source: trunk/abcl/doc/asdf/asdf.texinfo @ 14150

Last change on this file since 14150 was 14150, checked in by Mark Evenson, 8 years ago

Upgrade to asdf-2.24.

File size: 152.0 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2012 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2012 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2012 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex require-system
201@findex compile-system
202@findex test-system
203@cindex system directory designator
204@findex operate
205@findex oos
206
207@c @menu
208@c * Installing ASDF::
209@c @end menu
210
211
212@section Loading a pre-installed ASDF
213
214Many Lisp implementations include a copy of ASDF.
215You can usually load this copy using Common Lisp's @code{require} function:
216
217@lisp
218(require "asdf")
219@end lisp
220
221As of the writing of this manual,
222the following implementations provide ASDF 2 this way:
223abcl allegro ccl clisp cmucl ecl lispworks mkcl sbcl xcl.
224The following implementation doesn't provide it yet but will in a future release:
225scl.
226The following implementations are obsolete, not actively maintained,
227and most probably will never bundle it:
228cormanlisp gcl genera mcl.
229
230If the implementation you are using doesn't provide ASDF 2,
231see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
232If that implementation is still actively maintained,
233you may also send a bug report to your Lisp vendor and complain
234about their failing to provide ASDF.
235
236NB: all implementations except clisp also accept
237@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
238For portability's sake, you probably want to use @code{(require "asdf")}.
239
240
241@section Checking whether ASDF is loaded
242
243To check whether ASDF is properly loaded in your current Lisp image,
244you can run this form:
245
246@lisp
247(asdf:asdf-version)
248@end lisp
249
250If it returns a string,
251that is the version of ASDF that is currently installed.
252
253If it raises an error,
254then either ASDF is not loaded, or
255you are using an old version of ASDF.
256
257You can check whether an old version is loaded
258by checking if the ASDF package is present.
259The form below will allow you to programmatically determine
260whether a recent version is loaded, an old version is loaded,
261or none at all:
262
263@lisp
264(or #+asdf2 (asdf:asdf-version) #+asdf :old)
265@end lisp
266
267If it returns a version number, that's the version of ASDF installed.
268If it returns the keyword @code{:OLD},
269then you're using an old version of ASDF (from before 1.635).
270If it returns @code{NIL} then ASDF is not installed.
271
272If you are experiencing problems with ASDF,
273please try upgrading to the latest released version,
274using the method below,
275before you contact us and raise an issue.
276
277
278@section Upgrading ASDF
279
280If your implementation does provide ASDF 2 or later,
281and you want to upgrade to a more recent version,
282just install ASDF like any other package
283(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
284configure ASDF as usual (see @pxref{Configuring ASDF} below),
285and upgrade with:
286
287@lisp
288(require "asdf")
289(asdf:load-system :asdf)
290@end lisp
291
292If on the other hand, your implementation only provides an old ASDF,
293you will require a special configuration step and an old-style loading:
294
295@lisp
296(require "asdf")
297(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
298(asdf:oos 'asdf:load-op :asdf)
299@end lisp
300
301Don't forget the trailing @code{/} at the end of your pathname.
302
303Also, note that older versions of ASDF won't redirect their output,
304or at least won't do it according to your usual ASDF 2 configuration.
305You therefore need write access on the directory
306where you install the new ASDF,
307and make sure you're not using it
308for multiple mutually incompatible implementations.
309At worst, you may have to have multiple copies of the new ASDF,
310e.g. one per implementation installation, to avoid clashes.
311Note that to our knowledge all implementations that provide ASDF
312provide ASDF 2 in their latest release, so
313you may want to upgrade your implementation rather than go through that hoop.
314
315Finally, note that there are some limitations to upgrading ASDF:
316@itemize
317@item
318Any ASDF extension becomes invalid, and will need to be reloaded.
319This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
320Starting with ASDF 2.014.8, ASDF will actually invalidate
321all previously loaded systems when it is loaded on top of
322a different ASDF version.
323@item
324Until all implementations provide ASDF 2.015 or later,
325it is safer if you upgrade ASDF and its extensions as a special step
326at the very beginning of whatever script you are running,
327before you start using ASDF to load anything else.
328@item
329Until all implementations provide ASDF 2.015 or later,
330it is unsafe to upgrade ASDF as part of loading a system
331that depends on a more recent version of ASDF,
332since the new one might shadow the old one while the old one is running,
333and the running old one will be confused
334when extensions are loaded into the new one.
335In the meantime, we recommend that your systems should @emph{not} specify
336@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
337but instead that they check that a recent enough ASDF is installed,
338with such code as:
339@example
340(unless (or #+asdf2 (asdf:version-satisfies
341                     (asdf:asdf-version) *required-asdf-version*))
342  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
343@end example
344@end itemize
345
346
347@section Loading an otherwise installed ASDF
348
349If your implementation doesn't include ASDF,
350if for some reason the upgrade somehow fails,
351does not or cannot apply to your case,
352you will have to install the file @file{asdf.lisp}
353somewhere and load it with:
354
355@lisp
356(load "/path/to/your/installed/asdf.lisp")
357@end lisp
358
359The single file @file{asdf.lisp} is all you normally need to use ASDF.
360
361You can extract this file from latest release tarball on the
362@url{http://common-lisp.net/project/asdf/,ASDF website}.
363If you are daring and willing to report bugs, you can get
364the latest and greatest version of ASDF from its git repository.
365@xref{Getting the latest version}.
366
367For maximum convenience you might want to have ASDF loaded
368whenever you start your Lisp implementation,
369for example by loading it from the startup script or dumping a custom core
370--- check your Lisp implementation's manual for details.
371
372
373@node Configuring ASDF, Using ASDF, Loading ASDF, Top
374@comment  node-name,  next,  previous,  up
375
376@chapter Configuring ASDF
377
378@section Configuring ASDF to find your systems
379
380So it may compile and load your systems, ASDF must be configured to find
381the @file{.asd} files that contain system definitions.
382
383Since ASDF 2, the preferred way to configure where ASDF finds your systems is
384the @code{source-registry} facility,
385fully described in its own chapter of this manual.
386@xref{Controlling where ASDF searches for systems}.
387
388The default location for a user to install Common Lisp software is under
389@file{~/.local/share/common-lisp/source/}.
390If you install software there, you don't need further configuration.
391If you're installing software yourself at a location that isn't standard,
392you have to tell ASDF where you installed it. See below.
393If you're using some tool to install software (e.g. Quicklisp),
394the authors of that tool should already have configured ASDF.
395
396The simplest way to add a path to your search path,
397say @file{/home/luser/.asd-link-farm/}
398is to create the directory
399@file{~/.config/common-lisp/source-registry.conf.d/}
400and there create a file with any name of your choice,
401and with the type @file{conf},
402for instance @file{42-asd-link-farm.conf}
403containing the line:
404
405@kbd{(:directory "/home/luser/.asd-link-farm/")}
406
407If you want all the subdirectories under @file{/home/luser/lisp/}
408to be recursively scanned for @file{.asd} files, instead use:
409
410@kbd{(:tree "/home/luser/lisp/")}
411
412Note that your Operating System distribution or your system administrator
413may already have configured system-managed libraries for you.
414
415The required @file{.conf} extension allows you to have disabled files
416or editor backups (ending in @file{~}), and works portably
417(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
418Excluded are files the name of which start with a @file{.} character.
419It is customary to start the filename with two digits
420that specify the order in which the directories will be scanned.
421
422ASDF will automatically read your configuration
423the first time you try to find a system.
424You can reset the source-registry configuration with:
425
426@lisp
427(asdf:clear-source-registry)
428@end lisp
429
430And you probably should do so before you dump your Lisp image,
431if the configuration may change
432between the machine where you save it at the time you save it
433and the machine you resume it at the time you resume it.
434
435
436@section Configuring ASDF to find your systems --- old style
437
438The old way to configure ASDF to find your systems is by
439@code{push}ing directory pathnames onto the variable
440@code{asdf:*central-registry*}.
441
442You must configure this variable between the time you load ASDF
443and the time you first try to use it.
444Loading and configuring ASDF presumably happen
445as part of some initialization script that builds or starts
446your Common Lisp software system.
447(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
448
449The @code{asdf:*central-registry*} is empty by default in ASDF 2,
450but is still supported for compatibility with ASDF 1.
451When used, it takes precedence over the above source-registry@footnote{
452It is possible to further customize
453the system definition file search.
454That's considered advanced use, and covered later:
455search forward for
456@code{*system-definition-search-functions*}.
457@xref{Defining systems with defsystem}.}.
458
459For instance, if you wanted ASDF to find the @file{.asd} file
460@file{/home/me/src/foo/foo.asd} your initialization script
461could after it loads ASDF with @code{(require "asdf")}
462configure it with:
463
464@lisp
465(push "/home/me/src/foo/" asdf:*central-registry*)
466@end lisp
467
468Note the trailing slash: when searching for a system,
469ASDF will evaluate each entry of the central registry
470and coerce the result to a pathname@footnote{
471ASDF will indeed call @code{EVAL} on each entry.
472It will also skip entries that evaluate to @code{NIL}.
473
474Strings and pathname objects are self-evaluating,
475in which case the @code{EVAL} step does nothing;
476but you may push arbitrary SEXP onto the central registry,
477that will be evaluated to compute e.g. things that depend
478on the value of shell variables or the identity of the user.
479
480The variable @code{asdf:*central-registry*} is thus a list of
481``system directory designators''.
482A @dfn{system directory designator} is a form
483which will be evaluated whenever a system is to be found,
484and must evaluate to a directory to look in.
485By ``directory'' here, we mean
486``designator for a pathname with a supplied DIRECTORY component''.
487}
488at which point the presence of the trailing directory name separator
489is necessary to tell Lisp that you're discussing a directory
490rather than a file.
491
492Typically, however, there are a lot of @file{.asd} files, and
493a common idiom was to have to put
494a bunch of @emph{symbolic links} to @file{.asd} files
495in a common directory
496and push @emph{that} directory (the ``link farm'')
497to the
498@code{asdf:*central-registry*}
499instead of pushing each of the many involved directories
500to the @code{asdf:*central-registry*}.
501ASDF knows how to follow such @emph{symlinks}
502to the actual file location when resolving the paths of system components
503(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
504if you try aliases under MacOS, we are curious to hear about your experience).
505
506For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
507is a member of @code{*central-registry*}, you could set up the
508system @var{foo} for loading with asdf with the following
509commands at the shell:
510
511@example
512$ cd /home/me/cl/systems/
513$ ln -s ~/src/foo/foo.asd .
514@end example
515
516This old style for configuring ASDF is not recommended for new users,
517but it is supported for old users, and for users who want to programmatically
518control what directories are added to the ASDF search path.
519
520
521@section Configuring where ASDF stores object files
522@findex clear-output-translations
523
524ASDF lets you configure where object files will be stored.
525Sensible defaults are provided and
526you shouldn't normally have to worry about it.
527
528This allows the same source code repository may be shared
529between several versions of several Common Lisp implementations,
530between several users using different compilation options
531and without write privileges on shared source directories, etc.
532This also allows to keep source directories uncluttered
533by plenty of object files.
534
535Starting with ASDF 2, the @code{asdf-output-translations} facility
536was added to ASDF itself, that controls where object files will be stored.
537This facility is fully described in a chapter of this manual,
538@ref{Controlling where ASDF saves compiled files}.
539
540The simplest way to add a translation to your search path,
541say from @file{/foo/bar/baz/quux/}
542to @file{/where/i/want/my/fasls/}
543is to create the directory
544@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
545and there create a file with any name of your choice and the type @file{conf},
546for instance @file{42-bazquux.conf}
547containing the line:
548
549@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
550
551To disable output translations for source under a given directory,
552say @file{/toto/tata/}
553you can create a file @file{40-disable-toto.conf}
554with the line:
555
556@kbd{("/toto/tata/")}
557
558To wholly disable output translations for all directories,
559you can create a file @file{00-disable.conf}
560with the line:
561
562@kbd{(t t)}
563
564Note that your Operating System distribution or your system administrator
565may already have configured translations for you.
566In absence of any configuration, the default is to redirect everything
567under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
568@xref{Controlling where ASDF searches for systems}, for full details.
569
570The required @file{.conf} extension allows you to have disabled files
571or editor backups (ending in @file{~}), and works portably
572(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
573Excluded are files the name of which start with a @file{.} character.
574It is customary to start the filename with two digits
575that specify the order in which the directories will be scanned.
576
577ASDF will automatically read your configuration
578the first time you try to find a system.
579You can reset the source-registry configuration with:
580
581@lisp
582(asdf:clear-output-translations)
583@end lisp
584
585And you probably should do so before you dump your Lisp image,
586if the configuration may change
587between the machine where you save it at the time you save it
588and the machine you resume it at the time you resume it.
589
590Finally note that before ASDF 2,
591other ASDF add-ons offered the same functionality,
592each in subtly different and incompatible ways:
593ASDF-Binary-Locations, cl-launch, common-lisp-controller.
594ASDF-Binary-Locations is now not needed anymore and should not be used.
595cl-launch 3.000 and common-lisp-controller 7.2 have been updated
596to just delegate this functionality to ASDF.
597
598@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
599@comment  node-name,  next,  previous,  up
600
601
602@section Resetting Configuration
603
604When you dump and restore an image, or when you tweak your configuration,
605you may want to reset the ASDF configuration.
606For that you may use the following function:
607
608@defun clear-configuration
609   undoes any ASDF configuration,
610   regarding source-registry or output-translations.
611@end defun
612
613If you use SBCL, CMUCL or SCL, you may use this snippet
614so that the ASDF configuration be cleared automatically as you dump an image:
615
616@example
617#+(or cmu sbcl scl)
618(pushnew 'clear-configuration
619         #+(or cmu scl) ext:*before-save-initializations*
620         #+sbcl sb-ext:*save-hooks*)
621@end example
622
623For compatibility with all Lisp implementations, however,
624you might want instead your build script to explicitly call
625@code{(asdf:clear-configuration)} at an appropriate moment before dumping.
626
627
628@chapter Using ASDF
629
630@section Loading a system
631
632The system @var{foo} is loaded (and compiled, if necessary)
633by evaluating the following Lisp form:
634
635@example
636(asdf:load-system :@var{foo})
637@end example
638
639On some implementations (namely recent versions of
640ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, SBCL and XCL),
641ASDF hooks into the @code{CL:REQUIRE} facility
642and you can just use:
643
644@example
645(require :@var{foo})
646@end example
647
648In older versions of ASDF, you needed to use
649@code{(asdf:oos 'asdf:load-op :@var{foo})}.
650If your ASDF is too old to provide @code{asdf:load-system} though
651we recommend that you upgrade to ASDF 2.
652@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
653
654Note the name of a system is specified as a string or a symbol,
655typically a keyword.
656If a symbol (including a keyword), its name is taken and lowercased.
657The name must be a suitable value for the @code{:name} initarg
658to @code{make-pathname} in whatever filesystem the system is to be found.
659The lower-casing-symbols behaviour is unconventional,
660but was selected after some consideration.
661Observations suggest that the type of systems we want to support
662either have lowercase as customary case (unix, mac, windows)
663or silently convert lowercase to uppercase (lpns),
664so this makes more sense than attempting to use @code{:case :common},
665which is reported not to work on some implementations
666
667
668@section Other Operations
669
670ASDF provides three commands for the most common system operations:
671@code{load-system}, @code{compile-system} or @code{test-system}.
672It also provides @code{require-system}, a version of @code{load-system}
673that skips trying to update systems that are already loaded.
674
675Because ASDF is an extensible system
676for defining @emph{operations} on @emph{components},
677it also provides a generic function @code{operate}
678(which is usually abbreviated by @code{oos}).
679You'll use @code{oos} whenever you want to do something beyond
680compiling, loading and testing.
681
682Output from ASDF and ASDF extensions are supposed to be sent
683to the CL stream @code{*standard-output*},
684and so rebinding that stream around calls to @code{asdf:operate}
685should redirect all output from ASDF operations.
686
687Reminder: before ASDF can operate on a system, however,
688it must be able to find and load that system's definition.
689@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
690
691
692@section Summary
693
694To use ASDF:
695
696@itemize
697@item
698Load ASDF itself into your Lisp image, either through
699@code{(require "asdf")} or else through
700@code{(load "/path/to/asdf.lisp")}.
701
702@item
703Make sure ASDF can find system definitions
704thanks to proper source-registry configuration.
705
706@item
707Load a system with @code{(load-system :my-system)}
708or use some other operation on some system of your choice.
709
710@end itemize
711
712@section Moving on
713
714That's all you need to know to use ASDF to load systems written by others.
715The rest of this manual deals with writing system definitions
716for Common Lisp software you write yourself,
717including how to extend ASDF to define new operation and component types.
718
719
720@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
721@comment  node-name,  next,  previous,  up
722@chapter Defining systems with defsystem
723
724This chapter describes how to use asdf to define systems and develop
725software.
726
727
728@menu
729* The defsystem form::
730* A more involved example::
731* The defsystem grammar::
732* Other code in .asd files::
733@end menu
734
735@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
736@comment  node-name,  next,  previous,  up
737@section The defsystem form
738
739Systems can be constructed programmatically
740by instantiating components using @code{make-instance}.
741Most of the time, however, it is much more practical to use
742a static @code{defsystem} form.
743This section begins with an example of a system definition,
744then gives the full grammar of @code{defsystem}.
745
746Let's look at a simple system.
747This is a complete file that would
748usually be saved as @file{hello-lisp.asd}:
749
750@lisp
751(in-package :asdf)
752
753(defsystem "hello-lisp"
754  :description "hello-lisp: a sample Lisp system."
755  :version "0.2.1"
756  :author "Joe User <joe@@example.com>"
757  :licence "Public Domain"
758  :components ((:file "packages")
759               (:file "macros" :depends-on ("packages"))
760               (:file "hello" :depends-on ("macros"))))
761@end lisp
762
763Some notes about this example:
764
765@itemize
766
767@item
768The file starts with an @code{in-package} form
769to use package @code{asdf}.
770You could instead start your definition by using
771a qualified name @code{asdf:defsystem}.
772
773@item
774If in addition to simply using @code{defsystem},
775you are going to define functions,
776create ASDF extension, globally bind symbols, etc.,
777it is recommended that to avoid namespace pollution between systems,
778you should create your own package for that purpose,
779for instance replacing the above @code{(in-package :asdf)} with:
780
781@lisp
782(defpackage :foo-system
783  (:use :cl :asdf))
784
785(in-package :foo-system)
786@end lisp
787
788@item
789The @code{defsystem} form defines a system named @code{hello-lisp}
790that contains three source files:
791@file{packages}, @file{macros} and @file{hello}.
792
793@item
794The file @file{macros} depends on @file{packages}
795(presumably because the package it's in is defined in @file{packages}),
796and the file @file{hello} depends on @file{macros}
797(and hence, transitively on @file{packages}).
798This means that ASDF will compile and load @file{packages} and @file{macros}
799before starting the compilation of file @file{hello}.
800
801@item
802The files are located in the same directory
803as the file with the system definition.
804ASDF resolves symbolic links (or Windows shortcuts)
805before loading the system definition file and
806stores its location in the resulting system@footnote{
807It is possible, though almost never necessary, to override this behaviour.}.
808This is a good thing because the user can move the system sources
809without having to edit the system definition.
810
811@c FIXME: Should have cross-reference to "Version specifiers" in the
812@c defsystem grammar, but the cross-referencing is so broken by
813@c insufficient node breakdown that I have not put one in.
814@item
815Make sure you know how the @code{:version} numbers will be parsed!  They
816are parsed as period-separated lists of integers.  I.e., in the example,
817@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
818In particular, version @code{0.2.1} is interpreted the same as
819@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
820even though the two are the same when interpreted as decimal fractions.
821@cindex version specifiers
822@cindex :version
823
824@end itemize
825
826@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
827@comment  node-name,  next,  previous,  up
828@section A more involved example
829
830Let's illustrate some more involved uses of @code{defsystem} via a
831slightly convoluted example:
832
833@lisp
834(defsystem "foo"
835  :version "1.0.0"
836  :components ((:module "mod"
837                            :components ((:file "bar")
838                                                  (:file"baz")
839                                                  (:file "quux"))
840                            :perform (compile-op :after (op c)
841                                                  (do-something c))
842                            :explain (compile-op :after (op c)
843                                            (explain-something c)))
844                         (:file "blah")))
845@end lisp
846
847The @code{:module} component named @code{"mod"} is a collection of three files,
848which will be located in a subdirectory of the main code directory named
849@file{mod} (this location can be overridden; see the discussion of the
850@code{:pathname} option in @ref{The defsystem grammar}).
851
852The method-form tokens provide a shorthand for defining methods on
853particular components.  This part
854
855@lisp
856                :perform (compile-op :after (op c)
857                          (do-something c))
858                :explain (compile-op :after (op c)
859                          (explain-something c))
860@end lisp
861
862has the effect of
863
864@lisp
865(defmethod perform :after ((op compile-op) (c (eql ...)))
866           (do-something c))
867(defmethod explain :after ((op compile-op) (c (eql ...)))
868           (explain-something c))
869@end lisp
870
871where @code{...} is the component in question.
872In this case @code{...} would expand to something like
873
874@lisp
875(find-component (find-system "foo") "mod")
876@end lisp
877
878For more details on the syntax of such forms, see @ref{The defsystem
879grammar}.
880For more details on what these methods do, @pxref{Operations} in
881@ref{The object model of ASDF}.
882
883@c The following plunge into the weeds is not appropriate in this
884@c location. [2010/10/03:rpg]
885@c note that although this also supports @code{:before} methods,
886@c they may not do what you want them to ---
887@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
888@c will run after all the dependencies and sub-components have been processed,
889@c but before the component in question has been compiled.
890
891@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
892@comment  node-name,  next,  previous,  up
893@section The defsystem grammar
894
895@c FIXME: @var typesetting not consistently used here.  We should either expand
896@c its use to everywhere, or we should kill it everywhere.
897
898
899@example
900system-definition := ( defsystem system-designator @var{system-option}* )
901
902system-option := :defsystem-depends-on system-list
903                 | :weakly-depends-on @var{system-list}
904                 | :class class-name (see discussion below)
905                 | module-option
906                 | option
907
908module-option := :components component-list
909                 | :serial [ t | nil ]
910                 | :if-component-dep-fails component-dep-fail-option
911
912option :=
913        | :pathname pathname-specifier
914        | :default-component-class class-name
915        | :perform method-form
916        | :explain method-form
917        | :output-files method-form
918        | :operation-done-p method-form
919        | :depends-on ( @var{dependency-def}* )
920        | :in-order-to ( @var{dependency}+ )
921
922
923system-list := ( @var{simple-component-name}* )
924
925component-list := ( @var{component-def}* )
926
927component-def  := ( component-type simple-component-name @var{option}* )
928
929component-type := :system | :module | :file | :static-file | other-component-type
930
931other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
932
933dependency-def := simple-component-name
934               | ( :feature name )
935               | ( :version simple-component-name version-specifier)
936
937dependency := (dependent-op @var{requirement}+)
938requirement := (required-op @var{required-component}+)
939             | (feature feature-name)
940dependent-op := operation-name
941required-op := operation-name | feature
942
943simple-component-name := string
944                      |  symbol
945
946pathname-specifier := pathname | string | symbol
947
948method-form := (operation-name qual lambda-list @&rest body)
949qual := method qualifier
950
951component-dep-fail-option := :fail | :try-next | :ignore
952@end example
953
954
955@subsection Component names
956
957Component names (@code{simple-component-name})
958may be either strings or symbols.
959
960@subsection Component types
961
962Component type names, even if expressed as keywords, will be looked up
963by name in the current package and in the asdf package, if not found in
964the current package.  So a component type @code{my-component-type}, in
965the current package @code{my-system-asd} can be specified as
966@code{:my-component-type}, or @code{my-component-type}.
967
968@subsection System class names
969
970A system class name will be looked up in the same way as a Component
971type (see above).  Typically, one will not need to specify a system
972class name, unless using a non-standard system class defined in some
973ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
974see below.  For such class names in the ASDF package, we recommend that
975the @code{:class} option be specified using a keyword symbol, such as
976
977@example
978:class :MY-NEW-SYSTEM-SUBCLASS
979@end example
980
981This practice will ensure that package name conflicts are avoided.
982Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
983the current package @emph{before} it has been exported from the ASDF
984extension loaded by @code{:defsystem-depends-on}, causing a name
985conflict in the current package.
986
987@subsection Defsystem depends on
988@cindex :defsystem-depends-on
989
990The @code{:defsystem-depends-on} option to @code{defsystem} allows the
991programmer to specify another ASDF-defined system or set of systems that
992must be loaded @emph{before} the system definition is processed.
993Typically this is used to load an ASDF extension that is used in the
994system definition.
995
996@subsection Weakly depends on
997@cindex :weakly-depends-on
998
999The @code{:weakly-depends-on} option to @code{defsystem} allows the
1000programmer to specify another ASDF-defined system or set of systems that
1001ASDF should @emph{try} to load, but need not load in order to be
1002successful.  Typically this is used if there are a number of systems
1003that, if present, could provide additional functionality, but which are
1004not necessary for basic function.
1005
1006Currently, although it is specified to be an option only to
1007@code{defsystem}, this option is accepted at any component, but it probably
1008only makes sense at the @code{defsystem} level. Programmers are cautioned not
1009to use this component option except at the @code{defsystem} level, as
1010this anomalous behavior may be removed without warning.
1011
1012@subsection Pathname specifiers
1013@cindex pathname specifiers
1014
1015A pathname specifier (@code{pathname-specifier})
1016may be a pathname, a string or a symbol.
1017When no pathname specifier is given for a component,
1018which is the usual case, the component name itself is used.
1019
1020If a string is given, which is the usual case,
1021the string will be interpreted as a Unix-style pathname
1022where @code{/} characters will be interpreted as directory separators.
1023Usually, Unix-style relative pathnames are used
1024(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1025they are relative to the path of the parent component.
1026Finally, depending on the @code{component-type},
1027the pathname may be interpreted as either a file or a directory,
1028and if it's a file,
1029a file type may be added corresponding to the @code{component-type},
1030or else it will be extracted from the string itself (if applicable).
1031
1032For instance, the @code{component-type} @code{:module}
1033wants a directory pathname, and so a string @code{"foo/bar"}
1034will be interpreted as the pathname @file{#p"foo/bar/"}.
1035On the other hand, the @code{component-type} @code{:file}
1036wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1037will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1038and a string @code{"foo/bar.quux"}
1039will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1040Finally, the @code{component-type} @code{:static-file}
1041wants a file without specifying a type, and so a string @code{"foo/bar"}
1042will be interpreted as the pathname @file{#p"foo/bar"},
1043and a string @code{"foo/bar.quux"}
1044will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1045
1046ASDF does not interpret the string @code{".."} to designate the parent
1047directory.  This string will be passed through to the underlying
1048operating system for interpretation.  We @emph{believe} that this will
1049work on all platforms where ASDF is deployed, but do not guarantee this
1050behavior.  A pathname object with a relative directory component of
1051@code{:up} or @code{:back} is the only guaranteed way to specify a
1052parent directory.
1053
1054If a symbol is given, it will be translated into a string,
1055and downcased in the process.
1056The downcasing of symbols is unconventional,
1057but was selected after some consideration.
1058Observations suggest that the type of systems we want to support
1059either have lowercase as customary case (Unix, Mac, windows)
1060or silently convert lowercase to uppercase (lpns),
1061so this makes more sense than attempting to use @code{:case :common}
1062as argument to @code{make-pathname},
1063which is reported not to work on some implementations.
1064
1065Pathname objects may be given to override the path for a component.
1066Such objects are typically specified using reader macros such as @code{#p}
1067or @code{#.(make-pathname ...)}.
1068Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1069and that the behavior of @code{parse-namestring} is completely non-portable,
1070unless you are using Common Lisp @code{logical-pathname}s
1071(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1072Pathnames made with @code{#.(make-pathname ...)}
1073can usually be done more easily with the string syntax above.
1074The only case that you really need a pathname object is to override
1075the component-type default file type for a given component.
1076Therefore, pathname objects should only rarely be used.
1077Unhappily, ASDF 1 didn't properly support
1078parsing component names as strings specifying paths with directories,
1079and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1080
1081Note that when specifying pathname objects,
1082ASDF does not do any special interpretation of the pathname
1083influenced by the component type, unlike the procedure for
1084pathname-specifying strings.
1085On the one hand, you have to be careful to provide a pathname that correctly
1086fulfills whatever constraints are required from that component type
1087(e.g. naming a directory or a file with appropriate type);
1088on the other hand, you can circumvent the file type that would otherwise
1089be forced upon you if you were specifying a string.
1090
1091@subsection Version specifiers
1092@cindex version specifiers
1093@cindex :version
1094
1095Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1096@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1097In particular, version @code{0.2.1} is interpreted the same as
1098@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1099even though the two are the same when interpreted as decimal fractions.
1100
1101System definers are encouraged to use version identifiers of the form
1102@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1103API) and patch level.
1104
1105@xref{Common attributes of components}.
1106
1107
1108@subsection Using logical pathnames
1109@cindex logical pathnames
1110
1111We do not generally recommend the use of logical pathnames,
1112especially not so to newcomers to Common Lisp.
1113However, we do support the use of logical pathnames by old timers,
1114when such is their preference.
1115
1116To use logical pathnames,
1117you will have to provide a pathname object as a @code{:pathname} specifier
1118to components that use it, using such syntax as
1119@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1120
1121You only have to specify such logical pathname
1122for your system or some top-level component.
1123Sub-components' relative pathnames,
1124specified using the string syntax for names,
1125will be properly merged with the pathnames of their parents.
1126The specification of a logical pathname host however is @emph{not}
1127otherwise directly supported in the ASDF syntax
1128for pathname specifiers as strings.
1129
1130The @code{asdf-output-translation} layer will
1131avoid trying to resolve and translate logical pathnames.
1132The advantage of this is that
1133you can define yourself what translations you want to use
1134with the logical pathname facility.
1135The disadvantage is that if you do not define such translations,
1136any system that uses logical pathnames will behave differently under
1137asdf-output-translations than other systems you use.
1138
1139If you wish to use logical pathnames you will have to configure the
1140translations yourself before they may be used.
1141ASDF currently provides no specific support
1142for defining logical pathname translations.
1143
1144Note that the reasons we do not recommend logical pathnames are that
1145(1) there is no portable way to set up logical pathnames before they are used,
1146(2) logical pathnames are limited to only portably use
1147a single character case, digits and hyphens.
1148While you can solve the first issue on your own,
1149describing how to do it on each of fifteen implementations supported by ASDF
1150is more than we can document.
1151As for the second issue, mind that the limitation is notably enforced on SBCL,
1152and that you therefore can't portably violate the limitations
1153but must instead define some encoding of your own and add individual mappings
1154to name physical pathnames that do not fit the restrictions.
1155This can notably be a problem when your Lisp files are part of a larger project
1156in which it is common to name files or directories in a way that
1157includes the version numbers of supported protocols,
1158or in which files are shared with software written
1159in different programming languages where conventions include the use of
1160underscores, dots or CamelCase in pathnames.
1161
1162
1163@subsection Serial dependencies
1164@cindex serial dependencies
1165
1166If the @code{:serial t} option is specified for a module,
1167ASDF will add dependencies for each child component,
1168on all the children textually preceding it.
1169This is done as if by @code{:depends-on}.
1170
1171@lisp
1172:serial t
1173:components ((:file "a") (:file "b") (:file "c"))
1174@end lisp
1175
1176is equivalent to
1177
1178@lisp
1179:components ((:file "a")
1180             (:file "b" :depends-on ("a"))
1181             (:file "c" :depends-on ("a" "b")))
1182@end lisp
1183
1184
1185@subsection Source location
1186
1187The @code{:pathname} option is optional in all cases for systems
1188defined via @code{defsystem},
1189and in the usual case the user is recommended not to supply it.
1190
1191Instead, ASDF follows a hairy set of rules that are designed so that
1192@enumerate
1193@item
1194@code{find-system}
1195will load a system from disk
1196and have its pathname default to the right place.
1197@item
1198This pathname information will not be overwritten with
1199@code{*default-pathname-defaults*}
1200(which could be somewhere else altogether)
1201if the user loads up the @file{.asd} file into his editor
1202and interactively re-evaluates that form.
1203@end enumerate
1204
1205If a system is being loaded for the first time,
1206its top-level pathname will be set to:
1207
1208@itemize
1209@item
1210The host/device/directory parts of @code{*load-truename*},
1211if it is bound.
1212@item
1213@code{*default-pathname-defaults*}, otherwise.
1214@end itemize
1215
1216If a system is being redefined, the top-level pathname will be
1217
1218@itemize
1219@item
1220changed, if explicitly supplied or obtained from @code{*load-truename*}
1221(so that an updated source location is reflected in the system definition)
1222@item
1223changed if it had previously been set from @code{*default-pathname-defaults*}
1224@item
1225left as before, if it had previously been set from @code{*load-truename*}
1226and @code{*load-truename*} is currently unbound
1227(so that a developer can evaluate a @code{defsystem} form
1228from within an editor without clobbering its source location)
1229@end itemize
1230
1231@subsection if-component-dep-fails option
1232
1233This option is only appropriate for module components (including
1234systems), not individual source files.
1235
1236For more information about this option, @pxref{Pre-defined subclasses of component}.
1237
1238@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1239@section Other code in .asd files
1240
1241Files containing @code{defsystem} forms
1242are regular Lisp files that are executed by @code{load}.
1243Consequently, you can put whatever Lisp code you like into these files
1244(e.g., code that examines the compile-time environment
1245and adds appropriate features to @code{*features*}).
1246However, some conventions should be followed,
1247so that users can control certain details of execution
1248of the Lisp in @file{.asd} files:
1249
1250@itemize
1251@item
1252Any informative output
1253(other than warnings and errors,
1254which are the condition system's to dispose of)
1255should be sent to the standard CL stream @code{*standard-output*},
1256so that users can easily control the disposition
1257of output from ASDF operations.
1258@end itemize
1259
1260
1261@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1262@comment  node-name,  next,  previous,  up
1263@chapter The object model of ASDF
1264
1265ASDF is designed in an object-oriented way from the ground up.
1266Both a system's structure and the operations that can be performed on systems
1267follow a protocol.
1268ASDF is extensible to new operations and to new component types.
1269This allows the addition of behaviours:
1270for example, a new component could be added for Java JAR archives,
1271and methods specialised on @code{compile-op} added for it
1272that would accomplish the relevant actions.
1273
1274This chapter deals with @emph{components}, the building blocks of a system,
1275and @emph{operations}, the actions that can be performed on a system.
1276
1277
1278
1279@menu
1280* Operations::
1281* Components::
1282* Functions::
1283@end menu
1284
1285@node  Operations, Components, The object model of ASDF, The object model of ASDF
1286@comment  node-name,  next,  previous,  up
1287@section Operations
1288@cindex operation
1289
1290An @dfn{operation} object of the appropriate type is instantiated
1291whenever the user wants to do something with a system like
1292
1293@itemize
1294@item compile all its files
1295@item load the files into a running lisp environment
1296@item copy its source files somewhere else
1297@end itemize
1298
1299Operations can be invoked directly, or examined
1300to see what their effects would be without performing them.
1301@emph{FIXME: document how!}
1302There are a bunch of methods specialised on operation and component type
1303that actually do the grunt work.
1304
1305The operation object contains whatever state is relevant for this purpose
1306(perhaps a list of visited nodes, for example)
1307but primarily is a nice thing to specialise operation methods on
1308and easier than having them all be @code{EQL} methods.
1309
1310Operations are invoked on systems via @code{operate}.
1311@anchor{operate}
1312@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1313@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1314@code{operate} invokes @var{operation} on @var{system}.
1315@code{oos} is a synonym for @code{operate}.
1316
1317@var{operation} is a symbol that is passed, along with the supplied
1318@var{initargs}, to @code{make-instance} to create the operation object.
1319@var{system} is a system designator.
1320
1321The @var{initargs} are passed to the @code{make-instance} call
1322when creating the operation object.
1323Note that dependencies may cause the operation
1324to invoke other operations on the system or its components:
1325the new operations will be created
1326with the same @var{initargs} as the original one.
1327
1328@end deffn
1329
1330@menu
1331* Predefined operations of ASDF::
1332* Creating new operations::
1333@end menu
1334
1335@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1336@comment  node-name,  next,  previous,  up
1337@subsection Predefined operations of ASDF
1338
1339All the operations described in this section are in the @code{asdf} package.
1340They are invoked via the @code{operate} generic function.
1341
1342@lisp
1343(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1344@end lisp
1345
1346@deffn Operation @code{compile-op} @&key @code{proclamations}
1347
1348This operation compiles the specified component.
1349If proclamations are supplied, they will be proclaimed.
1350This is a good place to specify optimization settings.
1351
1352When creating a new component type,
1353you should provide methods for @code{compile-op}.
1354
1355When @code{compile-op} is invoked,
1356component dependencies often cause some parts of the system
1357to be loaded as well as compiled.
1358Invoking @code{compile-op}
1359does not necessarily load all the parts of the system, though;
1360use @code{load-op} to load a system.
1361@end deffn
1362
1363@deffn Operation @code{load-op} @&key @code{proclamations}
1364
1365This operation loads a system.
1366
1367The default methods for @code{load-op} compile files before loading them.
1368For parity, your own methods on new component types should probably do so too.
1369@end deffn
1370
1371@deffn Operation @code{load-source-op}
1372
1373This operation will load the source for the files in a module
1374even if the source files have been compiled.
1375Systems sometimes have knotty dependencies
1376which require that sources are loaded
1377before they can be compiled.
1378This is how you do that.
1379
1380If you are creating a component type,
1381you need to implement this operation --- at least, where meaningful.
1382@end deffn
1383
1384@anchor{test-op}
1385@deffn Operation @code{test-op}
1386
1387This operation will perform some tests on the module.
1388The default method will do nothing.
1389The default dependency is to require
1390@code{load-op} to be performed on the module first.
1391The default @code{operation-done-p} is that the operation is @emph{never} done
1392---
1393we assume that if you invoke the @code{test-op},
1394you want to test the system, even if you have already done so.
1395
1396The results of this operation are not defined by ASDF.
1397It has proven difficult to define how the test operation
1398should signal its results to the user
1399in a way that is compatible with all of the various test libraries
1400and test techniques in use in the community.
1401@end deffn
1402
1403@c @deffn Operation test-system-version @&key minimum
1404
1405@c Asks the system whether it satisfies a version requirement.
1406
1407@c The default method accepts a string, which is expected to contain of a
1408@c number of integers separated by #\. characters.  The method is not
1409@c recursive.  The component satisfies the version dependency if it has
1410@c the same major number as required and each of its sub-versions is
1411@c greater than or equal to the sub-version number required.
1412
1413@c @lisp
1414@c (defun version-satisfies (x y)
1415@c   (labels ((bigger (x y)
1416@c           (cond ((not y) t)
1417@c                 ((not x) nil)
1418@c                 ((> (car x) (car y)) t)
1419@c                 ((= (car x) (car y))
1420@c                  (bigger (cdr x) (cdr y))))))
1421@c     (and (= (car x) (car y))
1422@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1423@c @end lisp
1424
1425@c If that doesn't work for your system, you can override it.  I hope
1426@c you have as much fun writing the new method as @verb{|#lisp|} did
1427@c reimplementing this one.
1428@c @end deffn
1429
1430@c @deffn Operation feature-dependent-op
1431
1432@c An instance of @code{feature-dependent-op} will ignore any components
1433@c which have a @code{features} attribute, unless the feature combination
1434@c it designates is satisfied by @code{*features*}.  This operation is
1435@c not intended to be instantiated directly, but other operations may
1436@c inherit from it.
1437
1438@c @end deffn
1439
1440@node  Creating new operations,  , Predefined operations of ASDF, Operations
1441@comment  node-name,  next,  previous,  up
1442@subsection Creating new operations
1443
1444ASDF was designed to be extensible in an object-oriented fashion.
1445To teach ASDF new tricks, a programmer can implement the behaviour he wants
1446by creating a subclass of @code{operation}.
1447
1448ASDF's pre-defined operations are in no way ``privileged'',
1449but it is requested that developers never use the @code{asdf} package
1450for operations they develop themselves.
1451The rationale for this rule is that we don't want to establish a
1452``global asdf operation name registry'',
1453but also want to avoid name clashes.
1454
1455An operation must provide methods for the following generic functions
1456when invoked with an object of type @code{source-file}:
1457@emph{FIXME describe this better}
1458
1459@itemize
1460
1461@item @code{output-files}
1462The @code{output-files} method determines where the method will put its files.
1463It returns two values, a list of pathnames, and a boolean.
1464If the boolean is @code{T} then the pathnames are marked
1465not be translated by enclosing @code{:around} methods.
1466If the boolean is @code{NIL} then enclosing @code{:around} methods
1467may translate these pathnames, e.g. to ensure object files
1468are somehow stored in some implementation-dependent cache.
1469@item @code{perform}
1470The @code{perform} method must call @code{output-files}
1471to find out where to put its files,
1472because the user is allowed to override.
1473@item @code{output-files}
1474for local policy @code{explain}
1475@item @code{operation-done-p},
1476if you don't like the default one
1477
1478@end itemize
1479
1480Operations that print output should send that output to the standard
1481CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1482
1483@node Components, Functions, Operations, The object model of ASDF
1484@comment  node-name,  next,  previous,  up
1485@section Components
1486@cindex component
1487@cindex system
1488@cindex system designator
1489@vindex *system-definition-search-functions*
1490
1491A @dfn{component} represents a source file or
1492(recursively) a collection of components.
1493A @dfn{system} is (roughly speaking) a top-level component
1494that can be found via @code{find-system}.
1495
1496A @dfn{system designator} is a string or symbol
1497and behaves just like any other component name
1498(including with regard to the case conversion rules for component names).
1499
1500
1501@defun find-system system-designator &optional (error-p t)
1502
1503Given a system designator, @code{find-system} finds and returns a system.
1504If no system is found, an error of type
1505@code{missing-component} is thrown,
1506or @code{nil} is returned if @code{error-p} is false.
1507
1508To find and update systems, @code{find-system} funcalls each element
1509in the @code{*system-definition-search-functions*} list,
1510expecting a pathname to be returned, or a system object,
1511from which a pathname may be extracted, and that will be registered.
1512The resulting pathname (if any) is loaded
1513if one of the following conditions is true:
1514
1515@itemize
1516@item
1517there is no system of that name in memory
1518@item
1519the pathname is different from that which was previously loaded
1520@item
1521the file's @code{last-modified} time exceeds the @code{last-modified} time
1522of the system in memory
1523@end itemize
1524
1525When system definitions are loaded from @file{.asd} files,
1526a new scratch package is created for them to load into,
1527so that different systems do not overwrite each others operations.
1528The user may also wish to (and is recommended to)
1529include @code{defpackage} and @code{in-package} forms
1530in his system definition files, however,
1531so that they can be loaded manually if need be.
1532
1533The default value of @code{*system-definition-search-functions*}
1534is a list of two functions.
1535The first function looks in each of the directories given
1536by evaluating members of @code{*central-registry*}
1537for a file whose name is the name of the system and whose type is @file{asd}.
1538The first such file is returned,
1539whether or not it turns out to actually define the appropriate system.
1540The second function does something similar,
1541for the directories specified in the @code{source-registry}.
1542Hence, it is strongly advised to define a system
1543@var{foo} in the corresponding file @var{foo.asd}.
1544@end defun
1545
1546
1547@menu
1548* Common attributes of components::
1549* Pre-defined subclasses of component::
1550* Creating new component types::
1551@end menu
1552
1553@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1554@comment  node-name,  next,  previous,  up
1555@subsection Common attributes of components
1556
1557All components, regardless of type, have the following attributes.
1558All attributes except @code{name} are optional.
1559
1560@subsubsection Name
1561
1562A component name is a string or a symbol.
1563If a symbol, its name is taken and lowercased.
1564
1565Unless overridden by a @code{:pathname} attribute,
1566the name will be interpreted as a pathname specifier according
1567to a Unix-style syntax.
1568@xref{The defsystem grammar,,Pathname specifiers}.
1569
1570@subsubsection Version identifier
1571@findex version-satisfies
1572@cindex :version
1573
1574This optional attribute is used by the generic function
1575@code{version-satisfies}, which tests to see if @code{:version}
1576dependencies are satisfied.
1577the version should be a string of integers separated by dots,
1578for example @samp{1.0.11}.
1579For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1580
1581@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1582@c @xref{Predefined operations of ASDF}.
1583@c @emph{Nota Bene}:
1584@c This operation, planned for ASDF 1,
1585@c is still not implemented yet as of ASDF 2.
1586@c Don't hold your breath.
1587
1588
1589
1590@subsubsection Required features
1591
1592@emph{FIXME: This subsection seems to contradict the
1593@code{defsystem} grammar subsection,
1594which doesn't provide any obvious way to specify required features.
1595Furthermore, in 2009, discussions on the
1596@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1597suggested that the specification of required features may be broken,
1598and that no one may have been using them for a while.
1599Please contact the
1600@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1601if you are interested in getting this features feature fixed.}
1602
1603Traditionally defsystem users have used reader conditionals
1604to include or exclude specific per-implementation files.
1605This means that any single implementation cannot read the entire system,
1606which becomes a problem if it doesn't wish to compile it,
1607but instead for example to create an archive file containing all the sources,
1608as it will omit to process the system-dependent sources for other systems.
1609
1610Each component in an asdf system may therefore specify features using
1611the same syntax as @code{#+} does, and it will (somehow) be ignored for
1612certain operations unless the feature conditional is a member of
1613@code{*features*}.
1614
1615
1616@subsubsection Dependencies
1617
1618This attribute specifies dependencies of the component on its siblings.
1619It is optional but often necessary.
1620
1621There is an excitingly complicated relationship between the initarg
1622and the method that you use to ask about dependencies
1623
1624Dependencies are between (operation component) pairs.
1625In your initargs for the component, you can say
1626
1627@lisp
1628:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1629              (load-op (load-op "foo")))
1630@end lisp
1631
1632This means the following things:
1633@itemize
1634@item
1635before performing compile-op on this component, we must perform
1636load-op on @var{a} and @var{b}, and compile-op on @var{c},
1637@item
1638before performing @code{load-op}, we have to load @var{foo}
1639@end itemize
1640
1641The syntax is approximately
1642
1643@verbatim
1644(this-op {(other-op required-components)}+)
1645
1646simple-component-name := string
1647                      |  symbol
1648
1649required-components := simple-component-name
1650                     | (required-components required-components)
1651
1652component-name := simple-component-name
1653                | (:version simple-component-name minimum-version-object)
1654@end verbatim
1655
1656Side note:
1657
1658This is on a par with what ACL defsystem does.
1659mk-defsystem is less general: it has an implied dependency
1660
1661@verbatim
1662  for all x, (load x) depends on (compile x)
1663@end verbatim
1664
1665and using a @code{:depends-on} argument to say that @var{b} depends on
1666@var{a} @emph{actually} means that
1667
1668@verbatim
1669  (compile b) depends on (load a)
1670@end verbatim
1671
1672This is insufficient for e.g. the McCLIM system, which requires that
1673all the files are loaded before any of them can be compiled ]
1674
1675End side note
1676
1677In ASDF, the dependency information for a given component and operation
1678can be queried using @code{(component-depends-on operation component)},
1679which returns a list
1680
1681@lisp
1682((load-op "a") (load-op "b") (compile-op "c") ...)
1683@end lisp
1684
1685@code{component-depends-on} can be subclassed for more specific
1686component/operation types: these need to @code{(call-next-method)}
1687and append the answer to their dependency, unless
1688they have a good reason for completely overriding the default dependencies.
1689
1690If it weren't for CLISP, we'd be using @code{LIST} method
1691combination to do this transparently.
1692But, we need to support CLISP.
1693If you have the time for some CLISP hacking,
1694I'm sure they'd welcome your fixes.
1695@c Doesn't CLISP now support LIST method combination?
1696
1697See the discussion of the semantics of @code{:version} in the defsystem
1698grammar.
1699
1700@c FIXME: Should have cross-reference to "Version specifiers" in the
1701@c defsystem grammar, but the cross-referencing is so broken by
1702@c insufficient node breakdown that I have not put one in.
1703
1704
1705@subsubsection pathname
1706
1707This attribute is optional and if absent (which is the usual case),
1708the component name will be used.
1709
1710@xref{The defsystem grammar,,Pathname specifiers},
1711for an explanation of how this attribute is interpreted.
1712
1713Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1714does additional processing to set the filesystem location of
1715the top component in that system.
1716This is detailed elsewhere. @xref{Defining systems with defsystem}.
1717
1718
1719@subsubsection properties
1720
1721This attribute is optional.
1722
1723Packaging systems often require information about files or systems
1724in addition to that specified by ASDF's pre-defined component attributes.
1725Programs that create vendor packages out of ASDF systems therefore
1726have to create ``placeholder'' information to satisfy these systems.
1727Sometimes the creator of an ASDF system may know the additional
1728information and wish to provide it directly.
1729
1730@code{(component-property component property-name)} and
1731associated @code{setf} method will allow
1732the programmatic update of this information.
1733Property names are compared as if by @code{EQL},
1734so use symbols or keywords or something.
1735
1736@menu
1737* Pre-defined subclasses of component::
1738* Creating new component types::
1739@end menu
1740
1741@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1742@comment  node-name,  next,  previous,  up
1743@subsection Pre-defined subclasses of component
1744
1745@deffn Component source-file
1746
1747A source file is any file that the system does not know how to
1748generate from other components of the system.
1749
1750Note that this is not necessarily the same thing as
1751``a file containing data that is typically fed to a compiler''.
1752If a file is generated by some pre-processor stage
1753(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1754then it is not, by this definition, a source file.
1755Conversely, we might have a graphic file
1756that cannot be automatically regenerated,
1757or a proprietary shared library that we received as a binary:
1758these do count as source files for our purposes.
1759
1760Subclasses of source-file exist for various languages.
1761@emph{FIXME: describe these.}
1762@end deffn
1763
1764@deffn Component module
1765
1766A module is a collection of sub-components.
1767
1768A module component has the following extra initargs:
1769
1770@itemize
1771@item
1772@code{:components} the components contained in this module
1773
1774@item
1775@code{:default-component-class}
1776All children components which don't specify their class explicitly
1777are inferred to be of this type.
1778
1779@item
1780@code{:if-component-dep-fails}
1781This attribute takes one of the values
1782@code{:fail}, @code{:try-next}, @code{:ignore},
1783its default value is @code{:fail}.
1784The other values can be used for implementing conditional compilation
1785based on implementation @code{*features*},
1786for the case where it is not necessary for all files in a module to be
1787compiled.
1788@emph{FIXME: such conditional compilation has been reported
1789to be broken in 2009.}
1790
1791@item
1792@code{:serial} When this attribute is set,
1793each subcomponent of this component is assumed to depend on all subcomponents
1794before it in the list given to @code{:components}, i.e.
1795all of them are loaded before a compile or load operation is performed on it.
1796
1797@end itemize
1798
1799The default operation knows how to traverse a module, so
1800most operations will not need to provide methods specialised on modules.
1801
1802@code{module} may be subclassed to represent components such as
1803foreign-language linked libraries or archive files.
1804@end deffn
1805
1806@deffn Component system
1807
1808@code{system} is a subclass of @code{module}.
1809
1810A system is a module with a few extra attributes for documentation
1811purposes; these are given elsewhere.
1812@xref{The defsystem grammar}.
1813
1814Users can create new classes for their systems:
1815the default @code{defsystem} macro takes a @code{:class} keyword argument.
1816@end deffn
1817
1818@node  Creating new component types,  , Pre-defined subclasses of component, Components
1819@comment  node-name,  next,  previous,  up
1820@subsection Creating new component types
1821
1822New component types are defined by subclassing one of the existing
1823component classes and specializing methods on the new component class.
1824
1825@emph{FIXME: this should perhaps be explained more throughly,
1826not only by example ...}
1827
1828As an example, suppose we have some implementation-dependent
1829functionality that we want to isolate
1830in one subdirectory per Lisp implementation our system supports.
1831We create a subclass of
1832@code{cl-source-file}:
1833
1834@lisp
1835(defclass unportable-cl-source-file (cl-source-file)
1836  ())
1837@end lisp
1838
1839Function @code{asdf:implementation-type} (exported since 2.014.14)
1840gives us the name of the subdirectory.
1841All that's left is to define how to calculate the pathname
1842of an @code{unportable-cl-source-file}.
1843
1844@lisp
1845(defmethod component-pathname ((component unportable-cl-source-file))
1846  (merge-pathnames*
1847   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1848   (call-next-method)))
1849@end lisp
1850
1851The new component type is used in a @code{defsystem} form in this way:
1852
1853@lisp
1854(defsystem :foo
1855    :components
1856    ((:file "packages")
1857     ...
1858     (:unportable-cl-source-file "threads"
1859      :depends-on ("packages" ...))
1860     ...
1861    )
1862@end lisp
1863
1864@node Functions,  , Components, The object model of ASDF
1865@comment  node-name,  next,  previous,  up
1866@section Functions
1867@findex version-satisfies
1868
1869@deffn version-satisfies @var{version} @var{version-spec}
1870Does @var{version} satisfy the @var{version-spec}.  A generic function.
1871ASDF provides built-in methods for @var{version} being a
1872@code{component} or @code{string}.  @var{version-spec} should be a
1873string.
1874
1875In the wild, we typically see version numbering only on components of
1876type @code{system}.
1877
1878For more information about how @code{version-satisfies} interprets
1879version strings and specifications, @pxref{The defsystem grammar} and
1880@ref{Common attributes of components}.
1881@end deffn
1882
1883@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1884@comment  node-name,  next,  previous,  up
1885@chapter Controlling where ASDF searches for systems
1886
1887@section Configurations
1888
1889Configurations specify paths where to find system files.
1890
1891@enumerate
1892
1893@item
1894The search registry may use some hardcoded wrapping registry specification.
1895This allows some implementations (notably SBCL) to specify where to find
1896some special implementation-provided systems that
1897need to precisely match the version of the implementation itself.
1898
1899@item
1900An application may explicitly initialize the source-registry configuration
1901using the configuration API
1902(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1903in which case this takes precedence.
1904It may itself compute this configuration from the command-line,
1905from a script, from its own configuration file, etc.
1906
1907@item
1908The source registry will be configured from
1909the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1910
1911@item
1912The source registry will be configured from
1913user configuration file
1914@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1915(which defaults to
1916@file{~/.config/common-lisp/source-registry.conf})
1917if it exists.
1918
1919@item
1920The source registry will be configured from
1921user configuration directory
1922@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1923(which defaults to
1924@file{~/.config/common-lisp/source-registry.conf.d/})
1925if it exists.
1926
1927@item
1928The source registry will be configured from
1929system configuration file
1930@file{/etc/common-lisp/source-registry.conf}
1931if it exists/
1932
1933@item
1934The source registry will be configured from
1935system configuration directory
1936@file{/etc/common-lisp/source-registry.conf.d/}
1937if it exists.
1938
1939@item
1940The source registry will be configured from a default configuration.
1941This configuration may allow for implementation-specific systems
1942to be found, for systems to be found the current directory
1943(at the time that the configuration is initialized) as well as
1944@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1945@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1946For instance, SBCL will include directories for its contribs
1947when it can find them; it will look for them where SBCL was installed,
1948or at the location specified by the @code{SBCL_HOME} environment variable.
1949
1950@end enumerate
1951
1952Each of these configurations is specified as an s-expression
1953in a trivial domain-specific language (defined below).
1954Additionally, a more shell-friendly syntax is available
1955for the environment variable (defined yet below).
1956
1957Each of these configurations is only used if the previous
1958configuration explicitly or implicitly specifies that it
1959includes its inherited configuration.
1960
1961Additionally, some implementation-specific directories
1962may be automatically prepended to whatever directories are specified
1963in configuration files, no matter if the last one inherits or not.
1964
1965@section XDG base directory
1966
1967Note that we purport to respect the XDG base directory specification
1968as to where configuration files are located,
1969where data files are located,
1970where output file caches are located.
1971Mentions of XDG variables refer to that document.
1972
1973@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1974
1975This specification allows the user to specify some environment variables
1976to customize how applications behave to his preferences.
1977
1978On Windows platforms, when not using Cygwin,
1979instead of the XDG base directory specification,
1980we try to use folder configuration from the registry regarding
1981@code{Common AppData} and similar directories.
1982However, support for querying the Windows registry is limited as of ASDF 2,
1983and on many implementations, we may fall back to always using the defaults
1984without consulting the registry.
1985Patches welcome.
1986
1987@section Backward Compatibility
1988
1989For backward compatibility as well as to provide a practical backdoor for hackers,
1990ASDF will first search for @code{.asd} files in the directories specified in
1991@code{asdf:*central-registry*}
1992before it searches in the source registry above.
1993
1994@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
1995
1996By default, @code{asdf:*central-registry*} will be empty.
1997
1998This old mechanism will therefore not affect you if you don't use it,
1999but will take precedence over the new mechanism if you do use it.
2000
2001@section Configuration DSL
2002
2003Here is the grammar of the s-expression (SEXP) DSL for source-registry
2004configuration:
2005
2006@c FIXME: This is too wide for happy compilation into pdf.
2007
2008@example
2009;; A configuration is a single SEXP starting with keyword :source-registry
2010;; followed by a list of directives.
2011CONFIGURATION := (:source-registry DIRECTIVE ...)
2012
2013;; A directive is one of the following:
2014DIRECTIVE :=
2015    ;; INHERITANCE DIRECTIVE:
2016    ;; Your configuration expression MUST contain
2017    ;; exactly one of either of these:
2018    :inherit-configuration | ; splices inherited configuration (often specified last)
2019    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2020
2021    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2022    ;; you want to use new configuration features but have to bootstrap a
2023    ;; the newer required ASDF from an older release that doesn't sport said features:
2024    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2025
2026    ;; add a single directory to be scanned (no recursion)
2027    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
2028
2029    ;; add a directory hierarchy, recursing but excluding specified patterns
2030    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
2031
2032    ;; override the defaults for exclusion patterns
2033    (:exclude EXCLUSION-PATTERN ...) |
2034    ;; augment the defaults for exclusion patterns
2035    (:also-exclude EXCLUSION-PATTERN ...) |
2036    ;; Note that the scope of a an exclude pattern specification is
2037    ;; the rest of the current configuration expression or file.
2038
2039    ;; splice the parsed contents of another config file
2040    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
2041
2042    ;; This directive specifies that some default must be spliced.
2043    :default-registry
2044
2045REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
2046DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2047
2048PATHNAME-DESIGNATOR :=
2049    NIL | ;; Special: skip this entry.
2050    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
2051
2052EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2053  against the name of a any subdirectory in the directory component
2054        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2055@end example
2056
2057Pathnames are designated using another DSL,
2058shared with the output-translations configuration DSL below.
2059The DSL is resolved by the function @code{asdf::resolve-location},
2060to be documented and exported at some point in the future.
2061
2062@example
2063ABSOLUTE-COMPONENT-DESIGNATOR :=
2064    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2065    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2066             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2067             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2068             ;; Note that none of the above applies to strings used in *central-registry*,
2069             ;; which doesn't use this DSL: they are processed as normal namestrings.
2070             ;; however, you can compute what you put in the *central-registry*
2071             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2072    PATHNAME | ;; pathname (better be an absolute path, or bust)
2073               ;; In output-translations, unless followed by relative components,
2074               ;; it better have appropriate wildcards, as in **/*.*.*
2075    :HOME | ;; designates the user-homedir-pathname ~/
2076    :USER-CACHE | ;; designates the default location for the user cache
2077    :HERE | ;; designates the location of the configuration file
2078            ;; (or *default-pathname-defaults*, if invoked interactively)
2079    :ROOT ;; magic, for output-translations source only: paths that are relative
2080          ;; to the root of the source host and device
2081    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2082
2083RELATIVE-COMPONENT-DESIGNATOR :=
2084    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2085    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
2086             ;; In output translations, if last component, **/*.*.* is added
2087    PATHNAME | ;; pathname; unless last component, directory is assumed.
2088    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
2089    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2090    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2091    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2092    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2093    :*.*.* | ;; any file (since ASDF 2.011.4)
2094    ;; Not supported (anymore): :UID and :USERNAME
2095@end example
2096
2097For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2098which is the default place ASDF looks for this configuration, once contained:
2099@example
2100(:source-registry
2101  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2102  :inherit-configuration)
2103@end example
2104
2105@section Configuration Directories
2106
2107Configuration directories consist in files each containing
2108a list of directives without any enclosing @code{(:source-registry ...)} form.
2109The files will be sorted by namestring as if by @code{string<} and
2110the lists of directives of these files with be concatenated in order.
2111An implicit @code{:inherit-configuration} will be included
2112at the @emph{end} of the list.
2113
2114This allows for packaging software that has file granularity
2115(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2116to easily include configuration information about distributed software.
2117
2118The convention is that, for sorting purposes,
2119the names of files in such a directory begin with two digits
2120that determine the order in which these entries will be read.
2121Also, the type of these files is conventionally @code{"conf"}
2122and as a limitation to some implementations (e.g. GNU clisp),
2123the type cannot be @code{NIL}.
2124
2125Directories may be included by specifying a directory pathname
2126or namestring in an @code{:include} directive, e.g.:
2127
2128@example
2129  (:include "/foo/bar/")
2130@end example
2131
2132Hence, to achieve the same effect as
2133my example @file{~/.config/common-lisp/source-registry.conf} above,
2134I could simply create a file
2135@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2136alone in its directory with the following contents:
2137@example
2138(:tree "/home/fare/cl/")
2139@end example
2140
2141@subsection The :here directive
2142
2143The @code{:here} directive is an absolute pathname designator that
2144refers to the directory containing the configuration file currently
2145being processed.
2146
2147The @code{:here} directive is intended to simplify the delivery of
2148complex CL systems, and for easy configuration of projects shared through
2149revision control systems, in accordance with our design principle that
2150each participant should be able to provide all and only the information
2151available to him or her.
2152
2153Consider a person X who has set up the source code repository for a
2154complex project with a master directory @file{dir/}.  Ordinarily, one
2155might simply have the user add a directive that would look something
2156like this:
2157@example
2158   (:tree  "path/to/dir")
2159@end example
2160But what if X knows that there are very large subtrees
2161under dir that are filled with, e.g., Java source code, image files for
2162icons, etc.?  All of the asdf system definitions are contained in the
2163subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2164these are the only directories that should be searched.
2165
2166In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2167contains the following:
2168@example
2169(:source-registry
2170   (:tree (:here "src/lisp/"))
2171   (:tree (:here "extlib/lisp"))
2172   (:directory (:here "outlier/")))
2173@end example
2174
2175Then when someone else (call her Y) checks out a copy of this
2176repository, she need only add
2177@example
2178(:include "/path/to/my/checkout/directory/asdf.conf")
2179@end example
2180to one of her previously-existing asdf source location configuration
2181files, or invoke @code{initialize-source-registry} with a configuration
2182form containing that s-expression.  ASDF will find the .conf file that X
2183has provided, and then set up source locations within the working
2184directory according to X's (relative) instructions.
2185
2186@section Shell-friendly syntax for configuration
2187
2188When considering environment variable @code{CL_SOURCE_REGISTRY}
2189ASDF will skip to next configuration if it's an empty string.
2190It will @code{READ} the string as a SEXP in the DSL
2191if it begins with a paren @code{(}
2192and it will be interpreted much like @code{TEXINPUTS}
2193list of paths, where
2194
2195  * paths are separated
2196   by a @code{:} (colon) on Unix platforms (including cygwin),
2197   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2198
2199  * each entry is a directory to add to the search path.
2200
2201  * if the entry ends with a double slash @code{//}
2202    then it instead indicates a tree in the subdirectories
2203    of which to recurse.
2204
2205  * if the entry is the empty string (which may only appear once),
2206    then it indicates that the inherited configuration should be
2207    spliced there.
2208
2209
2210@section Search Algorithm
2211@vindex *default-source-registry-exclusions*
2212
2213In case that isn't clear, the semantics of the configuration is that
2214when searching for a system of a given name,
2215directives are processed in order.
2216
2217When looking in a directory, if the system is found, the search succeeds,
2218otherwise it continues.
2219
2220When looking in a tree, if one system is found, the search succeeds.
2221If multiple systems are found, the consequences are unspecified:
2222the search may succeed with any of the found systems,
2223or an error may be raised.
2224ASDF currently returns the first system found,
2225XCVB currently raised an error.
2226If none is found, the search continues.
2227
2228Exclude statements specify patterns of subdirectories
2229the systems from which to ignore.
2230Typically you don't want to use copies of files kept by such
2231version control systems as Darcs.
2232Exclude statements are not propagated to further included or inherited
2233configuration files or expressions;
2234instead the defaults are reset around every configuration statement
2235to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2236
2237Include statements cause the search to recurse with the path specifications
2238from the file specified.
2239
2240An inherit-configuration statement cause the search to recurse with the path
2241specifications from the next configuration
2242(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2243
2244
2245@section Caching Results
2246
2247The implementation is allowed to either eagerly compute the information
2248from the configurations and file system, or to lazily re-compute it
2249every time, or to cache any part of it as it goes.
2250To explicitly flush any information cached by the system, use the API below.
2251
2252
2253@section Configuration API
2254
2255The specified functions are exported from your build system's package.
2256Thus for ASDF the corresponding functions are in package ASDF,
2257and for XCVB the corresponding functions are in package XCVB.
2258
2259@defun initialize-source-registry @&optional PARAMETER
2260   will read the configuration and initialize all internal variables.
2261   You may extend or override configuration
2262   from the environment and configuration files
2263   with the given @var{PARAMETER}, which can be
2264   @code{NIL} (no configuration override),
2265   or a SEXP (in the SEXP DSL),
2266   a string (as in the string DSL),
2267   a pathname (of a file or directory with configuration),
2268   or a symbol (fbound to function that when called returns one of the above).
2269@end defun
2270
2271@defun clear-source-registry
2272   undoes any source registry configuration
2273   and clears any cache for the search algorithm.
2274   You might want to call this function
2275   (or better, @code{clear-configuration})
2276   before you dump an image that would be resumed
2277   with a different configuration,
2278   and return an empty configuration.
2279   Note that this does not include clearing information about
2280   systems defined in the current image, only about
2281   where to look for systems not yet defined.
2282@end defun
2283
2284@defun ensure-source-registry @&optional PARAMETER
2285   checks whether a source registry has been initialized.
2286   If not, initialize it with the given @var{PARAMETER}.
2287@end defun
2288
2289Every time you use ASDF's @code{find-system}, or
2290anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2291@code{ensure-source-registry} is called with parameter NIL,
2292which the first time around causes your configuration to be read.
2293If you change a configuration file,
2294you need to explicitly @code{initialize-source-registry} again,
2295or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2296which will cause the initialization to happen next time around.
2297
2298
2299@section Future
2300
2301If this mechanism is successful, in the future, we may declare
2302@code{asdf:*central-registry*} obsolete and eventually remove it.
2303Any hook into implementation-specific search mechanisms will by then
2304have been integrated in the @code{:default-configuration} which everyone
2305should either explicitly use or implicit inherit. Some shell syntax
2306for it should probably be added somehow.
2307
2308But we're not there yet. For now, let's see how practical this new
2309source-registry is.
2310
2311
2312@section Rejected ideas
2313
2314Alternatives I considered and rejected included:
2315
2316@enumerate
2317@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2318   and somehow the configuration parser expands the new configuration
2319   language into a expanded series of directories of subdirectories to
2320   lookup, pre-recursing through specified hierarchies. This is kludgy,
2321   and leaves little space of future cleanups and extensions.
2322
2323@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2324   in completely new ways, so that new kinds of entries may be implemented
2325   as a recursive search, etc. This seems somewhat backwards.
2326
2327@item Completely remove @code{asdf:*central-registry*}
2328   and break backwards compatibility.
2329   Hopefully this will happen in a few years after everyone migrate to
2330   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2331
2332@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2333   when you dereference it or setf it. Only the new variable with new
2334   semantics is handled by the new search procedure.
2335   Complex and still introduces subtle semantic issues.
2336@end enumerate
2337
2338
2339I've been suggested the below features, but have rejected them,
2340for the sake of keeping ASDF no more complex than strictly necessary.
2341
2342@itemize
2343@item
2344  More syntactic sugar: synonyms for the configuration directives, such as
2345  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2346  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2347
2348@item
2349   The possibility to register individual files instead of directories.
2350
2351@item
2352  Integrate Xach Beane's tilde expander into the parser,
2353  or something similar that is shell-friendly or shell-compatible.
2354  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2355  minimal by removing the need for evaluated entries that ASDF has?
2356  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2357  Hopefully, these are already superseded by the @code{:default-registry}
2358
2359@item
2360  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2361  down a filesystem tree in the environment variable.
2362  It isn't that Lisp friendly either.
2363@end itemize
2364
2365@section TODO
2366
2367@itemize
2368@item Add examples
2369@end itemize
2370
2371
2372@section Credits for the source-registry
2373
2374Thanks a lot to Stelian Ionescu for the initial idea.
2375
2376Thanks to Rommel Martinez for the initial implementation attempt.
2377
2378All bad design ideas and implementation bugs are to mine, not theirs.
2379But so are good design ideas and elegant implementation tricks.
2380
2381 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2382
2383
2384
2385@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2386@comment  node-name,  next,  previous,  up
2387@chapter Controlling where ASDF saves compiled files
2388@cindex asdf-output-translations
2389@vindex ASDF_OUTPUT_TRANSLATIONS
2390
2391Each Common Lisp implementation has its own format
2392for compiled files (fasls for short, short for ``fast loading'').
2393If you use multiple implementations
2394(or multiple versions of the same implementation),
2395you'll soon find your source directories
2396littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2397Worse yet, some implementations use the same file extension
2398while changing formats from version to version (or platform to platform)
2399which means that you'll have to recompile binaries
2400as you switch from one implementation to the next.
2401
2402ASDF 2 includes the @code{asdf-output-translations} facility
2403to mitigate the problem.
2404
2405@section Configurations
2406
2407Configurations specify mappings from input locations to output locations.
2408Once again we rely on the XDG base directory specification for configuration.
2409@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2410
2411@enumerate
2412
2413@item
2414Some hardcoded wrapping output translations configuration may be used.
2415This allows special output translations (or usually, invariant directories)
2416to be specified corresponding to the similar special entries in the source registry.
2417
2418@item
2419An application may explicitly initialize the output-translations
2420configuration using the Configuration API
2421in which case this takes precedence.
2422(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2423It may itself compute this configuration from the command-line,
2424from a script, from its own configuration file, etc.
2425
2426@item
2427The source registry will be configured from
2428the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2429
2430@item
2431The source registry will be configured from
2432user configuration file
2433@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2434(which defaults to
2435@file{~/.config/common-lisp/asdf-output-translations.conf})
2436if it exists.
2437
2438@item
2439The source registry will be configured from
2440user configuration directory
2441@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2442(which defaults to
2443@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2444if it exists.
2445
2446@item
2447The source registry will be configured from
2448system configuration file
2449@file{/etc/common-lisp/asdf-output-translations.conf}
2450if it exists.
2451
2452@item
2453The source registry will be configured from
2454system configuration directory
2455@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2456if it exists.
2457
2458@end enumerate
2459
2460Each of these configurations is specified as a SEXP
2461in a trival domain-specific language (defined below).
2462Additionally, a more shell-friendly syntax is available
2463for the environment variable (defined yet below).
2464
2465Each of these configurations is only used if the previous
2466configuration explicitly or implicitly specifies that it
2467includes its inherited configuration.
2468
2469Note that by default, a per-user cache is used for output files.
2470This allows the seamless use of shared installations of software
2471between several users, and takes files out of the way of the developers
2472when they browse source code,
2473at the expense of taking a small toll when developers have to clean up
2474output files and find they need to get familiar with output-translations first.
2475
2476
2477@section Backward Compatibility
2478@cindex ASDF-BINARY-LOCATIONS compatibility
2479
2480
2481We purposefully do NOT provide backward compatibility with earlier versions of
2482@code{ASDF-Binary-Locations} (8 Sept 2009),
2483@code{common-lisp-controller} (7.0) or
2484@code{cl-launch} (2.35),
2485each of which had similar general capabilities.
2486The previous APIs of these programs were not designed
2487for configuration by the end-user
2488in an easy way with configuration files.
2489Recent versions of same packages use
2490the new @code{asdf-output-translations} API as defined below:
2491@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2492@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2493
2494This incompatibility shouldn't inconvenience many people.
2495Indeed, few people use and customize these packages;
2496these few people are experts who can trivially adapt to the new configuration.
2497Most people are not experts, could not properly configure these features
2498(except inasmuch as the default configuration of
2499@code{common-lisp-controller} and/or @code{cl-launch}
2500might have been doing the right thing for some users),
2501and yet will experience software that ``just works'',
2502as configured by the system distributor, or by default.
2503
2504Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2505we provide a limited emulation mode:
2506
2507@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2508This function will initialize the new @code{asdf-output-translations} facility in a way
2509that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2510Where you would previously set global variables
2511@var{*centralize-lisp-binaries*},
2512@var{*default-toplevel-directory*},
2513@var{*include-per-user-information*},
2514@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2515you will now have to pass the same values as keyword arguments to this function.
2516Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2517will accept any valid pathname designator for @code{asdf-output-translations}
2518instead of just strings and pathnames.
2519@end defun
2520
2521If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2522(the one available as an extension to load of top of ASDF,
2523not the one built into a few old versions of ASDF),
2524but first you must disable @code{asdf-output-translations}
2525with @code{(asdf:disable-output-translations)},
2526or you might experience ``interesting'' issues.
2527
2528Also, note that output translation is enabled by default.
2529To disable it, use @code{(asdf:disable-output-translations)}.
2530
2531
2532@section Configuration DSL
2533
2534Here is the grammar of the SEXP DSL
2535for @code{asdf-output-translations} configuration:
2536
2537@verbatim
2538;; A configuration is single SEXP starting with keyword :source-registry
2539;; followed by a list of directives.
2540CONFIGURATION := (:output-translations DIRECTIVE ...)
2541
2542;; A directive is one of the following:
2543DIRECTIVE :=
2544    ;; INHERITANCE DIRECTIVE:
2545    ;; Your configuration expression MUST contain
2546    ;; exactly one of either of these:
2547    :inherit-configuration | ; splices inherited configuration (often specified last)
2548    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2549
2550    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2551    ;; you want to use new configuration features but have to bootstrap a
2552    ;; the newer required ASDF from an older release that doesn't sport said features:
2553    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2554
2555    ;; include a configuration file or directory
2556    (:include PATHNAME-DESIGNATOR) |
2557
2558    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2559    :enable-user-cache |
2560    ;; Disable global cache. Map / to /
2561    :disable-cache |
2562
2563    ;; add a single directory to be scanned (no recursion)
2564    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2565
2566    ;; use a function to return the translation of a directory designator
2567    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2568
2569DIRECTORY-DESIGNATOR :=
2570    NIL | ;; As source: skip this entry. As destination: same as source
2571    T | ;; as source matches anything, as destination leaves pathname unmapped.
2572    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2573
2574TRANSLATION-FUNCTION :=
2575    SYMBOL | ;; symbol of a function that takes two arguments,
2576             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2577    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2578             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2579
2580@end verbatim
2581
2582Relative components better be either relative
2583or subdirectories of the path before them, or bust.
2584
2585The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2586You can specify more fine-grained patterns
2587by using a pathname object as the last component
2588e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2589
2590You may use @code{#+features} to customize the configuration file.
2591
2592The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2593to anything but themselves (same as if the second designator was the same as the first).
2594
2595When the first designator is @code{t},
2596the mapping always matches.
2597When the first designator starts with @code{:root},
2598the mapping matches any host and device.
2599In either of these cases, if the second designator
2600isn't @code{t} and doesn't start with @code{:root},
2601then strings indicating the host and pathname are somehow copied
2602in the beginning of the directory component of the source pathname
2603before it is translated.
2604
2605When the second designator is @code{t}, the mapping is the identity.
2606When the second designator starts with @code{:root},
2607the mapping preserves the host and device of the original pathname.
2608Notably, this allows you to map files
2609to a subdirectory of the whichever directory the file is in.
2610Though the syntax is not quite as easy to use as we'd like,
2611you can have an (source destination) mapping entry such as follows
2612in your configuration file,
2613or you may use @code{enable-asdf-binary-locations-compatibility}
2614with @code{:centralize-lisp-binaries nil}
2615which will do the same thing internally for you:
2616@verbatim
2617  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2618          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2619     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2620       (:root ,wild-subdir :implementation ,wild-file)))
2621@end verbatim
2622Starting with ASDF 2.011.4, you can use the simpler:
2623  @code{`(:root (:root :**/ :implementation :*.*.*))}
2624
2625
2626
2627@code{:include} statements cause the search to recurse with the path specifications
2628from the file specified.
2629
2630If the @code{translate-pathname} mechanism cannot achieve a desired
2631translation, the user may provide a function which provides the
2632required algorithim.  Such a translation function is specified by
2633supplying a list as the second @code{directory-designator}
2634the first element of which is the keyword @code{:function},
2635and the second element of which is
2636either a symbol which designates a function or a lambda expression.
2637The function designated by the second argument must take two arguments,
2638the first being the pathname of the source file,
2639the second being the wildcard that was matched.
2640The result of the function invocation should be the translated pathname.
2641
2642An @code{:inherit-configuration} statement cause the search to recurse with the path
2643specifications from the next configuration.
2644@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2645
2646@itemize
2647@item
2648@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2649@item
2650@code{:disable-cache} is the same as @code{(t t)}.
2651@item
2652@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2653which by default is the same as using
2654@code{(:home ".cache" "common-lisp" :implementation)}.
2655@item
2656@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2657which by default is the same as using
2658@code{("/var/cache/common-lisp" :uid :implementation-type)}
2659(on Unix and cygwin), or something semi-sensible on Windows.
2660@end itemize
2661
2662
2663@section Configuration Directories
2664
2665Configuration directories consist in files each contains
2666a list of directives without any enclosing
2667@code{(:output-translations ...)} form.
2668The files will be sorted by namestring as if by @code{string<} and
2669the lists of directives of these files with be concatenated in order.
2670An implicit @code{:inherit-configuration} will be included
2671at the @emph{end} of the list.
2672
2673This allows for packaging software that has file granularity
2674(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2675to easily include configuration information about software being distributed.
2676
2677The convention is that, for sorting purposes,
2678the names of files in such a directory begin with two digits
2679that determine the order in which these entries will be read.
2680Also, the type of these files is conventionally @code{"conf"}
2681and as a limitation of some implementations, the type cannot be @code{NIL}.
2682
2683Directories may be included by specifying a directory pathname
2684or namestring in an @code{:include} directive, e.g.:
2685@verbatim
2686  (:include "/foo/bar/")
2687@end verbatim
2688
2689@section Shell-friendly syntax for configuration
2690
2691When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2692ASDF will skip to next configuration if it's an empty string.
2693It will @code{READ} the string as an SEXP in the DSL
2694if it begins with a paren @code{(}
2695and it will be interpreted as a list of directories.
2696Directories should come by pairs, indicating a mapping directive.
2697Entries are separated
2698by a @code{:} (colon) on Unix platforms (including cygwin),
2699by a @code{;} (semicolon) on other platforms (mainly, Windows).
2700
2701The magic empty entry,
2702if it comes in what would otherwise be the first entry in a pair,
2703indicates the splicing of inherited configuration.
2704If it comes as the second entry in a pair,
2705it indicates that the directory specified first is to be left untranslated
2706(which has the same effect as if the directory had been repeated).
2707
2708
2709@section Semantics of Output Translations
2710
2711From the specified configuration,
2712a list of mappings is extracted in a straightforward way:
2713mappings are collected in order, recursing through
2714included or inherited configuration as specified.
2715To this list is prepended some implementation-specific mappings,
2716and is appended a global default.
2717
2718The list is then compiled to a mapping table as follows:
2719for each entry, in order, resolve the first designated directory
2720into an actual directory pathname for source locations.
2721If no mapping was specified yet for that location,
2722resolve the second designated directory to an output location directory
2723add a mapping to the table mapping the source location to the output location,
2724and add another mapping from the output location to itself
2725(unless a mapping already exists for the output location).
2726
2727Based on the table, a mapping function is defined,
2728mapping source pathnames to output pathnames:
2729given a source pathname, locate the longest matching prefix
2730in the source column of the mapping table.
2731Replace that prefix by the corresponding output column
2732in the same row of the table, and return the result.
2733If no match is found, return the source pathname.
2734(A global default mapping the filesystem root to itself
2735may ensure that there will always be a match,
2736with same fall-through semantics).
2737
2738@section Caching Results
2739
2740The implementation is allowed to either eagerly compute the information
2741from the configurations and file system, or to lazily re-compute it
2742every time, or to cache any part of it as it goes.
2743To explicitly flush any information cached by the system, use the API below.
2744
2745
2746@section Output location API
2747
2748The specified functions are exported from package ASDF.
2749
2750@defun initialize-output-translations @&optional PARAMETER
2751   will read the configuration and initialize all internal variables.
2752   You may extend or override configuration
2753   from the environment and configuration files
2754   with the given @var{PARAMETER}, which can be
2755   @code{NIL} (no configuration override),
2756   or a SEXP (in the SEXP DSL),
2757   a string (as in the string DSL),
2758   a pathname (of a file or directory with configuration),
2759   or a symbol (fbound to function that when called returns one of the above).
2760@end defun
2761
2762@defun disable-output-translations
2763   will initialize output translations in a way
2764   that maps every pathname to itself,
2765   effectively disabling the output translation facility.
2766@end defun
2767
2768@defun clear-output-translations
2769   undoes any output translation configuration
2770   and clears any cache for the mapping algorithm.
2771   You might want to call this function
2772   (or better, @code{clear-configuration})
2773   before you dump an image that would be resumed
2774   with a different configuration,
2775   and return an empty configuration.
2776   Note that this does not include clearing information about
2777   systems defined in the current image, only about
2778   where to look for systems not yet defined.
2779@end defun
2780
2781@defun ensure-output-translations @&optional PARAMETER
2782   checks whether output translations have been initialized.
2783   If not, initialize them with the given @var{PARAMETER}.
2784   This function will be called before any attempt to operate on a system.
2785@end defun
2786
2787@defun apply-output-translations PATHNAME
2788   Applies the configured output location translations to @var{PATHNAME}
2789   (calls @code{ensure-output-translations} for the translations).
2790@end defun
2791
2792Every time you use ASDF's @code{output-files}, or
2793anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2794@code{ensure-output-translations} is called with parameter NIL,
2795which the first time around causes your configuration to be read.
2796If you change a configuration file,
2797you need to explicitly @code{initialize-output-translations} again,
2798or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2799which will cause the initialization to happen next time around.
2800
2801
2802@section Credits for output translations
2803
2804Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2805and to Peter van Eynde for @code{Common Lisp Controller}.
2806
2807All bad design ideas and implementation bugs are to mine, not theirs.
2808But so are good design ideas and elegant implementation tricks.
2809
2810 --- Francois-Rene Rideau @email{fare@@tunes.org}
2811
2812@c @section Default locations
2813@c @findex output-files-for-system-and-operation
2814
2815@c The default binary location for each Lisp implementation
2816@c is a subdirectory of each source directory.
2817@c To account for different Lisps, Operating Systems, Implementation versions,
2818@c and so on, ASDF borrows code from SLIME
2819@c to create reasonable custom directory names.
2820@c Here are some examples:
2821
2822@c @itemize
2823@c @item
2824@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2825
2826@c @item
2827@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2828
2829@c @item
2830@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2831@c @end itemize
2832
2833@c By default, all output file pathnames will be relocated
2834@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2835
2836@c See the document @file{README.asdf-output-translations}
2837@c for a full specification on how to configure @code{asdf-output-translations}.
2838
2839@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2840@comment  node-name,  next,  previous,  up
2841@chapter Error handling
2842@findex SYSTEM-DEFINITION-ERROR
2843@findex OPERATION-ERROR
2844
2845@section ASDF errors
2846
2847If ASDF detects an incorrect system definition, it will signal a generalised instance of
2848@code{SYSTEM-DEFINITION-ERROR}.
2849
2850Operations may go wrong (for example when source files contain errors).
2851These are signalled using generalised instances of
2852@code{OPERATION-ERROR}.
2853
2854@section Compilation error and warning handling
2855@vindex *compile-file-warnings-behaviour*
2856@vindex *compile-file-errors-behavior*
2857
2858ASDF checks for warnings and errors when a file is compiled.
2859The variables @var{*compile-file-warnings-behaviour*} and
2860@var{*compile-file-errors-behavior*}
2861control the handling of any such events.
2862The valid values for these variables are
2863@code{:error}, @code{:warn}, and @code{:ignore}.
2864
2865@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2866@comment  node-name,  next,  previous,  up
2867@chapter Miscellaneous additional functionality
2868
2869ASDF includes several additional features that are generally
2870useful for system definition and development.
2871
2872@section Controlling file compilation
2873
2874When declaring a component (system, module, file),
2875you can specify a keyword argument @code{:around-compile function}.
2876If left unspecified,
2877the value will be inherited from the parent component if any,
2878or with a default of @code{nil}
2879if no value is specified in any transitive parent.
2880
2881The argument must be a either @code{nil}, a fbound symbol,
2882a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
2883a function object (e.g. using @code{#.#'} but that's discouraged
2884because it prevents the introspection done by e.g. asdf-dependency-grovel),
2885or a string that when read yields a symbol or a lambda-expression.
2886@code{nil} means the normal compile-file function will be called.
2887A non-nil value designates a function of one argument
2888that will be called with a function that
2889calls the @code{*compile-op-compile-file-function*} (usually @code{compile-file*})
2890with proper arguments;
2891the around-compile hook may supply additional arguments
2892to pass to that @code{*compile-op-compile-file-function*}.
2893One notable argument that is heeded by @code{compile-file*} is
2894@code{:compile-check}, a function called when the compilation was otherwise a success,
2895with the same arguments as @code{compile-file},
2896to determine whether
2897(NB: The ability to pass such extra flags is only available starting with asdf 2.22.1.)
2898
2899Note that by using a string, you may reference
2900a function, symbol and/or package
2901that will only be created later during the build, but
2902isn't yet present at the time the defsystem form is evaluated.
2903However, if your entire system is using such a hook, you may have to
2904explicitly override the hook with @code{nil} for all the modules and files
2905that are compiled before the hook is defined.
2906
2907Using this hook, you may achieve such effects as:
2908locally renaming packages,
2909binding @var{*readtables*} and other syntax-controlling variables,
2910handling warnings and other conditions,
2911proclaiming consistent optimization settings,
2912saving code coverage information,
2913maintaining meta-data about compilation timings,
2914setting gensym counters and PRNG seeds and other sources of non-determinism,
2915overriding the source-location and/or timestamping systems,
2916checking that some compile-time side-effects were properly balanced,
2917etc.
2918
2919Note that there is no around-load hook. This is on purpose.
2920Some implementations such as ECL, GCL or MKCL link object files,
2921which allows for no such hook.
2922Other implementations allow for concatenating FASL files,
2923which doesn't allow for such a hook either.
2924We aim to discourage something that's not portable,
2925and has some dubious impact on performance and semantics
2926even when it is possible.
2927Things you might want to do with an around-load hook
2928are better done around-compile,
2929though it may at times require some creativity
2930(see e.g. the @code{package-renaming} system).
2931
2932
2933@section Controlling source file character encoding
2934
2935Starting with ASDF 2.21, components accept a @code{:encoding} option
2936so authors may specify which character encoding should be used
2937to read and evaluate their source code.
2938When left unspecified, the encoding is inherited
2939from the parent module or system;
2940if no encoding is specified at any point,
2941the default @code{:autodetect} is assumed.
2942By default, only @code{:default}, @code{:utf-8}
2943and @code{:autodetect} are accepted.
2944@code{:autodetect}, the default, calls
2945@code{*encoding-detection-hook*} which by default always returns
2946@code{*default-encoding*} which itself defaults to @code{:default}.
2947
2948In other words, there now are plenty of extension hooks, but
2949by default ASDF follows the backwards compatible behavior
2950of using whichever @code{:default} encoding your implementation uses,
2951which itself may or may not vary based on environment variables
2952and other locale settings.
2953In practice this means that only source code that only uses ASCII
2954is guaranteed to be read the same on all implementations
2955independently from any user setting.
2956
2957Additionally, for backward-compatibility with older versions of ASDF
2958and/or with implementations that do not support unicode and its many encodings,
2959you may want to use
2960the reader conditionals @code{#+asdf-unicode #+asdf-unicode}
2961to protect any @code{:encoding @emph{encoding}} statement
2962as @code{:asdf-unicode} will be present in @code{*features*}
2963only if you're using a recent ASDF
2964on an implementation that supports unicode.
2965We recommend that you avoid using unprotected @code{:encoding} specifications
2966until after ASDF 2.21 or later becomes widespread, hopefully by the end of 2012.
2967
2968While it offers plenty of hooks for extension,
2969and one such extension is being developed (see below),
2970ASDF itself only recognizes one encoding beside @code{:default},
2971and that is @code{:utf-8}, which is the @emph{de facto} standard,
2972already used by the vast majority of libraries that use more than ASCII.
2973On implementations that do not support unicode,
2974the feature @code{:asdf-unicode} is absent, and
2975the @code{:default} external-format is used
2976to read even source files declared as @code{:utf-8}.
2977On these implementations, non-ASCII characters
2978intended to be read as one CL character
2979may thus end up being read as multiple CL characters.
2980In most cases, this shouldn't affect the software's semantics:
2981comments will be skipped just the same, strings with be read and printed
2982with slightly different lengths, symbol names will be accordingly longer,
2983but none of it should matter.
2984But a few systems that actually depend on unicode characters
2985may fail to work properly, or may work in a subtly different way.
2986See for instance @code{lambda-reader}.
2987
2988We invite you to embrace UTF-8
2989as the encoding for non-ASCII characters starting today,
2990even without any explicit specification in your @code{.asd} files.
2991Indeed, on some implementations and configurations,
2992UTF-8 is already the @code{:default},
2993and loading your code may cause errors if it is encoded in anything but UTF-8.
2994Therefore, even with the legacy behavior,
2995non-UTF-8 is guaranteed to break for some users,
2996whereas UTF-8 is pretty much guaranteed not to break anywhere
2997(provided you do @emph{not} use a BOM),
2998although it might be read incorrectly on some implementations.
2999In the future, we intend to make @code{:utf-8}
3000the default value of @code{*default-encoding*},
3001to be enforced everywhere, so at least the code is guaranteed
3002to be read correctly everywhere it can be.
3003
3004If you need non-standard character encodings for your source code,
3005use the extension system @code{asdf-encodings}, by specifying
3006@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
3007This extension system will register support for more encodings using the
3008@code{*encoding-external-format-hook*} facility,
3009so you can explicitly specify @code{:encoding :latin1}
3010in your @code{.asd} file.
3011Using the @code{*encoding-detection-hook*} it will also
3012eventually implement some autodetection of a file's encoding
3013from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
3014or otherwise based on an analysis of octet patterns in the file.
3015At this point, asdf-encoding only supports the encodings
3016that are supported as part of your implementation.
3017Since the list varies depending on implementations,
3018we once again recommend you use @code{:utf-8} everywhere,
3019which is the most portable (next is @code{:latin1}).
3020
3021If you're not using a version of Quicklisp that has it,
3022you may get the source for @code{asdf-encodings} using git:
3023@kbd{git clone git://common-lisp.net/projects/asdf/asdf-encodings.git}
3024or
3025@kbd{git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git}.
3026You can also browse the repository on
3027@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git}.
3028
3029In the future, we intend to change the default @code{*default-encoding*}
3030to @code{:utf-8}, which is already the de facto standard
3031for most libraries that use non-ASCII characters:
3032utf-8 works everywhere and was backhandedly enforced by
3033a lot of people using SBCL and utf-8 and sending reports to authors
3034so they make their packages compatible.
3035A survey showed only about a handful few libraries
3036are incompatible with non-UTF-8, and then, only in comments,
3037and we believe that authors will adopt UTF-8 when prompted.
3038See the April 2012 discussion on the asdf-devel mailing-list.
3039For backwards compatibility with users who insist on a non-UTF-8 encoding,
3040but cannot immediately transition to using @code{asdf-encodings}
3041(maybe because it isn't ready), it will still be possible to use
3042the @code{:encoding :default} option in your @code{defsystem} form
3043to restore the behavior of ASDF 2.20 and earlier.
3044This shouldn't be required in libraries,
3045because user pressure as mentioned above will already have pushed
3046library authors towards using UTF-8;
3047but authors of end-user programs might care.
3048
3049When you use @code{asdf-encodings}, any further loaded @code{.asd} file
3050will use the autodetection algorithm to determine its encoding;
3051yet if you depend on this detection happening,
3052you may want to explicitly load @code{asdf-encodings} early in your build,
3053for by the time you can use @code{:defsystem-depends-on},
3054it is already too late to load it.
3055In practice, this means that the @code{*default-encoding*}
3056is usually used for @code{.asd} files.
3057Currently, this defaults to @code{:default} for backwards compatibility,
3058and that means that you shouldn't rely on non-ASCII characters in a .asd file.
3059Since component (path)names are the only real data in these files,
3060and non-ASCII characters are not very portable for file names,
3061this isn't too much of an issue.
3062We still encourage you to use either plain ASCII or UTF-8
3063in @code{.asd} files,
3064as we intend to make @code{:utf-8} the default encoding in the future.
3065This might matter, for instance, in meta-data about author's names.
3066
3067
3068@section Miscellaneous Exported Functions
3069
3070@defun coerce-pathname name @&key type defaults
3071
3072This function (available starting with ASDF 2.012.11)
3073takes an argument, and portably interprets it as a pathname.
3074If the argument @var{name} is a pathname or @code{nil}, it is passed through;
3075if it's a symbol, it's interpreted as a string by downcasing it;
3076if it's a string, it is first separated using @code{/} into substrings;
3077the leading substrings denote subdirectories of a relative pathname.
3078If @var{type} is @code{:directory} or the string ends with @code{/},
3079the last substring is also a subdirectory;
3080if @var{type} is a string, it is used as the type of the pathname, and
3081the last substring is the name component of the pathname;
3082if @var{type} is @code{nil}, the last substring specifies both name and type components
3083of the pathname, with the last @code{.} separating them, or only the name component
3084if there's no last @code{.} or if there is only one dot and it's the first character.
3085The host, device and version components come from @var{defaults}, which defaults to
3086@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
3087
3088@end defun
3089
3090@defun merge-pathnames* @&key specified defaults
3091
3092This function is a replacement for @code{merge-pathnames} that uses the host and device
3093from the @var{defaults} rather than the @var{specified} pathname when the latter
3094is a relative pathname. This allows ASDF and its users to create and use relative pathnames
3095without having to know beforehand what are the host and device
3096of the absolute pathnames they are relative to.
3097
3098@end defun
3099
3100@defun system-relative-pathname system name @&key type
3101
3102It's often handy to locate a file relative to some system.
3103The @code{system-relative-pathname} function meets this need.
3104
3105It takes two mandatory arguments @var{system} and @var{name}
3106and a keyword argument @var{type}:
3107@var{system} is name of a system, whereas @var{name} and optionally @var{type}
3108specify a relative pathname, interpreted like a component pathname specifier
3109by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
3110
3111It returns a pathname built from the location of the system's
3112source directory and the relative pathname. For example:
3113
3114@lisp
3115> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
3116#P"/repository/other/cl-ppcre/regex.data"
3117@end lisp
3118
3119@end defun
3120
3121@defun system-source-directory system-designator
3122
3123ASDF does not provide a turnkey solution for locating
3124data (or other miscellaneous) files
3125that are distributed together with the source code of a system.
3126Programmers can use @code{system-source-directory} to find such files.
3127Returns a pathname object.
3128The @var{system-designator} may be a string, symbol, or ASDF system object.
3129@end defun
3130
3131@defun clear-system system-designator
3132
3133It is sometimes useful to force recompilation of a previously loaded system.
3134In these cases, it may be useful to @code{(asdf:clear-system :foo)}
3135to remove the system from the table of currently loaded systems;
3136the next time the system @code{foo} or one that depends on it is re-loaded,
3137@code{foo} will then be loaded again.
3138Alternatively, you could touch @code{foo.asd} or
3139remove the corresponding fasls from the output file cache.
3140(It was once conceived that one should provide
3141a list of systems the recompilation of which to force
3142as the @code{:force} keyword argument to @code{load-system};
3143but this has never worked, and though the feature was fixed in ASDF 2.000,
3144it remains @code{cerror}'ed out as nobody ever used it.)
3145
3146Note that this does not and cannot by itself undo the previous loading
3147of the system. Common Lisp has no provision for such an operation,
3148and its reliance on irreversible side-effects to global datastructures
3149makes such a thing impossible in the general case.
3150If the software being re-loaded is not conceived with hot upgrade in mind,
3151this re-loading may cause many errors, warnings or subtle silent problems,
3152as packages, generic function signatures, structures, types, macros, constants, etc.
3153are being redefined incompatibly.
3154It is up to the user to make sure that reloading is possible and has the desired effect.
3155In some cases, extreme measures such as recursively deleting packages,
3156unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
3157and much more are necessary for reloading to happen smoothly.
3158ASDF itself goes through notable pains to make such a hot upgrade possible
3159with respect to its own code, and what it does is ridiculously complex;
3160look at the beginning of @file{asdf.lisp} to see what it does.
3161@end defun
3162
3163@defun run-shell-command
3164
3165This function is obsolete and present only for the sake of backwards-compatibility:
3166``If it's not backwards, it's not compatible''. We strongly discourage its use.
3167Its current behavior is only well-defined on Unix platforms
3168(which include MacOS X and cygwin). On Windows, anything goes.
3169
3170Instead we recommend the use of such a function as
3171@code{xcvb-driver:run-program/}
3172from the @code{xcvb-driver} system that is distributed with XCVB:
3173@url{http://common-lisp.net/project/xcvb}.
3174It's the only alternative that supports
3175as many implementations and operating systems as ASDF does,
3176and provides well-defined behavior outside Unix (i.e. on Windows).
3177(The only unsupported exception is Genera, since on it
3178@code{run-shell-command} doesn't make sense anyway on that platform).
3179
3180@code{run-shell-command} takes as arguments a @code{format} control-string
3181and arguments to be passed to @code{format} after this control-string
3182to produce a string.
3183This string is a command that will be evaluated with a POSIX shell if possible;
3184yet, on Windows, some implementations will use CMD.EXE,
3185while others (like SBCL) will make an attempt at invoking a POSIX shell
3186(and fail if it is not present).
3187@end defun
3188
3189
3190@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
3191@comment  node-name,  next,  previous,  up
3192@chapter Getting the latest version
3193
3194Decide which version you want.
3195The @code{master} branch is where development happens;
3196its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
3197but an occasional regression may happen despite our (limited) test suite.
3198
3199The @code{release} branch is what cautious people should be using;
3200it has usually been tested more, and releases are cut at a point
3201where there isn't any known unresolved issue.
3202
3203You may get the ASDF source repository using git:
3204@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
3205
3206You will find the above referenced tags in this repository.
3207You can also browse the repository on
3208@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
3209
3210Discussion of ASDF development is conducted on the
3211mailing list
3212@kbd{asdf-devel@@common-lisp.net}.
3213@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
3214
3215
3216@node FAQ, TODO list, Getting the latest version, Top
3217@comment  node-name,  next,  previous,  up
3218@chapter FAQ
3219
3220@section  ``Where do I report a bug?''
3221
3222ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
3223
3224If you're unsure about whether something is a bug, or for general discussion,
3225use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
3226
3227
3228@section ``What has changed between ASDF 1 and ASDF 2?''
3229
3230@subsection What are ASDF 1 and ASDF 2?
3231
3232On May 31st 2010, we have released ASDF 2.
3233ASDF 2 refers to release 2.000 and later.
3234(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
3235ASDF 1 to any release earlier than 1.369 or so.
3236If your ASDF doesn't sport a version, it's an old ASDF 1.
3237
3238ASDF 2 and its release candidates push
3239@code{:asdf2} onto @code{*features*} so that if you are writing
3240ASDF-dependent code you may check for this feature
3241to see if the new API is present.
3242@emph{All} versions of ASDF should have the @code{:asdf} feature.
3243
3244Additionally, all versions of asdf 2
3245define a function @code{(asdf:asdf-version)} you may use to query the version;
3246and the source code of recent versions of asdf 2 features the version number
3247prominently on the second line of its source code.
3248
3249If you are experiencing problems or limitations of any sort with ASDF 1,
3250we recommend that you should upgrade to ASDF 2,
3251or whatever is the latest release.
3252
3253
3254@subsection ASDF can portably name files in subdirectories
3255
3256Common Lisp namestrings are not portable,
3257except maybe for logical pathnamestrings,
3258that themselves have various limitations and require a lot of setup
3259that is itself ultimately non-portable.
3260
3261In ASDF 1, the only portable ways to refer to pathnames inside systems and components
3262were very awkward, using @code{#.(make-pathname ...)} and
3263@code{#.(merge-pathnames ...)}.
3264Even the above were themselves were inadequate in the general case
3265due to host and device issues, unless horribly complex patterns were used.
3266Plenty of simple cases that looked portable actually weren't,
3267leading to much confusion and greavance.
3268
3269ASDF 2 implements its own portable syntax for strings as pathname specifiers.
3270Naming files within a system definition becomes easy and portable again.
3271@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
3272@code{merge-pathnames*},
3273@code{coerce-pathname}.
3274
3275On the other hand, there are places where systems used to accept namestrings
3276where you must now use an explicit pathname object:
3277@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3278must now be written with the @code{#p} syntax:
3279@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3280
3281@xref{The defsystem grammar,,Pathname specifiers}.
3282
3283
3284@subsection Output translations
3285
3286A popular feature added to ASDF was output pathname translation:
3287@code{asdf-binary-locations}, @code{common-lisp-controller},
3288@code{cl-launch} and other hacks were all implementing it in ways
3289both mutually incompatible and difficult to configure.
3290
3291Output pathname translation is essential to share
3292source directories of portable systems across multiple implementations
3293or variants thereof,
3294or source directories of shared installations of systems across multiple users,
3295or combinations of the above.
3296
3297In ASDF 2, a standard mechanism is provided for that,
3298@code{asdf-output-translations},
3299with sensible defaults, adequate configuration languages,
3300a coherent set of configuration files and hooks,
3301and support for non-Unix platforms.
3302
3303@xref{Controlling where ASDF saves compiled files}.
3304
3305@subsection Source Registry Configuration
3306
3307Configuring ASDF used to require special magic
3308to be applied just at the right moment,
3309between the moment ASDF is loaded and the moment it is used,
3310in a way that is specific to the user,
3311the implementation he is using and the application he is building.
3312
3313This made for awkward configuration files and startup scripts
3314that could not be shared between users, managed by administrators
3315or packaged by distributions.
3316
3317ASDF 2 provides a well-documented way to configure ASDF,
3318with sensible defaults, adequate configuration languages,
3319and a coherent set of configuration files and hooks.
3320
3321We believe it's a vast improvement because it decouples
3322application distribution from library distribution.
3323The application writer can avoid thinking where the libraries are,
3324and the library distributor (dpkg, clbuild, advanced user, etc.)
3325can configure them once and for every application.
3326Yet settings can be easily overridden where needed,
3327so whoever needs control has exactly as much as required.
3328
3329At the same time, ASDF 2 remains compatible
3330with the old magic you may have in your build scripts
3331(using @code{*central-registry*} and
3332@code{*system-definition-search-functions*})
3333to tailor the ASDF configuration to your build automation needs,
3334and also allows for new magic, simpler and more powerful magic.
3335
3336@xref{Controlling where ASDF searches for systems}.
3337
3338
3339@subsection Usual operations are made easier to the user
3340
3341In ASDF 1, you had to use the awkward syntax
3342@code{(asdf:oos 'asdf:load-op :foo)}
3343to load a system,
3344and similarly for @code{compile-op}, @code{test-op}.
3345
3346In ASDF 2, you can use shortcuts for the usual operations:
3347@code{(asdf:load-system :foo)}, and
3348similarly for @code{compile-system}, @code{test-system}.
3349
3350
3351@subsection Many bugs have been fixed
3352
3353The following issues and many others have been fixed:
3354
3355@itemize
3356@item
3357The infamous TRAVERSE function has been revamped significantly,
3358with many bugs squashed.
3359In particular, dependencies were not correctly propagated
3360across submodules within a system but now are.
3361The :version and :feature features and
3362the :force (system1 .. systemN) feature have been fixed.
3363
3364@item
3365Performance has been notably improved for large systems
3366(say with thousands of components) by using
3367hash-tables instead of linear search,
3368and linear-time list accumulation
3369instead of quadratic-time recursive appends.
3370
3371@item
3372Many features used to not be portable,
3373especially where pathnames were involved.
3374Windows support was notably quirky because of such non-portability.
3375
3376@item
3377The internal test suite used to massively fail on many implementations.
3378While still incomplete, it now fully passes
3379on all implementations supported by the test suite,
3380except for GCL (due to GCL bugs).
3381
3382@item
3383Support was lacking for some implementations.
3384ABCL and GCL were notably wholly broken.
3385ECL extensions were not integrated in the ASDF release.
3386
3387@item
3388The documentation was grossly out of date.
3389
3390@end itemize
3391
3392
3393@subsection ASDF itself is versioned
3394
3395Between new features, old bugs fixed, and new bugs introduced,
3396there were various releases of ASDF in the wild,
3397and no simple way to check which release had which feature set.
3398People using or writing systems had to either make worst-case assumptions
3399as to what features were available and worked,
3400or take great pains to have the correct version of ASDF installed.
3401
3402With ASDF 2, we provide a new stable set of working features
3403that everyone can rely on from now on.
3404Use @code{#+asdf2} to detect presence of ASDF 2,
3405@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3406to check the availability of a version no earlier than required.
3407
3408
3409@subsection ASDF can be upgraded
3410
3411When an old version of ASDF was loaded,
3412it was very hard to upgrade ASDF in your current image
3413without breaking everything.
3414Instead you had to exit the Lisp process and
3415somehow arrange to start a new one from a simpler image.
3416Something that can't be done from within Lisp,
3417making automation of it difficult,
3418which compounded with difficulty in configuration,
3419made the task quite hard.
3420Yet as we saw before, the task would have been required
3421to not have to live with the worst case or non-portable
3422subset of ASDF features.
3423
3424With ASDF 2, it is easy to upgrade
3425from ASDF 2 to later versions from within Lisp,
3426and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3427We support hot upgrade of ASDF and any breakage is a bug
3428that we will do our best to fix.
3429There are still limitations on upgrade, though,
3430most notably the fact that after you upgrade ASDF,
3431you must also reload or upgrade all ASDF extensions.
3432
3433@subsection Decoupled release cycle
3434
3435When vendors were releasing their Lisp implementations with ASDF,
3436they had to basically never change version
3437because neither upgrade nor downgrade was possible
3438without breaking something for someone,
3439and no obvious upgrade path was visible and recommendable.
3440
3441With ASDF 2, upgrade is possible, easy and can be recommended.
3442This means that vendors can safely ship a recent version of ASDF,
3443confident that if a user isn't fully satisfied,
3444he can easily upgrade ASDF and deal
3445with a supported recent version of it.
3446This means that release cycles will be causally decoupled,
3447the practical consequence of which will mean faster convergence
3448towards the latest version for everyone.
3449
3450
3451@subsection Pitfalls of the transition to ASDF 2
3452
3453The main pitfalls in upgrading to ASDF 2 seem to be related
3454to the output translation mechanism.
3455
3456@itemize
3457
3458@item
3459Output translations is enabled by default. This may surprise some users,
3460most of them in pleasant way (we hope), a few of them in an unpleasant way.
3461It is trivial to disable output translations.
3462@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3463
3464@item
3465Some systems in the large have been known not to play well with output translations.
3466They were relatively easy to fix.
3467Once again, it is also easy to disable output translations,
3468or to override its configuration.
3469
3470@item
3471The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3472They replace A-B-L, and there is compatibility mode to emulate
3473your previous A-B-L configuration.
3474See @code{enable-asdf-binary-locations-compatibility} in
3475@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3476But thou shalt not load ABL on top of ASDF 2.
3477
3478@end itemize
3479
3480Other issues include the following:
3481
3482@itemize
3483
3484@item
3485ASDF pathname designators are now specified in places where they were unspecified,
3486and a few small adjustments have to be made to some non-portable defsystems.
3487Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
3488a logical pathname (or implementation-dependent hierarchical pathname)
3489must now be specified with @code{#p} syntax
3490where the namestring might have previously sufficed;
3491moreover when evaluation is desired @code{#.} must be used,
3492where it wasn't necessary in the toplevel @code{:pathname} argument
3493(but necessary in other @code{:pathname} arguments).
3494
3495@item
3496There is a slight performance bug, notably on SBCL,
3497when initially searching for @file{asd} files,
3498the implicit @code{(directory "/configured/path/**/*.asd")}
3499for every configured path @code{(:tree "/configured/path/")}
3500in your @code{source-registry} configuration can cause a slight pause.
3501Try to @code{(time (asdf:initialize-source-registry))}
3502to see how bad it is or isn't on your system.
3503If you insist on not having this pause,
3504you can avoid the pause by overriding the default source-registry configuration
3505and not use any deep @code{:tree} entry but only @code{:directory} entries
3506or shallow @code{:tree} entries.
3507Or you can fix your implementation to not be quite that slow
3508when recursing through directories.
3509@emph{Update}: This performance bug fixed the hard way in 2.010.
3510
3511@item
3512On Windows, only LispWorks supports proper default configuration pathnames
3513based on the Windows registry.
3514Other implementations make do with environment variables,
3515that you may have to define yourself if you're using an older version of Windows.
3516Windows support is somewhat less tested than Unix support.
3517Please help report and fix bugs.
3518
3519@item
3520The mechanism by which one customizes a system so that Lisp files
3521may use a different extension from the default @file{.lisp} has changed.
3522Previously, the pathname for a component was lazily computed when operating on a system,
3523and you would
3524@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3525  (declare (ignorable component system)) "lis")}.
3526Now, the pathname for a component is eagerly computed when defining the system,
3527and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3528and use @code{:default-component-class cl-source-file.lis} as argument to @code{defsystem},
3529as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3530
3531@findex source-file-type
3532
3533
3534@end itemize
3535
3536
3537@section Issues with installing the proper version of ASDF
3538
3539@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3540
3541We recommend you upgrade ASDF.
3542@xref{Loading ASDF,,Upgrading ASDF}.
3543
3544If this does not work, it is a bug, and you should report it.
3545@xref{FAQ, report-bugs, Where do I report a bug}.
3546In the meantime, you can load @file{asdf.lisp} directly.
3547@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3548
3549
3550@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3551
3552Starting with current candidate releases of ASDF 2,
3553it should always be a good time to upgrade to a recent ASDF.
3554You may consult with the maintainer for which specific version they recommend,
3555but the latest @code{release} should be correct.
3556We trust you to thoroughly test it with your implementation before you release it.
3557If there are any issues with the current release,
3558it's a bug that you should report upstream and that we will fix ASAP.
3559
3560As to how to include ASDF, we recommend the following:
3561
3562@itemize
3563@item
3564If ASDF isn't loaded yet, then @code{(require "asdf")}
3565should load the version of ASDF that is bundled with your system.
3566If possible so should @code{(require "ASDF")}.
3567You may have it load some other version configured by the user,
3568if you allow such configuration.
3569
3570@item
3571If your system provides a mechanism to hook into @code{CL:REQUIRE},
3572then it would be nice to add ASDF to this hook the same way that
3573ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3574
3575@item
3576You may, like SBCL, have ASDF be implicitly used to require systems
3577that are bundled with your Lisp distribution.
3578If you do have a few magic systems that come with your implementation
3579in a precompiled way such that one should only use the binary version
3580that goes with your distribution, like SBCL does,
3581then you should add them in the beginning of @code{wrapping-source-registry}.
3582
3583@item
3584If you have magic systems as above, like SBCL does,
3585then we explicitly ask you to @emph{NOT} distribute
3586@file{asdf.asd} as part of those magic systems.
3587You should still include the file @file{asdf.lisp} in your source distribution
3588and precompile it in your binary distribution,
3589but @file{asdf.asd} if included at all,
3590should be secluded from the magic systems,
3591in a separate file hierarchy;
3592alternatively, you may provide the system
3593after renaming it and its @file{.asd} file to e.g.
3594@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3595@code{sb-asdf} and @file{sb-asdf.asd}.
3596Indeed, if you made @file{asdf.asd} a magic system,
3597then users would no longer be able to upgrade ASDF using ASDF itself
3598to some version of their preference that
3599they maintain independently from your Lisp distribution.
3600
3601@item
3602If you do not have any such magic systems, or have other non-magic systems
3603that you want to bundle with your implementation,
3604then you may add them to the @code{default-source-registry},
3605and you are welcome to include @file{asdf.asd} amongst them.
3606
3607@item
3608Please send us upstream any patches you make to ASDF itself,
3609so we can merge them back in for the benefit of your users
3610when they upgrade to the upstream version.
3611
3612@end itemize
3613
3614
3615
3616@section Issues with configuring ASDF
3617
3618@subsection ``How can I customize where fasl files are stored?''
3619
3620@xref{Controlling where ASDF saves compiled files}.
3621
3622Note that in the past there was an add-on to ASDF called
3623@code{ASDF-binary-locations}, developed by Gary King.
3624That add-on has been merged into ASDF proper,
3625then superseded by the @code{asdf-output-translations} facility.
3626
3627Note that use of @code{asdf-output-translations}
3628can interfere with one aspect of your systems
3629--- if your system uses @code{*load-truename*} to find files
3630(e.g., if you have some data files stored with your program),
3631then the relocation that this ASDF customization performs
3632is likely to interfere.
3633Use @code{asdf:system-relative-pathname} to locate a file
3634in the source directory of some system, and
3635use @code{asdf:apply-output-translations} to locate a file
3636whose pathname has been translated by the facility.
3637
3638@subsection ``How can I wholly disable the compiler output cache?''
3639
3640To permanently disable the compiler output cache
3641for all future runs of ASDF, you can:
3642
3643@example
3644mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3645echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3646@end example
3647
3648This assumes that you didn't otherwise configure the ASDF files
3649(if you did, edit them again),
3650and don't somehow override the configuration at runtime
3651with a shell variable (see below) or some other runtime command
3652(e.g. some call to @code{asdf:initialize-output-translations}).
3653
3654To disable the compiler output cache in Lisp processes
3655run by your current shell, try (assuming @code{bash} or @code{zsh})
3656(on Unix and cygwin only):
3657
3658@example
3659export ASDF_OUTPUT_TRANSLATIONS=/:
3660@end example
3661
3662To disable the compiler output cache just in the current Lisp process,
3663use (after loading ASDF but before using it):
3664
3665@example
3666(asdf:disable-output-translations)
3667@end example
3668
3669@section Issues with using and extending ASDF to define systems
3670
3671@subsection ``How can I cater for unit-testing in my system?''
3672
3673ASDF provides a predefined test operation, @code{test-op}.
3674@xref{Predefined operations of ASDF, test-op}.
3675The test operation, however, is largely left to the system definer to specify.
3676@code{test-op} has been
3677a topic of considerable discussion on the
3678@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3679and on the
3680@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3681
3682Here are some guidelines:
3683
3684@itemize
3685@item
3686For a given system, @var{foo}, you will want to define a corresponding
3687test system, such as @var{foo-test}.  The reason that you will want this
3688separate system is that ASDF does not out of the box supply components
3689that are conditionally loaded.  So if you want to have source files
3690(with the test definitions) that will not be loaded except when testing,
3691they should be put elsewhere.
3692
3693@item
3694The @var{foo-test} system can be defined in an asd file of its own or
3695together with @var{foo}.  An aesthetic preference against cluttering up
3696the filesystem with extra asd files should be balanced against the
3697question of whether one might want to directly load @var{foo-test}.
3698Typically one would not want to do this except in early stages of
3699debugging.
3700
3701@item
3702Record that testing is implemented by @var{foo-test}.  For example:
3703@example
3704(defsystem @var{foo}
3705   :in-order-to ((test-op (test-op @var{foo-test})))
3706   ....)
3707
3708(defsystem @var{foo-test}
3709   :depends-on (@var{foo} @var{my-test-library} ...)
3710   ....)
3711@end example
3712@end itemize
3713
3714This procedure will allow you to support users who do not wish to
3715install your test framework.
3716
3717One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3718does not return a value.  So in current versions of ASDF there is no
3719reliable programmatic means of determining whether or not a set of tests
3720has passed, or which tests have failed.  The user must simply read the
3721console output.  This limitation has been the subject of much
3722discussion.
3723
3724@subsection ``How can I cater for documentation generation in my system?''
3725
3726The ASDF developers are currently working to add a @code{doc-op}
3727to the set of predefined ASDF operations.
3728@xref{Predefined operations of ASDF}.
3729See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3730
3731
3732
3733@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3734
3735See @code{cffi}'s @code{cffi-grovel}.
3736
3737@anchor{report-bugs}
3738
3739
3740@subsection ``I want to put my module's files at the top level.  How do I do this?''
3741
3742By default, the files contained in an asdf module go
3743in a subdirectory with the same name as the module.
3744However, this can be overridden by adding a @code{:pathname ""} argument
3745to the module description.
3746For example, here is how it could be done
3747in the spatial-trees ASDF system definition for ASDF 2:
3748
3749@example
3750(asdf:defsystem :spatial-trees
3751  :components
3752  ((:module base
3753            :pathname ""
3754            :components
3755            ((:file "package")
3756             (:file "basedefs" :depends-on ("package"))
3757             (:file "rectangles" :depends-on ("package"))))
3758   (:module tree-impls
3759            :depends-on (base)
3760            :pathname ""
3761            :components
3762            ((:file "r-trees")
3763             (:file "greene-trees" :depends-on ("r-trees"))
3764             (:file "rstar-trees" :depends-on ("r-trees"))
3765             (:file "rplus-trees" :depends-on ("r-trees"))
3766             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3767   (:module viz
3768            :depends-on (base)
3769            :pathname ""
3770            :components
3771            ((:static-file "spatial-tree-viz.lisp")))
3772   (:module tests
3773            :depends-on (base)
3774            :pathname ""
3775            :components
3776            ((:static-file "spatial-tree-test.lisp")))
3777   (:static-file "LICENCE")
3778   (:static-file "TODO")))
3779@end example
3780
3781All of the files in the @code{tree-impls} module are at the top level,
3782instead of in a @file{tree-impls/} subdirectory.
3783
3784Note that the argument to @code{:pathname} can be either a pathname object or a string.
3785A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3786but this is discouraged because the results of parsing a namestring are not portable.
3787A pathname can only be portably constructed with such syntax as
3788@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3789and similarly the current directory can only be portably specified as
3790@code{#.(make-pathname :directory '(:relative))}.
3791However, as of ASDF 2, you can portably use a string to denote a pathname.
3792The string will be parsed as a @code{/}-separated path from the current directory,
3793such that the empty string @code{""} denotes the current directory, and
3794@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3795portably denotes the same subdirectory as above.
3796When files are specified, the last @code{/}-separated component is interpreted
3797either as the name component of a pathname
3798(if the component class specifies a pathname type),
3799or as a name component plus optional dot-separated type component
3800(if the component class doesn't specifies a pathname type).
3801
3802@subsection How do I create a system definition where all the source files have a .cl extension?
3803
3804Starting with ASDF 2.014.14, you may just pass
3805the builtin class @code{cl-source-file.cl} as
3806the @code{:default-component-class} argument to @code{defsystem}:
3807
3808@lisp
3809(defsystem my-cl-system
3810  :default-component-class cl-source-file.cl
3811  ...)
3812@end lisp
3813
3814Another builtin class @code{cl-source-file.lsp} is offered
3815for files ending in @file{.lsp}.
3816
3817If you want to use a different extension
3818for which ASDF doesn't provide builtin support,
3819or want to support versions of ASDF
3820earlier than 2.014.14 (but later than 2.000),
3821you can define a class as follows:
3822
3823@lisp
3824;; Prologue: make sure we're using a sane package.
3825(defpackage :my-asdf-extension
3826   (:use :asdf :common-lisp)
3827   (:export #:cl-source-file.lis))
3828(in-package :my-asdf-extension)
3829
3830(defclass cl-source-file.lis (cl-source-file)
3831   ((type :initform "lis")))
3832@end lisp
3833
3834Then you can use it as follows:
3835@lisp
3836(defsystem my-cl-system
3837  :default-component-class my-asdf-extension:cl-source-file.lis
3838  ...)
3839@end lisp
3840
3841Of course, if you're in the same package, e.g. in the same file,
3842you won't need to use the package qualifier before @code{cl-source-file.lis}.
3843Actually, if all you're doing is defining this class
3844and using it in the same file without other fancy definitions,
3845you might skip package complications:
3846
3847@lisp
3848(in-package :asdf)
3849(defclass cl-source-file.lis (cl-source-file)
3850   ((type :initform "lis")))
3851(defsystem my-cl-system
3852  :default-component-class cl-source-file.lis
3853  ...)
3854@end lisp
3855
3856It is possible to achieve the same effect
3857in a way that supports both ASDF 1 and ASDF 2,
3858but really, friends don't let friends use ASDF 1.
3859Please upgrade to ASDF 2.
3860In short, though: do same as above, but
3861@emph{before} you use the class in a @code{defsystem},
3862you also define the following method:
3863
3864@lisp
3865(defmethod source-file-type ((f cl-source-file.lis) (m module))
3866  (declare (ignorable f m))
3867  "lis")
3868@end lisp
3869
3870@comment FIXME: Add a FAQ about how to use a new system class...
3871
3872
3873@node  TODO list, Inspiration, FAQ, Top
3874@comment  node-name,  next,  previous,  up
3875@chapter TODO list
3876
3877Here is an old list of things to do,
3878in addition to the bugs that are now tracked on launchpad:
3879@url{https://launchpad.net/asdf}.
3880
3881@section Outstanding spec questions, things to add
3882
3883** packaging systems
3884
3885*** manual page component?
3886
3887** style guide for .asd files
3888
3889You should either use keywords or be careful
3890with the package that you evaluate defsystem forms in.
3891Otherwise @code{(defsystem partition ...)}
3892being read in the @code{cl-user} package
3893will intern a @code{cl-user:partition} symbol,
3894which will then collide with the @code{partition:partition} symbol.
3895
3896Actually there's a hairier packages problem to think about too.
3897@code{in-order-to} is not a keyword:
3898if you read @code{defsystem} forms in a package that doesn't use ASDF,
3899odd things might happen.
3900
3901
3902** extending defsystem with new options
3903
3904You might not want to write a whole parser,
3905but just to add options to the existing syntax.
3906Reinstate @code{parse-option} or something akin.
3907
3908
3909** document all the error classes
3910
3911** what to do with compile-file failure
3912
3913Should check the primary return value from compile-file and see if
3914that gets us any closer to a sensible error handling strategy
3915
3916** foreign files
3917
3918lift unix-dso stuff from db-sockets
3919
3920** Diagnostics
3921
3922A ``dry run'' of an operation can be made with the following form:
3923
3924@lisp
3925(traverse (make-instance '<operation-name>)
3926          (find-system <system-name>)
3927          'explain)
3928@end lisp
3929
3930This uses unexported symbols.
3931What would be a nice interface for this functionality?
3932
3933@section Missing bits in implementation
3934
3935** reuse the same scratch package whenever a system is reloaded from disk
3936
3937** proclamations probably aren't
3938
3939** when a system is reloaded with fewer components than it previously had, odd things happen
3940
3941We should do something inventive when processing a @code{defsystem} form,
3942like take the list of kids and @code{setf} the slot to @code{nil},
3943then transfer children from old to new list as they're found.
3944
3945** (stuff that might happen later)
3946
3947*** Propagation of the @code{:force} option.
3948
3949``I notice that
3950
3951        @code{(asdf:compile-system :araneida :force t)}
3952
3953also forces compilation of every other system the @code{:araneida} system depends on.
3954This is rarely useful to me;
3955usually, when I want to force recompilation of something more than a single source file,
3956I want to recompile only one system.
3957So it would be more useful to have @code{make-sub-operation}
3958refuse to propagate @code{:force t} to other systems, and
3959propagate only something like @code{:force :recursively}.
3960
3961Ideally what we actually want is some kind of criterion that says
3962to which systems (and which operations) a @code{:force} switch will propagate.
3963
3964The problem is perhaps that ``force'' is a pretty meaningless concept.
3965How obvious is it that @code{load :force t} should force @emph{compilation}?
3966But we don't really have the right dependency setup
3967for the user to compile @code{:force t} and expect it to work
3968(files will not be loaded after compilation, so the compile
3969environment for subsequent files will be emptier than it needs to be)
3970
3971What does the user actually want to do when he forces?
3972Usually, for me, update for use with a new version of the Lisp compiler.
3973Perhaps for recovery when he suspects that something has gone wrong.
3974Or else when he's changed compilation options or configuration
3975in some way that's not reflected in the dependency graph.
3976
3977Other possible interface: have a ``revert'' function akin to @code{make clean}.
3978
3979@lisp
3980(asdf:revert 'asdf:compile-op 'araneida)
3981@end lisp
3982
3983would delete any files produced by @code{(compile-system :araneida)}.
3984Of course, it wouldn't be able to do much about stuff in the image itself.
3985
3986How would this work?
3987
3988@code{traverse}
3989
3990There's a difference between a module's dependencies (peers)
3991and its components (children).
3992Perhaps there's a similar difference in operations?
3993For example, @code{(load "use") depends-on (load "macros")} is a peer,
3994whereas @code{(load "use") depends-on (compile "use")}
3995is more of a ``subservient'' relationship.
3996
3997@node  Inspiration, Concept Index, TODO list, Top
3998@comment  node-name,  next,  previous,  up
3999@chapter Inspiration
4000
4001@section mk-defsystem (defsystem-3.x)
4002
4003We aim to solve basically the same problems as @code{mk-defsystem} does.
4004However, our architecture for extensibility
4005better exploits CL language features (and is documented),
4006and we intend to be portable rather than just widely-ported.
4007No slight on the @code{mk-defsystem} authors and maintainers is intended here;
4008that implementation has the unenviable task
4009of supporting pre-ANSI implementations, which is no longer necessary.
4010
4011The surface defsystem syntax of asdf is more-or-less compatible with
4012@code{mk-defsystem}, except that we do not support
4013the @code{source-foo} and @code{binary-foo} prefixes
4014for separating source and binary files, and
4015we advise the removal of all options to specify pathnames.
4016
4017The @code{mk-defsystem} code for topologically sorting
4018a module's dependency list was very useful.
4019
4020@section defsystem-4 proposal
4021
4022Marco and Peter's proposal for defsystem 4 served as the driver for
4023many of the features in here.  Notable differences are:
4024
4025@itemize
4026@item
4027We don't specify output files or output file extensions
4028as part of the system.
4029
4030If you want to find out what files an operation would create,
4031ask the operation.
4032
4033@item
4034We don't deal with CL packages
4035
4036If you want to compile in a particular package, use an @code{in-package} form
4037in that file (ilisp / SLIME will like you more if you do this anyway)
4038
4039@item
4040There is no proposal here that @code{defsystem} does version control.
4041
4042A system has a given version which can be used to check dependencies,
4043but that's all.
4044@end itemize
4045
4046The defsystem 4 proposal tends to look more at the external features,
4047whereas this one centres on a protocol for system introspection.
4048
4049@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
4050
4051Available in updated-for-CL form on the web at
4052@url{http://nhplace.com/kent/Papers/Large-Systems.html}
4053
4054In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
4055and concept to deal with creating component trees
4056from @code{defsystem} surface syntax.
4057[ this is not true right now, though it used to be and
4058probably will be again soon ]
4059
4060
4061@c -------------------
4062
4063
4064@node Concept Index, Function and Class Index, Inspiration, Top
4065@unnumbered Concept Index
4066
4067@printindex cp
4068
4069@node Function and Class Index, Variable Index, Concept Index, Top
4070@unnumbered Function and Class Index
4071
4072@printindex fn
4073
4074@node Variable Index,  , Function and Class Index, Top
4075@unnumbered Variable Index
4076
4077@printindex vr
4078
4079@bye
Note: See TracBrowser for help on using the repository browser.