source: branches/typed-asm/abcl/doc/asdf/asdf.texinfo

Last change on this file was 14397, checked in by Mark Evenson, 12 years ago

Upgrade to asdf-2.30.

File size: 168.6 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24@macro &curly
25{
26@end macro
27@macro &ylruc
28}
29@end macro
30
31@c for install-info
32@dircategory Software development
33@direntry
34* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
35@end direntry
36
37@copying
38This manual describes ASDF, a system definition facility
39for Common Lisp programs and libraries.
40
41You can find the latest version of this manual at
42@url{http://common-lisp.net/project/asdf/asdf.html}.
43
44ASDF Copyright @copyright{} 2001-2013 Daniel Barlow and contributors.
45
46This manual Copyright @copyright{} 2001-2013 Daniel Barlow and contributors.
47
48This manual revised @copyright{} 2009-2013 Robert P. Goldman and Francois-Rene Rideau.
49
50Permission is hereby granted, free of charge, to any person obtaining
51a copy of this software and associated documentation files (the
52``Software''), to deal in the Software without restriction, including
53without limitation the rights to use, copy, modify, merge, publish,
54distribute, sublicense, and/or sell copies of the Software, and to
55permit persons to whom the Software is furnished to do so, subject to
56the following conditions:
57
58The above copyright notice and this permission notice shall be
59included in all copies or substantial portions of the Software.
60
61THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
62EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
63MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
64NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
65LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
66OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
67WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
68
69@end copying
70
71
72
73@titlepage
74@title ASDF: Another System Definition Facility
75
76@c The following two commands start the copyright page.
77@page
78@vskip 0pt plus 1filll
79@insertcopying
80@end titlepage
81
82@c Output the table of contents at the beginning.
83@contents
84
85@c -------------------
86
87@ifnottex
88
89@node Top, Introduction, (dir), (dir)
90@top asdf: another system definition facility
91
92@insertcopying
93
94@menu
95* Introduction::
96* Loading ASDF::
97* Configuring ASDF::
98* Using ASDF::
99* Defining systems with defsystem::
100* The object model of ASDF::
101* Controlling where ASDF searches for systems::
102* Controlling where ASDF saves compiled files::
103* Error handling::
104* Miscellaneous additional functionality::
105* Getting the latest version::
106* FAQ::
107* TODO list::
108* Inspiration::
109* Concept Index::
110* Function and Class Index::
111* Variable Index::
112
113@c @detailmenu
114@c  --- The Detailed Node Listing ---
115
116@c Defining systems with defsystem
117
118@c * The defsystem form::
119@c * A more involved example::
120@c * The defsystem grammar::
121@c * Other code in .asd files::
122
123@c The object model of ASDF
124
125@c * Operations::
126@c * Components::
127@c * Functions::
128
129@c Operations
130
131@c * Predefined operations of ASDF::
132@c * Creating new operations::
133
134@c Components
135
136@c * Common attributes of components::
137@c * Pre-defined subclasses of component::
138@c * Creating new component types::
139
140@c properties
141
142@c * Pre-defined subclasses of component::
143@c * Creating new component types::
144
145@c @end detailmenu
146@end menu
147
148@end ifnottex
149
150@c -------------------
151
152@node Introduction, Loading ASDF, Top, Top
153@comment  node-name,  next,  previous,  up
154@chapter Introduction
155@cindex ASDF-related features
156@vindex *features*
157@cindex Testing for ASDF
158@cindex ASDF versions
159@cindex :asdf
160@cindex :asdf2
161@cindex :asdf3
162
163ASDF is Another System Definition Facility:
164a tool for specifying how systems of Common Lisp software
165are comprised of components (sub-systems and files),
166and how to operate on these components in the right order
167so that they can be compiled, loaded, tested, etc.
168
169ASDF presents three faces:
170one for users of Common Lisp software who want to reuse other people's code,
171one for writers of Common Lisp software who want to specify how to build their systems,
172one for implementers of Common Lisp extensions who want to extend the build system.
173@xref{Using ASDF,,Loading a system},
174to learn how to use ASDF to load a system.
175@xref{Defining systems with defsystem},
176to learn how to define a system of your own.
177@xref{The object model of ASDF}, for a description of
178the ASDF internals and how to extend ASDF.
179
180@emph{Nota Bene}:
181We have released ASDF 2.000 on May 31st 2010,
182and ASDF 3.0 on January 31st 2013.
183Releases of ASDF 2 and later have since then been included
184in all actively maintained CL implementations that used to bundle ASDF 1,
185plus some implementations that didn't use to,
186and has been made to work with all actively used CL implementations and a few more.
187@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
188Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly.
189For this reason, we have stopped supporting ASDF 1 and ASDF 2.
190If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
191we recommend you upgrade to ASDF 3
192--- and we explain how to do that. @xref{Loading ASDF}.
193
194Also note that ASDF is not to be confused with ASDF-Install.
195ASDF-Install is not part of ASDF, but a separate piece of software.
196ASDF-Install is also unmaintained and obsolete.
197We recommend you use Quicklisp instead,
198which works great and is being actively maintained.
199If you want to download software from version control instead of tarballs,
200so you may more easily modify it, we recommend clbuild.
201
202
203@node Loading ASDF, Configuring ASDF, Introduction, Top
204@comment  node-name,  next,  previous,  up
205@chapter Loading ASDF
206@vindex *central-registry*
207@cindex link farm
208@findex load-system
209@findex require-system
210@findex compile-system
211@findex test-system
212@cindex system directory designator
213@findex operate
214@findex oos
215
216@c @menu
217@c * Installing ASDF::
218@c @end menu
219
220
221@section Loading a pre-installed ASDF
222
223Most recent Lisp implementations include a copy of ASDF 2, and soon ASDF 3.
224You can usually load this copy using Common Lisp's @code{require} function:
225
226@lisp
227(require "asdf")
228@end lisp
229
230As of the writing of this manual,
231the following implementations provide ASDF 2 this way:
232abcl allegro ccl clisp cmucl ecl lispworks mkcl sbcl xcl.
233The following implementation doesn't provide it yet but will in an upcoming release:
234scl.
235The following implementations are obsolete, not actively maintained,
236and most probably will never bundle it:
237cormanlisp gcl genera mcl.
238
239If the implementation you are using doesn't provide ASDF 2 or ASDF 3,
240see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
241If that implementation is still actively maintained,
242you may also send a bug report to your Lisp vendor and complain
243about their failing to provide ASDF.
244
245NB: all implementations except clisp also accept
246@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
247For portability's sake, you probably want to use @code{(require "asdf")}.
248
249
250@section Checking whether ASDF is loaded
251
252To check whether ASDF is properly loaded in your current Lisp image,
253you can run this form:
254
255@lisp
256(asdf:asdf-version)
257@end lisp
258
259If it returns a string,
260that is the version of ASDF that is currently installed.
261
262If it raises an error,
263then either ASDF is not loaded, or
264you are using an old version of ASDF.
265
266You can check whether an old version is loaded
267by checking if the ASDF package is present.
268The form below will allow you to programmatically determine
269whether a recent version is loaded, an old version is loaded,
270or none at all:
271
272@lisp
273(when (find-package :asdf)
274  (let ((ver (symbol-value (or (find-symbol (string :*asdf-version*) :asdf)
275                               (find-symbol (string :*asdf-revision*) :asdf)))))
276    (etypecase ver
277      (string ver)
278      (cons (with-output-to-string (s)
279              (loop for (n . m) on ver do (princ n s) (when m (princ "." s)))))
280      (null "1.0"))))
281@end lisp
282
283If it returns @code{NIL} then ASDF is not installed.
284Otherwise it should return a string.
285If it returns @code{"1.0"}, then it can actually be
286any version before 1.77 or so, or some buggy variant of 1.x.
287
288If you are experiencing problems with ASDF,
289please try upgrading to the latest released version,
290using the method below,
291before you contact us and raise an issue.
292
293
294@section Upgrading ASDF
295
296If your implementation provides ASDF 3 or later,
297you only need to @code{(require "asdf")}:
298ASDF will automatically look whether an updated version of itself is available
299amongst the regularly configured systems, before it compiles anything else.
300See @pxref{Configuring ASDF} below.
301
302If your implementation does provide ASDF 2 or later,
303but not ASDF 3 or later,
304and you want to upgrade to a more recent version,
305you need to install and configure your ASDF as above,
306and additionally, you need to explicitly tell ASDF to load itself,
307right after you require your implementation's old ASDF 2:
308
309@lisp
310(require "asdf")
311(asdf:load-system :asdf)
312@end lisp
313
314If on the other hand, your implementation only provides an old ASDF,
315you will require a special configuration step and an old-style loading.
316Take special attention to not omit the trailing directory separator
317@code{/} at the end of your pathname:
318
319@lisp
320(require "asdf")
321(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
322(asdf:oos 'asdf:load-op :asdf)
323@end lisp
324
325Note that ASDF 1 won't redirect its output files,
326or at least won't do it according to your usual ASDF 2 configuration.
327You therefore need write access on the directory
328where you install the new ASDF,
329and make sure you're not using it
330for multiple mutually incompatible implementations.
331At worst, you may have to have multiple copies of the new ASDF,
332e.g. one per implementation installation, to avoid clashes.
333Note that to our knowledge all implementations that provide ASDF
334provide ASDF 2 in their latest release, so
335you may want to upgrade your implementation rather than go through that hoop.
336
337Finally, if you are using an unmaintained implementation
338that does not provide ASDF at all,
339see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
340
341Note that there are some limitations to upgrading ASDF:
342@itemize
343@item
344Previously loaded ASDF extension becomes invalid, and will need to be reloaded.
345This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
346Since it isn't possible to automatically detect what extensions are present
347that need to be invalidated,
348ASDF will actually invalidate all previously loaded systems
349when it is loaded on top of a different ASDF version,
350starting with ASDF 2.014.8 (as far as releases go, 2.015);
351and it will automatically attempt this self-upgrade as its very first step
352starting with ASDF 3.
353
354@item
355For this an many other reasons,
356it important reason to load, configure and upgrade ASDF (if needed)
357as one of the very first things done by your build and startup scripts.
358Until all implementations provide ASDF 3 or later,
359it is safer if you upgrade ASDF and its extensions as a special step
360at the very beginning of whatever script you are running,
361before you start using ASDF to load anything else;
362even afterwards, it is still a good idea, to avoid having to
363load and reload code twice as it gets invalidated.
364
365@item
366Until all implementations provide ASDF 3 or later,
367it is unsafe to upgrade ASDF as part of loading a system
368that depends on a more recent version of ASDF,
369since the new one might shadow the old one while the old one is running,
370and the running old one will be confused
371when extensions are loaded into the new one.
372In the meantime, we recommend that your systems should @emph{not} specify
373@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
374but instead that they check that a recent enough ASDF is installed,
375with such code as:
376@example
377(unless (or #+asdf2 (asdf:version-satisfies
378                     (asdf:asdf-version) *required-asdf-version*))
379  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
380@end example
381@item
382Until all implementations provide ASDF 3 or later,
383it is unsafe for a system to transitively depend on ASDF
384and not directly depend on ASDF;
385if any of the system you use either depends-on asdf,
386system-depends-on asdf, or transitively does,
387you should also do as well.
388@end itemize
389
390
391@section Loading an otherwise installed ASDF
392
393If your implementation doesn't include ASDF,
394if for some reason the upgrade somehow fails,
395does not or cannot apply to your case,
396you will have to install the file @file{asdf.lisp}
397somewhere and load it with:
398
399@lisp
400(load "/path/to/your/installed/asdf.lisp")
401@end lisp
402
403The single file @file{asdf.lisp} is all you normally need to use ASDF.
404
405You can extract this file from latest release tarball on the
406@url{http://common-lisp.net/project/asdf/,ASDF website}.
407If you are daring and willing to report bugs, you can get
408the latest and greatest version of ASDF from its git repository.
409@xref{Getting the latest version}.
410
411For maximum convenience you might want to have ASDF loaded
412whenever you start your Lisp implementation,
413for example by loading it from the startup script or dumping a custom core
414--- check your Lisp implementation's manual for details.
415
416
417@node Configuring ASDF, Using ASDF, Loading ASDF, Top
418@comment  node-name,  next,  previous,  up
419
420@chapter Configuring ASDF
421
422@section Configuring ASDF to find your systems
423
424So it may compile and load your systems, ASDF must be configured to find
425the @file{.asd} files that contain system definitions.
426
427Since ASDF 2, the preferred way to configure where ASDF finds your systems is
428the @code{source-registry} facility,
429fully described in its own chapter of this manual.
430@xref{Controlling where ASDF searches for systems}.
431
432The default location for a user to install Common Lisp software is under
433@file{~/.local/share/common-lisp/source/}.
434If you install software there (it can be a symlink),
435you don't need further configuration.
436If you're installing software yourself at a location that isn't standard,
437you have to tell ASDF where you installed it. See below.
438If you're using some tool to install software (e.g. Quicklisp),
439the authors of that tool should already have configured ASDF.
440
441The simplest way to add a path to your search path,
442say @file{/home/luser/.asd-link-farm/}
443is to create the directory
444@file{~/.config/common-lisp/source-registry.conf.d/}
445and there create a file with any name of your choice,
446and with the type @file{conf},
447for instance @file{42-asd-link-farm.conf}
448containing the line:
449
450@kbd{(:directory "/home/luser/.asd-link-farm/")}
451
452If you want all the subdirectories under @file{/home/luser/lisp/}
453to be recursively scanned for @file{.asd} files, instead use:
454
455@kbd{(:tree "/home/luser/lisp/")}
456
457Note that your Operating System distribution or your system administrator
458may already have configured system-managed libraries for you.
459
460The required @file{.conf} extension allows you to have disabled files
461or editor backups (ending in @file{~}), and works portably
462(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
463Excluded are files the name of which start with a @file{.} character.
464It is customary to start the filename with two digits
465that specify the order in which the directories will be scanned.
466
467ASDF will automatically read your configuration
468the first time you try to find a system.
469You can reset the source-registry configuration with:
470
471@lisp
472(asdf:clear-source-registry)
473@end lisp
474
475And you probably should do so before you dump your Lisp image,
476if the configuration may change
477between the machine where you save it at the time you save it
478and the machine you resume it at the time you resume it.
479Actually, you should use @code{(asdf:clear-configuration)}
480before you dump your Lisp image, which includes the above.
481
482
483@section Configuring ASDF to find your systems --- old style
484
485The old way to configure ASDF to find your systems is by
486@code{push}ing directory pathnames onto the variable
487@code{asdf:*central-registry*}.
488
489You must configure this variable between the time you load ASDF
490and the time you first try to use it.
491Loading and configuring ASDF presumably happen
492as part of some initialization script that builds or starts
493your Common Lisp software system.
494(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
495
496The @code{asdf:*central-registry*} is empty by default in ASDF 2 or ASDF 3,
497but is still supported for compatibility with ASDF 1.
498When used, it takes precedence over the above source-registry@footnote{
499It is possible to further customize
500the system definition file search.
501That's considered advanced use, and covered later:
502search forward for
503@code{*system-definition-search-functions*}.
504@xref{Defining systems with defsystem}.}.
505
506For instance, if you wanted ASDF to find the @file{.asd} file
507@file{/home/me/src/foo/foo.asd} your initialization script
508could after it loads ASDF with @code{(require "asdf")}
509configure it with:
510
511@lisp
512(push "/home/me/src/foo/" asdf:*central-registry*)
513@end lisp
514
515Note the trailing slash: when searching for a system,
516ASDF will evaluate each entry of the central registry
517and coerce the result to a pathname@footnote{
518ASDF will indeed call @code{EVAL} on each entry.
519It will also skip entries that evaluate to @code{NIL}.
520
521Strings and pathname objects are self-evaluating,
522in which case the @code{EVAL} step does nothing;
523but you may push arbitrary SEXP onto the central registry,
524that will be evaluated to compute e.g. things that depend
525on the value of shell variables or the identity of the user.
526
527The variable @code{asdf:*central-registry*} is thus a list of
528``system directory designators''.
529A @dfn{system directory designator} is a form
530which will be evaluated whenever a system is to be found,
531and must evaluate to a directory to look in.
532By ``directory'' here, we mean
533``designator for a pathname with a supplied DIRECTORY component''.
534}
535at which point the presence of the trailing directory name separator
536is necessary to tell Lisp that you're discussing a directory
537rather than a file.
538
539Typically, however, there are a lot of @file{.asd} files, and
540a common idiom was to have to put
541a bunch of @emph{symbolic links} to @file{.asd} files
542in a common directory
543and push @emph{that} directory (the ``link farm'')
544to the
545@code{asdf:*central-registry*}
546instead of pushing each of the many involved directories
547to the @code{asdf:*central-registry*}.
548ASDF knows how to follow such @emph{symlinks}
549to the actual file location when resolving the paths of system components
550(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
551if you try aliases under MacOS, we are curious to hear about your experience).
552
553For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
554is a member of @code{*central-registry*}, you could set up the
555system @var{foo} for loading with asdf with the following
556commands at the shell:
557
558@example
559$ cd /home/me/cl/systems/
560$ ln -s ~/src/foo/foo.asd .
561@end example
562
563This old style for configuring ASDF is not recommended for new users,
564but it is supported for old users, and for users who want to programmatically
565control what directories are added to the ASDF search path.
566
567
568@section Configuring where ASDF stores object files
569@findex clear-output-translations
570
571ASDF lets you configure where object files will be stored.
572Sensible defaults are provided and
573you shouldn't normally have to worry about it.
574
575This allows the same source code repository may be shared
576between several versions of several Common Lisp implementations,
577between several users using different compilation options
578and without write privileges on shared source directories, etc.
579This also allows to keep source directories uncluttered
580by plenty of object files.
581
582Starting with ASDF 2, the @code{asdf-output-translations} facility
583was added to ASDF itself, that controls where object files will be stored.
584This facility is fully described in a chapter of this manual,
585@ref{Controlling where ASDF saves compiled files}.
586
587The simplest way to add a translation to your search path,
588say from @file{/foo/bar/baz/quux/}
589to @file{/where/i/want/my/fasls/}
590is to create the directory
591@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
592and there create a file with any name of your choice and the type @file{conf},
593for instance @file{42-bazquux.conf}
594containing the line:
595
596@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
597
598To disable output translations for source under a given directory,
599say @file{/toto/tata/}
600you can create a file @file{40-disable-toto.conf}
601with the line:
602
603@kbd{("/toto/tata/")}
604
605To wholly disable output translations for all directories,
606you can create a file @file{00-disable.conf}
607with the line:
608
609@kbd{(t t)}
610
611Note that your Operating System distribution or your system administrator
612may already have configured translations for you.
613In absence of any configuration, the default is to redirect everything
614under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
615@xref{Controlling where ASDF searches for systems}, for full details.
616
617The required @file{.conf} extension allows you to have disabled files
618or editor backups (ending in @file{~}), and works portably
619(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
620Excluded are files the name of which start with a @file{.} character.
621It is customary to start the filename with two digits
622that specify the order in which the directories will be scanned.
623
624ASDF will automatically read your configuration
625the first time you try to find a system.
626You can reset the source-registry configuration with:
627
628@lisp
629(asdf:clear-output-translations)
630@end lisp
631
632And you probably should do so before you dump your Lisp image,
633if the configuration may change
634between the machine where you save it at the time you save it
635and the machine you resume it at the time you resume it.
636(Once again, you should use @code{(asdf:clear-configuration)}
637before you dump your Lisp image, which includes the above.)
638
639Finally note that before ASDF 2,
640other ASDF add-ons offered the same functionality,
641each in subtly different and incompatible ways:
642ASDF-Binary-Locations, cl-launch, common-lisp-controller.
643ASDF-Binary-Locations is now not needed anymore and should not be used.
644cl-launch 3.000 and common-lisp-controller 7.2 have been updated
645to just delegate this functionality to ASDF.
646
647@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
648@comment  node-name,  next,  previous,  up
649
650
651@section Resetting Configuration
652
653When you dump and restore an image, or when you tweak your configuration,
654you may want to reset the ASDF configuration.
655For that you may use the following function:
656
657@defun clear-configuration
658   undoes any ASDF configuration,
659   regarding source-registry or output-translations.
660@end defun
661
662If you use SBCL, CMUCL or SCL, you may use this snippet
663so that the ASDF configuration be cleared automatically as you dump an image:
664
665@example
666#+(or cmu sbcl scl)
667(pushnew 'clear-configuration
668         #+(or cmu scl) ext:*before-save-initializations*
669         #+sbcl sb-ext:*save-hooks*)
670@end example
671
672For compatibility with all Lisp implementations, however,
673you might want instead your build script to explicitly call
674@code{(asdf:clear-configuration)} at an appropriate moment before dumping.
675
676
677@chapter Using ASDF
678
679@section Loading a system
680
681The system @var{foo} is loaded (and compiled, if necessary)
682by evaluating the following Lisp form:
683
684@example
685(asdf:load-system :@var{foo})
686@end example
687
688On some implementations (namely recent versions of
689ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP,
690LispWorks, MKCL, SBCL and XCL),
691ASDF hooks into the @code{CL:REQUIRE} facility
692and you can just use:
693
694@example
695(require :@var{foo})
696@end example
697
698In older versions of ASDF, you needed to use
699@code{(asdf:oos 'asdf:load-op :@var{foo})}.
700If your ASDF is too old to provide @code{asdf:load-system} though
701we recommend that you upgrade to ASDF 3.
702@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
703
704Note the name of a system is specified as a string or a symbol,
705typically a keyword.
706If a symbol (including a keyword), its name is taken and lowercased.
707The name must be a suitable value for the @code{:name} initarg
708to @code{make-pathname} in whatever filesystem the system is to be found.
709The lower-casing-symbols behaviour is unconventional,
710but was selected after some consideration.
711Observations suggest that the type of systems we want to support
712either have lowercase as customary case (unix, mac, windows)
713or silently convert lowercase to uppercase (lpns),
714so this makes more sense than attempting to use @code{:case :common},
715which is reported not to work on some implementations
716
717
718@section Other Operations
719
720ASDF provides three commands for the most common system operations:
721@code{load-system}, @code{compile-system} or @code{test-system}.
722It also provides @code{require-system}, a version of @code{load-system}
723that skips trying to update systems that are already loaded.
724
725Because ASDF is an extensible system
726for defining @emph{operations} on @emph{components},
727it also provides a generic function @code{operate}
728(which is usually abbreviated by @code{oos}).
729You'll use @code{oos} whenever you want to do something beyond
730compiling, loading and testing.
731
732Output from ASDF and ASDF extensions are supposed to be sent
733to the CL stream @code{*standard-output*},
734and so rebinding that stream around calls to @code{asdf:operate}
735should redirect all output from ASDF operations.
736
737Reminder: before ASDF can operate on a system, however,
738it must be able to find and load that system's definition.
739@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
740
741For the advanced users, note that
742@code{require-system} calls @code{load-system}
743with keyword arguments @code{:force-not (loaded-systems)}.
744@code{loaded-systems} returns a list of the names of loaded systems.
745@code{load-system} applies @code{operate} with the operation from
746@code{*load-system-operation*}, which by default is @code{load-op},
747the system, and any provided keyword arguments.
748
749@section Summary
750
751To use ASDF:
752
753@itemize
754@item
755Load ASDF itself into your Lisp image, either through
756@code{(require "asdf")} or else through
757@code{(load "/path/to/asdf.lisp")}.
758
759@item
760Make sure ASDF can find system definitions
761thanks to proper source-registry configuration.
762
763@item
764Load a system with @code{(asdf:load-system :my-system)}
765or use some other operation on some system of your choice.
766
767@end itemize
768
769@section Moving on
770
771That's all you need to know to use ASDF to load systems written by others.
772The rest of this manual deals with writing system definitions
773for Common Lisp software you write yourself,
774including how to extend ASDF to define new operation and component types.
775
776
777@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
778@comment  node-name,  next,  previous,  up
779@chapter Defining systems with defsystem
780
781This chapter describes how to use asdf to define systems and develop
782software.
783
784
785@menu
786* The defsystem form::
787* A more involved example::
788* The defsystem grammar::
789* Other code in .asd files::
790@end menu
791
792@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
793@comment  node-name,  next,  previous,  up
794@section The defsystem form
795
796Systems can be constructed programmatically
797by instantiating components using @code{make-instance}.
798Most of the time, however, it is much more practical to use
799a static @code{defsystem} form.
800This section begins with an example of a system definition,
801then gives the full grammar of @code{defsystem}.
802
803Let's look at a simple system.
804This is a complete file that would
805usually be saved as @file{hello-lisp.asd}:
806
807@lisp
808(in-package :asdf)
809
810(defsystem "hello-lisp"
811  :description "hello-lisp: a sample Lisp system."
812  :version "0.2.1"
813  :author "Joe User <joe@@example.com>"
814  :licence "Public Domain"
815  :components ((:file "packages")
816               (:file "macros" :depends-on ("packages"))
817               (:file "hello" :depends-on ("macros"))))
818@end lisp
819
820Some notes about this example:
821
822@itemize
823
824@item
825The file starts with an @code{in-package} form
826to use package @code{asdf}.
827You could instead start your definition by using
828a qualified name @code{asdf:defsystem}.
829
830@item
831If in addition to simply using @code{defsystem},
832you are going to define functions,
833create ASDF extension, globally bind symbols, etc.,
834it is recommended that to avoid namespace pollution between systems,
835you should create your own package for that purpose,
836for instance replacing the above @code{(in-package :asdf)} with:
837
838@lisp
839(defpackage :foo-system
840  (:use :cl :asdf))
841
842(in-package :foo-system)
843@end lisp
844
845@item
846The @code{defsystem} form defines a system named @code{hello-lisp}
847that contains three source files:
848@file{packages}, @file{macros} and @file{hello}.
849
850@item
851The file @file{macros} depends on @file{packages}
852(presumably because the package it's in is defined in @file{packages}),
853and the file @file{hello} depends on @file{macros}
854(and hence, transitively on @file{packages}).
855This means that ASDF will compile and load @file{packages} and @file{macros}
856before starting the compilation of file @file{hello}.
857
858@item
859The files are located in the same directory
860as the file with the system definition.
861ASDF resolves symbolic links (or Windows shortcuts)
862before loading the system definition file and
863stores its location in the resulting system@footnote{
864It is possible, though almost never necessary, to override this behaviour.}.
865This is a good thing because the user can move the system sources
866without having to edit the system definition.
867
868@c FIXME: Should have cross-reference to "Version specifiers" in the
869@c defsystem grammar, but the cross-referencing is so broken by
870@c insufficient node breakdown that I have not put one in.
871@item
872Make sure you know how the @code{:version} numbers will be parsed!  They
873are parsed as period-separated lists of integers.  I.e., in the example,
874@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
875In particular, version @code{0.2.1} is interpreted the same as
876@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
877even though the two are the same when interpreted as decimal fractions.
878Instead of a string representing the version,
879the @code{:version} argument can be an expression that is resolved to
880such a string using the following trivial domain-specific language:
881in addition to being a literal string, it can be an expression of the form
882@code{(:read-file-form <pathname-or-string> :at <access-at-specifier>)},
883which will be resolved by reading a form
884in the specified pathname
885(read as a subpathname of the current system if relative or a unix-namestring).
886You may use an access-at specifier with the (optional) :at keyword,
887by default the specifier is 0, meaning the first form is returned.
888
889@cindex :version
890
891@end itemize
892
893@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
894@comment  node-name,  next,  previous,  up
895@section A more involved example
896
897Let's illustrate some more involved uses of @code{defsystem} via a
898slightly convoluted example:
899
900@lisp
901(defsystem "foo"
902  :version "1.0.0"
903  :components ((:module "mod"
904                            :components ((:file "bar")
905                                                  (:file"baz")
906                                                  (:file "quux"))
907                            :perform (compile-op :after (op c)
908                                                  (do-something c))
909                            :explain (compile-op :after (op c)
910                                            (explain-something c)))
911                         (:file "blah")))
912@end lisp
913
914The @code{:module} component named @code{"mod"} is a collection of three files,
915which will be located in a subdirectory of the main code directory named
916@file{mod} (this location can be overridden; see the discussion of the
917@code{:pathname} option in @ref{The defsystem grammar}).
918
919The method-form tokens provide a shorthand for defining methods on
920particular components.  This part
921
922@lisp
923                :perform (compile-op :after (op c)
924                          (do-something c))
925                :explain (compile-op :after (op c)
926                          (explain-something c))
927@end lisp
928
929has the effect of
930
931@lisp
932(defmethod perform :after ((op compile-op) (c (eql ...)))
933           (do-something c))
934(defmethod explain :after ((op compile-op) (c (eql ...)))
935           (explain-something c))
936@end lisp
937
938where @code{...} is the component in question.
939In this case @code{...} would expand to something like
940
941@lisp
942(find-component (find-system "foo") "mod")
943@end lisp
944
945For more details on the syntax of such forms, see @ref{The defsystem
946grammar}.
947For more details on what these methods do, @pxref{Operations} in
948@ref{The object model of ASDF}.
949
950@c The following plunge into the weeds is not appropriate in this
951@c location. [2010/10/03:rpg]
952@c note that although this also supports @code{:before} methods,
953@c they may not do what you want them to ---
954@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
955@c will run after all the dependencies and sub-components have been processed,
956@c but before the component in question has been compiled.
957
958@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
959@comment  node-name,  next,  previous,  up
960@section The defsystem grammar
961
962@c FIXME: @var typesetting not consistently used here.  We should either expand
963@c its use to everywhere, or we should kill it everywhere.
964
965
966@example
967system-definition := ( defsystem system-designator @var{system-option}* )
968
969system-option := :defsystem-depends-on system-list
970                 | :weakly-depends-on @var{system-list}
971                 | :class class-name (see discussion below)
972                 | module-option
973                 | option
974
975module-option := :components component-list
976                 | :serial [ t | nil ]
977
978option :=
979        | :pathname pathname-specifier
980        | :default-component-class class-name
981        | :perform method-form
982        | :explain method-form
983        | :output-files method-form
984        | :operation-done-p method-form
985        | :if-feature feature-expression
986        | :depends-on ( @var{dependency-def}* )
987        | :in-order-to ( @var{dependency}+ )
988
989
990system-list := ( @var{simple-component-name}* )
991
992component-list := ( @var{component-def}* )
993
994component-def  := ( component-type simple-component-name @var{option}* )
995
996component-type := :system | :module | :file | :static-file | other-component-type
997
998other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
999
1000dependency-def := simple-component-name
1001               | ( :feature name )
1002               | ( :version simple-component-name version-specifier)
1003
1004dependency := (dependent-op @var{requirement}+)
1005requirement := (required-op @var{required-component}+)
1006             | (feature feature-name)
1007dependent-op := operation-name
1008required-op := operation-name | feature
1009
1010simple-component-name := string
1011                      |  symbol
1012
1013pathname-specifier := pathname | string | symbol
1014
1015method-form := (operation-name qual lambda-list @&rest body)
1016qual := method qualifier
1017
1018component-dep-fail-option := :fail | :try-next | :ignore
1019
1020feature-expression := keyword | (:and @var{feature-expression}*)
1021                      | (:or @var{feature-expression}*) | (:not @var{feature-expression})
1022@end example
1023
1024
1025@subsection Component names
1026
1027Component names (@code{simple-component-name})
1028may be either strings or symbols.
1029
1030@subsection Component types
1031
1032Component type names, even if expressed as keywords, will be looked up
1033by name in the current package and in the asdf package, if not found in
1034the current package.  So a component type @code{my-component-type}, in
1035the current package @code{my-system-asd} can be specified as
1036@code{:my-component-type}, or @code{my-component-type}.
1037
1038@subsection System class names
1039
1040A system class name will be looked up in the same way as a Component
1041type (see above).  Typically, one will not need to specify a system
1042class name, unless using a non-standard system class defined in some
1043ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
1044see below.  For such class names in the ASDF package, we recommend that
1045the @code{:class} option be specified using a keyword symbol, such as
1046
1047@example
1048:class :MY-NEW-SYSTEM-SUBCLASS
1049@end example
1050
1051This practice will ensure that package name conflicts are avoided.
1052Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
1053the current package @emph{before} it has been exported from the ASDF
1054extension loaded by @code{:defsystem-depends-on}, causing a name
1055conflict in the current package.
1056
1057@subsection Defsystem depends on
1058@cindex :defsystem-depends-on
1059
1060The @code{:defsystem-depends-on} option to @code{defsystem} allows the
1061programmer to specify another ASDF-defined system or set of systems that
1062must be loaded @emph{before} the system definition is processed.
1063Typically this is used to load an ASDF extension that is used in the
1064system definition.
1065
1066@subsection Weakly depends on
1067@cindex :weakly-depends-on
1068
1069We do @emph{NOT} recommend you use this feature.
1070If you are tempted to write a system @var{foo}
1071that weakly-depends-on a system @var{bar},
1072we recommend that you should instead
1073write system @var{foo} in a parametric way,
1074and offer some special variable and/or some hook to specialize its behavior;
1075then you should write a system @var{foo+bar}
1076that does the hooking of things together.
1077
1078The (deprecated) @code{:weakly-depends-on} option to @code{defsystem}
1079allows the programmer to specify another ASDF-defined system or set of systems
1080that ASDF should @emph{try} to load,
1081but need not load in order to be successful.
1082Typically this is used if there are a number of systems
1083that, if present, could provide additional functionality,
1084but which are not necessary for basic function.
1085
1086Currently, although it is specified to be an option only to @code{defsystem},
1087this option is accepted at any component, but it probably
1088only makes sense at the @code{defsystem} level.
1089Programmers are cautioned not
1090to use this component option except at the @code{defsystem} level, as
1091this anomalous behavior may be removed without warning.
1092
1093Finally, you might look into the @code{asdf-system-connections} extension,
1094that will let you define additional code to be loaded
1095when two systems are simultaneously loaded.
1096It may or may not be considered good style, but at least it can be used
1097in a way that has deterministic behavior independent of load order,
1098unlike @code{weakly-depends-on}.
1099
1100
1101@subsection Pathname specifiers
1102@cindex pathname specifiers
1103
1104A pathname specifier (@code{pathname-specifier})
1105may be a pathname, a string or a symbol.
1106When no pathname specifier is given for a component,
1107which is the usual case, the component name itself is used.
1108
1109If a string is given, which is the usual case,
1110the string will be interpreted as a Unix-style pathname
1111where @code{/} characters will be interpreted as directory separators.
1112Usually, Unix-style relative pathnames are used
1113(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1114they are relative to the path of the parent component.
1115Finally, depending on the @code{component-type},
1116the pathname may be interpreted as either a file or a directory,
1117and if it's a file,
1118a file type may be added corresponding to the @code{component-type},
1119or else it will be extracted from the string itself (if applicable).
1120
1121For instance, the @code{component-type} @code{:module}
1122wants a directory pathname, and so a string @code{"foo/bar"}
1123will be interpreted as the pathname @file{#p"foo/bar/"}.
1124On the other hand, the @code{component-type} @code{:file}
1125wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1126will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1127and a string @code{"foo/bar.quux"}
1128will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1129Finally, the @code{component-type} @code{:static-file}
1130wants a file without specifying a type, and so a string @code{"foo/bar"}
1131will be interpreted as the pathname @file{#p"foo/bar"},
1132and a string @code{"foo/bar.quux"}
1133will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1134
1135ASDF does not interpret the string @code{".."} to designate the parent
1136directory.  This string will be passed through to the underlying
1137operating system for interpretation.  We @emph{believe} that this will
1138work on all platforms where ASDF is deployed, but do not guarantee this
1139behavior.  A pathname object with a relative directory component of
1140@code{:up} or @code{:back} is the only guaranteed way to specify a
1141parent directory.
1142
1143If a symbol is given, it will be translated into a string,
1144and downcased in the process.
1145The downcasing of symbols is unconventional,
1146but was selected after some consideration.
1147Observations suggest that the type of systems we want to support
1148either have lowercase as customary case (Unix, Mac, windows)
1149or silently convert lowercase to uppercase (lpns),
1150so this makes more sense than attempting to use @code{:case :common}
1151as argument to @code{make-pathname},
1152which is reported not to work on some implementations.
1153
1154Pathname objects may be given to override the path for a component.
1155Such objects are typically specified using reader macros such as @code{#p}
1156or @code{#.(make-pathname ...)}.
1157Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1158and that the behavior of @code{parse-namestring} is completely non-portable,
1159unless you are using Common Lisp @code{logical-pathname}s
1160(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1161Pathnames made with @code{#.(make-pathname ...)}
1162can usually be done more easily with the string syntax above.
1163The only case that you really need a pathname object is to override
1164the component-type default file type for a given component.
1165Therefore, pathname objects should only rarely be used.
1166Unhappily, ASDF 1 didn't properly support
1167parsing component names as strings specifying paths with directories,
1168and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1169An alternative to @code{#.} read-time evaluation is to use
1170@code{(eval `(defsystem ... ,pathname ...))}.
1171
1172Note that when specifying pathname objects,
1173ASDF does not do any special interpretation of the pathname
1174influenced by the component type, unlike the procedure for
1175pathname-specifying strings.
1176On the one hand, you have to be careful to provide a pathname that correctly
1177fulfills whatever constraints are required from that component type
1178(e.g. naming a directory or a file with appropriate type);
1179on the other hand, you can circumvent the file type that would otherwise
1180be forced upon you if you were specifying a string.
1181
1182@subsection Version specifiers
1183@cindex version specifiers
1184@cindex :version
1185
1186Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1187@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1188In particular, version @code{0.2.1} is interpreted the same as
1189@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1190even though the two are the same when interpreted as decimal fractions.
1191
1192System definers are encouraged to use version identifiers of the form
1193@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1194API) and patch level.
1195
1196@xref{Common attributes of components}.
1197
1198
1199@subsection Using logical pathnames
1200@cindex logical pathnames
1201
1202We do not generally recommend the use of logical pathnames,
1203especially not so to newcomers to Common Lisp.
1204However, we do support the use of logical pathnames by old timers,
1205when such is their preference.
1206
1207To use logical pathnames,
1208you will have to provide a pathname object as a @code{:pathname} specifier
1209to components that use it, using such syntax as
1210@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1211
1212You only have to specify such logical pathname
1213for your system or some top-level component.
1214Sub-components' relative pathnames,
1215specified using the string syntax for names,
1216will be properly merged with the pathnames of their parents.
1217The specification of a logical pathname host however is @emph{not}
1218otherwise directly supported in the ASDF syntax
1219for pathname specifiers as strings.
1220
1221The @code{asdf-output-translation} layer will
1222avoid trying to resolve and translate logical pathnames.
1223The advantage of this is that
1224you can define yourself what translations you want to use
1225with the logical pathname facility.
1226The disadvantage is that if you do not define such translations,
1227any system that uses logical pathnames will behave differently under
1228asdf-output-translations than other systems you use.
1229
1230If you wish to use logical pathnames you will have to configure the
1231translations yourself before they may be used.
1232ASDF currently provides no specific support
1233for defining logical pathname translations.
1234
1235Note that the reasons we do not recommend logical pathnames are that
1236(1) there is no portable way to set up logical pathnames before they are used,
1237(2) logical pathnames are limited to only portably use
1238a single character case, digits and hyphens.
1239While you can solve the first issue on your own,
1240describing how to do it on each of fifteen implementations supported by ASDF
1241is more than we can document.
1242As for the second issue, mind that the limitation is notably enforced on SBCL,
1243and that you therefore can't portably violate the limitations
1244but must instead define some encoding of your own and add individual mappings
1245to name physical pathnames that do not fit the restrictions.
1246This can notably be a problem when your Lisp files are part of a larger project
1247in which it is common to name files or directories in a way that
1248includes the version numbers of supported protocols,
1249or in which files are shared with software written
1250in different programming languages where conventions include the use of
1251underscores, dots or CamelCase in pathnames.
1252
1253
1254@subsection Serial dependencies
1255@cindex serial dependencies
1256
1257If the @code{:serial t} option is specified for a module,
1258ASDF will add dependencies for each child component,
1259on all the children textually preceding it.
1260This is done as if by @code{:depends-on}.
1261
1262@lisp
1263:serial t
1264:components ((:file "a") (:file "b") (:file "c"))
1265@end lisp
1266
1267is equivalent to
1268
1269@lisp
1270:components ((:file "a")
1271             (:file "b" :depends-on ("a"))
1272             (:file "c" :depends-on ("a" "b")))
1273@end lisp
1274
1275
1276@subsection Source location
1277
1278The @code{:pathname} option is optional in all cases for systems
1279defined via @code{defsystem},
1280and in the usual case the user is recommended not to supply it.
1281
1282Instead, ASDF follows a hairy set of rules that are designed so that
1283@enumerate
1284@item
1285@code{find-system}
1286will load a system from disk
1287and have its pathname default to the right place.
1288
1289@item
1290This pathname information will not be overwritten with
1291@code{*default-pathname-defaults*}
1292(which could be somewhere else altogether)
1293if the user loads up the @file{.asd} file into his editor
1294and interactively re-evaluates that form.
1295@end enumerate
1296
1297If a system is being loaded for the first time,
1298its top-level pathname will be set to:
1299
1300@itemize
1301@item
1302The host/device/directory parts of @code{*load-truename*},
1303if it is bound.
1304@item
1305@code{*default-pathname-defaults*}, otherwise.
1306@end itemize
1307
1308If a system is being redefined, the top-level pathname will be
1309
1310@itemize
1311@item
1312changed, if explicitly supplied or obtained from @code{*load-truename*}
1313(so that an updated source location is reflected in the system definition)
1314
1315@item
1316changed if it had previously been set from @code{*default-pathname-defaults*}
1317
1318@item
1319left as before, if it had previously been set from @code{*load-truename*}
1320and @code{*load-truename*} is currently unbound
1321(so that a developer can evaluate a @code{defsystem} form
1322from within an editor without clobbering its source location)
1323@end itemize
1324
1325@subsection if-feature option
1326This option allows you to specify a feature expression to be evaluated
1327as if by @code{#+} to conditionally include a component in your build.
1328If the expression is false, the component is dropped
1329as well as any dependency pointing to it.
1330As compared to using @code{#+} which is expanded at read-time,
1331this allows you to have an object in your component hierarchy
1332that can be used for manipulations beside building your project.
1333This option was added in ASDF 3.
1334
1335@subsection if-component-dep-fails option
1336This option was removed in ASDF 3.
1337Its semantics was limited in purpose and dubious to explain,
1338and its implementation was breaking a hole into the ASDF object model.
1339Please use the @code{if-feature} option instead.
1340
1341@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1342@section Other code in .asd files
1343
1344Files containing @code{defsystem} forms
1345are regular Lisp files that are executed by @code{load}.
1346Consequently, you can put whatever Lisp code you like into these files
1347(e.g., code that examines the compile-time environment
1348and adds appropriate features to @code{*features*}).
1349However, some conventions should be followed,
1350so that users can control certain details of execution
1351of the Lisp in @file{.asd} files:
1352
1353@itemize
1354@item
1355Any informative output
1356(other than warnings and errors,
1357which are the condition system's to dispose of)
1358should be sent to the standard CL stream @code{*standard-output*},
1359so that users can easily control the disposition
1360of output from ASDF operations.
1361@end itemize
1362
1363
1364@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1365@comment  node-name,  next,  previous,  up
1366@chapter The object model of ASDF
1367
1368ASDF is designed in an object-oriented way from the ground up.
1369Both a system's structure and the operations that can be performed on systems
1370follow a extensible protocol.
1371
1372This allows the addition of behaviours:
1373for example, @code{cffi} adds support of special FFI description files
1374to interface with C libraries and of wrapper files to embed C code in Lisp;
1375@code{abcl-jar} supports creating Java JAR archives in ABCL;
1376and @code{poiu} supports for compiling code in parallel using background processes.
1377
1378This chapter deals with @code{component}s and @code{operation}s.
1379
1380A @code{component} represents an individual source file or a group of source files,
1381and the things that get transformed into.
1382A @code{system} is a component at the top level of the component hierarchy.
1383A @code{source-file} is a component representing a single source-file
1384and the successive output files into which it is transformed.
1385A @code{module} is an intermediate component itself grouping several other components,
1386themselves source-files or further modules.
1387
1388An @code{Operation} represents a transformation that can be performed on a component,
1389turning them from source files to intermediate results to final outputs.
1390
1391A pair of an @code{operation} and a @code{component} is called an @code{action}.
1392An @code{action} represents a particular build step to be @code{perform}ed,
1393after all its dependencies have been fulfilled.
1394In the ASDF model, actions depend on other actions.
1395The term @emph{action} itself was used by Kent Pitman in his old article,
1396but was only used by ASDF hackers starting with the ASDF 2;
1397but the concept is ubiquitous since the very beginning of ASDF 1,
1398though previously implicit.
1399
1400Then, there are many @emph{functions} available
1401to users, extenders and implementers of ASDF
1402to use, define or implement the activities
1403that are part of building your software.
1404Though they manipulate @code{action}s,
1405most of these functions do not take as an argument
1406a reified pair (a CONS cell) of an operation and a component;
1407instead, they usually take two separate arguments,
1408which allows to take advantage of the power CLOS-style multiple dispatch
1409for fun and profit.
1410
1411There are many @emph{hooks} in which to add functionality,
1412by customizing the behavior of existing @emph{functions}.
1413
1414Last but not least is the notion of @emph{dependency} between two actions.
1415The structure of dependencies between actions is
1416a directed @emph{dependency graph}.
1417ASDF is invoked by being told to @emph{operate}
1418with some @emph{operation} on some toplevel @emph{system};
1419it will then @emph{traverse} the graph and build a @emph{plan}
1420that follows its structure.
1421To be successfully buildable, this graph of actions but be acyclic.
1422If, as a user, extender or implementer of ASDF, you fail
1423to keep the dependency graph without cycles,
1424ASDF will fail loudly as it eventually finds one.
1425To clearly distinguish the direction of dependencies,
1426ASDF 3 uses the words @emph{requiring} and @emph{required}
1427as applied to an action depending on the other:
1428the requiring action @code{depends-on} the completion of all required actions
1429before it may itself be @code{perform}ed.
1430
1431Using the @code{defsystem} syntax, users may easily express
1432direct dependencies along the graph of the object hierarchy:
1433between a component and its parent, its children, and its siblings.
1434By defining custom CLOS methods, you can express more elaborate dependencies as you wish.
1435Most common operations, such as @code{load-op}, @code{compile-op} or @code{load-source-op}
1436are automatically propagate ``downward'' the component hierarchy and are ``covariant'' with it:
1437to act the operation on the parent module, you must first act it on all the children components,
1438with the action on the parent being parent of the action on each child.
1439Other operations, such as @code{prepare-op} and @code{prepare-source-op}
1440(introduced in ASDF 3) are automatically propagated ``upward'' the component hierarchy
1441and are ``contravariant'' with it:
1442to perform the operation of preparing for compilation of a child component,
1443you must perform the operation of preparing for compilation of its parent component, and so on,
1444ensuring that all the parent's dependencies are (compiled and) loaded
1445before the child component may be compiled and loaded.
1446Yet other operations, such as @code{test-op} or @code{load-fasl-op}
1447remain at the system level, and are not propagated along the hierarchy,
1448but instead do something global on the system.
1449
1450@menu
1451* Operations::
1452* Components::
1453* Functions::
1454@end menu
1455
1456@node  Operations, Components, The object model of ASDF, The object model of ASDF
1457@comment  node-name,  next,  previous,  up
1458@section Operations
1459@cindex operation
1460
1461An @dfn{operation} object of the appropriate type is instantiated
1462whenever the user wants to do something with a system like
1463
1464@itemize
1465@item compile all its files
1466@item load the files into a running lisp environment
1467@item copy its source files somewhere else
1468@end itemize
1469
1470Operations can be invoked directly, or examined
1471to see what their effects would be without performing them.
1472There are a bunch of methods specialised on operation and component type
1473that actually do the grunt work.
1474
1475The operation object contains whatever state is relevant for this purpose
1476(perhaps a list of visited nodes, for example)
1477but primarily is a nice thing to specialise operation methods on
1478and easier than having them all be @code{EQL} methods.
1479
1480Operations are invoked on systems via @code{operate}.
1481@anchor{operate}
1482@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs} @&key @code{force} @code{force-not} @code{verbose} @&allow-other-keys
1483@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs} @&key @&allow-other-keys
1484@code{operate} invokes @var{operation} on @var{system}.
1485@code{oos} is a synonym for @code{operate}.
1486
1487@var{operation} is a symbol that is passed, along with the supplied
1488@var{initargs}, to @code{make-instance} to create the operation object.
1489@var{system} is a system designator.
1490
1491The @var{initargs} are passed to the @code{make-instance} call
1492when creating the operation object.
1493Note that dependencies may cause the operation
1494to invoke other operations on the system or its components:
1495the new operations will be created
1496with the same @var{initargs} as the original one.
1497
1498If @var{force} is @code{:all}, then all systems
1499are forced to be recompiled even if not modified since last compilation.
1500If @var{force} is @code{t}, then only the system being loaded
1501is forced to be recompiled even if not modified since last compilation,
1502but other systems are not affected.
1503If @var{force} is a list, then it specifies a list of systems that
1504are forced to be recompiled even if not modified since last compilation.
1505If @var{force-not} is @code{:all}, then all systems
1506are forced not to be recompiled even if modified since last compilation.
1507If @var{force-not} is @code{t}, then only the system being loaded
1508is forced not to be recompiled even if modified since last compilation,
1509but other systems are not affected.
1510If @var{force-not} is a list, then it specifies a list of systems that
1511are forced not to be recompiled even if modified since last compilation.
1512@var{force} takes precedences over @var{force-not};
1513both of them apply to systems that are dependencies and were already compiled.
1514
1515To see what @code{operate} would do, you can use:
1516@example
1517(asdf::traverse (make-instance operation-class initargs ...) (find-system system-name))
1518@end example
1519
1520@end deffn
1521
1522@menu
1523* Predefined operations of ASDF::
1524* Creating new operations::
1525@end menu
1526
1527@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1528@comment  node-name,  next,  previous,  up
1529@subsection Predefined operations of ASDF
1530
1531All the operations described in this section are in the @code{asdf} package.
1532They are invoked via the @code{operate} generic function.
1533
1534@lisp
1535(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1536@end lisp
1537
1538@deffn Operation @code{compile-op} @&key @code{proclamations}
1539
1540This operation compiles the specified component.
1541If proclamations are supplied, they will be proclaimed.
1542This is a good place to specify optimization settings.
1543
1544When creating a new component type,
1545you should provide methods for @code{compile-op}.
1546
1547When @code{compile-op} is invoked,
1548component dependencies often cause some parts of the system
1549to be loaded as well as compiled.
1550Invoking @code{compile-op}
1551does not necessarily load all the parts of the system, though;
1552use @code{load-op} to load a system.
1553@end deffn
1554
1555@deffn Operation @code{load-op} @&key @code{proclamations}
1556
1557This operation loads a system.
1558
1559The default methods for @code{load-op} compile files before loading them.
1560For parity, your own methods on new component types should probably do so too.
1561@end deffn
1562
1563@deffn Operation @code{parent-load-op} @&key @code{proclamations}
1564
1565This operation ensures that the dependencies
1566of a module, and its parent, and so on, are loaded (as per @code{load-op})
1567before the components within that module may be operated upon.
1568
1569By default, all operations depend on this @code{parent-operation}
1570for actions on components to depend on this ``parent operation'' being acted on the parent.
1571
1572The default methods for @code{load-op} compile files before loading them.
1573For parity, your own methods on new component types should probably do so too.
1574@end deffn
1575
1576@deffn Operation @code{load-source-op}
1577
1578This operation will load the source for the files in a module
1579even if the source files have been compiled.
1580Systems sometimes have knotty dependencies
1581which require that sources are loaded
1582before they can be compiled.
1583This is how you do that.
1584
1585If you are creating a component type,
1586you need to implement this operation --- at least, where meaningful.
1587@end deffn
1588
1589@anchor{test-op}
1590@deffn Operation @code{test-op}
1591
1592This operation will perform some tests on the module.
1593The default method will do nothing.
1594The default dependency is to require
1595@code{load-op} to be performed on the module first.
1596The default @code{operation-done-p} is that the operation is @emph{never} done
1597---
1598we assume that if you invoke the @code{test-op},
1599you want to test the system, even if you have already done so.
1600
1601The results of this operation are not defined by ASDF.
1602It has proven difficult to define how the test operation
1603should signal its results to the user
1604in a way that is compatible with all of the various test libraries
1605and test techniques in use in the community.
1606
1607People typically define @code{test-op} methods like thus:
1608@example
1609(defmethod perform ((o asdf:test-op) (s (eql (asdf:find-system @var{:mysystem}))))
1610  (asdf:load-system @var{:mysystem})
1611  (eval (read-from-string "(some expression that runs the tests)"))
1612  t)
1613@end example
1614@end deffn
1615
1616@deffn Operation @code{load-fasl-op}
1617
1618This operation will load and create if need be
1619a single fasl file for all the files in each loaded system.
1620(Its compilation-only equivalent is @code{asdf::fasl-op}.)
1621
1622Once you have created such a fasl,
1623you can use @code{precompiled-system} to deliver it in a way
1624that is compatible with clients having asdf dependencies
1625on your system whether it is distributed as source of as a single binary.
1626
1627On your build platform, you run something like that:
1628@example
1629@code{(asdf:operate 'load-fasl-op @var{:mysystem})}
1630@end example
1631
1632And on your delivery platform, a form like this is evaluated
1633in a prologue or at some point before you save your image:
1634@example
1635(defsystem :mysystem :class :precompiled-system
1636    :fasl (some expression that will evaluate to a pathname))
1637@end example
1638
1639Of course, @emph{before} you define such systems,
1640you should not forget to @code{(asdf:clear-configuration)}.
1641
1642@code{load-fasl-op} is available on all actively supported Lisp implementations,
1643and on those implementations only, and only since ASDF 3.
1644This functionality was previously available for select implementations,
1645as part of a separate system @code{asdf-bundle},
1646itself descended from @code{asdf-ecl}.
1647@end deffn
1648
1649
1650@c @deffn Operation test-system-version @&key minimum
1651
1652@c Asks the system whether it satisfies a version requirement.
1653
1654@c The default method accepts a string, which is expected to contain of a
1655@c number of integers separated by #\. characters.  The method is not
1656@c recursive.  The component satisfies the version dependency if it has
1657@c the same major number as required and each of its sub-versions is
1658@c greater than or equal to the sub-version number required.
1659
1660@c @lisp
1661@c (defun version-satisfies (x y)
1662@c   (labels ((bigger (x y)
1663@c           (cond ((not y) t)
1664@c                 ((not x) nil)
1665@c                 ((> (car x) (car y)) t)
1666@c                 ((= (car x) (car y))
1667@c                  (bigger (cdr x) (cdr y))))))
1668@c     (and (= (car x) (car y))
1669@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1670@c @end lisp
1671
1672@c If that doesn't work for your system, you can override it.  I hope
1673@c you have as much fun writing the new method as @verb{|#lisp|} did
1674@c reimplementing this one.
1675@c @end deffn
1676
1677@c @deffn Operation feature-dependent-op
1678
1679@c An instance of @code{feature-dependent-op} will ignore any components
1680@c which have a @code{features} attribute, unless the feature combination
1681@c it designates is satisfied by @code{*features*}.  This operation is
1682@c not intended to be instantiated directly, but other operations may
1683@c inherit from it.
1684
1685@c @end deffn
1686
1687@node  Creating new operations,  , Predefined operations of ASDF, Operations
1688@comment  node-name,  next,  previous,  up
1689@subsection Creating new operations
1690
1691ASDF was designed to be extensible in an object-oriented fashion.
1692To teach ASDF new tricks, a programmer can implement the behaviour he wants
1693by creating a subclass of @code{operation}.
1694
1695ASDF's pre-defined operations are in no way ``privileged'',
1696but it is requested that developers never use the @code{asdf} package
1697for operations they develop themselves.
1698The rationale for this rule is that we don't want to establish a
1699``global asdf operation name registry'',
1700but also want to avoid name clashes.
1701
1702An operation must provide methods for the following generic functions
1703when invoked with an object of type @code{source-file}:
1704@emph{FIXME describe this better}
1705
1706@itemize
1707
1708@item @code{input-files}
1709ASDF has a pretty clever default @code{input-files} mechanism.
1710You only need create a method if there are multiple ultimate input files,
1711and/or the bottom one doesn't depend
1712on the @code{component-pathname} of the component.
1713
1714@item @code{output-files}
1715The @code{output-files} method determines where the method will put its files.
1716It returns two values, a list of pathnames, and a boolean.
1717If the boolean is @code{T} then the pathnames are marked
1718not be translated by enclosing @code{:around} methods.
1719If the boolean is @code{NIL} then enclosing @code{:around} methods
1720may translate these pathnames, e.g. to ensure object files
1721are somehow stored in some implementation-dependent cache.
1722
1723@item @code{perform}
1724The @code{perform} method must call @code{output-files}
1725to find out where to put its files,
1726because the user is allowed to override.
1727@item @code{output-files}
1728for local policy @code{explain}
1729
1730@item @code{operation-done-p}
1731You only need to define a method on that function
1732if you can detect conditions that invalidate previous runs of the operation,
1733even though no filesystem timestamp has changed,
1734in which case you return @code{nil} (the default is @code{t}).
1735
1736For instance, the method for @code{test-op} always returns @code{nil},
1737so that tests are always run afresh.
1738Of course, the @code{test-op} for your system could depend
1739on a deterministically repeatable @code{test-report-op},
1740and just read the results from the report files.
1741
1742@item @code{component-depends-on}
1743When you add new operations, you probably need to explain
1744how they relate to loading, compiling, testing, etc.,
1745in terms of dependencies between actions.
1746
1747That's where you typically define methods on @code{component-depends-on}.
1748Your method will take as arguments
1749some properly specialized operation
1750and a component denoting a current action,
1751and return a list of entries,
1752denoting the children actions that the current action depends on.
1753The format of entries is described below.
1754
1755It is @emph{strongly} advised that
1756you should always append the results of @code{(call-next-method)}
1757to the results of your method,
1758or ``interesting'' failures will likely occur,
1759unless you're a true specialist of ASDF internals.
1760
1761Each entry returned by @code{component-depends-on} is itself a list.
1762
1763The first element of an entry is the name of an operation:
1764a symbol that you can use with @code{make-instance}
1765(ASDF will instead use with @code{asdf::make-sub-operation}),
1766to create a related operation for use in a build plan.
1767For instance, @code{load-op} and @code{compile-op}
1768are common such names, denoting the respective operations.
1769
1770The rest of an entry is a list of identifiers
1771each denote a component such that
1772the pair of the previous operation and this component
1773is a children action of current action.
1774
1775Identifiers follow the @code{defsystem} grammar
1776previously documented.
1777The main format for identifiers is a string or symbol
1778(that will be downcase as per @code{coerce-name}),
1779and looked up against the sibling list of the parent module's children components,
1780as per @code{find-component}.
1781As a special case, @code{nil} denotes the parent itself.
1782Other syntaxes are allowed, for instance to specify a component with a version.
1783
1784@end itemize
1785
1786Operations that print output should send that output to the standard
1787CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1788
1789@node Components, Functions, Operations, The object model of ASDF
1790@comment  node-name,  next,  previous,  up
1791@section Components
1792@cindex component
1793@cindex system
1794@cindex system designator
1795@vindex *system-definition-search-functions*
1796
1797A @dfn{component} represents a source file or
1798(recursively) a collection of components.
1799A @dfn{system} is (roughly speaking) a top-level component
1800that can be found via @code{find-system}.
1801
1802A @dfn{system designator} is a string or symbol
1803and behaves just like any other component name
1804(including with regard to the case conversion rules for component names).
1805
1806
1807@defun find-system system-designator &optional (error-p t)
1808
1809Given a system designator, @code{find-system} finds and returns a system.
1810If no system is found, an error of type
1811@code{missing-component} is thrown,
1812or @code{nil} is returned if @code{error-p} is false.
1813
1814To find and update systems, @code{find-system} funcalls each element
1815in the @code{*system-definition-search-functions*} list,
1816expecting a pathname to be returned, or a system object,
1817from which a pathname may be extracted, and that will be registered.
1818The resulting pathname (if any) is loaded
1819if one of the following conditions is true:
1820
1821@itemize
1822@item
1823there is no system of that name in memory
1824@item
1825the pathname is different from that which was previously loaded
1826@item
1827the file's @code{last-modified} time exceeds the @code{last-modified} time
1828of the system in memory
1829@end itemize
1830
1831When system definitions are loaded from @file{.asd} files,
1832a new scratch package is created for them to load into,
1833so that different systems do not overwrite each others operations.
1834The user may also wish to (and is recommended to)
1835include @code{defpackage} and @code{in-package} forms
1836in his system definition files, however,
1837so that they can be loaded manually if need be.
1838
1839The default value of @code{*system-definition-search-functions*}
1840is a list of two functions.
1841The first function looks in each of the directories given
1842by evaluating members of @code{*central-registry*}
1843for a file whose name is the name of the system and whose type is @file{asd}.
1844The first such file is returned,
1845whether or not it turns out to actually define the appropriate system.
1846The second function does something similar,
1847for the directories specified in the @code{source-registry}.
1848Hence, it is strongly advised to define a system
1849@var{foo} in the corresponding file @var{foo.asd}.
1850@end defun
1851
1852
1853@menu
1854* Common attributes of components::
1855* Pre-defined subclasses of component::
1856* Creating new component types::
1857@end menu
1858
1859@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1860@comment  node-name,  next,  previous,  up
1861@subsection Common attributes of components
1862
1863All components, regardless of type, have the following attributes.
1864All attributes except @code{name} are optional.
1865
1866@subsubsection Name
1867
1868A component name is a string or a symbol.
1869If a symbol, its name is taken and lowercased.
1870
1871Unless overridden by a @code{:pathname} attribute,
1872the name will be interpreted as a pathname specifier according
1873to a Unix-style syntax.
1874@xref{The defsystem grammar,,Pathname specifiers}.
1875
1876@subsubsection Version identifier
1877@findex version-satisfies
1878@cindex :version
1879
1880This optional attribute is used by the generic function
1881@code{version-satisfies}, which tests to see if @code{:version}
1882dependencies are satisfied.
1883the version should be a string of integers separated by dots,
1884for example @samp{1.0.11}.
1885For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1886
1887@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1888@c @xref{Predefined operations of ASDF}.
1889@c @emph{Nota Bene}:
1890@c This operation, planned for ASDF 1,
1891@c is still not implemented yet as of ASDF 3.
1892@c Don't hold your breath.
1893
1894
1895
1896@subsubsection Required features
1897
1898@emph{FIXME: This subsection seems to contradict the
1899@code{defsystem} grammar subsection,
1900which doesn't provide any obvious way to specify required features.
1901Furthermore, in 2009, discussions on the
1902@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1903suggested that the specification of required features may be broken,
1904and that no one may have been using them for a while.
1905Please contact the
1906@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1907if you are interested in getting this features feature fixed.}
1908
1909Traditionally defsystem users have used reader conditionals
1910to include or exclude specific per-implementation files.
1911This means that any single implementation cannot read the entire system,
1912which becomes a problem if it doesn't wish to compile it,
1913but instead for example to create an archive file containing all the sources,
1914as it will omit to process the system-dependent sources for other systems.
1915
1916Each component in an asdf system may therefore specify features using
1917the same syntax as @code{#+} does, and it will (somehow) be ignored for
1918certain operations unless the feature conditional is a member of
1919@code{*features*}.
1920
1921
1922@subsubsection Dependencies
1923
1924This attribute specifies dependencies of the component on its siblings.
1925It is optional but often necessary.
1926
1927There is an excitingly complicated relationship between the initarg
1928and the method that you use to ask about dependencies
1929
1930Dependencies are between (operation component) pairs.
1931In your initargs for the component, you can say
1932
1933@lisp
1934:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1935              (load-op (load-op "foo")))
1936@end lisp
1937
1938This means the following things:
1939@itemize
1940@item
1941before performing compile-op on this component, we must perform
1942load-op on @var{a} and @var{b}, and compile-op on @var{c},
1943@item
1944before performing @code{load-op}, we have to load @var{foo}
1945@end itemize
1946
1947The syntax is approximately
1948
1949@verbatim
1950(this-op {(other-op required-components)}+)
1951
1952simple-component-name := string
1953                      |  symbol
1954
1955required-components := simple-component-name
1956                     | (required-components required-components)
1957
1958component-name := simple-component-name
1959                | (:version simple-component-name minimum-version-object)
1960@end verbatim
1961
1962Side note:
1963
1964This is on a par with what ACL defsystem does.
1965mk-defsystem is less general: it has an implied dependency
1966
1967@verbatim
1968  for all source file x, (load x) depends on (compile x)
1969@end verbatim
1970
1971and using a @code{:depends-on} argument to say that @var{b} depends on
1972@var{a} @emph{actually} means that
1973
1974@verbatim
1975  (compile b) depends on (load a)
1976@end verbatim
1977
1978This is insufficient for e.g. the McCLIM system, which requires that
1979all the files are loaded before any of them can be compiled ]
1980
1981End side note
1982
1983In ASDF, the dependency information for a given component and operation
1984can be queried using @code{(component-depends-on operation component)},
1985which returns a list
1986
1987@lisp
1988((load-op "a") (load-op "b") (compile-op "c") ...)
1989@end lisp
1990
1991@code{component-depends-on} can be subclassed for more specific
1992component/operation types: these need to @code{(call-next-method)}
1993and append the answer to their dependency, unless
1994they have a good reason for completely overriding the default dependencies.
1995
1996If it weren't for CLISP, we'd be using @code{LIST} method
1997combination to do this transparently.
1998But, we need to support CLISP.
1999If you have the time for some CLISP hacking,
2000I'm sure they'd welcome your fixes.
2001@c Doesn't CLISP now support LIST method combination?
2002
2003See the discussion of the semantics of @code{:version} in the defsystem
2004grammar.
2005
2006@c FIXME: Should have cross-reference to "Version specifiers" in the
2007@c defsystem grammar, but the cross-referencing is so broken by
2008@c insufficient node breakdown that I have not put one in.
2009
2010
2011@subsubsection pathname
2012
2013This attribute is optional and if absent (which is the usual case),
2014the component name will be used.
2015
2016@xref{The defsystem grammar,,Pathname specifiers},
2017for an explanation of how this attribute is interpreted.
2018
2019Note that the @code{defsystem} macro (used to create a ``top-level'' system)
2020does additional processing to set the filesystem location of
2021the top component in that system.
2022This is detailed elsewhere. @xref{Defining systems with defsystem}.
2023
2024
2025@subsubsection properties
2026
2027This attribute is optional.
2028
2029Packaging systems often require information about files or systems
2030in addition to that specified by ASDF's pre-defined component attributes.
2031Programs that create vendor packages out of ASDF systems therefore
2032have to create ``placeholder'' information to satisfy these systems.
2033Sometimes the creator of an ASDF system may know the additional
2034information and wish to provide it directly.
2035
2036@code{(component-property component property-name)} and
2037associated @code{setf} method will allow
2038the programmatic update of this information.
2039Property names are compared as if by @code{EQL},
2040so use symbols or keywords or something.
2041
2042@menu
2043* Pre-defined subclasses of component::
2044* Creating new component types::
2045@end menu
2046
2047@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
2048@comment  node-name,  next,  previous,  up
2049@subsection Pre-defined subclasses of component
2050
2051@deffn Component source-file
2052
2053A source file is any file that the system does not know how to
2054generate from other components of the system.
2055
2056Note that this is not necessarily the same thing as
2057``a file containing data that is typically fed to a compiler''.
2058If a file is generated by some pre-processor stage
2059(e.g. a @file{.h} file from @file{.h.in} by autoconf)
2060then it is not, by this definition, a source file.
2061Conversely, we might have a graphic file
2062that cannot be automatically regenerated,
2063or a proprietary shared library that we received as a binary:
2064these do count as source files for our purposes.
2065
2066Subclasses of source-file exist for various languages.
2067@emph{FIXME: describe these.}
2068@end deffn
2069
2070@deffn Component module
2071
2072A module is a collection of sub-components.
2073
2074A module component has the following extra initargs:
2075
2076@itemize
2077@item
2078@code{:components} the components contained in this module
2079
2080@item
2081@code{:default-component-class}
2082All children components which don't specify their class explicitly
2083are inferred to be of this type.
2084
2085@item
2086@code{:if-component-dep-fails}
2087This attribute was removed in ASDF 3. Do not use it.
2088Use @code{:if-feature} instead.
2089
2090@item
2091@code{:serial} When this attribute is set,
2092each subcomponent of this component is assumed to depend on all subcomponents
2093before it in the list given to @code{:components}, i.e.
2094all of them are loaded before a compile or load operation is performed on it.
2095
2096@end itemize
2097
2098The default operation knows how to traverse a module, so
2099most operations will not need to provide methods specialised on modules.
2100
2101@code{module} may be subclassed to represent components such as
2102foreign-language linked libraries or archive files.
2103@end deffn
2104
2105@deffn Component system
2106
2107@code{system} is a subclass of @code{module}.
2108
2109A system is a module with a few extra attributes for documentation
2110purposes; these are given elsewhere.
2111@xref{The defsystem grammar}.
2112
2113Users can create new classes for their systems:
2114the default @code{defsystem} macro takes a @code{:class} keyword argument.
2115@end deffn
2116
2117@node  Creating new component types,  , Pre-defined subclasses of component, Components
2118@comment  node-name,  next,  previous,  up
2119@subsection Creating new component types
2120
2121New component types are defined by subclassing one of the existing
2122component classes and specializing methods on the new component class.
2123
2124@emph{FIXME: this should perhaps be explained more throughly,
2125not only by example ...}
2126
2127As an example, suppose we have some implementation-dependent
2128functionality that we want to isolate
2129in one subdirectory per Lisp implementation our system supports.
2130We create a subclass of
2131@code{cl-source-file}:
2132
2133@lisp
2134(defclass unportable-cl-source-file (cl-source-file)
2135  ())
2136@end lisp
2137
2138Function @code{asdf:implementation-type} (exported since 2.014.14)
2139gives us the name of the subdirectory.
2140All that's left is to define how to calculate the pathname
2141of an @code{unportable-cl-source-file}.
2142
2143@lisp
2144(defmethod component-pathname ((component unportable-cl-source-file))
2145  (merge-pathnames*
2146   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
2147   (call-next-method)))
2148@end lisp
2149
2150The new component type is used in a @code{defsystem} form in this way:
2151
2152@lisp
2153(defsystem :foo
2154    :components
2155    ((:file "packages")
2156     ...
2157     (:unportable-cl-source-file "threads"
2158      :depends-on ("packages" ...))
2159     ...
2160    )
2161@end lisp
2162
2163@node Functions,  , Components, The object model of ASDF
2164@comment  node-name,  next,  previous,  up
2165@section Functions
2166@findex version-satisfies
2167
2168@deffn version-satisfies @var{version} @var{version-spec}
2169Does @var{version} satisfy the @var{version-spec}.  A generic function.
2170ASDF provides built-in methods for @var{version} being a
2171@code{component} or @code{string}.  @var{version-spec} should be a
2172string.
2173
2174In the wild, we typically see version numbering only on components of
2175type @code{system}.
2176
2177For more information about how @code{version-satisfies} interprets
2178version strings and specifications, @pxref{The defsystem grammar} and
2179@ref{Common attributes of components}.
2180@end deffn
2181
2182@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
2183@comment  node-name,  next,  previous,  up
2184@chapter Controlling where ASDF searches for systems
2185
2186@section Configurations
2187
2188Configurations specify paths where to find system files.
2189
2190@enumerate
2191
2192@item
2193The search registry may use some hardcoded wrapping registry specification.
2194This allows some implementations (notably SBCL) to specify where to find
2195some special implementation-provided systems that
2196need to precisely match the version of the implementation itself.
2197
2198@item
2199An application may explicitly initialize the source-registry configuration
2200using the configuration API
2201(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
2202in which case this takes precedence.
2203It may itself compute this configuration from the command-line,
2204from a script, from its own configuration file, etc.
2205
2206@item
2207The source registry will be configured from
2208the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
2209
2210@item
2211The source registry will be configured from
2212user configuration file
2213@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
2214(which defaults to
2215@file{~/.config/common-lisp/source-registry.conf})
2216if it exists.
2217
2218@item
2219The source registry will be configured from
2220user configuration directory
2221@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
2222(which defaults to
2223@file{~/.config/common-lisp/source-registry.conf.d/})
2224if it exists.
2225
2226@item
2227The source registry will be configured from
2228system configuration file
2229@file{/etc/common-lisp/source-registry.conf}
2230if it exists/
2231
2232@item
2233The source registry will be configured from
2234system configuration directory
2235@file{/etc/common-lisp/source-registry.conf.d/}
2236if it exists.
2237
2238@item
2239The source registry will be configured from a default configuration.
2240This configuration may allow for implementation-specific systems
2241to be found, for systems to be found the current directory
2242(at the time that the configuration is initialized) as well as
2243@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
2244@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
2245For instance, SBCL will include directories for its contribs
2246when it can find them; it will look for them where SBCL was installed,
2247or at the location specified by the @code{SBCL_HOME} environment variable.
2248
2249@end enumerate
2250
2251Each of these configurations is specified as an s-expression
2252in a trivial domain-specific language (defined below).
2253Additionally, a more shell-friendly syntax is available
2254for the environment variable (defined yet below).
2255
2256Each of these configurations is only used if the previous
2257configuration explicitly or implicitly specifies that it
2258includes its inherited configuration.
2259
2260Additionally, some implementation-specific directories
2261may be automatically prepended to whatever directories are specified
2262in configuration files, no matter if the last one inherits or not.
2263
2264
2265@section Truenames and other dangers
2266
2267One great innovation of the original ASDF was its ability to leverage
2268@code{CL:TRUENAME} to locate where your source code was and where to build it,
2269allowing for symlink farms as a simple but effective configuration mechanism
2270that is easy to control programmatically.
2271ASDF 3 still supports this configuration style, and it is enabled by default;
2272however we recommend you instead use
2273our source-registry configuration mechanism described below,
2274because it is easier to setup in a portable way across users and implementations.
2275
2276Addtionally, some people dislike truename,
2277either because it is very slow on their system, or
2278because they are using content-addressed storage where the truename of a file
2279is related to a digest of its individual contents,
2280and not to other files in the same intended project.
2281For these people, ASDF 3 allows to eschew the @code{TRUENAME} mechanism,
2282by setting the variable @var{asdf:*resolve-symlinks*} to @code{NIL}.
2283
2284PS: Yes, if you haven't read Vernor Vinge's short but great classic
2285``True Names... and Other Dangers'' then you're in for a treat.
2286
2287
2288@section XDG base directory
2289
2290Note that we purport to respect the XDG base directory specification
2291as to where configuration files are located,
2292where data files are located,
2293where output file caches are located.
2294Mentions of XDG variables refer to that document.
2295
2296@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
2297
2298This specification allows the user to specify some environment variables
2299to customize how applications behave to his preferences.
2300
2301On Windows platforms, when not using Cygwin,
2302instead of the XDG base directory specification,
2303we try to use folder configuration from the registry regarding
2304@code{Common AppData} and similar directories.
2305Since support for querying the Windows registry
2306is not possible to do in reasonable amounts of portable Common Lisp code,
2307ASDF 3 relies on the environment variables that Windows usually exports.
2308
2309@section Backward Compatibility
2310
2311For backward compatibility as well as to provide a practical backdoor for hackers,
2312ASDF will first search for @code{.asd} files in the directories specified in
2313@code{asdf:*central-registry*}
2314before it searches in the source registry above.
2315
2316@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
2317
2318By default, @code{asdf:*central-registry*} will be empty.
2319
2320This old mechanism will therefore not affect you if you don't use it,
2321but will take precedence over the new mechanism if you do use it.
2322
2323@section Configuration DSL
2324
2325Here is the grammar of the s-expression (SEXP) DSL for source-registry
2326configuration:
2327
2328@c FIXME: This is too wide for happy compilation into pdf.
2329
2330@example
2331;; A configuration is a single SEXP starting with keyword :source-registry
2332;; followed by a list of directives.
2333CONFIGURATION := (:source-registry DIRECTIVE ...)
2334
2335;; A directive is one of the following:
2336DIRECTIVE :=
2337    ;; INHERITANCE DIRECTIVE:
2338    ;; Your configuration expression MUST contain
2339    ;; exactly one of either of these:
2340    :inherit-configuration | ; splices inherited configuration (often specified last)
2341    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2342
2343    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2344    ;; you want to use new configuration features but have to bootstrap a
2345    ;; the newer required ASDF from an older release that doesn't sport said features:
2346    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2347
2348    ;; add a single directory to be scanned (no recursion)
2349    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
2350
2351    ;; add a directory hierarchy, recursing but excluding specified patterns
2352    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
2353
2354    ;; override the defaults for exclusion patterns
2355    (:exclude EXCLUSION-PATTERN ...) |
2356    ;; augment the defaults for exclusion patterns
2357    (:also-exclude EXCLUSION-PATTERN ...) |
2358    ;; Note that the scope of a an exclude pattern specification is
2359    ;; the rest of the current configuration expression or file.
2360
2361    ;; splice the parsed contents of another config file
2362    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
2363
2364    ;; This directive specifies that some default must be spliced.
2365    :default-registry
2366
2367REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
2368DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2369
2370PATHNAME-DESIGNATOR :=
2371    NIL | ;; Special: skip this entry.
2372    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
2373
2374EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2375  against the name of a any subdirectory in the directory component
2376        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2377@end example
2378
2379Pathnames are designated using another DSL,
2380shared with the output-translations configuration DSL below.
2381The DSL is resolved by the function @code{asdf::resolve-location},
2382to be documented and exported at some point in the future.
2383
2384@example
2385ABSOLUTE-COMPONENT-DESIGNATOR :=
2386    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2387    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2388             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2389             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2390             ;; Note that none of the above applies to strings used in *central-registry*,
2391             ;; which doesn't use this DSL: they are processed as normal namestrings.
2392             ;; however, you can compute what you put in the *central-registry*
2393             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2394    PATHNAME | ;; pathname (better be an absolute path, or bust)
2395               ;; In output-translations, unless followed by relative components,
2396               ;; it better have appropriate wildcards, as in **/*.*.*
2397    :HOME | ;; designates the user-homedir-pathname ~/
2398    :USER-CACHE | ;; designates the default location for the user cache
2399    :HERE | ;; designates the location of the configuration file
2400            ;; (or *default-pathname-defaults*, if invoked interactively)
2401    :ROOT ;; magic, for output-translations source only: paths that are relative
2402          ;; to the root of the source host and device
2403    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2404
2405RELATIVE-COMPONENT-DESIGNATOR :=
2406    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2407    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
2408             ;; In output translations, if last component, **/*.*.* is added
2409    PATHNAME | ;; pathname; unless last component, directory is assumed.
2410    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
2411    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2412    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2413    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2414    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2415    :*.*.* | ;; any file (since ASDF 2.011.4)
2416    ;; Not supported (anymore): :UID and :USERNAME
2417@end example
2418
2419For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2420which is the default place ASDF looks for this configuration, once contained:
2421@example
2422(:source-registry
2423  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2424  :inherit-configuration)
2425@end example
2426
2427@section Configuration Directories
2428
2429Configuration directories consist in files each containing
2430a list of directives without any enclosing @code{(:source-registry ...)} form.
2431The files will be sorted by namestring as if by @code{string<} and
2432the lists of directives of these files with be concatenated in order.
2433An implicit @code{:inherit-configuration} will be included
2434at the @emph{end} of the list.
2435
2436This allows for packaging software that has file granularity
2437(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2438to easily include configuration information about distributed software.
2439
2440The convention is that, for sorting purposes,
2441the names of files in such a directory begin with two digits
2442that determine the order in which these entries will be read.
2443Also, the type of these files is conventionally @code{"conf"}
2444and as a limitation to some implementations (e.g. GNU clisp),
2445the type cannot be @code{NIL}.
2446
2447Directories may be included by specifying a directory pathname
2448or namestring in an @code{:include} directive, e.g.:
2449
2450@example
2451  (:include "/foo/bar/")
2452@end example
2453
2454Hence, to achieve the same effect as
2455my example @file{~/.config/common-lisp/source-registry.conf} above,
2456I could simply create a file
2457@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2458alone in its directory with the following contents:
2459@example
2460(:tree "/home/fare/cl/")
2461@end example
2462
2463@subsection The :here directive
2464
2465The @code{:here} directive is an absolute pathname designator that
2466refers to the directory containing the configuration file currently
2467being processed.
2468
2469The @code{:here} directive is intended to simplify the delivery of
2470complex CL systems, and for easy configuration of projects shared through
2471revision control systems, in accordance with our design principle that
2472each participant should be able to provide all and only the information
2473available to him or her.
2474
2475Consider a person X who has set up the source code repository for a
2476complex project with a master directory @file{dir/}.  Ordinarily, one
2477might simply have the user add a directive that would look something
2478like this:
2479@example
2480   (:tree  "path/to/dir")
2481@end example
2482But what if X knows that there are very large subtrees
2483under dir that are filled with, e.g., Java source code, image files for
2484icons, etc.?  All of the asdf system definitions are contained in the
2485subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2486these are the only directories that should be searched.
2487
2488In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2489contains the following:
2490@example
2491(:source-registry
2492   (:tree (:here "src/lisp/"))
2493   (:tree (:here "extlib/lisp"))
2494   (:directory (:here "outlier/")))
2495@end example
2496
2497Then when someone else (call her Y) checks out a copy of this
2498repository, she need only add
2499@example
2500(:include "/path/to/my/checkout/directory/asdf.conf")
2501@end example
2502to one of her previously-existing asdf source location configuration
2503files, or invoke @code{initialize-source-registry} with a configuration
2504form containing that s-expression.  ASDF will find the .conf file that X
2505has provided, and then set up source locations within the working
2506directory according to X's (relative) instructions.
2507
2508@section Shell-friendly syntax for configuration
2509
2510When considering environment variable @code{CL_SOURCE_REGISTRY}
2511ASDF will skip to next configuration if it's an empty string.
2512It will @code{READ} the string as a SEXP in the DSL
2513if it begins with a paren @code{(}
2514and it will be interpreted much like @code{TEXINPUTS}
2515list of paths, where
2516
2517  * paths are separated
2518   by a @code{:} (colon) on Unix platforms (including cygwin),
2519   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2520
2521  * each entry is a directory to add to the search path.
2522
2523  * if the entry ends with a double slash @code{//}
2524    then it instead indicates a tree in the subdirectories
2525    of which to recurse.
2526
2527  * if the entry is the empty string (which may only appear once),
2528    then it indicates that the inherited configuration should be
2529    spliced there.
2530
2531
2532@section Search Algorithm
2533@vindex *default-source-registry-exclusions*
2534
2535In case that isn't clear, the semantics of the configuration is that
2536when searching for a system of a given name,
2537directives are processed in order.
2538
2539When looking in a directory, if the system is found, the search succeeds,
2540otherwise it continues.
2541
2542When looking in a tree, if one system is found, the search succeeds.
2543If multiple systems are found, the consequences are unspecified:
2544the search may succeed with any of the found systems,
2545or an error may be raised.
2546ASDF currently returns the first system found,
2547XCVB currently raised an error.
2548If none is found, the search continues.
2549
2550Exclude statements specify patterns of subdirectories
2551the systems from which to ignore.
2552Typically you don't want to use copies of files kept by such
2553version control systems as Darcs.
2554Exclude statements are not propagated to further included or inherited
2555configuration files or expressions;
2556instead the defaults are reset around every configuration statement
2557to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2558
2559Include statements cause the search to recurse with the path specifications
2560from the file specified.
2561
2562An inherit-configuration statement cause the search to recurse with the path
2563specifications from the next configuration
2564(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2565
2566
2567@section Caching Results
2568
2569The implementation is allowed to either eagerly compute the information
2570from the configurations and file system, or to lazily re-compute it
2571every time, or to cache any part of it as it goes.
2572To explicitly flush any information cached by the system, use the API below.
2573
2574
2575@section Configuration API
2576
2577The specified functions are exported from your build system's package.
2578Thus for ASDF the corresponding functions are in package ASDF,
2579and for XCVB the corresponding functions are in package XCVB.
2580
2581@defun initialize-source-registry @&optional PARAMETER
2582   will read the configuration and initialize all internal variables.
2583   You may extend or override configuration
2584   from the environment and configuration files
2585   with the given @var{PARAMETER}, which can be
2586   @code{NIL} (no configuration override),
2587   or a SEXP (in the SEXP DSL),
2588   a string (as in the string DSL),
2589   a pathname (of a file or directory with configuration),
2590   or a symbol (fbound to function that when called returns one of the above).
2591@end defun
2592
2593@defun clear-source-registry
2594   undoes any source registry configuration
2595   and clears any cache for the search algorithm.
2596   You might want to call this function
2597   (or better, @code{clear-configuration})
2598   before you dump an image that would be resumed
2599   with a different configuration,
2600   and return an empty configuration.
2601   Note that this does not include clearing information about
2602   systems defined in the current image, only about
2603   where to look for systems not yet defined.
2604@end defun
2605
2606@defun ensure-source-registry @&optional PARAMETER
2607   checks whether a source registry has been initialized.
2608   If not, initialize it with the given @var{PARAMETER}.
2609@end defun
2610
2611Every time you use ASDF's @code{find-system}, or
2612anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2613@code{ensure-source-registry} is called with parameter NIL,
2614which the first time around causes your configuration to be read.
2615If you change a configuration file,
2616you need to explicitly @code{initialize-source-registry} again,
2617or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2618which will cause the initialization to happen next time around.
2619
2620
2621@section Status
2622
2623This mechanism is vastly successful, and we have declared
2624that @code{asdf:*central-registry*} is not recommended anymore,
2625though we will continue to support it.
2626All hooks into implementation-specific search mechanisms
2627have been integrated in the @code{wrapping-source-registry}
2628that everyone uses implicitly.
2629
2630
2631@section Rejected ideas
2632
2633Alternatives I considered and rejected included:
2634
2635@enumerate
2636@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2637   and somehow the configuration parser expands the new configuration
2638   language into a expanded series of directories of subdirectories to
2639   lookup, pre-recursing through specified hierarchies. This is kludgy,
2640   and leaves little space of future cleanups and extensions.
2641
2642@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2643   in completely new ways, so that new kinds of entries may be implemented
2644   as a recursive search, etc. This seems somewhat backwards.
2645
2646@item Completely remove @code{asdf:*central-registry*}
2647   and break backwards compatibility.
2648   Hopefully this will happen in a few years after everyone migrate to
2649   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2650
2651@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2652   when you dereference it or setf it. Only the new variable with new
2653   semantics is handled by the new search procedure.
2654   Complex and still introduces subtle semantic issues.
2655@end enumerate
2656
2657
2658I've been suggested the below features, but have rejected them,
2659for the sake of keeping ASDF no more complex than strictly necessary.
2660
2661@itemize
2662@item
2663  More syntactic sugar: synonyms for the configuration directives, such as
2664  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2665  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2666
2667@item
2668   The possibility to register individual files instead of directories.
2669
2670@item
2671  Integrate Xach Beane's tilde expander into the parser,
2672  or something similar that is shell-friendly or shell-compatible.
2673  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2674  minimal by removing the need for evaluated entries that ASDF has?
2675  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2676  Hopefully, these are already superseded by the @code{:default-registry}
2677
2678@item
2679  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2680  down a filesystem tree in the environment variable.
2681  It isn't that Lisp friendly either.
2682@end itemize
2683
2684@section TODO
2685
2686@itemize
2687@item Add examples
2688@end itemize
2689
2690
2691@section Credits for the source-registry
2692
2693Thanks a lot to Stelian Ionescu for the initial idea.
2694
2695Thanks to Rommel Martinez for the initial implementation attempt.
2696
2697All bad design ideas and implementation bugs are to mine, not theirs.
2698But so are good design ideas and elegant implementation tricks.
2699
2700 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2701
2702
2703
2704@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2705@comment  node-name,  next,  previous,  up
2706@chapter Controlling where ASDF saves compiled files
2707@cindex asdf-output-translations
2708@vindex ASDF_OUTPUT_TRANSLATIONS
2709
2710Each Common Lisp implementation has its own format
2711for compiled files (fasls for short, short for ``fast loading'').
2712If you use multiple implementations
2713(or multiple versions of the same implementation),
2714you'll soon find your source directories
2715littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2716Worse yet, some implementations use the same file extension
2717while changing formats from version to version (or platform to platform)
2718which means that you'll have to recompile binaries
2719as you switch from one implementation to the next.
2720
2721Since ASDF 2, ASDF includes the @code{asdf-output-translations} facility
2722to mitigate the problem.
2723
2724@section Configurations
2725
2726Configurations specify mappings from input locations to output locations.
2727Once again we rely on the XDG base directory specification for configuration.
2728@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2729
2730@enumerate
2731
2732@item
2733Some hardcoded wrapping output translations configuration may be used.
2734This allows special output translations (or usually, invariant directories)
2735to be specified corresponding to the similar special entries in the source registry.
2736
2737@item
2738An application may explicitly initialize the output-translations
2739configuration using the Configuration API
2740in which case this takes precedence.
2741(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2742It may itself compute this configuration from the command-line,
2743from a script, from its own configuration file, etc.
2744
2745@item
2746The source registry will be configured from
2747the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2748
2749@item
2750The source registry will be configured from
2751user configuration file
2752@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2753(which defaults to
2754@file{~/.config/common-lisp/asdf-output-translations.conf})
2755if it exists.
2756
2757@item
2758The source registry will be configured from
2759user configuration directory
2760@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2761(which defaults to
2762@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2763if it exists.
2764
2765@item
2766The source registry will be configured from
2767system configuration file
2768@file{/etc/common-lisp/asdf-output-translations.conf}
2769if it exists.
2770
2771@item
2772The source registry will be configured from
2773system configuration directory
2774@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2775if it exists.
2776
2777@end enumerate
2778
2779Each of these configurations is specified as a SEXP
2780in a trival domain-specific language (defined below).
2781Additionally, a more shell-friendly syntax is available
2782for the environment variable (defined yet below).
2783
2784Each of these configurations is only used if the previous
2785configuration explicitly or implicitly specifies that it
2786includes its inherited configuration.
2787
2788Note that by default, a per-user cache is used for output files.
2789This allows the seamless use of shared installations of software
2790between several users, and takes files out of the way of the developers
2791when they browse source code,
2792at the expense of taking a small toll when developers have to clean up
2793output files and find they need to get familiar with output-translations first.
2794
2795
2796@section Backward Compatibility
2797@cindex ASDF-BINARY-LOCATIONS compatibility
2798
2799
2800We purposefully do NOT provide backward compatibility with earlier versions of
2801@code{ASDF-Binary-Locations} (8 Sept 2009),
2802@code{common-lisp-controller} (7.0) or
2803@code{cl-launch} (2.35),
2804each of which had similar general capabilities.
2805The previous APIs of these programs were not designed
2806for configuration by the end-user
2807in an easy way with configuration files.
2808Recent versions of same packages use
2809the new @code{asdf-output-translations} API as defined below:
2810@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2811@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2812
2813This incompatibility shouldn't inconvenience many people.
2814Indeed, few people use and customize these packages;
2815these few people are experts who can trivially adapt to the new configuration.
2816Most people are not experts, could not properly configure these features
2817(except inasmuch as the default configuration of
2818@code{common-lisp-controller} and/or @code{cl-launch}
2819might have been doing the right thing for some users),
2820and yet will experience software that ``just works'',
2821as configured by the system distributor, or by default.
2822
2823Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2824we provide a limited emulation mode:
2825
2826@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2827This function will initialize the new @code{asdf-output-translations} facility in a way
2828that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2829Where you would previously set global variables
2830@var{*centralize-lisp-binaries*},
2831@var{*default-toplevel-directory*},
2832@var{*include-per-user-information*},
2833@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2834you will now have to pass the same values as keyword arguments to this function.
2835Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2836will accept any valid pathname designator for @code{asdf-output-translations}
2837instead of just strings and pathnames.
2838@end defun
2839
2840If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2841(the one available as an extension to load of top of ASDF,
2842not the one built into a few old versions of ASDF),
2843but first you must disable @code{asdf-output-translations}
2844with @code{(asdf:disable-output-translations)},
2845or you might experience ``interesting'' issues.
2846
2847Also, note that output translation is enabled by default.
2848To disable it, use @code{(asdf:disable-output-translations)}.
2849
2850
2851@section Configuration DSL
2852
2853Here is the grammar of the SEXP DSL
2854for @code{asdf-output-translations} configuration:
2855
2856@verbatim
2857;; A configuration is single SEXP starting with keyword :source-registry
2858;; followed by a list of directives.
2859CONFIGURATION := (:output-translations DIRECTIVE ...)
2860
2861;; A directive is one of the following:
2862DIRECTIVE :=
2863    ;; INHERITANCE DIRECTIVE:
2864    ;; Your configuration expression MUST contain
2865    ;; exactly one of either of these:
2866    :inherit-configuration | ; splices inherited configuration (often specified last)
2867    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2868
2869    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2870    ;; you want to use new configuration features but have to bootstrap a
2871    ;; the newer required ASDF from an older release that doesn't sport said features:
2872    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2873
2874    ;; include a configuration file or directory
2875    (:include PATHNAME-DESIGNATOR) |
2876
2877    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2878    :enable-user-cache |
2879    ;; Disable global cache. Map / to /
2880    :disable-cache |
2881
2882    ;; add a single directory to be scanned (no recursion)
2883    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2884
2885    ;; use a function to return the translation of a directory designator
2886    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2887
2888DIRECTORY-DESIGNATOR :=
2889    NIL | ;; As source: skip this entry. As destination: same as source
2890    T | ;; as source matches anything, as destination leaves pathname unmapped.
2891    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2892
2893TRANSLATION-FUNCTION :=
2894    SYMBOL | ;; symbol of a function that takes two arguments,
2895             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2896    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2897             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2898
2899@end verbatim
2900
2901Relative components better be either relative
2902or subdirectories of the path before them, or bust.
2903
2904The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2905You can specify more fine-grained patterns
2906by using a pathname object as the last component
2907e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2908
2909You may use @code{#+features} to customize the configuration file.
2910
2911The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2912to anything but themselves (same as if the second designator was the same as the first).
2913
2914When the first designator is @code{t},
2915the mapping always matches.
2916When the first designator starts with @code{:root},
2917the mapping matches any host and device.
2918In either of these cases, if the second designator
2919isn't @code{t} and doesn't start with @code{:root},
2920then strings indicating the host and pathname are somehow copied
2921in the beginning of the directory component of the source pathname
2922before it is translated.
2923
2924When the second designator is @code{t}, the mapping is the identity.
2925When the second designator starts with @code{:root},
2926the mapping preserves the host and device of the original pathname.
2927Notably, this allows you to map files
2928to a subdirectory of the whichever directory the file is in.
2929Though the syntax is not quite as easy to use as we'd like,
2930you can have an (source destination) mapping entry such as follows
2931in your configuration file,
2932or you may use @code{enable-asdf-binary-locations-compatibility}
2933with @code{:centralize-lisp-binaries nil}
2934which will do the same thing internally for you:
2935@verbatim
2936  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2937          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2938     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2939       (:root ,wild-subdir :implementation ,wild-file)))
2940@end verbatim
2941Starting with ASDF 2.011.4, you can use the simpler:
2942  @code{`(:root (:root :**/ :implementation :*.*.*))}
2943
2944
2945
2946@code{:include} statements cause the search to recurse with the path specifications
2947from the file specified.
2948
2949If the @code{translate-pathname} mechanism cannot achieve a desired
2950translation, the user may provide a function which provides the
2951required algorithim.  Such a translation function is specified by
2952supplying a list as the second @code{directory-designator}
2953the first element of which is the keyword @code{:function},
2954and the second element of which is
2955either a symbol which designates a function or a lambda expression.
2956The function designated by the second argument must take two arguments,
2957the first being the pathname of the source file,
2958the second being the wildcard that was matched.
2959The result of the function invocation should be the translated pathname.
2960
2961An @code{:inherit-configuration} statement cause the search to recurse with the path
2962specifications from the next configuration.
2963@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2964
2965@itemize
2966@item
2967@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2968@item
2969@code{:disable-cache} is the same as @code{(t t)}.
2970@item
2971@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2972which by default is the same as using
2973@code{(:home ".cache" "common-lisp" :implementation)}.
2974@item
2975@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2976which by default is the same as using
2977@code{("/var/cache/common-lisp" :uid :implementation-type)}
2978(on Unix and cygwin), or something semi-sensible on Windows.
2979@end itemize
2980
2981
2982@section Configuration Directories
2983
2984Configuration directories consist in files each contains
2985a list of directives without any enclosing
2986@code{(:output-translations ...)} form.
2987The files will be sorted by namestring as if by @code{string<} and
2988the lists of directives of these files with be concatenated in order.
2989An implicit @code{:inherit-configuration} will be included
2990at the @emph{end} of the list.
2991
2992This allows for packaging software that has file granularity
2993(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2994to easily include configuration information about software being distributed.
2995
2996The convention is that, for sorting purposes,
2997the names of files in such a directory begin with two digits
2998that determine the order in which these entries will be read.
2999Also, the type of these files is conventionally @code{"conf"}
3000and as a limitation of some implementations, the type cannot be @code{NIL}.
3001
3002Directories may be included by specifying a directory pathname
3003or namestring in an @code{:include} directive, e.g.:
3004@verbatim
3005  (:include "/foo/bar/")
3006@end verbatim
3007
3008@section Shell-friendly syntax for configuration
3009
3010When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
3011ASDF will skip to next configuration if it's an empty string.
3012It will @code{READ} the string as an SEXP in the DSL
3013if it begins with a paren @code{(}
3014and it will be interpreted as a list of directories.
3015Directories should come by pairs, indicating a mapping directive.
3016Entries are separated
3017by a @code{:} (colon) on Unix platforms (including cygwin),
3018by a @code{;} (semicolon) on other platforms (mainly, Windows).
3019
3020The magic empty entry,
3021if it comes in what would otherwise be the first entry in a pair,
3022indicates the splicing of inherited configuration.
3023If it comes as the second entry in a pair,
3024it indicates that the directory specified first is to be left untranslated
3025(which has the same effect as if the directory had been repeated).
3026
3027
3028@section Semantics of Output Translations
3029
3030From the specified configuration,
3031a list of mappings is extracted in a straightforward way:
3032mappings are collected in order, recursing through
3033included or inherited configuration as specified.
3034To this list is prepended some implementation-specific mappings,
3035and is appended a global default.
3036
3037The list is then compiled to a mapping table as follows:
3038for each entry, in order, resolve the first designated directory
3039into an actual directory pathname for source locations.
3040If no mapping was specified yet for that location,
3041resolve the second designated directory to an output location directory
3042add a mapping to the table mapping the source location to the output location,
3043and add another mapping from the output location to itself
3044(unless a mapping already exists for the output location).
3045
3046Based on the table, a mapping function is defined,
3047mapping source pathnames to output pathnames:
3048given a source pathname, locate the longest matching prefix
3049in the source column of the mapping table.
3050Replace that prefix by the corresponding output column
3051in the same row of the table, and return the result.
3052If no match is found, return the source pathname.
3053(A global default mapping the filesystem root to itself
3054may ensure that there will always be a match,
3055with same fall-through semantics).
3056
3057@section Caching Results
3058
3059The implementation is allowed to either eagerly compute the information
3060from the configurations and file system, or to lazily re-compute it
3061every time, or to cache any part of it as it goes.
3062To explicitly flush any information cached by the system, use the API below.
3063
3064
3065@section Output location API
3066
3067The specified functions are exported from package ASDF.
3068
3069@defun initialize-output-translations @&optional PARAMETER
3070   will read the configuration and initialize all internal variables.
3071   You may extend or override configuration
3072   from the environment and configuration files
3073   with the given @var{PARAMETER}, which can be
3074   @code{NIL} (no configuration override),
3075   or a SEXP (in the SEXP DSL),
3076   a string (as in the string DSL),
3077   a pathname (of a file or directory with configuration),
3078   or a symbol (fbound to function that when called returns one of the above).
3079@end defun
3080
3081@defun disable-output-translations
3082   will initialize output translations in a way
3083   that maps every pathname to itself,
3084   effectively disabling the output translation facility.
3085@end defun
3086
3087@defun clear-output-translations
3088   undoes any output translation configuration
3089   and clears any cache for the mapping algorithm.
3090   You might want to call this function
3091   (or better, @code{clear-configuration})
3092   before you dump an image that would be resumed
3093   with a different configuration,
3094   and return an empty configuration.
3095   Note that this does not include clearing information about
3096   systems defined in the current image, only about
3097   where to look for systems not yet defined.
3098@end defun
3099
3100@defun ensure-output-translations @&optional PARAMETER
3101   checks whether output translations have been initialized.
3102   If not, initialize them with the given @var{PARAMETER}.
3103   This function will be called before any attempt to operate on a system.
3104@end defun
3105
3106@defun apply-output-translations PATHNAME
3107   Applies the configured output location translations to @var{PATHNAME}
3108   (calls @code{ensure-output-translations} for the translations).
3109@end defun
3110
3111Every time you use ASDF's @code{output-files}, or
3112anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
3113@code{ensure-output-translations} is called with parameter NIL,
3114which the first time around causes your configuration to be read.
3115If you change a configuration file,
3116you need to explicitly @code{initialize-output-translations} again,
3117or maybe @code{clear-output-translations} (or @code{clear-configuration}),
3118which will cause the initialization to happen next time around.
3119
3120
3121@section Credits for output translations
3122
3123Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
3124and to Peter van Eynde for @code{Common Lisp Controller}.
3125
3126All bad design ideas and implementation bugs are to mine, not theirs.
3127But so are good design ideas and elegant implementation tricks.
3128
3129 --- Francois-Rene Rideau @email{fare@@tunes.org}
3130
3131@c @section Default locations
3132@c @findex output-files-for-system-and-operation
3133
3134@c The default binary location for each Lisp implementation
3135@c is a subdirectory of each source directory.
3136@c To account for different Lisps, Operating Systems, Implementation versions,
3137@c and so on, ASDF borrows code from SLIME
3138@c to create reasonable custom directory names.
3139@c Here are some examples:
3140
3141@c @itemize
3142@c @item
3143@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
3144
3145@c @item
3146@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
3147
3148@c @item
3149@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
3150@c @end itemize
3151
3152@c By default, all output file pathnames will be relocated
3153@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
3154
3155@c See the document @file{README.asdf-output-translations}
3156@c for a full specification on how to configure @code{asdf-output-translations}.
3157
3158@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
3159@comment  node-name,  next,  previous,  up
3160@chapter Error handling
3161@findex SYSTEM-DEFINITION-ERROR
3162@findex OPERATION-ERROR
3163
3164@section ASDF errors
3165
3166If ASDF detects an incorrect system definition, it will signal a generalised instance of
3167@code{SYSTEM-DEFINITION-ERROR}.
3168
3169Operations may go wrong (for example when source files contain errors).
3170These are signalled using generalised instances of
3171@code{OPERATION-ERROR}.
3172
3173@section Compilation error and warning handling
3174@vindex *compile-file-warnings-behaviour*
3175@vindex *compile-file-errors-behavior*
3176
3177ASDF checks for warnings and errors when a file is compiled.
3178The variables @var{*compile-file-warnings-behaviour*} and
3179@var{*compile-file-errors-behavior*}
3180control the handling of any such events.
3181The valid values for these variables are
3182@code{:error}, @code{:warn}, and @code{:ignore}.
3183
3184@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
3185@comment  node-name,  next,  previous,  up
3186@chapter Miscellaneous additional functionality
3187
3188ASDF includes several additional features that are generally
3189useful for system definition and development.
3190
3191@section Controlling file compilation
3192
3193When declaring a component (system, module, file),
3194you can specify a keyword argument @code{:around-compile function}.
3195If left unspecified (and therefore unbound),
3196the value will be inherited from the parent component if any,
3197or with a default of @code{nil}
3198if no value is specified in any transitive parent.
3199
3200The argument must be a either @code{nil}, a fbound symbol,
3201a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
3202a function object (e.g. using @code{#.#'} but that's discouraged
3203because it prevents the introspection done by e.g. asdf-dependency-grovel),
3204or a string that when @code{read} yields a symbol or a lambda-expression.
3205@code{nil} means the normal compile-file function will be called.
3206A non-nil value designates a function of one argument
3207that will be called with a function that will
3208invoke @code{compile-file*} with various arguments;
3209the around-compile hook may supply additional keyword arguments
3210to pass to that call to @code{compile-file*}.
3211
3212One notable argument that is heeded by @code{compile-file*} is
3213@code{:compile-check},
3214a function called when the compilation was otherwise a success,
3215with the same arguments as @code{compile-file};
3216the function shall return true if the compilation
3217and its resulting compiled file respected all system-specific invariants,
3218and false (@code{nil}) if it broke any of those invariants;
3219it may issue warnings or errors before it returns @code{nil}.
3220(NB: The ability to pass such extra flags
3221is only available starting with ASDF 2.22.3.)
3222This feature is notably exercised by asdf-finalizers.
3223
3224By using a string, you may reference
3225a function, symbol and/or package
3226that will only be created later during the build, but
3227isn't yet present at the time the defsystem form is evaluated.
3228However, if your entire system is using such a hook, you may have to
3229explicitly override the hook with @code{nil} for all the modules and files
3230that are compiled before the hook is defined.
3231
3232Using this hook, you may achieve such effects as:
3233locally renaming packages,
3234binding @var{*readtables*} and other syntax-controlling variables,
3235handling warnings and other conditions,
3236proclaiming consistent optimization settings,
3237saving code coverage information,
3238maintaining meta-data about compilation timings,
3239setting gensym counters and PRNG seeds and other sources of non-determinism,
3240overriding the source-location and/or timestamping systems,
3241checking that some compile-time side-effects were properly balanced,
3242etc.
3243
3244Note that there is no around-load hook. This is on purpose.
3245Some implementations such as ECL, GCL or MKCL link object files,
3246which allows for no such hook.
3247Other implementations allow for concatenating FASL files,
3248which doesn't allow for such a hook either.
3249We aim to discourage something that's not portable,
3250and has some dubious impact on performance and semantics
3251even when it is possible.
3252Things you might want to do with an around-load hook
3253are better done around-compile,
3254though it may at times require some creativity
3255(see e.g. the @code{package-renaming} system).
3256
3257
3258@section Controlling source file character encoding
3259
3260Starting with ASDF 2.21, components accept a @code{:encoding} option
3261so authors may specify which character encoding should be used
3262to read and evaluate their source code.
3263When left unspecified, the encoding is inherited
3264from the parent module or system;
3265if no encoding is specified at any point,
3266the default @code{:autodetect} is assumed.
3267By default, only @code{:default}, @code{:utf-8}
3268and @code{:autodetect} are accepted.
3269@code{:autodetect}, the default, calls
3270@code{*encoding-detection-hook*} which by default always returns
3271@code{*default-encoding*} which itself defaults to @code{:default}.
3272
3273In other words, there now are plenty of extension hooks, but
3274by default ASDF follows the backwards compatible behavior
3275of using whichever @code{:default} encoding your implementation uses,
3276which itself may or may not vary based on environment variables
3277and other locale settings.
3278In practice this means that only source code that only uses ASCII
3279is guaranteed to be read the same on all implementations
3280independently from any user setting.
3281
3282Additionally, for backward-compatibility with older versions of ASDF
3283and/or with implementations that do not support unicode and its many encodings,
3284you may want to use
3285the reader conditionals @code{#+asdf-unicode #+asdf-unicode}
3286to protect any @code{:encoding @emph{encoding}} statement
3287as @code{:asdf-unicode} will be present in @code{*features*}
3288only if you're using a recent ASDF
3289on an implementation that supports unicode.
3290We recommend that you avoid using unprotected @code{:encoding} specifications
3291until after ASDF 2.21 or later becomes widespread, hopefully by the end of 2012.
3292
3293While it offers plenty of hooks for extension,
3294and one such extension is being developed (see below),
3295ASDF itself only recognizes one encoding beside @code{:default},
3296and that is @code{:utf-8}, which is the @emph{de facto} standard,
3297already used by the vast majority of libraries that use more than ASCII.
3298On implementations that do not support unicode,
3299the feature @code{:asdf-unicode} is absent, and
3300the @code{:default} external-format is used
3301to read even source files declared as @code{:utf-8}.
3302On these implementations, non-ASCII characters
3303intended to be read as one CL character
3304may thus end up being read as multiple CL characters.
3305In most cases, this shouldn't affect the software's semantics:
3306comments will be skipped just the same, strings with be read and printed
3307with slightly different lengths, symbol names will be accordingly longer,
3308but none of it should matter.
3309But a few systems that actually depend on unicode characters
3310may fail to work properly, or may work in a subtly different way.
3311See for instance @code{lambda-reader}.
3312
3313We invite you to embrace UTF-8
3314as the encoding for non-ASCII characters starting today,
3315even without any explicit specification in your @code{.asd} files.
3316Indeed, on some implementations and configurations,
3317UTF-8 is already the @code{:default},
3318and loading your code may cause errors if it is encoded in anything but UTF-8.
3319Therefore, even with the legacy behavior,
3320non-UTF-8 is guaranteed to break for some users,
3321whereas UTF-8 is pretty much guaranteed not to break anywhere
3322(provided you do @emph{not} use a BOM),
3323although it might be read incorrectly on some implementations.
3324In the future, we intend to make @code{:utf-8}
3325the default value of @code{*default-encoding*},
3326to be enforced everywhere, so at least the code is guaranteed
3327to be read correctly everywhere it can be.
3328
3329If you need non-standard character encodings for your source code,
3330use the extension system @code{asdf-encodings}, by specifying
3331@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
3332This extension system will register support for more encodings using the
3333@code{*encoding-external-format-hook*} facility,
3334so you can explicitly specify @code{:encoding :latin1}
3335in your @code{.asd} file.
3336Using the @code{*encoding-detection-hook*} it will also
3337eventually implement some autodetection of a file's encoding
3338from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
3339or otherwise based on an analysis of octet patterns in the file.
3340At this point, asdf-encoding only supports the encodings
3341that are supported as part of your implementation.
3342Since the list varies depending on implementations,
3343we once again recommend you use @code{:utf-8} everywhere,
3344which is the most portable (next is @code{:latin1}).
3345
3346If you're not using a version of Quicklisp that has it,
3347you may get the source for @code{asdf-encodings} using git:
3348@kbd{git clone git://common-lisp.net/projects/asdf/asdf-encodings.git}
3349or
3350@kbd{git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git}.
3351You can also browse the repository on
3352@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git}.
3353
3354In the future, we intend to change the default @code{*default-encoding*}
3355to @code{:utf-8}, which is already the de facto standard
3356for most libraries that use non-ASCII characters:
3357utf-8 works everywhere and was backhandedly enforced by
3358a lot of people using SBCL and utf-8 and sending reports to authors
3359so they make their packages compatible.
3360A survey showed only about a handful few libraries
3361are incompatible with non-UTF-8, and then, only in comments,
3362and we believe that authors will adopt UTF-8 when prompted.
3363See the April 2012 discussion on the asdf-devel mailing-list.
3364For backwards compatibility with users who insist on a non-UTF-8 encoding,
3365but cannot immediately transition to using @code{asdf-encodings}
3366(maybe because it isn't ready), it will still be possible to use
3367the @code{:encoding :default} option in your @code{defsystem} form
3368to restore the behavior of ASDF 2.20 and earlier.
3369This shouldn't be required in libraries,
3370because user pressure as mentioned above will already have pushed
3371library authors towards using UTF-8;
3372but authors of end-user programs might care.
3373
3374When you use @code{asdf-encodings}, any further loaded @code{.asd} file
3375will use the autodetection algorithm to determine its encoding;
3376yet if you depend on this detection happening,
3377you may want to explicitly load @code{asdf-encodings} early in your build,
3378for by the time you can use @code{:defsystem-depends-on},
3379it is already too late to load it.
3380In practice, this means that the @code{*default-encoding*}
3381is usually used for @code{.asd} files.
3382Currently, this defaults to @code{:default} for backwards compatibility,
3383and that means that you shouldn't rely on non-ASCII characters in a .asd file.
3384Since component (path)names are the only real data in these files,
3385and non-ASCII characters are not very portable for file names,
3386this isn't too much of an issue.
3387We still encourage you to use either plain ASCII or UTF-8
3388in @code{.asd} files,
3389as we intend to make @code{:utf-8} the default encoding in the future.
3390This might matter, for instance, in meta-data about author's names.
3391
3392
3393@section Miscellaneous Functions
3394
3395Most of these functions are not exported by ASDF anymore,
3396but only used for private purposes of ASDF.
3397Please use ASDF-UTILS for the same functions exported from a stable library.
3398
3399@defun coerce-pathname name @&key type defaults
3400
3401This function (available starting with ASDF 2.012.11)
3402takes an argument, and portably interprets it as a pathname.
3403If the argument @var{name} is a pathname or @code{nil}, it is passed through;
3404if it's a symbol, it's interpreted as a string by downcasing it;
3405if it's a string, it is first separated using @code{/} into substrings;
3406the leading substrings denote subdirectories of a relative pathname.
3407If @var{type} is @code{:directory} or the string ends with @code{/},
3408the last substring is also a subdirectory;
3409if @var{type} is a string, it is used as the type of the pathname, and
3410the last substring is the name component of the pathname;
3411if @var{type} is @code{nil}, the last substring specifies both name and type components
3412of the pathname, with the last @code{.} separating them, or only the name component
3413if there's no last @code{.} or if there is only one dot and it's the first character.
3414The host, device and version components come from @var{defaults}, which defaults to
3415@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
3416
3417@end defun
3418
3419@defun merge-pathnames* @&key specified defaults
3420
3421This function is a replacement for @code{merge-pathnames} that uses the host and device
3422from the @var{defaults} rather than the @var{specified} pathname when the latter
3423is a relative pathname. This allows ASDF and its users to create and use relative pathnames
3424without having to know beforehand what are the host and device
3425of the absolute pathnames they are relative to.
3426
3427@end defun
3428
3429@defun system-relative-pathname system name @&key type
3430
3431It's often handy to locate a file relative to some system.
3432The @code{system-relative-pathname} function meets this need.
3433
3434It takes two mandatory arguments @var{system} and @var{name}
3435and a keyword argument @var{type}:
3436@var{system} is name of a system, whereas @var{name} and optionally @var{type}
3437specify a relative pathname, interpreted like a component pathname specifier
3438by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
3439
3440It returns a pathname built from the location of the system's
3441source directory and the relative pathname. For example:
3442
3443@lisp
3444> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
3445#P"/repository/other/cl-ppcre/regex.data"
3446@end lisp
3447
3448@end defun
3449
3450@defun system-source-directory system-designator
3451
3452ASDF does not provide a turnkey solution for locating
3453data (or other miscellaneous) files
3454that are distributed together with the source code of a system.
3455Programmers can use @code{system-source-directory} to find such files.
3456Returns a pathname object.
3457The @var{system-designator} may be a string, symbol, or ASDF system object.
3458@end defun
3459
3460@defun clear-system system-designator
3461
3462It is sometimes useful to force recompilation of a previously loaded system.
3463In these cases, it may be useful to @code{(asdf:clear-system :foo)}
3464to remove the system from the table of currently loaded systems;
3465the next time the system @code{foo} or one that depends on it is re-loaded,
3466@code{foo} will then be loaded again.
3467Alternatively, you could touch @code{foo.asd} or
3468remove the corresponding fasls from the output file cache.
3469(It was once conceived that one should provide
3470a list of systems the recompilation of which to force
3471as the @code{:force} keyword argument to @code{load-system};
3472but this has never worked, and though the feature was fixed in ASDF 2.000,
3473it remains @code{cerror}'ed out as nobody ever used it.)
3474
3475Note that this does not and cannot by itself undo the previous loading
3476of the system. Common Lisp has no provision for such an operation,
3477and its reliance on irreversible side-effects to global datastructures
3478makes such a thing impossible in the general case.
3479If the software being re-loaded is not conceived with hot upgrade in mind,
3480this re-loading may cause many errors, warnings or subtle silent problems,
3481as packages, generic function signatures, structures, types, macros, constants, etc.
3482are being redefined incompatibly.
3483It is up to the user to make sure that reloading is possible and has the desired effect.
3484In some cases, extreme measures such as recursively deleting packages,
3485unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
3486and much more are necessary for reloading to happen smoothly.
3487ASDF itself goes through notable pains to make such a hot upgrade possible
3488with respect to its own code, and what it does is ridiculously complex;
3489look at the beginning of @file{asdf.lisp} to see what it does.
3490@end defun
3491
3492@defun run-program
3493
3494run-program takes a @var{COMMAND} argument that is either
3495a list of a program path and its arguments,
3496or a string to be executed by a shell.
3497It spawns the command, waits for it to return,
3498verifies that it exited cleanly (unless told not too below),
3499and optionally captures and processes its output.
3500It accepts many keyword arguments to configure its behavior.
3501
3502@code{output} is its most important argument;
3503it specifies how the output is captured and processed.
3504If it is @code{nil}, then the output is not captured.
3505If it is @code{:interactive}, then
3506input and output are inherited from the current process,
3507which the subprocess can control until it exits.
3508Otherwise, the output is captured and redirected to a stream,
3509and processed by @code{slurp-input-stream} with the object as first argument.
3510See below.
3511
3512@code{element-type} and @code{external-format} are passed on
3513to your Lisp implementation, when available, for creation of the output stream.
3514
3515@code{force-shell} forces evaluation of the command through a shell,
3516even if it was passed as a list rather than a string.
3517
3518@code{ignore-error-status} causes @code{run-program}
3519to not raise an error if the spawned program exits in error.
3520Following POSIX convention, an error is anything but
3521a normal exit with status code zero.
3522
3523run-program works on all platforms supported by ASDF, except Genera.
3524
3525@end defun
3526
3527@defun slurp-input-stream
3528
3529It's a generic function of two arguments, a target object and an input stream,
3530and accepting keyword arguments.
3531Predefined methods based on the target object are as follow:
3532
3533If the object is a function, the function is called with the stream as argument.
3534
3535If the object is a cons, its first element is applied to its rest appended by
3536a list of the input stream.
3537
3538If the object is an output stream, the contents of the input stream are copied to it.
3539If the linewise argument is provided, copying happens line by line,
3540and an optional prefix is printed before each line.
3541Otherwise, copying happen based on a buffer of size buffer-size,
3542using the element-type.
3543
3544If the object is 'string or :string, the content is captured into a string
3545of the given element-type.
3546
3547If the object is :lines, the content is captured as a list of strings,
3548one per line, without line ending. If the count argument is provided,
3549it is a maximum count of lines to be read.
3550
3551If the object is :line, the content is capture as with :lines above,
3552and then its sub-object is extracted with the path argument,
3553which defaults to 0, extracting the first line.
3554A number will extract the corresponding line.
3555See the documentation for asdf-driver:sub-object.
3556
3557If the object is :forms, the content is captured as a list of S-expressions,
3558as read by the Lisp reader.
3559If the count argument is provided,
3560it is a maximum count of lines to be read.
3561We recommend you control the syntax with such macro as
3562asdf-driver:with-safe-io-syntax.
3563
3564If the object is :form, the content is capture as with :forms above,
3565and then its sub-object is extracted with the path argument,
3566which defaults to 0, extracting the first form.
3567A number will extract the corresponding form.
3568See the documentation for asdf-driver:sub-object.
3569We recommend you control the syntax with such macro as
3570asdf-driver:with-safe-io-syntax.
3571
3572@end defun
3573
3574@defun run-shell-command
3575
3576This function is obsolete and present only for the sake of backwards-compatibility:
3577``If it's not backwards, it's not compatible''. We strongly discourage its use.
3578Its current behavior is only well-defined on Unix platforms
3579(which include MacOS X and cygwin). On Windows, anything goes.
3580The following documentation is only for the purpose of your migrating away from it
3581in a way that preserves semantics.
3582
3583Instead we recommend the use @code{run-program} above
3584available as part of ASDF since ASDF 3.
3585
3586@code{run-shell-command} takes as arguments a @code{format} control-string
3587and arguments to be passed to @code{format} after this control-string
3588to produce a string.
3589This string is a command that will be evaluated with a POSIX shell if possible;
3590yet, on Windows, some implementations will use CMD.EXE,
3591while others (like SBCL) will make an attempt at invoking a POSIX shell
3592(and fail if it is not present).
3593@end defun
3594
3595
3596@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
3597@comment  node-name,  next,  previous,  up
3598@chapter Getting the latest version
3599
3600Decide which version you want.
3601The @code{master} branch is where development happens;
3602its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
3603but an occasional regression may happen despite our (limited) test suite.
3604
3605The @code{release} branch is what cautious people should be using;
3606it has usually been tested more, and releases are cut at a point
3607where there isn't any known unresolved issue.
3608
3609You may get the ASDF source repository using git:
3610@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
3611
3612You will find the above referenced tags in this repository.
3613You can also browse the repository on
3614@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
3615
3616Discussion of ASDF development is conducted on the
3617mailing list
3618@kbd{asdf-devel@@common-lisp.net}.
3619@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
3620
3621
3622@node FAQ, TODO list, Getting the latest version, Top
3623@comment  node-name,  next,  previous,  up
3624@chapter FAQ
3625
3626@section  ``Where do I report a bug?''
3627
3628ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
3629
3630If you're unsure about whether something is a bug, or for general discussion,
3631use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
3632
3633
3634@section ``What has changed between ASDF 1 and ASDF 2?''
3635
3636@subsection What are ASDF 1 and ASDF 2?
3637
3638On May 31st 2010, we have released ASDF 2.
3639ASDF 2 refers to release 2.000 and later.
3640(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
3641ASDF 1 to any release earlier than 1.369 or so.
3642If your ASDF doesn't sport a version, it's an old ASDF 1.
3643
3644ASDF 2 and its release candidates push
3645@code{:asdf2} onto @code{*features*} so that if you are writing
3646ASDF-dependent code you may check for this feature
3647to see if the new API is present.
3648@emph{All} versions of ASDF should have the @code{:asdf} feature.
3649
3650Additionally, all versions of ASDF 2
3651define a function @code{(asdf:asdf-version)} you may use to query the version;
3652and the source code of recent versions of ASDF 2 features the version number
3653prominently on the second line of its source code.
3654
3655If you are experiencing problems or limitations of any sort with ASDF 1,
3656we recommend that you should upgrade to ASDF 2,
3657or whatever is the latest release.
3658
3659
3660@subsection ASDF can portably name files in subdirectories
3661
3662Common Lisp namestrings are not portable,
3663except maybe for logical pathnamestrings,
3664that themselves have various limitations and require a lot of setup
3665that is itself ultimately non-portable.
3666
3667In ASDF 1, the only portable ways to refer to pathnames inside systems and components
3668were very awkward, using @code{#.(make-pathname ...)} and
3669@code{#.(merge-pathnames ...)}.
3670Even the above were themselves were inadequate in the general case
3671due to host and device issues, unless horribly complex patterns were used.
3672Plenty of simple cases that looked portable actually weren't,
3673leading to much confusion and greavance.
3674
3675ASDF 2 implements its own portable syntax for strings as pathname specifiers.
3676Naming files within a system definition becomes easy and portable again.
3677@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
3678@code{merge-pathnames*},
3679@code{coerce-pathname}.
3680
3681On the other hand, there are places where systems used to accept namestrings
3682where you must now use an explicit pathname object:
3683@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3684must now be written with the @code{#p} syntax:
3685@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3686
3687@xref{The defsystem grammar,,Pathname specifiers}.
3688
3689
3690@subsection Output translations
3691
3692A popular feature added to ASDF was output pathname translation:
3693@code{asdf-binary-locations}, @code{common-lisp-controller},
3694@code{cl-launch} and other hacks were all implementing it in ways
3695both mutually incompatible and difficult to configure.
3696
3697Output pathname translation is essential to share
3698source directories of portable systems across multiple implementations
3699or variants thereof,
3700or source directories of shared installations of systems across multiple users,
3701or combinations of the above.
3702
3703In ASDF 2, a standard mechanism is provided for that,
3704@code{asdf-output-translations},
3705with sensible defaults, adequate configuration languages,
3706a coherent set of configuration files and hooks,
3707and support for non-Unix platforms.
3708
3709@xref{Controlling where ASDF saves compiled files}.
3710
3711@subsection Source Registry Configuration
3712
3713Configuring ASDF used to require special magic
3714to be applied just at the right moment,
3715between the moment ASDF is loaded and the moment it is used,
3716in a way that is specific to the user,
3717the implementation he is using and the application he is building.
3718
3719This made for awkward configuration files and startup scripts
3720that could not be shared between users, managed by administrators
3721or packaged by distributions.
3722
3723ASDF 2 provides a well-documented way to configure ASDF,
3724with sensible defaults, adequate configuration languages,
3725and a coherent set of configuration files and hooks.
3726
3727We believe it's a vast improvement because it decouples
3728application distribution from library distribution.
3729The application writer can avoid thinking where the libraries are,
3730and the library distributor (dpkg, clbuild, advanced user, etc.)
3731can configure them once and for every application.
3732Yet settings can be easily overridden where needed,
3733so whoever needs control has exactly as much as required.
3734
3735At the same time, ASDF 2 remains compatible
3736with the old magic you may have in your build scripts
3737(using @code{*central-registry*} and
3738@code{*system-definition-search-functions*})
3739to tailor the ASDF configuration to your build automation needs,
3740and also allows for new magic, simpler and more powerful magic.
3741
3742@xref{Controlling where ASDF searches for systems}.
3743
3744
3745@subsection Usual operations are made easier to the user
3746
3747In ASDF 1, you had to use the awkward syntax
3748@code{(asdf:oos 'asdf:load-op :foo)}
3749to load a system,
3750and similarly for @code{compile-op}, @code{test-op}.
3751
3752In ASDF 2, you can use shortcuts for the usual operations:
3753@code{(asdf:load-system :foo)}, and
3754similarly for @code{compile-system}, @code{test-system}.
3755
3756
3757@subsection Many bugs have been fixed
3758
3759The following issues and many others have been fixed:
3760
3761@itemize
3762@item
3763The infamous TRAVERSE function has been revamped completely
3764between ASDF 1 and ASDF 2, with many bugs squashed.
3765In particular, dependencies were not correctly propagated
3766across modules but now are.
3767It has been completely rewritten many times over
3768between ASDF 2.000 and ASDF 3,
3769with fundamental issues in the original model being fixed.
3770Timestamps were not propagated at all, and now are.
3771The internal model of how actions depend on each other
3772is now both consistent and complete.
3773The :version and
3774the :force (system1 .. systemN) feature have been fixed.
3775
3776@item
3777Performance has been notably improved for large systems
3778(say with thousands of components) by using
3779hash-tables instead of linear search,
3780and linear-time list accumulation
3781instead of quadratic-time recursive appends.
3782
3783@item
3784Many features used to not be portable,
3785especially where pathnames were involved.
3786Windows support was notably quirky because of such non-portability.
3787
3788@item
3789The internal test suite used to massively fail on many implementations.
3790While still incomplete, it now fully passes
3791on all implementations supported by the test suite,
3792except for GCL (due to GCL bugs).
3793
3794@item
3795Support was lacking for some implementations.
3796ABCL and GCL were notably wholly broken.
3797ECL extensions were not integrated with ASDF release.
3798
3799@item
3800The documentation was grossly out of date.
3801
3802@end itemize
3803
3804
3805@subsection ASDF itself is versioned
3806
3807Between new features, old bugs fixed, and new bugs introduced,
3808there were various releases of ASDF in the wild,
3809and no simple way to check which release had which feature set.
3810People using or writing systems had to either make worst-case assumptions
3811as to what features were available and worked,
3812or take great pains to have the correct version of ASDF installed.
3813
3814With ASDF 2, we provide a new stable set of working features
3815that everyone can rely on from now on.
3816Use @code{#+asdf2} to detect presence of ASDF 2,
3817@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3818to check the availability of a version no earlier than required.
3819
3820
3821@subsection ASDF can be upgraded
3822
3823When an old version of ASDF was loaded,
3824it was very hard to upgrade ASDF in your current image
3825without breaking everything.
3826Instead you had to exit the Lisp process and
3827somehow arrange to start a new one from a simpler image.
3828Something that can't be done from within Lisp,
3829making automation of it difficult,
3830which compounded with difficulty in configuration,
3831made the task quite hard.
3832Yet as we saw before, the task would have been required
3833to not have to live with the worst case or non-portable
3834subset of ASDF features.
3835
3836With ASDF 2, it is easy to upgrade
3837from ASDF 2 to later versions from within Lisp,
3838and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3839We support hot upgrade of ASDF and any breakage is a bug
3840that we will do our best to fix.
3841There are still limitations on upgrade, though,
3842most notably the fact that after you upgrade ASDF,
3843you must also reload or upgrade all ASDF extensions.
3844
3845@subsection Decoupled release cycle
3846
3847When vendors were releasing their Lisp implementations with ASDF,
3848they had to basically never change version
3849because neither upgrade nor downgrade was possible
3850without breaking something for someone,
3851and no obvious upgrade path was visible and recommendable.
3852
3853With ASDF 2, upgrade is possible, easy and can be recommended.
3854This means that vendors can safely ship a recent version of ASDF,
3855confident that if a user isn't fully satisfied,
3856he can easily upgrade ASDF and deal
3857with a supported recent version of it.
3858This means that release cycles will be causally decoupled,
3859the practical consequence of which will mean faster convergence
3860towards the latest version for everyone.
3861
3862
3863@subsection Pitfalls of the transition to ASDF 2
3864
3865The main pitfalls in upgrading to ASDF 2 seem to be related
3866to the output translation mechanism.
3867
3868@itemize
3869
3870@item
3871Output translations is enabled by default. This may surprise some users,
3872most of them in pleasant way (we hope), a few of them in an unpleasant way.
3873It is trivial to disable output translations.
3874@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3875
3876@item
3877Some systems in the large have been known
3878not to play well with output translations.
3879They were relatively easy to fix.
3880Once again, it is also easy to disable output translations,
3881or to override its configuration.
3882
3883@item
3884The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3885They replace A-B-L, and there is compatibility mode to emulate
3886your previous A-B-L configuration.
3887See @code{enable-asdf-binary-locations-compatibility} in
3888@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3889But thou shalt not load ABL on top of ASDF 2.
3890
3891@end itemize
3892
3893Other issues include the following:
3894
3895@itemize
3896
3897@item
3898ASDF pathname designators are now specified
3899in places where they were unspecified,
3900and a few small adjustments have to be made to some non-portable defsystems.
3901Notably, in the @code{:pathname} argument
3902to a @code{defsystem} and its components,
3903a logical pathname (or implementation-dependent hierarchical pathname)
3904must now be specified with @code{#p} syntax
3905where the namestring might have previously sufficed;
3906moreover when evaluation is desired @code{#.} must be used,
3907where it wasn't necessary in the toplevel @code{:pathname} argument
3908(but necessary in other @code{:pathname} arguments).
3909
3910@item
3911There is a slight performance bug, notably on SBCL,
3912when initially searching for @file{asd} files,
3913the implicit @code{(directory "/configured/path/**/*.asd")}
3914for every configured path @code{(:tree "/configured/path/")}
3915in your @code{source-registry} configuration can cause a slight pause.
3916Try to @code{(time (asdf:initialize-source-registry))}
3917to see how bad it is or isn't on your system.
3918If you insist on not having this pause,
3919you can avoid the pause by overriding the default source-registry configuration
3920and not use any deep @code{:tree} entry but only @code{:directory} entries
3921or shallow @code{:tree} entries.
3922Or you can fix your implementation to not be quite that slow
3923when recursing through directories.
3924@emph{Update}: This performance bug fixed the hard way in 2.010.
3925
3926@item
3927On Windows, only LispWorks supports proper default configuration pathnames
3928based on the Windows registry.
3929Other implementations make do with environment variables,
3930that you may have to define yourself
3931if you're using an older version of Windows.
3932Windows support is somewhat less tested than Unix support.
3933Please help report and fix bugs.
3934@emph{Update}: As of ASDF 2.21, all implementations
3935should now use the same proper default configuration pathnames
3936and they should actually work, though they haven't all been tested.
3937
3938@item
3939The mechanism by which one customizes a system so that Lisp files
3940may use a different extension from the default @file{.lisp} has changed.
3941Previously, the pathname for a component
3942was lazily computed when operating on a system,
3943and you would
3944@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3945  (declare (ignorable component system)) "lis")}.
3946Now, the pathname for a component is eagerly computed when defining the system,
3947and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3948and use @code{:default-component-class cl-source-file.lis}
3949as argument to @code{defsystem},
3950as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3951
3952@findex source-file-type
3953
3954
3955@end itemize
3956
3957
3958@section Issues with installing the proper version of ASDF
3959
3960@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3961
3962We recommend you upgrade ASDF.
3963@xref{Loading ASDF,,Upgrading ASDF}.
3964
3965If this does not work, it is a bug, and you should report it.
3966@xref{FAQ, report-bugs, Where do I report a bug}.
3967In the meantime, you can load @file{asdf.lisp} directly.
3968@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3969
3970
3971@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3972
3973Since ASDF 2,
3974it should always be a good time to upgrade to a recent version of ASDF.
3975You may consult with the maintainer for which specific version they recommend,
3976but the latest @code{release} should be correct.
3977We trust you to thoroughly test it with your implementation
3978before you release it.
3979If there are any issues with the current release,
3980it's a bug that you should report upstream and that we will fix ASAP.
3981
3982As to how to include ASDF, we recommend the following:
3983
3984@itemize
3985@item
3986If ASDF isn't loaded yet, then @code{(require "asdf")}
3987should load the version of ASDF that is bundled with your system.
3988If possible so should @code{(require "ASDF")}.
3989You may have it load some other version configured by the user,
3990if you allow such configuration.
3991
3992@item
3993If your system provides a mechanism to hook into @code{CL:REQUIRE},
3994then it would be nice to add ASDF to this hook the same way that
3995ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3996Please send us appropriate code to this end.
3997
3998@item
3999You may, like SBCL, have ASDF be implicitly used to require systems
4000that are bundled with your Lisp distribution.
4001If you do have a few magic systems that come with your implementation
4002in a precompiled way such that one should only use the binary version
4003that goes with your distribution, like SBCL does,
4004then you should add them in the beginning of @code{wrapping-source-registry}.
4005
4006@item
4007If you have magic systems as above, like SBCL does,
4008then we explicitly ask you to @emph{NOT} distribute
4009@file{asdf.asd} as part of those magic systems.
4010You should still include the file @file{asdf.lisp} in your source distribution
4011and precompile it in your binary distribution,
4012but @file{asdf.asd} if included at all,
4013should be secluded from the magic systems,
4014in a separate file hierarchy.
4015Alternatively, you may provide the system
4016after renaming it and its @file{.asd} file to e.g.
4017@code{asdf-ecl} and @file{asdf-ecl.asd}, or
4018@code{sb-asdf} and @file{sb-asdf.asd}.
4019Indeed, if you made @file{asdf.asd} a magic system,
4020then users would no longer be able to upgrade ASDF using ASDF itself
4021to some version of their preference that
4022they maintain independently from your Lisp distribution.
4023
4024@item
4025If you do not have any such magic systems, or have other non-magic systems
4026that you want to bundle with your implementation,
4027then you may add them to the @code{wrapping-source-registry},
4028and you are welcome to include @file{asdf.asd} amongst them.
4029Non-magic systems should be at the back of the @code{wrapping-source-registry}
4030while magic systems are at the front.
4031
4032@item
4033Please send us upstream any patches you make to ASDF itself,
4034so we can merge them back in for the benefit of your users
4035when they upgrade to the upstream version.
4036
4037@end itemize
4038
4039
4040
4041@section Issues with configuring ASDF
4042
4043@subsection ``How can I customize where fasl files are stored?''
4044
4045@xref{Controlling where ASDF saves compiled files}.
4046
4047Note that in the past there was an add-on to ASDF called
4048@code{ASDF-binary-locations}, developed by Gary King.
4049That add-on has been merged into ASDF proper,
4050then superseded by the @code{asdf-output-translations} facility.
4051
4052Note that use of @code{asdf-output-translations}
4053can interfere with one aspect of your systems
4054--- if your system uses @code{*load-truename*} to find files
4055(e.g., if you have some data files stored with your program),
4056then the relocation that this ASDF customization performs
4057is likely to interfere.
4058Use @code{asdf:system-relative-pathname} to locate a file
4059in the source directory of some system, and
4060use @code{asdf:apply-output-translations} to locate a file
4061whose pathname has been translated by the facility.
4062
4063@subsection ``How can I wholly disable the compiler output cache?''
4064
4065To permanently disable the compiler output cache
4066for all future runs of ASDF, you can:
4067
4068@example
4069mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
4070echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
4071@end example
4072
4073This assumes that you didn't otherwise configure the ASDF files
4074(if you did, edit them again),
4075and don't somehow override the configuration at runtime
4076with a shell variable (see below) or some other runtime command
4077(e.g. some call to @code{asdf:initialize-output-translations}).
4078
4079To disable the compiler output cache in Lisp processes
4080run by your current shell, try (assuming @code{bash} or @code{zsh})
4081(on Unix and cygwin only):
4082
4083@example
4084export ASDF_OUTPUT_TRANSLATIONS=/:
4085@end example
4086
4087To disable the compiler output cache just in the current Lisp process,
4088use (after loading ASDF but before using it):
4089
4090@example
4091(asdf:disable-output-translations)
4092@end example
4093
4094@section Issues with using and extending ASDF to define systems
4095
4096@subsection ``How can I cater for unit-testing in my system?''
4097
4098ASDF provides a predefined test operation, @code{test-op}.
4099@xref{Predefined operations of ASDF, test-op}.
4100The test operation, however, is largely left to the system definer to specify.
4101@code{test-op} has been
4102a topic of considerable discussion on the
4103@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
4104and on the
4105@url{https://launchpad.net/asdf,launchpad bug-tracker}.
4106
4107Here are some guidelines:
4108
4109@itemize
4110@item
4111For a given system, @var{foo}, you will want to define a corresponding
4112test system, such as @var{foo-test}.  The reason that you will want this
4113separate system is that ASDF does not out of the box supply components
4114that are conditionally loaded.  So if you want to have source files
4115(with the test definitions) that will not be loaded except when testing,
4116they should be put elsewhere.
4117
4118@item
4119The @var{foo-test} system can be defined in an asd file of its own or
4120together with @var{foo}.  An aesthetic preference against cluttering up
4121the filesystem with extra asd files should be balanced against the
4122question of whether one might want to directly load @var{foo-test}.
4123Typically one would not want to do this except in early stages of
4124debugging.
4125
4126@item
4127Record that testing is implemented by @var{foo-test}.  For example:
4128@example
4129(defsystem @var{foo}
4130   :in-order-to ((test-op (test-op @var{foo-test})))
4131   ....)
4132
4133(defsystem @var{foo-test}
4134   :depends-on (@var{foo} @var{my-test-library} ...)
4135   ....)
4136@end example
4137@end itemize
4138
4139This procedure will allow you to support users who do not wish to
4140install your test framework.
4141
4142One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
4143does not return a value.  So in current versions of ASDF there is no
4144reliable programmatic means of determining whether or not a set of tests
4145has passed, or which tests have failed.  The user must simply read the
4146console output.  This limitation has been the subject of much
4147discussion.
4148
4149@subsection ``How can I cater for documentation generation in my system?''
4150
4151The ASDF developers are currently working to add a @code{doc-op}
4152to the set of predefined ASDF operations.
4153@xref{Predefined operations of ASDF}.
4154See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
4155
4156
4157
4158@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
4159
4160See @code{cffi}'s @code{cffi-grovel}.
4161
4162@anchor{report-bugs}
4163
4164
4165@subsection ``I want to put my module's files at the top level.  How do I do this?''
4166
4167By default, the files contained in an asdf module go
4168in a subdirectory with the same name as the module.
4169However, this can be overridden by adding a @code{:pathname ""} argument
4170to the module description.
4171For example, here is how it could be done
4172in the spatial-trees ASDF system definition for ASDF 2:
4173
4174@example
4175(asdf:defsystem :spatial-trees
4176  :components
4177  ((:module base
4178            :pathname ""
4179            :components
4180            ((:file "package")
4181             (:file "basedefs" :depends-on ("package"))
4182             (:file "rectangles" :depends-on ("package"))))
4183   (:module tree-impls
4184            :depends-on (base)
4185            :pathname ""
4186            :components
4187            ((:file "r-trees")
4188             (:file "greene-trees" :depends-on ("r-trees"))
4189             (:file "rstar-trees" :depends-on ("r-trees"))
4190             (:file "rplus-trees" :depends-on ("r-trees"))
4191             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
4192   (:module viz
4193            :depends-on (base)
4194            :pathname ""
4195            :components
4196            ((:static-file "spatial-tree-viz.lisp")))
4197   (:module tests
4198            :depends-on (base)
4199            :pathname ""
4200            :components
4201            ((:static-file "spatial-tree-test.lisp")))
4202   (:static-file "LICENCE")
4203   (:static-file "TODO")))
4204@end example
4205
4206All of the files in the @code{tree-impls} module are at the top level,
4207instead of in a @file{tree-impls/} subdirectory.
4208
4209Note that the argument to @code{:pathname} can be either a pathname object or a string.
4210A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
4211but this is discouraged because the results of parsing a namestring are not portable.
4212A pathname can only be portably constructed with such syntax as
4213@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
4214and similarly the current directory can only be portably specified as
4215@code{#.(make-pathname :directory '(:relative))}.
4216However, as of ASDF 2, you can portably use a string to denote a pathname.
4217The string will be parsed as a @code{/}-separated path from the current directory,
4218such that the empty string @code{""} denotes the current directory, and
4219@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
4220portably denotes the same subdirectory as above.
4221When files are specified, the last @code{/}-separated component is interpreted
4222either as the name component of a pathname
4223(if the component class specifies a pathname type),
4224or as a name component plus optional dot-separated type component
4225(if the component class doesn't specifies a pathname type).
4226
4227@subsection How do I create a system definition where all the source files have a .cl extension?
4228
4229Starting with ASDF 2.014.14, you may just pass
4230the builtin class @code{cl-source-file.cl} as
4231the @code{:default-component-class} argument to @code{defsystem}:
4232
4233@lisp
4234(defsystem my-cl-system
4235  :default-component-class cl-source-file.cl
4236  ...)
4237@end lisp
4238
4239Another builtin class @code{cl-source-file.lsp} is offered
4240for files ending in @file{.lsp}.
4241
4242If you want to use a different extension
4243for which ASDF doesn't provide builtin support,
4244or want to support versions of ASDF
4245earlier than 2.014.14 (but later than 2.000),
4246you can define a class as follows:
4247
4248@lisp
4249;; Prologue: make sure we're using a sane package.
4250(defpackage :my-asdf-extension
4251   (:use :asdf :common-lisp)
4252   (:export #:cl-source-file.lis))
4253(in-package :my-asdf-extension)
4254
4255(defclass cl-source-file.lis (cl-source-file)
4256  ((type :initform "lis")))
4257@end lisp
4258
4259Then you can use it as follows:
4260@lisp
4261(defsystem my-cl-system
4262  :default-component-class my-asdf-extension:cl-source-file.lis
4263  ...)
4264@end lisp
4265
4266Of course, if you're in the same package, e.g. in the same file,
4267you won't need to use the package qualifier before @code{cl-source-file.lis}.
4268Actually, if all you're doing is defining this class
4269and using it in the same file without other fancy definitions,
4270you might skip package complications:
4271
4272@lisp
4273(in-package :asdf)
4274(defclass cl-source-file.lis (cl-source-file)
4275   ((type :initform "lis")))
4276(defsystem my-cl-system
4277  :default-component-class cl-source-file.lis
4278  ...)
4279@end lisp
4280
4281It is possible to achieve the same effect
4282in a way that supports both ASDF 1 and ASDF 2,
4283but really, friends don't let friends use ASDF 1.
4284Please upgrade to ASDF 3.
4285In short, though: do same as above, but
4286@emph{before} you use the class in a @code{defsystem},
4287you also define the following method:
4288
4289@lisp
4290(defmethod source-file-type ((f cl-source-file.lis) (s system))
4291  (declare (ignorable f s))
4292  "lis")
4293@end lisp
4294
4295@comment FIXME: Add a FAQ about how to use a new system class...
4296
4297
4298@node  TODO list, Inspiration, FAQ, Top
4299@comment  node-name,  next,  previous,  up
4300@chapter TODO list
4301
4302Here is an old list of things to do,
4303in addition to the bugs that are now tracked on launchpad:
4304@url{https://launchpad.net/asdf}.
4305
4306@section Outstanding spec questions, things to add
4307
4308** packaging systems
4309
4310*** manual page component?
4311
4312** style guide for .asd files
4313
4314You should either use keywords or be careful
4315with the package that you evaluate defsystem forms in.
4316Otherwise @code{(defsystem partition ...)}
4317being read in the @code{cl-user} package
4318will intern a @code{cl-user:partition} symbol,
4319which will then collide with the @code{partition:partition} symbol.
4320
4321Actually there's a hairier packages problem to think about too.
4322@code{in-order-to} is not a keyword:
4323if you read @code{defsystem} forms in a package that doesn't use ASDF,
4324odd things might happen.
4325
4326
4327** extending defsystem with new options
4328
4329You might not want to write a whole parser,
4330but just to add options to the existing syntax.
4331Reinstate @code{parse-option} or something akin.
4332
4333
4334** Diagnostics
4335
4336A ``dry run'' of an operation can be made with the following form:
4337
4338@lisp
4339(let ((asdf::*verbose-out* *standard-output*))
4340  (loop :for (op . comp) :in
4341    (asdf::traverse (make-instance '<operation-name> :force t)
4342                    (asdf:find-system <system-name>))
4343    :do (asdf:explain op comp)))
4344@end lisp
4345
4346This uses unexported symbols.
4347What would be a nice interface for this functionality?
4348
4349@section Missing bits in implementation
4350
4351** reuse the same scratch package whenever a system is reloaded from disk
4352
4353Have a package ASDF-USER instead of all these temporary packages?
4354
4355** proclamations probably aren't
4356
4357** A revert function
4358
4359Other possible interface: have a ``revert'' function akin to @code{make clean}.
4360
4361@lisp
4362(asdf:revert 'asdf:compile-op 'araneida)
4363@end lisp
4364
4365would delete any files produced by @code{(compile-system :araneida)}.
4366Of course, it wouldn't be able to do much about stuff in the image itself.
4367
4368How would this work?
4369
4370@code{traverse}
4371
4372There's a difference between a module's dependencies (peers)
4373and its components (children).
4374Perhaps there's a similar difference in operations?
4375For example, @code{(load "use") depends-on (load "macros")} is a peer,
4376whereas @code{(load "use") depends-on (compile "use")}
4377is more of a ``subservient'' relationship.
4378
4379@node  Inspiration, Concept Index, TODO list, Top
4380@comment  node-name,  next,  previous,  up
4381@chapter Inspiration
4382
4383@section mk-defsystem (defsystem-3.x)
4384
4385We aim to solve basically the same problems as @code{mk-defsystem} does.
4386However, our architecture for extensibility
4387better exploits CL language features (and is documented),
4388and we intend to be portable rather than just widely-ported.
4389No slight on the @code{mk-defsystem} authors and maintainers is intended here;
4390that implementation has the unenviable task
4391of supporting pre-ANSI implementations, which is no longer necessary.
4392
4393The surface defsystem syntax of asdf is more-or-less compatible with
4394@code{mk-defsystem}, except that we do not support
4395the @code{source-foo} and @code{binary-foo} prefixes
4396for separating source and binary files, and
4397we advise the removal of all options to specify pathnames.
4398
4399The @code{mk-defsystem} code for topologically sorting
4400a module's dependency list was very useful.
4401
4402@section defsystem-4 proposal
4403
4404Marco and Peter's proposal for defsystem 4 served as the driver for
4405many of the features in here.  Notable differences are:
4406
4407@itemize
4408@item
4409We don't specify output files or output file extensions
4410as part of the system.
4411
4412If you want to find out what files an operation would create,
4413ask the operation.
4414
4415@item
4416We don't deal with CL packages
4417
4418If you want to compile in a particular package, use an @code{in-package} form
4419in that file (ilisp / SLIME will like you more if you do this anyway)
4420
4421@item
4422There is no proposal here that @code{defsystem} does version control.
4423
4424A system has a given version which can be used to check dependencies,
4425but that's all.
4426@end itemize
4427
4428The defsystem 4 proposal tends to look more at the external features,
4429whereas this one centres on a protocol for system introspection.
4430
4431@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
4432
4433Available in updated-for-CL form on the web at
4434@url{http://nhplace.com/kent/Papers/Large-Systems.html}
4435
4436In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
4437and concept to deal with creating component trees
4438from @code{defsystem} surface syntax.
4439[ this is not true right now, though it used to be and
4440probably will be again soon ]
4441
4442
4443@c -------------------
4444
4445
4446@node Concept Index, Function and Class Index, Inspiration, Top
4447@unnumbered Concept Index
4448
4449@printindex cp
4450
4451@node Function and Class Index, Variable Index, Concept Index, Top
4452@unnumbered Function and Class Index
4453
4454@printindex fn
4455
4456@node Variable Index,  , Function and Class Index, Top
4457@unnumbered Variable Index
4458
4459@printindex vr
4460
4461@bye
Note: See TracBrowser for help on using the repository browser.