source: trunk/abcl/doc/asdf/asdf.texinfo @ 12655

Last change on this file since 12655 was 12655, checked in by Mark Evenson, 12 years ago

Update to ASDF 1.719 as recommended by ASDF developers.

File size: 114.0 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2010 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title asdf: another system definition facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We are preparing for a release of ASDF 2, hopefully for May 2010,
174which will have version 2.000 and later.
175Current releases, in the 1.700 series and beyond,
176should be considered as release candidates.
177We're still working on polishing the code and documentation.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179
180
181@node Loading ASDF, Configuring ASDF, Introduction, Top
182@comment  node-name,  next,  previous,  up
183@chapter Loading ASDF
184@vindex *central-registry*
185@cindex link farm
186@findex load-system
187@findex compile-system
188@findex test-system
189@cindex system directory designator
190@findex operate
191@findex oos
192
193@c @menu
194@c * Installing ASDF::
195@c @end menu
196
197
198@section Loading a pre-installed ASDF
199
200Many Lisp implementations include a copy of ASDF.
201You can usually load this copy using Common Lisp's @code{require} function:
202
203@lisp
204(require :asdf)
205@end lisp
206
207Consult your Lisp implementation's documentation for details.
208
209Hopefully, ASDF 2 will soon be bundled with every Common Lisp implementation,
210and you can load it that way.
211
212
213@section Checking whether ASDF is loaded
214
215To check whether ASDF is properly loaded in your current Lisp image,
216you can run this form:
217
218@lisp
219(asdf:asdf-version)
220@end lisp
221
222If it returns a string,
223that is the version of ASDF that is currently installed.
224
225If it raises an error,
226then either ASDF is not loaded, or
227you are using an old version of ASDF.
228
229You can check whether an old version is loaded
230by checking if the ASDF package is present.
231The form below will allow you to programmatically determine
232whether a recent version is loaded, an old version is loaded,
233or none at all:
234
235@lisp
236(or #+asdf2 (asdf:asdf-version) #+asdf :old)
237@end lisp
238
239If it returns a version number, that's the version of ASDF installed.
240If it returns the keyword @code{:OLD},
241then you're using an old version of ASDF (from before 1.635).
242If it returns @code{NIL} then ASDF is not installed.
243
244If you are running a version older than 1.711,
245we recommend that you load a newer ASDF using the method below.
246
247
248@section Upgrading ASDF
249
250If your implementation does provide ASDF 2 or later,
251and you want to upgrade to a more recent version,
252just install ASDF like any other package
253(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
254configure ASDF as usual (see @pxref{Configuring ASDF} below),
255and upgrade with:
256
257@lisp
258(require :asdf)
259(asdf:load-system :asdf)
260@end lisp
261
262If on the other hand, your implementation only provides an old ASDF,
263you will require a special configuration step and an old-style loading:
264
265@lisp
266(require :asdf)
267(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
268(asdf:oos 'asdf:load-op :asdf)
269@end lisp
270
271Don't forget the trailing @code{/} at the end of your pathname.
272
273Also, note that older versions of ASDF won't redirect their output,
274or at least won't do it according to your usual ASDF 2 configuration.
275You therefore need write access on the directory
276where you install the new ASDF,
277and make sure you're not using it
278for multiple mutually incompatible implementations.
279At worst, you may have to have multiple copies of the new ASDF,
280e.g. one per implementation installation, to avoid clashes.
281
282Finally, note that there are some limitations to upgrading ASDF:
283@itemize
284@item
285Any ASDF extension is invalidated, and will need to be reloaded.
286@item
287It is safer if you upgrade ASDF and its extensions as a special step
288at the very beginning of whatever script you are running,
289before you start using ASDF to load anything else.
290@end itemize
291
292
293@section Loading an otherwise installed ASDF
294
295If your implementation doesn't include ASDF,
296if for some reason the upgrade somehow fails,
297does not or cannot apply to your case,
298you will have to install the file @file{asdf.lisp}
299somewhere and load it with:
300
301@lisp
302(load "/path/to/your/installed/asdf.lisp")
303@end lisp
304
305The single file @file{asdf.lisp} is all you normally need to use ASDF.
306
307You can extract this file from latest release tarball on the
308@url{http://common-lisp.net/project/asdf/,ASDF website}.
309If you are daring and willing to report bugs, you can get
310the latest and greatest version of ASDF from its git repository.
311@xref{Getting the latest version}.
312
313For maximum convenience you might want to have ASDF loaded
314whenever you start your Lisp implementation,
315for example by loading it from the startup script or dumping a custom core
316--- check your Lisp implementation's manual for details.
317
318
319@node Configuring ASDF, Using ASDF, Loading ASDF, Top
320@comment  node-name,  next,  previous,  up
321
322@chapter Configuring ASDF
323
324@section Configuring ASDF to find your systems
325
326So it may compile and load your systems, ASDF must be configured to find
327the @file{.asd} files that contain system definitions.
328
329Since ASDF 2, the preferred way to configure where ASDF finds your systems is
330the @code{source-registry} facility,
331fully described in its own chapter of this manual.
332@xref{Controlling where ASDF searches for systems}.
333
334The default location for a user to install Common Lisp software is under
335@file{~/.local/share/common-lisp/source/}.
336If you install software there, you don't need further configuration.
337If you're installing software yourself at a location that isn't standard,
338you have to tell ASDF where you installed it. See below.
339If you're using some tool to install software,
340the authors of that tool should already have configured ASDF.
341
342The simplest way to add a path to your search path,
343say @file{/foo/bar/baz/quux/}
344is to create the directory
345@file{~/.config/common-lisp/source-registry.conf.d/}
346and there create a file with any name of your choice,
347for instance @file{42-bazquux.conf}
348containing the line:
349
350@kbd{(:directory "/foo/bar/baz/quux/")}
351
352If you want all the subdirectories under @file{/foo/bar/baz/}
353to be recursively scanned for @file{.asd} files, instead use:
354
355@kbd{(:tree "/foo/bar/baz/quux/")}
356
357Note that your Operating System distribution or your system administrator
358may already have configured system-managed libraries for you.
359
360Also note that when choosing a filename, the convention is to use
361the @file{.conf} extension
362(and a non-empty extension is required for CLISP compatibility),
363and it is customary to start the filename with two digits
364that specify the order in which the directories will be scanned.
365
366ASDF will automatically read your configuration
367the first time you try to find a system.
368You can reset the source-registry configuration with:
369
370@lisp
371(asdf:clear-source-registry)
372@end lisp
373
374And you probably should do so before you dump your Lisp image,
375if the configuration may change
376between the machine where you save it at the time you save it
377and the machine you resume it at the time you resume it.
378
379
380@section Configuring ASDF to find your systems -- old style
381
382The old way to configure ASDF to find your systems is by
383@code{push}ing directory pathnames onto the variable
384@code{asdf:*central-registry*}.
385
386You must configure this variable between the time you load ASDF
387and the time you first try to use it.
388Loading and configuring ASDF presumably happen
389as part of some initialization script that builds or starts
390your Common Lisp software system.
391(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
392
393The @code{asdf:*central-registry*} is empty by default in ASDF 2,
394but is still supported for compatibility with ASDF 1.
395When used, it takes precedence over the above source-registry@footnote{
396It is possible to further customize
397the system definition file search.
398That's considered advanced use, and covered later:
399search forward for
400@code{*system-definition-search-functions*}.
401@xref{Defining systems with defsystem}.}.
402
403For instance, if you wanted ASDF to find the @file{.asd} file
404@file{/home/me/src/foo/foo.asd} your initialization script
405could after it loads ASDF with @code{(require :asdf)}
406configure it with:
407
408@lisp
409(push "/home/me/src/foo/" asdf:*central-registry*)
410@end lisp
411
412Note the trailing slash: when searching for a system,
413ASDF will evaluate each entry of the central registry
414and coerce the result to a pathname@footnote{
415ASDF will indeed call @code{EVAL} on each entry.
416It will also skip entries that evaluate to @code{NIL}.
417
418Strings and pathname objects are self-evaluating,
419in which case the @code{EVAL} step does nothing;
420but you may push arbitrary SEXP onto the central registry,
421that will be evaluated to compute e.g. things that depend
422on the value of shell variables or the identity of the user.
423
424The variable @code{asdf:*central-registry*} is thus a list of
425``system directory designators''.
426A @dfn{system directory designator} is a form
427which will be evaluated whenever a system is to be found,
428and must evaluate to a directory to look in.
429By ``directory'' here, we mean
430``designator for a pathname with a supplied DIRECTORY component''.
431}
432at which point the presence of the trailing directory name separator
433is necessary to tell Lisp that you're discussing a directory
434rather than a file.
435
436Typically, however, there are a lot of @file{.asd} files, and
437a common idiom was to have to put
438a bunch of @emph{symbolic links} to @file{.asd} files
439in a common directory
440and push @emph{that} directory (the ``link farm'')
441to the
442@code{asdf:*central-registry*}
443instead of pushing each of the many involved directories
444to the @code{asdf:*central-registry*}.
445ASDF knows how to follow such @emph{symlinks}
446to the actual file location when resolving the paths of system components
447(on Windows, you can use Windows shortcuts instead of POSIX symlinks).
448
449For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
450is a member of @code{*central-registry*}, you could set up the
451system @var{foo} for loading with asdf with the following
452commands at the shell:
453
454@example
455$ cd /home/me/cl/systems/
456$ ln -s ~/src/foo/foo.asd .
457@end example
458
459This old style for configuring ASDF is not recommended for new users,
460but it is supported for old users, and for users who want to programmatically
461control what directories are added to the ASDF search path.
462
463
464@section Configuring where ASDF stores object files
465@findex clear-output-locations
466
467ASDF lets you configure where object files will be stored.
468Sensible defaults are provided and
469you shouldn't normally have to worry about it.
470
471This allows the same source code repository may be shared
472between several versions of several Common Lisp implementations,
473between several users using different compilation options
474and without write privileges on shared source directories, etc.
475This also allows to keep source directories uncluttered
476by plenty of object files.
477
478Starting with ASDF 2, the @code{asdf-output-translations} facility
479was added to ASDF itself, that controls where object files will be stored.
480This facility is fully described in a chapter of this manual,
481@ref{Controlling where ASDF saves compiled files}.
482
483The simplest way to add a translation to your search path,
484say from @file{/foo/bar/baz/quux/}
485to @file{/where/i/want/my/fasls/}
486is to create the directory
487@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
488and there create a file with any name of your choice,
489for instance @file{42-bazquux.conf}
490containing the line:
491
492@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
493
494To disable output translations for source under a given directory,
495say @file{/toto/tata/}
496you can create a file @file{40-disable-toto.conf}
497with the line:
498
499@kbd{("/toto/tata/")}
500
501To wholly disable output translations for all directories,
502you can create a file @file{00-disable.conf}
503with the line:
504
505@kbd{(t t)}
506
507Note that your Operating System distribution or your system administrator
508may already have configured translations for you.
509In absence of any configuration, the default is to redirect everything
510under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
511@xref{Controlling where ASDF searches for systems}, for full details.
512
513
514Also note that when choosing a filename, the convention is to use
515the @file{.conf} extension
516(and a non-empty extension is required for CLISP compatibility),
517and it is customary to start the filename with two digits
518that specify the order in which the directories will be scanned.
519
520ASDF will automatically read your configuration
521the first time you try to find a system.
522You can reset the source-registry configuration with:
523
524@lisp
525(asdf:clear-output-translations)
526@end lisp
527
528And you probably should do so before you dump your Lisp image,
529if the configuration may change
530between the machine where you save it at the time you save it
531and the machine you resume it at the time you resume it.
532
533Finally note that before ASDF 2,
534other ASDF add-ons offered the same functionality,
535each in subtly different and incompatible ways:
536ASDF-Binary-Locations, cl-launch, common-lisp-controller.
537ASDF-Binary-Locations is now not needed anymore and should not be used.
538cl-launch 2.900 and common-lisp-controller 7.1 have been updated
539to just delegate this functionality to ASDF.
540
541@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
542@comment  node-name,  next,  previous,  up
543
544@chapter Using ASDF
545
546@section Loading a system
547
548The system @var{foo} is loaded (and compiled, if necessary)
549by evaluating the following Lisp form:
550
551@example
552(asdf:load-system :@var{foo})
553@end example
554
555On some implementations (namely ABCL, Clozure CL, CMUCL, ECL and SBCL),
556ASDF hooks into the @code{CL:REQUIRE} facility
557and you can just use:
558
559@example
560(require :@var{foo})
561@end example
562
563In older versions of ASDF, you needed to use
564@code{(asdf:oos 'asdf:load-op :@var{foo})}.
565If your ASDF is too old to provide @code{asdf:load-system} though
566we recommend that you upgrade to ASDF 2.
567@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
568
569
570@section Other Operations
571
572ASDF provides three commands for the most common system operations:
573@code{load-system}, @code{compile-system} or @code{test-system}.
574
575Because ASDF is an extensible system
576for defining @emph{operations} on @emph{components},
577it also provides a generic function @code{operate}
578(which is usually abbreviated by @code{oos}).
579You'll use @code{oos} whenever you want to do something beyond
580compiling, loading and testing.
581
582Output from ASDF and ASDF extensions are supposed to be sent
583to the CL stream @code{*standard-output*},
584and so rebinding that stream around calls to @code{asdf:operate}
585should redirect all output from ASDF operations.
586
587Reminder: before ASDF can operate on a system, however,
588it must be able to find and load that system's definition.
589@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
590
591
592@section Summary
593
594To use ASDF:
595
596@itemize
597@item
598Load ASDF itself into your Lisp image, either through
599@code{(require :asdf)} or else through
600@code{(load "/path/to/asdf.lisp")}.
601
602@item
603Make sure ASDF can find system definitions
604thanks to proper source-registry configuration.
605
606@item
607Load a system with @code{(load-system :my-system)}
608or use some other operation on some system of your choice.
609
610@end itemize
611
612@section Moving on
613
614That's all you need to know to use ASDF to load systems written by others.
615The rest of this manual deals with writing system definitions
616for Common Lisp software you write yourself,
617including how to extend ASDF to define new operation and component types.
618
619
620@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
621@comment  node-name,  next,  previous,  up
622@chapter Defining systems with defsystem
623
624This chapter describes how to use asdf to define systems and develop
625software.
626
627
628@menu
629* The defsystem form::
630* A more involved example::
631* The defsystem grammar::
632* Other code in .asd files::
633@end menu
634
635@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
636@comment  node-name,  next,  previous,  up
637@section The defsystem form
638
639Systems can be constructed programmatically
640by instantiating components using @code{make-instance}.
641Most of the time, however, it is much more practical to use
642a static @code{defsystem} form.
643This section begins with an example of a system definition,
644then gives the full grammar of @code{defsystem}.
645
646Let's look at a simple system.
647This is a complete file that would
648usually be saved as @file{hello-lisp.asd}:
649
650@lisp
651(in-package :asdf)
652
653(defsystem "hello-lisp"
654  :description "hello-lisp: a sample Lisp system."
655  :version "0.2"
656  :author "Joe User <joe@@example.com>"
657  :licence "Public Domain"
658  :components ((:file "packages")
659               (:file "macros" :depends-on ("packages"))
660               (:file "hello" :depends-on ("macros"))))
661@end lisp
662
663Some notes about this example:
664
665@itemize
666
667@item
668The file starts with an @code{in-package} form
669to use package @code{asdf}.
670You could instead start your definition by using
671a qualified name @code{asdf:defsystem}.
672
673@item
674If in addition to simply using @code{defsystem},
675you are going to define functions,
676create ASDF extension, globally bind symbols, etc.,
677it is recommended that to avoid namespace pollution between systems,
678you should create your own package for that purpose,
679for instance replacing the above @code{(in-package :asdf)} with:
680
681@lisp
682(defpackage :foo-system
683  (:use :cl :asdf))
684
685(in-package :foo-system)
686@end lisp
687
688@item
689The @code{defsystem} form defines a system named @code{hello-lisp}
690that contains three source files:
691@file{packages}, @file{macros} and @file{hello}.
692
693@item
694The file @file{macros} depends on @file{packages}
695(presumably because the package it's in is defined in @file{packages}),
696and the file @file{hello} depends on @file{macros}
697(and hence, transitively on @file{packages}).
698This means that ASDF will compile and load @file{packages} and @file{macros}
699before starting the compilation of file @file{hello}.
700
701@item
702The files are located in the same directory
703as the file with the system definition.
704ASDF resolves symbolic links (or Windows shortcuts)
705before loading the system definition file and
706stores its location in the resulting system@footnote{
707It is possible, though almost never necessary, to override this behaviour.}.
708This is a good thing because the user can move the system sources
709without having to edit the system definition.
710
711@end itemize
712
713@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
714@comment  node-name,  next,  previous,  up
715@section A more involved example
716
717Let's illustrate some more involved uses of @code{defsystem} via a
718slightly convoluted example:
719
720@lisp
721(defsystem "foo"
722  :version "1.0"
723  :components ((:module "foo" :components ((:file "bar") (:file"baz")
724                                           (:file "quux"))
725                :perform (compile-op :after (op c)
726                          (do-something c))
727                :explain (compile-op :after (op c)
728                          (explain-something c)))
729               (:file "blah")))
730@end lisp
731
732The method-form tokens need explaining: essentially, this part:
733
734@lisp
735                :perform (compile-op :after (op c)
736                          (do-something c))
737                :explain (compile-op :after (op c)
738                          (explain-something c))
739@end lisp
740
741has the effect of
742
743@lisp
744(defmethod perform :after ((op compile-op) (c (eql ...)))
745           (do-something c))
746(defmethod explain :after ((op compile-op) (c (eql ...)))
747           (explain-something c))
748@end lisp
749
750where @code{...} is the component in question;
751note that although this also supports @code{:before} methods,
752they may not do what you want them to ---
753a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
754will run after all the dependencies and sub-components have been processed,
755but before the component in question has been compiled.
756
757@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
758@comment  node-name,  next,  previous,  up
759@section The defsystem grammar
760
761@example
762system-definition := ( defsystem system-designator @var{option}* )
763
764option := :components component-list
765        | :pathname pathname-specifier
766        | :default-component-class
767        | :perform method-form
768        | :explain method-form
769        | :output-files method-form
770        | :operation-done-p method-form
771        | :depends-on ( @var{dependency-def}* )
772        | :serial [ t | nil ]
773        | :in-order-to ( @var{dependency}+ )
774
775component-list := ( @var{component-def}* )
776
777component-def  := ( component-type simple-component-name @var{option}* )
778
779component-type := :system | :module | :file | :static-file | other-component-type
780
781other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
782
783dependency-def := simple-component-name
784               | ( :feature name )
785               | ( :version simple-component-name version-specifier)
786
787dependency := (dependent-op @var{requirement}+)
788requirement := (required-op @var{required-component}+)
789             | (feature feature-name)
790dependent-op := operation-name
791required-op := operation-name | feature
792
793simple-component-name := string
794                      |  symbol
795
796pathname-specifier := pathname | string | symbol
797
798method-form := (operation-name qual lambda-list @&rest body)
799qual := method qualifier
800@end example
801
802@subsection Component names
803
804Component names (@code{simple-component-name})
805may be either strings or symbols.
806
807@subsection Component types
808
809Component type names, even if expressed as keywords, will be looked up
810by name in the current package and in the asdf package, if not found in
811the current package.  So a component type @code{my-component-type}, in
812the current package @code{my-system-asd} can be specified as
813@code{:my-component-type}, or @code{my-component-type}.
814
815@subsection Pathname specifiers
816
817A pathname specifier (@code{pathname-specifier})
818may be a pathname, a string or a symbol.
819When no pathname specifier is given for a component,
820which is the usual case, the component name itself is used.
821
822If a string is given, which is the usual case,
823the string will be interpreted as a Unix-style pathname
824where @code{/} characters will be interpreted as directory separators.
825Usually, Unix-style relative pathnames are used
826(i.e. not starting with @code{/}, as opposed to absolute pathnames);
827they are relative to the path of the parent component.
828Finally, depending on the @code{component-type},
829the pathname may be interpreted as either a file or a directory,
830and if it's a file,
831a file type may be added corresponding to the @code{component-type},
832or else it will be extracted from the string itself (if applicable).
833
834For instance, the @code{component-type} @code{:module}
835wants a directory pathname, and so a string @code{"foo/bar"}
836will be interpreted as the pathname @file{#p"foo/bar/"}.
837On the other hand, the @code{component-type} @code{:file}
838wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
839will be interpreted as the pathname @file{#p"foo/bar.lisp"},
840and a string @code{"foo/bar.quux"}
841will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
842Finally, the @code{component-type} @code{:static-file}
843wants a file without specifying a type, and so a string @code{"foo/bar"}
844will be interpreted as the pathname @file{#p"foo/bar"},
845and a string @code{"foo/bar.quux"}
846will be interpreted as the pathname @file{#p"foo/bar.quux"}.
847
848If a symbol is given, it will be translated into a string,
849and downcased in the process.
850The downcasing of symbols is unconventional,
851but was selected after some consideration.
852Observations suggest that the type of systems we want to support
853either have lowercase as customary case (Unix, Mac, windows)
854or silently convert lowercase to uppercase (lpns),
855so this makes more sense than attempting to use @code{:case :common}
856as argument to @code{make-pathname},
857which is reported not to work on some implementations.
858
859Pathnames objects may be given to override the path for a component.
860Such objects are typically specified using reader macros such as @code{#p}
861or @code{#.(make-pathname ...)}.
862Note however, that @code{#p...} is a short for @code{#.(parse-namestring ...)}
863and that the behavior @code{parse-namestring} is completely non-portable,
864unless you are using Common Lisp @code{logical-pathname}s.
865(@xref{The defsystem grammar,,Warning about logical pathnames}, below.)
866Pathnames made with @code{#.(make-pathname ...)}
867can usually be done more easily with the string syntax above.
868The only case that you really need a pathname object is to override
869the component-type default file type for a given component.
870Therefore, it is a rare case that pathname objects should be used at all.
871Unhappily, ASDF 1 didn't properly support
872parsing component names as strings specifying paths with directories,
873and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
874Note that when specifying pathname objects, no magic interpretation of the pathname
875is made depending on the component type.
876On the one hand, you have to be careful to provide a pathname that correctly
877fulfills whatever constraints are required from that component type
878(e.g. naming a directory or a file with appropriate type);
879on the other hand, you can circumvent the file type that would otherwise
880be forced upon you if you were specifying a string.
881
882
883@subsection Warning about logical pathnames
884
885To use logical pathnames,
886you will have to provide a pathname object as a @code{:pathname} specifier
887to components that use it, using such syntax as
888@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
889
890You only have to specify such logical pathname for your system or
891some top-level component, as sub-components using the usual string syntax
892for names will be properly merged with the pathname of their parent.
893The specification of a logical pathname host however is @emph{not}
894otherwise directly supported in the ASDF syntax
895for pathname specifiers as strings.
896
897Logical pathnames are not specifically recommended to newcomers,
898but are otherwise supported.
899Moreover, the @code{asdf-output-translation} layer will
900avoid trying to resolve and translate logical-pathnames,
901so you can define yourself what translations you want to use
902with the logical pathname facility.
903
904The user of logical pathnames will have to configure logical pathnames himself,
905before they may be used, and ASDF provides no specific support for that.
906
907
908@subsection Serial dependencies
909
910If the @code{:serial t} option is specified for a module,
911ASDF will add dependencies for each each child component,
912on all the children textually preceding it.
913This is done as if by @code{:depends-on}.
914
915@lisp
916:components ((:file "a") (:file "b") (:file "c"))
917:serial t
918@end lisp
919
920is equivalent to
921
922@lisp
923:components ((:file "a")
924             (:file "b" :depends-on ("a"))
925             (:file "c" :depends-on ("a" "b")))
926@end lisp
927
928
929@subsection Source location
930
931The @code{:pathname} option is optional in all cases for systems
932defined via @code{defsystem},
933and in the usual case the user is recommended not to supply it.
934
935Instead, ASDF follows a hairy set of rules that are designed so that
936@enumerate
937@item
938@code{find-system}
939will load a system from disk
940and have its pathname default to the right place.
941@item
942This pathname information will not be overwritten with
943@code{*default-pathname-defaults*}
944(which could be somewhere else altogether)
945if the user loads up the @file{.asd} file into his editor
946and interactively re-evaluates that form.
947@end enumerate
948
949If a system is being loaded for the first time,
950its top-level pathname will be set to:
951
952@itemize
953@item
954The host/device/directory parts of @code{*load-truename*},
955if it is bound.
956@item
957@code{*default-pathname-defaults*}, otherwise.
958@end itemize
959
960If a system is being redefined, the top-level pathname will be
961
962@itemize
963@item
964changed, if explicitly supplied or obtained from @code{*load-truename*}
965(so that an updated source location is reflected in the system definition)
966@item
967changed if it had previously been set from @code{*default-pathname-defaults*}
968@item
969left as before, if it had previously been set from @code{*load-truename*}
970and @code{*load-truename*} is currently unbound
971(so that a developer can evaluate a @code{defsystem} form
972from within an editor without clobbering its source location)
973@end itemize
974
975@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
976@section Other code in .asd files
977
978Files containing @code{defsystem} forms
979are regular Lisp files that are executed by @code{load}.
980Consequently, you can put whatever Lisp code you like into these files
981(e.g., code that examines the compile-time environment
982and adds appropriate features to @code{*features*}).
983However, some conventions should be followed,
984so that users can control certain details of execution
985of the Lisp in @file{.asd} files:
986
987@itemize
988@item
989Any informative output
990(other than warnings and errors,
991which are the condition system's to dispose of)
992should be sent to the standard CL stream @code{*standard-output*},
993so that users can easily control the disposition
994of output from ASDF operations.
995@end itemize
996
997
998@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
999@comment  node-name,  next,  previous,  up
1000@chapter The object model of ASDF
1001
1002ASDF is designed in an object-oriented way from the ground up.
1003Both a system's structure and the operations that can be performed on systems
1004follow a protocol.
1005ASDF is extensible to new operations and to new component types.
1006This allows the addition of behaviours:
1007for example, a new component could be added for Java JAR archives,
1008and methods specialised on @code{compile-op} added for it
1009that would accomplish the relevant actions.
1010
1011This chapter deals with @emph{components}, the building blocks of a system,
1012and @emph{operations}, the actions that can be performed on a system.
1013
1014
1015
1016@menu
1017* Operations::
1018* Components::
1019@end menu
1020
1021@node  Operations, Components, The object model of ASDF, The object model of ASDF
1022@comment  node-name,  next,  previous,  up
1023@section Operations
1024@cindex operation
1025
1026An @dfn{operation} object of the appropriate type is instantiated
1027whenever the user wants to do something with a system like
1028
1029@itemize
1030@item compile all its files
1031@item load the files into a running lisp environment
1032@item copy its source files somewhere else
1033@end itemize
1034
1035Operations can be invoked directly, or examined
1036to see what their effects would be without performing them.
1037@emph{FIXME: document how!}
1038There are a bunch of methods specialised on operation and component type
1039that actually do the grunt work.
1040
1041The operation object contains whatever state is relevant for this purpose
1042(perhaps a list of visited nodes, for example)
1043but primarily is a nice thing to specialise operation methods on
1044and easier than having them all be @code{EQL} methods.
1045
1046Operations are invoked on systems via @code{operate}.
1047@anchor{operate}
1048@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1049@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1050@code{operate} invokes @var{operation} on @var{system}.
1051@code{oos} is a synonym for @code{operate}.
1052
1053@var{operation} is a symbol that is passed, along with the supplied
1054@var{initargs}, to @code{make-instance} to create the operation object.
1055@var{system} is a system designator.
1056
1057The @var{initargs} are passed to the @code{make-instance} call
1058when creating the operation object.
1059Note that dependencies may cause the operation
1060to invoke other operations on the system or its components:
1061the new operations will be created
1062with the same @var{initargs} as the original one.
1063
1064@end deffn
1065
1066@menu
1067* Predefined operations of ASDF::
1068* Creating new operations::
1069@end menu
1070
1071@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1072@comment  node-name,  next,  previous,  up
1073@subsection Predefined operations of ASDF
1074
1075All the operations described in this section are in the @code{asdf} package.
1076They are invoked via the @code{operate} generic function.
1077
1078@lisp
1079(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1080@end lisp
1081
1082@deffn Operation @code{compile-op} @&key @code{proclamations}
1083
1084This operation compiles the specified component.
1085If proclamations are supplied, they will be proclaimed.
1086This is a good place to specify optimization settings.
1087
1088When creating a new component type,
1089you should provide methods for @code{compile-op}.
1090
1091When @code{compile-op} is invoked,
1092component dependencies often cause some parts of the system
1093to be loaded as well as compiled.
1094Invoking @code{compile-op}
1095does not necessarily load all the parts of the system, though;
1096use @code{load-op} to load a system.
1097@end deffn
1098
1099@deffn Operation @code{load-op} @&key @code{proclamations}
1100
1101This operation loads a system.
1102
1103The default methods for @code{load-op} compile files before loading them.
1104For parity, your own methods on new component types should probably do so too.
1105@end deffn
1106
1107@deffn Operation @code{load-source-op}
1108
1109This operation will load the source for the files in a module
1110even if the source files have been compiled.
1111Systems sometimes have knotty dependencies
1112which require that sources are loaded
1113before they can be compiled.
1114This is how you do that.
1115
1116If you are creating a component type,
1117you need to implement this operation --- at least, where meaningful.
1118@end deffn
1119
1120@anchor{test-op}
1121@deffn Operation @code{test-op}
1122
1123This operation will perform some tests on the module.
1124The default method will do nothing.
1125The default dependency is to require
1126@code{load-op} to be performed on the module first.
1127The default @code{operation-done-p} is that the operation is @emph{never} done
1128---
1129we assume that if you invoke the @code{test-op},
1130you want to test the system, even if you have already done so.
1131
1132The results of this operation are not defined by ASDF.
1133It has proven difficult to define how the test operation
1134should signal its results to the user
1135in a way that is compatible with all of the various test libraries
1136and test techniques in use in the community.
1137@end deffn
1138
1139@c @deffn Operation test-system-version @&key minimum
1140
1141@c Asks the system whether it satisfies a version requirement.
1142
1143@c The default method accepts a string, which is expected to contain of a
1144@c number of integers separated by #\. characters.  The method is not
1145@c recursive.  The component satisfies the version dependency if it has
1146@c the same major number as required and each of its sub-versions is
1147@c greater than or equal to the sub-version number required.
1148
1149@c @lisp
1150@c (defun version-satisfies (x y)
1151@c   (labels ((bigger (x y)
1152@c           (cond ((not y) t)
1153@c                 ((not x) nil)
1154@c                 ((> (car x) (car y)) t)
1155@c                 ((= (car x) (car y))
1156@c                  (bigger (cdr x) (cdr y))))))
1157@c     (and (= (car x) (car y))
1158@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1159@c @end lisp
1160
1161@c If that doesn't work for your system, you can override it.  I hope
1162@c you have as much fun writing the new method as @verb{|#lisp|} did
1163@c reimplementing this one.
1164@c @end deffn
1165
1166@c @deffn Operation feature-dependent-op
1167
1168@c An instance of @code{feature-dependent-op} will ignore any components
1169@c which have a @code{features} attribute, unless the feature combination
1170@c it designates is satisfied by @code{*features*}.  This operation is
1171@c not intended to be instantiated directly, but other operations may
1172@c inherit from it.
1173
1174@c @end deffn
1175
1176@node  Creating new operations,  , Predefined operations of ASDF, Operations
1177@comment  node-name,  next,  previous,  up
1178@subsection Creating new operations
1179
1180ASDF was designed to be extensible in an object-oriented fashion.
1181To teach ASDF new tricks, a programmer can implement the behaviour he wants
1182by creating a subclass of @code{operation}.
1183
1184ASDF's pre-defined operations are in no way ``privileged'',
1185but it is requested that developers never use the @code{asdf} package
1186for operations they develop themselves.
1187The rationale for this rule is that we don't want to establish a
1188``global asdf operation name registry'',
1189but also want to avoid name clashes.
1190
1191An operation must provide methods for the following generic functions
1192when invoked with an object of type @code{source-file}:
1193@emph{FIXME describe this better}
1194
1195@itemize
1196
1197@item @code{output-files}
1198The @code{output-files} method determines where the method will put its files.
1199It returns two values, a list of pathnames, and a boolean.
1200If the boolean is @code{T} then the pathnames are marked
1201not be translated by enclosing @code{:around} methods.
1202If the boolean is @code{NIL} then enclosing @code{:around} methods
1203may translate these pathnames, e.g. to ensure object files
1204are somehow stored in some implementation-dependent cache.
1205@item @code{perform}
1206The @code{perform} method must call @code{output-files}
1207to find out where to put its files,
1208because the user is allowed to override.
1209@item @code{output-files}
1210for local policy @code{explain}
1211@item @code{operation-done-p},
1212if you don't like the default one
1213
1214@end itemize
1215
1216Operations that print output should send that output to the standard
1217CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1218
1219@node Components,  , Operations, The object model of ASDF
1220@comment  node-name,  next,  previous,  up
1221@section Components
1222@cindex component
1223@cindex system
1224@cindex system designator
1225@vindex *system-definition-search-functions*
1226
1227A @dfn{component} represents a source file or
1228(recursively) a collection of components.
1229A @dfn{system} is (roughly speaking) a top-level component
1230that can be found via @code{find-system}.
1231
1232A @dfn{system designator} is a string or symbol
1233and behaves just like any other component name
1234(including with regard to the case conversion rules for component names).
1235
1236
1237@defun find-system system-designator &optional (error-p t)
1238
1239Given a system designator, @code{find-system} finds and returns a system.
1240If no system is found, an error of type
1241@code{missing-component} is thrown,
1242or @code{nil} is returned if @code{error-p} is false.
1243
1244To find and update systems, @code{find-system} funcalls each element
1245in the @code{*system-definition-search-functions*} list,
1246expecting a pathname to be returned.
1247The resulting pathname is loaded if either of the following conditions is true:
1248
1249@itemize
1250@item
1251there is no system of that name in memory
1252@item
1253the file's @code{last-modified} time exceeds the @code{last-modified} time
1254of the system in memory
1255@end itemize
1256
1257When system definitions are loaded from @file{.asd} files,
1258a new scratch package is created for them to load into,
1259so that different systems do not overwrite each others operations.
1260The user may also wish to (and is recommended to)
1261include @code{defpackage} and @code{in-package} forms
1262in his system definition files, however,
1263so that they can be loaded manually if need be.
1264
1265The default value of @code{*system-definition-search-functions*}
1266is a list of two functions.
1267The first function looks in each of the directories given
1268by evaluating members of @code{*central-registry*}
1269for a file whose name is the name of the system and whose type is @file{asd}.
1270The first such file is returned,
1271whether or not it turns out to actually define the appropriate system.
1272The second function does something similar,
1273for the directories specified in the @code{source-registry}.
1274Hence, it is strongly advised to define a system
1275@var{foo} in the corresponding file @var{foo.asd}.
1276@end defun
1277
1278
1279@menu
1280* Common attributes of components::
1281* Pre-defined subclasses of component::
1282* Creating new component types::
1283@end menu
1284
1285@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1286@comment  node-name,  next,  previous,  up
1287@subsection Common attributes of components
1288
1289All components, regardless of type, have the following attributes.
1290All attributes except @code{name} are optional.
1291
1292@subsubsection Name
1293
1294A component name is a string or a symbol.
1295If a symbol, its name is taken and lowercased.
1296
1297Unless overridden by a @code{:pathname} attribute,
1298the name will be interpreted as a pathname specifier according
1299to a Unix-style syntax.
1300@xref{The defsystem grammar,,Pathname specifiers}.
1301
1302@subsubsection Version identifier
1303
1304This optional attribute is used by the @code{test-system-version} operation.
1305@xref{Predefined operations of ASDF}.
1306For the default method of @code{test-system-version},
1307the version should be a string of integers separated by dots,
1308for example @samp{1.0.11}.
1309
1310@emph{Nota Bene}:
1311This operation, planned for ASDF 1,
1312is still not implement yet as of ASDF 2.
1313Don't hold your breath.
1314
1315
1316@subsubsection Required features
1317
1318@emph{FIXME: This subsection seems to contradict the
1319@code{defsystem} grammar subsection,
1320which doesn't provide any obvious way to specify required features.
1321Furthermore, in 2009, discussions on the
1322@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1323suggested that the specification of required features may be broken,
1324and that no one may have been using them for a while.
1325Please contact the
1326@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1327if you are interested in getting this features feature fixed.}
1328
1329Traditionally defsystem users have used reader conditionals
1330to include or exclude specific per-implementation files.
1331This means that any single implementation cannot read the entire system,
1332which becomes a problem if it doesn't wish to compile it,
1333but instead for example to create an archive file containing all the sources,
1334as it will omit to process the system-dependent sources for other systems.
1335
1336Each component in an asdf system may therefore specify features using
1337the same syntax as @code{#+} does, and it will (somehow) be ignored for
1338certain operations unless the feature conditional is a member of
1339@code{*features*}.
1340
1341
1342@subsubsection Dependencies
1343
1344This attribute specifies dependencies of the component on its siblings.
1345It is optional but often necessary.
1346
1347There is an excitingly complicated relationship between the initarg
1348and the method that you use to ask about dependencies
1349
1350Dependencies are between (operation component) pairs.
1351In your initargs for the component, you can say
1352
1353@lisp
1354:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1355              (load-op (load-op "foo")))
1356@end lisp
1357
1358This means the following things:
1359@itemize
1360@item
1361before performing compile-op on this component, we must perform
1362load-op on @var{a} and @var{b}, and compile-op on @var{c},
1363@item
1364before performing @code{load-op}, we have to load @var{foo}
1365@end itemize
1366
1367The syntax is approximately
1368
1369@verbatim
1370(this-op {(other-op required-components)}+)
1371
1372required-components := component-name
1373                     | (required-components required-components)
1374
1375component-name := string
1376                | (:version string minimum-version-object)
1377@end verbatim
1378
1379Side note:
1380
1381This is on a par with what ACL defsystem does.
1382mk-defsystem is less general: it has an implied dependency
1383
1384@verbatim
1385  for all x, (load x) depends on (compile x)
1386@end verbatim
1387
1388and using a @code{:depends-on} argument to say that @var{b} depends on
1389@var{a} @emph{actually} means that
1390
1391@verbatim
1392  (compile b) depends on (load a)
1393@end verbatim
1394
1395This is insufficient for e.g. the McCLIM system, which requires that
1396all the files are loaded before any of them can be compiled ]
1397
1398End side note
1399
1400In ASDF, the dependency information for a given component and operation
1401can be queried using @code{(component-depends-on operation component)},
1402which returns a list
1403
1404@lisp
1405((load-op "a") (load-op "b") (compile-op "c") ...)
1406@end lisp
1407
1408@code{component-depends-on} can be subclassed for more specific
1409component/operation types: these need to @code{(call-next-method)}
1410and append the answer to their dependency, unless
1411they have a good reason for completely overriding the default dependencies.
1412
1413If it weren't for CLISP, we'd be using @code{LIST} method
1414combination to do this transparently.
1415But, we need to support CLISP.
1416If you have the time for some CLISP hacking,
1417I'm sure they'd welcome your fixes.
1418@c Doesn't CLISP now support LIST method combination?
1419
1420@subsubsection pathname
1421
1422This attribute is optional and if absent (which is the usual case),
1423the component name will be used.
1424
1425@xref{The defsystem grammar,,Pathname specifiers},
1426for an explanation of how this attribute is interpreted.
1427
1428Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1429does additional processing to set the filesystem location of
1430the top component in that system.
1431This is detailed elsewhere. @xref{Defining systems with defsystem}.
1432
1433The answer to the frequently asked question
1434``how do I create a system definition
1435where all the source files have a @file{.cl} extension''
1436is thus
1437
1438@lisp
1439(defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys))))
1440  "cl")
1441@end lisp
1442
1443@subsubsection properties
1444
1445This attribute is optional.
1446
1447Packaging systems often require information about files or systems
1448in addition to that specified by ASDF's pre-defined component attributes.
1449Programs that create vendor packages out of ASDF systems therefore
1450have to create ``placeholder'' information to satisfy these systems.
1451Sometimes the creator of an ASDF system may know the additional
1452information and wish to provide it directly.
1453
1454@code{(component-property component property-name)} and
1455associated @code{setf} method will allow
1456the programmatic update of this information.
1457Property names are compared as if by @code{EQL},
1458so use symbols or keywords or something.
1459
1460@menu
1461* Pre-defined subclasses of component::
1462* Creating new component types::
1463@end menu
1464
1465@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1466@comment  node-name,  next,  previous,  up
1467@subsection Pre-defined subclasses of component
1468
1469@deffn Component source-file
1470
1471A source file is any file that the system does not know how to
1472generate from other components of the system.
1473
1474Note that this is not necessarily the same thing as
1475``a file containing data that is typically fed to a compiler''.
1476If a file is generated by some pre-processor stage
1477(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1478then it is not, by this definition, a source file.
1479Conversely, we might have a graphic file
1480that cannot be automatically regenerated,
1481or a proprietary shared library that we received as a binary:
1482these do count as source files for our purposes.
1483
1484Subclasses of source-file exist for various languages.
1485@emph{FIXME: describe these.}
1486@end deffn
1487
1488@deffn Component module
1489
1490A module is a collection of sub-components.
1491
1492A module component has the following extra initargs:
1493
1494@itemize
1495@item
1496@code{:components} the components contained in this module
1497
1498@item
1499@code{:default-component-class}
1500All children components which don't specify their class explicitly
1501are inferred to be of this type.
1502
1503@item
1504@code{:if-component-dep-fails}
1505This attribute takes one of the values
1506@code{:fail}, @code{:try-next}, @code{:ignore},
1507its default value is @code{:fail}.
1508The other values can be used for implementing conditional compilation
1509based on implementation @code{*features*},
1510for the case where it is not necessary for all files in a module to be
1511compiled.
1512@emph{FIXME: such conditional compilation has been reported
1513to be broken in 2009.}
1514
1515@item
1516@code{:serial} When this attribute is set,
1517each subcomponent of this component is assumed to depend on all subcomponents
1518before it in the list given to @code{:components}, i.e.
1519all of them are loaded before a compile or load operation is performed on it.
1520
1521@end itemize
1522
1523The default operation knows how to traverse a module, so
1524most operations will not need to provide methods specialised on modules.
1525
1526@code{module} may be subclassed to represent components such as
1527foreign-language linked libraries or archive files.
1528@end deffn
1529
1530@deffn Component system
1531
1532@code{system} is a subclass of @code{module}.
1533
1534A system is a module with a few extra attributes for documentation
1535purposes; these are given elsewhere.
1536@xref{The defsystem grammar}.
1537
1538Users can create new classes for their systems:
1539the default @code{defsystem} macro takes a @code{:class} keyword argument.
1540@end deffn
1541
1542@node  Creating new component types,  , Pre-defined subclasses of component, Components
1543@comment  node-name,  next,  previous,  up
1544@subsection Creating new component types
1545
1546New component types are defined by subclassing one of the existing
1547component classes and specializing methods on the new component class.
1548
1549@emph{FIXME: this should perhaps be explained more throughly,
1550not only by example ...}
1551
1552As an example, suppose we have some implementation-dependent
1553functionality that we want to isolate
1554in one subdirectory per Lisp implementation our system supports.
1555We create a subclass of
1556@code{cl-source-file}:
1557
1558@lisp
1559(defclass unportable-cl-source-file (cl-source-file)
1560  ())
1561@end lisp
1562
1563A hypothetical function @code{system-dependent-dirname}
1564gives us the name of the subdirectory.
1565All that's left is to define how to calculate the pathname
1566of an @code{unportable-cl-source-file}.
1567
1568@lisp
1569(defmethod component-pathname ((component unportable-cl-source-file))
1570  (let ((pathname (call-next-method))
1571        (name (string-downcase (system-dependent-dirname))))
1572    (merge-pathnames*
1573     (make-pathname :directory (list :relative name))
1574     pathname)))
1575@end lisp
1576
1577The new component type is used in a @code{defsystem} form in this way:
1578
1579@lisp
1580(defsystem :foo
1581    :components
1582    ((:file "packages")
1583     ...
1584     (:unportable-cl-source-file "threads"
1585      :depends-on ("packages" ...))
1586     ...
1587    )
1588@end lisp
1589
1590@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1591@comment  node-name,  next,  previous,  up
1592@chapter Controlling where ASDF searches for systems
1593
1594@section Configurations
1595
1596Configurations specify paths where to find system files.
1597
1598@enumerate
1599
1600@item
1601The search registry may use some hardcoded wrapping registry specification.
1602This allows some implementations (notably SBCL) to specify where to find
1603some special implementation-provided systems that
1604need to precisely match the version of the implementation itself.
1605
1606@item
1607An application may explicitly initialize the source-registry configuration
1608using the configuration API
1609(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1610in which case this takes precedence.
1611It may itself compute this configuration from the command-line,
1612from a script, from its own configuration file, etc.
1613
1614@item
1615The source registry will be configured from
1616the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1617
1618@item
1619The source registry will be configured from
1620user configuration file
1621@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1622(which defaults to
1623@file{~/.config/common-lisp/source-registry.conf})
1624if it exists.
1625
1626@item
1627The source registry will be configured from
1628user configuration directory
1629@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1630(which defaults to
1631@file{~/.config/common-lisp/source-registry.conf.d/})
1632if it exists.
1633
1634@item
1635The source registry will be configured from
1636system configuration file
1637@file{/etc/common-lisp/source-registry.conf}
1638if it exists/
1639
1640@item
1641The source registry will be configured from
1642system configuration directory
1643@file{/etc/common-lisp/source-registry.conf.d/}
1644if it exists.
1645
1646@item
1647The source registry will be configured from a default configuration.
1648This configuration may allow for implementation-specific systems
1649to be found, for systems to be found the current directory
1650(at the time that the configuration is initialized) as well as
1651@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1652@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1653
1654@end enumerate
1655
1656Each of these configuration is specified as a SEXP
1657in a trival domain-specific language (defined below).
1658Additionally, a more shell-friendly syntax is available
1659for the environment variable (defined yet below).
1660
1661Each of these configurations is only used if the previous
1662configuration explicitly or implicitly specifies that it
1663includes its inherited configuration.
1664
1665Additionally, some implementation-specific directories
1666may be automatically prepended to whatever directories are specified
1667in configuration files, no matter if the last one inherits or not.
1668
1669@section XDG base directory
1670
1671Note that we purport to respect the XDG base directory specification
1672as to where configuration files are located,
1673where data files are located,
1674where output file caches are located.
1675Mentions of XDG variables refer to that document.
1676
1677@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1678
1679This specification allows the user to specify some environment variables
1680to customize how applications behave to his preferences.
1681
1682On Windows platforms, when not using Cygwin,
1683instead of the XDG base directory specification,
1684we try to use folder configuration from the registry regarding
1685@code{Common AppData} and similar directories.
1686However, support querying the Windows registry is limited as of ASDF 2,
1687and on many implementations, we may fall back to always using the defaults
1688without consulting the registry.
1689Patches welcome.
1690
1691@section Backward Compatibility
1692
1693For backward compatibility as well as for a practical backdoor for hackers,
1694ASDF will first search for @code{.asd} files in the directories specified in
1695@code{asdf:*central-registry*}
1696before it searches in the source registry above.
1697
1698@xref{Configuring ASDF,,Configuring ASDF to find your systems -- old style}.
1699
1700By default, @code{asdf:*central-registry*} will be empty.
1701
1702This old mechanism will therefore not affect you if you don't use it,
1703but will take precedence over the new mechanism if you do use it.
1704
1705@section Configuration DSL
1706
1707Here is the grammar of the SEXP DSL for source-registry configuration:
1708
1709@example
1710;; A configuration is single SEXP starting with keyword :source-registry
1711;; followed by a list of directives.
1712CONFIGURATION := (:source-registry DIRECTIVE ...)
1713
1714;; A directive is one of the following:
1715DIRECTIVE :=
1716    ;; add a single directory to be scanned (no recursion)
1717    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1718
1719    ;; add a directory hierarchy, recursing but excluding specified patterns
1720    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1721
1722    ;; override the default defaults for exclusion patterns
1723    (:exclude PATTERN ...) |
1724
1725    ;; splice the parsed contents of another config file
1726    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1727
1728    ;; Your configuration expression MUST contain
1729    ;; exactly one of either of these:
1730    :inherit-configuration | ; splices contents of inherited configuration
1731    :ignore-inherited-configuration | ; drop contents of inherited configuration
1732
1733    ;; This directive specifies that some default must be spliced.
1734    :default-registry
1735
1736PATTERN := a string without wildcards, that will be matched exactly
1737  against the name of a any subdirectory in the directory component
1738        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
1739@end example
1740
1741
1742@section Configuration Directories
1743
1744Configuration directories consist in files each contains
1745a list of directives without any enclosing @code{(:source-registry ...)} form.
1746The files will be sorted by namestring as if by @code{string<} and
1747the lists of directives of these files with be concatenated in order.
1748An implicit @code{:inherit-configuration} will be included
1749at the end of the list.
1750
1751This allows for packaging software that has file granularity
1752(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
1753to easily include configuration information about distributed software.
1754
1755The convention is that, for sorting purposes,
1756the names of files in such a directory begin with two digits
1757that determine the order in which these entries will be read.
1758Also, the type of these files is conventionally @code{"conf"}
1759and as a limitation to some implementations (e.g. GNU clisp),
1760the type cannot be @code{NIL}.
1761
1762Directories may be included by specifying a directory pathname
1763or namestring in an @code{:include} directive, e.g.:
1764
1765@example
1766  (:include "/foo/bar/")
1767@end example
1768
1769
1770@section Shell-friendly syntax for configuration
1771
1772When considering environment variable @code{CL_SOURCE_REGISTRY}
1773ASDF will skip to next configuration if it's an empty string.
1774It will @code{READ} the string as a SEXP in the DSL
1775if it begins with a paren @code{(}
1776and it will be interpreted much like @code{TEXINPUTS}
1777list of paths, where
1778
1779  * paths are separated
1780   by a @code{:} (colon) on Unix platforms (including cygwin),
1781   by a @code{;} (semicolon) on other platforms (mainly, Windows).
1782
1783  * each entry is a directory to add to the search path.
1784
1785  * if the entry ends with a double slash @code{//}
1786    then it instead indicates a tree in the subdirectories
1787    of which to recurse.
1788
1789  * if the entry is the empty string (which may only appear once),
1790    then it indicates that the inherited configuration should be
1791    spliced there.
1792
1793
1794@section Search Algorithm
1795
1796In case that isn't clear, the semantics of the configuration is that
1797when searching for a system of a given name,
1798directives are processed in order.
1799
1800When looking in a directory, if the system is found, the search succeeds,
1801otherwise it continues.
1802
1803When looking in a tree, if one system is found, the search succeeds.
1804If multiple systems are found, the consequences are unspecified:
1805the search may succeed with any of the found systems,
1806or an error may be raised.
1807ASDF currently returns the first system found,
1808XCVB currently raised an error.
1809If none is found, the search continues.
1810
1811Exclude statements specify patterns of subdirectories the systems of which
1812to ignore. Typically you don't want to use copies of files kept by such
1813version control systems as Darcs.
1814
1815Include statements cause the search to recurse with the path specifications
1816from the file specified.
1817
1818An inherit-configuration statement cause the search to recurse with the path
1819specifications from the next configuration
1820(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
1821
1822
1823@section Caching Results
1824
1825The implementation is allowed to either eagerly compute the information
1826from the configurations and file system, or to lazily re-compute it
1827every time, or to cache any part of it as it goes.
1828To explicitly flush any information cached by the system, use the API below.
1829
1830
1831@section Configuration API
1832
1833The specified functions are exported from your build system's package.
1834Thus for ASDF the corresponding functions are in package ASDF,
1835and for XCVB the corresponding functions are in package XCVB.
1836
1837@defun initialize-source-registry @&optional PARAMETER
1838   will read the configuration and initialize all internal variables.
1839   You may extend or override configuration
1840   from the environment and configuration files
1841   with the given @var{PARAMETER}, which can be
1842   @code{NIL} (no configuration override),
1843   or a SEXP (in the SEXP DSL),
1844   a string (as in the string DSL),
1845   a pathname (of a file or directory with configuration),
1846   or a symbol (fbound to function that when called returns one of the above).
1847@end defun
1848
1849@defun clear-source-registry
1850   undoes any source registry configuration
1851   and clears any cache for the search algorithm.
1852   You might want to call that before you
1853   dump an image that would be resumed with a different configuration,
1854   and return an empty configuration.
1855   Note that this does not include clearing information about
1856   systems defined in the current image, only about
1857   where to look for systems not yet defined.
1858@end defun
1859
1860@defun ensure-source-registry @&optional PARAMETER
1861   checks whether a source registry has been initialized.
1862   If not, initialize it with the given @var{PARAMETER}.
1863@end defun
1864
1865
1866@section Future
1867
1868If this mechanism is successful, in the future, we may declare
1869@code{asdf:*central-registry*} obsolete and eventually remove it.
1870Any hook into implementation-specific search mechanisms will by then
1871have been integrated in the @code{:default-configuration} which everyone
1872should either explicitly use or implicit inherit. Some shell syntax
1873for it should probably be added somehow.
1874
1875But we're not there yet. For now, let's see how practical this new
1876source-registry is.
1877
1878
1879@section Rejected ideas
1880
1881Alternatives I considered and rejected included:
1882
1883@enumerate
1884@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
1885   and somehow the configuration parser expands the new configuration
1886   language into a expanded series of directories of subdirectories to
1887   lookup, pre-recursing through specified hierarchies. This is kludgy,
1888   and leaves little space of future cleanups and extensions.
1889
1890@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
1891   in completely new ways, so that new kinds of entries may be implemented
1892   as a recursive search, etc. This seems somewhat backwards.
1893
1894@item Completely remove @code{asdf:*central-registry*}
1895   and break backwards compatibility.
1896   Hopefully this will happen in a few years after everyone migrate to
1897   a better ASDF and/or to XCVB, but it would be very bad to do it now.
1898
1899@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
1900   when you dereference it or setf it. Only the new variable with new
1901   semantics is handled by the new search procedure.
1902   Complex and still introduces subtle semantic issues.
1903@end enumerate
1904
1905
1906I've been suggested the below features, but have rejected them,
1907for the sake of keeping ASDF no more complex than strictly necessary.
1908
1909@itemize
1910@item
1911  More syntactic sugar: synonyms for the configuration directives, such as
1912  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
1913  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
1914
1915@item
1916   The possibility to register individual files instead of directories.
1917
1918@item
1919  Integrate Xach Beane's tilde expander into the parser,
1920  or something similar that is shell-friendly or shell-compatible.
1921  I'd rather keep ASDF minimal. But maybe this precisely keeps it
1922  minimal by removing the need for evaluated entries that ASDF has?
1923  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
1924  Hopefully, these are already superseded by the @code{:default-registry}
1925
1926@item
1927  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
1928  down a filesystem tree in the environment variable.
1929  It isn't that Lisp friendly either.
1930@end itemize
1931
1932@section TODO
1933
1934@itemize
1935@item Add examples
1936@end itemize
1937
1938
1939@section Credits for the source-registry
1940
1941Thanks a lot to Stelian Ionescu for the initial idea.
1942
1943Thanks to Rommel Martinez for the initial implementation attempt.
1944
1945All bad design ideas and implementation bugs are to mine, not theirs.
1946But so are good design ideas and elegant implementation tricks.
1947
1948 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
1949
1950
1951
1952@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
1953@comment  node-name,  next,  previous,  up
1954@chapter Controlling where ASDF saves compiled files
1955@cindex asdf-output-translations
1956@vindex ASDF_OUTPUT_TRANSLATIONS
1957
1958Each Common Lisp implementation has its own format
1959for compiled files (fasls for short, short for ``fast loading'').
1960If you use multiple implementations
1961(or multiple versions of the same implementation),
1962you'll soon find your source directories
1963littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
1964Worse yet, some implementations use the same file extension
1965while changing formats from version to version (or platform to platform)
1966which means that you'll have to recompile binaries
1967as you switch from one implementation to the next.
1968
1969ASDF 2 includes the @code{asdf-output-translations} facility
1970to mitigate the problem.
1971
1972@section Configurations
1973
1974Configurations specify mappings from input locations to output locations.
1975Once again we rely on the XDG base directory specification for configuration.
1976@xref{Controlling where ASDF searches for systems,,XDG base directory}.
1977
1978@enumerate
1979
1980@item
1981Some hardcoded wrapping output translations configuration may be used.
1982This allows special output translations (or usually, invariant directories)
1983to be specified corresponding to the similar special entries in the source registry.
1984
1985@item
1986An application may explicitly initialize the output-translations
1987configuration using the Configuration API
1988in which case this takes precedence.
1989(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
1990It may itself compute this configuration from the command-line,
1991from a script, from its own configuration file, etc.
1992
1993@item
1994The source registry will be configured from
1995the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
1996
1997@item
1998The source registry will be configured from
1999user configuration file
2000@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2001(which defaults to
2002@file{~/.config/common-lisp/asdf-output-translations.conf})
2003if it exists.
2004
2005@item
2006The source registry will be configured from
2007user configuration directory
2008@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2009(which defaults to
2010@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2011if it exists.
2012
2013@item
2014The source registry will be configured from
2015system configuration file
2016@file{/etc/common-lisp/asdf-output-translations.conf}
2017if it exists.
2018
2019@item
2020The source registry will be configured from
2021system configuration directory
2022@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2023if it exists.
2024
2025@end enumerate
2026
2027Each of these configurations is specified as a SEXP
2028in a trival domain-specific language (defined below).
2029Additionally, a more shell-friendly syntax is available
2030for the environment variable (defined yet below).
2031
2032Each of these configurations is only used if the previous
2033configuration explicitly or implicitly specifies that it
2034includes its inherited configuration.
2035
2036Note that by default, a per-user cache is used for output files.
2037This allows the seamless use of shared installations of software
2038between several users, and takes files out of the way of the developers
2039when they browse source code,
2040at the expense of taking a small toll when developers have to clean up
2041output files and find they need to get familiar with output-translations first.
2042
2043
2044@section Backward Compatibility
2045
2046@c FIXME -- I think we should provide an easy way
2047@c to get behavior equivalent to A-B-L and
2048@c I will propose a technique for doing this.
2049
2050We purposefully do NOT provide backward compatibility with earlier versions of
2051@code{ASDF-Binary-Locations} (8 Sept 2009),
2052@code{common-lisp-controller} (7.0) or
2053@code{cl-launch} (2.35),
2054each of which had similar general capabilities.
2055The previous APIs of these programs were not designed
2056for configuration by the end-user
2057in an easy way with configuration files.
2058Recent versions of same packages use
2059the new @code{asdf-output-translations} API as defined below:
2060@code{common-lisp-controller} (7.1) and @code{cl-launch} (3.00);
2061@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2062
2063This incompatibility shouldn't inconvenience many people.
2064Indeed, few people use and customize these packages;
2065these few people are experts who can trivially adapt to the new configuration.
2066Most people are not experts, could not properly configure these features
2067(except inasmuch as the default configuration of
2068@code{common-lisp-controller} and/or @code{cl-launch}
2069might have been doing the right thing for some users),
2070and yet will experience software that ``just works'',
2071as configured by the system distributor, or by default.
2072
2073Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2074we provide a limited emulation mode:
2075
2076@defun asdf:enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2077This function will initialize the new @code{asdf-output-translations} facility in a way
2078that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2079Where you would previously set global variables
2080@var{*centralize-lisp-binaries*},
2081@var{*default-toplevel-directory*},
2082@var{*include-per-user-information*},
2083@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2084you will now have to pass the same values as keyword arguments to this function.
2085Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2086will accept any valid pathname designator for @code{asdf-output-translations}
2087instead of just strings and pathnames.
2088@end defun
2089
2090If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2091(the one available as an extension to load of top of ASDF,
2092not the one built into a few old versions of ASDF),
2093but first you must disable @code{asdf-output-translations}
2094with @code{(asdf:disable-output-translations)},
2095or you might experience ``interesting'' issues.
2096
2097Also, note that output translation is enabled by default.
2098To disable it, use @code{(asdf:disable-output-translations)}.
2099
2100
2101@section Configuration DSL
2102
2103Here is the grammar of the SEXP DSL
2104for @code{asdf-output-translations} configuration:
2105
2106@verbatim
2107;; A configuration is single SEXP starting with keyword :source-registry
2108;; followed by a list of directives.
2109CONFIGURATION := (:output-translations DIRECTIVE ...)
2110
2111;; A directive is one of the following:
2112DIRECTIVE :=
2113    ;; include a configuration file or directory
2114    (:include PATHNAME-DESIGNATOR) |
2115
2116    ;; Your configuration expression MUST contain
2117    ;; exactly one of either of these:
2118    :inherit-configuration | ; splices contents of inherited configuration
2119    :ignore-inherited-configuration | ; drop contents of inherited configuration
2120
2121    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.35-x86-64/ or something.
2122    :enable-user-cache |
2123    ;; Disable global cache. Map / to /
2124    :disable-cache |
2125
2126    ;; add a single directory to be scanned (no recursion)
2127    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2128
2129    ;; use a function to return the translation of a directory designator
2130    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2131
2132DIRECTORY-DESIGNATOR :=
2133    T | ;; as source matches anything, as destination leaves pathname unmapped.
2134    ABSOLUTE-COMPONENT-DESIGNATOR |
2135    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
2136
2137ABSOLUTE-COMPONENT-DESIGNATOR :=
2138    NULL | ;; As source: skip this entry. As destination: same as source
2139    :ROOT | ;; magic: paths that are relative to the root of the source host and device
2140    STRING | ;; namestring (directory is assumed, better be absolute or bust, ``/**/*.*'' added)
2141    PATHNAME | ;; pathname (better be an absolute directory or bust)
2142    :HOME | ;; designates the user-homedir-pathname ~/
2143    :USER-CACHE | ;; designates the default location for the user cache
2144    :SYSTEM-CACHE | ;; designates the default location for the system cache
2145    :CURRENT-DIRECTORY ;; the current directory
2146
2147RELATIVE-COMPONENT-DESIGNATOR :=
2148    STRING | ;; namestring, directory is assumed. If the last component, /**/*.* is added
2149    PATHNAME | ;; pathname unless last component, directory is assumed.
2150    :IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.32.30-linux-x86-64
2151    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2152    :CURRENT-DIRECTORY | ;; all components of the current directory, without the :absolute
2153    :UID | ;; current UID -- not available on Windows
2154    :USER ;; current USER name -- NOT IMPLEMENTED(!)
2155
2156TRANSLATION-FUNCTION :=
2157    SYMBOL | ;; symbol of a function that takes two arguments,
2158             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2159    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2160             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2161
2162@end verbatim
2163
2164Relative components better be either relative
2165or subdirectories of the path before them, or bust.
2166
2167The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2168You can specify more fine-grained patterns
2169by using a pathname object as the last component
2170e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2171
2172You may use @code{#+features} to customize the configuration file.
2173
2174The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2175to anything but themselves (same as if the second designator was the same as the first).
2176
2177When the first designator is @code{t},
2178the mapping always matches.
2179When the first designator starts with @code{:root},
2180the mapping matches any host and device.
2181In either of these cases, if the second designator
2182isn't @code{t} and doesn't start with @code{:root},
2183then strings indicating the host and pathname are somehow copied
2184in the beginning of the directory component of the source pathname
2185before it is translated.
2186
2187When the second designator is @code{t}, the mapping is the identity.
2188When the second designator starts with @code{root},
2189the mapping preserves the host and device of the original pathname.
2190
2191@code{:include} statements cause the search to recurse with the path specifications
2192from the file specified.
2193
2194If the @code{translate-pathname} mechanism cannot achieve a desired
2195translation, the user may provide a function which provides the
2196required algorithim.  Such a translation function is specified by
2197supplying a list as the second @code{directory-designator}
2198the first element of which is the keyword @code{:function},
2199and the second element of which is
2200either a symbol which designates a function or a lambda expression.
2201The function designated by the second argument must take two arguments,
2202the first being the pathname of the source file,
2203the second being the wildcard that was matched.
2204The result of the function invocation should be the translated pathname.
2205
2206An @code{:inherit-configuration} statement cause the search to recurse with the path
2207specifications from the next configuration.
2208@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2209
2210@itemize
2211@item
2212@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2213@item
2214@code{:disable-cache} is the same as @code{(t t)}.
2215@item
2216@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2217which by default is the same as using
2218@code{(:home ".cache" "common-lisp" :implementation)}.
2219@item
2220@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2221which by default is the same as using
2222@code{("/var/cache/common-lisp" :uid :implementation-type)}
2223(on Unix and cygwin), or something semi-sensible on Windows.
2224@end itemize
2225
2226
2227@section Configuration Directories
2228
2229Configuration directories consist in files each contains
2230a list of directives without any enclosing
2231@code{(:output-translations ...)} form.
2232The files will be sorted by namestring as if by @code{string<} and
2233the lists of directives of these files with be concatenated in order.
2234An implicit @code{:inherit-configuration} will be included
2235at the end of the list.
2236
2237This allows for packaging software that has file granularity
2238(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2239to easily include configuration information about software being distributed.
2240
2241The convention is that, for sorting purposes,
2242the names of files in such a directory begin with two digits
2243that determine the order in which these entries will be read.
2244Also, the type of these files is conventionally @code{"conf"}
2245and as a limitation of some implementations, the type cannot be @code{NIL}.
2246
2247Directories may be included by specifying a directory pathname
2248or namestring in an @code{:include} directive, e.g.:
2249@verbatim
2250  (:include "/foo/bar/")
2251@end verbatim
2252
2253@section Shell-friendly syntax for configuration
2254
2255When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2256ASDF will skip to next configuration if it's an empty string.
2257It will @code{READ} the string as an SEXP in the DSL
2258if it begins with a paren @code{(}
2259and it will be interpreted as a list of directories.
2260Directories should come by pairs, indicating a mapping directive.
2261Entries are separated
2262by a @code{:} (colon) on Unix platforms (including cygwin),
2263by a @code{;} (semicolon) on other platforms (mainly, Windows).
2264
2265The magic empty entry,
2266if it comes in what would otherwise be the first entry in a pair,
2267indicates the splicing of inherited configuration.
2268If it comes as the second entry in a pair,
2269it indicates that the directory specified first is to be left untranslated
2270(which has the same effect as if the directory had been repeated).
2271
2272
2273@section Semantics of Output Translations
2274
2275From the specified configuration,
2276a list of mappings is extracted in a straightforward way:
2277mappings are collected in order, recursing through
2278included or inherited configuration as specified.
2279To this list is prepended some implementation-specific mappings,
2280and is appended a global default.
2281
2282The list is then compiled to a mapping table as follows:
2283for each entry, in order, resolve the first designated directory
2284into an actual directory pathname for source locations.
2285If no mapping was specified yet for that location,
2286resolve the second designated directory to an output location directory
2287add a mapping to the table mapping the source location to the output location,
2288and add another mapping from the output location to itself
2289(unless a mapping already exists for the output location).
2290
2291Based on the table, a mapping function is defined,
2292mapping source pathnames to output pathnames:
2293given a source pathname, locate the longest matching prefix
2294in the source column of the mapping table.
2295Replace that prefix by the corresponding output column
2296in the same row of the table, and return the result.
2297If no match is found, return the source pathname.
2298(A global default mapping the filesystem root to itself
2299may ensure that there will always be a match,
2300with same fall-through semantics).
2301
2302@section Caching Results
2303
2304The implementation is allowed to either eagerly compute the information
2305from the configurations and file system, or to lazily re-compute it
2306every time, or to cache any part of it as it goes.
2307To explicitly flush any information cached by the system, use the API below.
2308
2309
2310@section Output location API
2311
2312The specified functions are exported from package ASDF.
2313
2314@defun initialize-output-translations @&optional PARAMETER
2315   will read the configuration and initialize all internal variables.
2316   You may extend or override configuration
2317   from the environment and configuration files
2318   with the given @var{PARAMETER}, which can be
2319   @code{NIL} (no configuration override),
2320   or a SEXP (in the SEXP DSL),
2321   a string (as in the string DSL),
2322   a pathname (of a file or directory with configuration),
2323   or a symbol (fbound to function that when called returns one of the above).
2324@end defun
2325
2326@defun disable-output-translations
2327   will initialize output translations in a way
2328   that maps every pathname to itself,
2329   effectively disabling the output translation facility.
2330@end defun
2331
2332@defun clear-output-translations
2333   undoes any output translation configuration
2334   and clears any cache for the mapping algorithm.
2335   You might want to call that before you
2336   dump an image that would be resumed with a different configuration,
2337   and return an empty configuration.
2338   Note that this does not include clearing information about
2339   systems defined in the current image, only about
2340   where to look for systems not yet defined.
2341@end defun
2342
2343@defun ensure-output-translations @&optional PARAMETER
2344   checks whether output translations have been initialized.
2345   If not, initialize them with the given @var{PARAMETER}.
2346   This function will be called before any attempt to operate on a system.
2347@end defun
2348
2349@defun apply-output-translations PATHNAME
2350   Applies the configured output location translations to @var{PATHNAME}
2351   (calls @code{ensure-output-translations} for the translations).
2352@end defun
2353
2354
2355@section Credits for output translations
2356
2357Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2358and to Peter van Eynde for @code{Common Lisp Controller}.
2359
2360All bad design ideas and implementation bugs are to mine, not theirs.
2361But so are good design ideas and elegant implementation tricks.
2362
2363 --- Francois-Rene Rideau @email{fare@@tunes.org}
2364
2365@c @section Default locations
2366@c @findex output-files-for-system-and-operation
2367
2368@c The default binary location for each Lisp implementation
2369@c is a subdirectory of each source directory.
2370@c To account for different Lisps, Operating Systems, Implementation versions,
2371@c and so on, ASDF borrows code from SLIME
2372@c to create reasonable custom directory names.
2373@c Here are some examples:
2374
2375@c @itemize
2376@c @item
2377@c SBCL, version 1.0 on Mac OS X for intel: @code{sbcl-1.0-darwin-x86}
2378
2379@c @item
2380@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2381
2382@c @item
2383@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2384@c @end itemize
2385
2386@c By default, all output file pathnames will be relocated
2387@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2388
2389@c See the document @file{README.asdf-output-translations}
2390@c for a full specification on how to configure @code{asdf-output-translations}.
2391
2392@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2393@comment  node-name,  next,  previous,  up
2394@chapter Error handling
2395@findex SYSTEM-DEFINITION-ERROR
2396@findex OPERATION-ERROR
2397
2398@section ASDF errors
2399
2400If ASDF detects an incorrect system definition, it will signal a generalised instance of
2401@code{SYSTEM-DEFINITION-ERROR}.
2402
2403Operations may go wrong (for example when source files contain errors).
2404These are signalled using generalised instances of
2405@code{OPERATION-ERROR}.
2406
2407@section Compilation error and warning handling
2408@vindex *compile-file-warnings-behaviour*
2409@vindex *compile-file-errors-behavior*
2410
2411ASDF checks for warnings and errors when a file is compiled.
2412The variables @var{*compile-file-warnings-behaviour*} and
2413@var{*compile-file-errors-behavior*}
2414control the handling of any such events.
2415The valid values for these variables are
2416@code{:error}, @code{:warn}, and @code{:ignore}.
2417
2418@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2419@comment  node-name,  next,  previous,  up
2420@chapter Miscellaneous additional functionality
2421
2422@emph{FIXME:  Add discussion of @code{run-shell-command}?  Others?}
2423
2424ASDF includes several additional features that are generally
2425useful for system definition and development. These include:
2426
2427@defun system-relative-pathname system name @&key type
2428
2429It's often handy to locate a file relative to some system.
2430The @code{system-relative-pathname} function meets this need.
2431It takes two arguments: the name of a system and a relative pathname.
2432It returns a pathname built from the location of the system's source file
2433and the relative pathname. For example
2434
2435@lisp
2436> (asdf:system-relative-pathname 'cl-ppcre #p"regex.data")
2437#P"/repository/other/cl-ppcre/regex.data"
2438@end lisp
2439
2440Instead of a pathname, you can provide a symbol or a string,
2441and optionally a keyword argument @code{type}.
2442The arguments will then be interpreted in the same way
2443as pathname specifiers for components.
2444@xref{The defsystem grammar,,Pathname specifiers}.
2445@end defun
2446
2447@defun system-source-directory system-designator
2448
2449ASDF does not provide a turnkey solution for locating data (or other
2450miscellaneous) files that are distributed together with the source code
2451of a system.  Programmers can use @code{system-source-directory} to find
2452such files.  Returns a pathname object.  The @var{system-designator} may
2453be a string, symbol, or ASDF system object.
2454@end defun
2455
2456
2457@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2458@comment  node-name,  next,  previous,  up
2459@chapter Getting the latest version
2460
2461Decide which version you want.
2462HEAD is the newest version and usually OK, whereas
2463RELEASE is for cautious people
2464(e.g. who already have systems using ASDF that they don't want broken),
2465a slightly older version about which none of the HEAD users have complained.
2466There is also a STABLE version, which is earlier than release.
2467
2468You may get the ASDF source repository using git:
2469@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2470
2471You will find the above referenced tags in this repository.
2472You can also browse the repository on
2473@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2474
2475Discussion of ASDF development is conducted on the
2476mailing list
2477@kbd{asdf-devel@@common-lisp.net}.
2478@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
2479
2480
2481@node FAQ, TODO list, Getting the latest version, Top
2482@comment  node-name,  next,  previous,  up
2483@chapter FAQ
2484
2485@section  ``Where do I report a bug?''
2486
2487ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
2488
2489If you're unsure about whether something is a bug, of for general discussion,
2490use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
2491
2492
2493@section ``What has changed between ASDF 1 and ASDF 2?''
2494
2495@subsection What are ASDF 1 and ASDF 2?
2496
2497We are preparing for a release of ASDF 2,
2498which will have version 2.000 and later.
2499While the code and documentation are essentially complete
2500we are still working on polishing them before release.
2501
2502Releases in the 1.700 series and beyond
2503should be considered as release candidates.
2504For all practical purposes,
2505ASDF 2 refers to releases later than 1.656,
2506and ASDF 1 to any release earlier than 1.369 or so.
2507If your ASDF doesn't have a version, it's old.
2508
2509ASDF 2 release candidates and beyond will have
2510@code{:asdf2} onto @code{*features*} so that if you are writing
2511ASDF-dependent code you may check for this feature
2512to see if the new API is present.
2513@emph{All} versions of ASDF should have the @code{:asdf} feature.
2514
2515If you are experiencing problems or limitations of any sort with ASDF 1,
2516we recommend that you should upgrade to ASDF 2 or its latest release candidate.
2517
2518
2519@subsection ASDF can portably name files in subdirectories
2520
2521Common Lisp namestrings are not portable,
2522except maybe for logical pathnamestrings,
2523that themselves have various limitations and require a lot of setup
2524that is itself ultimately non-portable.
2525
2526In ASDF 1, the only portable ways to refer to pathnames inside systems and components
2527were very awkward, using @code{#.(make-pathname ...)} and
2528@code{#.(merge-pathnames ...)}.
2529Even the above were themselves were inadequate in the general case
2530due to host and device issues, unless horribly complex patterns were used.
2531Plenty of simple cases that looked portable actually weren't,
2532leading to much confusion and greavance.
2533
2534ASDF 2 implements its own portable syntax for strings as pathname specifiers.
2535Naming files within a system definition becomes easy and portable again.
2536@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
2537@code{asdf-utilities:merge-pathnames*},
2538@code{asdf::merge-component-name-type}.
2539
2540@xref{The defsystem grammar,,Pathname specifiers}.
2541
2542
2543@subsection Output translations
2544
2545A popular feature added to ASDF was output pathname translation:
2546@code{asdf-binary-locations}, @code{common-lisp-controller},
2547@code{cl-launch} and other hacks were all implementing it in ways
2548both mutually incompatible and difficult to configure.
2549
2550Output pathname translation is essential to share
2551source directories of portable systems across multiple implementations
2552or variants thereof,
2553or source directories of shared installations of systems across multiple users,
2554or combinations of the above.
2555
2556In ASDF 2, a standard mechanism is provided for that,
2557@code{asdf-output-translations},
2558with sensible defaults, adequate configuration languages,
2559a coherent set of configuration files and hooks,
2560and support for non-Unix platforms.
2561
2562@xref{Controlling where ASDF saves compiled files}.
2563
2564@subsection Source Registry Configuration
2565
2566Configuring ASDF used to require special magic
2567to be applied just at the right moment,
2568between the moment ASDF is loaded and the moment it is used,
2569in a way that is specific to the user,
2570the implementation he is using and the application he is building.
2571
2572This made for awkward configuration files and startup scripts
2573that could not be shared between users, managed by administrators
2574or packaged by distributions.
2575
2576ASDF 2 provides a well-documented way to configure ASDF,
2577with sensible defaults, adequate configuration languages,
2578and a coherent set of configuration files and hooks.
2579
2580We believe it's a vast improvement because it decouples
2581application distribution from library distribution.
2582The application writer can avoid thinking where the libraries are,
2583and the library distributor (dpkg, clbuild, advanced user, etc.)
2584can configure them once and for every application.
2585Yet settings can be easily overridden where needed,
2586so whoever needs control has exactly as much as required.
2587
2588At the same time, ASDF 2 remains compatible
2589with the old magic you may have in your build scripts
2590(using @code{*central-registry*} and
2591@code{*system-definition-search-functions*})
2592to tailor the ASDF configuration to your build automation needs,
2593and also allows for new magic, simpler and more powerful magic.
2594
2595@xref{Controlling where ASDF searches for systems}.
2596
2597
2598@subsection Usual operations are made easier to the user
2599
2600In ASDF 1, you had to use the awkward syntax
2601@code{(asdf:oos 'asdf:load-op :foo)}
2602to load a system,
2603and similarly for @code{compile-op}, @code{test-op}.
2604
2605In ASDF 2, you can use shortcuts for the usual operations:
2606@code{(asdf:load-system :foo)}, and
2607similarly for @code{compile-system}, @code{test-system}.
2608
2609
2610@subsection Many bugs have been fixed
2611
2612The following issues and many others have been fixed:
2613
2614@itemize
2615@item
2616The infamous TRAVERSE function has been revamped significantly,
2617with many bugs squashed.
2618In particular, dependencies were not correctly propagated
2619across submodules within a system but now are.
2620The :version and :feature features and
2621the :force (system1 .. systemN) feature have been fixed.
2622
2623@item
2624Performance has been notably improved for large systems
2625(say with thousands of components) by using
2626hash-tables instead of linear search,
2627and linear-time list accumulation
2628instead of quadratic-time recursive appends.
2629
2630@item
2631Many features used to not be portable,
2632especially where pathnames were involved.
2633Windows support was notably quirky because of such non-portability.
2634
2635@item
2636The internal test suite used to massively fail on many implementations.
2637While still incomplete, it now fully passes
2638on all implementations supported by the test suite.
2639
2640@item
2641Support was lacking for some implementations.
2642ABCL was notably wholly broken.
2643ECL extensions were not integrated in the ASDF release.
2644
2645@item
2646The documentation was grossly out of date.
2647
2648@end itemize
2649
2650
2651@subsection ASDF itself is versioned
2652
2653Between new features, old bugs fixed, and new bugs introduced,
2654there were various releases of ASDF in the wild,
2655and no simple way to check which release had which feature set.
2656People using or writing systems had to either make worst-case assumptions
2657as to what features were available and worked,
2658or take great pains to have the correct version of ASDF installed.
2659
2660With ASDF 2, we provide a new stable set of working features
2661that everyone can rely on from now on.
2662Use @code{#+asdf2} to detect presence of ASDF 2,
2663@code{(asdf:version-satisfies (asdf:asdf-version) "1.711")}
2664to check the availability of a version no earlier than required.
2665
2666
2667@subsection ASDF can be upgraded
2668
2669When an old version of ASDF was loaded,
2670it was very hard to upgrade ASDF in your current image
2671without breaking everything.
2672Instead you have to exit the Lisp process and
2673somehow arrange to start a new one from a simpler image.
2674Something that can't be done from within Lisp,
2675making automation of it difficult,
2676which compounded with difficulty in configuration,
2677made the task quite hard.
2678Yet as we saw before, the task would have been required
2679to not have to live with the worst case or non-portable
2680subset of ASDF features.
2681
2682With ASDF 2, it is easy to upgrade
2683from ASDF 2 to later versions from within Lisp,
2684and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
2685We support hot upgrade of ASDF and any breakage is a bug
2686that we will do our best to fix.
2687There are still limitations on upgrade, though,
2688most notably the fact that after you upgrade ASDF,
2689you must also reload or upgrade all ASDF extensions.
2690
2691@subsection Decoupled release cycle
2692
2693When vendors were releasing their Lisp implementations with ASDF,
2694they had to basically never change version
2695because neither upgrade nor downgrade was possible
2696without breaking something for someone,
2697and no obvious upgrade path was visible and recommendable.
2698
2699With ASDF 2, upgrade is possible, easy and can be recommended.
2700This means that vendors can safely ship a recent version of ASDF,
2701confident that if a user isn't fully satisfied,
2702he can easily upgrade ASDF and deal
2703with a supported recent version of it.
2704This means that release cycles will be causally decoupled,
2705the practical consequence of which will mean faster convergence
2706towards the latest version for everyone.
2707
2708
2709@subsection Pitfalls of ASDF 2
2710
2711The main pitfalls in upgrading to ASDF 2 seem to be related
2712to the output translation mechanism.
2713
2714@itemize
2715
2716@item
2717Output translations is enabled by default. This may surprise some users,
2718most of them in pleasant way (we hope), a few of them in an unpleasant way.
2719It is trivial to disable output translations.
2720@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
2721
2722@item
2723Some systems in the large have been known not to play well with output translations.
2724They were relatively easy to fix.
2725Once again, it is also easy to disable output translations,
2726or to override its configuration.
2727
2728@item
2729The new ASDF output translations are incompatible with ASDF-Binary-Locations.
2730They replace A-B-L, and there is compatibility mode to emulate
2731your previous A-B-L configuration.
2732See @code{asdf:enable-asdf-binary-locations-compatibility} in
2733@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
2734But thou shall not load ABL on top of ASDF 2.
2735
2736@end itemize
2737
2738Other issues include the following:
2739
2740@itemize
2741
2742@item
2743There is a slight performance bug, notably on SBCL,
2744when initially searching for @file{asd} files,
2745the implicit @code{(directory "/configured/path/**/*.asd")}
2746for every configured path @code{(:tree "/configured/path/")}
2747in your @code{source-registry} configuration can cause a slight pause.
2748Try to @code{(time (asdf:initialize-source-registry))}
2749to see how bad it is or isn't on your system.
2750If you insist on not having this pause,
2751you can avoid the pause by overriding the default source-registry configuration
2752and not use any deep @code{:tree} entry but only @code{:directory} entries
2753or shallow @code{:tree} entries.
2754Or you can fix your implementation to not be quite that slow
2755when recursing through directories.
2756
2757@item
2758On Windows, only LispWorks supports proper default configuration pathnames
2759based on the Windows registry.
2760Other implementations make do.
2761Windows support is largely untested, so please help report and fix bugs.
2762
2763@end itemize
2764
2765
2766@section Issues with installing the proper version of ASDF
2767
2768@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
2769
2770We recommend you upgrade ASDF.
2771@xref{Loading ASDF,,Upgrading ASDF}.
2772
2773If this does not work, it is a bug, and you should report it.
2774@xref{FAQ, report-bugs, Where do I report a bug}.
2775In the meantime, you can load @file{asdf.lisp} directly.
2776@xref{Loading ASDF,Loading an otherwise installed ASDF}.
2777
2778
2779@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
2780
2781Starting with current candidate releases of ASDF 2,
2782it should always be a good time to upgrade to a recent version of ASDF.
2783You may consult with the maintainer for which specific version they recommend,
2784but the latest RELEASE should be correct.
2785We trust you to thoroughly test it with your implementation before you release it.
2786If there are any issues with the current release,
2787it's a bug that you should report upstream and that we will fix ASAP.
2788
2789As to how to include ASDF, we recommend the following:
2790
2791@itemize
2792@item
2793If ASDF isn't installed yet, then @code{(require :asdf)}
2794should load the version of ASDF that is bundled with your system.
2795You may have it load some other version configured by the user,
2796if you allow such configuration.
2797
2798@item
2799If your system provides a mechanism to hook into @code{CL:REQUIRE},
2800then it would be nice to add ASDF to this hook the same way that
2801ABCL, CCL, CMUCL, ECL and SBCL do it.
2802
2803@item
2804You may, like SBCL, have ASDF be implicitly used to require systems
2805that are bundled with your Lisp distribution.
2806If you do have a few magic systems that come with your implementation
2807in a precompiled way such that one should only use the binary version
2808that goes with your distribution, like SBCL does,
2809then you should add them in the beginning of @code{wrapping-source-registry}.
2810
2811@item
2812If you have magic systems as above, like SBCL does,
2813then we explicitly ask you to @emph{NOT} distribute
2814@file{asdf.asd} as part of those magic systems.
2815You should still include the file @file{asdf.lisp} in your source distribution
2816and precompile it in your binary distribution,
2817but @file{asdf.asd} if included at all,
2818should be secluded from the magic systems,
2819in a separate file hierarchy,
2820or you may otherwise rename the system and its file to e.g.
2821@code{asdf-ecl} and @file{asdf-ecl.asd}, or
2822@code{sb-asdf} and @file{sb-asdf.asd}.
2823Indeed, if you made @file{asdf.asd} a magic system,
2824then users would no longer be able to upgrade ASDF using ASDF itself
2825to some version of their preference that
2826they maintain independently from your Lisp distribution.
2827
2828@item
2829If you do not have any such magic systems, or have other non-magic systems
2830that you want to bundle with your implementation,
2831then you may add them to the @code{default-source-registry},
2832and you are welcome to include @file{asdf.asd} amongst them.
2833
2834@item
2835Please send us upstream any patches you make to ASDF itself,
2836so we can merge them back in for the benefit of your users
2837when they upgrade to the upstream version.
2838
2839@end itemize
2840
2841
2842
2843@section Issues with configuring ASDF
2844
2845@subsection ``How can I customize where fasl files are stored?''
2846
2847@xref{Controlling where ASDF saves compiled files}.
2848
2849Note that in the past there was an add-on to ASDF called
2850@code{ASDF-binary-locations}, developed by Gary King.
2851That add-on has been merged into ASDF proper,
2852then superseded by the @code{asdf-output-translations} facility.
2853
2854Note that use of @code{asdf-output-translations}
2855can interfere with one aspect of your systems
2856--- if your system uses @code{*load-truename*} to find files
2857(e.g., if you have some data files stored with your program),
2858then the relocation that this ASDF customization performs
2859is likely to interfere.
2860Use @code{asdf:system-relative-pathname} to locate a file
2861in the source directory of some system, and
2862use @code{asdf:apply-output-translations} to locate a file
2863whose pathname has been translated by the facility.
2864
2865@subsection ``How can I wholly disable the compiler output cache?''
2866
2867To permanently disable the compiler output cache
2868for all future runs of ASDF, you can:
2869
2870@example
2871mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
2872echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
2873@end example
2874
2875This assumes that you didn't otherwise configure the ASDF files
2876(if you did, edit them again),
2877and don't somehow override the configuration at runtime
2878with a shell variable (see below) or some other runtime command
2879(e.g. some call to @code{asdf:initialize-output-translations}).
2880
2881To disable the compiler output cache in Lisp processes
2882run by your current shell, try (assuming @code{bash} or @code{zsh})
2883(on Unix and cygwin only):
2884
2885@example
2886export ASDF_OUTPUT_TRANSLATIONS=/:
2887@end example
2888
2889To disable the compiler output cache just in the current Lisp process,
2890use (after loading ASDF but before using it):
2891
2892@example
2893(asdf:disable-output-translations)
2894@end example
2895
2896@section Issues with using and extending ASDF to define systems
2897
2898@subsection ``How can I cater for unit-testing in my system?''
2899
2900ASDF provides a predefined test operation, @code{test-op}.
2901@xref{Predefined operations of ASDF, test-op}.
2902The test operation, however, is largely left to the system definer to specify.
2903@code{test-op} has been
2904a topic of considerable discussion on the
2905@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
2906and on the
2907@url{https://launchpad.net/asdf,launchpad bug-tracker}.
2908
2909Here are some guidelines:
2910
2911@itemize
2912@item
2913For a given system, @var{foo}, you will want to define a corresponding
2914test system, such as @var{foo-test}.  The reason that you will want this
2915separate system is that ASDF does not out of the box supply components
2916that are conditionally loaded.  So if you want to have source files
2917(with the test definitions) that will not be loaded except when testing,
2918they should be put elsewhere.
2919
2920@item
2921The @var{foo-test} system can be defined in an asd file of its own or
2922together with @var{foo}.  An aesthetic preference against cluttering up
2923the filesystem with extra asd files should be balanced against the
2924question of whether one might want to directly load @var{foo-test}.
2925Typically one would not want to do this except in early stages of
2926debugging.
2927
2928@item
2929Record that testing is implemented by @var{foo-test}.  For example:
2930@example
2931(defsystem @var{foo}
2932   :in-order-to ((test-op (test-op @var{foo-test})))
2933   ....)
2934
2935(defsystem @var{foo-test}
2936   :depends-on (@var{foo} @var{my-test-library} ...)
2937   ....)
2938@end example
2939@end itemize
2940
2941This procedure will allow you to support users who do not wish to
2942install your test framework.
2943
2944One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
2945does not return a value.  So in current versions of ASDF there is no
2946reliable programmatic means of determining whether or not a set of tests
2947has passed, or which tests have failed.  The user must simply read the
2948console output.  This limitation has been the subject of much
2949discussion.
2950
2951@subsection ``How can I cater for documentation generation in my system?''
2952
2953The ASDF developers are currently working to add a @code{doc-op}
2954to the set of predefined ASDF operations.
2955@xref{Predefined operations of ASDF}.
2956See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
2957
2958
2959
2960@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
2961
2962See @code{cffi}'s @code{cffi-grovel}.
2963
2964@anchor{report-bugs}
2965
2966
2967@subsection ``I want to put my module's files at the top level.  How do I do this?''
2968
2969By default, the files contained in an asdf module go
2970in a subdirectory with the same name as the module.
2971However, this can be overridden by adding a @code{:pathname ""} argument
2972to the module description.
2973For example, here is how it could be done
2974in the spatial-trees ASDF system definition for ASDF 2:
2975
2976@example
2977(asdf:defsystem :spatial-trees
2978  :components
2979  ((:module base
2980            :pathname ""
2981            :components
2982            ((:file "package")
2983             (:file "basedefs" :depends-on ("package"))
2984             (:file "rectangles" :depends-on ("package"))))
2985   (:module tree-impls
2986            :depends-on (base)
2987            :pathname ""
2988            :components
2989            ((:file "r-trees")
2990             (:file "greene-trees" :depends-on ("r-trees"))
2991             (:file "rstar-trees" :depends-on ("r-trees"))
2992             (:file "rplus-trees" :depends-on ("r-trees"))
2993             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
2994   (:module viz
2995            :depends-on (base)
2996            :pathname ""
2997            :components
2998            ((:static-file "spatial-tree-viz.lisp")))
2999   (:module tests
3000            :depends-on (base)
3001            :pathname ""
3002            :components
3003            ((:static-file "spatial-tree-test.lisp")))
3004   (:static-file "LICENCE")
3005   (:static-file "TODO")))
3006@end example
3007
3008All of the files in the @code{tree-impls} module are at the top level,
3009instead of in a @file{tree-impls/} subdirectory.
3010
3011Note that the argument to @code{:pathname} can be either a pathname object or a string.
3012A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3013but this is discouraged because the results of parsing a namestring are not portable.
3014A pathname can only be portably constructed with such syntax as
3015@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3016and similarly the current directory can only be portably specified as
3017@code{#.(make-pathname :directory '(:relative))}.
3018However, as of ASDF 2, you can portably use a string to denote a pathname.
3019The string will be parsed as a @code{/}-separated path from the current directory,
3020such that the empty string @code{""} denotes the current directory, and
3021@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3022portably denotes the same subdirectory as above.
3023When files are specified, the last @code{/}-separated component is interpreted
3024either as the name component of a pathname
3025(if the component class specifies a pathname type),
3026or as a name component plus optional dot-separated type component
3027(if the component class doesn't specifies a pathname type).
3028
3029
3030@node  TODO list, Inspiration, FAQ, Top
3031@comment  node-name,  next,  previous,  up
3032@chapter TODO list
3033
3034Here is an old list of things to do,
3035in addition to the bugs that are now tracked on launchpad:
3036@url{https://launchpad.net/asdf}.
3037
3038@section Outstanding spec questions, things to add
3039
3040** packaging systems
3041
3042*** manual page component?
3043
3044** style guide for .asd files
3045
3046You should either use keywords or be careful
3047with the package that you evaluate defsystem forms in.
3048Otherwise @code{(defsystem partition ...)}
3049being read in the @code{cl-user} package
3050will intern a @code{cl-user:partition} symbol,
3051which will then collide with the @code{partition:partition} symbol.
3052
3053Actually there's a hairier packages problem to think about too.
3054@code{in-order-to} is not a keyword:
3055if you read @code{defsystem} forms in a package that doesn't use ASDF,
3056odd things might happen.
3057
3058
3059** extending defsystem with new options
3060
3061You might not want to write a whole parser,
3062but just to add options to the existing syntax.
3063Reinstate @code{parse-option} or something akin.
3064
3065
3066** document all the error classes
3067
3068** what to do with compile-file failure
3069
3070Should check the primary return value from compile-file and see if
3071that gets us any closer to a sensible error handling strategy
3072
3073** foreign files
3074
3075lift unix-dso stuff from db-sockets
3076
3077** Diagnostics
3078
3079A ``dry run'' of an operation can be made with the following form:
3080
3081@lisp
3082(traverse (make-instance '<operation-name>)
3083          (find-system <system-name>)
3084          'explain)
3085@end lisp
3086
3087This uses unexported symbols.
3088What would be a nice interface for this functionality?
3089
3090@section Missing bits in implementation
3091
3092** all of the above
3093
3094** reuse the same scratch package whenever a system is reloaded from disk
3095
3096** rules for system pathname defaulting are not yet implemented properly
3097
3098** proclamations probably aren't
3099
3100** when a system is reloaded with fewer components than it previously had, odd things happen
3101
3102We should do something inventive when processing a @code{defsystem} form,
3103like take the list of kids and @code{setf} the slot to @code{nil},
3104then transfer children from old to new list as they're found.
3105
3106**  traverse may become a normal function
3107
3108If you're defining methods on @code{traverse}, speak up.
3109
3110
3111** a lot of load-op methods can be rewritten to use input-files
3112
3113so should be.
3114
3115
3116** (stuff that might happen later)
3117
3118*** Propagation of the @code{:force} option.
3119
3120``I notice that
3121
3122        @code{(asdf:compile-system :araneida :force t)}
3123
3124also forces compilation of every other system the @code{:araneida} system depends on.
3125This is rarely useful to me;
3126usually, when I want to force recompilation of something more than a single source file,
3127I want to recompile only one system.
3128So it would be more useful to have @code{make-sub-operation}
3129refuse to propagate @code{:force t} to other systems, and
3130propagate only something like @code{:force :recursively}.
3131
3132Ideally what we actually want is some kind of criterion that says
3133to which systems (and which operations) a @code{:force} switch will propagate.
3134
3135The problem is perhaps that ``force'' is a pretty meaningless concept.
3136How obvious is it that @code{load :force t} should force @emph{compilation}?
3137But we don't really have the right dependency setup
3138for the user to compile @code{:force t} and expect it to work
3139(files will not be loaded after compilation, so the compile
3140environment for subsequent files will be emptier than it needs to be)
3141
3142What does the user actually want to do when he forces?
3143Usually, for me, update for use with a new version of the Lisp compiler.
3144Perhaps for recovery when he suspects that something has gone wrong.
3145Or else when he's changed compilation options or configuration
3146in some way that's not reflected in the dependency graph.
3147
3148Other possible interface: have a ``revert'' function akin to @code{make clean}.
3149
3150@lisp
3151(asdf:revert 'asdf:compile-op 'araneida)
3152@end lisp
3153
3154would delete any files produced by @code{(compile-system :araneida)}.
3155Of course, it wouldn't be able to do much about stuff in the image itself.
3156
3157How would this work?
3158
3159@code{traverse}
3160
3161There's a difference between a module's dependencies (peers)
3162and its components (children).
3163Perhaps there's a similar difference in operations?
3164For example, @code{(load "use") depends-on (load "macros")} is a peer,
3165whereas @code{(load "use") depends-on (compile "use")}
3166is more of a ``subservient'' relationship.
3167
3168@node  Inspiration, Concept Index, TODO list, Top
3169@comment  node-name,  next,  previous,  up
3170@chapter Inspiration
3171
3172@section mk-defsystem (defsystem-3.x)
3173
3174We aim to solve basically the same problems as @code{mk-defsystem} does.
3175However, our architecture for extensibility
3176better exploits CL language features (and is documented),
3177and we intend to be portable rather than just widely-ported.
3178No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3179that implementation has the unenviable task
3180of supporting pre-ANSI implementations, which is no longer necessary.
3181
3182The surface defsystem syntax of asdf is more-or-less compatible with
3183@code{mk-defsystem}, except that we do not support
3184the @code{source-foo} and @code{binary-foo} prefixes
3185for separating source and binary files, and
3186we advise the removal of all options to specify pathnames.
3187
3188The @code{mk-defsystem} code for topologically sorting
3189a module's dependency list was very useful.
3190
3191@section defsystem-4 proposal
3192
3193Marco and Peter's proposal for defsystem 4 served as the driver for
3194many of the features in here.  Notable differences are:
3195
3196@itemize
3197@item
3198We don't specify output files or output file extensions
3199as part of the system.
3200
3201If you want to find out what files an operation would create,
3202ask the operation.
3203
3204@item
3205We don't deal with CL packages
3206
3207If you want to compile in a particular package, use an @code{in-package} form
3208in that file (ilisp / SLIME will like you more if you do this anyway)
3209
3210@item
3211There is no proposal here that @code{defsystem} does version control.
3212
3213A system has a given version which can be used to check dependencies,
3214but that's all.
3215@end itemize
3216
3217The defsystem 4 proposal tends to look more at the external features,
3218whereas this one centres on a protocol for system introspection.
3219
3220@section kmp's ``The Description of Large Systems'', MIT AI Memu 801
3221
3222Available in updated-for-CL form on the web at
3223@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3224
3225In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3226and concept to deal with creating component trees
3227from @code{defsystem} surface syntax.
3228[ this is not true right now, though it used to be and
3229probably will be again soon ]
3230
3231
3232@c -------------------
3233
3234
3235@node Concept Index, Function and Class Index, Inspiration, Top
3236@unnumbered Concept Index
3237
3238@printindex cp
3239
3240@node Function and Class Index, Variable Index, Concept Index, Top
3241@unnumbered Function and Class Index
3242
3243@printindex fn
3244
3245@node Variable Index,  , Function and Class Index, Top
3246@unnumbered Variable Index
3247
3248@printindex vr
3249
3250@bye
Note: See TracBrowser for help on using the repository browser.