source: trunk/abcl/doc/asdf/asdf.texinfo @ 12765

Last change on this file since 12765 was 12765, checked in by Mark Evenson, 11 years ago

Update to ASDF-2.003 with local patches.

Local patches differentiate output location by FASL and Java version.

File size: 116.5 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2010 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2010 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title asdf: another system definition facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174It hopefully will have been it included
175in all CL maintained implementations shortly afterwards.
176@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
177
178
179@node Loading ASDF, Configuring ASDF, Introduction, Top
180@comment  node-name,  next,  previous,  up
181@chapter Loading ASDF
182@vindex *central-registry*
183@cindex link farm
184@findex load-system
185@findex compile-system
186@findex test-system
187@cindex system directory designator
188@findex operate
189@findex oos
190
191@c @menu
192@c * Installing ASDF::
193@c @end menu
194
195
196@section Loading a pre-installed ASDF
197
198Many Lisp implementations include a copy of ASDF.
199You can usually load this copy using Common Lisp's @code{require} function:
200
201@lisp
202(require :asdf)
203@end lisp
204
205Consult your Lisp implementation's documentation for details.
206
207Hopefully, ASDF 2 will soon be bundled with every Common Lisp implementation,
208and you can load it that way.
209
210
211@section Checking whether ASDF is loaded
212
213To check whether ASDF is properly loaded in your current Lisp image,
214you can run this form:
215
216@lisp
217(asdf:asdf-version)
218@end lisp
219
220If it returns a string,
221that is the version of ASDF that is currently installed.
222
223If it raises an error,
224then either ASDF is not loaded, or
225you are using an old version of ASDF.
226
227You can check whether an old version is loaded
228by checking if the ASDF package is present.
229The form below will allow you to programmatically determine
230whether a recent version is loaded, an old version is loaded,
231or none at all:
232
233@lisp
234(or #+asdf2 (asdf:asdf-version) #+asdf :old)
235@end lisp
236
237If it returns a version number, that's the version of ASDF installed.
238If it returns the keyword @code{:OLD},
239then you're using an old version of ASDF (from before 1.635).
240If it returns @code{NIL} then ASDF is not installed.
241
242If you are running a version older than 2.000,
243we recommend that you load a newer ASDF using the method below.
244
245
246@section Upgrading ASDF
247
248If your implementation does provide ASDF 2 or later,
249and you want to upgrade to a more recent version,
250just install ASDF like any other package
251(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
252configure ASDF as usual (see @pxref{Configuring ASDF} below),
253and upgrade with:
254
255@lisp
256(require :asdf)
257(asdf:load-system :asdf)
258@end lisp
259
260If on the other hand, your implementation only provides an old ASDF,
261you will require a special configuration step and an old-style loading:
262
263@lisp
264(require :asdf)
265(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
266(asdf:oos 'asdf:load-op :asdf)
267@end lisp
268
269Don't forget the trailing @code{/} at the end of your pathname.
270
271Also, note that older versions of ASDF won't redirect their output,
272or at least won't do it according to your usual ASDF 2 configuration.
273You therefore need write access on the directory
274where you install the new ASDF,
275and make sure you're not using it
276for multiple mutually incompatible implementations.
277At worst, you may have to have multiple copies of the new ASDF,
278e.g. one per implementation installation, to avoid clashes.
279
280Finally, note that there are some limitations to upgrading ASDF:
281@itemize
282@item
283Any ASDF extension is invalidated, and will need to be reloaded.
284@item
285It is safer if you upgrade ASDF and its extensions as a special step
286at the very beginning of whatever script you are running,
287before you start using ASDF to load anything else.
288@end itemize
289
290
291@section Loading an otherwise installed ASDF
292
293If your implementation doesn't include ASDF,
294if for some reason the upgrade somehow fails,
295does not or cannot apply to your case,
296you will have to install the file @file{asdf.lisp}
297somewhere and load it with:
298
299@lisp
300(load "/path/to/your/installed/asdf.lisp")
301@end lisp
302
303The single file @file{asdf.lisp} is all you normally need to use ASDF.
304
305You can extract this file from latest release tarball on the
306@url{http://common-lisp.net/project/asdf/,ASDF website}.
307If you are daring and willing to report bugs, you can get
308the latest and greatest version of ASDF from its git repository.
309@xref{Getting the latest version}.
310
311For maximum convenience you might want to have ASDF loaded
312whenever you start your Lisp implementation,
313for example by loading it from the startup script or dumping a custom core
314--- check your Lisp implementation's manual for details.
315
316
317@node Configuring ASDF, Using ASDF, Loading ASDF, Top
318@comment  node-name,  next,  previous,  up
319
320@chapter Configuring ASDF
321
322@section Configuring ASDF to find your systems
323
324So it may compile and load your systems, ASDF must be configured to find
325the @file{.asd} files that contain system definitions.
326
327Since ASDF 2, the preferred way to configure where ASDF finds your systems is
328the @code{source-registry} facility,
329fully described in its own chapter of this manual.
330@xref{Controlling where ASDF searches for systems}.
331
332The default location for a user to install Common Lisp software is under
333@file{~/.local/share/common-lisp/source/}.
334If you install software there, you don't need further configuration.
335If you're installing software yourself at a location that isn't standard,
336you have to tell ASDF where you installed it. See below.
337If you're using some tool to install software,
338the authors of that tool should already have configured ASDF.
339
340The simplest way to add a path to your search path,
341say @file{/home/luser/.asd-link-farm/}
342is to create the directory
343@file{~/.config/common-lisp/source-registry.conf.d/}
344and there create a file with any name of your choice but the type @file{conf},
345for instance @file{42-asd-link-farm.conf}
346containing the line:
347
348@kbd{(:directory "/home/luser/.asd-link-farm/")}
349
350If you want all the subdirectories under @file{/home/luser/lisp/}
351to be recursively scanned for @file{.asd} files, instead use:
352
353@kbd{(:tree "/home/luser/lisp/")}
354
355Note that your Operating System distribution or your system administrator
356may already have configured system-managed libraries for you.
357
358The required @file{.conf} extension allows you to have disabled files
359or editor backups (ending in @file{~}), and works portably
360(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
361Excluded are files the name of which start with a @file{.} character.
362It is customary to start the filename with two digits
363that specify the order in which the directories will be scanned.
364
365ASDF will automatically read your configuration
366the first time you try to find a system.
367You can reset the source-registry configuration with:
368
369@lisp
370(asdf:clear-source-registry)
371@end lisp
372
373And you probably should do so before you dump your Lisp image,
374if the configuration may change
375between the machine where you save it at the time you save it
376and the machine you resume it at the time you resume it.
377
378
379@section Configuring ASDF to find your systems -- old style
380
381The old way to configure ASDF to find your systems is by
382@code{push}ing directory pathnames onto the variable
383@code{asdf:*central-registry*}.
384
385You must configure this variable between the time you load ASDF
386and the time you first try to use it.
387Loading and configuring ASDF presumably happen
388as part of some initialization script that builds or starts
389your Common Lisp software system.
390(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
391
392The @code{asdf:*central-registry*} is empty by default in ASDF 2,
393but is still supported for compatibility with ASDF 1.
394When used, it takes precedence over the above source-registry@footnote{
395It is possible to further customize
396the system definition file search.
397That's considered advanced use, and covered later:
398search forward for
399@code{*system-definition-search-functions*}.
400@xref{Defining systems with defsystem}.}.
401
402For instance, if you wanted ASDF to find the @file{.asd} file
403@file{/home/me/src/foo/foo.asd} your initialization script
404could after it loads ASDF with @code{(require :asdf)}
405configure it with:
406
407@lisp
408(push "/home/me/src/foo/" asdf:*central-registry*)
409@end lisp
410
411Note the trailing slash: when searching for a system,
412ASDF will evaluate each entry of the central registry
413and coerce the result to a pathname@footnote{
414ASDF will indeed call @code{EVAL} on each entry.
415It will also skip entries that evaluate to @code{NIL}.
416
417Strings and pathname objects are self-evaluating,
418in which case the @code{EVAL} step does nothing;
419but you may push arbitrary SEXP onto the central registry,
420that will be evaluated to compute e.g. things that depend
421on the value of shell variables or the identity of the user.
422
423The variable @code{asdf:*central-registry*} is thus a list of
424``system directory designators''.
425A @dfn{system directory designator} is a form
426which will be evaluated whenever a system is to be found,
427and must evaluate to a directory to look in.
428By ``directory'' here, we mean
429``designator for a pathname with a supplied DIRECTORY component''.
430}
431at which point the presence of the trailing directory name separator
432is necessary to tell Lisp that you're discussing a directory
433rather than a file.
434
435Typically, however, there are a lot of @file{.asd} files, and
436a common idiom was to have to put
437a bunch of @emph{symbolic links} to @file{.asd} files
438in a common directory
439and push @emph{that} directory (the ``link farm'')
440to the
441@code{asdf:*central-registry*}
442instead of pushing each of the many involved directories
443to the @code{asdf:*central-registry*}.
444ASDF knows how to follow such @emph{symlinks}
445to the actual file location when resolving the paths of system components
446(on Windows, you can use Windows shortcuts instead of POSIX symlinks).
447
448For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
449is a member of @code{*central-registry*}, you could set up the
450system @var{foo} for loading with asdf with the following
451commands at the shell:
452
453@example
454$ cd /home/me/cl/systems/
455$ ln -s ~/src/foo/foo.asd .
456@end example
457
458This old style for configuring ASDF is not recommended for new users,
459but it is supported for old users, and for users who want to programmatically
460control what directories are added to the ASDF search path.
461
462
463@section Configuring where ASDF stores object files
464@findex clear-output-locations
465
466ASDF lets you configure where object files will be stored.
467Sensible defaults are provided and
468you shouldn't normally have to worry about it.
469
470This allows the same source code repository may be shared
471between several versions of several Common Lisp implementations,
472between several users using different compilation options
473and without write privileges on shared source directories, etc.
474This also allows to keep source directories uncluttered
475by plenty of object files.
476
477Starting with ASDF 2, the @code{asdf-output-translations} facility
478was added to ASDF itself, that controls where object files will be stored.
479This facility is fully described in a chapter of this manual,
480@ref{Controlling where ASDF saves compiled files}.
481
482The simplest way to add a translation to your search path,
483say from @file{/foo/bar/baz/quux/}
484to @file{/where/i/want/my/fasls/}
485is to create the directory
486@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
487and there create a file with any name of your choice and the type @file{conf},
488for instance @file{42-bazquux.conf}
489containing the line:
490
491@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
492
493To disable output translations for source under a given directory,
494say @file{/toto/tata/}
495you can create a file @file{40-disable-toto.conf}
496with the line:
497
498@kbd{("/toto/tata/")}
499
500To wholly disable output translations for all directories,
501you can create a file @file{00-disable.conf}
502with the line:
503
504@kbd{(t t)}
505
506Note that your Operating System distribution or your system administrator
507may already have configured translations for you.
508In absence of any configuration, the default is to redirect everything
509under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
510@xref{Controlling where ASDF searches for systems}, for full details.
511
512The required @file{.conf} extension allows you to have disabled files
513or editor backups (ending in @file{~}), and works portably
514(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
515Excluded are files the name of which start with a @file{.} character.
516It is customary to start the filename with two digits
517that specify the order in which the directories will be scanned.
518
519ASDF will automatically read your configuration
520the first time you try to find a system.
521You can reset the source-registry configuration with:
522
523@lisp
524(asdf:clear-output-translations)
525@end lisp
526
527And you probably should do so before you dump your Lisp image,
528if the configuration may change
529between the machine where you save it at the time you save it
530and the machine you resume it at the time you resume it.
531
532Finally note that before ASDF 2,
533other ASDF add-ons offered the same functionality,
534each in subtly different and incompatible ways:
535ASDF-Binary-Locations, cl-launch, common-lisp-controller.
536ASDF-Binary-Locations is now not needed anymore and should not be used.
537cl-launch 3.000 and common-lisp-controller 7.2 have been updated
538to just delegate this functionality to ASDF.
539
540@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
541@comment  node-name,  next,  previous,  up
542
543@chapter Using ASDF
544
545@section Loading a system
546
547The system @var{foo} is loaded (and compiled, if necessary)
548by evaluating the following Lisp form:
549
550@example
551(asdf:load-system :@var{foo})
552@end example
553
554On some implementations (namely ABCL, Clozure CL, CMUCL, ECL and SBCL),
555ASDF hooks into the @code{CL:REQUIRE} facility
556and you can just use:
557
558@example
559(require :@var{foo})
560@end example
561
562In older versions of ASDF, you needed to use
563@code{(asdf:oos 'asdf:load-op :@var{foo})}.
564If your ASDF is too old to provide @code{asdf:load-system} though
565we recommend that you upgrade to ASDF 2.
566@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
567
568
569@section Other Operations
570
571ASDF provides three commands for the most common system operations:
572@code{load-system}, @code{compile-system} or @code{test-system}.
573
574Because ASDF is an extensible system
575for defining @emph{operations} on @emph{components},
576it also provides a generic function @code{operate}
577(which is usually abbreviated by @code{oos}).
578You'll use @code{oos} whenever you want to do something beyond
579compiling, loading and testing.
580
581Output from ASDF and ASDF extensions are supposed to be sent
582to the CL stream @code{*standard-output*},
583and so rebinding that stream around calls to @code{asdf:operate}
584should redirect all output from ASDF operations.
585
586Reminder: before ASDF can operate on a system, however,
587it must be able to find and load that system's definition.
588@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
589
590
591@section Summary
592
593To use ASDF:
594
595@itemize
596@item
597Load ASDF itself into your Lisp image, either through
598@code{(require :asdf)} or else through
599@code{(load "/path/to/asdf.lisp")}.
600
601@item
602Make sure ASDF can find system definitions
603thanks to proper source-registry configuration.
604
605@item
606Load a system with @code{(load-system :my-system)}
607or use some other operation on some system of your choice.
608
609@end itemize
610
611@section Moving on
612
613That's all you need to know to use ASDF to load systems written by others.
614The rest of this manual deals with writing system definitions
615for Common Lisp software you write yourself,
616including how to extend ASDF to define new operation and component types.
617
618
619@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
620@comment  node-name,  next,  previous,  up
621@chapter Defining systems with defsystem
622
623This chapter describes how to use asdf to define systems and develop
624software.
625
626
627@menu
628* The defsystem form::
629* A more involved example::
630* The defsystem grammar::
631* Other code in .asd files::
632@end menu
633
634@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
635@comment  node-name,  next,  previous,  up
636@section The defsystem form
637
638Systems can be constructed programmatically
639by instantiating components using @code{make-instance}.
640Most of the time, however, it is much more practical to use
641a static @code{defsystem} form.
642This section begins with an example of a system definition,
643then gives the full grammar of @code{defsystem}.
644
645Let's look at a simple system.
646This is a complete file that would
647usually be saved as @file{hello-lisp.asd}:
648
649@lisp
650(in-package :asdf)
651
652(defsystem "hello-lisp"
653  :description "hello-lisp: a sample Lisp system."
654  :version "0.2"
655  :author "Joe User <joe@@example.com>"
656  :licence "Public Domain"
657  :components ((:file "packages")
658               (:file "macros" :depends-on ("packages"))
659               (:file "hello" :depends-on ("macros"))))
660@end lisp
661
662Some notes about this example:
663
664@itemize
665
666@item
667The file starts with an @code{in-package} form
668to use package @code{asdf}.
669You could instead start your definition by using
670a qualified name @code{asdf:defsystem}.
671
672@item
673If in addition to simply using @code{defsystem},
674you are going to define functions,
675create ASDF extension, globally bind symbols, etc.,
676it is recommended that to avoid namespace pollution between systems,
677you should create your own package for that purpose,
678for instance replacing the above @code{(in-package :asdf)} with:
679
680@lisp
681(defpackage :foo-system
682  (:use :cl :asdf))
683
684(in-package :foo-system)
685@end lisp
686
687@item
688The @code{defsystem} form defines a system named @code{hello-lisp}
689that contains three source files:
690@file{packages}, @file{macros} and @file{hello}.
691
692@item
693The file @file{macros} depends on @file{packages}
694(presumably because the package it's in is defined in @file{packages}),
695and the file @file{hello} depends on @file{macros}
696(and hence, transitively on @file{packages}).
697This means that ASDF will compile and load @file{packages} and @file{macros}
698before starting the compilation of file @file{hello}.
699
700@item
701The files are located in the same directory
702as the file with the system definition.
703ASDF resolves symbolic links (or Windows shortcuts)
704before loading the system definition file and
705stores its location in the resulting system@footnote{
706It is possible, though almost never necessary, to override this behaviour.}.
707This is a good thing because the user can move the system sources
708without having to edit the system definition.
709
710@end itemize
711
712@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
713@comment  node-name,  next,  previous,  up
714@section A more involved example
715
716Let's illustrate some more involved uses of @code{defsystem} via a
717slightly convoluted example:
718
719@lisp
720(defsystem "foo"
721  :version "1.0"
722  :components ((:module "foo" :components ((:file "bar") (:file"baz")
723                                           (:file "quux"))
724                :perform (compile-op :after (op c)
725                          (do-something c))
726                :explain (compile-op :after (op c)
727                          (explain-something c)))
728               (:file "blah")))
729@end lisp
730
731The method-form tokens need explaining: essentially, this part:
732
733@lisp
734                :perform (compile-op :after (op c)
735                          (do-something c))
736                :explain (compile-op :after (op c)
737                          (explain-something c))
738@end lisp
739
740has the effect of
741
742@lisp
743(defmethod perform :after ((op compile-op) (c (eql ...)))
744           (do-something c))
745(defmethod explain :after ((op compile-op) (c (eql ...)))
746           (explain-something c))
747@end lisp
748
749where @code{...} is the component in question;
750note that although this also supports @code{:before} methods,
751they may not do what you want them to ---
752a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
753will run after all the dependencies and sub-components have been processed,
754but before the component in question has been compiled.
755
756@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
757@comment  node-name,  next,  previous,  up
758@section The defsystem grammar
759
760@example
761system-definition := ( defsystem system-designator @var{option}* )
762
763option := :components component-list
764        | :pathname pathname-specifier
765        | :default-component-class
766        | :perform method-form
767        | :explain method-form
768        | :output-files method-form
769        | :operation-done-p method-form
770        | :depends-on ( @var{dependency-def}* )
771        | :serial [ t | nil ]
772        | :in-order-to ( @var{dependency}+ )
773
774component-list := ( @var{component-def}* )
775
776component-def  := ( component-type simple-component-name @var{option}* )
777
778component-type := :system | :module | :file | :static-file | other-component-type
779
780other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
781
782dependency-def := simple-component-name
783               | ( :feature name )
784               | ( :version simple-component-name version-specifier)
785
786dependency := (dependent-op @var{requirement}+)
787requirement := (required-op @var{required-component}+)
788             | (feature feature-name)
789dependent-op := operation-name
790required-op := operation-name | feature
791
792simple-component-name := string
793                      |  symbol
794
795pathname-specifier := pathname | string | symbol
796
797method-form := (operation-name qual lambda-list @&rest body)
798qual := method qualifier
799@end example
800
801@subsection Component names
802
803Component names (@code{simple-component-name})
804may be either strings or symbols.
805
806@subsection Component types
807
808Component type names, even if expressed as keywords, will be looked up
809by name in the current package and in the asdf package, if not found in
810the current package.  So a component type @code{my-component-type}, in
811the current package @code{my-system-asd} can be specified as
812@code{:my-component-type}, or @code{my-component-type}.
813
814@subsection Pathname specifiers
815@cindex pathname specifiers
816
817A pathname specifier (@code{pathname-specifier})
818may be a pathname, a string or a symbol.
819When no pathname specifier is given for a component,
820which is the usual case, the component name itself is used.
821
822If a string is given, which is the usual case,
823the string will be interpreted as a Unix-style pathname
824where @code{/} characters will be interpreted as directory separators.
825Usually, Unix-style relative pathnames are used
826(i.e. not starting with @code{/}, as opposed to absolute pathnames);
827they are relative to the path of the parent component.
828Finally, depending on the @code{component-type},
829the pathname may be interpreted as either a file or a directory,
830and if it's a file,
831a file type may be added corresponding to the @code{component-type},
832or else it will be extracted from the string itself (if applicable).
833
834For instance, the @code{component-type} @code{:module}
835wants a directory pathname, and so a string @code{"foo/bar"}
836will be interpreted as the pathname @file{#p"foo/bar/"}.
837On the other hand, the @code{component-type} @code{:file}
838wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
839will be interpreted as the pathname @file{#p"foo/bar.lisp"},
840and a string @code{"foo/bar.quux"}
841will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
842Finally, the @code{component-type} @code{:static-file}
843wants a file without specifying a type, and so a string @code{"foo/bar"}
844will be interpreted as the pathname @file{#p"foo/bar"},
845and a string @code{"foo/bar.quux"}
846will be interpreted as the pathname @file{#p"foo/bar.quux"}.
847
848ASDF does not interpret the string @code{".."} to designate the parent
849directory.  This string will be passed through to the underlying
850operating system for interpretation.  We @emph{believe} that this will
851work on all platforms where ASDF is deployed, but do not guarantee this
852behavior.  A pathname object with a relative directory component of
853@code{:up} or @code{:back} is the only guaranteed way to specify a
854parent directory.
855
856If a symbol is given, it will be translated into a string,
857and downcased in the process.
858The downcasing of symbols is unconventional,
859but was selected after some consideration.
860Observations suggest that the type of systems we want to support
861either have lowercase as customary case (Unix, Mac, windows)
862or silently convert lowercase to uppercase (lpns),
863so this makes more sense than attempting to use @code{:case :common}
864as argument to @code{make-pathname},
865which is reported not to work on some implementations.
866
867Pathname objects may be given to override the path for a component.
868Such objects are typically specified using reader macros such as @code{#p}
869or @code{#.(make-pathname ...)}.
870Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
871and that the behavior of @code{parse-namestring} is completely non-portable,
872unless you are using Common Lisp @code{logical-pathname}s
873(@pxref{The defsystem grammar,,Warning about logical pathnames}, below).
874Pathnames made with @code{#.(make-pathname ...)}
875can usually be done more easily with the string syntax above.
876The only case that you really need a pathname object is to override
877the component-type default file type for a given component.
878Therefore, pathname objects should only rarely be used.
879Unhappily, ASDF 1 didn't properly support
880parsing component names as strings specifying paths with directories,
881and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
882
883Note that when specifying pathname objects,
884ASDF does not do any special interpretation of the pathname
885influenced by the component type, unlike the procedure for
886pathname-specifying strings.
887On the one hand, you have to be careful to provide a pathname that correctly
888fulfills whatever constraints are required from that component type
889(e.g. naming a directory or a file with appropriate type);
890on the other hand, you can circumvent the file type that would otherwise
891be forced upon you if you were specifying a string.
892
893
894@subsection Warning about logical pathnames
895@cindex logical pathnames
896
897We recommend that you not use logical pathnames
898in your asdf system definitions at this point,
899but logical pathnames @emph{are} supported.
900
901To use logical pathnames,
902you will have to provide a pathname object as a @code{:pathname} specifier
903to components that use it, using such syntax as
904@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
905
906You only have to specify such logical pathname for your system or
907some top-level component.  Sub-components' relative pathnames, specified
908using the string syntax
909for names, will be properly merged with the pathnames of their parents.
910The specification of a logical pathname host however is @emph{not}
911otherwise directly supported in the ASDF syntax
912for pathname specifiers as strings.
913
914The @code{asdf-output-translation} layer will
915avoid trying to resolve and translate logical-pathnames.
916The advantage of this is that you can define yourself what translations you want to use
917with the logical pathname facility.
918The disadvantage is that if you do not define such translations, any
919system that uses logical pathnames will be have differently under
920asdf-output-translations than other systems you use.
921
922If you wish to use logical pathnames you will have to configure the
923translations yourself before they may be used.
924ASDF currently provides no specific support
925for defining logical pathname translations.
926
927
928@subsection Serial dependencies
929@cindex serial dependencies
930
931If the @code{:serial t} option is specified for a module,
932ASDF will add dependencies for each each child component,
933on all the children textually preceding it.
934This is done as if by @code{:depends-on}.
935
936@lisp
937:serial t
938:components ((:file "a") (:file "b") (:file "c"))
939@end lisp
940
941is equivalent to
942
943@lisp
944:components ((:file "a")
945             (:file "b" :depends-on ("a"))
946             (:file "c" :depends-on ("a" "b")))
947@end lisp
948
949
950@subsection Source location
951
952The @code{:pathname} option is optional in all cases for systems
953defined via @code{defsystem},
954and in the usual case the user is recommended not to supply it.
955
956Instead, ASDF follows a hairy set of rules that are designed so that
957@enumerate
958@item
959@code{find-system}
960will load a system from disk
961and have its pathname default to the right place.
962@item
963This pathname information will not be overwritten with
964@code{*default-pathname-defaults*}
965(which could be somewhere else altogether)
966if the user loads up the @file{.asd} file into his editor
967and interactively re-evaluates that form.
968@end enumerate
969
970If a system is being loaded for the first time,
971its top-level pathname will be set to:
972
973@itemize
974@item
975The host/device/directory parts of @code{*load-truename*},
976if it is bound.
977@item
978@code{*default-pathname-defaults*}, otherwise.
979@end itemize
980
981If a system is being redefined, the top-level pathname will be
982
983@itemize
984@item
985changed, if explicitly supplied or obtained from @code{*load-truename*}
986(so that an updated source location is reflected in the system definition)
987@item
988changed if it had previously been set from @code{*default-pathname-defaults*}
989@item
990left as before, if it had previously been set from @code{*load-truename*}
991and @code{*load-truename*} is currently unbound
992(so that a developer can evaluate a @code{defsystem} form
993from within an editor without clobbering its source location)
994@end itemize
995
996@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
997@section Other code in .asd files
998
999Files containing @code{defsystem} forms
1000are regular Lisp files that are executed by @code{load}.
1001Consequently, you can put whatever Lisp code you like into these files
1002(e.g., code that examines the compile-time environment
1003and adds appropriate features to @code{*features*}).
1004However, some conventions should be followed,
1005so that users can control certain details of execution
1006of the Lisp in @file{.asd} files:
1007
1008@itemize
1009@item
1010Any informative output
1011(other than warnings and errors,
1012which are the condition system's to dispose of)
1013should be sent to the standard CL stream @code{*standard-output*},
1014so that users can easily control the disposition
1015of output from ASDF operations.
1016@end itemize
1017
1018
1019@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1020@comment  node-name,  next,  previous,  up
1021@chapter The object model of ASDF
1022
1023ASDF is designed in an object-oriented way from the ground up.
1024Both a system's structure and the operations that can be performed on systems
1025follow a protocol.
1026ASDF is extensible to new operations and to new component types.
1027This allows the addition of behaviours:
1028for example, a new component could be added for Java JAR archives,
1029and methods specialised on @code{compile-op} added for it
1030that would accomplish the relevant actions.
1031
1032This chapter deals with @emph{components}, the building blocks of a system,
1033and @emph{operations}, the actions that can be performed on a system.
1034
1035
1036
1037@menu
1038* Operations::
1039* Components::
1040@end menu
1041
1042@node  Operations, Components, The object model of ASDF, The object model of ASDF
1043@comment  node-name,  next,  previous,  up
1044@section Operations
1045@cindex operation
1046
1047An @dfn{operation} object of the appropriate type is instantiated
1048whenever the user wants to do something with a system like
1049
1050@itemize
1051@item compile all its files
1052@item load the files into a running lisp environment
1053@item copy its source files somewhere else
1054@end itemize
1055
1056Operations can be invoked directly, or examined
1057to see what their effects would be without performing them.
1058@emph{FIXME: document how!}
1059There are a bunch of methods specialised on operation and component type
1060that actually do the grunt work.
1061
1062The operation object contains whatever state is relevant for this purpose
1063(perhaps a list of visited nodes, for example)
1064but primarily is a nice thing to specialise operation methods on
1065and easier than having them all be @code{EQL} methods.
1066
1067Operations are invoked on systems via @code{operate}.
1068@anchor{operate}
1069@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1070@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1071@code{operate} invokes @var{operation} on @var{system}.
1072@code{oos} is a synonym for @code{operate}.
1073
1074@var{operation} is a symbol that is passed, along with the supplied
1075@var{initargs}, to @code{make-instance} to create the operation object.
1076@var{system} is a system designator.
1077
1078The @var{initargs} are passed to the @code{make-instance} call
1079when creating the operation object.
1080Note that dependencies may cause the operation
1081to invoke other operations on the system or its components:
1082the new operations will be created
1083with the same @var{initargs} as the original one.
1084
1085@end deffn
1086
1087@menu
1088* Predefined operations of ASDF::
1089* Creating new operations::
1090@end menu
1091
1092@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1093@comment  node-name,  next,  previous,  up
1094@subsection Predefined operations of ASDF
1095
1096All the operations described in this section are in the @code{asdf} package.
1097They are invoked via the @code{operate} generic function.
1098
1099@lisp
1100(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1101@end lisp
1102
1103@deffn Operation @code{compile-op} @&key @code{proclamations}
1104
1105This operation compiles the specified component.
1106If proclamations are supplied, they will be proclaimed.
1107This is a good place to specify optimization settings.
1108
1109When creating a new component type,
1110you should provide methods for @code{compile-op}.
1111
1112When @code{compile-op} is invoked,
1113component dependencies often cause some parts of the system
1114to be loaded as well as compiled.
1115Invoking @code{compile-op}
1116does not necessarily load all the parts of the system, though;
1117use @code{load-op} to load a system.
1118@end deffn
1119
1120@deffn Operation @code{load-op} @&key @code{proclamations}
1121
1122This operation loads a system.
1123
1124The default methods for @code{load-op} compile files before loading them.
1125For parity, your own methods on new component types should probably do so too.
1126@end deffn
1127
1128@deffn Operation @code{load-source-op}
1129
1130This operation will load the source for the files in a module
1131even if the source files have been compiled.
1132Systems sometimes have knotty dependencies
1133which require that sources are loaded
1134before they can be compiled.
1135This is how you do that.
1136
1137If you are creating a component type,
1138you need to implement this operation --- at least, where meaningful.
1139@end deffn
1140
1141@anchor{test-op}
1142@deffn Operation @code{test-op}
1143
1144This operation will perform some tests on the module.
1145The default method will do nothing.
1146The default dependency is to require
1147@code{load-op} to be performed on the module first.
1148The default @code{operation-done-p} is that the operation is @emph{never} done
1149---
1150we assume that if you invoke the @code{test-op},
1151you want to test the system, even if you have already done so.
1152
1153The results of this operation are not defined by ASDF.
1154It has proven difficult to define how the test operation
1155should signal its results to the user
1156in a way that is compatible with all of the various test libraries
1157and test techniques in use in the community.
1158@end deffn
1159
1160@c @deffn Operation test-system-version @&key minimum
1161
1162@c Asks the system whether it satisfies a version requirement.
1163
1164@c The default method accepts a string, which is expected to contain of a
1165@c number of integers separated by #\. characters.  The method is not
1166@c recursive.  The component satisfies the version dependency if it has
1167@c the same major number as required and each of its sub-versions is
1168@c greater than or equal to the sub-version number required.
1169
1170@c @lisp
1171@c (defun version-satisfies (x y)
1172@c   (labels ((bigger (x y)
1173@c           (cond ((not y) t)
1174@c                 ((not x) nil)
1175@c                 ((> (car x) (car y)) t)
1176@c                 ((= (car x) (car y))
1177@c                  (bigger (cdr x) (cdr y))))))
1178@c     (and (= (car x) (car y))
1179@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1180@c @end lisp
1181
1182@c If that doesn't work for your system, you can override it.  I hope
1183@c you have as much fun writing the new method as @verb{|#lisp|} did
1184@c reimplementing this one.
1185@c @end deffn
1186
1187@c @deffn Operation feature-dependent-op
1188
1189@c An instance of @code{feature-dependent-op} will ignore any components
1190@c which have a @code{features} attribute, unless the feature combination
1191@c it designates is satisfied by @code{*features*}.  This operation is
1192@c not intended to be instantiated directly, but other operations may
1193@c inherit from it.
1194
1195@c @end deffn
1196
1197@node  Creating new operations,  , Predefined operations of ASDF, Operations
1198@comment  node-name,  next,  previous,  up
1199@subsection Creating new operations
1200
1201ASDF was designed to be extensible in an object-oriented fashion.
1202To teach ASDF new tricks, a programmer can implement the behaviour he wants
1203by creating a subclass of @code{operation}.
1204
1205ASDF's pre-defined operations are in no way ``privileged'',
1206but it is requested that developers never use the @code{asdf} package
1207for operations they develop themselves.
1208The rationale for this rule is that we don't want to establish a
1209``global asdf operation name registry'',
1210but also want to avoid name clashes.
1211
1212An operation must provide methods for the following generic functions
1213when invoked with an object of type @code{source-file}:
1214@emph{FIXME describe this better}
1215
1216@itemize
1217
1218@item @code{output-files}
1219The @code{output-files} method determines where the method will put its files.
1220It returns two values, a list of pathnames, and a boolean.
1221If the boolean is @code{T} then the pathnames are marked
1222not be translated by enclosing @code{:around} methods.
1223If the boolean is @code{NIL} then enclosing @code{:around} methods
1224may translate these pathnames, e.g. to ensure object files
1225are somehow stored in some implementation-dependent cache.
1226@item @code{perform}
1227The @code{perform} method must call @code{output-files}
1228to find out where to put its files,
1229because the user is allowed to override.
1230@item @code{output-files}
1231for local policy @code{explain}
1232@item @code{operation-done-p},
1233if you don't like the default one
1234
1235@end itemize
1236
1237Operations that print output should send that output to the standard
1238CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1239
1240@node Components,  , Operations, The object model of ASDF
1241@comment  node-name,  next,  previous,  up
1242@section Components
1243@cindex component
1244@cindex system
1245@cindex system designator
1246@vindex *system-definition-search-functions*
1247
1248A @dfn{component} represents a source file or
1249(recursively) a collection of components.
1250A @dfn{system} is (roughly speaking) a top-level component
1251that can be found via @code{find-system}.
1252
1253A @dfn{system designator} is a string or symbol
1254and behaves just like any other component name
1255(including with regard to the case conversion rules for component names).
1256
1257
1258@defun find-system system-designator &optional (error-p t)
1259
1260Given a system designator, @code{find-system} finds and returns a system.
1261If no system is found, an error of type
1262@code{missing-component} is thrown,
1263or @code{nil} is returned if @code{error-p} is false.
1264
1265To find and update systems, @code{find-system} funcalls each element
1266in the @code{*system-definition-search-functions*} list,
1267expecting a pathname to be returned.
1268The resulting pathname is loaded if either of the following conditions is true:
1269
1270@itemize
1271@item
1272there is no system of that name in memory
1273@item
1274the file's @code{last-modified} time exceeds the @code{last-modified} time
1275of the system in memory
1276@end itemize
1277
1278When system definitions are loaded from @file{.asd} files,
1279a new scratch package is created for them to load into,
1280so that different systems do not overwrite each others operations.
1281The user may also wish to (and is recommended to)
1282include @code{defpackage} and @code{in-package} forms
1283in his system definition files, however,
1284so that they can be loaded manually if need be.
1285
1286The default value of @code{*system-definition-search-functions*}
1287is a list of two functions.
1288The first function looks in each of the directories given
1289by evaluating members of @code{*central-registry*}
1290for a file whose name is the name of the system and whose type is @file{asd}.
1291The first such file is returned,
1292whether or not it turns out to actually define the appropriate system.
1293The second function does something similar,
1294for the directories specified in the @code{source-registry}.
1295Hence, it is strongly advised to define a system
1296@var{foo} in the corresponding file @var{foo.asd}.
1297@end defun
1298
1299
1300@menu
1301* Common attributes of components::
1302* Pre-defined subclasses of component::
1303* Creating new component types::
1304@end menu
1305
1306@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1307@comment  node-name,  next,  previous,  up
1308@subsection Common attributes of components
1309
1310All components, regardless of type, have the following attributes.
1311All attributes except @code{name} are optional.
1312
1313@subsubsection Name
1314
1315A component name is a string or a symbol.
1316If a symbol, its name is taken and lowercased.
1317
1318Unless overridden by a @code{:pathname} attribute,
1319the name will be interpreted as a pathname specifier according
1320to a Unix-style syntax.
1321@xref{The defsystem grammar,,Pathname specifiers}.
1322
1323@subsubsection Version identifier
1324
1325This optional attribute is used by the @code{test-system-version} operation.
1326@xref{Predefined operations of ASDF}.
1327For the default method of @code{test-system-version},
1328the version should be a string of integers separated by dots,
1329for example @samp{1.0.11}.
1330
1331@emph{Nota Bene}:
1332This operation, planned for ASDF 1,
1333is still not implement yet as of ASDF 2.
1334Don't hold your breath.
1335
1336
1337@subsubsection Required features
1338
1339@emph{FIXME: This subsection seems to contradict the
1340@code{defsystem} grammar subsection,
1341which doesn't provide any obvious way to specify required features.
1342Furthermore, in 2009, discussions on the
1343@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1344suggested that the specification of required features may be broken,
1345and that no one may have been using them for a while.
1346Please contact the
1347@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1348if you are interested in getting this features feature fixed.}
1349
1350Traditionally defsystem users have used reader conditionals
1351to include or exclude specific per-implementation files.
1352This means that any single implementation cannot read the entire system,
1353which becomes a problem if it doesn't wish to compile it,
1354but instead for example to create an archive file containing all the sources,
1355as it will omit to process the system-dependent sources for other systems.
1356
1357Each component in an asdf system may therefore specify features using
1358the same syntax as @code{#+} does, and it will (somehow) be ignored for
1359certain operations unless the feature conditional is a member of
1360@code{*features*}.
1361
1362
1363@subsubsection Dependencies
1364
1365This attribute specifies dependencies of the component on its siblings.
1366It is optional but often necessary.
1367
1368There is an excitingly complicated relationship between the initarg
1369and the method that you use to ask about dependencies
1370
1371Dependencies are between (operation component) pairs.
1372In your initargs for the component, you can say
1373
1374@lisp
1375:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1376              (load-op (load-op "foo")))
1377@end lisp
1378
1379This means the following things:
1380@itemize
1381@item
1382before performing compile-op on this component, we must perform
1383load-op on @var{a} and @var{b}, and compile-op on @var{c},
1384@item
1385before performing @code{load-op}, we have to load @var{foo}
1386@end itemize
1387
1388The syntax is approximately
1389
1390@verbatim
1391(this-op {(other-op required-components)}+)
1392
1393required-components := component-name
1394                     | (required-components required-components)
1395
1396component-name := string
1397                | (:version string minimum-version-object)
1398@end verbatim
1399
1400Side note:
1401
1402This is on a par with what ACL defsystem does.
1403mk-defsystem is less general: it has an implied dependency
1404
1405@verbatim
1406  for all x, (load x) depends on (compile x)
1407@end verbatim
1408
1409and using a @code{:depends-on} argument to say that @var{b} depends on
1410@var{a} @emph{actually} means that
1411
1412@verbatim
1413  (compile b) depends on (load a)
1414@end verbatim
1415
1416This is insufficient for e.g. the McCLIM system, which requires that
1417all the files are loaded before any of them can be compiled ]
1418
1419End side note
1420
1421In ASDF, the dependency information for a given component and operation
1422can be queried using @code{(component-depends-on operation component)},
1423which returns a list
1424
1425@lisp
1426((load-op "a") (load-op "b") (compile-op "c") ...)
1427@end lisp
1428
1429@code{component-depends-on} can be subclassed for more specific
1430component/operation types: these need to @code{(call-next-method)}
1431and append the answer to their dependency, unless
1432they have a good reason for completely overriding the default dependencies.
1433
1434If it weren't for CLISP, we'd be using @code{LIST} method
1435combination to do this transparently.
1436But, we need to support CLISP.
1437If you have the time for some CLISP hacking,
1438I'm sure they'd welcome your fixes.
1439@c Doesn't CLISP now support LIST method combination?
1440
1441@subsubsection pathname
1442
1443This attribute is optional and if absent (which is the usual case),
1444the component name will be used.
1445
1446@xref{The defsystem grammar,,Pathname specifiers},
1447for an explanation of how this attribute is interpreted.
1448
1449Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1450does additional processing to set the filesystem location of
1451the top component in that system.
1452This is detailed elsewhere. @xref{Defining systems with defsystem}.
1453
1454The answer to the frequently asked question
1455``how do I create a system definition
1456where all the source files have a @file{.cl} extension''
1457is thus
1458
1459@lisp
1460(defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys))))
1461  "cl")
1462@end lisp
1463
1464@subsubsection properties
1465
1466This attribute is optional.
1467
1468Packaging systems often require information about files or systems
1469in addition to that specified by ASDF's pre-defined component attributes.
1470Programs that create vendor packages out of ASDF systems therefore
1471have to create ``placeholder'' information to satisfy these systems.
1472Sometimes the creator of an ASDF system may know the additional
1473information and wish to provide it directly.
1474
1475@code{(component-property component property-name)} and
1476associated @code{setf} method will allow
1477the programmatic update of this information.
1478Property names are compared as if by @code{EQL},
1479so use symbols or keywords or something.
1480
1481@menu
1482* Pre-defined subclasses of component::
1483* Creating new component types::
1484@end menu
1485
1486@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1487@comment  node-name,  next,  previous,  up
1488@subsection Pre-defined subclasses of component
1489
1490@deffn Component source-file
1491
1492A source file is any file that the system does not know how to
1493generate from other components of the system.
1494
1495Note that this is not necessarily the same thing as
1496``a file containing data that is typically fed to a compiler''.
1497If a file is generated by some pre-processor stage
1498(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1499then it is not, by this definition, a source file.
1500Conversely, we might have a graphic file
1501that cannot be automatically regenerated,
1502or a proprietary shared library that we received as a binary:
1503these do count as source files for our purposes.
1504
1505Subclasses of source-file exist for various languages.
1506@emph{FIXME: describe these.}
1507@end deffn
1508
1509@deffn Component module
1510
1511A module is a collection of sub-components.
1512
1513A module component has the following extra initargs:
1514
1515@itemize
1516@item
1517@code{:components} the components contained in this module
1518
1519@item
1520@code{:default-component-class}
1521All children components which don't specify their class explicitly
1522are inferred to be of this type.
1523
1524@item
1525@code{:if-component-dep-fails}
1526This attribute takes one of the values
1527@code{:fail}, @code{:try-next}, @code{:ignore},
1528its default value is @code{:fail}.
1529The other values can be used for implementing conditional compilation
1530based on implementation @code{*features*},
1531for the case where it is not necessary for all files in a module to be
1532compiled.
1533@emph{FIXME: such conditional compilation has been reported
1534to be broken in 2009.}
1535
1536@item
1537@code{:serial} When this attribute is set,
1538each subcomponent of this component is assumed to depend on all subcomponents
1539before it in the list given to @code{:components}, i.e.
1540all of them are loaded before a compile or load operation is performed on it.
1541
1542@end itemize
1543
1544The default operation knows how to traverse a module, so
1545most operations will not need to provide methods specialised on modules.
1546
1547@code{module} may be subclassed to represent components such as
1548foreign-language linked libraries or archive files.
1549@end deffn
1550
1551@deffn Component system
1552
1553@code{system} is a subclass of @code{module}.
1554
1555A system is a module with a few extra attributes for documentation
1556purposes; these are given elsewhere.
1557@xref{The defsystem grammar}.
1558
1559Users can create new classes for their systems:
1560the default @code{defsystem} macro takes a @code{:class} keyword argument.
1561@end deffn
1562
1563@node  Creating new component types,  , Pre-defined subclasses of component, Components
1564@comment  node-name,  next,  previous,  up
1565@subsection Creating new component types
1566
1567New component types are defined by subclassing one of the existing
1568component classes and specializing methods on the new component class.
1569
1570@emph{FIXME: this should perhaps be explained more throughly,
1571not only by example ...}
1572
1573As an example, suppose we have some implementation-dependent
1574functionality that we want to isolate
1575in one subdirectory per Lisp implementation our system supports.
1576We create a subclass of
1577@code{cl-source-file}:
1578
1579@lisp
1580(defclass unportable-cl-source-file (cl-source-file)
1581  ())
1582@end lisp
1583
1584A hypothetical function @code{system-dependent-dirname}
1585gives us the name of the subdirectory.
1586All that's left is to define how to calculate the pathname
1587of an @code{unportable-cl-source-file}.
1588
1589@lisp
1590(defmethod component-pathname ((component unportable-cl-source-file))
1591  (let ((pathname (call-next-method))
1592        (name (string-downcase (system-dependent-dirname))))
1593    (merge-pathnames*
1594     (make-pathname :directory (list :relative name))
1595     pathname)))
1596@end lisp
1597
1598The new component type is used in a @code{defsystem} form in this way:
1599
1600@lisp
1601(defsystem :foo
1602    :components
1603    ((:file "packages")
1604     ...
1605     (:unportable-cl-source-file "threads"
1606      :depends-on ("packages" ...))
1607     ...
1608    )
1609@end lisp
1610
1611@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1612@comment  node-name,  next,  previous,  up
1613@chapter Controlling where ASDF searches for systems
1614
1615@section Configurations
1616
1617Configurations specify paths where to find system files.
1618
1619@enumerate
1620
1621@item
1622The search registry may use some hardcoded wrapping registry specification.
1623This allows some implementations (notably SBCL) to specify where to find
1624some special implementation-provided systems that
1625need to precisely match the version of the implementation itself.
1626
1627@item
1628An application may explicitly initialize the source-registry configuration
1629using the configuration API
1630(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1631in which case this takes precedence.
1632It may itself compute this configuration from the command-line,
1633from a script, from its own configuration file, etc.
1634
1635@item
1636The source registry will be configured from
1637the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1638
1639@item
1640The source registry will be configured from
1641user configuration file
1642@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1643(which defaults to
1644@file{~/.config/common-lisp/source-registry.conf})
1645if it exists.
1646
1647@item
1648The source registry will be configured from
1649user configuration directory
1650@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1651(which defaults to
1652@file{~/.config/common-lisp/source-registry.conf.d/})
1653if it exists.
1654
1655@item
1656The source registry will be configured from
1657system configuration file
1658@file{/etc/common-lisp/source-registry.conf}
1659if it exists/
1660
1661@item
1662The source registry will be configured from
1663system configuration directory
1664@file{/etc/common-lisp/source-registry.conf.d/}
1665if it exists.
1666
1667@item
1668The source registry will be configured from a default configuration.
1669This configuration may allow for implementation-specific systems
1670to be found, for systems to be found the current directory
1671(at the time that the configuration is initialized) as well as
1672@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1673@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1674
1675@end enumerate
1676
1677Each of these configuration is specified as a SEXP
1678in a trival domain-specific language (defined below).
1679Additionally, a more shell-friendly syntax is available
1680for the environment variable (defined yet below).
1681
1682Each of these configurations is only used if the previous
1683configuration explicitly or implicitly specifies that it
1684includes its inherited configuration.
1685
1686Additionally, some implementation-specific directories
1687may be automatically prepended to whatever directories are specified
1688in configuration files, no matter if the last one inherits or not.
1689
1690@section XDG base directory
1691
1692Note that we purport to respect the XDG base directory specification
1693as to where configuration files are located,
1694where data files are located,
1695where output file caches are located.
1696Mentions of XDG variables refer to that document.
1697
1698@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1699
1700This specification allows the user to specify some environment variables
1701to customize how applications behave to his preferences.
1702
1703On Windows platforms, when not using Cygwin,
1704instead of the XDG base directory specification,
1705we try to use folder configuration from the registry regarding
1706@code{Common AppData} and similar directories.
1707However, support querying the Windows registry is limited as of ASDF 2,
1708and on many implementations, we may fall back to always using the defaults
1709without consulting the registry.
1710Patches welcome.
1711
1712@section Backward Compatibility
1713
1714For backward compatibility as well as for a practical backdoor for hackers,
1715ASDF will first search for @code{.asd} files in the directories specified in
1716@code{asdf:*central-registry*}
1717before it searches in the source registry above.
1718
1719@xref{Configuring ASDF,,Configuring ASDF to find your systems -- old style}.
1720
1721By default, @code{asdf:*central-registry*} will be empty.
1722
1723This old mechanism will therefore not affect you if you don't use it,
1724but will take precedence over the new mechanism if you do use it.
1725
1726@section Configuration DSL
1727
1728Here is the grammar of the SEXP DSL for source-registry configuration:
1729
1730@example
1731;; A configuration is single SEXP starting with keyword :source-registry
1732;; followed by a list of directives.
1733CONFIGURATION := (:source-registry DIRECTIVE ...)
1734
1735;; A directive is one of the following:
1736DIRECTIVE :=
1737    ;; INHERITANCE DIRECTIVE:
1738    ;; Your configuration expression MUST contain
1739    ;; exactly one of either of these:
1740    :inherit-configuration | ; splices inherited configuration (often specified last)
1741    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1742
1743    ;; add a single directory to be scanned (no recursion)
1744    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1745
1746    ;; add a directory hierarchy, recursing but excluding specified patterns
1747    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1748
1749    ;; override the defaults for exclusion patterns
1750    (:exclude PATTERN ...) |
1751    ;; augment the defaults for exclusion patterns
1752    (:also-exclude PATTERN ...) |
1753
1754    ;; splice the parsed contents of another config file
1755    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1756
1757    ;; This directive specifies that some default must be spliced.
1758    :default-registry
1759
1760PATTERN := a string without wildcards, that will be matched exactly
1761  against the name of a any subdirectory in the directory component
1762        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
1763@end example
1764
1765For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
1766which is the default place ASDF looks for this configuration,
1767once contained:
1768@example
1769(:source-registry
1770  (:tree "/home/fare/cl/")
1771  :inherit-configuration)
1772@end example
1773
1774
1775@section Configuration Directories
1776
1777Configuration directories consist in files each contains
1778a list of directives without any enclosing @code{(:source-registry ...)} form.
1779The files will be sorted by namestring as if by @code{string<} and
1780the lists of directives of these files with be concatenated in order.
1781An implicit @code{:inherit-configuration} will be included
1782at the @emph{end} of the list.
1783
1784This allows for packaging software that has file granularity
1785(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
1786to easily include configuration information about distributed software.
1787
1788The convention is that, for sorting purposes,
1789the names of files in such a directory begin with two digits
1790that determine the order in which these entries will be read.
1791Also, the type of these files is conventionally @code{"conf"}
1792and as a limitation to some implementations (e.g. GNU clisp),
1793the type cannot be @code{NIL}.
1794
1795Directories may be included by specifying a directory pathname
1796or namestring in an @code{:include} directive, e.g.:
1797
1798@example
1799  (:include "/foo/bar/")
1800@end example
1801
1802Hence, to achieve the same effect as
1803my example @file{~/.config/common-lisp/source-registry.conf} above,
1804I could simply create a file
1805@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
1806alone in its directory with the following contents:
1807@example
1808(:tree "/home/fare/cl/")
1809@end example
1810
1811
1812@section Shell-friendly syntax for configuration
1813
1814When considering environment variable @code{CL_SOURCE_REGISTRY}
1815ASDF will skip to next configuration if it's an empty string.
1816It will @code{READ} the string as a SEXP in the DSL
1817if it begins with a paren @code{(}
1818and it will be interpreted much like @code{TEXINPUTS}
1819list of paths, where
1820
1821  * paths are separated
1822   by a @code{:} (colon) on Unix platforms (including cygwin),
1823   by a @code{;} (semicolon) on other platforms (mainly, Windows).
1824
1825  * each entry is a directory to add to the search path.
1826
1827  * if the entry ends with a double slash @code{//}
1828    then it instead indicates a tree in the subdirectories
1829    of which to recurse.
1830
1831  * if the entry is the empty string (which may only appear once),
1832    then it indicates that the inherited configuration should be
1833    spliced there.
1834
1835
1836@section Search Algorithm
1837
1838In case that isn't clear, the semantics of the configuration is that
1839when searching for a system of a given name,
1840directives are processed in order.
1841
1842When looking in a directory, if the system is found, the search succeeds,
1843otherwise it continues.
1844
1845When looking in a tree, if one system is found, the search succeeds.
1846If multiple systems are found, the consequences are unspecified:
1847the search may succeed with any of the found systems,
1848or an error may be raised.
1849ASDF currently returns the first system found,
1850XCVB currently raised an error.
1851If none is found, the search continues.
1852
1853Exclude statements specify patterns of subdirectories
1854the systems from which to ignore.
1855Typically you don't want to use copies of files kept by such
1856version control systems as Darcs.
1857Exclude statements are not propagated to further included or inherited
1858configuration files or expressions;
1859instead the defaults are reset around every configuration statement
1860to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
1861
1862Include statements cause the search to recurse with the path specifications
1863from the file specified.
1864
1865An inherit-configuration statement cause the search to recurse with the path
1866specifications from the next configuration
1867(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
1868
1869
1870@section Caching Results
1871
1872The implementation is allowed to either eagerly compute the information
1873from the configurations and file system, or to lazily re-compute it
1874every time, or to cache any part of it as it goes.
1875To explicitly flush any information cached by the system, use the API below.
1876
1877
1878@section Configuration API
1879
1880The specified functions are exported from your build system's package.
1881Thus for ASDF the corresponding functions are in package ASDF,
1882and for XCVB the corresponding functions are in package XCVB.
1883
1884@defun initialize-source-registry @&optional PARAMETER
1885   will read the configuration and initialize all internal variables.
1886   You may extend or override configuration
1887   from the environment and configuration files
1888   with the given @var{PARAMETER}, which can be
1889   @code{NIL} (no configuration override),
1890   or a SEXP (in the SEXP DSL),
1891   a string (as in the string DSL),
1892   a pathname (of a file or directory with configuration),
1893   or a symbol (fbound to function that when called returns one of the above).
1894@end defun
1895
1896@defun clear-source-registry
1897   undoes any source registry configuration
1898   and clears any cache for the search algorithm.
1899   You might want to call that before you
1900   dump an image that would be resumed with a different configuration,
1901   and return an empty configuration.
1902   Note that this does not include clearing information about
1903   systems defined in the current image, only about
1904   where to look for systems not yet defined.
1905@end defun
1906
1907@defun ensure-source-registry @&optional PARAMETER
1908   checks whether a source registry has been initialized.
1909   If not, initialize it with the given @var{PARAMETER}.
1910@end defun
1911
1912
1913@section Future
1914
1915If this mechanism is successful, in the future, we may declare
1916@code{asdf:*central-registry*} obsolete and eventually remove it.
1917Any hook into implementation-specific search mechanisms will by then
1918have been integrated in the @code{:default-configuration} which everyone
1919should either explicitly use or implicit inherit. Some shell syntax
1920for it should probably be added somehow.
1921
1922But we're not there yet. For now, let's see how practical this new
1923source-registry is.
1924
1925
1926@section Rejected ideas
1927
1928Alternatives I considered and rejected included:
1929
1930@enumerate
1931@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
1932   and somehow the configuration parser expands the new configuration
1933   language into a expanded series of directories of subdirectories to
1934   lookup, pre-recursing through specified hierarchies. This is kludgy,
1935   and leaves little space of future cleanups and extensions.
1936
1937@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
1938   in completely new ways, so that new kinds of entries may be implemented
1939   as a recursive search, etc. This seems somewhat backwards.
1940
1941@item Completely remove @code{asdf:*central-registry*}
1942   and break backwards compatibility.
1943   Hopefully this will happen in a few years after everyone migrate to
1944   a better ASDF and/or to XCVB, but it would be very bad to do it now.
1945
1946@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
1947   when you dereference it or setf it. Only the new variable with new
1948   semantics is handled by the new search procedure.
1949   Complex and still introduces subtle semantic issues.
1950@end enumerate
1951
1952
1953I've been suggested the below features, but have rejected them,
1954for the sake of keeping ASDF no more complex than strictly necessary.
1955
1956@itemize
1957@item
1958  More syntactic sugar: synonyms for the configuration directives, such as
1959  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
1960  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
1961
1962@item
1963   The possibility to register individual files instead of directories.
1964
1965@item
1966  Integrate Xach Beane's tilde expander into the parser,
1967  or something similar that is shell-friendly or shell-compatible.
1968  I'd rather keep ASDF minimal. But maybe this precisely keeps it
1969  minimal by removing the need for evaluated entries that ASDF has?
1970  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
1971  Hopefully, these are already superseded by the @code{:default-registry}
1972
1973@item
1974  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
1975  down a filesystem tree in the environment variable.
1976  It isn't that Lisp friendly either.
1977@end itemize
1978
1979@section TODO
1980
1981@itemize
1982@item Add examples
1983@end itemize
1984
1985
1986@section Credits for the source-registry
1987
1988Thanks a lot to Stelian Ionescu for the initial idea.
1989
1990Thanks to Rommel Martinez for the initial implementation attempt.
1991
1992All bad design ideas and implementation bugs are to mine, not theirs.
1993But so are good design ideas and elegant implementation tricks.
1994
1995 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
1996
1997
1998
1999@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2000@comment  node-name,  next,  previous,  up
2001@chapter Controlling where ASDF saves compiled files
2002@cindex asdf-output-translations
2003@vindex ASDF_OUTPUT_TRANSLATIONS
2004
2005Each Common Lisp implementation has its own format
2006for compiled files (fasls for short, short for ``fast loading'').
2007If you use multiple implementations
2008(or multiple versions of the same implementation),
2009you'll soon find your source directories
2010littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2011Worse yet, some implementations use the same file extension
2012while changing formats from version to version (or platform to platform)
2013which means that you'll have to recompile binaries
2014as you switch from one implementation to the next.
2015
2016ASDF 2 includes the @code{asdf-output-translations} facility
2017to mitigate the problem.
2018
2019@section Configurations
2020
2021Configurations specify mappings from input locations to output locations.
2022Once again we rely on the XDG base directory specification for configuration.
2023@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2024
2025@enumerate
2026
2027@item
2028Some hardcoded wrapping output translations configuration may be used.
2029This allows special output translations (or usually, invariant directories)
2030to be specified corresponding to the similar special entries in the source registry.
2031
2032@item
2033An application may explicitly initialize the output-translations
2034configuration using the Configuration API
2035in which case this takes precedence.
2036(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2037It may itself compute this configuration from the command-line,
2038from a script, from its own configuration file, etc.
2039
2040@item
2041The source registry will be configured from
2042the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2043
2044@item
2045The source registry will be configured from
2046user configuration file
2047@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2048(which defaults to
2049@file{~/.config/common-lisp/asdf-output-translations.conf})
2050if it exists.
2051
2052@item
2053The source registry will be configured from
2054user configuration directory
2055@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2056(which defaults to
2057@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2058if it exists.
2059
2060@item
2061The source registry will be configured from
2062system configuration file
2063@file{/etc/common-lisp/asdf-output-translations.conf}
2064if it exists.
2065
2066@item
2067The source registry will be configured from
2068system configuration directory
2069@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2070if it exists.
2071
2072@end enumerate
2073
2074Each of these configurations is specified as a SEXP
2075in a trival domain-specific language (defined below).
2076Additionally, a more shell-friendly syntax is available
2077for the environment variable (defined yet below).
2078
2079Each of these configurations is only used if the previous
2080configuration explicitly or implicitly specifies that it
2081includes its inherited configuration.
2082
2083Note that by default, a per-user cache is used for output files.
2084This allows the seamless use of shared installations of software
2085between several users, and takes files out of the way of the developers
2086when they browse source code,
2087at the expense of taking a small toll when developers have to clean up
2088output files and find they need to get familiar with output-translations first.
2089
2090
2091@section Backward Compatibility
2092
2093@c FIXME -- I think we should provide an easy way
2094@c to get behavior equivalent to A-B-L and
2095@c I will propose a technique for doing this.
2096
2097We purposefully do NOT provide backward compatibility with earlier versions of
2098@code{ASDF-Binary-Locations} (8 Sept 2009),
2099@code{common-lisp-controller} (7.0) or
2100@code{cl-launch} (2.35),
2101each of which had similar general capabilities.
2102The previous APIs of these programs were not designed
2103for configuration by the end-user
2104in an easy way with configuration files.
2105Recent versions of same packages use
2106the new @code{asdf-output-translations} API as defined below:
2107@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2108@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2109
2110This incompatibility shouldn't inconvenience many people.
2111Indeed, few people use and customize these packages;
2112these few people are experts who can trivially adapt to the new configuration.
2113Most people are not experts, could not properly configure these features
2114(except inasmuch as the default configuration of
2115@code{common-lisp-controller} and/or @code{cl-launch}
2116might have been doing the right thing for some users),
2117and yet will experience software that ``just works'',
2118as configured by the system distributor, or by default.
2119
2120Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2121we provide a limited emulation mode:
2122
2123@defun asdf:enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2124This function will initialize the new @code{asdf-output-translations} facility in a way
2125that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2126Where you would previously set global variables
2127@var{*centralize-lisp-binaries*},
2128@var{*default-toplevel-directory*},
2129@var{*include-per-user-information*},
2130@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2131you will now have to pass the same values as keyword arguments to this function.
2132Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2133will accept any valid pathname designator for @code{asdf-output-translations}
2134instead of just strings and pathnames.
2135@end defun
2136
2137If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2138(the one available as an extension to load of top of ASDF,
2139not the one built into a few old versions of ASDF),
2140but first you must disable @code{asdf-output-translations}
2141with @code{(asdf:disable-output-translations)},
2142or you might experience ``interesting'' issues.
2143
2144Also, note that output translation is enabled by default.
2145To disable it, use @code{(asdf:disable-output-translations)}.
2146
2147
2148@section Configuration DSL
2149
2150Here is the grammar of the SEXP DSL
2151for @code{asdf-output-translations} configuration:
2152
2153@verbatim
2154;; A configuration is single SEXP starting with keyword :source-registry
2155;; followed by a list of directives.
2156CONFIGURATION := (:output-translations DIRECTIVE ...)
2157
2158;; A directive is one of the following:
2159DIRECTIVE :=
2160    ;; INHERITANCE DIRECTIVE:
2161    ;; Your configuration expression MUST contain
2162    ;; exactly one of either of these:
2163    :inherit-configuration | ; splices inherited configuration (often specified last)
2164    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2165
2166    ;; include a configuration file or directory
2167    (:include PATHNAME-DESIGNATOR) |
2168
2169    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.35-x86-64/ or something.
2170    :enable-user-cache |
2171    ;; Disable global cache. Map / to /
2172    :disable-cache |
2173
2174    ;; add a single directory to be scanned (no recursion)
2175    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2176
2177    ;; use a function to return the translation of a directory designator
2178    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2179
2180DIRECTORY-DESIGNATOR :=
2181    T | ;; as source matches anything, as destination leaves pathname unmapped.
2182    ABSOLUTE-COMPONENT-DESIGNATOR |
2183    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
2184
2185ABSOLUTE-COMPONENT-DESIGNATOR :=
2186    NULL | ;; As source: skip this entry. As destination: same as source
2187    :ROOT | ;; magic: paths that are relative to the root of the source host and device
2188    STRING | ;; namestring (directory is assumed, better be absolute or bust, ``/**/*.*'' added)
2189    PATHNAME | ;; pathname (better be an absolute directory or bust)
2190    :HOME | ;; designates the user-homedir-pathname ~/
2191    :USER-CACHE | ;; designates the default location for the user cache
2192    :SYSTEM-CACHE | ;; designates the default location for the system cache
2193    :CURRENT-DIRECTORY ;; the current directory
2194
2195RELATIVE-COMPONENT-DESIGNATOR :=
2196    STRING | ;; namestring, directory is assumed. If the last component, /**/*.* is added
2197    PATHNAME | ;; pathname unless last component, directory is assumed.
2198    :IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.32.30-linux-x86-64
2199    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2200    :CURRENT-DIRECTORY | ;; all components of the current directory, without the :absolute
2201    :UID | ;; current UID -- not available on Windows
2202    :USER ;; current USER name -- NOT IMPLEMENTED(!)
2203
2204TRANSLATION-FUNCTION :=
2205    SYMBOL | ;; symbol of a function that takes two arguments,
2206             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2207    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2208             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2209
2210@end verbatim
2211
2212Relative components better be either relative
2213or subdirectories of the path before them, or bust.
2214
2215The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2216You can specify more fine-grained patterns
2217by using a pathname object as the last component
2218e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2219
2220You may use @code{#+features} to customize the configuration file.
2221
2222The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2223to anything but themselves (same as if the second designator was the same as the first).
2224
2225When the first designator is @code{t},
2226the mapping always matches.
2227When the first designator starts with @code{:root},
2228the mapping matches any host and device.
2229In either of these cases, if the second designator
2230isn't @code{t} and doesn't start with @code{:root},
2231then strings indicating the host and pathname are somehow copied
2232in the beginning of the directory component of the source pathname
2233before it is translated.
2234
2235When the second designator is @code{t}, the mapping is the identity.
2236When the second designator starts with @code{root},
2237the mapping preserves the host and device of the original pathname.
2238
2239@code{:include} statements cause the search to recurse with the path specifications
2240from the file specified.
2241
2242If the @code{translate-pathname} mechanism cannot achieve a desired
2243translation, the user may provide a function which provides the
2244required algorithim.  Such a translation function is specified by
2245supplying a list as the second @code{directory-designator}
2246the first element of which is the keyword @code{:function},
2247and the second element of which is
2248either a symbol which designates a function or a lambda expression.
2249The function designated by the second argument must take two arguments,
2250the first being the pathname of the source file,
2251the second being the wildcard that was matched.
2252The result of the function invocation should be the translated pathname.
2253
2254An @code{:inherit-configuration} statement cause the search to recurse with the path
2255specifications from the next configuration.
2256@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2257
2258@itemize
2259@item
2260@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2261@item
2262@code{:disable-cache} is the same as @code{(t t)}.
2263@item
2264@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2265which by default is the same as using
2266@code{(:home ".cache" "common-lisp" :implementation)}.
2267@item
2268@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2269which by default is the same as using
2270@code{("/var/cache/common-lisp" :uid :implementation-type)}
2271(on Unix and cygwin), or something semi-sensible on Windows.
2272@end itemize
2273
2274
2275@section Configuration Directories
2276
2277Configuration directories consist in files each contains
2278a list of directives without any enclosing
2279@code{(:output-translations ...)} form.
2280The files will be sorted by namestring as if by @code{string<} and
2281the lists of directives of these files with be concatenated in order.
2282An implicit @code{:inherit-configuration} will be included
2283at the @emph{end} of the list.
2284
2285This allows for packaging software that has file granularity
2286(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2287to easily include configuration information about software being distributed.
2288
2289The convention is that, for sorting purposes,
2290the names of files in such a directory begin with two digits
2291that determine the order in which these entries will be read.
2292Also, the type of these files is conventionally @code{"conf"}
2293and as a limitation of some implementations, the type cannot be @code{NIL}.
2294
2295Directories may be included by specifying a directory pathname
2296or namestring in an @code{:include} directive, e.g.:
2297@verbatim
2298  (:include "/foo/bar/")
2299@end verbatim
2300
2301@section Shell-friendly syntax for configuration
2302
2303When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2304ASDF will skip to next configuration if it's an empty string.
2305It will @code{READ} the string as an SEXP in the DSL
2306if it begins with a paren @code{(}
2307and it will be interpreted as a list of directories.
2308Directories should come by pairs, indicating a mapping directive.
2309Entries are separated
2310by a @code{:} (colon) on Unix platforms (including cygwin),
2311by a @code{;} (semicolon) on other platforms (mainly, Windows).
2312
2313The magic empty entry,
2314if it comes in what would otherwise be the first entry in a pair,
2315indicates the splicing of inherited configuration.
2316If it comes as the second entry in a pair,
2317it indicates that the directory specified first is to be left untranslated
2318(which has the same effect as if the directory had been repeated).
2319
2320
2321@section Semantics of Output Translations
2322
2323From the specified configuration,
2324a list of mappings is extracted in a straightforward way:
2325mappings are collected in order, recursing through
2326included or inherited configuration as specified.
2327To this list is prepended some implementation-specific mappings,
2328and is appended a global default.
2329
2330The list is then compiled to a mapping table as follows:
2331for each entry, in order, resolve the first designated directory
2332into an actual directory pathname for source locations.
2333If no mapping was specified yet for that location,
2334resolve the second designated directory to an output location directory
2335add a mapping to the table mapping the source location to the output location,
2336and add another mapping from the output location to itself
2337(unless a mapping already exists for the output location).
2338
2339Based on the table, a mapping function is defined,
2340mapping source pathnames to output pathnames:
2341given a source pathname, locate the longest matching prefix
2342in the source column of the mapping table.
2343Replace that prefix by the corresponding output column
2344in the same row of the table, and return the result.
2345If no match is found, return the source pathname.
2346(A global default mapping the filesystem root to itself
2347may ensure that there will always be a match,
2348with same fall-through semantics).
2349
2350@section Caching Results
2351
2352The implementation is allowed to either eagerly compute the information
2353from the configurations and file system, or to lazily re-compute it
2354every time, or to cache any part of it as it goes.
2355To explicitly flush any information cached by the system, use the API below.
2356
2357
2358@section Output location API
2359
2360The specified functions are exported from package ASDF.
2361
2362@defun initialize-output-translations @&optional PARAMETER
2363   will read the configuration and initialize all internal variables.
2364   You may extend or override configuration
2365   from the environment and configuration files
2366   with the given @var{PARAMETER}, which can be
2367   @code{NIL} (no configuration override),
2368   or a SEXP (in the SEXP DSL),
2369   a string (as in the string DSL),
2370   a pathname (of a file or directory with configuration),
2371   or a symbol (fbound to function that when called returns one of the above).
2372@end defun
2373
2374@defun disable-output-translations
2375   will initialize output translations in a way
2376   that maps every pathname to itself,
2377   effectively disabling the output translation facility.
2378@end defun
2379
2380@defun clear-output-translations
2381   undoes any output translation configuration
2382   and clears any cache for the mapping algorithm.
2383   You might want to call that before you
2384   dump an image that would be resumed with a different configuration,
2385   and return an empty configuration.
2386   Note that this does not include clearing information about
2387   systems defined in the current image, only about
2388   where to look for systems not yet defined.
2389@end defun
2390
2391@defun ensure-output-translations @&optional PARAMETER
2392   checks whether output translations have been initialized.
2393   If not, initialize them with the given @var{PARAMETER}.
2394   This function will be called before any attempt to operate on a system.
2395@end defun
2396
2397@defun apply-output-translations PATHNAME
2398   Applies the configured output location translations to @var{PATHNAME}
2399   (calls @code{ensure-output-translations} for the translations).
2400@end defun
2401
2402
2403@section Credits for output translations
2404
2405Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2406and to Peter van Eynde for @code{Common Lisp Controller}.
2407
2408All bad design ideas and implementation bugs are to mine, not theirs.
2409But so are good design ideas and elegant implementation tricks.
2410
2411 --- Francois-Rene Rideau @email{fare@@tunes.org}
2412
2413@c @section Default locations
2414@c @findex output-files-for-system-and-operation
2415
2416@c The default binary location for each Lisp implementation
2417@c is a subdirectory of each source directory.
2418@c To account for different Lisps, Operating Systems, Implementation versions,
2419@c and so on, ASDF borrows code from SLIME
2420@c to create reasonable custom directory names.
2421@c Here are some examples:
2422
2423@c @itemize
2424@c @item
2425@c SBCL, version 1.0 on Mac OS X for intel: @code{sbcl-1.0-darwin-x86}
2426
2427@c @item
2428@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2429
2430@c @item
2431@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2432@c @end itemize
2433
2434@c By default, all output file pathnames will be relocated
2435@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2436
2437@c See the document @file{README.asdf-output-translations}
2438@c for a full specification on how to configure @code{asdf-output-translations}.
2439
2440@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2441@comment  node-name,  next,  previous,  up
2442@chapter Error handling
2443@findex SYSTEM-DEFINITION-ERROR
2444@findex OPERATION-ERROR
2445
2446@section ASDF errors
2447
2448If ASDF detects an incorrect system definition, it will signal a generalised instance of
2449@code{SYSTEM-DEFINITION-ERROR}.
2450
2451Operations may go wrong (for example when source files contain errors).
2452These are signalled using generalised instances of
2453@code{OPERATION-ERROR}.
2454
2455@section Compilation error and warning handling
2456@vindex *compile-file-warnings-behaviour*
2457@vindex *compile-file-errors-behavior*
2458
2459ASDF checks for warnings and errors when a file is compiled.
2460The variables @var{*compile-file-warnings-behaviour*} and
2461@var{*compile-file-errors-behavior*}
2462control the handling of any such events.
2463The valid values for these variables are
2464@code{:error}, @code{:warn}, and @code{:ignore}.
2465
2466@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2467@comment  node-name,  next,  previous,  up
2468@chapter Miscellaneous additional functionality
2469
2470@emph{FIXME:  Add discussion of @code{run-shell-command}?  Others?}
2471
2472ASDF includes several additional features that are generally
2473useful for system definition and development. These include:
2474
2475@defun system-relative-pathname system name @&key type
2476
2477It's often handy to locate a file relative to some system.
2478The @code{system-relative-pathname} function meets this need.
2479It takes two arguments: the name of a system and a relative pathname.
2480It returns a pathname built from the location of the system's source file
2481and the relative pathname. For example
2482
2483@lisp
2484> (asdf:system-relative-pathname 'cl-ppcre #p"regex.data")
2485#P"/repository/other/cl-ppcre/regex.data"
2486@end lisp
2487
2488Instead of a pathname, you can provide a symbol or a string,
2489and optionally a keyword argument @code{type}.
2490The arguments will then be interpreted in the same way
2491as pathname specifiers for components.
2492@xref{The defsystem grammar,,Pathname specifiers}.
2493@end defun
2494
2495@defun system-source-directory system-designator
2496
2497ASDF does not provide a turnkey solution for locating data (or other
2498miscellaneous) files that are distributed together with the source code
2499of a system.  Programmers can use @code{system-source-directory} to find
2500such files.  Returns a pathname object.  The @var{system-designator} may
2501be a string, symbol, or ASDF system object.
2502@end defun
2503
2504
2505@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2506@comment  node-name,  next,  previous,  up
2507@chapter Getting the latest version
2508
2509Decide which version you want.
2510HEAD is the newest version and usually OK, whereas
2511RELEASE is for cautious people
2512(e.g. who already have systems using ASDF that they don't want broken),
2513a slightly older version about which none of the HEAD users have complained.
2514There is also a STABLE version, which is earlier than release.
2515
2516You may get the ASDF source repository using git:
2517@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2518
2519You will find the above referenced tags in this repository.
2520You can also browse the repository on
2521@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2522
2523Discussion of ASDF development is conducted on the
2524mailing list
2525@kbd{asdf-devel@@common-lisp.net}.
2526@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
2527
2528
2529@node FAQ, TODO list, Getting the latest version, Top
2530@comment  node-name,  next,  previous,  up
2531@chapter FAQ
2532
2533@section  ``Where do I report a bug?''
2534
2535ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
2536
2537If you're unsure about whether something is a bug, of for general discussion,
2538use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
2539
2540
2541@section ``What has changed between ASDF 1 and ASDF 2?''
2542
2543@subsection What are ASDF 1 and ASDF 2?
2544
2545On May 31st 2010, we have released ASDF 2.
2546ASDF 2 refers to release 2.000 and later.
2547(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
2548ASDF 1 to any release earlier than 1.369 or so.
2549If your ASDF doesn't sport a version, it's an old ASDF 1.
2550
2551ASDF 2 and its release candidates push
2552@code{:asdf2} onto @code{*features*} so that if you are writing
2553ASDF-dependent code you may check for this feature
2554to see if the new API is present.
2555@emph{All} versions of ASDF should have the @code{:asdf} feature.
2556
2557If you are experiencing problems or limitations of any sort with ASDF 1,
2558we recommend that you should upgrade to ASDF 2,
2559or whatever is the latest release.
2560
2561
2562@subsection ASDF can portably name files in subdirectories
2563
2564Common Lisp namestrings are not portable,
2565except maybe for logical pathnamestrings,
2566that themselves have various limitations and require a lot of setup
2567that is itself ultimately non-portable.
2568
2569In ASDF 1, the only portable ways to refer to pathnames inside systems and components
2570were very awkward, using @code{#.(make-pathname ...)} and
2571@code{#.(merge-pathnames ...)}.
2572Even the above were themselves were inadequate in the general case
2573due to host and device issues, unless horribly complex patterns were used.
2574Plenty of simple cases that looked portable actually weren't,
2575leading to much confusion and greavance.
2576
2577ASDF 2 implements its own portable syntax for strings as pathname specifiers.
2578Naming files within a system definition becomes easy and portable again.
2579@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
2580@code{asdf-utilities:merge-pathnames*},
2581@code{asdf::merge-component-name-type}.
2582
2583On the other hand, there are places where systems used to accept namestrings
2584where you must now use an explicit pathname object:
2585@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2586must now be written with the @code{#p} syntax:
2587@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2588
2589@xref{The defsystem grammar,,Pathname specifiers}.
2590
2591
2592@subsection Output translations
2593
2594A popular feature added to ASDF was output pathname translation:
2595@code{asdf-binary-locations}, @code{common-lisp-controller},
2596@code{cl-launch} and other hacks were all implementing it in ways
2597both mutually incompatible and difficult to configure.
2598
2599Output pathname translation is essential to share
2600source directories of portable systems across multiple implementations
2601or variants thereof,
2602or source directories of shared installations of systems across multiple users,
2603or combinations of the above.
2604
2605In ASDF 2, a standard mechanism is provided for that,
2606@code{asdf-output-translations},
2607with sensible defaults, adequate configuration languages,
2608a coherent set of configuration files and hooks,
2609and support for non-Unix platforms.
2610
2611@xref{Controlling where ASDF saves compiled files}.
2612
2613@subsection Source Registry Configuration
2614
2615Configuring ASDF used to require special magic
2616to be applied just at the right moment,
2617between the moment ASDF is loaded and the moment it is used,
2618in a way that is specific to the user,
2619the implementation he is using and the application he is building.
2620
2621This made for awkward configuration files and startup scripts
2622that could not be shared between users, managed by administrators
2623or packaged by distributions.
2624
2625ASDF 2 provides a well-documented way to configure ASDF,
2626with sensible defaults, adequate configuration languages,
2627and a coherent set of configuration files and hooks.
2628
2629We believe it's a vast improvement because it decouples
2630application distribution from library distribution.
2631The application writer can avoid thinking where the libraries are,
2632and the library distributor (dpkg, clbuild, advanced user, etc.)
2633can configure them once and for every application.
2634Yet settings can be easily overridden where needed,
2635so whoever needs control has exactly as much as required.
2636
2637At the same time, ASDF 2 remains compatible
2638with the old magic you may have in your build scripts
2639(using @code{*central-registry*} and
2640@code{*system-definition-search-functions*})
2641to tailor the ASDF configuration to your build automation needs,
2642and also allows for new magic, simpler and more powerful magic.
2643
2644@xref{Controlling where ASDF searches for systems}.
2645
2646
2647@subsection Usual operations are made easier to the user
2648
2649In ASDF 1, you had to use the awkward syntax
2650@code{(asdf:oos 'asdf:load-op :foo)}
2651to load a system,
2652and similarly for @code{compile-op}, @code{test-op}.
2653
2654In ASDF 2, you can use shortcuts for the usual operations:
2655@code{(asdf:load-system :foo)}, and
2656similarly for @code{compile-system}, @code{test-system}.
2657
2658
2659@subsection Many bugs have been fixed
2660
2661The following issues and many others have been fixed:
2662
2663@itemize
2664@item
2665The infamous TRAVERSE function has been revamped significantly,
2666with many bugs squashed.
2667In particular, dependencies were not correctly propagated
2668across submodules within a system but now are.
2669The :version and :feature features and
2670the :force (system1 .. systemN) feature have been fixed.
2671
2672@item
2673Performance has been notably improved for large systems
2674(say with thousands of components) by using
2675hash-tables instead of linear search,
2676and linear-time list accumulation
2677instead of quadratic-time recursive appends.
2678
2679@item
2680Many features used to not be portable,
2681especially where pathnames were involved.
2682Windows support was notably quirky because of such non-portability.
2683
2684@item
2685The internal test suite used to massively fail on many implementations.
2686While still incomplete, it now fully passes
2687on all implementations supported by the test suite,
2688except for GCL (due to GCL bugs).
2689
2690@item
2691Support was lacking for some implementations.
2692ABCL and GCL were notably wholly broken.
2693ECL extensions were not integrated in the ASDF release.
2694
2695@item
2696The documentation was grossly out of date.
2697
2698@end itemize
2699
2700
2701@subsection ASDF itself is versioned
2702
2703Between new features, old bugs fixed, and new bugs introduced,
2704there were various releases of ASDF in the wild,
2705and no simple way to check which release had which feature set.
2706People using or writing systems had to either make worst-case assumptions
2707as to what features were available and worked,
2708or take great pains to have the correct version of ASDF installed.
2709
2710With ASDF 2, we provide a new stable set of working features
2711that everyone can rely on from now on.
2712Use @code{#+asdf2} to detect presence of ASDF 2,
2713@code{(asdf:version-satisfies (asdf:asdf-version) "2.000")}
2714to check the availability of a version no earlier than required.
2715
2716
2717@subsection ASDF can be upgraded
2718
2719When an old version of ASDF was loaded,
2720it was very hard to upgrade ASDF in your current image
2721without breaking everything.
2722Instead you have to exit the Lisp process and
2723somehow arrange to start a new one from a simpler image.
2724Something that can't be done from within Lisp,
2725making automation of it difficult,
2726which compounded with difficulty in configuration,
2727made the task quite hard.
2728Yet as we saw before, the task would have been required
2729to not have to live with the worst case or non-portable
2730subset of ASDF features.
2731
2732With ASDF 2, it is easy to upgrade
2733from ASDF 2 to later versions from within Lisp,
2734and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
2735We support hot upgrade of ASDF and any breakage is a bug
2736that we will do our best to fix.
2737There are still limitations on upgrade, though,
2738most notably the fact that after you upgrade ASDF,
2739you must also reload or upgrade all ASDF extensions.
2740
2741@subsection Decoupled release cycle
2742
2743When vendors were releasing their Lisp implementations with ASDF,
2744they had to basically never change version
2745because neither upgrade nor downgrade was possible
2746without breaking something for someone,
2747and no obvious upgrade path was visible and recommendable.
2748
2749With ASDF 2, upgrade is possible, easy and can be recommended.
2750This means that vendors can safely ship a recent version of ASDF,
2751confident that if a user isn't fully satisfied,
2752he can easily upgrade ASDF and deal
2753with a supported recent version of it.
2754This means that release cycles will be causally decoupled,
2755the practical consequence of which will mean faster convergence
2756towards the latest version for everyone.
2757
2758
2759@subsection Pitfalls of ASDF 2
2760
2761The main pitfalls in upgrading to ASDF 2 seem to be related
2762to the output translation mechanism.
2763
2764@itemize
2765
2766@item
2767Output translations is enabled by default. This may surprise some users,
2768most of them in pleasant way (we hope), a few of them in an unpleasant way.
2769It is trivial to disable output translations.
2770@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
2771
2772@item
2773Some systems in the large have been known not to play well with output translations.
2774They were relatively easy to fix.
2775Once again, it is also easy to disable output translations,
2776or to override its configuration.
2777
2778@item
2779The new ASDF output translations are incompatible with ASDF-Binary-Locations.
2780They replace A-B-L, and there is compatibility mode to emulate
2781your previous A-B-L configuration.
2782See @code{asdf:enable-asdf-binary-locations-compatibility} in
2783@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
2784But thou shall not load ABL on top of ASDF 2.
2785
2786@item
2787ASDF pathname designators are now specified in places where they were unspecified,
2788and a few small adjustments have to be made to some non-portable defsystems.
2789Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
2790a logical pathname (or implementation-dependent hierarchical pathname)
2791must now be specified with @code{#p} syntax
2792where the namestring might have previously sufficed;
2793moreover when evaluation is desired @code{#.} must be used,
2794where it wasn't necessary in the toplevel @code{:pathname} argument.
2795
2796@end itemize
2797
2798Other issues include the following:
2799
2800@itemize
2801
2802@item
2803There is a slight performance bug, notably on SBCL,
2804when initially searching for @file{asd} files,
2805the implicit @code{(directory "/configured/path/**/*.asd")}
2806for every configured path @code{(:tree "/configured/path/")}
2807in your @code{source-registry} configuration can cause a slight pause.
2808Try to @code{(time (asdf:initialize-source-registry))}
2809to see how bad it is or isn't on your system.
2810If you insist on not having this pause,
2811you can avoid the pause by overriding the default source-registry configuration
2812and not use any deep @code{:tree} entry but only @code{:directory} entries
2813or shallow @code{:tree} entries.
2814Or you can fix your implementation to not be quite that slow
2815when recursing through directories.
2816
2817@item
2818On Windows, only LispWorks supports proper default configuration pathnames
2819based on the Windows registry.
2820Other implementations make do.
2821Windows support is largely untested, so please help report and fix bugs.
2822
2823@end itemize
2824
2825
2826@section Issues with installing the proper version of ASDF
2827
2828@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
2829
2830We recommend you upgrade ASDF.
2831@xref{Loading ASDF,,Upgrading ASDF}.
2832
2833If this does not work, it is a bug, and you should report it.
2834@xref{FAQ, report-bugs, Where do I report a bug}.
2835In the meantime, you can load @file{asdf.lisp} directly.
2836@xref{Loading ASDF,Loading an otherwise installed ASDF}.
2837
2838
2839@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
2840
2841Starting with current candidate releases of ASDF 2,
2842it should always be a good time to upgrade to a recent version of ASDF.
2843You may consult with the maintainer for which specific version they recommend,
2844but the latest RELEASE should be correct.
2845We trust you to thoroughly test it with your implementation before you release it.
2846If there are any issues with the current release,
2847it's a bug that you should report upstream and that we will fix ASAP.
2848
2849As to how to include ASDF, we recommend the following:
2850
2851@itemize
2852@item
2853If ASDF isn't installed yet, then @code{(require :asdf)}
2854should load the version of ASDF that is bundled with your system.
2855You may have it load some other version configured by the user,
2856if you allow such configuration.
2857
2858@item
2859If your system provides a mechanism to hook into @code{CL:REQUIRE},
2860then it would be nice to add ASDF to this hook the same way that
2861ABCL, CCL, CMUCL, ECL and SBCL do it.
2862
2863@item
2864You may, like SBCL, have ASDF be implicitly used to require systems
2865that are bundled with your Lisp distribution.
2866If you do have a few magic systems that come with your implementation
2867in a precompiled way such that one should only use the binary version
2868that goes with your distribution, like SBCL does,
2869then you should add them in the beginning of @code{wrapping-source-registry}.
2870
2871@item
2872If you have magic systems as above, like SBCL does,
2873then we explicitly ask you to @emph{NOT} distribute
2874@file{asdf.asd} as part of those magic systems.
2875You should still include the file @file{asdf.lisp} in your source distribution
2876and precompile it in your binary distribution,
2877but @file{asdf.asd} if included at all,
2878should be secluded from the magic systems,
2879in a separate file hierarchy,
2880or you may otherwise rename the system and its file to e.g.
2881@code{asdf-ecl} and @file{asdf-ecl.asd}, or
2882@code{sb-asdf} and @file{sb-asdf.asd}.
2883Indeed, if you made @file{asdf.asd} a magic system,
2884then users would no longer be able to upgrade ASDF using ASDF itself
2885to some version of their preference that
2886they maintain independently from your Lisp distribution.
2887
2888@item
2889If you do not have any such magic systems, or have other non-magic systems
2890that you want to bundle with your implementation,
2891then you may add them to the @code{default-source-registry},
2892and you are welcome to include @file{asdf.asd} amongst them.
2893
2894@item
2895Please send us upstream any patches you make to ASDF itself,
2896so we can merge them back in for the benefit of your users
2897when they upgrade to the upstream version.
2898
2899@end itemize
2900
2901
2902
2903@section Issues with configuring ASDF
2904
2905@subsection ``How can I customize where fasl files are stored?''
2906
2907@xref{Controlling where ASDF saves compiled files}.
2908
2909Note that in the past there was an add-on to ASDF called
2910@code{ASDF-binary-locations}, developed by Gary King.
2911That add-on has been merged into ASDF proper,
2912then superseded by the @code{asdf-output-translations} facility.
2913
2914Note that use of @code{asdf-output-translations}
2915can interfere with one aspect of your systems
2916--- if your system uses @code{*load-truename*} to find files
2917(e.g., if you have some data files stored with your program),
2918then the relocation that this ASDF customization performs
2919is likely to interfere.
2920Use @code{asdf:system-relative-pathname} to locate a file
2921in the source directory of some system, and
2922use @code{asdf:apply-output-translations} to locate a file
2923whose pathname has been translated by the facility.
2924
2925@subsection ``How can I wholly disable the compiler output cache?''
2926
2927To permanently disable the compiler output cache
2928for all future runs of ASDF, you can:
2929
2930@example
2931mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
2932echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
2933@end example
2934
2935This assumes that you didn't otherwise configure the ASDF files
2936(if you did, edit them again),
2937and don't somehow override the configuration at runtime
2938with a shell variable (see below) or some other runtime command
2939(e.g. some call to @code{asdf:initialize-output-translations}).
2940
2941To disable the compiler output cache in Lisp processes
2942run by your current shell, try (assuming @code{bash} or @code{zsh})
2943(on Unix and cygwin only):
2944
2945@example
2946export ASDF_OUTPUT_TRANSLATIONS=/:
2947@end example
2948
2949To disable the compiler output cache just in the current Lisp process,
2950use (after loading ASDF but before using it):
2951
2952@example
2953(asdf:disable-output-translations)
2954@end example
2955
2956@section Issues with using and extending ASDF to define systems
2957
2958@subsection ``How can I cater for unit-testing in my system?''
2959
2960ASDF provides a predefined test operation, @code{test-op}.
2961@xref{Predefined operations of ASDF, test-op}.
2962The test operation, however, is largely left to the system definer to specify.
2963@code{test-op} has been
2964a topic of considerable discussion on the
2965@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
2966and on the
2967@url{https://launchpad.net/asdf,launchpad bug-tracker}.
2968
2969Here are some guidelines:
2970
2971@itemize
2972@item
2973For a given system, @var{foo}, you will want to define a corresponding
2974test system, such as @var{foo-test}.  The reason that you will want this
2975separate system is that ASDF does not out of the box supply components
2976that are conditionally loaded.  So if you want to have source files
2977(with the test definitions) that will not be loaded except when testing,
2978they should be put elsewhere.
2979
2980@item
2981The @var{foo-test} system can be defined in an asd file of its own or
2982together with @var{foo}.  An aesthetic preference against cluttering up
2983the filesystem with extra asd files should be balanced against the
2984question of whether one might want to directly load @var{foo-test}.
2985Typically one would not want to do this except in early stages of
2986debugging.
2987
2988@item
2989Record that testing is implemented by @var{foo-test}.  For example:
2990@example
2991(defsystem @var{foo}
2992   :in-order-to ((test-op (test-op @var{foo-test})))
2993   ....)
2994
2995(defsystem @var{foo-test}
2996   :depends-on (@var{foo} @var{my-test-library} ...)
2997   ....)
2998@end example
2999@end itemize
3000
3001This procedure will allow you to support users who do not wish to
3002install your test framework.
3003
3004One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3005does not return a value.  So in current versions of ASDF there is no
3006reliable programmatic means of determining whether or not a set of tests
3007has passed, or which tests have failed.  The user must simply read the
3008console output.  This limitation has been the subject of much
3009discussion.
3010
3011@subsection ``How can I cater for documentation generation in my system?''
3012
3013The ASDF developers are currently working to add a @code{doc-op}
3014to the set of predefined ASDF operations.
3015@xref{Predefined operations of ASDF}.
3016See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3017
3018
3019
3020@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3021
3022See @code{cffi}'s @code{cffi-grovel}.
3023
3024@anchor{report-bugs}
3025
3026
3027@subsection ``I want to put my module's files at the top level.  How do I do this?''
3028
3029By default, the files contained in an asdf module go
3030in a subdirectory with the same name as the module.
3031However, this can be overridden by adding a @code{:pathname ""} argument
3032to the module description.
3033For example, here is how it could be done
3034in the spatial-trees ASDF system definition for ASDF 2:
3035
3036@example
3037(asdf:defsystem :spatial-trees
3038  :components
3039  ((:module base
3040            :pathname ""
3041            :components
3042            ((:file "package")
3043             (:file "basedefs" :depends-on ("package"))
3044             (:file "rectangles" :depends-on ("package"))))
3045   (:module tree-impls
3046            :depends-on (base)
3047            :pathname ""
3048            :components
3049            ((:file "r-trees")
3050             (:file "greene-trees" :depends-on ("r-trees"))
3051             (:file "rstar-trees" :depends-on ("r-trees"))
3052             (:file "rplus-trees" :depends-on ("r-trees"))
3053             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3054   (:module viz
3055            :depends-on (base)
3056            :pathname ""
3057            :components
3058            ((:static-file "spatial-tree-viz.lisp")))
3059   (:module tests
3060            :depends-on (base)
3061            :pathname ""
3062            :components
3063            ((:static-file "spatial-tree-test.lisp")))
3064   (:static-file "LICENCE")
3065   (:static-file "TODO")))
3066@end example
3067
3068All of the files in the @code{tree-impls} module are at the top level,
3069instead of in a @file{tree-impls/} subdirectory.
3070
3071Note that the argument to @code{:pathname} can be either a pathname object or a string.
3072A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3073but this is discouraged because the results of parsing a namestring are not portable.
3074A pathname can only be portably constructed with such syntax as
3075@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3076and similarly the current directory can only be portably specified as
3077@code{#.(make-pathname :directory '(:relative))}.
3078However, as of ASDF 2, you can portably use a string to denote a pathname.
3079The string will be parsed as a @code{/}-separated path from the current directory,
3080such that the empty string @code{""} denotes the current directory, and
3081@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3082portably denotes the same subdirectory as above.
3083When files are specified, the last @code{/}-separated component is interpreted
3084either as the name component of a pathname
3085(if the component class specifies a pathname type),
3086or as a name component plus optional dot-separated type component
3087(if the component class doesn't specifies a pathname type).
3088
3089
3090@node  TODO list, Inspiration, FAQ, Top
3091@comment  node-name,  next,  previous,  up
3092@chapter TODO list
3093
3094Here is an old list of things to do,
3095in addition to the bugs that are now tracked on launchpad:
3096@url{https://launchpad.net/asdf}.
3097
3098@section Outstanding spec questions, things to add
3099
3100** packaging systems
3101
3102*** manual page component?
3103
3104** style guide for .asd files
3105
3106You should either use keywords or be careful
3107with the package that you evaluate defsystem forms in.
3108Otherwise @code{(defsystem partition ...)}
3109being read in the @code{cl-user} package
3110will intern a @code{cl-user:partition} symbol,
3111which will then collide with the @code{partition:partition} symbol.
3112
3113Actually there's a hairier packages problem to think about too.
3114@code{in-order-to} is not a keyword:
3115if you read @code{defsystem} forms in a package that doesn't use ASDF,
3116odd things might happen.
3117
3118
3119** extending defsystem with new options
3120
3121You might not want to write a whole parser,
3122but just to add options to the existing syntax.
3123Reinstate @code{parse-option} or something akin.
3124
3125
3126** document all the error classes
3127
3128** what to do with compile-file failure
3129
3130Should check the primary return value from compile-file and see if
3131that gets us any closer to a sensible error handling strategy
3132
3133** foreign files
3134
3135lift unix-dso stuff from db-sockets
3136
3137** Diagnostics
3138
3139A ``dry run'' of an operation can be made with the following form:
3140
3141@lisp
3142(traverse (make-instance '<operation-name>)
3143          (find-system <system-name>)
3144          'explain)
3145@end lisp
3146
3147This uses unexported symbols.
3148What would be a nice interface for this functionality?
3149
3150@section Missing bits in implementation
3151
3152** reuse the same scratch package whenever a system is reloaded from disk
3153
3154** proclamations probably aren't
3155
3156** when a system is reloaded with fewer components than it previously had, odd things happen
3157
3158We should do something inventive when processing a @code{defsystem} form,
3159like take the list of kids and @code{setf} the slot to @code{nil},
3160then transfer children from old to new list as they're found.
3161
3162** (stuff that might happen later)
3163
3164*** Propagation of the @code{:force} option.
3165
3166``I notice that
3167
3168        @code{(asdf:compile-system :araneida :force t)}
3169
3170also forces compilation of every other system the @code{:araneida} system depends on.
3171This is rarely useful to me;
3172usually, when I want to force recompilation of something more than a single source file,
3173I want to recompile only one system.
3174So it would be more useful to have @code{make-sub-operation}
3175refuse to propagate @code{:force t} to other systems, and
3176propagate only something like @code{:force :recursively}.
3177
3178Ideally what we actually want is some kind of criterion that says
3179to which systems (and which operations) a @code{:force} switch will propagate.
3180
3181The problem is perhaps that ``force'' is a pretty meaningless concept.
3182How obvious is it that @code{load :force t} should force @emph{compilation}?
3183But we don't really have the right dependency setup
3184for the user to compile @code{:force t} and expect it to work
3185(files will not be loaded after compilation, so the compile
3186environment for subsequent files will be emptier than it needs to be)
3187
3188What does the user actually want to do when he forces?
3189Usually, for me, update for use with a new version of the Lisp compiler.
3190Perhaps for recovery when he suspects that something has gone wrong.
3191Or else when he's changed compilation options or configuration
3192in some way that's not reflected in the dependency graph.
3193
3194Other possible interface: have a ``revert'' function akin to @code{make clean}.
3195
3196@lisp
3197(asdf:revert 'asdf:compile-op 'araneida)
3198@end lisp
3199
3200would delete any files produced by @code{(compile-system :araneida)}.
3201Of course, it wouldn't be able to do much about stuff in the image itself.
3202
3203How would this work?
3204
3205@code{traverse}
3206
3207There's a difference between a module's dependencies (peers)
3208and its components (children).
3209Perhaps there's a similar difference in operations?
3210For example, @code{(load "use") depends-on (load "macros")} is a peer,
3211whereas @code{(load "use") depends-on (compile "use")}
3212is more of a ``subservient'' relationship.
3213
3214@node  Inspiration, Concept Index, TODO list, Top
3215@comment  node-name,  next,  previous,  up
3216@chapter Inspiration
3217
3218@section mk-defsystem (defsystem-3.x)
3219
3220We aim to solve basically the same problems as @code{mk-defsystem} does.
3221However, our architecture for extensibility
3222better exploits CL language features (and is documented),
3223and we intend to be portable rather than just widely-ported.
3224No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3225that implementation has the unenviable task
3226of supporting pre-ANSI implementations, which is no longer necessary.
3227
3228The surface defsystem syntax of asdf is more-or-less compatible with
3229@code{mk-defsystem}, except that we do not support
3230the @code{source-foo} and @code{binary-foo} prefixes
3231for separating source and binary files, and
3232we advise the removal of all options to specify pathnames.
3233
3234The @code{mk-defsystem} code for topologically sorting
3235a module's dependency list was very useful.
3236
3237@section defsystem-4 proposal
3238
3239Marco and Peter's proposal for defsystem 4 served as the driver for
3240many of the features in here.  Notable differences are:
3241
3242@itemize
3243@item
3244We don't specify output files or output file extensions
3245as part of the system.
3246
3247If you want to find out what files an operation would create,
3248ask the operation.
3249
3250@item
3251We don't deal with CL packages
3252
3253If you want to compile in a particular package, use an @code{in-package} form
3254in that file (ilisp / SLIME will like you more if you do this anyway)
3255
3256@item
3257There is no proposal here that @code{defsystem} does version control.
3258
3259A system has a given version which can be used to check dependencies,
3260but that's all.
3261@end itemize
3262
3263The defsystem 4 proposal tends to look more at the external features,
3264whereas this one centres on a protocol for system introspection.
3265
3266@section kmp's ``The Description of Large Systems'', MIT AI Memu 801
3267
3268Available in updated-for-CL form on the web at
3269@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3270
3271In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3272and concept to deal with creating component trees
3273from @code{defsystem} surface syntax.
3274[ this is not true right now, though it used to be and
3275probably will be again soon ]
3276
3277
3278@c -------------------
3279
3280
3281@node Concept Index, Function and Class Index, Inspiration, Top
3282@unnumbered Concept Index
3283
3284@printindex cp
3285
3286@node Function and Class Index, Variable Index, Concept Index, Top
3287@unnumbered Function and Class Index
3288
3289@printindex fn
3290
3291@node Variable Index,  , Function and Class Index, Top
3292@unnumbered Variable Index
3293
3294@printindex vr
3295
3296@bye
Note: See TracBrowser for help on using the repository browser.