source: trunk/abcl/doc/asdf/asdf.texinfo @ 13417

Last change on this file since 13417 was 13417, checked in by Mark Evenson, 10 years ago

Upgrade to asdf-2.017.

File size: 137.4 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2011 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex compile-system
201@findex test-system
202@cindex system directory designator
203@findex operate
204@findex oos
205
206@c @menu
207@c * Installing ASDF::
208@c @end menu
209
210
211@section Loading a pre-installed ASDF
212
213Many Lisp implementations include a copy of ASDF.
214You can usually load this copy using Common Lisp's @code{require} function:
215
216@lisp
217(require "asdf")
218@end lisp
219
220As of the writing of this manual,
221the following implementations provide ASDF 2 this way:
222abcl allegro ccl clisp cmucl ecl sbcl xcl.
223The following implementations don't provide it yet but will in a future release:
224lispworks scl.
225The following implementations are obsolete and most probably will never bundle it:
226cormancl gcl genera mcl.
227
228If the implementation you are using doesn't provide ASDF 2,
229see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
230If that implementation is still actively maintained,
231you may also send a bug report to your Lisp vendor and complain
232about their failing to provide ASDF.
233
234NB: all implementations except clisp also accept
235@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
236For portability's sake, you probably want to use @code{(require "asdf")}.
237
238
239@section Checking whether ASDF is loaded
240
241To check whether ASDF is properly loaded in your current Lisp image,
242you can run this form:
243
244@lisp
245(asdf:asdf-version)
246@end lisp
247
248If it returns a string,
249that is the version of ASDF that is currently installed.
250
251If it raises an error,
252then either ASDF is not loaded, or
253you are using an old version of ASDF.
254
255You can check whether an old version is loaded
256by checking if the ASDF package is present.
257The form below will allow you to programmatically determine
258whether a recent version is loaded, an old version is loaded,
259or none at all:
260
261@lisp
262(or #+asdf2 (asdf:asdf-version) #+asdf :old)
263@end lisp
264
265If it returns a version number, that's the version of ASDF installed.
266If it returns the keyword @code{:OLD},
267then you're using an old version of ASDF (from before 1.635).
268If it returns @code{NIL} then ASDF is not installed.
269
270If you are experiencing problems with ASDF,
271please try upgrading to the latest released version,
272using the method below,
273before you contact us and raise an issue.
274
275
276@section Upgrading ASDF
277
278If your implementation does provide ASDF 2 or later,
279and you want to upgrade to a more recent version,
280just install ASDF like any other package
281(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
282configure ASDF as usual (see @pxref{Configuring ASDF} below),
283and upgrade with:
284
285@lisp
286(require "asdf")
287(asdf:load-system :asdf)
288@end lisp
289
290If on the other hand, your implementation only provides an old ASDF,
291you will require a special configuration step and an old-style loading:
292
293@lisp
294(require "asdf")
295(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
296(asdf:oos 'asdf:load-op :asdf)
297@end lisp
298
299Don't forget the trailing @code{/} at the end of your pathname.
300
301Also, note that older versions of ASDF won't redirect their output,
302or at least won't do it according to your usual ASDF 2 configuration.
303You therefore need write access on the directory
304where you install the new ASDF,
305and make sure you're not using it
306for multiple mutually incompatible implementations.
307At worst, you may have to have multiple copies of the new ASDF,
308e.g. one per implementation installation, to avoid clashes.
309Note that to our knowledge all implementations that provide ASDF
310provide ASDF 2 in their latest release, so
311you may want to upgrade your implementation rather than go through that hoop.
312
313Finally, note that there are some limitations to upgrading ASDF:
314@itemize
315@item
316Any ASDF extension becomes invalid, and will need to be reloaded.
317This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
318Starting with ASDF 2.014.8, ASDF will actually invalidate
319all previously loaded systems when it is loaded on top of
320a different ASDF version.
321@item
322Until all implementations provide ASDF 2.015 or later,
323it is safer if you upgrade ASDF and its extensions as a special step
324at the very beginning of whatever script you are running,
325before you start using ASDF to load anything else.
326@item
327Until all implementations provide ASDF 2.015 or later,
328it is unsafe to upgrade ASDF as part of loading a system
329that depends on a more recent version of ASDF,
330since the new one might shadow the old one while the old one is running,
331and the running old one will be confused
332when extensions are loaded into the new one.
333In the meantime, we recommend that your systems should @emph{not} specify
334@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
335but instead that they check that a recent enough ASDF is installed,
336with such code as:
337@example
338(unless (or #+asdf2 (asdf:version-satisfies
339                     (asdf:asdf-version) *required-asdf-version*))
340  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
341@end example
342@end itemize
343
344
345@section Loading an otherwise installed ASDF
346
347If your implementation doesn't include ASDF,
348if for some reason the upgrade somehow fails,
349does not or cannot apply to your case,
350you will have to install the file @file{asdf.lisp}
351somewhere and load it with:
352
353@lisp
354(load "/path/to/your/installed/asdf.lisp")
355@end lisp
356
357The single file @file{asdf.lisp} is all you normally need to use ASDF.
358
359You can extract this file from latest release tarball on the
360@url{http://common-lisp.net/project/asdf/,ASDF website}.
361If you are daring and willing to report bugs, you can get
362the latest and greatest version of ASDF from its git repository.
363@xref{Getting the latest version}.
364
365For maximum convenience you might want to have ASDF loaded
366whenever you start your Lisp implementation,
367for example by loading it from the startup script or dumping a custom core
368--- check your Lisp implementation's manual for details.
369
370
371@node Configuring ASDF, Using ASDF, Loading ASDF, Top
372@comment  node-name,  next,  previous,  up
373
374@chapter Configuring ASDF
375
376@section Configuring ASDF to find your systems
377
378So it may compile and load your systems, ASDF must be configured to find
379the @file{.asd} files that contain system definitions.
380
381Since ASDF 2, the preferred way to configure where ASDF finds your systems is
382the @code{source-registry} facility,
383fully described in its own chapter of this manual.
384@xref{Controlling where ASDF searches for systems}.
385
386The default location for a user to install Common Lisp software is under
387@file{~/.local/share/common-lisp/source/}.
388If you install software there, you don't need further configuration.
389If you're installing software yourself at a location that isn't standard,
390you have to tell ASDF where you installed it. See below.
391If you're using some tool to install software (e.g. Quicklisp),
392the authors of that tool should already have configured ASDF.
393
394The simplest way to add a path to your search path,
395say @file{/home/luser/.asd-link-farm/}
396is to create the directory
397@file{~/.config/common-lisp/source-registry.conf.d/}
398and there create a file with any name of your choice,
399and with the type @file{conf},
400for instance @file{42-asd-link-farm.conf}
401containing the line:
402
403@kbd{(:directory "/home/luser/.asd-link-farm/")}
404
405If you want all the subdirectories under @file{/home/luser/lisp/}
406to be recursively scanned for @file{.asd} files, instead use:
407
408@kbd{(:tree "/home/luser/lisp/")}
409
410Note that your Operating System distribution or your system administrator
411may already have configured system-managed libraries for you.
412
413The required @file{.conf} extension allows you to have disabled files
414or editor backups (ending in @file{~}), and works portably
415(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
416Excluded are files the name of which start with a @file{.} character.
417It is customary to start the filename with two digits
418that specify the order in which the directories will be scanned.
419
420ASDF will automatically read your configuration
421the first time you try to find a system.
422You can reset the source-registry configuration with:
423
424@lisp
425(asdf:clear-source-registry)
426@end lisp
427
428And you probably should do so before you dump your Lisp image,
429if the configuration may change
430between the machine where you save it at the time you save it
431and the machine you resume it at the time you resume it.
432
433
434@section Configuring ASDF to find your systems --- old style
435
436The old way to configure ASDF to find your systems is by
437@code{push}ing directory pathnames onto the variable
438@code{asdf:*central-registry*}.
439
440You must configure this variable between the time you load ASDF
441and the time you first try to use it.
442Loading and configuring ASDF presumably happen
443as part of some initialization script that builds or starts
444your Common Lisp software system.
445(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
446
447The @code{asdf:*central-registry*} is empty by default in ASDF 2,
448but is still supported for compatibility with ASDF 1.
449When used, it takes precedence over the above source-registry@footnote{
450It is possible to further customize
451the system definition file search.
452That's considered advanced use, and covered later:
453search forward for
454@code{*system-definition-search-functions*}.
455@xref{Defining systems with defsystem}.}.
456
457For instance, if you wanted ASDF to find the @file{.asd} file
458@file{/home/me/src/foo/foo.asd} your initialization script
459could after it loads ASDF with @code{(require "asdf")}
460configure it with:
461
462@lisp
463(push "/home/me/src/foo/" asdf:*central-registry*)
464@end lisp
465
466Note the trailing slash: when searching for a system,
467ASDF will evaluate each entry of the central registry
468and coerce the result to a pathname@footnote{
469ASDF will indeed call @code{EVAL} on each entry.
470It will also skip entries that evaluate to @code{NIL}.
471
472Strings and pathname objects are self-evaluating,
473in which case the @code{EVAL} step does nothing;
474but you may push arbitrary SEXP onto the central registry,
475that will be evaluated to compute e.g. things that depend
476on the value of shell variables or the identity of the user.
477
478The variable @code{asdf:*central-registry*} is thus a list of
479``system directory designators''.
480A @dfn{system directory designator} is a form
481which will be evaluated whenever a system is to be found,
482and must evaluate to a directory to look in.
483By ``directory'' here, we mean
484``designator for a pathname with a supplied DIRECTORY component''.
485}
486at which point the presence of the trailing directory name separator
487is necessary to tell Lisp that you're discussing a directory
488rather than a file.
489
490Typically, however, there are a lot of @file{.asd} files, and
491a common idiom was to have to put
492a bunch of @emph{symbolic links} to @file{.asd} files
493in a common directory
494and push @emph{that} directory (the ``link farm'')
495to the
496@code{asdf:*central-registry*}
497instead of pushing each of the many involved directories
498to the @code{asdf:*central-registry*}.
499ASDF knows how to follow such @emph{symlinks}
500to the actual file location when resolving the paths of system components
501(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
502if you try aliases under MacOS, we are curious to hear about your experience).
503
504For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
505is a member of @code{*central-registry*}, you could set up the
506system @var{foo} for loading with asdf with the following
507commands at the shell:
508
509@example
510$ cd /home/me/cl/systems/
511$ ln -s ~/src/foo/foo.asd .
512@end example
513
514This old style for configuring ASDF is not recommended for new users,
515but it is supported for old users, and for users who want to programmatically
516control what directories are added to the ASDF search path.
517
518
519@section Configuring where ASDF stores object files
520@findex clear-output-locations
521
522ASDF lets you configure where object files will be stored.
523Sensible defaults are provided and
524you shouldn't normally have to worry about it.
525
526This allows the same source code repository may be shared
527between several versions of several Common Lisp implementations,
528between several users using different compilation options
529and without write privileges on shared source directories, etc.
530This also allows to keep source directories uncluttered
531by plenty of object files.
532
533Starting with ASDF 2, the @code{asdf-output-translations} facility
534was added to ASDF itself, that controls where object files will be stored.
535This facility is fully described in a chapter of this manual,
536@ref{Controlling where ASDF saves compiled files}.
537
538The simplest way to add a translation to your search path,
539say from @file{/foo/bar/baz/quux/}
540to @file{/where/i/want/my/fasls/}
541is to create the directory
542@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
543and there create a file with any name of your choice and the type @file{conf},
544for instance @file{42-bazquux.conf}
545containing the line:
546
547@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
548
549To disable output translations for source under a given directory,
550say @file{/toto/tata/}
551you can create a file @file{40-disable-toto.conf}
552with the line:
553
554@kbd{("/toto/tata/")}
555
556To wholly disable output translations for all directories,
557you can create a file @file{00-disable.conf}
558with the line:
559
560@kbd{(t t)}
561
562Note that your Operating System distribution or your system administrator
563may already have configured translations for you.
564In absence of any configuration, the default is to redirect everything
565under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
566@xref{Controlling where ASDF searches for systems}, for full details.
567
568The required @file{.conf} extension allows you to have disabled files
569or editor backups (ending in @file{~}), and works portably
570(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
571Excluded are files the name of which start with a @file{.} character.
572It is customary to start the filename with two digits
573that specify the order in which the directories will be scanned.
574
575ASDF will automatically read your configuration
576the first time you try to find a system.
577You can reset the source-registry configuration with:
578
579@lisp
580(asdf:clear-output-translations)
581@end lisp
582
583And you probably should do so before you dump your Lisp image,
584if the configuration may change
585between the machine where you save it at the time you save it
586and the machine you resume it at the time you resume it.
587
588Finally note that before ASDF 2,
589other ASDF add-ons offered the same functionality,
590each in subtly different and incompatible ways:
591ASDF-Binary-Locations, cl-launch, common-lisp-controller.
592ASDF-Binary-Locations is now not needed anymore and should not be used.
593cl-launch 3.000 and common-lisp-controller 7.2 have been updated
594to just delegate this functionality to ASDF.
595
596@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
597@comment  node-name,  next,  previous,  up
598
599@chapter Using ASDF
600
601@section Loading a system
602
603The system @var{foo} is loaded (and compiled, if necessary)
604by evaluating the following Lisp form:
605
606@example
607(asdf:load-system :@var{foo})
608@end example
609
610On some implementations (namely recent versions of
611ABCL, Clozure CL, CLISP, CMUCL, ECL, SBCL and SCL),
612ASDF hooks into the @code{CL:REQUIRE} facility
613and you can just use:
614
615@example
616(require :@var{foo})
617@end example
618
619In older versions of ASDF, you needed to use
620@code{(asdf:oos 'asdf:load-op :@var{foo})}.
621If your ASDF is too old to provide @code{asdf:load-system} though
622we recommend that you upgrade to ASDF 2.
623@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
624
625Note the name of a system is specified as a string or a symbol,
626typically a keyword.
627If a symbol (including a keyword), its name is taken and lowercased.
628The name must be a suitable value for the @code{:name} initarg
629to @code{make-pathname} in whatever filesystem the system is to be found.
630The lower-casing-symbols behaviour is unconventional,
631but was selected after some consideration.
632Observations suggest that the type of systems we want to support
633either have lowercase as customary case (unix, mac, windows)
634or silently convert lowercase to uppercase (lpns),
635so this makes more sense than attempting to use @code{:case :common},
636which is reported not to work on some implementations
637
638
639@section Other Operations
640
641ASDF provides three commands for the most common system operations:
642@code{load-system}, @code{compile-system} or @code{test-system}.
643
644Because ASDF is an extensible system
645for defining @emph{operations} on @emph{components},
646it also provides a generic function @code{operate}
647(which is usually abbreviated by @code{oos}).
648You'll use @code{oos} whenever you want to do something beyond
649compiling, loading and testing.
650
651Output from ASDF and ASDF extensions are supposed to be sent
652to the CL stream @code{*standard-output*},
653and so rebinding that stream around calls to @code{asdf:operate}
654should redirect all output from ASDF operations.
655
656Reminder: before ASDF can operate on a system, however,
657it must be able to find and load that system's definition.
658@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
659
660
661@section Summary
662
663To use ASDF:
664
665@itemize
666@item
667Load ASDF itself into your Lisp image, either through
668@code{(require "asdf")} or else through
669@code{(load "/path/to/asdf.lisp")}.
670
671@item
672Make sure ASDF can find system definitions
673thanks to proper source-registry configuration.
674
675@item
676Load a system with @code{(load-system :my-system)}
677or use some other operation on some system of your choice.
678
679@end itemize
680
681@section Moving on
682
683That's all you need to know to use ASDF to load systems written by others.
684The rest of this manual deals with writing system definitions
685for Common Lisp software you write yourself,
686including how to extend ASDF to define new operation and component types.
687
688
689@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
690@comment  node-name,  next,  previous,  up
691@chapter Defining systems with defsystem
692
693This chapter describes how to use asdf to define systems and develop
694software.
695
696
697@menu
698* The defsystem form::
699* A more involved example::
700* The defsystem grammar::
701* Other code in .asd files::
702@end menu
703
704@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
705@comment  node-name,  next,  previous,  up
706@section The defsystem form
707
708Systems can be constructed programmatically
709by instantiating components using @code{make-instance}.
710Most of the time, however, it is much more practical to use
711a static @code{defsystem} form.
712This section begins with an example of a system definition,
713then gives the full grammar of @code{defsystem}.
714
715Let's look at a simple system.
716This is a complete file that would
717usually be saved as @file{hello-lisp.asd}:
718
719@lisp
720(in-package :asdf)
721
722(defsystem "hello-lisp"
723  :description "hello-lisp: a sample Lisp system."
724  :version "0.2.1"
725  :author "Joe User <joe@@example.com>"
726  :licence "Public Domain"
727  :components ((:file "packages")
728               (:file "macros" :depends-on ("packages"))
729               (:file "hello" :depends-on ("macros"))))
730@end lisp
731
732Some notes about this example:
733
734@itemize
735
736@item
737The file starts with an @code{in-package} form
738to use package @code{asdf}.
739You could instead start your definition by using
740a qualified name @code{asdf:defsystem}.
741
742@item
743If in addition to simply using @code{defsystem},
744you are going to define functions,
745create ASDF extension, globally bind symbols, etc.,
746it is recommended that to avoid namespace pollution between systems,
747you should create your own package for that purpose,
748for instance replacing the above @code{(in-package :asdf)} with:
749
750@lisp
751(defpackage :foo-system
752  (:use :cl :asdf))
753
754(in-package :foo-system)
755@end lisp
756
757@item
758The @code{defsystem} form defines a system named @code{hello-lisp}
759that contains three source files:
760@file{packages}, @file{macros} and @file{hello}.
761
762@item
763The file @file{macros} depends on @file{packages}
764(presumably because the package it's in is defined in @file{packages}),
765and the file @file{hello} depends on @file{macros}
766(and hence, transitively on @file{packages}).
767This means that ASDF will compile and load @file{packages} and @file{macros}
768before starting the compilation of file @file{hello}.
769
770@item
771The files are located in the same directory
772as the file with the system definition.
773ASDF resolves symbolic links (or Windows shortcuts)
774before loading the system definition file and
775stores its location in the resulting system@footnote{
776It is possible, though almost never necessary, to override this behaviour.}.
777This is a good thing because the user can move the system sources
778without having to edit the system definition.
779
780@c FIXME: Should have cross-reference to "Version specifiers" in the
781@c defsystem grammar, but the cross-referencing is so broken by
782@c insufficient node breakdown that I have not put one in.
783@item
784Make sure you know how the @code{:version} numbers will be parsed!  They
785are parsed as period-separated lists of integers.  I.e., in the example,
786@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
787In particular, version @code{0.2.1} is interpreted the same as
788@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
789even though the two are the same when interpreted as decimal fractions.
790@cindex version specifiers
791@cindex :version
792
793@end itemize
794
795@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
796@comment  node-name,  next,  previous,  up
797@section A more involved example
798
799Let's illustrate some more involved uses of @code{defsystem} via a
800slightly convoluted example:
801
802@lisp
803(defsystem "foo"
804  :version "1.0.0"
805  :components ((:module "mod"
806                            :components ((:file "bar")
807                                                  (:file"baz")
808                                                  (:file "quux"))
809                            :perform (compile-op :after (op c)
810                                                  (do-something c))
811                            :explain (compile-op :after (op c)
812                                            (explain-something c)))
813                         (:file "blah")))
814@end lisp
815
816The @code{:module} component named @code{"mod"} is a collection of three files,
817which will be located in a subdirectory of the main code directory named
818@file{mod} (this location can be overridden; see the discussion of the
819@code{:pathname} option in @ref{The defsystem grammar}).
820
821The method-form tokens provide a shorthand for defining methods on
822particular components.  This part
823
824@lisp
825                :perform (compile-op :after (op c)
826                          (do-something c))
827                :explain (compile-op :after (op c)
828                          (explain-something c))
829@end lisp
830
831has the effect of
832
833@lisp
834(defmethod perform :after ((op compile-op) (c (eql ...)))
835           (do-something c))
836(defmethod explain :after ((op compile-op) (c (eql ...)))
837           (explain-something c))
838@end lisp
839
840where @code{...} is the component in question.
841In this case @code{...} would expand to something like
842
843@lisp
844(find-component (find-system "foo") "mod")
845@end lisp
846
847For more details on the syntax of such forms, see @ref{The defsystem
848grammar}.
849For more details on what these methods do, @pxref{Operations} in
850@ref{The object model of ASDF}.
851
852@c The following plunge into the weeds is not appropriate in this
853@c location. [2010/10/03:rpg]
854@c note that although this also supports @code{:before} methods,
855@c they may not do what you want them to ---
856@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
857@c will run after all the dependencies and sub-components have been processed,
858@c but before the component in question has been compiled.
859
860@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
861@comment  node-name,  next,  previous,  up
862@section The defsystem grammar
863
864@c FIXME: @var typesetting not consistently used here.  We should either expand
865@c its use to everywhere, or we should kill it everywhere.
866
867
868@example
869system-definition := ( defsystem system-designator @var{system-option}* )
870
871system-option := :defsystem-depends-on system-list
872                 | module-option
873                 | option
874
875module-option := :components component-list
876                 | :serial [ t | nil ]
877                 | :if-component-dep-fails component-dep-fail-option
878
879option :=
880        | :pathname pathname-specifier
881        | :default-component-class class-name
882        | :perform method-form
883        | :explain method-form
884        | :output-files method-form
885        | :operation-done-p method-form
886        | :depends-on ( @var{dependency-def}* )
887        | :in-order-to ( @var{dependency}+ )
888
889
890system-list := ( @var{simple-component-name}* )
891
892component-list := ( @var{component-def}* )
893
894component-def  := ( component-type simple-component-name @var{option}* )
895
896component-type := :system | :module | :file | :static-file | other-component-type
897
898other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
899
900dependency-def := simple-component-name
901               | ( :feature name )
902               | ( :version simple-component-name version-specifier)
903
904dependency := (dependent-op @var{requirement}+)
905requirement := (required-op @var{required-component}+)
906             | (feature feature-name)
907dependent-op := operation-name
908required-op := operation-name | feature
909
910simple-component-name := string
911                      |  symbol
912
913pathname-specifier := pathname | string | symbol
914
915method-form := (operation-name qual lambda-list @&rest body)
916qual := method qualifier
917
918component-dep-fail-option := :fail | :try-next | :ignore
919@end example
920
921
922@subsection Component names
923
924Component names (@code{simple-component-name})
925may be either strings or symbols.
926
927@subsection Component types
928
929Component type names, even if expressed as keywords, will be looked up
930by name in the current package and in the asdf package, if not found in
931the current package.  So a component type @code{my-component-type}, in
932the current package @code{my-system-asd} can be specified as
933@code{:my-component-type}, or @code{my-component-type}.
934
935@subsection Defsystem depends on
936
937The @code{:defsystem-depends-on} option to @code{defsystem} allows the
938programmer to specify another ASDF-defined system or set of systems that
939must be loaded @emph{before} the system definition is processed.
940Typically this is used to load an ASDF extension that is used in the
941system definition.
942
943@subsection Pathname specifiers
944@cindex pathname specifiers
945
946A pathname specifier (@code{pathname-specifier})
947may be a pathname, a string or a symbol.
948When no pathname specifier is given for a component,
949which is the usual case, the component name itself is used.
950
951If a string is given, which is the usual case,
952the string will be interpreted as a Unix-style pathname
953where @code{/} characters will be interpreted as directory separators.
954Usually, Unix-style relative pathnames are used
955(i.e. not starting with @code{/}, as opposed to absolute pathnames);
956they are relative to the path of the parent component.
957Finally, depending on the @code{component-type},
958the pathname may be interpreted as either a file or a directory,
959and if it's a file,
960a file type may be added corresponding to the @code{component-type},
961or else it will be extracted from the string itself (if applicable).
962
963For instance, the @code{component-type} @code{:module}
964wants a directory pathname, and so a string @code{"foo/bar"}
965will be interpreted as the pathname @file{#p"foo/bar/"}.
966On the other hand, the @code{component-type} @code{:file}
967wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
968will be interpreted as the pathname @file{#p"foo/bar.lisp"},
969and a string @code{"foo/bar.quux"}
970will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
971Finally, the @code{component-type} @code{:static-file}
972wants a file without specifying a type, and so a string @code{"foo/bar"}
973will be interpreted as the pathname @file{#p"foo/bar"},
974and a string @code{"foo/bar.quux"}
975will be interpreted as the pathname @file{#p"foo/bar.quux"}.
976
977ASDF does not interpret the string @code{".."} to designate the parent
978directory.  This string will be passed through to the underlying
979operating system for interpretation.  We @emph{believe} that this will
980work on all platforms where ASDF is deployed, but do not guarantee this
981behavior.  A pathname object with a relative directory component of
982@code{:up} or @code{:back} is the only guaranteed way to specify a
983parent directory.
984
985If a symbol is given, it will be translated into a string,
986and downcased in the process.
987The downcasing of symbols is unconventional,
988but was selected after some consideration.
989Observations suggest that the type of systems we want to support
990either have lowercase as customary case (Unix, Mac, windows)
991or silently convert lowercase to uppercase (lpns),
992so this makes more sense than attempting to use @code{:case :common}
993as argument to @code{make-pathname},
994which is reported not to work on some implementations.
995
996Pathname objects may be given to override the path for a component.
997Such objects are typically specified using reader macros such as @code{#p}
998or @code{#.(make-pathname ...)}.
999Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1000and that the behavior of @code{parse-namestring} is completely non-portable,
1001unless you are using Common Lisp @code{logical-pathname}s
1002(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1003Pathnames made with @code{#.(make-pathname ...)}
1004can usually be done more easily with the string syntax above.
1005The only case that you really need a pathname object is to override
1006the component-type default file type for a given component.
1007Therefore, pathname objects should only rarely be used.
1008Unhappily, ASDF 1 didn't properly support
1009parsing component names as strings specifying paths with directories,
1010and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1011
1012Note that when specifying pathname objects,
1013ASDF does not do any special interpretation of the pathname
1014influenced by the component type, unlike the procedure for
1015pathname-specifying strings.
1016On the one hand, you have to be careful to provide a pathname that correctly
1017fulfills whatever constraints are required from that component type
1018(e.g. naming a directory or a file with appropriate type);
1019on the other hand, you can circumvent the file type that would otherwise
1020be forced upon you if you were specifying a string.
1021
1022@subsection Version specifiers
1023@cindex version specifiers
1024@cindex :version
1025
1026Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1027@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1028In particular, version @code{0.2.1} is interpreted the same as
1029@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1030even though the two are the same when interpreted as decimal fractions.
1031
1032System definers are encouraged to use version identifiers of the form
1033@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1034API) and patch level.
1035
1036@xref{Common attributes of components}.
1037
1038
1039@subsection Using logical pathnames
1040@cindex logical pathnames
1041
1042We do not generally recommend the use of logical pathnames,
1043especially not so to newcomers to Common Lisp.
1044However, we do support the use of logical pathnames by old timers,
1045when such is their preference.
1046
1047To use logical pathnames,
1048you will have to provide a pathname object as a @code{:pathname} specifier
1049to components that use it, using such syntax as
1050@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1051
1052You only have to specify such logical pathname
1053for your system or some top-level component.
1054Sub-components' relative pathnames,
1055specified using the string syntax for names,
1056will be properly merged with the pathnames of their parents.
1057The specification of a logical pathname host however is @emph{not}
1058otherwise directly supported in the ASDF syntax
1059for pathname specifiers as strings.
1060
1061The @code{asdf-output-translation} layer will
1062avoid trying to resolve and translate logical pathnames.
1063The advantage of this is that
1064you can define yourself what translations you want to use
1065with the logical pathname facility.
1066The disadvantage is that if you do not define such translations,
1067any system that uses logical pathnames will behave differently under
1068asdf-output-translations than other systems you use.
1069
1070If you wish to use logical pathnames you will have to configure the
1071translations yourself before they may be used.
1072ASDF currently provides no specific support
1073for defining logical pathname translations.
1074
1075Note that the reasons we do not recommend logical pathnames are that
1076(1) there is no portable way to set up logical pathnames before they are used,
1077(2) logical pathnames are limited to only portably use
1078a single character case, digits and hyphens.
1079While you can solve the first issue on your own,
1080describing how to do it on each of fifteen implementations supported by ASDF
1081is more than we can document.
1082As for the second issue, mind that the limitation is notably enforced on SBCL,
1083and that you therefore can't portably violate the limitations
1084but must instead define some encoding of your own and add individual mappings
1085to name physical pathnames that do not fit the restrictions.
1086This can notably be a problem when your Lisp files are part of a larger project
1087in which it is common to name files or directories in a way that
1088includes the version numbers of supported protocols,
1089or in which files are shared with software written
1090in different programming languages where conventions include the use of
1091underscores, dots or CamelCase in pathnames.
1092
1093
1094@subsection Serial dependencies
1095@cindex serial dependencies
1096
1097If the @code{:serial t} option is specified for a module,
1098ASDF will add dependencies for each child component,
1099on all the children textually preceding it.
1100This is done as if by @code{:depends-on}.
1101
1102@lisp
1103:serial t
1104:components ((:file "a") (:file "b") (:file "c"))
1105@end lisp
1106
1107is equivalent to
1108
1109@lisp
1110:components ((:file "a")
1111             (:file "b" :depends-on ("a"))
1112             (:file "c" :depends-on ("a" "b")))
1113@end lisp
1114
1115
1116@subsection Source location
1117
1118The @code{:pathname} option is optional in all cases for systems
1119defined via @code{defsystem},
1120and in the usual case the user is recommended not to supply it.
1121
1122Instead, ASDF follows a hairy set of rules that are designed so that
1123@enumerate
1124@item
1125@code{find-system}
1126will load a system from disk
1127and have its pathname default to the right place.
1128@item
1129This pathname information will not be overwritten with
1130@code{*default-pathname-defaults*}
1131(which could be somewhere else altogether)
1132if the user loads up the @file{.asd} file into his editor
1133and interactively re-evaluates that form.
1134@end enumerate
1135
1136If a system is being loaded for the first time,
1137its top-level pathname will be set to:
1138
1139@itemize
1140@item
1141The host/device/directory parts of @code{*load-truename*},
1142if it is bound.
1143@item
1144@code{*default-pathname-defaults*}, otherwise.
1145@end itemize
1146
1147If a system is being redefined, the top-level pathname will be
1148
1149@itemize
1150@item
1151changed, if explicitly supplied or obtained from @code{*load-truename*}
1152(so that an updated source location is reflected in the system definition)
1153@item
1154changed if it had previously been set from @code{*default-pathname-defaults*}
1155@item
1156left as before, if it had previously been set from @code{*load-truename*}
1157and @code{*load-truename*} is currently unbound
1158(so that a developer can evaluate a @code{defsystem} form
1159from within an editor without clobbering its source location)
1160@end itemize
1161
1162@subsection if-component-dep-fails option
1163
1164This option is only appropriate for module components (including
1165systems), not individual source files.
1166
1167For more information about this option, @pxref{Pre-defined subclasses of component}.
1168
1169@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1170@section Other code in .asd files
1171
1172Files containing @code{defsystem} forms
1173are regular Lisp files that are executed by @code{load}.
1174Consequently, you can put whatever Lisp code you like into these files
1175(e.g., code that examines the compile-time environment
1176and adds appropriate features to @code{*features*}).
1177However, some conventions should be followed,
1178so that users can control certain details of execution
1179of the Lisp in @file{.asd} files:
1180
1181@itemize
1182@item
1183Any informative output
1184(other than warnings and errors,
1185which are the condition system's to dispose of)
1186should be sent to the standard CL stream @code{*standard-output*},
1187so that users can easily control the disposition
1188of output from ASDF operations.
1189@end itemize
1190
1191
1192@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1193@comment  node-name,  next,  previous,  up
1194@chapter The object model of ASDF
1195
1196ASDF is designed in an object-oriented way from the ground up.
1197Both a system's structure and the operations that can be performed on systems
1198follow a protocol.
1199ASDF is extensible to new operations and to new component types.
1200This allows the addition of behaviours:
1201for example, a new component could be added for Java JAR archives,
1202and methods specialised on @code{compile-op} added for it
1203that would accomplish the relevant actions.
1204
1205This chapter deals with @emph{components}, the building blocks of a system,
1206and @emph{operations}, the actions that can be performed on a system.
1207
1208
1209
1210@menu
1211* Operations::
1212* Components::
1213@end menu
1214
1215@node  Operations, Components, The object model of ASDF, The object model of ASDF
1216@comment  node-name,  next,  previous,  up
1217@section Operations
1218@cindex operation
1219
1220An @dfn{operation} object of the appropriate type is instantiated
1221whenever the user wants to do something with a system like
1222
1223@itemize
1224@item compile all its files
1225@item load the files into a running lisp environment
1226@item copy its source files somewhere else
1227@end itemize
1228
1229Operations can be invoked directly, or examined
1230to see what their effects would be without performing them.
1231@emph{FIXME: document how!}
1232There are a bunch of methods specialised on operation and component type
1233that actually do the grunt work.
1234
1235The operation object contains whatever state is relevant for this purpose
1236(perhaps a list of visited nodes, for example)
1237but primarily is a nice thing to specialise operation methods on
1238and easier than having them all be @code{EQL} methods.
1239
1240Operations are invoked on systems via @code{operate}.
1241@anchor{operate}
1242@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1243@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1244@code{operate} invokes @var{operation} on @var{system}.
1245@code{oos} is a synonym for @code{operate}.
1246
1247@var{operation} is a symbol that is passed, along with the supplied
1248@var{initargs}, to @code{make-instance} to create the operation object.
1249@var{system} is a system designator.
1250
1251The @var{initargs} are passed to the @code{make-instance} call
1252when creating the operation object.
1253Note that dependencies may cause the operation
1254to invoke other operations on the system or its components:
1255the new operations will be created
1256with the same @var{initargs} as the original one.
1257
1258@end deffn
1259
1260@menu
1261* Predefined operations of ASDF::
1262* Creating new operations::
1263@end menu
1264
1265@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1266@comment  node-name,  next,  previous,  up
1267@subsection Predefined operations of ASDF
1268
1269All the operations described in this section are in the @code{asdf} package.
1270They are invoked via the @code{operate} generic function.
1271
1272@lisp
1273(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1274@end lisp
1275
1276@deffn Operation @code{compile-op} @&key @code{proclamations}
1277
1278This operation compiles the specified component.
1279If proclamations are supplied, they will be proclaimed.
1280This is a good place to specify optimization settings.
1281
1282When creating a new component type,
1283you should provide methods for @code{compile-op}.
1284
1285When @code{compile-op} is invoked,
1286component dependencies often cause some parts of the system
1287to be loaded as well as compiled.
1288Invoking @code{compile-op}
1289does not necessarily load all the parts of the system, though;
1290use @code{load-op} to load a system.
1291@end deffn
1292
1293@deffn Operation @code{load-op} @&key @code{proclamations}
1294
1295This operation loads a system.
1296
1297The default methods for @code{load-op} compile files before loading them.
1298For parity, your own methods on new component types should probably do so too.
1299@end deffn
1300
1301@deffn Operation @code{load-source-op}
1302
1303This operation will load the source for the files in a module
1304even if the source files have been compiled.
1305Systems sometimes have knotty dependencies
1306which require that sources are loaded
1307before they can be compiled.
1308This is how you do that.
1309
1310If you are creating a component type,
1311you need to implement this operation --- at least, where meaningful.
1312@end deffn
1313
1314@anchor{test-op}
1315@deffn Operation @code{test-op}
1316
1317This operation will perform some tests on the module.
1318The default method will do nothing.
1319The default dependency is to require
1320@code{load-op} to be performed on the module first.
1321The default @code{operation-done-p} is that the operation is @emph{never} done
1322---
1323we assume that if you invoke the @code{test-op},
1324you want to test the system, even if you have already done so.
1325
1326The results of this operation are not defined by ASDF.
1327It has proven difficult to define how the test operation
1328should signal its results to the user
1329in a way that is compatible with all of the various test libraries
1330and test techniques in use in the community.
1331@end deffn
1332
1333@c @deffn Operation test-system-version @&key minimum
1334
1335@c Asks the system whether it satisfies a version requirement.
1336
1337@c The default method accepts a string, which is expected to contain of a
1338@c number of integers separated by #\. characters.  The method is not
1339@c recursive.  The component satisfies the version dependency if it has
1340@c the same major number as required and each of its sub-versions is
1341@c greater than or equal to the sub-version number required.
1342
1343@c @lisp
1344@c (defun version-satisfies (x y)
1345@c   (labels ((bigger (x y)
1346@c           (cond ((not y) t)
1347@c                 ((not x) nil)
1348@c                 ((> (car x) (car y)) t)
1349@c                 ((= (car x) (car y))
1350@c                  (bigger (cdr x) (cdr y))))))
1351@c     (and (= (car x) (car y))
1352@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1353@c @end lisp
1354
1355@c If that doesn't work for your system, you can override it.  I hope
1356@c you have as much fun writing the new method as @verb{|#lisp|} did
1357@c reimplementing this one.
1358@c @end deffn
1359
1360@c @deffn Operation feature-dependent-op
1361
1362@c An instance of @code{feature-dependent-op} will ignore any components
1363@c which have a @code{features} attribute, unless the feature combination
1364@c it designates is satisfied by @code{*features*}.  This operation is
1365@c not intended to be instantiated directly, but other operations may
1366@c inherit from it.
1367
1368@c @end deffn
1369
1370@node  Creating new operations,  , Predefined operations of ASDF, Operations
1371@comment  node-name,  next,  previous,  up
1372@subsection Creating new operations
1373
1374ASDF was designed to be extensible in an object-oriented fashion.
1375To teach ASDF new tricks, a programmer can implement the behaviour he wants
1376by creating a subclass of @code{operation}.
1377
1378ASDF's pre-defined operations are in no way ``privileged'',
1379but it is requested that developers never use the @code{asdf} package
1380for operations they develop themselves.
1381The rationale for this rule is that we don't want to establish a
1382``global asdf operation name registry'',
1383but also want to avoid name clashes.
1384
1385An operation must provide methods for the following generic functions
1386when invoked with an object of type @code{source-file}:
1387@emph{FIXME describe this better}
1388
1389@itemize
1390
1391@item @code{output-files}
1392The @code{output-files} method determines where the method will put its files.
1393It returns two values, a list of pathnames, and a boolean.
1394If the boolean is @code{T} then the pathnames are marked
1395not be translated by enclosing @code{:around} methods.
1396If the boolean is @code{NIL} then enclosing @code{:around} methods
1397may translate these pathnames, e.g. to ensure object files
1398are somehow stored in some implementation-dependent cache.
1399@item @code{perform}
1400The @code{perform} method must call @code{output-files}
1401to find out where to put its files,
1402because the user is allowed to override.
1403@item @code{output-files}
1404for local policy @code{explain}
1405@item @code{operation-done-p},
1406if you don't like the default one
1407
1408@end itemize
1409
1410Operations that print output should send that output to the standard
1411CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1412
1413@node Components,  , Operations, The object model of ASDF
1414@comment  node-name,  next,  previous,  up
1415@section Components
1416@cindex component
1417@cindex system
1418@cindex system designator
1419@vindex *system-definition-search-functions*
1420
1421A @dfn{component} represents a source file or
1422(recursively) a collection of components.
1423A @dfn{system} is (roughly speaking) a top-level component
1424that can be found via @code{find-system}.
1425
1426A @dfn{system designator} is a string or symbol
1427and behaves just like any other component name
1428(including with regard to the case conversion rules for component names).
1429
1430
1431@defun find-system system-designator &optional (error-p t)
1432
1433Given a system designator, @code{find-system} finds and returns a system.
1434If no system is found, an error of type
1435@code{missing-component} is thrown,
1436or @code{nil} is returned if @code{error-p} is false.
1437
1438To find and update systems, @code{find-system} funcalls each element
1439in the @code{*system-definition-search-functions*} list,
1440expecting a pathname to be returned, or a system object,
1441from which a pathname may be extracted, and that will be registered.
1442The resulting pathname (if any) is loaded
1443if one of the following conditions is true:
1444
1445@itemize
1446@item
1447there is no system of that name in memory
1448@item
1449the pathname is different from that which was previously loaded
1450@item
1451the file's @code{last-modified} time exceeds the @code{last-modified} time
1452of the system in memory
1453@end itemize
1454
1455When system definitions are loaded from @file{.asd} files,
1456a new scratch package is created for them to load into,
1457so that different systems do not overwrite each others operations.
1458The user may also wish to (and is recommended to)
1459include @code{defpackage} and @code{in-package} forms
1460in his system definition files, however,
1461so that they can be loaded manually if need be.
1462
1463The default value of @code{*system-definition-search-functions*}
1464is a list of two functions.
1465The first function looks in each of the directories given
1466by evaluating members of @code{*central-registry*}
1467for a file whose name is the name of the system and whose type is @file{asd}.
1468The first such file is returned,
1469whether or not it turns out to actually define the appropriate system.
1470The second function does something similar,
1471for the directories specified in the @code{source-registry}.
1472Hence, it is strongly advised to define a system
1473@var{foo} in the corresponding file @var{foo.asd}.
1474@end defun
1475
1476
1477@menu
1478* Common attributes of components::
1479* Pre-defined subclasses of component::
1480* Creating new component types::
1481@end menu
1482
1483@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1484@comment  node-name,  next,  previous,  up
1485@subsection Common attributes of components
1486
1487All components, regardless of type, have the following attributes.
1488All attributes except @code{name} are optional.
1489
1490@subsubsection Name
1491
1492A component name is a string or a symbol.
1493If a symbol, its name is taken and lowercased.
1494
1495Unless overridden by a @code{:pathname} attribute,
1496the name will be interpreted as a pathname specifier according
1497to a Unix-style syntax.
1498@xref{The defsystem grammar,,Pathname specifiers}.
1499
1500@subsubsection Version identifier
1501@findex version-satisfies
1502@cindex :version
1503
1504This optional attribute is used by the generic function
1505@code{version-satisfies}, which tests to see if @code{:version}
1506dependencies are satisfied.
1507the version should be a string of integers separated by dots,
1508for example @samp{1.0.11}.
1509For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1510
1511@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1512@c @xref{Predefined operations of ASDF}.
1513@c @emph{Nota Bene}:
1514@c This operation, planned for ASDF 1,
1515@c is still not implemented yet as of ASDF 2.
1516@c Don't hold your breath.
1517
1518
1519
1520@subsubsection Required features
1521
1522@emph{FIXME: This subsection seems to contradict the
1523@code{defsystem} grammar subsection,
1524which doesn't provide any obvious way to specify required features.
1525Furthermore, in 2009, discussions on the
1526@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1527suggested that the specification of required features may be broken,
1528and that no one may have been using them for a while.
1529Please contact the
1530@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1531if you are interested in getting this features feature fixed.}
1532
1533Traditionally defsystem users have used reader conditionals
1534to include or exclude specific per-implementation files.
1535This means that any single implementation cannot read the entire system,
1536which becomes a problem if it doesn't wish to compile it,
1537but instead for example to create an archive file containing all the sources,
1538as it will omit to process the system-dependent sources for other systems.
1539
1540Each component in an asdf system may therefore specify features using
1541the same syntax as @code{#+} does, and it will (somehow) be ignored for
1542certain operations unless the feature conditional is a member of
1543@code{*features*}.
1544
1545
1546@subsubsection Dependencies
1547
1548This attribute specifies dependencies of the component on its siblings.
1549It is optional but often necessary.
1550
1551There is an excitingly complicated relationship between the initarg
1552and the method that you use to ask about dependencies
1553
1554Dependencies are between (operation component) pairs.
1555In your initargs for the component, you can say
1556
1557@lisp
1558:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1559              (load-op (load-op "foo")))
1560@end lisp
1561
1562This means the following things:
1563@itemize
1564@item
1565before performing compile-op on this component, we must perform
1566load-op on @var{a} and @var{b}, and compile-op on @var{c},
1567@item
1568before performing @code{load-op}, we have to load @var{foo}
1569@end itemize
1570
1571The syntax is approximately
1572
1573@verbatim
1574(this-op {(other-op required-components)}+)
1575
1576required-components := component-name
1577                     | (required-components required-components)
1578
1579component-name := string
1580                | (:version string minimum-version-object)
1581@end verbatim
1582
1583Side note:
1584
1585This is on a par with what ACL defsystem does.
1586mk-defsystem is less general: it has an implied dependency
1587
1588@verbatim
1589  for all x, (load x) depends on (compile x)
1590@end verbatim
1591
1592and using a @code{:depends-on} argument to say that @var{b} depends on
1593@var{a} @emph{actually} means that
1594
1595@verbatim
1596  (compile b) depends on (load a)
1597@end verbatim
1598
1599This is insufficient for e.g. the McCLIM system, which requires that
1600all the files are loaded before any of them can be compiled ]
1601
1602End side note
1603
1604In ASDF, the dependency information for a given component and operation
1605can be queried using @code{(component-depends-on operation component)},
1606which returns a list
1607
1608@lisp
1609((load-op "a") (load-op "b") (compile-op "c") ...)
1610@end lisp
1611
1612@code{component-depends-on} can be subclassed for more specific
1613component/operation types: these need to @code{(call-next-method)}
1614and append the answer to their dependency, unless
1615they have a good reason for completely overriding the default dependencies.
1616
1617If it weren't for CLISP, we'd be using @code{LIST} method
1618combination to do this transparently.
1619But, we need to support CLISP.
1620If you have the time for some CLISP hacking,
1621I'm sure they'd welcome your fixes.
1622@c Doesn't CLISP now support LIST method combination?
1623
1624See the discussion of the semantics of @code{:version} in the defsystem
1625grammar.
1626
1627@c FIXME: Should have cross-reference to "Version specifiers" in the
1628@c defsystem grammar, but the cross-referencing is so broken by
1629@c insufficient node breakdown that I have not put one in.
1630
1631
1632@subsubsection pathname
1633
1634This attribute is optional and if absent (which is the usual case),
1635the component name will be used.
1636
1637@xref{The defsystem grammar,,Pathname specifiers},
1638for an explanation of how this attribute is interpreted.
1639
1640Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1641does additional processing to set the filesystem location of
1642the top component in that system.
1643This is detailed elsewhere. @xref{Defining systems with defsystem}.
1644
1645
1646@subsubsection properties
1647
1648This attribute is optional.
1649
1650Packaging systems often require information about files or systems
1651in addition to that specified by ASDF's pre-defined component attributes.
1652Programs that create vendor packages out of ASDF systems therefore
1653have to create ``placeholder'' information to satisfy these systems.
1654Sometimes the creator of an ASDF system may know the additional
1655information and wish to provide it directly.
1656
1657@code{(component-property component property-name)} and
1658associated @code{setf} method will allow
1659the programmatic update of this information.
1660Property names are compared as if by @code{EQL},
1661so use symbols or keywords or something.
1662
1663@menu
1664* Pre-defined subclasses of component::
1665* Creating new component types::
1666@end menu
1667
1668@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1669@comment  node-name,  next,  previous,  up
1670@subsection Pre-defined subclasses of component
1671
1672@deffn Component source-file
1673
1674A source file is any file that the system does not know how to
1675generate from other components of the system.
1676
1677Note that this is not necessarily the same thing as
1678``a file containing data that is typically fed to a compiler''.
1679If a file is generated by some pre-processor stage
1680(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1681then it is not, by this definition, a source file.
1682Conversely, we might have a graphic file
1683that cannot be automatically regenerated,
1684or a proprietary shared library that we received as a binary:
1685these do count as source files for our purposes.
1686
1687Subclasses of source-file exist for various languages.
1688@emph{FIXME: describe these.}
1689@end deffn
1690
1691@deffn Component module
1692
1693A module is a collection of sub-components.
1694
1695A module component has the following extra initargs:
1696
1697@itemize
1698@item
1699@code{:components} the components contained in this module
1700
1701@item
1702@code{:default-component-class}
1703All children components which don't specify their class explicitly
1704are inferred to be of this type.
1705
1706@item
1707@code{:if-component-dep-fails}
1708This attribute takes one of the values
1709@code{:fail}, @code{:try-next}, @code{:ignore},
1710its default value is @code{:fail}.
1711The other values can be used for implementing conditional compilation
1712based on implementation @code{*features*},
1713for the case where it is not necessary for all files in a module to be
1714compiled.
1715@emph{FIXME: such conditional compilation has been reported
1716to be broken in 2009.}
1717
1718@item
1719@code{:serial} When this attribute is set,
1720each subcomponent of this component is assumed to depend on all subcomponents
1721before it in the list given to @code{:components}, i.e.
1722all of them are loaded before a compile or load operation is performed on it.
1723
1724@end itemize
1725
1726The default operation knows how to traverse a module, so
1727most operations will not need to provide methods specialised on modules.
1728
1729@code{module} may be subclassed to represent components such as
1730foreign-language linked libraries or archive files.
1731@end deffn
1732
1733@deffn Component system
1734
1735@code{system} is a subclass of @code{module}.
1736
1737A system is a module with a few extra attributes for documentation
1738purposes; these are given elsewhere.
1739@xref{The defsystem grammar}.
1740
1741Users can create new classes for their systems:
1742the default @code{defsystem} macro takes a @code{:class} keyword argument.
1743@end deffn
1744
1745@node  Creating new component types,  , Pre-defined subclasses of component, Components
1746@comment  node-name,  next,  previous,  up
1747@subsection Creating new component types
1748
1749New component types are defined by subclassing one of the existing
1750component classes and specializing methods on the new component class.
1751
1752@emph{FIXME: this should perhaps be explained more throughly,
1753not only by example ...}
1754
1755As an example, suppose we have some implementation-dependent
1756functionality that we want to isolate
1757in one subdirectory per Lisp implementation our system supports.
1758We create a subclass of
1759@code{cl-source-file}:
1760
1761@lisp
1762(defclass unportable-cl-source-file (cl-source-file)
1763  ())
1764@end lisp
1765
1766Function @code{asdf:implementation-type} (exported since 2.014.14)
1767gives us the name of the subdirectory.
1768All that's left is to define how to calculate the pathname
1769of an @code{unportable-cl-source-file}.
1770
1771@lisp
1772(defmethod component-pathname ((component unportable-cl-source-file))
1773  (merge-pathnames*
1774   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1775   (call-next-method)))
1776@end lisp
1777
1778The new component type is used in a @code{defsystem} form in this way:
1779
1780@lisp
1781(defsystem :foo
1782    :components
1783    ((:file "packages")
1784     ...
1785     (:unportable-cl-source-file "threads"
1786      :depends-on ("packages" ...))
1787     ...
1788    )
1789@end lisp
1790
1791@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1792@comment  node-name,  next,  previous,  up
1793@chapter Controlling where ASDF searches for systems
1794
1795@section Configurations
1796
1797Configurations specify paths where to find system files.
1798
1799@enumerate
1800
1801@item
1802The search registry may use some hardcoded wrapping registry specification.
1803This allows some implementations (notably SBCL) to specify where to find
1804some special implementation-provided systems that
1805need to precisely match the version of the implementation itself.
1806
1807@item
1808An application may explicitly initialize the source-registry configuration
1809using the configuration API
1810(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1811in which case this takes precedence.
1812It may itself compute this configuration from the command-line,
1813from a script, from its own configuration file, etc.
1814
1815@item
1816The source registry will be configured from
1817the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1818
1819@item
1820The source registry will be configured from
1821user configuration file
1822@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1823(which defaults to
1824@file{~/.config/common-lisp/source-registry.conf})
1825if it exists.
1826
1827@item
1828The source registry will be configured from
1829user configuration directory
1830@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1831(which defaults to
1832@file{~/.config/common-lisp/source-registry.conf.d/})
1833if it exists.
1834
1835@item
1836The source registry will be configured from
1837system configuration file
1838@file{/etc/common-lisp/source-registry.conf}
1839if it exists/
1840
1841@item
1842The source registry will be configured from
1843system configuration directory
1844@file{/etc/common-lisp/source-registry.conf.d/}
1845if it exists.
1846
1847@item
1848The source registry will be configured from a default configuration.
1849This configuration may allow for implementation-specific systems
1850to be found, for systems to be found the current directory
1851(at the time that the configuration is initialized) as well as
1852@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1853@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1854For instance, SBCL will include directories for its contribs
1855when it can find them; it will look for them where SBCL was installed,
1856or at the location specified by the @code{SBCL_HOME} environment variable.
1857
1858@end enumerate
1859
1860Each of these configurations is specified as an s-expression
1861in a trivial domain-specific language (defined below).
1862Additionally, a more shell-friendly syntax is available
1863for the environment variable (defined yet below).
1864
1865Each of these configurations is only used if the previous
1866configuration explicitly or implicitly specifies that it
1867includes its inherited configuration.
1868
1869Additionally, some implementation-specific directories
1870may be automatically prepended to whatever directories are specified
1871in configuration files, no matter if the last one inherits or not.
1872
1873@section XDG base directory
1874
1875Note that we purport to respect the XDG base directory specification
1876as to where configuration files are located,
1877where data files are located,
1878where output file caches are located.
1879Mentions of XDG variables refer to that document.
1880
1881@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1882
1883This specification allows the user to specify some environment variables
1884to customize how applications behave to his preferences.
1885
1886On Windows platforms, when not using Cygwin,
1887instead of the XDG base directory specification,
1888we try to use folder configuration from the registry regarding
1889@code{Common AppData} and similar directories.
1890However, support for querying the Windows registry is limited as of ASDF 2,
1891and on many implementations, we may fall back to always using the defaults
1892without consulting the registry.
1893Patches welcome.
1894
1895@section Backward Compatibility
1896
1897For backward compatibility as well as to provide a practical backdoor for hackers,
1898ASDF will first search for @code{.asd} files in the directories specified in
1899@code{asdf:*central-registry*}
1900before it searches in the source registry above.
1901
1902@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
1903
1904By default, @code{asdf:*central-registry*} will be empty.
1905
1906This old mechanism will therefore not affect you if you don't use it,
1907but will take precedence over the new mechanism if you do use it.
1908
1909@section Configuration DSL
1910
1911Here is the grammar of the s-expression (SEXP) DSL for source-registry
1912configuration:
1913
1914@c FIXME: This is too wide for happy compilation into pdf.
1915
1916@example
1917;; A configuration is a single SEXP starting with keyword :source-registry
1918;; followed by a list of directives.
1919CONFIGURATION := (:source-registry DIRECTIVE ...)
1920
1921;; A directive is one of the following:
1922DIRECTIVE :=
1923    ;; INHERITANCE DIRECTIVE:
1924    ;; Your configuration expression MUST contain
1925    ;; exactly one of either of these:
1926    :inherit-configuration | ; splices inherited configuration (often specified last)
1927    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1928
1929    ;; forward compatibility directive (since ASDF 2.011.4), useful when
1930    ;; you want to use new configuration features but have to bootstrap a
1931    ;; the newer required ASDF from an older release that doesn't sport said features:
1932    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
1933
1934    ;; add a single directory to be scanned (no recursion)
1935    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1936
1937    ;; add a directory hierarchy, recursing but excluding specified patterns
1938    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1939
1940    ;; override the defaults for exclusion patterns
1941    (:exclude EXCLUSION-PATTERN ...) |
1942    ;; augment the defaults for exclusion patterns
1943    (:also-exclude EXCLUSION-PATTERN ...) |
1944    ;; Note that the scope of a an exclude pattern specification is
1945    ;; the rest of the current configuration expression or file.
1946
1947    ;; splice the parsed contents of another config file
1948    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1949
1950    ;; This directive specifies that some default must be spliced.
1951    :default-registry
1952
1953REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
1954DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
1955
1956PATHNAME-DESIGNATOR :=
1957    NIL | ;; Special: skip this entry.
1958    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
1959
1960EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
1961  against the name of a any subdirectory in the directory component
1962        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
1963@end example
1964
1965Pathnames are designated using another DSL,
1966shared with the output-translations configuration DSL below.
1967The DSL is resolved by the function @code{asdf::resolve-location},
1968to be documented and exported at some point in the future.
1969
1970@example
1971ABSOLUTE-COMPONENT-DESIGNATOR :=
1972    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
1973    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
1974             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
1975             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
1976             ;; Note that none of the above applies to strings used in *central-registry*,
1977             ;; which doesn't use this DSL: they are processed as normal namestrings.
1978             ;; however, you can compute what you put in the *central-registry*
1979             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
1980    PATHNAME | ;; pathname (better be an absolute path, or bust)
1981               ;; In output-translations, unless followed by relative components,
1982               ;; it better have appropriate wildcards, as in **/*.*.*
1983    :HOME | ;; designates the user-homedir-pathname ~/
1984    :USER-CACHE | ;; designates the default location for the user cache
1985    :HERE | ;; designates the location of the configuration file
1986            ;; (or *default-pathname-defaults*, if invoked interactively)
1987    :ROOT ;; magic, for output-translations source only: paths that are relative
1988          ;; to the root of the source host and device
1989    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
1990
1991RELATIVE-COMPONENT-DESIGNATOR :=
1992    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
1993    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
1994             ;; In output translations, if last component, **/*.*.* is added
1995    PATHNAME | ;; pathname; unless last component, directory is assumed.
1996    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.49-linux-x64
1997    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
1998    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
1999    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2000    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2001    :*.*.* | ;; any file (since ASDF 2.011.4)
2002    ;; Not supported (anymore): :UID and :USERNAME
2003@end example
2004
2005For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2006which is the default place ASDF looks for this configuration, once contained:
2007@example
2008(:source-registry
2009  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2010  :inherit-configuration)
2011@end example
2012
2013@section Configuration Directories
2014
2015Configuration directories consist in files each containing
2016a list of directives without any enclosing @code{(:source-registry ...)} form.
2017The files will be sorted by namestring as if by @code{string<} and
2018the lists of directives of these files with be concatenated in order.
2019An implicit @code{:inherit-configuration} will be included
2020at the @emph{end} of the list.
2021
2022This allows for packaging software that has file granularity
2023(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2024to easily include configuration information about distributed software.
2025
2026The convention is that, for sorting purposes,
2027the names of files in such a directory begin with two digits
2028that determine the order in which these entries will be read.
2029Also, the type of these files is conventionally @code{"conf"}
2030and as a limitation to some implementations (e.g. GNU clisp),
2031the type cannot be @code{NIL}.
2032
2033Directories may be included by specifying a directory pathname
2034or namestring in an @code{:include} directive, e.g.:
2035
2036@example
2037  (:include "/foo/bar/")
2038@end example
2039
2040Hence, to achieve the same effect as
2041my example @file{~/.config/common-lisp/source-registry.conf} above,
2042I could simply create a file
2043@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2044alone in its directory with the following contents:
2045@example
2046(:tree "/home/fare/cl/")
2047@end example
2048
2049@subsection The :here directive
2050
2051The @code{:here} directive is an absolute pathname designator that
2052refers to the directory containing the configuration file currently
2053being processed.
2054
2055The @code{:here} directive is intended to simplify the delivery of
2056complex CL systems, and for easy configuration of projects shared through
2057revision control systems, in accordance with our design principle that
2058each participant should be able to provide all and only the information
2059available to him or her.
2060
2061Consider a person X who has set up the source code repository for a
2062complex project with a master directory @file{dir/}.  Ordinarily, one
2063might simply have the user add a directive that would look something
2064like this:
2065@example
2066   (:tree  "path/to/dir")
2067@end example
2068But what if X knows that there are very large subtrees
2069under dir that are filled with, e.g., Java source code, image files for
2070icons, etc.?  All of the asdf system definitions are contained in the
2071subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2072these are the only directories that should be searched.
2073
2074In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2075contains the following:
2076@example
2077(:source-registry
2078   (:tree (:here "src/lisp/"))
2079   (:tree (:here "extlib/lisp"))
2080   (:directory (:here "outlier/")))
2081@end example
2082
2083Then when someone else (call her Y) checks out a copy of this
2084repository, she need only add
2085@example
2086(:include "/path/to/my/checkout/directory/asdf.conf")
2087@end example
2088to one of her previously-existing asdf source location configuration
2089files, or invoke @code{initialize-source-registry} with a configuration
2090form containing that s-expression.  ASDF will find the .conf file that X
2091has provided, and then set up source locations within the working
2092directory according to X's (relative) instructions.
2093
2094@section Shell-friendly syntax for configuration
2095
2096When considering environment variable @code{CL_SOURCE_REGISTRY}
2097ASDF will skip to next configuration if it's an empty string.
2098It will @code{READ} the string as a SEXP in the DSL
2099if it begins with a paren @code{(}
2100and it will be interpreted much like @code{TEXINPUTS}
2101list of paths, where
2102
2103  * paths are separated
2104   by a @code{:} (colon) on Unix platforms (including cygwin),
2105   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2106
2107  * each entry is a directory to add to the search path.
2108
2109  * if the entry ends with a double slash @code{//}
2110    then it instead indicates a tree in the subdirectories
2111    of which to recurse.
2112
2113  * if the entry is the empty string (which may only appear once),
2114    then it indicates that the inherited configuration should be
2115    spliced there.
2116
2117
2118@section Search Algorithm
2119@vindex *default-source-registry-exclusions*
2120
2121In case that isn't clear, the semantics of the configuration is that
2122when searching for a system of a given name,
2123directives are processed in order.
2124
2125When looking in a directory, if the system is found, the search succeeds,
2126otherwise it continues.
2127
2128When looking in a tree, if one system is found, the search succeeds.
2129If multiple systems are found, the consequences are unspecified:
2130the search may succeed with any of the found systems,
2131or an error may be raised.
2132ASDF currently returns the first system found,
2133XCVB currently raised an error.
2134If none is found, the search continues.
2135
2136Exclude statements specify patterns of subdirectories
2137the systems from which to ignore.
2138Typically you don't want to use copies of files kept by such
2139version control systems as Darcs.
2140Exclude statements are not propagated to further included or inherited
2141configuration files or expressions;
2142instead the defaults are reset around every configuration statement
2143to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2144
2145Include statements cause the search to recurse with the path specifications
2146from the file specified.
2147
2148An inherit-configuration statement cause the search to recurse with the path
2149specifications from the next configuration
2150(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2151
2152
2153@section Caching Results
2154
2155The implementation is allowed to either eagerly compute the information
2156from the configurations and file system, or to lazily re-compute it
2157every time, or to cache any part of it as it goes.
2158To explicitly flush any information cached by the system, use the API below.
2159
2160
2161@section Configuration API
2162
2163The specified functions are exported from your build system's package.
2164Thus for ASDF the corresponding functions are in package ASDF,
2165and for XCVB the corresponding functions are in package XCVB.
2166
2167@defun initialize-source-registry @&optional PARAMETER
2168   will read the configuration and initialize all internal variables.
2169   You may extend or override configuration
2170   from the environment and configuration files
2171   with the given @var{PARAMETER}, which can be
2172   @code{NIL} (no configuration override),
2173   or a SEXP (in the SEXP DSL),
2174   a string (as in the string DSL),
2175   a pathname (of a file or directory with configuration),
2176   or a symbol (fbound to function that when called returns one of the above).
2177@end defun
2178
2179@defun clear-source-registry
2180   undoes any source registry configuration
2181   and clears any cache for the search algorithm.
2182   You might want to call this function
2183   (or better, @code{clear-configuration})
2184   before you dump an image that would be resumed
2185   with a different configuration,
2186   and return an empty configuration.
2187   Note that this does not include clearing information about
2188   systems defined in the current image, only about
2189   where to look for systems not yet defined.
2190@end defun
2191
2192@defun ensure-source-registry @&optional PARAMETER
2193   checks whether a source registry has been initialized.
2194   If not, initialize it with the given @var{PARAMETER}.
2195@end defun
2196
2197Every time you use ASDF's @code{find-system}, or
2198anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2199@code{ensure-source-registry} is called with parameter NIL,
2200which the first time around causes your configuration to be read.
2201If you change a configuration file,
2202you need to explicitly @code{initialize-source-registry} again,
2203or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2204which will cause the initialization to happen next time around.
2205
2206
2207@section Future
2208
2209If this mechanism is successful, in the future, we may declare
2210@code{asdf:*central-registry*} obsolete and eventually remove it.
2211Any hook into implementation-specific search mechanisms will by then
2212have been integrated in the @code{:default-configuration} which everyone
2213should either explicitly use or implicit inherit. Some shell syntax
2214for it should probably be added somehow.
2215
2216But we're not there yet. For now, let's see how practical this new
2217source-registry is.
2218
2219
2220@section Rejected ideas
2221
2222Alternatives I considered and rejected included:
2223
2224@enumerate
2225@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2226   and somehow the configuration parser expands the new configuration
2227   language into a expanded series of directories of subdirectories to
2228   lookup, pre-recursing through specified hierarchies. This is kludgy,
2229   and leaves little space of future cleanups and extensions.
2230
2231@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2232   in completely new ways, so that new kinds of entries may be implemented
2233   as a recursive search, etc. This seems somewhat backwards.
2234
2235@item Completely remove @code{asdf:*central-registry*}
2236   and break backwards compatibility.
2237   Hopefully this will happen in a few years after everyone migrate to
2238   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2239
2240@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2241   when you dereference it or setf it. Only the new variable with new
2242   semantics is handled by the new search procedure.
2243   Complex and still introduces subtle semantic issues.
2244@end enumerate
2245
2246
2247I've been suggested the below features, but have rejected them,
2248for the sake of keeping ASDF no more complex than strictly necessary.
2249
2250@itemize
2251@item
2252  More syntactic sugar: synonyms for the configuration directives, such as
2253  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2254  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2255
2256@item
2257   The possibility to register individual files instead of directories.
2258
2259@item
2260  Integrate Xach Beane's tilde expander into the parser,
2261  or something similar that is shell-friendly or shell-compatible.
2262  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2263  minimal by removing the need for evaluated entries that ASDF has?
2264  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2265  Hopefully, these are already superseded by the @code{:default-registry}
2266
2267@item
2268  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2269  down a filesystem tree in the environment variable.
2270  It isn't that Lisp friendly either.
2271@end itemize
2272
2273@section TODO
2274
2275@itemize
2276@item Add examples
2277@end itemize
2278
2279
2280@section Credits for the source-registry
2281
2282Thanks a lot to Stelian Ionescu for the initial idea.
2283
2284Thanks to Rommel Martinez for the initial implementation attempt.
2285
2286All bad design ideas and implementation bugs are to mine, not theirs.
2287But so are good design ideas and elegant implementation tricks.
2288
2289 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2290
2291
2292
2293@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2294@comment  node-name,  next,  previous,  up
2295@chapter Controlling where ASDF saves compiled files
2296@cindex asdf-output-translations
2297@vindex ASDF_OUTPUT_TRANSLATIONS
2298
2299Each Common Lisp implementation has its own format
2300for compiled files (fasls for short, short for ``fast loading'').
2301If you use multiple implementations
2302(or multiple versions of the same implementation),
2303you'll soon find your source directories
2304littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2305Worse yet, some implementations use the same file extension
2306while changing formats from version to version (or platform to platform)
2307which means that you'll have to recompile binaries
2308as you switch from one implementation to the next.
2309
2310ASDF 2 includes the @code{asdf-output-translations} facility
2311to mitigate the problem.
2312
2313@section Configurations
2314
2315Configurations specify mappings from input locations to output locations.
2316Once again we rely on the XDG base directory specification for configuration.
2317@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2318
2319@enumerate
2320
2321@item
2322Some hardcoded wrapping output translations configuration may be used.
2323This allows special output translations (or usually, invariant directories)
2324to be specified corresponding to the similar special entries in the source registry.
2325
2326@item
2327An application may explicitly initialize the output-translations
2328configuration using the Configuration API
2329in which case this takes precedence.
2330(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2331It may itself compute this configuration from the command-line,
2332from a script, from its own configuration file, etc.
2333
2334@item
2335The source registry will be configured from
2336the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2337
2338@item
2339The source registry will be configured from
2340user configuration file
2341@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2342(which defaults to
2343@file{~/.config/common-lisp/asdf-output-translations.conf})
2344if it exists.
2345
2346@item
2347The source registry will be configured from
2348user configuration directory
2349@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2350(which defaults to
2351@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2352if it exists.
2353
2354@item
2355The source registry will be configured from
2356system configuration file
2357@file{/etc/common-lisp/asdf-output-translations.conf}
2358if it exists.
2359
2360@item
2361The source registry will be configured from
2362system configuration directory
2363@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2364if it exists.
2365
2366@end enumerate
2367
2368Each of these configurations is specified as a SEXP
2369in a trival domain-specific language (defined below).
2370Additionally, a more shell-friendly syntax is available
2371for the environment variable (defined yet below).
2372
2373Each of these configurations is only used if the previous
2374configuration explicitly or implicitly specifies that it
2375includes its inherited configuration.
2376
2377Note that by default, a per-user cache is used for output files.
2378This allows the seamless use of shared installations of software
2379between several users, and takes files out of the way of the developers
2380when they browse source code,
2381at the expense of taking a small toll when developers have to clean up
2382output files and find they need to get familiar with output-translations first.
2383
2384
2385@section Backward Compatibility
2386@cindex ASDF-BINARY-LOCATIONS compatibility
2387
2388
2389We purposefully do NOT provide backward compatibility with earlier versions of
2390@code{ASDF-Binary-Locations} (8 Sept 2009),
2391@code{common-lisp-controller} (7.0) or
2392@code{cl-launch} (2.35),
2393each of which had similar general capabilities.
2394The previous APIs of these programs were not designed
2395for configuration by the end-user
2396in an easy way with configuration files.
2397Recent versions of same packages use
2398the new @code{asdf-output-translations} API as defined below:
2399@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2400@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2401
2402This incompatibility shouldn't inconvenience many people.
2403Indeed, few people use and customize these packages;
2404these few people are experts who can trivially adapt to the new configuration.
2405Most people are not experts, could not properly configure these features
2406(except inasmuch as the default configuration of
2407@code{common-lisp-controller} and/or @code{cl-launch}
2408might have been doing the right thing for some users),
2409and yet will experience software that ``just works'',
2410as configured by the system distributor, or by default.
2411
2412Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2413we provide a limited emulation mode:
2414
2415@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2416This function will initialize the new @code{asdf-output-translations} facility in a way
2417that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2418Where you would previously set global variables
2419@var{*centralize-lisp-binaries*},
2420@var{*default-toplevel-directory*},
2421@var{*include-per-user-information*},
2422@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2423you will now have to pass the same values as keyword arguments to this function.
2424Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2425will accept any valid pathname designator for @code{asdf-output-translations}
2426instead of just strings and pathnames.
2427@end defun
2428
2429If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2430(the one available as an extension to load of top of ASDF,
2431not the one built into a few old versions of ASDF),
2432but first you must disable @code{asdf-output-translations}
2433with @code{(asdf:disable-output-translations)},
2434or you might experience ``interesting'' issues.
2435
2436Also, note that output translation is enabled by default.
2437To disable it, use @code{(asdf:disable-output-translations)}.
2438
2439
2440@section Configuration DSL
2441
2442Here is the grammar of the SEXP DSL
2443for @code{asdf-output-translations} configuration:
2444
2445@verbatim
2446;; A configuration is single SEXP starting with keyword :source-registry
2447;; followed by a list of directives.
2448CONFIGURATION := (:output-translations DIRECTIVE ...)
2449
2450;; A directive is one of the following:
2451DIRECTIVE :=
2452    ;; INHERITANCE DIRECTIVE:
2453    ;; Your configuration expression MUST contain
2454    ;; exactly one of either of these:
2455    :inherit-configuration | ; splices inherited configuration (often specified last)
2456    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2457
2458    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2459    ;; you want to use new configuration features but have to bootstrap a
2460    ;; the newer required ASDF from an older release that doesn't sport said features:
2461    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2462
2463    ;; include a configuration file or directory
2464    (:include PATHNAME-DESIGNATOR) |
2465
2466    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2467    :enable-user-cache |
2468    ;; Disable global cache. Map / to /
2469    :disable-cache |
2470
2471    ;; add a single directory to be scanned (no recursion)
2472    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2473
2474    ;; use a function to return the translation of a directory designator
2475    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2476
2477DIRECTORY-DESIGNATOR :=
2478    NIL | ;; As source: skip this entry. As destination: same as source
2479    T | ;; as source matches anything, as destination leaves pathname unmapped.
2480    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2481
2482TRANSLATION-FUNCTION :=
2483    SYMBOL | ;; symbol of a function that takes two arguments,
2484             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2485    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2486             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2487
2488@end verbatim
2489
2490Relative components better be either relative
2491or subdirectories of the path before them, or bust.
2492
2493The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2494You can specify more fine-grained patterns
2495by using a pathname object as the last component
2496e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2497
2498You may use @code{#+features} to customize the configuration file.
2499
2500The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2501to anything but themselves (same as if the second designator was the same as the first).
2502
2503When the first designator is @code{t},
2504the mapping always matches.
2505When the first designator starts with @code{:root},
2506the mapping matches any host and device.
2507In either of these cases, if the second designator
2508isn't @code{t} and doesn't start with @code{:root},
2509then strings indicating the host and pathname are somehow copied
2510in the beginning of the directory component of the source pathname
2511before it is translated.
2512
2513When the second designator is @code{t}, the mapping is the identity.
2514When the second designator starts with @code{:root},
2515the mapping preserves the host and device of the original pathname.
2516Notably, this allows you to map files
2517to a subdirectory of the whichever directory the file is in.
2518Though the syntax is not quite as easy to use as we'd like,
2519you can have an (source destination) mapping entry such as follows
2520in your configuration file,
2521or you may use @code{enable-asdf-binary-locations-compatibility}
2522with @code{:centralize-lisp-binaries nil}
2523which will do the same thing internally for you:
2524@verbatim
2525  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2526          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2527     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2528       (:root ,wild-subdir :implementation ,wild-file)))
2529@end verbatim
2530Starting with ASDF 2.011.4, you can use the simpler:
2531  @code{`(:root (:root :**/ :implementation :*.*.*))}
2532
2533
2534
2535@code{:include} statements cause the search to recurse with the path specifications
2536from the file specified.
2537
2538If the @code{translate-pathname} mechanism cannot achieve a desired
2539translation, the user may provide a function which provides the
2540required algorithim.  Such a translation function is specified by
2541supplying a list as the second @code{directory-designator}
2542the first element of which is the keyword @code{:function},
2543and the second element of which is
2544either a symbol which designates a function or a lambda expression.
2545The function designated by the second argument must take two arguments,
2546the first being the pathname of the source file,
2547the second being the wildcard that was matched.
2548The result of the function invocation should be the translated pathname.
2549
2550An @code{:inherit-configuration} statement cause the search to recurse with the path
2551specifications from the next configuration.
2552@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2553
2554@itemize
2555@item
2556@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2557@item
2558@code{:disable-cache} is the same as @code{(t t)}.
2559@item
2560@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2561which by default is the same as using
2562@code{(:home ".cache" "common-lisp" :implementation)}.
2563@item
2564@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2565which by default is the same as using
2566@code{("/var/cache/common-lisp" :uid :implementation-type)}
2567(on Unix and cygwin), or something semi-sensible on Windows.
2568@end itemize
2569
2570
2571@section Configuration Directories
2572
2573Configuration directories consist in files each contains
2574a list of directives without any enclosing
2575@code{(:output-translations ...)} form.
2576The files will be sorted by namestring as if by @code{string<} and
2577the lists of directives of these files with be concatenated in order.
2578An implicit @code{:inherit-configuration} will be included
2579at the @emph{end} of the list.
2580
2581This allows for packaging software that has file granularity
2582(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2583to easily include configuration information about software being distributed.
2584
2585The convention is that, for sorting purposes,
2586the names of files in such a directory begin with two digits
2587that determine the order in which these entries will be read.
2588Also, the type of these files is conventionally @code{"conf"}
2589and as a limitation of some implementations, the type cannot be @code{NIL}.
2590
2591Directories may be included by specifying a directory pathname
2592or namestring in an @code{:include} directive, e.g.:
2593@verbatim
2594  (:include "/foo/bar/")
2595@end verbatim
2596
2597@section Shell-friendly syntax for configuration
2598
2599When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2600ASDF will skip to next configuration if it's an empty string.
2601It will @code{READ} the string as an SEXP in the DSL
2602if it begins with a paren @code{(}
2603and it will be interpreted as a list of directories.
2604Directories should come by pairs, indicating a mapping directive.
2605Entries are separated
2606by a @code{:} (colon) on Unix platforms (including cygwin),
2607by a @code{;} (semicolon) on other platforms (mainly, Windows).
2608
2609The magic empty entry,
2610if it comes in what would otherwise be the first entry in a pair,
2611indicates the splicing of inherited configuration.
2612If it comes as the second entry in a pair,
2613it indicates that the directory specified first is to be left untranslated
2614(which has the same effect as if the directory had been repeated).
2615
2616
2617@section Semantics of Output Translations
2618
2619From the specified configuration,
2620a list of mappings is extracted in a straightforward way:
2621mappings are collected in order, recursing through
2622included or inherited configuration as specified.
2623To this list is prepended some implementation-specific mappings,
2624and is appended a global default.
2625
2626The list is then compiled to a mapping table as follows:
2627for each entry, in order, resolve the first designated directory
2628into an actual directory pathname for source locations.
2629If no mapping was specified yet for that location,
2630resolve the second designated directory to an output location directory
2631add a mapping to the table mapping the source location to the output location,
2632and add another mapping from the output location to itself
2633(unless a mapping already exists for the output location).
2634
2635Based on the table, a mapping function is defined,
2636mapping source pathnames to output pathnames:
2637given a source pathname, locate the longest matching prefix
2638in the source column of the mapping table.
2639Replace that prefix by the corresponding output column
2640in the same row of the table, and return the result.
2641If no match is found, return the source pathname.
2642(A global default mapping the filesystem root to itself
2643may ensure that there will always be a match,
2644with same fall-through semantics).
2645
2646@section Caching Results
2647
2648The implementation is allowed to either eagerly compute the information
2649from the configurations and file system, or to lazily re-compute it
2650every time, or to cache any part of it as it goes.
2651To explicitly flush any information cached by the system, use the API below.
2652
2653
2654@section Output location API
2655
2656The specified functions are exported from package ASDF.
2657
2658@defun initialize-output-translations @&optional PARAMETER
2659   will read the configuration and initialize all internal variables.
2660   You may extend or override configuration
2661   from the environment and configuration files
2662   with the given @var{PARAMETER}, which can be
2663   @code{NIL} (no configuration override),
2664   or a SEXP (in the SEXP DSL),
2665   a string (as in the string DSL),
2666   a pathname (of a file or directory with configuration),
2667   or a symbol (fbound to function that when called returns one of the above).
2668@end defun
2669
2670@defun disable-output-translations
2671   will initialize output translations in a way
2672   that maps every pathname to itself,
2673   effectively disabling the output translation facility.
2674@end defun
2675
2676@defun clear-output-translations
2677   undoes any output translation configuration
2678   and clears any cache for the mapping algorithm.
2679   You might want to call this function
2680   (or better, @code{clear-configuration})
2681   before you dump an image that would be resumed
2682   with a different configuration,
2683   and return an empty configuration.
2684   Note that this does not include clearing information about
2685   systems defined in the current image, only about
2686   where to look for systems not yet defined.
2687@end defun
2688
2689@defun ensure-output-translations @&optional PARAMETER
2690   checks whether output translations have been initialized.
2691   If not, initialize them with the given @var{PARAMETER}.
2692   This function will be called before any attempt to operate on a system.
2693@end defun
2694
2695@defun apply-output-translations PATHNAME
2696   Applies the configured output location translations to @var{PATHNAME}
2697   (calls @code{ensure-output-translations} for the translations).
2698@end defun
2699
2700Every time you use ASDF's @code{output-files}, or
2701anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2702@code{ensure-output-translations} is called with parameter NIL,
2703which the first time around causes your configuration to be read.
2704If you change a configuration file,
2705you need to explicitly @code{initialize-output-translations} again,
2706or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2707which will cause the initialization to happen next time around.
2708
2709
2710@section Credits for output translations
2711
2712Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2713and to Peter van Eynde for @code{Common Lisp Controller}.
2714
2715All bad design ideas and implementation bugs are to mine, not theirs.
2716But so are good design ideas and elegant implementation tricks.
2717
2718 --- Francois-Rene Rideau @email{fare@@tunes.org}
2719
2720@c @section Default locations
2721@c @findex output-files-for-system-and-operation
2722
2723@c The default binary location for each Lisp implementation
2724@c is a subdirectory of each source directory.
2725@c To account for different Lisps, Operating Systems, Implementation versions,
2726@c and so on, ASDF borrows code from SLIME
2727@c to create reasonable custom directory names.
2728@c Here are some examples:
2729
2730@c @itemize
2731@c @item
2732@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2733
2734@c @item
2735@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2736
2737@c @item
2738@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2739@c @end itemize
2740
2741@c By default, all output file pathnames will be relocated
2742@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2743
2744@c See the document @file{README.asdf-output-translations}
2745@c for a full specification on how to configure @code{asdf-output-translations}.
2746
2747@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2748@comment  node-name,  next,  previous,  up
2749@chapter Error handling
2750@findex SYSTEM-DEFINITION-ERROR
2751@findex OPERATION-ERROR
2752
2753@section ASDF errors
2754
2755If ASDF detects an incorrect system definition, it will signal a generalised instance of
2756@code{SYSTEM-DEFINITION-ERROR}.
2757
2758Operations may go wrong (for example when source files contain errors).
2759These are signalled using generalised instances of
2760@code{OPERATION-ERROR}.
2761
2762@section Compilation error and warning handling
2763@vindex *compile-file-warnings-behaviour*
2764@vindex *compile-file-errors-behavior*
2765
2766ASDF checks for warnings and errors when a file is compiled.
2767The variables @var{*compile-file-warnings-behaviour*} and
2768@var{*compile-file-errors-behavior*}
2769control the handling of any such events.
2770The valid values for these variables are
2771@code{:error}, @code{:warn}, and @code{:ignore}.
2772
2773@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2774@comment  node-name,  next,  previous,  up
2775@chapter Miscellaneous additional functionality
2776
2777@emph{FIXME:  Add discussion of @code{run-shell-command}?  Others?}
2778
2779ASDF includes several additional features that are generally
2780useful for system definition and development. These include:
2781
2782@defun coerce-pathname name @&key type defaults
2783
2784This function (available starting with ASDF 2.012.11)
2785takes an argument, and portably interprets it as a pathname.
2786If the argument @var{name} is a pathname or @code{nil}, it is passed through;
2787if it's a symbol, it's interpreted as a string by downcasing it;
2788if it's a string, it is first separated using @code{/} into substrings;
2789the leading substrings denote subdirectories of a relative pathname.
2790If @var{type} is @code{:directory} or the string ends with @code{/},
2791the last substring is also a subdirectory;
2792if @var{type} is a string, it is used as the type of the pathname, and
2793the last substring is the name component of the pathname;
2794if @var{type} is @code{nil}, the last substring specifies both name and type components
2795of the pathname, with the last @code{.} separating them, or only the name component
2796if there's no last @code{.} or if there is only one dot and it's the first character.
2797The host, device and version components come from @var{defaults}, which defaults to
2798@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
2799
2800@end defun
2801
2802@defun merge-pathnames* @&key specified defaults
2803
2804This function is a replacement for @code{merge-pathnames} that uses the host and device
2805from the @var{defaults} rather than the @var{specified} pathname when the latter
2806is a relative pathname. This allows ASDF and its users to create and use relative pathnames
2807without having to know beforehand what are the host and device
2808of the absolute pathnames they are relative to.
2809
2810@end defun
2811
2812@defun system-relative-pathname system name @&key type
2813
2814It's often handy to locate a file relative to some system.
2815The @code{system-relative-pathname} function meets this need.
2816
2817It takes two mandatory arguments @var{system} and @var{name}
2818and a keyword argument @var{type}:
2819@var{system} is name of a system, whereas @var{name} and optionally @var{type}
2820specify a relative pathname, interpreted like a component pathname specifier
2821by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
2822
2823It returns a pathname built from the location of the system's
2824source directory and the relative pathname. For example:
2825
2826@lisp
2827> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
2828#P"/repository/other/cl-ppcre/regex.data"
2829@end lisp
2830
2831@end defun
2832
2833@defun system-source-directory system-designator
2834
2835ASDF does not provide a turnkey solution for locating
2836data (or other miscellaneous) files
2837that are distributed together with the source code of a system.
2838Programmers can use @code{system-source-directory} to find such files.
2839Returns a pathname object.
2840The @var{system-designator} may be a string, symbol, or ASDF system object.
2841@end defun
2842
2843@defun clear-system system-designator
2844
2845It is sometimes useful to force recompilation of a previously loaded system.
2846In these cases, it may be useful to @code{(asdf:clear-system :foo)}
2847to remove the system from the table of currently loaded systems;
2848the next time the system @code{foo} or one that depends on it is re-loaded,
2849@code{foo} will then be loaded again.
2850Alternatively, you could touch @code{foo.asd} or
2851remove the corresponding fasls from the output file cache.
2852(It was once conceived that one should provide
2853a list of systems the recompilation of which to force
2854as the @code{:force} keyword argument to @code{load-system};
2855but this has never worked, and though the feature was fixed in ASDF 2.000,
2856it remains @code{cerror}'ed out as nobody ever used it.)
2857
2858Note that this does not and cannot by itself undo the previous loading
2859of the system. Common Lisp has no provision for such an operation,
2860and its reliance on irreversible side-effects to global datastructures
2861makes such a thing impossible in the general case.
2862If the software being re-loaded is not conceived with hot upgrade in mind,
2863this re-loading may cause many errors, warnings or subtle silent problems,
2864as packages, generic function signatures, structures, types, macros, constants, etc.
2865are being redefined incompatibly.
2866It is up to the user to make sure that reloading is possible and has the desired effect.
2867In some cases, extreme measures such as recursively deleting packages,
2868unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
2869and much more are necessary for reloading to happen smoothly.
2870ASDF itself goes through notable pains to make such a hot upgrade possible
2871with respect to its own code, and what it does is ridiculously complex;
2872look at the beginning of @file{asdf.lisp} to see what it does.
2873@end defun
2874
2875@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2876@comment  node-name,  next,  previous,  up
2877@chapter Getting the latest version
2878
2879Decide which version you want.
2880The @code{master} branch is where development happens;
2881its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
2882but an occasional regression may happen despite our (limited) test suite.
2883
2884The @code{release} branch is what cautious people should be using;
2885it has usually been tested more, and releases are cut at a point
2886where there isn't any known unresolved issue.
2887
2888You may get the ASDF source repository using git:
2889@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2890
2891You will find the above referenced tags in this repository.
2892You can also browse the repository on
2893@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2894
2895Discussion of ASDF development is conducted on the
2896mailing list
2897@kbd{asdf-devel@@common-lisp.net}.
2898@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
2899
2900
2901@node FAQ, TODO list, Getting the latest version, Top
2902@comment  node-name,  next,  previous,  up
2903@chapter FAQ
2904
2905@section  ``Where do I report a bug?''
2906
2907ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
2908
2909If you're unsure about whether something is a bug, or for general discussion,
2910use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
2911
2912
2913@section ``What has changed between ASDF 1 and ASDF 2?''
2914
2915@subsection What are ASDF 1 and ASDF 2?
2916
2917On May 31st 2010, we have released ASDF 2.
2918ASDF 2 refers to release 2.000 and later.
2919(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
2920ASDF 1 to any release earlier than 1.369 or so.
2921If your ASDF doesn't sport a version, it's an old ASDF 1.
2922
2923ASDF 2 and its release candidates push
2924@code{:asdf2} onto @code{*features*} so that if you are writing
2925ASDF-dependent code you may check for this feature
2926to see if the new API is present.
2927@emph{All} versions of ASDF should have the @code{:asdf} feature.
2928
2929If you are experiencing problems or limitations of any sort with ASDF 1,
2930we recommend that you should upgrade to ASDF 2,
2931or whatever is the latest release.
2932
2933
2934@subsection ASDF can portably name files in subdirectories
2935
2936Common Lisp namestrings are not portable,
2937except maybe for logical pathnamestrings,
2938that themselves have various limitations and require a lot of setup
2939that is itself ultimately non-portable.
2940
2941In ASDF 1, the only portable ways to refer to pathnames inside systems and components
2942were very awkward, using @code{#.(make-pathname ...)} and
2943@code{#.(merge-pathnames ...)}.
2944Even the above were themselves were inadequate in the general case
2945due to host and device issues, unless horribly complex patterns were used.
2946Plenty of simple cases that looked portable actually weren't,
2947leading to much confusion and greavance.
2948
2949ASDF 2 implements its own portable syntax for strings as pathname specifiers.
2950Naming files within a system definition becomes easy and portable again.
2951@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
2952@code{merge-pathnames*},
2953@code{coerce-pathname}.
2954
2955On the other hand, there are places where systems used to accept namestrings
2956where you must now use an explicit pathname object:
2957@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2958must now be written with the @code{#p} syntax:
2959@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2960
2961@xref{The defsystem grammar,,Pathname specifiers}.
2962
2963
2964@subsection Output translations
2965
2966A popular feature added to ASDF was output pathname translation:
2967@code{asdf-binary-locations}, @code{common-lisp-controller},
2968@code{cl-launch} and other hacks were all implementing it in ways
2969both mutually incompatible and difficult to configure.
2970
2971Output pathname translation is essential to share
2972source directories of portable systems across multiple implementations
2973or variants thereof,
2974or source directories of shared installations of systems across multiple users,
2975or combinations of the above.
2976
2977In ASDF 2, a standard mechanism is provided for that,
2978@code{asdf-output-translations},
2979with sensible defaults, adequate configuration languages,
2980a coherent set of configuration files and hooks,
2981and support for non-Unix platforms.
2982
2983@xref{Controlling where ASDF saves compiled files}.
2984
2985@subsection Source Registry Configuration
2986
2987Configuring ASDF used to require special magic
2988to be applied just at the right moment,
2989between the moment ASDF is loaded and the moment it is used,
2990in a way that is specific to the user,
2991the implementation he is using and the application he is building.
2992
2993This made for awkward configuration files and startup scripts
2994that could not be shared between users, managed by administrators
2995or packaged by distributions.
2996
2997ASDF 2 provides a well-documented way to configure ASDF,
2998with sensible defaults, adequate configuration languages,
2999and a coherent set of configuration files and hooks.
3000
3001We believe it's a vast improvement because it decouples
3002application distribution from library distribution.
3003The application writer can avoid thinking where the libraries are,
3004and the library distributor (dpkg, clbuild, advanced user, etc.)
3005can configure them once and for every application.
3006Yet settings can be easily overridden where needed,
3007so whoever needs control has exactly as much as required.
3008
3009At the same time, ASDF 2 remains compatible
3010with the old magic you may have in your build scripts
3011(using @code{*central-registry*} and
3012@code{*system-definition-search-functions*})
3013to tailor the ASDF configuration to your build automation needs,
3014and also allows for new magic, simpler and more powerful magic.
3015
3016@xref{Controlling where ASDF searches for systems}.
3017
3018
3019@subsection Usual operations are made easier to the user
3020
3021In ASDF 1, you had to use the awkward syntax
3022@code{(asdf:oos 'asdf:load-op :foo)}
3023to load a system,
3024and similarly for @code{compile-op}, @code{test-op}.
3025
3026In ASDF 2, you can use shortcuts for the usual operations:
3027@code{(asdf:load-system :foo)}, and
3028similarly for @code{compile-system}, @code{test-system}.
3029
3030
3031@subsection Many bugs have been fixed
3032
3033The following issues and many others have been fixed:
3034
3035@itemize
3036@item
3037The infamous TRAVERSE function has been revamped significantly,
3038with many bugs squashed.
3039In particular, dependencies were not correctly propagated
3040across submodules within a system but now are.
3041The :version and :feature features and
3042the :force (system1 .. systemN) feature have been fixed.
3043
3044@item
3045Performance has been notably improved for large systems
3046(say with thousands of components) by using
3047hash-tables instead of linear search,
3048and linear-time list accumulation
3049instead of quadratic-time recursive appends.
3050
3051@item
3052Many features used to not be portable,
3053especially where pathnames were involved.
3054Windows support was notably quirky because of such non-portability.
3055
3056@item
3057The internal test suite used to massively fail on many implementations.
3058While still incomplete, it now fully passes
3059on all implementations supported by the test suite,
3060except for GCL (due to GCL bugs).
3061
3062@item
3063Support was lacking for some implementations.
3064ABCL and GCL were notably wholly broken.
3065ECL extensions were not integrated in the ASDF release.
3066
3067@item
3068The documentation was grossly out of date.
3069
3070@end itemize
3071
3072
3073@subsection ASDF itself is versioned
3074
3075Between new features, old bugs fixed, and new bugs introduced,
3076there were various releases of ASDF in the wild,
3077and no simple way to check which release had which feature set.
3078People using or writing systems had to either make worst-case assumptions
3079as to what features were available and worked,
3080or take great pains to have the correct version of ASDF installed.
3081
3082With ASDF 2, we provide a new stable set of working features
3083that everyone can rely on from now on.
3084Use @code{#+asdf2} to detect presence of ASDF 2,
3085@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3086to check the availability of a version no earlier than required.
3087
3088
3089@subsection ASDF can be upgraded
3090
3091When an old version of ASDF was loaded,
3092it was very hard to upgrade ASDF in your current image
3093without breaking everything.
3094Instead you had to exit the Lisp process and
3095somehow arrange to start a new one from a simpler image.
3096Something that can't be done from within Lisp,
3097making automation of it difficult,
3098which compounded with difficulty in configuration,
3099made the task quite hard.
3100Yet as we saw before, the task would have been required
3101to not have to live with the worst case or non-portable
3102subset of ASDF features.
3103
3104With ASDF 2, it is easy to upgrade
3105from ASDF 2 to later versions from within Lisp,
3106and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3107We support hot upgrade of ASDF and any breakage is a bug
3108that we will do our best to fix.
3109There are still limitations on upgrade, though,
3110most notably the fact that after you upgrade ASDF,
3111you must also reload or upgrade all ASDF extensions.
3112
3113@subsection Decoupled release cycle
3114
3115When vendors were releasing their Lisp implementations with ASDF,
3116they had to basically never change version
3117because neither upgrade nor downgrade was possible
3118without breaking something for someone,
3119and no obvious upgrade path was visible and recommendable.
3120
3121With ASDF 2, upgrade is possible, easy and can be recommended.
3122This means that vendors can safely ship a recent version of ASDF,
3123confident that if a user isn't fully satisfied,
3124he can easily upgrade ASDF and deal
3125with a supported recent version of it.
3126This means that release cycles will be causally decoupled,
3127the practical consequence of which will mean faster convergence
3128towards the latest version for everyone.
3129
3130
3131@subsection Pitfalls of the transition to ASDF 2
3132
3133The main pitfalls in upgrading to ASDF 2 seem to be related
3134to the output translation mechanism.
3135
3136@itemize
3137
3138@item
3139Output translations is enabled by default. This may surprise some users,
3140most of them in pleasant way (we hope), a few of them in an unpleasant way.
3141It is trivial to disable output translations.
3142@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3143
3144@item
3145Some systems in the large have been known not to play well with output translations.
3146They were relatively easy to fix.
3147Once again, it is also easy to disable output translations,
3148or to override its configuration.
3149
3150@item
3151The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3152They replace A-B-L, and there is compatibility mode to emulate
3153your previous A-B-L configuration.
3154See @code{enable-asdf-binary-locations-compatibility} in
3155@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3156But thou shall not load ABL on top of ASDF 2.
3157
3158@end itemize
3159
3160Other issues include the following:
3161
3162@itemize
3163
3164@item
3165ASDF pathname designators are now specified in places where they were unspecified,
3166and a few small adjustments have to be made to some non-portable defsystems.
3167Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
3168a logical pathname (or implementation-dependent hierarchical pathname)
3169must now be specified with @code{#p} syntax
3170where the namestring might have previously sufficed;
3171moreover when evaluation is desired @code{#.} must be used,
3172where it wasn't necessary in the toplevel @code{:pathname} argument.
3173
3174@item
3175There is a slight performance bug, notably on SBCL,
3176when initially searching for @file{asd} files,
3177the implicit @code{(directory "/configured/path/**/*.asd")}
3178for every configured path @code{(:tree "/configured/path/")}
3179in your @code{source-registry} configuration can cause a slight pause.
3180Try to @code{(time (asdf:initialize-source-registry))}
3181to see how bad it is or isn't on your system.
3182If you insist on not having this pause,
3183you can avoid the pause by overriding the default source-registry configuration
3184and not use any deep @code{:tree} entry but only @code{:directory} entries
3185or shallow @code{:tree} entries.
3186Or you can fix your implementation to not be quite that slow
3187when recursing through directories.
3188@emph{Update}: performance bug fixed the hard way in 2.010.
3189
3190@item
3191On Windows, only LispWorks supports proper default configuration pathnames
3192based on the Windows registry.
3193Other implementations make do with environment variables,
3194that you may have to define yourself if you're using an older version of Windows.
3195Windows support is somewhat less tested than Unix support.
3196Please help report and fix bugs.
3197
3198@item
3199The mechanism by which one customizes a system so that Lisp files
3200may use a different extension from the default @file{.lisp} has changed.
3201Previously, the pathname for a component was lazily computed when operating on a system,
3202and you would
3203@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3204  (declare (ignorable component system)) "lis")}.
3205Now, the pathname for a component is eagerly computed when defining the system,
3206and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3207and use @code{:default-component-class cl-source-file.lis} as argument to @code{defsystem},
3208as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3209
3210@findex source-file-type
3211
3212
3213@end itemize
3214
3215
3216@section Issues with installing the proper version of ASDF
3217
3218@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3219
3220We recommend you upgrade ASDF.
3221@xref{Loading ASDF,,Upgrading ASDF}.
3222
3223If this does not work, it is a bug, and you should report it.
3224@xref{FAQ, report-bugs, Where do I report a bug}.
3225In the meantime, you can load @file{asdf.lisp} directly.
3226@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3227
3228
3229@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3230
3231Starting with current candidate releases of ASDF 2,
3232it should always be a good time to upgrade to a recent ASDF.
3233You may consult with the maintainer for which specific version they recommend,
3234but the latest @code{release} should be correct.
3235We trust you to thoroughly test it with your implementation before you release it.
3236If there are any issues with the current release,
3237it's a bug that you should report upstream and that we will fix ASAP.
3238
3239As to how to include ASDF, we recommend the following:
3240
3241@itemize
3242@item
3243If ASDF isn't loaded yet, then @code{(require "asdf")}
3244should load the version of ASDF that is bundled with your system.
3245You may have it load some other version configured by the user,
3246if you allow such configuration.
3247
3248@item
3249If your system provides a mechanism to hook into @code{CL:REQUIRE},
3250then it would be nice to add ASDF to this hook the same way that
3251ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3252
3253@item
3254You may, like SBCL, have ASDF be implicitly used to require systems
3255that are bundled with your Lisp distribution.
3256If you do have a few magic systems that come with your implementation
3257in a precompiled way such that one should only use the binary version
3258that goes with your distribution, like SBCL does,
3259then you should add them in the beginning of @code{wrapping-source-registry}.
3260
3261@item
3262If you have magic systems as above, like SBCL does,
3263then we explicitly ask you to @emph{NOT} distribute
3264@file{asdf.asd} as part of those magic systems.
3265You should still include the file @file{asdf.lisp} in your source distribution
3266and precompile it in your binary distribution,
3267but @file{asdf.asd} if included at all,
3268should be secluded from the magic systems,
3269in a separate file hierarchy;
3270alternatively, you may provide the system
3271after renaming it and its @file{.asd} file to e.g.
3272@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3273@code{sb-asdf} and @file{sb-asdf.asd}.
3274Indeed, if you made @file{asdf.asd} a magic system,
3275then users would no longer be able to upgrade ASDF using ASDF itself
3276to some version of their preference that
3277they maintain independently from your Lisp distribution.
3278
3279@item
3280If you do not have any such magic systems, or have other non-magic systems
3281that you want to bundle with your implementation,
3282then you may add them to the @code{default-source-registry},
3283and you are welcome to include @file{asdf.asd} amongst them.
3284
3285@item
3286Please send us upstream any patches you make to ASDF itself,
3287so we can merge them back in for the benefit of your users
3288when they upgrade to the upstream version.
3289
3290@end itemize
3291
3292
3293
3294@section Issues with configuring ASDF
3295
3296@subsection ``How can I customize where fasl files are stored?''
3297
3298@xref{Controlling where ASDF saves compiled files}.
3299
3300Note that in the past there was an add-on to ASDF called
3301@code{ASDF-binary-locations}, developed by Gary King.
3302That add-on has been merged into ASDF proper,
3303then superseded by the @code{asdf-output-translations} facility.
3304
3305Note that use of @code{asdf-output-translations}
3306can interfere with one aspect of your systems
3307--- if your system uses @code{*load-truename*} to find files
3308(e.g., if you have some data files stored with your program),
3309then the relocation that this ASDF customization performs
3310is likely to interfere.
3311Use @code{asdf:system-relative-pathname} to locate a file
3312in the source directory of some system, and
3313use @code{asdf:apply-output-translations} to locate a file
3314whose pathname has been translated by the facility.
3315
3316@subsection ``How can I wholly disable the compiler output cache?''
3317
3318To permanently disable the compiler output cache
3319for all future runs of ASDF, you can:
3320
3321@example
3322mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3323echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3324@end example
3325
3326This assumes that you didn't otherwise configure the ASDF files
3327(if you did, edit them again),
3328and don't somehow override the configuration at runtime
3329with a shell variable (see below) or some other runtime command
3330(e.g. some call to @code{asdf:initialize-output-translations}).
3331
3332To disable the compiler output cache in Lisp processes
3333run by your current shell, try (assuming @code{bash} or @code{zsh})
3334(on Unix and cygwin only):
3335
3336@example
3337export ASDF_OUTPUT_TRANSLATIONS=/:
3338@end example
3339
3340To disable the compiler output cache just in the current Lisp process,
3341use (after loading ASDF but before using it):
3342
3343@example
3344(asdf:disable-output-translations)
3345@end example
3346
3347@section Issues with using and extending ASDF to define systems
3348
3349@subsection ``How can I cater for unit-testing in my system?''
3350
3351ASDF provides a predefined test operation, @code{test-op}.
3352@xref{Predefined operations of ASDF, test-op}.
3353The test operation, however, is largely left to the system definer to specify.
3354@code{test-op} has been
3355a topic of considerable discussion on the
3356@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3357and on the
3358@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3359
3360Here are some guidelines:
3361
3362@itemize
3363@item
3364For a given system, @var{foo}, you will want to define a corresponding
3365test system, such as @var{foo-test}.  The reason that you will want this
3366separate system is that ASDF does not out of the box supply components
3367that are conditionally loaded.  So if you want to have source files
3368(with the test definitions) that will not be loaded except when testing,
3369they should be put elsewhere.
3370
3371@item
3372The @var{foo-test} system can be defined in an asd file of its own or
3373together with @var{foo}.  An aesthetic preference against cluttering up
3374the filesystem with extra asd files should be balanced against the
3375question of whether one might want to directly load @var{foo-test}.
3376Typically one would not want to do this except in early stages of
3377debugging.
3378
3379@item
3380Record that testing is implemented by @var{foo-test}.  For example:
3381@example
3382(defsystem @var{foo}
3383   :in-order-to ((test-op (test-op @var{foo-test})))
3384   ....)
3385
3386(defsystem @var{foo-test}
3387   :depends-on (@var{foo} @var{my-test-library} ...)
3388   ....)
3389@end example
3390@end itemize
3391
3392This procedure will allow you to support users who do not wish to
3393install your test framework.
3394
3395One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3396does not return a value.  So in current versions of ASDF there is no
3397reliable programmatic means of determining whether or not a set of tests
3398has passed, or which tests have failed.  The user must simply read the
3399console output.  This limitation has been the subject of much
3400discussion.
3401
3402@subsection ``How can I cater for documentation generation in my system?''
3403
3404The ASDF developers are currently working to add a @code{doc-op}
3405to the set of predefined ASDF operations.
3406@xref{Predefined operations of ASDF}.
3407See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3408
3409
3410
3411@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3412
3413See @code{cffi}'s @code{cffi-grovel}.
3414
3415@anchor{report-bugs}
3416
3417
3418@subsection ``I want to put my module's files at the top level.  How do I do this?''
3419
3420By default, the files contained in an asdf module go
3421in a subdirectory with the same name as the module.
3422However, this can be overridden by adding a @code{:pathname ""} argument
3423to the module description.
3424For example, here is how it could be done
3425in the spatial-trees ASDF system definition for ASDF 2:
3426
3427@example
3428(asdf:defsystem :spatial-trees
3429  :components
3430  ((:module base
3431            :pathname ""
3432            :components
3433            ((:file "package")
3434             (:file "basedefs" :depends-on ("package"))
3435             (:file "rectangles" :depends-on ("package"))))
3436   (:module tree-impls
3437            :depends-on (base)
3438            :pathname ""
3439            :components
3440            ((:file "r-trees")
3441             (:file "greene-trees" :depends-on ("r-trees"))
3442             (:file "rstar-trees" :depends-on ("r-trees"))
3443             (:file "rplus-trees" :depends-on ("r-trees"))
3444             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3445   (:module viz
3446            :depends-on (base)
3447            :pathname ""
3448            :components
3449            ((:static-file "spatial-tree-viz.lisp")))
3450   (:module tests
3451            :depends-on (base)
3452            :pathname ""
3453            :components
3454            ((:static-file "spatial-tree-test.lisp")))
3455   (:static-file "LICENCE")
3456   (:static-file "TODO")))
3457@end example
3458
3459All of the files in the @code{tree-impls} module are at the top level,
3460instead of in a @file{tree-impls/} subdirectory.
3461
3462Note that the argument to @code{:pathname} can be either a pathname object or a string.
3463A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3464but this is discouraged because the results of parsing a namestring are not portable.
3465A pathname can only be portably constructed with such syntax as
3466@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3467and similarly the current directory can only be portably specified as
3468@code{#.(make-pathname :directory '(:relative))}.
3469However, as of ASDF 2, you can portably use a string to denote a pathname.
3470The string will be parsed as a @code{/}-separated path from the current directory,
3471such that the empty string @code{""} denotes the current directory, and
3472@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3473portably denotes the same subdirectory as above.
3474When files are specified, the last @code{/}-separated component is interpreted
3475either as the name component of a pathname
3476(if the component class specifies a pathname type),
3477or as a name component plus optional dot-separated type component
3478(if the component class doesn't specifies a pathname type).
3479
3480@subsection How do I create a system definition where all the source files have a .cl extension?
3481
3482Starting with ASDF 2.014.14, you may just pass
3483the builtin class @code{cl-source-file.cl} as
3484the @code{:default-component-class} argument to @code{defsystem}:
3485
3486@lisp
3487(defsystem my-cl-system
3488  :default-component-class cl-source-file.cl
3489  ...)
3490@end lisp
3491
3492Another builtin class @code{cl-source-file.lsp} is offered
3493for files ending in @file{.lsp}.
3494
3495If you want to use a different extension
3496for which ASDF doesn't provide builtin support,
3497or want to support versions of ASDF
3498earlier than 2.014.14 (but later than 2.000),
3499you can define a class as follows:
3500
3501@lisp
3502;; Prologue: make sure we're using a sane package.
3503(defpackage :my-asdf-extension
3504   (:use :asdf :common-lisp)
3505   (:export #:cl-source-file.lis))
3506(in-package :my-asdf-extension)
3507
3508(defclass cl-source-file.lis (cl-source-file)
3509   ((type :initform "lis")))
3510@end lisp
3511
3512Then you can use it as follows:
3513@lisp
3514(defsystem my-cl-system
3515  :default-component-class my-asdf-extension:cl-source-file.lis
3516  ...)
3517@end lisp
3518
3519Of course, if you're in the same package, e.g. in the same file,
3520you won't need to use the package qualifier before @code{cl-source-file.lis}.
3521Actually, if all you're doing is defining this class
3522and using it in the same file without other fancy definitions,
3523you might skip package complications:
3524
3525@lisp
3526(in-package :asdf)
3527(defclass cl-source-file.lis (cl-source-file)
3528   ((type :initform "lis")))
3529(defsystem my-cl-system
3530  :default-component-class cl-source-file.lis
3531  ...)
3532@end lisp
3533
3534It is possible to achieve the same effect
3535in a way that supports both ASDF 1 and ASDF 2,
3536but really, friends don't let friends use ASDF 1.
3537Please upgrade to ASDF 2.
3538In short, though: do same as above, but
3539@emph{before} you use the class in a @code{defsystem},
3540you also define the following method:
3541
3542@lisp
3543(defmethod source-file-type ((f cl-source-file.lis) (m module))
3544  (declare (ignorable f m))
3545  "lis")
3546@end lisp
3547
3548
3549@node  TODO list, Inspiration, FAQ, Top
3550@comment  node-name,  next,  previous,  up
3551@chapter TODO list
3552
3553Here is an old list of things to do,
3554in addition to the bugs that are now tracked on launchpad:
3555@url{https://launchpad.net/asdf}.
3556
3557@section Outstanding spec questions, things to add
3558
3559** packaging systems
3560
3561*** manual page component?
3562
3563** style guide for .asd files
3564
3565You should either use keywords or be careful
3566with the package that you evaluate defsystem forms in.
3567Otherwise @code{(defsystem partition ...)}
3568being read in the @code{cl-user} package
3569will intern a @code{cl-user:partition} symbol,
3570which will then collide with the @code{partition:partition} symbol.
3571
3572Actually there's a hairier packages problem to think about too.
3573@code{in-order-to} is not a keyword:
3574if you read @code{defsystem} forms in a package that doesn't use ASDF,
3575odd things might happen.
3576
3577
3578** extending defsystem with new options
3579
3580You might not want to write a whole parser,
3581but just to add options to the existing syntax.
3582Reinstate @code{parse-option} or something akin.
3583
3584
3585** document all the error classes
3586
3587** what to do with compile-file failure
3588
3589Should check the primary return value from compile-file and see if
3590that gets us any closer to a sensible error handling strategy
3591
3592** foreign files
3593
3594lift unix-dso stuff from db-sockets
3595
3596** Diagnostics
3597
3598A ``dry run'' of an operation can be made with the following form:
3599
3600@lisp
3601(traverse (make-instance '<operation-name>)
3602          (find-system <system-name>)
3603          'explain)
3604@end lisp
3605
3606This uses unexported symbols.
3607What would be a nice interface for this functionality?
3608
3609@section Missing bits in implementation
3610
3611** reuse the same scratch package whenever a system is reloaded from disk
3612
3613** proclamations probably aren't
3614
3615** when a system is reloaded with fewer components than it previously had, odd things happen
3616
3617We should do something inventive when processing a @code{defsystem} form,
3618like take the list of kids and @code{setf} the slot to @code{nil},
3619then transfer children from old to new list as they're found.
3620
3621** (stuff that might happen later)
3622
3623*** Propagation of the @code{:force} option.
3624
3625``I notice that
3626
3627        @code{(asdf:compile-system :araneida :force t)}
3628
3629also forces compilation of every other system the @code{:araneida} system depends on.
3630This is rarely useful to me;
3631usually, when I want to force recompilation of something more than a single source file,
3632I want to recompile only one system.
3633So it would be more useful to have @code{make-sub-operation}
3634refuse to propagate @code{:force t} to other systems, and
3635propagate only something like @code{:force :recursively}.
3636
3637Ideally what we actually want is some kind of criterion that says
3638to which systems (and which operations) a @code{:force} switch will propagate.
3639
3640The problem is perhaps that ``force'' is a pretty meaningless concept.
3641How obvious is it that @code{load :force t} should force @emph{compilation}?
3642But we don't really have the right dependency setup
3643for the user to compile @code{:force t} and expect it to work
3644(files will not be loaded after compilation, so the compile
3645environment for subsequent files will be emptier than it needs to be)
3646
3647What does the user actually want to do when he forces?
3648Usually, for me, update for use with a new version of the Lisp compiler.
3649Perhaps for recovery when he suspects that something has gone wrong.
3650Or else when he's changed compilation options or configuration
3651in some way that's not reflected in the dependency graph.
3652
3653Other possible interface: have a ``revert'' function akin to @code{make clean}.
3654
3655@lisp
3656(asdf:revert 'asdf:compile-op 'araneida)
3657@end lisp
3658
3659would delete any files produced by @code{(compile-system :araneida)}.
3660Of course, it wouldn't be able to do much about stuff in the image itself.
3661
3662How would this work?
3663
3664@code{traverse}
3665
3666There's a difference between a module's dependencies (peers)
3667and its components (children).
3668Perhaps there's a similar difference in operations?
3669For example, @code{(load "use") depends-on (load "macros")} is a peer,
3670whereas @code{(load "use") depends-on (compile "use")}
3671is more of a ``subservient'' relationship.
3672
3673@node  Inspiration, Concept Index, TODO list, Top
3674@comment  node-name,  next,  previous,  up
3675@chapter Inspiration
3676
3677@section mk-defsystem (defsystem-3.x)
3678
3679We aim to solve basically the same problems as @code{mk-defsystem} does.
3680However, our architecture for extensibility
3681better exploits CL language features (and is documented),
3682and we intend to be portable rather than just widely-ported.
3683No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3684that implementation has the unenviable task
3685of supporting pre-ANSI implementations, which is no longer necessary.
3686
3687The surface defsystem syntax of asdf is more-or-less compatible with
3688@code{mk-defsystem}, except that we do not support
3689the @code{source-foo} and @code{binary-foo} prefixes
3690for separating source and binary files, and
3691we advise the removal of all options to specify pathnames.
3692
3693The @code{mk-defsystem} code for topologically sorting
3694a module's dependency list was very useful.
3695
3696@section defsystem-4 proposal
3697
3698Marco and Peter's proposal for defsystem 4 served as the driver for
3699many of the features in here.  Notable differences are:
3700
3701@itemize
3702@item
3703We don't specify output files or output file extensions
3704as part of the system.
3705
3706If you want to find out what files an operation would create,
3707ask the operation.
3708
3709@item
3710We don't deal with CL packages
3711
3712If you want to compile in a particular package, use an @code{in-package} form
3713in that file (ilisp / SLIME will like you more if you do this anyway)
3714
3715@item
3716There is no proposal here that @code{defsystem} does version control.
3717
3718A system has a given version which can be used to check dependencies,
3719but that's all.
3720@end itemize
3721
3722The defsystem 4 proposal tends to look more at the external features,
3723whereas this one centres on a protocol for system introspection.
3724
3725@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
3726
3727Available in updated-for-CL form on the web at
3728@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3729
3730In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3731and concept to deal with creating component trees
3732from @code{defsystem} surface syntax.
3733[ this is not true right now, though it used to be and
3734probably will be again soon ]
3735
3736
3737@c -------------------
3738
3739
3740@node Concept Index, Function and Class Index, Inspiration, Top
3741@unnumbered Concept Index
3742
3743@printindex cp
3744
3745@node Function and Class Index, Variable Index, Concept Index, Top
3746@unnumbered Function and Class Index
3747
3748@printindex fn
3749
3750@node Variable Index,  , Function and Class Index, Top
3751@unnumbered Variable Index
3752
3753@printindex vr
3754
3755@bye
Note: See TracBrowser for help on using the repository browser.