source: branches/1.0.x/abcl/doc/asdf/asdf.texinfo @ 13717

Last change on this file since 13717 was 13717, checked in by Mark Evenson, 10 years ago

Backport r13702: update to asdf-2.019 with ABCL patch.

File size: 142.8 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2011 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex compile-system
201@findex test-system
202@cindex system directory designator
203@findex operate
204@findex oos
205
206@c @menu
207@c * Installing ASDF::
208@c @end menu
209
210
211@section Loading a pre-installed ASDF
212
213Many Lisp implementations include a copy of ASDF.
214You can usually load this copy using Common Lisp's @code{require} function:
215
216@lisp
217(require "asdf")
218@end lisp
219
220As of the writing of this manual,
221the following implementations provide ASDF 2 this way:
222abcl allegro ccl clisp cmucl ecl sbcl xcl.
223The following implementations don't provide it yet but will in a future release:
224lispworks scl.
225The following implementations are obsolete and most probably will never bundle it:
226cormancl gcl genera mcl.
227
228If the implementation you are using doesn't provide ASDF 2,
229see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
230If that implementation is still actively maintained,
231you may also send a bug report to your Lisp vendor and complain
232about their failing to provide ASDF.
233
234NB: all implementations except clisp also accept
235@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
236For portability's sake, you probably want to use @code{(require "asdf")}.
237
238
239@section Checking whether ASDF is loaded
240
241To check whether ASDF is properly loaded in your current Lisp image,
242you can run this form:
243
244@lisp
245(asdf:asdf-version)
246@end lisp
247
248If it returns a string,
249that is the version of ASDF that is currently installed.
250
251If it raises an error,
252then either ASDF is not loaded, or
253you are using an old version of ASDF.
254
255You can check whether an old version is loaded
256by checking if the ASDF package is present.
257The form below will allow you to programmatically determine
258whether a recent version is loaded, an old version is loaded,
259or none at all:
260
261@lisp
262(or #+asdf2 (asdf:asdf-version) #+asdf :old)
263@end lisp
264
265If it returns a version number, that's the version of ASDF installed.
266If it returns the keyword @code{:OLD},
267then you're using an old version of ASDF (from before 1.635).
268If it returns @code{NIL} then ASDF is not installed.
269
270If you are experiencing problems with ASDF,
271please try upgrading to the latest released version,
272using the method below,
273before you contact us and raise an issue.
274
275
276@section Upgrading ASDF
277
278If your implementation does provide ASDF 2 or later,
279and you want to upgrade to a more recent version,
280just install ASDF like any other package
281(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
282configure ASDF as usual (see @pxref{Configuring ASDF} below),
283and upgrade with:
284
285@lisp
286(require "asdf")
287(asdf:load-system :asdf)
288@end lisp
289
290If on the other hand, your implementation only provides an old ASDF,
291you will require a special configuration step and an old-style loading:
292
293@lisp
294(require "asdf")
295(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
296(asdf:oos 'asdf:load-op :asdf)
297@end lisp
298
299Don't forget the trailing @code{/} at the end of your pathname.
300
301Also, note that older versions of ASDF won't redirect their output,
302or at least won't do it according to your usual ASDF 2 configuration.
303You therefore need write access on the directory
304where you install the new ASDF,
305and make sure you're not using it
306for multiple mutually incompatible implementations.
307At worst, you may have to have multiple copies of the new ASDF,
308e.g. one per implementation installation, to avoid clashes.
309Note that to our knowledge all implementations that provide ASDF
310provide ASDF 2 in their latest release, so
311you may want to upgrade your implementation rather than go through that hoop.
312
313Finally, note that there are some limitations to upgrading ASDF:
314@itemize
315@item
316Any ASDF extension becomes invalid, and will need to be reloaded.
317This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
318Starting with ASDF 2.014.8, ASDF will actually invalidate
319all previously loaded systems when it is loaded on top of
320a different ASDF version.
321@item
322Until all implementations provide ASDF 2.015 or later,
323it is safer if you upgrade ASDF and its extensions as a special step
324at the very beginning of whatever script you are running,
325before you start using ASDF to load anything else.
326@item
327Until all implementations provide ASDF 2.015 or later,
328it is unsafe to upgrade ASDF as part of loading a system
329that depends on a more recent version of ASDF,
330since the new one might shadow the old one while the old one is running,
331and the running old one will be confused
332when extensions are loaded into the new one.
333In the meantime, we recommend that your systems should @emph{not} specify
334@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
335but instead that they check that a recent enough ASDF is installed,
336with such code as:
337@example
338(unless (or #+asdf2 (asdf:version-satisfies
339                     (asdf:asdf-version) *required-asdf-version*))
340  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
341@end example
342@end itemize
343
344
345@section Loading an otherwise installed ASDF
346
347If your implementation doesn't include ASDF,
348if for some reason the upgrade somehow fails,
349does not or cannot apply to your case,
350you will have to install the file @file{asdf.lisp}
351somewhere and load it with:
352
353@lisp
354(load "/path/to/your/installed/asdf.lisp")
355@end lisp
356
357The single file @file{asdf.lisp} is all you normally need to use ASDF.
358
359You can extract this file from latest release tarball on the
360@url{http://common-lisp.net/project/asdf/,ASDF website}.
361If you are daring and willing to report bugs, you can get
362the latest and greatest version of ASDF from its git repository.
363@xref{Getting the latest version}.
364
365For maximum convenience you might want to have ASDF loaded
366whenever you start your Lisp implementation,
367for example by loading it from the startup script or dumping a custom core
368--- check your Lisp implementation's manual for details.
369
370
371@node Configuring ASDF, Using ASDF, Loading ASDF, Top
372@comment  node-name,  next,  previous,  up
373
374@chapter Configuring ASDF
375
376@section Configuring ASDF to find your systems
377
378So it may compile and load your systems, ASDF must be configured to find
379the @file{.asd} files that contain system definitions.
380
381Since ASDF 2, the preferred way to configure where ASDF finds your systems is
382the @code{source-registry} facility,
383fully described in its own chapter of this manual.
384@xref{Controlling where ASDF searches for systems}.
385
386The default location for a user to install Common Lisp software is under
387@file{~/.local/share/common-lisp/source/}.
388If you install software there, you don't need further configuration.
389If you're installing software yourself at a location that isn't standard,
390you have to tell ASDF where you installed it. See below.
391If you're using some tool to install software (e.g. Quicklisp),
392the authors of that tool should already have configured ASDF.
393
394The simplest way to add a path to your search path,
395say @file{/home/luser/.asd-link-farm/}
396is to create the directory
397@file{~/.config/common-lisp/source-registry.conf.d/}
398and there create a file with any name of your choice,
399and with the type @file{conf},
400for instance @file{42-asd-link-farm.conf}
401containing the line:
402
403@kbd{(:directory "/home/luser/.asd-link-farm/")}
404
405If you want all the subdirectories under @file{/home/luser/lisp/}
406to be recursively scanned for @file{.asd} files, instead use:
407
408@kbd{(:tree "/home/luser/lisp/")}
409
410Note that your Operating System distribution or your system administrator
411may already have configured system-managed libraries for you.
412
413The required @file{.conf} extension allows you to have disabled files
414or editor backups (ending in @file{~}), and works portably
415(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
416Excluded are files the name of which start with a @file{.} character.
417It is customary to start the filename with two digits
418that specify the order in which the directories will be scanned.
419
420ASDF will automatically read your configuration
421the first time you try to find a system.
422You can reset the source-registry configuration with:
423
424@lisp
425(asdf:clear-source-registry)
426@end lisp
427
428And you probably should do so before you dump your Lisp image,
429if the configuration may change
430between the machine where you save it at the time you save it
431and the machine you resume it at the time you resume it.
432
433
434@section Configuring ASDF to find your systems --- old style
435
436The old way to configure ASDF to find your systems is by
437@code{push}ing directory pathnames onto the variable
438@code{asdf:*central-registry*}.
439
440You must configure this variable between the time you load ASDF
441and the time you first try to use it.
442Loading and configuring ASDF presumably happen
443as part of some initialization script that builds or starts
444your Common Lisp software system.
445(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
446
447The @code{asdf:*central-registry*} is empty by default in ASDF 2,
448but is still supported for compatibility with ASDF 1.
449When used, it takes precedence over the above source-registry@footnote{
450It is possible to further customize
451the system definition file search.
452That's considered advanced use, and covered later:
453search forward for
454@code{*system-definition-search-functions*}.
455@xref{Defining systems with defsystem}.}.
456
457For instance, if you wanted ASDF to find the @file{.asd} file
458@file{/home/me/src/foo/foo.asd} your initialization script
459could after it loads ASDF with @code{(require "asdf")}
460configure it with:
461
462@lisp
463(push "/home/me/src/foo/" asdf:*central-registry*)
464@end lisp
465
466Note the trailing slash: when searching for a system,
467ASDF will evaluate each entry of the central registry
468and coerce the result to a pathname@footnote{
469ASDF will indeed call @code{EVAL} on each entry.
470It will also skip entries that evaluate to @code{NIL}.
471
472Strings and pathname objects are self-evaluating,
473in which case the @code{EVAL} step does nothing;
474but you may push arbitrary SEXP onto the central registry,
475that will be evaluated to compute e.g. things that depend
476on the value of shell variables or the identity of the user.
477
478The variable @code{asdf:*central-registry*} is thus a list of
479``system directory designators''.
480A @dfn{system directory designator} is a form
481which will be evaluated whenever a system is to be found,
482and must evaluate to a directory to look in.
483By ``directory'' here, we mean
484``designator for a pathname with a supplied DIRECTORY component''.
485}
486at which point the presence of the trailing directory name separator
487is necessary to tell Lisp that you're discussing a directory
488rather than a file.
489
490Typically, however, there are a lot of @file{.asd} files, and
491a common idiom was to have to put
492a bunch of @emph{symbolic links} to @file{.asd} files
493in a common directory
494and push @emph{that} directory (the ``link farm'')
495to the
496@code{asdf:*central-registry*}
497instead of pushing each of the many involved directories
498to the @code{asdf:*central-registry*}.
499ASDF knows how to follow such @emph{symlinks}
500to the actual file location when resolving the paths of system components
501(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
502if you try aliases under MacOS, we are curious to hear about your experience).
503
504For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
505is a member of @code{*central-registry*}, you could set up the
506system @var{foo} for loading with asdf with the following
507commands at the shell:
508
509@example
510$ cd /home/me/cl/systems/
511$ ln -s ~/src/foo/foo.asd .
512@end example
513
514This old style for configuring ASDF is not recommended for new users,
515but it is supported for old users, and for users who want to programmatically
516control what directories are added to the ASDF search path.
517
518
519@section Configuring where ASDF stores object files
520@findex clear-output-translations
521
522ASDF lets you configure where object files will be stored.
523Sensible defaults are provided and
524you shouldn't normally have to worry about it.
525
526This allows the same source code repository may be shared
527between several versions of several Common Lisp implementations,
528between several users using different compilation options
529and without write privileges on shared source directories, etc.
530This also allows to keep source directories uncluttered
531by plenty of object files.
532
533Starting with ASDF 2, the @code{asdf-output-translations} facility
534was added to ASDF itself, that controls where object files will be stored.
535This facility is fully described in a chapter of this manual,
536@ref{Controlling where ASDF saves compiled files}.
537
538The simplest way to add a translation to your search path,
539say from @file{/foo/bar/baz/quux/}
540to @file{/where/i/want/my/fasls/}
541is to create the directory
542@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
543and there create a file with any name of your choice and the type @file{conf},
544for instance @file{42-bazquux.conf}
545containing the line:
546
547@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
548
549To disable output translations for source under a given directory,
550say @file{/toto/tata/}
551you can create a file @file{40-disable-toto.conf}
552with the line:
553
554@kbd{("/toto/tata/")}
555
556To wholly disable output translations for all directories,
557you can create a file @file{00-disable.conf}
558with the line:
559
560@kbd{(t t)}
561
562Note that your Operating System distribution or your system administrator
563may already have configured translations for you.
564In absence of any configuration, the default is to redirect everything
565under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
566@xref{Controlling where ASDF searches for systems}, for full details.
567
568The required @file{.conf} extension allows you to have disabled files
569or editor backups (ending in @file{~}), and works portably
570(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
571Excluded are files the name of which start with a @file{.} character.
572It is customary to start the filename with two digits
573that specify the order in which the directories will be scanned.
574
575ASDF will automatically read your configuration
576the first time you try to find a system.
577You can reset the source-registry configuration with:
578
579@lisp
580(asdf:clear-output-translations)
581@end lisp
582
583And you probably should do so before you dump your Lisp image,
584if the configuration may change
585between the machine where you save it at the time you save it
586and the machine you resume it at the time you resume it.
587
588Finally note that before ASDF 2,
589other ASDF add-ons offered the same functionality,
590each in subtly different and incompatible ways:
591ASDF-Binary-Locations, cl-launch, common-lisp-controller.
592ASDF-Binary-Locations is now not needed anymore and should not be used.
593cl-launch 3.000 and common-lisp-controller 7.2 have been updated
594to just delegate this functionality to ASDF.
595
596@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
597@comment  node-name,  next,  previous,  up
598
599
600@section Resetting Configuration
601
602When you dump and restore an image, or when you tweak your configuration,
603you may want to reset the ASDF configuration.
604For that you may use the following function:
605
606@defun clear-configuration
607   undoes any ASDF configuration,
608   regarding source-registry or output-translations.
609@end defun
610
611If you use SBCL, CMUCL or SCL, you may use this snippet
612so that the ASDF configuration be cleared automatically as you dump an image:
613
614@example
615#+(or cmu sbcl scl)
616(pushnew 'clear-configuration
617         #+(or cmu scl) ext:*before-save-initializations*
618         #+sbcl sb-ext:*save-hooks*)
619@end example
620
621For compatibility with all Lisp implementations, however,
622you might want instead your build script to explicitly call
623@code{(asdf:clear-configuration)} at an appropriate moment before dumping.
624
625
626@chapter Using ASDF
627
628@section Loading a system
629
630The system @var{foo} is loaded (and compiled, if necessary)
631by evaluating the following Lisp form:
632
633@example
634(asdf:load-system :@var{foo})
635@end example
636
637On some implementations (namely recent versions of
638ABCL, Clozure CL, CLISP, CMUCL, ECL, SBCL and SCL),
639ASDF hooks into the @code{CL:REQUIRE} facility
640and you can just use:
641
642@example
643(require :@var{foo})
644@end example
645
646In older versions of ASDF, you needed to use
647@code{(asdf:oos 'asdf:load-op :@var{foo})}.
648If your ASDF is too old to provide @code{asdf:load-system} though
649we recommend that you upgrade to ASDF 2.
650@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
651
652Note the name of a system is specified as a string or a symbol,
653typically a keyword.
654If a symbol (including a keyword), its name is taken and lowercased.
655The name must be a suitable value for the @code{:name} initarg
656to @code{make-pathname} in whatever filesystem the system is to be found.
657The lower-casing-symbols behaviour is unconventional,
658but was selected after some consideration.
659Observations suggest that the type of systems we want to support
660either have lowercase as customary case (unix, mac, windows)
661or silently convert lowercase to uppercase (lpns),
662so this makes more sense than attempting to use @code{:case :common},
663which is reported not to work on some implementations
664
665
666@section Other Operations
667
668ASDF provides three commands for the most common system operations:
669@code{load-system}, @code{compile-system} or @code{test-system}.
670
671Because ASDF is an extensible system
672for defining @emph{operations} on @emph{components},
673it also provides a generic function @code{operate}
674(which is usually abbreviated by @code{oos}).
675You'll use @code{oos} whenever you want to do something beyond
676compiling, loading and testing.
677
678Output from ASDF and ASDF extensions are supposed to be sent
679to the CL stream @code{*standard-output*},
680and so rebinding that stream around calls to @code{asdf:operate}
681should redirect all output from ASDF operations.
682
683Reminder: before ASDF can operate on a system, however,
684it must be able to find and load that system's definition.
685@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
686
687
688@section Summary
689
690To use ASDF:
691
692@itemize
693@item
694Load ASDF itself into your Lisp image, either through
695@code{(require "asdf")} or else through
696@code{(load "/path/to/asdf.lisp")}.
697
698@item
699Make sure ASDF can find system definitions
700thanks to proper source-registry configuration.
701
702@item
703Load a system with @code{(load-system :my-system)}
704or use some other operation on some system of your choice.
705
706@end itemize
707
708@section Moving on
709
710That's all you need to know to use ASDF to load systems written by others.
711The rest of this manual deals with writing system definitions
712for Common Lisp software you write yourself,
713including how to extend ASDF to define new operation and component types.
714
715
716@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
717@comment  node-name,  next,  previous,  up
718@chapter Defining systems with defsystem
719
720This chapter describes how to use asdf to define systems and develop
721software.
722
723
724@menu
725* The defsystem form::
726* A more involved example::
727* The defsystem grammar::
728* Other code in .asd files::
729@end menu
730
731@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
732@comment  node-name,  next,  previous,  up
733@section The defsystem form
734
735Systems can be constructed programmatically
736by instantiating components using @code{make-instance}.
737Most of the time, however, it is much more practical to use
738a static @code{defsystem} form.
739This section begins with an example of a system definition,
740then gives the full grammar of @code{defsystem}.
741
742Let's look at a simple system.
743This is a complete file that would
744usually be saved as @file{hello-lisp.asd}:
745
746@lisp
747(in-package :asdf)
748
749(defsystem "hello-lisp"
750  :description "hello-lisp: a sample Lisp system."
751  :version "0.2.1"
752  :author "Joe User <joe@@example.com>"
753  :licence "Public Domain"
754  :components ((:file "packages")
755               (:file "macros" :depends-on ("packages"))
756               (:file "hello" :depends-on ("macros"))))
757@end lisp
758
759Some notes about this example:
760
761@itemize
762
763@item
764The file starts with an @code{in-package} form
765to use package @code{asdf}.
766You could instead start your definition by using
767a qualified name @code{asdf:defsystem}.
768
769@item
770If in addition to simply using @code{defsystem},
771you are going to define functions,
772create ASDF extension, globally bind symbols, etc.,
773it is recommended that to avoid namespace pollution between systems,
774you should create your own package for that purpose,
775for instance replacing the above @code{(in-package :asdf)} with:
776
777@lisp
778(defpackage :foo-system
779  (:use :cl :asdf))
780
781(in-package :foo-system)
782@end lisp
783
784@item
785The @code{defsystem} form defines a system named @code{hello-lisp}
786that contains three source files:
787@file{packages}, @file{macros} and @file{hello}.
788
789@item
790The file @file{macros} depends on @file{packages}
791(presumably because the package it's in is defined in @file{packages}),
792and the file @file{hello} depends on @file{macros}
793(and hence, transitively on @file{packages}).
794This means that ASDF will compile and load @file{packages} and @file{macros}
795before starting the compilation of file @file{hello}.
796
797@item
798The files are located in the same directory
799as the file with the system definition.
800ASDF resolves symbolic links (or Windows shortcuts)
801before loading the system definition file and
802stores its location in the resulting system@footnote{
803It is possible, though almost never necessary, to override this behaviour.}.
804This is a good thing because the user can move the system sources
805without having to edit the system definition.
806
807@c FIXME: Should have cross-reference to "Version specifiers" in the
808@c defsystem grammar, but the cross-referencing is so broken by
809@c insufficient node breakdown that I have not put one in.
810@item
811Make sure you know how the @code{:version} numbers will be parsed!  They
812are parsed as period-separated lists of integers.  I.e., in the example,
813@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
814In particular, version @code{0.2.1} is interpreted the same as
815@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
816even though the two are the same when interpreted as decimal fractions.
817@cindex version specifiers
818@cindex :version
819
820@end itemize
821
822@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
823@comment  node-name,  next,  previous,  up
824@section A more involved example
825
826Let's illustrate some more involved uses of @code{defsystem} via a
827slightly convoluted example:
828
829@lisp
830(defsystem "foo"
831  :version "1.0.0"
832  :components ((:module "mod"
833                            :components ((:file "bar")
834                                                  (:file"baz")
835                                                  (:file "quux"))
836                            :perform (compile-op :after (op c)
837                                                  (do-something c))
838                            :explain (compile-op :after (op c)
839                                            (explain-something c)))
840                         (:file "blah")))
841@end lisp
842
843The @code{:module} component named @code{"mod"} is a collection of three files,
844which will be located in a subdirectory of the main code directory named
845@file{mod} (this location can be overridden; see the discussion of the
846@code{:pathname} option in @ref{The defsystem grammar}).
847
848The method-form tokens provide a shorthand for defining methods on
849particular components.  This part
850
851@lisp
852                :perform (compile-op :after (op c)
853                          (do-something c))
854                :explain (compile-op :after (op c)
855                          (explain-something c))
856@end lisp
857
858has the effect of
859
860@lisp
861(defmethod perform :after ((op compile-op) (c (eql ...)))
862           (do-something c))
863(defmethod explain :after ((op compile-op) (c (eql ...)))
864           (explain-something c))
865@end lisp
866
867where @code{...} is the component in question.
868In this case @code{...} would expand to something like
869
870@lisp
871(find-component (find-system "foo") "mod")
872@end lisp
873
874For more details on the syntax of such forms, see @ref{The defsystem
875grammar}.
876For more details on what these methods do, @pxref{Operations} in
877@ref{The object model of ASDF}.
878
879@c The following plunge into the weeds is not appropriate in this
880@c location. [2010/10/03:rpg]
881@c note that although this also supports @code{:before} methods,
882@c they may not do what you want them to ---
883@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
884@c will run after all the dependencies and sub-components have been processed,
885@c but before the component in question has been compiled.
886
887@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
888@comment  node-name,  next,  previous,  up
889@section The defsystem grammar
890
891@c FIXME: @var typesetting not consistently used here.  We should either expand
892@c its use to everywhere, or we should kill it everywhere.
893
894
895@example
896system-definition := ( defsystem system-designator @var{system-option}* )
897
898system-option := :defsystem-depends-on system-list
899                 | :class class-name (see discussion below)
900                 | module-option
901                 | option
902
903module-option := :components component-list
904                 | :serial [ t | nil ]
905                 | :if-component-dep-fails component-dep-fail-option
906
907option :=
908        | :pathname pathname-specifier
909        | :default-component-class class-name
910        | :perform method-form
911        | :explain method-form
912        | :output-files method-form
913        | :operation-done-p method-form
914        | :depends-on ( @var{dependency-def}* )
915        | :in-order-to ( @var{dependency}+ )
916
917
918system-list := ( @var{simple-component-name}* )
919
920component-list := ( @var{component-def}* )
921
922component-def  := ( component-type simple-component-name @var{option}* )
923
924component-type := :system | :module | :file | :static-file | other-component-type
925
926other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
927
928dependency-def := simple-component-name
929               | ( :feature name )
930               | ( :version simple-component-name version-specifier)
931
932dependency := (dependent-op @var{requirement}+)
933requirement := (required-op @var{required-component}+)
934             | (feature feature-name)
935dependent-op := operation-name
936required-op := operation-name | feature
937
938simple-component-name := string
939                      |  symbol
940
941pathname-specifier := pathname | string | symbol
942
943method-form := (operation-name qual lambda-list @&rest body)
944qual := method qualifier
945
946component-dep-fail-option := :fail | :try-next | :ignore
947@end example
948
949
950@subsection Component names
951
952Component names (@code{simple-component-name})
953may be either strings or symbols.
954
955@subsection Component types
956
957Component type names, even if expressed as keywords, will be looked up
958by name in the current package and in the asdf package, if not found in
959the current package.  So a component type @code{my-component-type}, in
960the current package @code{my-system-asd} can be specified as
961@code{:my-component-type}, or @code{my-component-type}.
962
963@subsection System class names
964
965A system class name will be looked up in the same way as a Component
966type (see above).  Typically, one will not need to specify a system
967class name, unless using a non-standard system class defined in some
968ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
969see below.  For such class names in the ASDF package, we recommend that
970the @code{:class} option be specified using a keyword symbol, such as
971
972@example
973:class :MY-NEW-SYSTEM-SUBCLASS
974@end example
975
976This practice will ensure that package name conflicts are avoided.
977Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
978the current package @emph{before} it has been exported from the ASDF
979extension loaded by @code{:defsystem-depends-on}, causing a name
980conflict in the current package.
981
982@subsection Defsystem depends on
983
984The @code{:defsystem-depends-on} option to @code{defsystem} allows the
985programmer to specify another ASDF-defined system or set of systems that
986must be loaded @emph{before} the system definition is processed.
987Typically this is used to load an ASDF extension that is used in the
988system definition.
989
990@subsection Pathname specifiers
991@cindex pathname specifiers
992
993A pathname specifier (@code{pathname-specifier})
994may be a pathname, a string or a symbol.
995When no pathname specifier is given for a component,
996which is the usual case, the component name itself is used.
997
998If a string is given, which is the usual case,
999the string will be interpreted as a Unix-style pathname
1000where @code{/} characters will be interpreted as directory separators.
1001Usually, Unix-style relative pathnames are used
1002(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1003they are relative to the path of the parent component.
1004Finally, depending on the @code{component-type},
1005the pathname may be interpreted as either a file or a directory,
1006and if it's a file,
1007a file type may be added corresponding to the @code{component-type},
1008or else it will be extracted from the string itself (if applicable).
1009
1010For instance, the @code{component-type} @code{:module}
1011wants a directory pathname, and so a string @code{"foo/bar"}
1012will be interpreted as the pathname @file{#p"foo/bar/"}.
1013On the other hand, the @code{component-type} @code{:file}
1014wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1015will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1016and a string @code{"foo/bar.quux"}
1017will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1018Finally, the @code{component-type} @code{:static-file}
1019wants a file without specifying a type, and so a string @code{"foo/bar"}
1020will be interpreted as the pathname @file{#p"foo/bar"},
1021and a string @code{"foo/bar.quux"}
1022will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1023
1024ASDF does not interpret the string @code{".."} to designate the parent
1025directory.  This string will be passed through to the underlying
1026operating system for interpretation.  We @emph{believe} that this will
1027work on all platforms where ASDF is deployed, but do not guarantee this
1028behavior.  A pathname object with a relative directory component of
1029@code{:up} or @code{:back} is the only guaranteed way to specify a
1030parent directory.
1031
1032If a symbol is given, it will be translated into a string,
1033and downcased in the process.
1034The downcasing of symbols is unconventional,
1035but was selected after some consideration.
1036Observations suggest that the type of systems we want to support
1037either have lowercase as customary case (Unix, Mac, windows)
1038or silently convert lowercase to uppercase (lpns),
1039so this makes more sense than attempting to use @code{:case :common}
1040as argument to @code{make-pathname},
1041which is reported not to work on some implementations.
1042
1043Pathname objects may be given to override the path for a component.
1044Such objects are typically specified using reader macros such as @code{#p}
1045or @code{#.(make-pathname ...)}.
1046Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1047and that the behavior of @code{parse-namestring} is completely non-portable,
1048unless you are using Common Lisp @code{logical-pathname}s
1049(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1050Pathnames made with @code{#.(make-pathname ...)}
1051can usually be done more easily with the string syntax above.
1052The only case that you really need a pathname object is to override
1053the component-type default file type for a given component.
1054Therefore, pathname objects should only rarely be used.
1055Unhappily, ASDF 1 didn't properly support
1056parsing component names as strings specifying paths with directories,
1057and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1058
1059Note that when specifying pathname objects,
1060ASDF does not do any special interpretation of the pathname
1061influenced by the component type, unlike the procedure for
1062pathname-specifying strings.
1063On the one hand, you have to be careful to provide a pathname that correctly
1064fulfills whatever constraints are required from that component type
1065(e.g. naming a directory or a file with appropriate type);
1066on the other hand, you can circumvent the file type that would otherwise
1067be forced upon you if you were specifying a string.
1068
1069@subsection Version specifiers
1070@cindex version specifiers
1071@cindex :version
1072
1073Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1074@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1075In particular, version @code{0.2.1} is interpreted the same as
1076@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1077even though the two are the same when interpreted as decimal fractions.
1078
1079System definers are encouraged to use version identifiers of the form
1080@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1081API) and patch level.
1082
1083@xref{Common attributes of components}.
1084
1085
1086@subsection Using logical pathnames
1087@cindex logical pathnames
1088
1089We do not generally recommend the use of logical pathnames,
1090especially not so to newcomers to Common Lisp.
1091However, we do support the use of logical pathnames by old timers,
1092when such is their preference.
1093
1094To use logical pathnames,
1095you will have to provide a pathname object as a @code{:pathname} specifier
1096to components that use it, using such syntax as
1097@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1098
1099You only have to specify such logical pathname
1100for your system or some top-level component.
1101Sub-components' relative pathnames,
1102specified using the string syntax for names,
1103will be properly merged with the pathnames of their parents.
1104The specification of a logical pathname host however is @emph{not}
1105otherwise directly supported in the ASDF syntax
1106for pathname specifiers as strings.
1107
1108The @code{asdf-output-translation} layer will
1109avoid trying to resolve and translate logical pathnames.
1110The advantage of this is that
1111you can define yourself what translations you want to use
1112with the logical pathname facility.
1113The disadvantage is that if you do not define such translations,
1114any system that uses logical pathnames will behave differently under
1115asdf-output-translations than other systems you use.
1116
1117If you wish to use logical pathnames you will have to configure the
1118translations yourself before they may be used.
1119ASDF currently provides no specific support
1120for defining logical pathname translations.
1121
1122Note that the reasons we do not recommend logical pathnames are that
1123(1) there is no portable way to set up logical pathnames before they are used,
1124(2) logical pathnames are limited to only portably use
1125a single character case, digits and hyphens.
1126While you can solve the first issue on your own,
1127describing how to do it on each of fifteen implementations supported by ASDF
1128is more than we can document.
1129As for the second issue, mind that the limitation is notably enforced on SBCL,
1130and that you therefore can't portably violate the limitations
1131but must instead define some encoding of your own and add individual mappings
1132to name physical pathnames that do not fit the restrictions.
1133This can notably be a problem when your Lisp files are part of a larger project
1134in which it is common to name files or directories in a way that
1135includes the version numbers of supported protocols,
1136or in which files are shared with software written
1137in different programming languages where conventions include the use of
1138underscores, dots or CamelCase in pathnames.
1139
1140
1141@subsection Serial dependencies
1142@cindex serial dependencies
1143
1144If the @code{:serial t} option is specified for a module,
1145ASDF will add dependencies for each child component,
1146on all the children textually preceding it.
1147This is done as if by @code{:depends-on}.
1148
1149@lisp
1150:serial t
1151:components ((:file "a") (:file "b") (:file "c"))
1152@end lisp
1153
1154is equivalent to
1155
1156@lisp
1157:components ((:file "a")
1158             (:file "b" :depends-on ("a"))
1159             (:file "c" :depends-on ("a" "b")))
1160@end lisp
1161
1162
1163@subsection Source location
1164
1165The @code{:pathname} option is optional in all cases for systems
1166defined via @code{defsystem},
1167and in the usual case the user is recommended not to supply it.
1168
1169Instead, ASDF follows a hairy set of rules that are designed so that
1170@enumerate
1171@item
1172@code{find-system}
1173will load a system from disk
1174and have its pathname default to the right place.
1175@item
1176This pathname information will not be overwritten with
1177@code{*default-pathname-defaults*}
1178(which could be somewhere else altogether)
1179if the user loads up the @file{.asd} file into his editor
1180and interactively re-evaluates that form.
1181@end enumerate
1182
1183If a system is being loaded for the first time,
1184its top-level pathname will be set to:
1185
1186@itemize
1187@item
1188The host/device/directory parts of @code{*load-truename*},
1189if it is bound.
1190@item
1191@code{*default-pathname-defaults*}, otherwise.
1192@end itemize
1193
1194If a system is being redefined, the top-level pathname will be
1195
1196@itemize
1197@item
1198changed, if explicitly supplied or obtained from @code{*load-truename*}
1199(so that an updated source location is reflected in the system definition)
1200@item
1201changed if it had previously been set from @code{*default-pathname-defaults*}
1202@item
1203left as before, if it had previously been set from @code{*load-truename*}
1204and @code{*load-truename*} is currently unbound
1205(so that a developer can evaluate a @code{defsystem} form
1206from within an editor without clobbering its source location)
1207@end itemize
1208
1209@subsection if-component-dep-fails option
1210
1211This option is only appropriate for module components (including
1212systems), not individual source files.
1213
1214For more information about this option, @pxref{Pre-defined subclasses of component}.
1215
1216@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1217@section Other code in .asd files
1218
1219Files containing @code{defsystem} forms
1220are regular Lisp files that are executed by @code{load}.
1221Consequently, you can put whatever Lisp code you like into these files
1222(e.g., code that examines the compile-time environment
1223and adds appropriate features to @code{*features*}).
1224However, some conventions should be followed,
1225so that users can control certain details of execution
1226of the Lisp in @file{.asd} files:
1227
1228@itemize
1229@item
1230Any informative output
1231(other than warnings and errors,
1232which are the condition system's to dispose of)
1233should be sent to the standard CL stream @code{*standard-output*},
1234so that users can easily control the disposition
1235of output from ASDF operations.
1236@end itemize
1237
1238
1239@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1240@comment  node-name,  next,  previous,  up
1241@chapter The object model of ASDF
1242
1243ASDF is designed in an object-oriented way from the ground up.
1244Both a system's structure and the operations that can be performed on systems
1245follow a protocol.
1246ASDF is extensible to new operations and to new component types.
1247This allows the addition of behaviours:
1248for example, a new component could be added for Java JAR archives,
1249and methods specialised on @code{compile-op} added for it
1250that would accomplish the relevant actions.
1251
1252This chapter deals with @emph{components}, the building blocks of a system,
1253and @emph{operations}, the actions that can be performed on a system.
1254
1255
1256
1257@menu
1258* Operations::
1259* Components::
1260* Functions::
1261@end menu
1262
1263@node  Operations, Components, The object model of ASDF, The object model of ASDF
1264@comment  node-name,  next,  previous,  up
1265@section Operations
1266@cindex operation
1267
1268An @dfn{operation} object of the appropriate type is instantiated
1269whenever the user wants to do something with a system like
1270
1271@itemize
1272@item compile all its files
1273@item load the files into a running lisp environment
1274@item copy its source files somewhere else
1275@end itemize
1276
1277Operations can be invoked directly, or examined
1278to see what their effects would be without performing them.
1279@emph{FIXME: document how!}
1280There are a bunch of methods specialised on operation and component type
1281that actually do the grunt work.
1282
1283The operation object contains whatever state is relevant for this purpose
1284(perhaps a list of visited nodes, for example)
1285but primarily is a nice thing to specialise operation methods on
1286and easier than having them all be @code{EQL} methods.
1287
1288Operations are invoked on systems via @code{operate}.
1289@anchor{operate}
1290@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1291@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1292@code{operate} invokes @var{operation} on @var{system}.
1293@code{oos} is a synonym for @code{operate}.
1294
1295@var{operation} is a symbol that is passed, along with the supplied
1296@var{initargs}, to @code{make-instance} to create the operation object.
1297@var{system} is a system designator.
1298
1299The @var{initargs} are passed to the @code{make-instance} call
1300when creating the operation object.
1301Note that dependencies may cause the operation
1302to invoke other operations on the system or its components:
1303the new operations will be created
1304with the same @var{initargs} as the original one.
1305
1306@end deffn
1307
1308@menu
1309* Predefined operations of ASDF::
1310* Creating new operations::
1311@end menu
1312
1313@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1314@comment  node-name,  next,  previous,  up
1315@subsection Predefined operations of ASDF
1316
1317All the operations described in this section are in the @code{asdf} package.
1318They are invoked via the @code{operate} generic function.
1319
1320@lisp
1321(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1322@end lisp
1323
1324@deffn Operation @code{compile-op} @&key @code{proclamations}
1325
1326This operation compiles the specified component.
1327If proclamations are supplied, they will be proclaimed.
1328This is a good place to specify optimization settings.
1329
1330When creating a new component type,
1331you should provide methods for @code{compile-op}.
1332
1333When @code{compile-op} is invoked,
1334component dependencies often cause some parts of the system
1335to be loaded as well as compiled.
1336Invoking @code{compile-op}
1337does not necessarily load all the parts of the system, though;
1338use @code{load-op} to load a system.
1339@end deffn
1340
1341@deffn Operation @code{load-op} @&key @code{proclamations}
1342
1343This operation loads a system.
1344
1345The default methods for @code{load-op} compile files before loading them.
1346For parity, your own methods on new component types should probably do so too.
1347@end deffn
1348
1349@deffn Operation @code{load-source-op}
1350
1351This operation will load the source for the files in a module
1352even if the source files have been compiled.
1353Systems sometimes have knotty dependencies
1354which require that sources are loaded
1355before they can be compiled.
1356This is how you do that.
1357
1358If you are creating a component type,
1359you need to implement this operation --- at least, where meaningful.
1360@end deffn
1361
1362@anchor{test-op}
1363@deffn Operation @code{test-op}
1364
1365This operation will perform some tests on the module.
1366The default method will do nothing.
1367The default dependency is to require
1368@code{load-op} to be performed on the module first.
1369The default @code{operation-done-p} is that the operation is @emph{never} done
1370---
1371we assume that if you invoke the @code{test-op},
1372you want to test the system, even if you have already done so.
1373
1374The results of this operation are not defined by ASDF.
1375It has proven difficult to define how the test operation
1376should signal its results to the user
1377in a way that is compatible with all of the various test libraries
1378and test techniques in use in the community.
1379@end deffn
1380
1381@c @deffn Operation test-system-version @&key minimum
1382
1383@c Asks the system whether it satisfies a version requirement.
1384
1385@c The default method accepts a string, which is expected to contain of a
1386@c number of integers separated by #\. characters.  The method is not
1387@c recursive.  The component satisfies the version dependency if it has
1388@c the same major number as required and each of its sub-versions is
1389@c greater than or equal to the sub-version number required.
1390
1391@c @lisp
1392@c (defun version-satisfies (x y)
1393@c   (labels ((bigger (x y)
1394@c           (cond ((not y) t)
1395@c                 ((not x) nil)
1396@c                 ((> (car x) (car y)) t)
1397@c                 ((= (car x) (car y))
1398@c                  (bigger (cdr x) (cdr y))))))
1399@c     (and (= (car x) (car y))
1400@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1401@c @end lisp
1402
1403@c If that doesn't work for your system, you can override it.  I hope
1404@c you have as much fun writing the new method as @verb{|#lisp|} did
1405@c reimplementing this one.
1406@c @end deffn
1407
1408@c @deffn Operation feature-dependent-op
1409
1410@c An instance of @code{feature-dependent-op} will ignore any components
1411@c which have a @code{features} attribute, unless the feature combination
1412@c it designates is satisfied by @code{*features*}.  This operation is
1413@c not intended to be instantiated directly, but other operations may
1414@c inherit from it.
1415
1416@c @end deffn
1417
1418@node  Creating new operations,  , Predefined operations of ASDF, Operations
1419@comment  node-name,  next,  previous,  up
1420@subsection Creating new operations
1421
1422ASDF was designed to be extensible in an object-oriented fashion.
1423To teach ASDF new tricks, a programmer can implement the behaviour he wants
1424by creating a subclass of @code{operation}.
1425
1426ASDF's pre-defined operations are in no way ``privileged'',
1427but it is requested that developers never use the @code{asdf} package
1428for operations they develop themselves.
1429The rationale for this rule is that we don't want to establish a
1430``global asdf operation name registry'',
1431but also want to avoid name clashes.
1432
1433An operation must provide methods for the following generic functions
1434when invoked with an object of type @code{source-file}:
1435@emph{FIXME describe this better}
1436
1437@itemize
1438
1439@item @code{output-files}
1440The @code{output-files} method determines where the method will put its files.
1441It returns two values, a list of pathnames, and a boolean.
1442If the boolean is @code{T} then the pathnames are marked
1443not be translated by enclosing @code{:around} methods.
1444If the boolean is @code{NIL} then enclosing @code{:around} methods
1445may translate these pathnames, e.g. to ensure object files
1446are somehow stored in some implementation-dependent cache.
1447@item @code{perform}
1448The @code{perform} method must call @code{output-files}
1449to find out where to put its files,
1450because the user is allowed to override.
1451@item @code{output-files}
1452for local policy @code{explain}
1453@item @code{operation-done-p},
1454if you don't like the default one
1455
1456@end itemize
1457
1458Operations that print output should send that output to the standard
1459CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1460
1461@node Components, Functions, Operations, The object model of ASDF
1462@comment  node-name,  next,  previous,  up
1463@section Components
1464@cindex component
1465@cindex system
1466@cindex system designator
1467@vindex *system-definition-search-functions*
1468
1469A @dfn{component} represents a source file or
1470(recursively) a collection of components.
1471A @dfn{system} is (roughly speaking) a top-level component
1472that can be found via @code{find-system}.
1473
1474A @dfn{system designator} is a string or symbol
1475and behaves just like any other component name
1476(including with regard to the case conversion rules for component names).
1477
1478
1479@defun find-system system-designator &optional (error-p t)
1480
1481Given a system designator, @code{find-system} finds and returns a system.
1482If no system is found, an error of type
1483@code{missing-component} is thrown,
1484or @code{nil} is returned if @code{error-p} is false.
1485
1486To find and update systems, @code{find-system} funcalls each element
1487in the @code{*system-definition-search-functions*} list,
1488expecting a pathname to be returned, or a system object,
1489from which a pathname may be extracted, and that will be registered.
1490The resulting pathname (if any) is loaded
1491if one of the following conditions is true:
1492
1493@itemize
1494@item
1495there is no system of that name in memory
1496@item
1497the pathname is different from that which was previously loaded
1498@item
1499the file's @code{last-modified} time exceeds the @code{last-modified} time
1500of the system in memory
1501@end itemize
1502
1503When system definitions are loaded from @file{.asd} files,
1504a new scratch package is created for them to load into,
1505so that different systems do not overwrite each others operations.
1506The user may also wish to (and is recommended to)
1507include @code{defpackage} and @code{in-package} forms
1508in his system definition files, however,
1509so that they can be loaded manually if need be.
1510
1511The default value of @code{*system-definition-search-functions*}
1512is a list of two functions.
1513The first function looks in each of the directories given
1514by evaluating members of @code{*central-registry*}
1515for a file whose name is the name of the system and whose type is @file{asd}.
1516The first such file is returned,
1517whether or not it turns out to actually define the appropriate system.
1518The second function does something similar,
1519for the directories specified in the @code{source-registry}.
1520Hence, it is strongly advised to define a system
1521@var{foo} in the corresponding file @var{foo.asd}.
1522@end defun
1523
1524
1525@menu
1526* Common attributes of components::
1527* Pre-defined subclasses of component::
1528* Creating new component types::
1529@end menu
1530
1531@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1532@comment  node-name,  next,  previous,  up
1533@subsection Common attributes of components
1534
1535All components, regardless of type, have the following attributes.
1536All attributes except @code{name} are optional.
1537
1538@subsubsection Name
1539
1540A component name is a string or a symbol.
1541If a symbol, its name is taken and lowercased.
1542
1543Unless overridden by a @code{:pathname} attribute,
1544the name will be interpreted as a pathname specifier according
1545to a Unix-style syntax.
1546@xref{The defsystem grammar,,Pathname specifiers}.
1547
1548@subsubsection Version identifier
1549@findex version-satisfies
1550@cindex :version
1551
1552This optional attribute is used by the generic function
1553@code{version-satisfies}, which tests to see if @code{:version}
1554dependencies are satisfied.
1555the version should be a string of integers separated by dots,
1556for example @samp{1.0.11}.
1557For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1558
1559@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1560@c @xref{Predefined operations of ASDF}.
1561@c @emph{Nota Bene}:
1562@c This operation, planned for ASDF 1,
1563@c is still not implemented yet as of ASDF 2.
1564@c Don't hold your breath.
1565
1566
1567
1568@subsubsection Required features
1569
1570@emph{FIXME: This subsection seems to contradict the
1571@code{defsystem} grammar subsection,
1572which doesn't provide any obvious way to specify required features.
1573Furthermore, in 2009, discussions on the
1574@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1575suggested that the specification of required features may be broken,
1576and that no one may have been using them for a while.
1577Please contact the
1578@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1579if you are interested in getting this features feature fixed.}
1580
1581Traditionally defsystem users have used reader conditionals
1582to include or exclude specific per-implementation files.
1583This means that any single implementation cannot read the entire system,
1584which becomes a problem if it doesn't wish to compile it,
1585but instead for example to create an archive file containing all the sources,
1586as it will omit to process the system-dependent sources for other systems.
1587
1588Each component in an asdf system may therefore specify features using
1589the same syntax as @code{#+} does, and it will (somehow) be ignored for
1590certain operations unless the feature conditional is a member of
1591@code{*features*}.
1592
1593
1594@subsubsection Dependencies
1595
1596This attribute specifies dependencies of the component on its siblings.
1597It is optional but often necessary.
1598
1599There is an excitingly complicated relationship between the initarg
1600and the method that you use to ask about dependencies
1601
1602Dependencies are between (operation component) pairs.
1603In your initargs for the component, you can say
1604
1605@lisp
1606:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1607              (load-op (load-op "foo")))
1608@end lisp
1609
1610This means the following things:
1611@itemize
1612@item
1613before performing compile-op on this component, we must perform
1614load-op on @var{a} and @var{b}, and compile-op on @var{c},
1615@item
1616before performing @code{load-op}, we have to load @var{foo}
1617@end itemize
1618
1619The syntax is approximately
1620
1621@verbatim
1622(this-op {(other-op required-components)}+)
1623
1624simple-component-name := string
1625                      |  symbol
1626
1627required-components := simple-component-name
1628                     | (required-components required-components)
1629
1630component-name := simple-component-name
1631                | (:version simple-component-name minimum-version-object)
1632@end verbatim
1633
1634Side note:
1635
1636This is on a par with what ACL defsystem does.
1637mk-defsystem is less general: it has an implied dependency
1638
1639@verbatim
1640  for all x, (load x) depends on (compile x)
1641@end verbatim
1642
1643and using a @code{:depends-on} argument to say that @var{b} depends on
1644@var{a} @emph{actually} means that
1645
1646@verbatim
1647  (compile b) depends on (load a)
1648@end verbatim
1649
1650This is insufficient for e.g. the McCLIM system, which requires that
1651all the files are loaded before any of them can be compiled ]
1652
1653End side note
1654
1655In ASDF, the dependency information for a given component and operation
1656can be queried using @code{(component-depends-on operation component)},
1657which returns a list
1658
1659@lisp
1660((load-op "a") (load-op "b") (compile-op "c") ...)
1661@end lisp
1662
1663@code{component-depends-on} can be subclassed for more specific
1664component/operation types: these need to @code{(call-next-method)}
1665and append the answer to their dependency, unless
1666they have a good reason for completely overriding the default dependencies.
1667
1668If it weren't for CLISP, we'd be using @code{LIST} method
1669combination to do this transparently.
1670But, we need to support CLISP.
1671If you have the time for some CLISP hacking,
1672I'm sure they'd welcome your fixes.
1673@c Doesn't CLISP now support LIST method combination?
1674
1675See the discussion of the semantics of @code{:version} in the defsystem
1676grammar.
1677
1678@c FIXME: Should have cross-reference to "Version specifiers" in the
1679@c defsystem grammar, but the cross-referencing is so broken by
1680@c insufficient node breakdown that I have not put one in.
1681
1682
1683@subsubsection pathname
1684
1685This attribute is optional and if absent (which is the usual case),
1686the component name will be used.
1687
1688@xref{The defsystem grammar,,Pathname specifiers},
1689for an explanation of how this attribute is interpreted.
1690
1691Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1692does additional processing to set the filesystem location of
1693the top component in that system.
1694This is detailed elsewhere. @xref{Defining systems with defsystem}.
1695
1696
1697@subsubsection properties
1698
1699This attribute is optional.
1700
1701Packaging systems often require information about files or systems
1702in addition to that specified by ASDF's pre-defined component attributes.
1703Programs that create vendor packages out of ASDF systems therefore
1704have to create ``placeholder'' information to satisfy these systems.
1705Sometimes the creator of an ASDF system may know the additional
1706information and wish to provide it directly.
1707
1708@code{(component-property component property-name)} and
1709associated @code{setf} method will allow
1710the programmatic update of this information.
1711Property names are compared as if by @code{EQL},
1712so use symbols or keywords or something.
1713
1714@menu
1715* Pre-defined subclasses of component::
1716* Creating new component types::
1717@end menu
1718
1719@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1720@comment  node-name,  next,  previous,  up
1721@subsection Pre-defined subclasses of component
1722
1723@deffn Component source-file
1724
1725A source file is any file that the system does not know how to
1726generate from other components of the system.
1727
1728Note that this is not necessarily the same thing as
1729``a file containing data that is typically fed to a compiler''.
1730If a file is generated by some pre-processor stage
1731(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1732then it is not, by this definition, a source file.
1733Conversely, we might have a graphic file
1734that cannot be automatically regenerated,
1735or a proprietary shared library that we received as a binary:
1736these do count as source files for our purposes.
1737
1738Subclasses of source-file exist for various languages.
1739@emph{FIXME: describe these.}
1740@end deffn
1741
1742@deffn Component module
1743
1744A module is a collection of sub-components.
1745
1746A module component has the following extra initargs:
1747
1748@itemize
1749@item
1750@code{:components} the components contained in this module
1751
1752@item
1753@code{:default-component-class}
1754All children components which don't specify their class explicitly
1755are inferred to be of this type.
1756
1757@item
1758@code{:if-component-dep-fails}
1759This attribute takes one of the values
1760@code{:fail}, @code{:try-next}, @code{:ignore},
1761its default value is @code{:fail}.
1762The other values can be used for implementing conditional compilation
1763based on implementation @code{*features*},
1764for the case where it is not necessary for all files in a module to be
1765compiled.
1766@emph{FIXME: such conditional compilation has been reported
1767to be broken in 2009.}
1768
1769@item
1770@code{:serial} When this attribute is set,
1771each subcomponent of this component is assumed to depend on all subcomponents
1772before it in the list given to @code{:components}, i.e.
1773all of them are loaded before a compile or load operation is performed on it.
1774
1775@end itemize
1776
1777The default operation knows how to traverse a module, so
1778most operations will not need to provide methods specialised on modules.
1779
1780@code{module} may be subclassed to represent components such as
1781foreign-language linked libraries or archive files.
1782@end deffn
1783
1784@deffn Component system
1785
1786@code{system} is a subclass of @code{module}.
1787
1788A system is a module with a few extra attributes for documentation
1789purposes; these are given elsewhere.
1790@xref{The defsystem grammar}.
1791
1792Users can create new classes for their systems:
1793the default @code{defsystem} macro takes a @code{:class} keyword argument.
1794@end deffn
1795
1796@node  Creating new component types,  , Pre-defined subclasses of component, Components
1797@comment  node-name,  next,  previous,  up
1798@subsection Creating new component types
1799
1800New component types are defined by subclassing one of the existing
1801component classes and specializing methods on the new component class.
1802
1803@emph{FIXME: this should perhaps be explained more throughly,
1804not only by example ...}
1805
1806As an example, suppose we have some implementation-dependent
1807functionality that we want to isolate
1808in one subdirectory per Lisp implementation our system supports.
1809We create a subclass of
1810@code{cl-source-file}:
1811
1812@lisp
1813(defclass unportable-cl-source-file (cl-source-file)
1814  ())
1815@end lisp
1816
1817Function @code{asdf:implementation-type} (exported since 2.014.14)
1818gives us the name of the subdirectory.
1819All that's left is to define how to calculate the pathname
1820of an @code{unportable-cl-source-file}.
1821
1822@lisp
1823(defmethod component-pathname ((component unportable-cl-source-file))
1824  (merge-pathnames*
1825   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1826   (call-next-method)))
1827@end lisp
1828
1829The new component type is used in a @code{defsystem} form in this way:
1830
1831@lisp
1832(defsystem :foo
1833    :components
1834    ((:file "packages")
1835     ...
1836     (:unportable-cl-source-file "threads"
1837      :depends-on ("packages" ...))
1838     ...
1839    )
1840@end lisp
1841
1842@node Functions,  , Components, The object model of ASDF
1843@comment  node-name,  next,  previous,  up
1844@section Functions
1845@findex version-satisfies
1846
1847@deffn version-satisfies @var{version} @var{version-spec}
1848Does @var{version} satisfy the @var{version-spec}.  A generic function.
1849ASDF provides built-in methods for @var{version} being a
1850@code{component} or @code{string}.  @var{version-spec} should be a
1851string.
1852
1853In the wild, we typically see version numbering only on components of
1854type @code{system}.
1855
1856For more information about how @code{version-satisfies} interprets
1857version strings and specifications, @pxref{The defsystem grammar} and
1858@ref{Common attributes of components}.
1859@end deffn
1860
1861@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1862@comment  node-name,  next,  previous,  up
1863@chapter Controlling where ASDF searches for systems
1864
1865@section Configurations
1866
1867Configurations specify paths where to find system files.
1868
1869@enumerate
1870
1871@item
1872The search registry may use some hardcoded wrapping registry specification.
1873This allows some implementations (notably SBCL) to specify where to find
1874some special implementation-provided systems that
1875need to precisely match the version of the implementation itself.
1876
1877@item
1878An application may explicitly initialize the source-registry configuration
1879using the configuration API
1880(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1881in which case this takes precedence.
1882It may itself compute this configuration from the command-line,
1883from a script, from its own configuration file, etc.
1884
1885@item
1886The source registry will be configured from
1887the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1888
1889@item
1890The source registry will be configured from
1891user configuration file
1892@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1893(which defaults to
1894@file{~/.config/common-lisp/source-registry.conf})
1895if it exists.
1896
1897@item
1898The source registry will be configured from
1899user configuration directory
1900@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1901(which defaults to
1902@file{~/.config/common-lisp/source-registry.conf.d/})
1903if it exists.
1904
1905@item
1906The source registry will be configured from
1907system configuration file
1908@file{/etc/common-lisp/source-registry.conf}
1909if it exists/
1910
1911@item
1912The source registry will be configured from
1913system configuration directory
1914@file{/etc/common-lisp/source-registry.conf.d/}
1915if it exists.
1916
1917@item
1918The source registry will be configured from a default configuration.
1919This configuration may allow for implementation-specific systems
1920to be found, for systems to be found the current directory
1921(at the time that the configuration is initialized) as well as
1922@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1923@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1924For instance, SBCL will include directories for its contribs
1925when it can find them; it will look for them where SBCL was installed,
1926or at the location specified by the @code{SBCL_HOME} environment variable.
1927
1928@end enumerate
1929
1930Each of these configurations is specified as an s-expression
1931in a trivial domain-specific language (defined below).
1932Additionally, a more shell-friendly syntax is available
1933for the environment variable (defined yet below).
1934
1935Each of these configurations is only used if the previous
1936configuration explicitly or implicitly specifies that it
1937includes its inherited configuration.
1938
1939Additionally, some implementation-specific directories
1940may be automatically prepended to whatever directories are specified
1941in configuration files, no matter if the last one inherits or not.
1942
1943@section XDG base directory
1944
1945Note that we purport to respect the XDG base directory specification
1946as to where configuration files are located,
1947where data files are located,
1948where output file caches are located.
1949Mentions of XDG variables refer to that document.
1950
1951@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1952
1953This specification allows the user to specify some environment variables
1954to customize how applications behave to his preferences.
1955
1956On Windows platforms, when not using Cygwin,
1957instead of the XDG base directory specification,
1958we try to use folder configuration from the registry regarding
1959@code{Common AppData} and similar directories.
1960However, support for querying the Windows registry is limited as of ASDF 2,
1961and on many implementations, we may fall back to always using the defaults
1962without consulting the registry.
1963Patches welcome.
1964
1965@section Backward Compatibility
1966
1967For backward compatibility as well as to provide a practical backdoor for hackers,
1968ASDF will first search for @code{.asd} files in the directories specified in
1969@code{asdf:*central-registry*}
1970before it searches in the source registry above.
1971
1972@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
1973
1974By default, @code{asdf:*central-registry*} will be empty.
1975
1976This old mechanism will therefore not affect you if you don't use it,
1977but will take precedence over the new mechanism if you do use it.
1978
1979@section Configuration DSL
1980
1981Here is the grammar of the s-expression (SEXP) DSL for source-registry
1982configuration:
1983
1984@c FIXME: This is too wide for happy compilation into pdf.
1985
1986@example
1987;; A configuration is a single SEXP starting with keyword :source-registry
1988;; followed by a list of directives.
1989CONFIGURATION := (:source-registry DIRECTIVE ...)
1990
1991;; A directive is one of the following:
1992DIRECTIVE :=
1993    ;; INHERITANCE DIRECTIVE:
1994    ;; Your configuration expression MUST contain
1995    ;; exactly one of either of these:
1996    :inherit-configuration | ; splices inherited configuration (often specified last)
1997    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1998
1999    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2000    ;; you want to use new configuration features but have to bootstrap a
2001    ;; the newer required ASDF from an older release that doesn't sport said features:
2002    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2003
2004    ;; add a single directory to be scanned (no recursion)
2005    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
2006
2007    ;; add a directory hierarchy, recursing but excluding specified patterns
2008    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
2009
2010    ;; override the defaults for exclusion patterns
2011    (:exclude EXCLUSION-PATTERN ...) |
2012    ;; augment the defaults for exclusion patterns
2013    (:also-exclude EXCLUSION-PATTERN ...) |
2014    ;; Note that the scope of a an exclude pattern specification is
2015    ;; the rest of the current configuration expression or file.
2016
2017    ;; splice the parsed contents of another config file
2018    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
2019
2020    ;; This directive specifies that some default must be spliced.
2021    :default-registry
2022
2023REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
2024DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2025
2026PATHNAME-DESIGNATOR :=
2027    NIL | ;; Special: skip this entry.
2028    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
2029
2030EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2031  against the name of a any subdirectory in the directory component
2032        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2033@end example
2034
2035Pathnames are designated using another DSL,
2036shared with the output-translations configuration DSL below.
2037The DSL is resolved by the function @code{asdf::resolve-location},
2038to be documented and exported at some point in the future.
2039
2040@example
2041ABSOLUTE-COMPONENT-DESIGNATOR :=
2042    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2043    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2044             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2045             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2046             ;; Note that none of the above applies to strings used in *central-registry*,
2047             ;; which doesn't use this DSL: they are processed as normal namestrings.
2048             ;; however, you can compute what you put in the *central-registry*
2049             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2050    PATHNAME | ;; pathname (better be an absolute path, or bust)
2051               ;; In output-translations, unless followed by relative components,
2052               ;; it better have appropriate wildcards, as in **/*.*.*
2053    :HOME | ;; designates the user-homedir-pathname ~/
2054    :USER-CACHE | ;; designates the default location for the user cache
2055    :HERE | ;; designates the location of the configuration file
2056            ;; (or *default-pathname-defaults*, if invoked interactively)
2057    :ROOT ;; magic, for output-translations source only: paths that are relative
2058          ;; to the root of the source host and device
2059    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2060
2061RELATIVE-COMPONENT-DESIGNATOR :=
2062    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2063    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
2064             ;; In output translations, if last component, **/*.*.* is added
2065    PATHNAME | ;; pathname; unless last component, directory is assumed.
2066    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.49-linux-x64
2067    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2068    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2069    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2070    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2071    :*.*.* | ;; any file (since ASDF 2.011.4)
2072    ;; Not supported (anymore): :UID and :USERNAME
2073@end example
2074
2075For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2076which is the default place ASDF looks for this configuration, once contained:
2077@example
2078(:source-registry
2079  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2080  :inherit-configuration)
2081@end example
2082
2083@section Configuration Directories
2084
2085Configuration directories consist in files each containing
2086a list of directives without any enclosing @code{(:source-registry ...)} form.
2087The files will be sorted by namestring as if by @code{string<} and
2088the lists of directives of these files with be concatenated in order.
2089An implicit @code{:inherit-configuration} will be included
2090at the @emph{end} of the list.
2091
2092This allows for packaging software that has file granularity
2093(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2094to easily include configuration information about distributed software.
2095
2096The convention is that, for sorting purposes,
2097the names of files in such a directory begin with two digits
2098that determine the order in which these entries will be read.
2099Also, the type of these files is conventionally @code{"conf"}
2100and as a limitation to some implementations (e.g. GNU clisp),
2101the type cannot be @code{NIL}.
2102
2103Directories may be included by specifying a directory pathname
2104or namestring in an @code{:include} directive, e.g.:
2105
2106@example
2107  (:include "/foo/bar/")
2108@end example
2109
2110Hence, to achieve the same effect as
2111my example @file{~/.config/common-lisp/source-registry.conf} above,
2112I could simply create a file
2113@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2114alone in its directory with the following contents:
2115@example
2116(:tree "/home/fare/cl/")
2117@end example
2118
2119@subsection The :here directive
2120
2121The @code{:here} directive is an absolute pathname designator that
2122refers to the directory containing the configuration file currently
2123being processed.
2124
2125The @code{:here} directive is intended to simplify the delivery of
2126complex CL systems, and for easy configuration of projects shared through
2127revision control systems, in accordance with our design principle that
2128each participant should be able to provide all and only the information
2129available to him or her.
2130
2131Consider a person X who has set up the source code repository for a
2132complex project with a master directory @file{dir/}.  Ordinarily, one
2133might simply have the user add a directive that would look something
2134like this:
2135@example
2136   (:tree  "path/to/dir")
2137@end example
2138But what if X knows that there are very large subtrees
2139under dir that are filled with, e.g., Java source code, image files for
2140icons, etc.?  All of the asdf system definitions are contained in the
2141subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2142these are the only directories that should be searched.
2143
2144In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2145contains the following:
2146@example
2147(:source-registry
2148   (:tree (:here "src/lisp/"))
2149   (:tree (:here "extlib/lisp"))
2150   (:directory (:here "outlier/")))
2151@end example
2152
2153Then when someone else (call her Y) checks out a copy of this
2154repository, she need only add
2155@example
2156(:include "/path/to/my/checkout/directory/asdf.conf")
2157@end example
2158to one of her previously-existing asdf source location configuration
2159files, or invoke @code{initialize-source-registry} with a configuration
2160form containing that s-expression.  ASDF will find the .conf file that X
2161has provided, and then set up source locations within the working
2162directory according to X's (relative) instructions.
2163
2164@section Shell-friendly syntax for configuration
2165
2166When considering environment variable @code{CL_SOURCE_REGISTRY}
2167ASDF will skip to next configuration if it's an empty string.
2168It will @code{READ} the string as a SEXP in the DSL
2169if it begins with a paren @code{(}
2170and it will be interpreted much like @code{TEXINPUTS}
2171list of paths, where
2172
2173  * paths are separated
2174   by a @code{:} (colon) on Unix platforms (including cygwin),
2175   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2176
2177  * each entry is a directory to add to the search path.
2178
2179  * if the entry ends with a double slash @code{//}
2180    then it instead indicates a tree in the subdirectories
2181    of which to recurse.
2182
2183  * if the entry is the empty string (which may only appear once),
2184    then it indicates that the inherited configuration should be
2185    spliced there.
2186
2187
2188@section Search Algorithm
2189@vindex *default-source-registry-exclusions*
2190
2191In case that isn't clear, the semantics of the configuration is that
2192when searching for a system of a given name,
2193directives are processed in order.
2194
2195When looking in a directory, if the system is found, the search succeeds,
2196otherwise it continues.
2197
2198When looking in a tree, if one system is found, the search succeeds.
2199If multiple systems are found, the consequences are unspecified:
2200the search may succeed with any of the found systems,
2201or an error may be raised.
2202ASDF currently returns the first system found,
2203XCVB currently raised an error.
2204If none is found, the search continues.
2205
2206Exclude statements specify patterns of subdirectories
2207the systems from which to ignore.
2208Typically you don't want to use copies of files kept by such
2209version control systems as Darcs.
2210Exclude statements are not propagated to further included or inherited
2211configuration files or expressions;
2212instead the defaults are reset around every configuration statement
2213to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2214
2215Include statements cause the search to recurse with the path specifications
2216from the file specified.
2217
2218An inherit-configuration statement cause the search to recurse with the path
2219specifications from the next configuration
2220(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2221
2222
2223@section Caching Results
2224
2225The implementation is allowed to either eagerly compute the information
2226from the configurations and file system, or to lazily re-compute it
2227every time, or to cache any part of it as it goes.
2228To explicitly flush any information cached by the system, use the API below.
2229
2230
2231@section Configuration API
2232
2233The specified functions are exported from your build system's package.
2234Thus for ASDF the corresponding functions are in package ASDF,
2235and for XCVB the corresponding functions are in package XCVB.
2236
2237@defun initialize-source-registry @&optional PARAMETER
2238   will read the configuration and initialize all internal variables.
2239   You may extend or override configuration
2240   from the environment and configuration files
2241   with the given @var{PARAMETER}, which can be
2242   @code{NIL} (no configuration override),
2243   or a SEXP (in the SEXP DSL),
2244   a string (as in the string DSL),
2245   a pathname (of a file or directory with configuration),
2246   or a symbol (fbound to function that when called returns one of the above).
2247@end defun
2248
2249@defun clear-source-registry
2250   undoes any source registry configuration
2251   and clears any cache for the search algorithm.
2252   You might want to call this function
2253   (or better, @code{clear-configuration})
2254   before you dump an image that would be resumed
2255   with a different configuration,
2256   and return an empty configuration.
2257   Note that this does not include clearing information about
2258   systems defined in the current image, only about
2259   where to look for systems not yet defined.
2260@end defun
2261
2262@defun ensure-source-registry @&optional PARAMETER
2263   checks whether a source registry has been initialized.
2264   If not, initialize it with the given @var{PARAMETER}.
2265@end defun
2266
2267Every time you use ASDF's @code{find-system}, or
2268anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2269@code{ensure-source-registry} is called with parameter NIL,
2270which the first time around causes your configuration to be read.
2271If you change a configuration file,
2272you need to explicitly @code{initialize-source-registry} again,
2273or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2274which will cause the initialization to happen next time around.
2275
2276
2277@section Future
2278
2279If this mechanism is successful, in the future, we may declare
2280@code{asdf:*central-registry*} obsolete and eventually remove it.
2281Any hook into implementation-specific search mechanisms will by then
2282have been integrated in the @code{:default-configuration} which everyone
2283should either explicitly use or implicit inherit. Some shell syntax
2284for it should probably be added somehow.
2285
2286But we're not there yet. For now, let's see how practical this new
2287source-registry is.
2288
2289
2290@section Rejected ideas
2291
2292Alternatives I considered and rejected included:
2293
2294@enumerate
2295@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2296   and somehow the configuration parser expands the new configuration
2297   language into a expanded series of directories of subdirectories to
2298   lookup, pre-recursing through specified hierarchies. This is kludgy,
2299   and leaves little space of future cleanups and extensions.
2300
2301@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2302   in completely new ways, so that new kinds of entries may be implemented
2303   as a recursive search, etc. This seems somewhat backwards.
2304
2305@item Completely remove @code{asdf:*central-registry*}
2306   and break backwards compatibility.
2307   Hopefully this will happen in a few years after everyone migrate to
2308   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2309
2310@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2311   when you dereference it or setf it. Only the new variable with new
2312   semantics is handled by the new search procedure.
2313   Complex and still introduces subtle semantic issues.
2314@end enumerate
2315
2316
2317I've been suggested the below features, but have rejected them,
2318for the sake of keeping ASDF no more complex than strictly necessary.
2319
2320@itemize
2321@item
2322  More syntactic sugar: synonyms for the configuration directives, such as
2323  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2324  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2325
2326@item
2327   The possibility to register individual files instead of directories.
2328
2329@item
2330  Integrate Xach Beane's tilde expander into the parser,
2331  or something similar that is shell-friendly or shell-compatible.
2332  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2333  minimal by removing the need for evaluated entries that ASDF has?
2334  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2335  Hopefully, these are already superseded by the @code{:default-registry}
2336
2337@item
2338  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2339  down a filesystem tree in the environment variable.
2340  It isn't that Lisp friendly either.
2341@end itemize
2342
2343@section TODO
2344
2345@itemize
2346@item Add examples
2347@end itemize
2348
2349
2350@section Credits for the source-registry
2351
2352Thanks a lot to Stelian Ionescu for the initial idea.
2353
2354Thanks to Rommel Martinez for the initial implementation attempt.
2355
2356All bad design ideas and implementation bugs are to mine, not theirs.
2357But so are good design ideas and elegant implementation tricks.
2358
2359 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2360
2361
2362
2363@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2364@comment  node-name,  next,  previous,  up
2365@chapter Controlling where ASDF saves compiled files
2366@cindex asdf-output-translations
2367@vindex ASDF_OUTPUT_TRANSLATIONS
2368
2369Each Common Lisp implementation has its own format
2370for compiled files (fasls for short, short for ``fast loading'').
2371If you use multiple implementations
2372(or multiple versions of the same implementation),
2373you'll soon find your source directories
2374littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2375Worse yet, some implementations use the same file extension
2376while changing formats from version to version (or platform to platform)
2377which means that you'll have to recompile binaries
2378as you switch from one implementation to the next.
2379
2380ASDF 2 includes the @code{asdf-output-translations} facility
2381to mitigate the problem.
2382
2383@section Configurations
2384
2385Configurations specify mappings from input locations to output locations.
2386Once again we rely on the XDG base directory specification for configuration.
2387@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2388
2389@enumerate
2390
2391@item
2392Some hardcoded wrapping output translations configuration may be used.
2393This allows special output translations (or usually, invariant directories)
2394to be specified corresponding to the similar special entries in the source registry.
2395
2396@item
2397An application may explicitly initialize the output-translations
2398configuration using the Configuration API
2399in which case this takes precedence.
2400(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2401It may itself compute this configuration from the command-line,
2402from a script, from its own configuration file, etc.
2403
2404@item
2405The source registry will be configured from
2406the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2407
2408@item
2409The source registry will be configured from
2410user configuration file
2411@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2412(which defaults to
2413@file{~/.config/common-lisp/asdf-output-translations.conf})
2414if it exists.
2415
2416@item
2417The source registry will be configured from
2418user configuration directory
2419@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2420(which defaults to
2421@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2422if it exists.
2423
2424@item
2425The source registry will be configured from
2426system configuration file
2427@file{/etc/common-lisp/asdf-output-translations.conf}
2428if it exists.
2429
2430@item
2431The source registry will be configured from
2432system configuration directory
2433@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2434if it exists.
2435
2436@end enumerate
2437
2438Each of these configurations is specified as a SEXP
2439in a trival domain-specific language (defined below).
2440Additionally, a more shell-friendly syntax is available
2441for the environment variable (defined yet below).
2442
2443Each of these configurations is only used if the previous
2444configuration explicitly or implicitly specifies that it
2445includes its inherited configuration.
2446
2447Note that by default, a per-user cache is used for output files.
2448This allows the seamless use of shared installations of software
2449between several users, and takes files out of the way of the developers
2450when they browse source code,
2451at the expense of taking a small toll when developers have to clean up
2452output files and find they need to get familiar with output-translations first.
2453
2454
2455@section Backward Compatibility
2456@cindex ASDF-BINARY-LOCATIONS compatibility
2457
2458
2459We purposefully do NOT provide backward compatibility with earlier versions of
2460@code{ASDF-Binary-Locations} (8 Sept 2009),
2461@code{common-lisp-controller} (7.0) or
2462@code{cl-launch} (2.35),
2463each of which had similar general capabilities.
2464The previous APIs of these programs were not designed
2465for configuration by the end-user
2466in an easy way with configuration files.
2467Recent versions of same packages use
2468the new @code{asdf-output-translations} API as defined below:
2469@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2470@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2471
2472This incompatibility shouldn't inconvenience many people.
2473Indeed, few people use and customize these packages;
2474these few people are experts who can trivially adapt to the new configuration.
2475Most people are not experts, could not properly configure these features
2476(except inasmuch as the default configuration of
2477@code{common-lisp-controller} and/or @code{cl-launch}
2478might have been doing the right thing for some users),
2479and yet will experience software that ``just works'',
2480as configured by the system distributor, or by default.
2481
2482Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2483we provide a limited emulation mode:
2484
2485@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2486This function will initialize the new @code{asdf-output-translations} facility in a way
2487that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2488Where you would previously set global variables
2489@var{*centralize-lisp-binaries*},
2490@var{*default-toplevel-directory*},
2491@var{*include-per-user-information*},
2492@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2493you will now have to pass the same values as keyword arguments to this function.
2494Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2495will accept any valid pathname designator for @code{asdf-output-translations}
2496instead of just strings and pathnames.
2497@end defun
2498
2499If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2500(the one available as an extension to load of top of ASDF,
2501not the one built into a few old versions of ASDF),
2502but first you must disable @code{asdf-output-translations}
2503with @code{(asdf:disable-output-translations)},
2504or you might experience ``interesting'' issues.
2505
2506Also, note that output translation is enabled by default.
2507To disable it, use @code{(asdf:disable-output-translations)}.
2508
2509
2510@section Configuration DSL
2511
2512Here is the grammar of the SEXP DSL
2513for @code{asdf-output-translations} configuration:
2514
2515@verbatim
2516;; A configuration is single SEXP starting with keyword :source-registry
2517;; followed by a list of directives.
2518CONFIGURATION := (:output-translations DIRECTIVE ...)
2519
2520;; A directive is one of the following:
2521DIRECTIVE :=
2522    ;; INHERITANCE DIRECTIVE:
2523    ;; Your configuration expression MUST contain
2524    ;; exactly one of either of these:
2525    :inherit-configuration | ; splices inherited configuration (often specified last)
2526    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2527
2528    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2529    ;; you want to use new configuration features but have to bootstrap a
2530    ;; the newer required ASDF from an older release that doesn't sport said features:
2531    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2532
2533    ;; include a configuration file or directory
2534    (:include PATHNAME-DESIGNATOR) |
2535
2536    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2537    :enable-user-cache |
2538    ;; Disable global cache. Map / to /
2539    :disable-cache |
2540
2541    ;; add a single directory to be scanned (no recursion)
2542    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2543
2544    ;; use a function to return the translation of a directory designator
2545    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2546
2547DIRECTORY-DESIGNATOR :=
2548    NIL | ;; As source: skip this entry. As destination: same as source
2549    T | ;; as source matches anything, as destination leaves pathname unmapped.
2550    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2551
2552TRANSLATION-FUNCTION :=
2553    SYMBOL | ;; symbol of a function that takes two arguments,
2554             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2555    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2556             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2557
2558@end verbatim
2559
2560Relative components better be either relative
2561or subdirectories of the path before them, or bust.
2562
2563The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2564You can specify more fine-grained patterns
2565by using a pathname object as the last component
2566e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2567
2568You may use @code{#+features} to customize the configuration file.
2569
2570The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2571to anything but themselves (same as if the second designator was the same as the first).
2572
2573When the first designator is @code{t},
2574the mapping always matches.
2575When the first designator starts with @code{:root},
2576the mapping matches any host and device.
2577In either of these cases, if the second designator
2578isn't @code{t} and doesn't start with @code{:root},
2579then strings indicating the host and pathname are somehow copied
2580in the beginning of the directory component of the source pathname
2581before it is translated.
2582
2583When the second designator is @code{t}, the mapping is the identity.
2584When the second designator starts with @code{:root},
2585the mapping preserves the host and device of the original pathname.
2586Notably, this allows you to map files
2587to a subdirectory of the whichever directory the file is in.
2588Though the syntax is not quite as easy to use as we'd like,
2589you can have an (source destination) mapping entry such as follows
2590in your configuration file,
2591or you may use @code{enable-asdf-binary-locations-compatibility}
2592with @code{:centralize-lisp-binaries nil}
2593which will do the same thing internally for you:
2594@verbatim
2595  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2596          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2597     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2598       (:root ,wild-subdir :implementation ,wild-file)))
2599@end verbatim
2600Starting with ASDF 2.011.4, you can use the simpler:
2601  @code{`(:root (:root :**/ :implementation :*.*.*))}
2602
2603
2604
2605@code{:include} statements cause the search to recurse with the path specifications
2606from the file specified.
2607
2608If the @code{translate-pathname} mechanism cannot achieve a desired
2609translation, the user may provide a function which provides the
2610required algorithim.  Such a translation function is specified by
2611supplying a list as the second @code{directory-designator}
2612the first element of which is the keyword @code{:function},
2613and the second element of which is
2614either a symbol which designates a function or a lambda expression.
2615The function designated by the second argument must take two arguments,
2616the first being the pathname of the source file,
2617the second being the wildcard that was matched.
2618The result of the function invocation should be the translated pathname.
2619
2620An @code{:inherit-configuration} statement cause the search to recurse with the path
2621specifications from the next configuration.
2622@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2623
2624@itemize
2625@item
2626@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2627@item
2628@code{:disable-cache} is the same as @code{(t t)}.
2629@item
2630@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2631which by default is the same as using
2632@code{(:home ".cache" "common-lisp" :implementation)}.
2633@item
2634@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2635which by default is the same as using
2636@code{("/var/cache/common-lisp" :uid :implementation-type)}
2637(on Unix and cygwin), or something semi-sensible on Windows.
2638@end itemize
2639
2640
2641@section Configuration Directories
2642
2643Configuration directories consist in files each contains
2644a list of directives without any enclosing
2645@code{(:output-translations ...)} form.
2646The files will be sorted by namestring as if by @code{string<} and
2647the lists of directives of these files with be concatenated in order.
2648An implicit @code{:inherit-configuration} will be included
2649at the @emph{end} of the list.
2650
2651This allows for packaging software that has file granularity
2652(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2653to easily include configuration information about software being distributed.
2654
2655The convention is that, for sorting purposes,
2656the names of files in such a directory begin with two digits
2657that determine the order in which these entries will be read.
2658Also, the type of these files is conventionally @code{"conf"}
2659and as a limitation of some implementations, the type cannot be @code{NIL}.
2660
2661Directories may be included by specifying a directory pathname
2662or namestring in an @code{:include} directive, e.g.:
2663@verbatim
2664  (:include "/foo/bar/")
2665@end verbatim
2666
2667@section Shell-friendly syntax for configuration
2668
2669When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2670ASDF will skip to next configuration if it's an empty string.
2671It will @code{READ} the string as an SEXP in the DSL
2672if it begins with a paren @code{(}
2673and it will be interpreted as a list of directories.
2674Directories should come by pairs, indicating a mapping directive.
2675Entries are separated
2676by a @code{:} (colon) on Unix platforms (including cygwin),
2677by a @code{;} (semicolon) on other platforms (mainly, Windows).
2678
2679The magic empty entry,
2680if it comes in what would otherwise be the first entry in a pair,
2681indicates the splicing of inherited configuration.
2682If it comes as the second entry in a pair,
2683it indicates that the directory specified first is to be left untranslated
2684(which has the same effect as if the directory had been repeated).
2685
2686
2687@section Semantics of Output Translations
2688
2689From the specified configuration,
2690a list of mappings is extracted in a straightforward way:
2691mappings are collected in order, recursing through
2692included or inherited configuration as specified.
2693To this list is prepended some implementation-specific mappings,
2694and is appended a global default.
2695
2696The list is then compiled to a mapping table as follows:
2697for each entry, in order, resolve the first designated directory
2698into an actual directory pathname for source locations.
2699If no mapping was specified yet for that location,
2700resolve the second designated directory to an output location directory
2701add a mapping to the table mapping the source location to the output location,
2702and add another mapping from the output location to itself
2703(unless a mapping already exists for the output location).
2704
2705Based on the table, a mapping function is defined,
2706mapping source pathnames to output pathnames:
2707given a source pathname, locate the longest matching prefix
2708in the source column of the mapping table.
2709Replace that prefix by the corresponding output column
2710in the same row of the table, and return the result.
2711If no match is found, return the source pathname.
2712(A global default mapping the filesystem root to itself
2713may ensure that there will always be a match,
2714with same fall-through semantics).
2715
2716@section Caching Results
2717
2718The implementation is allowed to either eagerly compute the information
2719from the configurations and file system, or to lazily re-compute it
2720every time, or to cache any part of it as it goes.
2721To explicitly flush any information cached by the system, use the API below.
2722
2723
2724@section Output location API
2725
2726The specified functions are exported from package ASDF.
2727
2728@defun initialize-output-translations @&optional PARAMETER
2729   will read the configuration and initialize all internal variables.
2730   You may extend or override configuration
2731   from the environment and configuration files
2732   with the given @var{PARAMETER}, which can be
2733   @code{NIL} (no configuration override),
2734   or a SEXP (in the SEXP DSL),
2735   a string (as in the string DSL),
2736   a pathname (of a file or directory with configuration),
2737   or a symbol (fbound to function that when called returns one of the above).
2738@end defun
2739
2740@defun disable-output-translations
2741   will initialize output translations in a way
2742   that maps every pathname to itself,
2743   effectively disabling the output translation facility.
2744@end defun
2745
2746@defun clear-output-translations
2747   undoes any output translation configuration
2748   and clears any cache for the mapping algorithm.
2749   You might want to call this function
2750   (or better, @code{clear-configuration})
2751   before you dump an image that would be resumed
2752   with a different configuration,
2753   and return an empty configuration.
2754   Note that this does not include clearing information about
2755   systems defined in the current image, only about
2756   where to look for systems not yet defined.
2757@end defun
2758
2759@defun ensure-output-translations @&optional PARAMETER
2760   checks whether output translations have been initialized.
2761   If not, initialize them with the given @var{PARAMETER}.
2762   This function will be called before any attempt to operate on a system.
2763@end defun
2764
2765@defun apply-output-translations PATHNAME
2766   Applies the configured output location translations to @var{PATHNAME}
2767   (calls @code{ensure-output-translations} for the translations).
2768@end defun
2769
2770Every time you use ASDF's @code{output-files}, or
2771anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2772@code{ensure-output-translations} is called with parameter NIL,
2773which the first time around causes your configuration to be read.
2774If you change a configuration file,
2775you need to explicitly @code{initialize-output-translations} again,
2776or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2777which will cause the initialization to happen next time around.
2778
2779
2780@section Credits for output translations
2781
2782Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2783and to Peter van Eynde for @code{Common Lisp Controller}.
2784
2785All bad design ideas and implementation bugs are to mine, not theirs.
2786But so are good design ideas and elegant implementation tricks.
2787
2788 --- Francois-Rene Rideau @email{fare@@tunes.org}
2789
2790@c @section Default locations
2791@c @findex output-files-for-system-and-operation
2792
2793@c The default binary location for each Lisp implementation
2794@c is a subdirectory of each source directory.
2795@c To account for different Lisps, Operating Systems, Implementation versions,
2796@c and so on, ASDF borrows code from SLIME
2797@c to create reasonable custom directory names.
2798@c Here are some examples:
2799
2800@c @itemize
2801@c @item
2802@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2803
2804@c @item
2805@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2806
2807@c @item
2808@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2809@c @end itemize
2810
2811@c By default, all output file pathnames will be relocated
2812@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2813
2814@c See the document @file{README.asdf-output-translations}
2815@c for a full specification on how to configure @code{asdf-output-translations}.
2816
2817@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2818@comment  node-name,  next,  previous,  up
2819@chapter Error handling
2820@findex SYSTEM-DEFINITION-ERROR
2821@findex OPERATION-ERROR
2822
2823@section ASDF errors
2824
2825If ASDF detects an incorrect system definition, it will signal a generalised instance of
2826@code{SYSTEM-DEFINITION-ERROR}.
2827
2828Operations may go wrong (for example when source files contain errors).
2829These are signalled using generalised instances of
2830@code{OPERATION-ERROR}.
2831
2832@section Compilation error and warning handling
2833@vindex *compile-file-warnings-behaviour*
2834@vindex *compile-file-errors-behavior*
2835
2836ASDF checks for warnings and errors when a file is compiled.
2837The variables @var{*compile-file-warnings-behaviour*} and
2838@var{*compile-file-errors-behavior*}
2839control the handling of any such events.
2840The valid values for these variables are
2841@code{:error}, @code{:warn}, and @code{:ignore}.
2842
2843@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2844@comment  node-name,  next,  previous,  up
2845@chapter Miscellaneous additional functionality
2846
2847ASDF includes several additional features that are generally
2848useful for system definition and development.
2849
2850@section Controlling file compilation
2851
2852When declaring a component (system, module, file),
2853you can specify a keyword argument @code{:around-compile function}.
2854If left unspecified,
2855the value will be inherited from the parent component if any,
2856or with a default of @code{nil}
2857if no value is specified in any transitive parent.
2858
2859The argument must be a either @code{nil}, a fbound symbol,
2860a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk) ...)})
2861a function object (e.g. using @code{#.#'} but that's discouraged
2862because it prevents the introspection done by e.g. asdf-dependency-grovel),
2863or a string that when read yields a symbol or a lambda-expression.
2864@code{nil} means the normal compile-file function will be called.
2865A non-nil value designates a function of one argument
2866that will be called with a thunk for calling
2867the compile-file function with proper arguments.
2868
2869Note that by using a string, you may reference
2870a function, symbol and/or package
2871that will only be created later during the build, but
2872isn't yet present at the time the defsystem form is evaluated.
2873However, if your entire system is using such a hook, you may have to
2874explicitly override the hook with @code{nil} for all the modules and files
2875that are compiled before the hook is defined.
2876
2877Using this hook, you may achieve such effects as:
2878locally renaming packages,
2879binding @var{*readtables*} and other syntax-controlling variables,
2880handling warnings and other conditions,
2881proclaiming consistent optimization settings,
2882saving code coverage information,
2883maintaining meta-data about compilation timings,
2884setting gensym counters and PRNG seeds and other sources of non-determinism,
2885overriding the source-location and/or timestamping systems,
2886checking that some compile-time side-effects were properly balanced,
2887etc.
2888
2889
2890@section Miscellaneous Exported Functions
2891
2892@defun coerce-pathname name @&key type defaults
2893
2894This function (available starting with ASDF 2.012.11)
2895takes an argument, and portably interprets it as a pathname.
2896If the argument @var{name} is a pathname or @code{nil}, it is passed through;
2897if it's a symbol, it's interpreted as a string by downcasing it;
2898if it's a string, it is first separated using @code{/} into substrings;
2899the leading substrings denote subdirectories of a relative pathname.
2900If @var{type} is @code{:directory} or the string ends with @code{/},
2901the last substring is also a subdirectory;
2902if @var{type} is a string, it is used as the type of the pathname, and
2903the last substring is the name component of the pathname;
2904if @var{type} is @code{nil}, the last substring specifies both name and type components
2905of the pathname, with the last @code{.} separating them, or only the name component
2906if there's no last @code{.} or if there is only one dot and it's the first character.
2907The host, device and version components come from @var{defaults}, which defaults to
2908@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
2909
2910@end defun
2911
2912@defun merge-pathnames* @&key specified defaults
2913
2914This function is a replacement for @code{merge-pathnames} that uses the host and device
2915from the @var{defaults} rather than the @var{specified} pathname when the latter
2916is a relative pathname. This allows ASDF and its users to create and use relative pathnames
2917without having to know beforehand what are the host and device
2918of the absolute pathnames they are relative to.
2919
2920@end defun
2921
2922@defun system-relative-pathname system name @&key type
2923
2924It's often handy to locate a file relative to some system.
2925The @code{system-relative-pathname} function meets this need.
2926
2927It takes two mandatory arguments @var{system} and @var{name}
2928and a keyword argument @var{type}:
2929@var{system} is name of a system, whereas @var{name} and optionally @var{type}
2930specify a relative pathname, interpreted like a component pathname specifier
2931by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
2932
2933It returns a pathname built from the location of the system's
2934source directory and the relative pathname. For example:
2935
2936@lisp
2937> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
2938#P"/repository/other/cl-ppcre/regex.data"
2939@end lisp
2940
2941@end defun
2942
2943@defun system-source-directory system-designator
2944
2945ASDF does not provide a turnkey solution for locating
2946data (or other miscellaneous) files
2947that are distributed together with the source code of a system.
2948Programmers can use @code{system-source-directory} to find such files.
2949Returns a pathname object.
2950The @var{system-designator} may be a string, symbol, or ASDF system object.
2951@end defun
2952
2953@defun clear-system system-designator
2954
2955It is sometimes useful to force recompilation of a previously loaded system.
2956In these cases, it may be useful to @code{(asdf:clear-system :foo)}
2957to remove the system from the table of currently loaded systems;
2958the next time the system @code{foo} or one that depends on it is re-loaded,
2959@code{foo} will then be loaded again.
2960Alternatively, you could touch @code{foo.asd} or
2961remove the corresponding fasls from the output file cache.
2962(It was once conceived that one should provide
2963a list of systems the recompilation of which to force
2964as the @code{:force} keyword argument to @code{load-system};
2965but this has never worked, and though the feature was fixed in ASDF 2.000,
2966it remains @code{cerror}'ed out as nobody ever used it.)
2967
2968Note that this does not and cannot by itself undo the previous loading
2969of the system. Common Lisp has no provision for such an operation,
2970and its reliance on irreversible side-effects to global datastructures
2971makes such a thing impossible in the general case.
2972If the software being re-loaded is not conceived with hot upgrade in mind,
2973this re-loading may cause many errors, warnings or subtle silent problems,
2974as packages, generic function signatures, structures, types, macros, constants, etc.
2975are being redefined incompatibly.
2976It is up to the user to make sure that reloading is possible and has the desired effect.
2977In some cases, extreme measures such as recursively deleting packages,
2978unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
2979and much more are necessary for reloading to happen smoothly.
2980ASDF itself goes through notable pains to make such a hot upgrade possible
2981with respect to its own code, and what it does is ridiculously complex;
2982look at the beginning of @file{asdf.lisp} to see what it does.
2983@end defun
2984
2985@defun run-shell-command
2986
2987This function is obsolete and present only for the sake of backwards-compatibility:
2988``If it's not backwards, it's not compatible''. We strongly discourage its use.
2989Its current behavior is only well-defined on Unix platforms
2990(which includes MacOS X and cygwin). On Windows, anything goes.
2991
2992Instead we recommend the use of such a function as
2993@code{xcvb-driver:run-program/process-output-stream}
2994from the @code{xcvb-driver} system that is distributed with XCVB:
2995@url{http://common-lisp.net/project/xcvb}.
2996It's only alternative that supports
2997as many implementations and operating systems as ASDF does,
2998and provides well-defined behavior outside Unix (i.e. on Windows).
2999(The only unsupported exception is Genera, since on it
3000@code{run-shell-command} doesn't make sense anyway on that platform).
3001
3002This function takes as arguments a @code{format} control-string
3003and arguments to be passed to @code{format} after this control-string
3004to produce a string.
3005This string is a command that will be evaluated with a POSIX shell if possible;
3006yet, on Windows, some implementations will use CMD.EXE,
3007while others (like SBCL) will make an attempt at invoking a POSIX shell
3008(and fail if it is not present).
3009@end defun
3010
3011
3012@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
3013@comment  node-name,  next,  previous,  up
3014@chapter Getting the latest version
3015
3016Decide which version you want.
3017The @code{master} branch is where development happens;
3018its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
3019but an occasional regression may happen despite our (limited) test suite.
3020
3021The @code{release} branch is what cautious people should be using;
3022it has usually been tested more, and releases are cut at a point
3023where there isn't any known unresolved issue.
3024
3025You may get the ASDF source repository using git:
3026@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
3027
3028You will find the above referenced tags in this repository.
3029You can also browse the repository on
3030@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
3031
3032Discussion of ASDF development is conducted on the
3033mailing list
3034@kbd{asdf-devel@@common-lisp.net}.
3035@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
3036
3037
3038@node FAQ, TODO list, Getting the latest version, Top
3039@comment  node-name,  next,  previous,  up
3040@chapter FAQ
3041
3042@section  ``Where do I report a bug?''
3043
3044ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
3045
3046If you're unsure about whether something is a bug, or for general discussion,
3047use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
3048
3049
3050@section ``What has changed between ASDF 1 and ASDF 2?''
3051
3052@subsection What are ASDF 1 and ASDF 2?
3053
3054On May 31st 2010, we have released ASDF 2.
3055ASDF 2 refers to release 2.000 and later.
3056(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
3057ASDF 1 to any release earlier than 1.369 or so.
3058If your ASDF doesn't sport a version, it's an old ASDF 1.
3059
3060ASDF 2 and its release candidates push
3061@code{:asdf2} onto @code{*features*} so that if you are writing
3062ASDF-dependent code you may check for this feature
3063to see if the new API is present.
3064@emph{All} versions of ASDF should have the @code{:asdf} feature.
3065
3066If you are experiencing problems or limitations of any sort with ASDF 1,
3067we recommend that you should upgrade to ASDF 2,
3068or whatever is the latest release.
3069
3070
3071@subsection ASDF can portably name files in subdirectories
3072
3073Common Lisp namestrings are not portable,
3074except maybe for logical pathnamestrings,
3075that themselves have various limitations and require a lot of setup
3076that is itself ultimately non-portable.
3077
3078In ASDF 1, the only portable ways to refer to pathnames inside systems and components
3079were very awkward, using @code{#.(make-pathname ...)} and
3080@code{#.(merge-pathnames ...)}.
3081Even the above were themselves were inadequate in the general case
3082due to host and device issues, unless horribly complex patterns were used.
3083Plenty of simple cases that looked portable actually weren't,
3084leading to much confusion and greavance.
3085
3086ASDF 2 implements its own portable syntax for strings as pathname specifiers.
3087Naming files within a system definition becomes easy and portable again.
3088@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
3089@code{merge-pathnames*},
3090@code{coerce-pathname}.
3091
3092On the other hand, there are places where systems used to accept namestrings
3093where you must now use an explicit pathname object:
3094@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3095must now be written with the @code{#p} syntax:
3096@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3097
3098@xref{The defsystem grammar,,Pathname specifiers}.
3099
3100
3101@subsection Output translations
3102
3103A popular feature added to ASDF was output pathname translation:
3104@code{asdf-binary-locations}, @code{common-lisp-controller},
3105@code{cl-launch} and other hacks were all implementing it in ways
3106both mutually incompatible and difficult to configure.
3107
3108Output pathname translation is essential to share
3109source directories of portable systems across multiple implementations
3110or variants thereof,
3111or source directories of shared installations of systems across multiple users,
3112or combinations of the above.
3113
3114In ASDF 2, a standard mechanism is provided for that,
3115@code{asdf-output-translations},
3116with sensible defaults, adequate configuration languages,
3117a coherent set of configuration files and hooks,
3118and support for non-Unix platforms.
3119
3120@xref{Controlling where ASDF saves compiled files}.
3121
3122@subsection Source Registry Configuration
3123
3124Configuring ASDF used to require special magic
3125to be applied just at the right moment,
3126between the moment ASDF is loaded and the moment it is used,
3127in a way that is specific to the user,
3128the implementation he is using and the application he is building.
3129
3130This made for awkward configuration files and startup scripts
3131that could not be shared between users, managed by administrators
3132or packaged by distributions.
3133
3134ASDF 2 provides a well-documented way to configure ASDF,
3135with sensible defaults, adequate configuration languages,
3136and a coherent set of configuration files and hooks.
3137
3138We believe it's a vast improvement because it decouples
3139application distribution from library distribution.
3140The application writer can avoid thinking where the libraries are,
3141and the library distributor (dpkg, clbuild, advanced user, etc.)
3142can configure them once and for every application.
3143Yet settings can be easily overridden where needed,
3144so whoever needs control has exactly as much as required.
3145
3146At the same time, ASDF 2 remains compatible
3147with the old magic you may have in your build scripts
3148(using @code{*central-registry*} and
3149@code{*system-definition-search-functions*})
3150to tailor the ASDF configuration to your build automation needs,
3151and also allows for new magic, simpler and more powerful magic.
3152
3153@xref{Controlling where ASDF searches for systems}.
3154
3155
3156@subsection Usual operations are made easier to the user
3157
3158In ASDF 1, you had to use the awkward syntax
3159@code{(asdf:oos 'asdf:load-op :foo)}
3160to load a system,
3161and similarly for @code{compile-op}, @code{test-op}.
3162
3163In ASDF 2, you can use shortcuts for the usual operations:
3164@code{(asdf:load-system :foo)}, and
3165similarly for @code{compile-system}, @code{test-system}.
3166
3167
3168@subsection Many bugs have been fixed
3169
3170The following issues and many others have been fixed:
3171
3172@itemize
3173@item
3174The infamous TRAVERSE function has been revamped significantly,
3175with many bugs squashed.
3176In particular, dependencies were not correctly propagated
3177across submodules within a system but now are.
3178The :version and :feature features and
3179the :force (system1 .. systemN) feature have been fixed.
3180
3181@item
3182Performance has been notably improved for large systems
3183(say with thousands of components) by using
3184hash-tables instead of linear search,
3185and linear-time list accumulation
3186instead of quadratic-time recursive appends.
3187
3188@item
3189Many features used to not be portable,
3190especially where pathnames were involved.
3191Windows support was notably quirky because of such non-portability.
3192
3193@item
3194The internal test suite used to massively fail on many implementations.
3195While still incomplete, it now fully passes
3196on all implementations supported by the test suite,
3197except for GCL (due to GCL bugs).
3198
3199@item
3200Support was lacking for some implementations.
3201ABCL and GCL were notably wholly broken.
3202ECL extensions were not integrated in the ASDF release.
3203
3204@item
3205The documentation was grossly out of date.
3206
3207@end itemize
3208
3209
3210@subsection ASDF itself is versioned
3211
3212Between new features, old bugs fixed, and new bugs introduced,
3213there were various releases of ASDF in the wild,
3214and no simple way to check which release had which feature set.
3215People using or writing systems had to either make worst-case assumptions
3216as to what features were available and worked,
3217or take great pains to have the correct version of ASDF installed.
3218
3219With ASDF 2, we provide a new stable set of working features
3220that everyone can rely on from now on.
3221Use @code{#+asdf2} to detect presence of ASDF 2,
3222@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3223to check the availability of a version no earlier than required.
3224
3225
3226@subsection ASDF can be upgraded
3227
3228When an old version of ASDF was loaded,
3229it was very hard to upgrade ASDF in your current image
3230without breaking everything.
3231Instead you had to exit the Lisp process and
3232somehow arrange to start a new one from a simpler image.
3233Something that can't be done from within Lisp,
3234making automation of it difficult,
3235which compounded with difficulty in configuration,
3236made the task quite hard.
3237Yet as we saw before, the task would have been required
3238to not have to live with the worst case or non-portable
3239subset of ASDF features.
3240
3241With ASDF 2, it is easy to upgrade
3242from ASDF 2 to later versions from within Lisp,
3243and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3244We support hot upgrade of ASDF and any breakage is a bug
3245that we will do our best to fix.
3246There are still limitations on upgrade, though,
3247most notably the fact that after you upgrade ASDF,
3248you must also reload or upgrade all ASDF extensions.
3249
3250@subsection Decoupled release cycle
3251
3252When vendors were releasing their Lisp implementations with ASDF,
3253they had to basically never change version
3254because neither upgrade nor downgrade was possible
3255without breaking something for someone,
3256and no obvious upgrade path was visible and recommendable.
3257
3258With ASDF 2, upgrade is possible, easy and can be recommended.
3259This means that vendors can safely ship a recent version of ASDF,
3260confident that if a user isn't fully satisfied,
3261he can easily upgrade ASDF and deal
3262with a supported recent version of it.
3263This means that release cycles will be causally decoupled,
3264the practical consequence of which will mean faster convergence
3265towards the latest version for everyone.
3266
3267
3268@subsection Pitfalls of the transition to ASDF 2
3269
3270The main pitfalls in upgrading to ASDF 2 seem to be related
3271to the output translation mechanism.
3272
3273@itemize
3274
3275@item
3276Output translations is enabled by default. This may surprise some users,
3277most of them in pleasant way (we hope), a few of them in an unpleasant way.
3278It is trivial to disable output translations.
3279@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3280
3281@item
3282Some systems in the large have been known not to play well with output translations.
3283They were relatively easy to fix.
3284Once again, it is also easy to disable output translations,
3285or to override its configuration.
3286
3287@item
3288The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3289They replace A-B-L, and there is compatibility mode to emulate
3290your previous A-B-L configuration.
3291See @code{enable-asdf-binary-locations-compatibility} in
3292@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3293But thou shall not load ABL on top of ASDF 2.
3294
3295@end itemize
3296
3297Other issues include the following:
3298
3299@itemize
3300
3301@item
3302ASDF pathname designators are now specified in places where they were unspecified,
3303and a few small adjustments have to be made to some non-portable defsystems.
3304Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
3305a logical pathname (or implementation-dependent hierarchical pathname)
3306must now be specified with @code{#p} syntax
3307where the namestring might have previously sufficed;
3308moreover when evaluation is desired @code{#.} must be used,
3309where it wasn't necessary in the toplevel @code{:pathname} argument.
3310
3311@item
3312There is a slight performance bug, notably on SBCL,
3313when initially searching for @file{asd} files,
3314the implicit @code{(directory "/configured/path/**/*.asd")}
3315for every configured path @code{(:tree "/configured/path/")}
3316in your @code{source-registry} configuration can cause a slight pause.
3317Try to @code{(time (asdf:initialize-source-registry))}
3318to see how bad it is or isn't on your system.
3319If you insist on not having this pause,
3320you can avoid the pause by overriding the default source-registry configuration
3321and not use any deep @code{:tree} entry but only @code{:directory} entries
3322or shallow @code{:tree} entries.
3323Or you can fix your implementation to not be quite that slow
3324when recursing through directories.
3325@emph{Update}: performance bug fixed the hard way in 2.010.
3326
3327@item
3328On Windows, only LispWorks supports proper default configuration pathnames
3329based on the Windows registry.
3330Other implementations make do with environment variables,
3331that you may have to define yourself if you're using an older version of Windows.
3332Windows support is somewhat less tested than Unix support.
3333Please help report and fix bugs.
3334
3335@item
3336The mechanism by which one customizes a system so that Lisp files
3337may use a different extension from the default @file{.lisp} has changed.
3338Previously, the pathname for a component was lazily computed when operating on a system,
3339and you would
3340@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3341  (declare (ignorable component system)) "lis")}.
3342Now, the pathname for a component is eagerly computed when defining the system,
3343and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3344and use @code{:default-component-class cl-source-file.lis} as argument to @code{defsystem},
3345as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3346
3347@findex source-file-type
3348
3349
3350@end itemize
3351
3352
3353@section Issues with installing the proper version of ASDF
3354
3355@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3356
3357We recommend you upgrade ASDF.
3358@xref{Loading ASDF,,Upgrading ASDF}.
3359
3360If this does not work, it is a bug, and you should report it.
3361@xref{FAQ, report-bugs, Where do I report a bug}.
3362In the meantime, you can load @file{asdf.lisp} directly.
3363@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3364
3365
3366@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3367
3368Starting with current candidate releases of ASDF 2,
3369it should always be a good time to upgrade to a recent ASDF.
3370You may consult with the maintainer for which specific version they recommend,
3371but the latest @code{release} should be correct.
3372We trust you to thoroughly test it with your implementation before you release it.
3373If there are any issues with the current release,
3374it's a bug that you should report upstream and that we will fix ASAP.
3375
3376As to how to include ASDF, we recommend the following:
3377
3378@itemize
3379@item
3380If ASDF isn't loaded yet, then @code{(require "asdf")}
3381should load the version of ASDF that is bundled with your system.
3382You may have it load some other version configured by the user,
3383if you allow such configuration.
3384
3385@item
3386If your system provides a mechanism to hook into @code{CL:REQUIRE},
3387then it would be nice to add ASDF to this hook the same way that
3388ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3389
3390@item
3391You may, like SBCL, have ASDF be implicitly used to require systems
3392that are bundled with your Lisp distribution.
3393If you do have a few magic systems that come with your implementation
3394in a precompiled way such that one should only use the binary version
3395that goes with your distribution, like SBCL does,
3396then you should add them in the beginning of @code{wrapping-source-registry}.
3397
3398@item
3399If you have magic systems as above, like SBCL does,
3400then we explicitly ask you to @emph{NOT} distribute
3401@file{asdf.asd} as part of those magic systems.
3402You should still include the file @file{asdf.lisp} in your source distribution
3403and precompile it in your binary distribution,
3404but @file{asdf.asd} if included at all,
3405should be secluded from the magic systems,
3406in a separate file hierarchy;
3407alternatively, you may provide the system
3408after renaming it and its @file{.asd} file to e.g.
3409@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3410@code{sb-asdf} and @file{sb-asdf.asd}.
3411Indeed, if you made @file{asdf.asd} a magic system,
3412then users would no longer be able to upgrade ASDF using ASDF itself
3413to some version of their preference that
3414they maintain independently from your Lisp distribution.
3415
3416@item
3417If you do not have any such magic systems, or have other non-magic systems
3418that you want to bundle with your implementation,
3419then you may add them to the @code{default-source-registry},
3420and you are welcome to include @file{asdf.asd} amongst them.
3421
3422@item
3423Please send us upstream any patches you make to ASDF itself,
3424so we can merge them back in for the benefit of your users
3425when they upgrade to the upstream version.
3426
3427@end itemize
3428
3429
3430
3431@section Issues with configuring ASDF
3432
3433@subsection ``How can I customize where fasl files are stored?''
3434
3435@xref{Controlling where ASDF saves compiled files}.
3436
3437Note that in the past there was an add-on to ASDF called
3438@code{ASDF-binary-locations}, developed by Gary King.
3439That add-on has been merged into ASDF proper,
3440then superseded by the @code{asdf-output-translations} facility.
3441
3442Note that use of @code{asdf-output-translations}
3443can interfere with one aspect of your systems
3444--- if your system uses @code{*load-truename*} to find files
3445(e.g., if you have some data files stored with your program),
3446then the relocation that this ASDF customization performs
3447is likely to interfere.
3448Use @code{asdf:system-relative-pathname} to locate a file
3449in the source directory of some system, and
3450use @code{asdf:apply-output-translations} to locate a file
3451whose pathname has been translated by the facility.
3452
3453@subsection ``How can I wholly disable the compiler output cache?''
3454
3455To permanently disable the compiler output cache
3456for all future runs of ASDF, you can:
3457
3458@example
3459mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3460echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3461@end example
3462
3463This assumes that you didn't otherwise configure the ASDF files
3464(if you did, edit them again),
3465and don't somehow override the configuration at runtime
3466with a shell variable (see below) or some other runtime command
3467(e.g. some call to @code{asdf:initialize-output-translations}).
3468
3469To disable the compiler output cache in Lisp processes
3470run by your current shell, try (assuming @code{bash} or @code{zsh})
3471(on Unix and cygwin only):
3472
3473@example
3474export ASDF_OUTPUT_TRANSLATIONS=/:
3475@end example
3476
3477To disable the compiler output cache just in the current Lisp process,
3478use (after loading ASDF but before using it):
3479
3480@example
3481(asdf:disable-output-translations)
3482@end example
3483
3484@section Issues with using and extending ASDF to define systems
3485
3486@subsection ``How can I cater for unit-testing in my system?''
3487
3488ASDF provides a predefined test operation, @code{test-op}.
3489@xref{Predefined operations of ASDF, test-op}.
3490The test operation, however, is largely left to the system definer to specify.
3491@code{test-op} has been
3492a topic of considerable discussion on the
3493@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3494and on the
3495@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3496
3497Here are some guidelines:
3498
3499@itemize
3500@item
3501For a given system, @var{foo}, you will want to define a corresponding
3502test system, such as @var{foo-test}.  The reason that you will want this
3503separate system is that ASDF does not out of the box supply components
3504that are conditionally loaded.  So if you want to have source files
3505(with the test definitions) that will not be loaded except when testing,
3506they should be put elsewhere.
3507
3508@item
3509The @var{foo-test} system can be defined in an asd file of its own or
3510together with @var{foo}.  An aesthetic preference against cluttering up
3511the filesystem with extra asd files should be balanced against the
3512question of whether one might want to directly load @var{foo-test}.
3513Typically one would not want to do this except in early stages of
3514debugging.
3515
3516@item
3517Record that testing is implemented by @var{foo-test}.  For example:
3518@example
3519(defsystem @var{foo}
3520   :in-order-to ((test-op (test-op @var{foo-test})))
3521   ....)
3522
3523(defsystem @var{foo-test}
3524   :depends-on (@var{foo} @var{my-test-library} ...)
3525   ....)
3526@end example
3527@end itemize
3528
3529This procedure will allow you to support users who do not wish to
3530install your test framework.
3531
3532One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3533does not return a value.  So in current versions of ASDF there is no
3534reliable programmatic means of determining whether or not a set of tests
3535has passed, or which tests have failed.  The user must simply read the
3536console output.  This limitation has been the subject of much
3537discussion.
3538
3539@subsection ``How can I cater for documentation generation in my system?''
3540
3541The ASDF developers are currently working to add a @code{doc-op}
3542to the set of predefined ASDF operations.
3543@xref{Predefined operations of ASDF}.
3544See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3545
3546
3547
3548@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3549
3550See @code{cffi}'s @code{cffi-grovel}.
3551
3552@anchor{report-bugs}
3553
3554
3555@subsection ``I want to put my module's files at the top level.  How do I do this?''
3556
3557By default, the files contained in an asdf module go
3558in a subdirectory with the same name as the module.
3559However, this can be overridden by adding a @code{:pathname ""} argument
3560to the module description.
3561For example, here is how it could be done
3562in the spatial-trees ASDF system definition for ASDF 2:
3563
3564@example
3565(asdf:defsystem :spatial-trees
3566  :components
3567  ((:module base
3568            :pathname ""
3569            :components
3570            ((:file "package")
3571             (:file "basedefs" :depends-on ("package"))
3572             (:file "rectangles" :depends-on ("package"))))
3573   (:module tree-impls
3574            :depends-on (base)
3575            :pathname ""
3576            :components
3577            ((:file "r-trees")
3578             (:file "greene-trees" :depends-on ("r-trees"))
3579             (:file "rstar-trees" :depends-on ("r-trees"))
3580             (:file "rplus-trees" :depends-on ("r-trees"))
3581             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3582   (:module viz
3583            :depends-on (base)
3584            :pathname ""
3585            :components
3586            ((:static-file "spatial-tree-viz.lisp")))
3587   (:module tests
3588            :depends-on (base)
3589            :pathname ""
3590            :components
3591            ((:static-file "spatial-tree-test.lisp")))
3592   (:static-file "LICENCE")
3593   (:static-file "TODO")))
3594@end example
3595
3596All of the files in the @code{tree-impls} module are at the top level,
3597instead of in a @file{tree-impls/} subdirectory.
3598
3599Note that the argument to @code{:pathname} can be either a pathname object or a string.
3600A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3601but this is discouraged because the results of parsing a namestring are not portable.
3602A pathname can only be portably constructed with such syntax as
3603@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3604and similarly the current directory can only be portably specified as
3605@code{#.(make-pathname :directory '(:relative))}.
3606However, as of ASDF 2, you can portably use a string to denote a pathname.
3607The string will be parsed as a @code{/}-separated path from the current directory,
3608such that the empty string @code{""} denotes the current directory, and
3609@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3610portably denotes the same subdirectory as above.
3611When files are specified, the last @code{/}-separated component is interpreted
3612either as the name component of a pathname
3613(if the component class specifies a pathname type),
3614or as a name component plus optional dot-separated type component
3615(if the component class doesn't specifies a pathname type).
3616
3617@subsection How do I create a system definition where all the source files have a .cl extension?
3618
3619Starting with ASDF 2.014.14, you may just pass
3620the builtin class @code{cl-source-file.cl} as
3621the @code{:default-component-class} argument to @code{defsystem}:
3622
3623@lisp
3624(defsystem my-cl-system
3625  :default-component-class cl-source-file.cl
3626  ...)
3627@end lisp
3628
3629Another builtin class @code{cl-source-file.lsp} is offered
3630for files ending in @file{.lsp}.
3631
3632If you want to use a different extension
3633for which ASDF doesn't provide builtin support,
3634or want to support versions of ASDF
3635earlier than 2.014.14 (but later than 2.000),
3636you can define a class as follows:
3637
3638@lisp
3639;; Prologue: make sure we're using a sane package.
3640(defpackage :my-asdf-extension
3641   (:use :asdf :common-lisp)
3642   (:export #:cl-source-file.lis))
3643(in-package :my-asdf-extension)
3644
3645(defclass cl-source-file.lis (cl-source-file)
3646   ((type :initform "lis")))
3647@end lisp
3648
3649Then you can use it as follows:
3650@lisp
3651(defsystem my-cl-system
3652  :default-component-class my-asdf-extension:cl-source-file.lis
3653  ...)
3654@end lisp
3655
3656Of course, if you're in the same package, e.g. in the same file,
3657you won't need to use the package qualifier before @code{cl-source-file.lis}.
3658Actually, if all you're doing is defining this class
3659and using it in the same file without other fancy definitions,
3660you might skip package complications:
3661
3662@lisp
3663(in-package :asdf)
3664(defclass cl-source-file.lis (cl-source-file)
3665   ((type :initform "lis")))
3666(defsystem my-cl-system
3667  :default-component-class cl-source-file.lis
3668  ...)
3669@end lisp
3670
3671It is possible to achieve the same effect
3672in a way that supports both ASDF 1 and ASDF 2,
3673but really, friends don't let friends use ASDF 1.
3674Please upgrade to ASDF 2.
3675In short, though: do same as above, but
3676@emph{before} you use the class in a @code{defsystem},
3677you also define the following method:
3678
3679@lisp
3680(defmethod source-file-type ((f cl-source-file.lis) (m module))
3681  (declare (ignorable f m))
3682  "lis")
3683@end lisp
3684
3685@comment FIXME: Add a FAQ about how to use a new system class...
3686
3687
3688@node  TODO list, Inspiration, FAQ, Top
3689@comment  node-name,  next,  previous,  up
3690@chapter TODO list
3691
3692Here is an old list of things to do,
3693in addition to the bugs that are now tracked on launchpad:
3694@url{https://launchpad.net/asdf}.
3695
3696@section Outstanding spec questions, things to add
3697
3698** packaging systems
3699
3700*** manual page component?
3701
3702** style guide for .asd files
3703
3704You should either use keywords or be careful
3705with the package that you evaluate defsystem forms in.
3706Otherwise @code{(defsystem partition ...)}
3707being read in the @code{cl-user} package
3708will intern a @code{cl-user:partition} symbol,
3709which will then collide with the @code{partition:partition} symbol.
3710
3711Actually there's a hairier packages problem to think about too.
3712@code{in-order-to} is not a keyword:
3713if you read @code{defsystem} forms in a package that doesn't use ASDF,
3714odd things might happen.
3715
3716
3717** extending defsystem with new options
3718
3719You might not want to write a whole parser,
3720but just to add options to the existing syntax.
3721Reinstate @code{parse-option} or something akin.
3722
3723
3724** document all the error classes
3725
3726** what to do with compile-file failure
3727
3728Should check the primary return value from compile-file and see if
3729that gets us any closer to a sensible error handling strategy
3730
3731** foreign files
3732
3733lift unix-dso stuff from db-sockets
3734
3735** Diagnostics
3736
3737A ``dry run'' of an operation can be made with the following form:
3738
3739@lisp
3740(traverse (make-instance '<operation-name>)
3741          (find-system <system-name>)
3742          'explain)
3743@end lisp
3744
3745This uses unexported symbols.
3746What would be a nice interface for this functionality?
3747
3748@section Missing bits in implementation
3749
3750** reuse the same scratch package whenever a system is reloaded from disk
3751
3752** proclamations probably aren't
3753
3754** when a system is reloaded with fewer components than it previously had, odd things happen
3755
3756We should do something inventive when processing a @code{defsystem} form,
3757like take the list of kids and @code{setf} the slot to @code{nil},
3758then transfer children from old to new list as they're found.
3759
3760** (stuff that might happen later)
3761
3762*** Propagation of the @code{:force} option.
3763
3764``I notice that
3765
3766        @code{(asdf:compile-system :araneida :force t)}
3767
3768also forces compilation of every other system the @code{:araneida} system depends on.
3769This is rarely useful to me;
3770usually, when I want to force recompilation of something more than a single source file,
3771I want to recompile only one system.
3772So it would be more useful to have @code{make-sub-operation}
3773refuse to propagate @code{:force t} to other systems, and
3774propagate only something like @code{:force :recursively}.
3775
3776Ideally what we actually want is some kind of criterion that says
3777to which systems (and which operations) a @code{:force} switch will propagate.
3778
3779The problem is perhaps that ``force'' is a pretty meaningless concept.
3780How obvious is it that @code{load :force t} should force @emph{compilation}?
3781But we don't really have the right dependency setup
3782for the user to compile @code{:force t} and expect it to work
3783(files will not be loaded after compilation, so the compile
3784environment for subsequent files will be emptier than it needs to be)
3785
3786What does the user actually want to do when he forces?
3787Usually, for me, update for use with a new version of the Lisp compiler.
3788Perhaps for recovery when he suspects that something has gone wrong.
3789Or else when he's changed compilation options or configuration
3790in some way that's not reflected in the dependency graph.
3791
3792Other possible interface: have a ``revert'' function akin to @code{make clean}.
3793
3794@lisp
3795(asdf:revert 'asdf:compile-op 'araneida)
3796@end lisp
3797
3798would delete any files produced by @code{(compile-system :araneida)}.
3799Of course, it wouldn't be able to do much about stuff in the image itself.
3800
3801How would this work?
3802
3803@code{traverse}
3804
3805There's a difference between a module's dependencies (peers)
3806and its components (children).
3807Perhaps there's a similar difference in operations?
3808For example, @code{(load "use") depends-on (load "macros")} is a peer,
3809whereas @code{(load "use") depends-on (compile "use")}
3810is more of a ``subservient'' relationship.
3811
3812@node  Inspiration, Concept Index, TODO list, Top
3813@comment  node-name,  next,  previous,  up
3814@chapter Inspiration
3815
3816@section mk-defsystem (defsystem-3.x)
3817
3818We aim to solve basically the same problems as @code{mk-defsystem} does.
3819However, our architecture for extensibility
3820better exploits CL language features (and is documented),
3821and we intend to be portable rather than just widely-ported.
3822No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3823that implementation has the unenviable task
3824of supporting pre-ANSI implementations, which is no longer necessary.
3825
3826The surface defsystem syntax of asdf is more-or-less compatible with
3827@code{mk-defsystem}, except that we do not support
3828the @code{source-foo} and @code{binary-foo} prefixes
3829for separating source and binary files, and
3830we advise the removal of all options to specify pathnames.
3831
3832The @code{mk-defsystem} code for topologically sorting
3833a module's dependency list was very useful.
3834
3835@section defsystem-4 proposal
3836
3837Marco and Peter's proposal for defsystem 4 served as the driver for
3838many of the features in here.  Notable differences are:
3839
3840@itemize
3841@item
3842We don't specify output files or output file extensions
3843as part of the system.
3844
3845If you want to find out what files an operation would create,
3846ask the operation.
3847
3848@item
3849We don't deal with CL packages
3850
3851If you want to compile in a particular package, use an @code{in-package} form
3852in that file (ilisp / SLIME will like you more if you do this anyway)
3853
3854@item
3855There is no proposal here that @code{defsystem} does version control.
3856
3857A system has a given version which can be used to check dependencies,
3858but that's all.
3859@end itemize
3860
3861The defsystem 4 proposal tends to look more at the external features,
3862whereas this one centres on a protocol for system introspection.
3863
3864@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
3865
3866Available in updated-for-CL form on the web at
3867@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3868
3869In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3870and concept to deal with creating component trees
3871from @code{defsystem} surface syntax.
3872[ this is not true right now, though it used to be and
3873probably will be again soon ]
3874
3875
3876@c -------------------
3877
3878
3879@node Concept Index, Function and Class Index, Inspiration, Top
3880@unnumbered Concept Index
3881
3882@printindex cp
3883
3884@node Function and Class Index, Variable Index, Concept Index, Top
3885@unnumbered Function and Class Index
3886
3887@printindex fn
3888
3889@node Variable Index,  , Function and Class Index, Top
3890@unnumbered Variable Index
3891
3892@printindex vr
3893
3894@bye
Note: See TracBrowser for help on using the repository browser.