source: trunk/abcl/doc/asdf/asdf.texinfo @ 14277

Last change on this file since 14277 was 14277, checked in by Mark Evenson, 8 years ago

Sync to asdf-2.26.6.

Fixes #271.

File size: 153.5 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2012 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2012 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2012 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex require-system
201@findex compile-system
202@findex test-system
203@cindex system directory designator
204@findex operate
205@findex oos
206
207@c @menu
208@c * Installing ASDF::
209@c @end menu
210
211
212@section Loading a pre-installed ASDF
213
214Many Lisp implementations include a copy of ASDF.
215You can usually load this copy using Common Lisp's @code{require} function:
216
217@lisp
218(require "asdf")
219@end lisp
220
221As of the writing of this manual,
222the following implementations provide ASDF 2 this way:
223abcl allegro ccl clisp cmucl ecl lispworks mkcl sbcl xcl.
224The following implementation doesn't provide it yet but will in a future release:
225scl.
226The following implementations are obsolete, not actively maintained,
227and most probably will never bundle it:
228cormanlisp gcl genera mcl.
229
230If the implementation you are using doesn't provide ASDF 2,
231see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
232If that implementation is still actively maintained,
233you may also send a bug report to your Lisp vendor and complain
234about their failing to provide ASDF.
235
236NB: all implementations except clisp also accept
237@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
238For portability's sake, you probably want to use @code{(require "asdf")}.
239
240
241@section Checking whether ASDF is loaded
242
243To check whether ASDF is properly loaded in your current Lisp image,
244you can run this form:
245
246@lisp
247(asdf:asdf-version)
248@end lisp
249
250If it returns a string,
251that is the version of ASDF that is currently installed.
252
253If it raises an error,
254then either ASDF is not loaded, or
255you are using an old version of ASDF.
256
257You can check whether an old version is loaded
258by checking if the ASDF package is present.
259The form below will allow you to programmatically determine
260whether a recent version is loaded, an old version is loaded,
261or none at all:
262
263@lisp
264(or #+asdf2 (asdf:asdf-version) #+asdf :old)
265@end lisp
266
267If it returns a version number, that's the version of ASDF installed.
268If it returns the keyword @code{:OLD},
269then you're using an old version of ASDF (from before 1.635).
270If it returns @code{NIL} then ASDF is not installed.
271
272If you are experiencing problems with ASDF,
273please try upgrading to the latest released version,
274using the method below,
275before you contact us and raise an issue.
276
277
278@section Upgrading ASDF
279
280If your implementation does provide ASDF 2 or later,
281and you want to upgrade to a more recent version,
282just install ASDF like any other package
283(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
284configure ASDF as usual (see @pxref{Configuring ASDF} below),
285and upgrade with:
286
287@lisp
288(require "asdf")
289(asdf:load-system :asdf)
290@end lisp
291
292If on the other hand, your implementation only provides an old ASDF,
293you will require a special configuration step and an old-style loading:
294
295@lisp
296(require "asdf")
297(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
298(asdf:oos 'asdf:load-op :asdf)
299@end lisp
300
301Don't forget the trailing @code{/} at the end of your pathname.
302
303Also, note that older versions of ASDF won't redirect their output,
304or at least won't do it according to your usual ASDF 2 configuration.
305You therefore need write access on the directory
306where you install the new ASDF,
307and make sure you're not using it
308for multiple mutually incompatible implementations.
309At worst, you may have to have multiple copies of the new ASDF,
310e.g. one per implementation installation, to avoid clashes.
311Note that to our knowledge all implementations that provide ASDF
312provide ASDF 2 in their latest release, so
313you may want to upgrade your implementation rather than go through that hoop.
314
315Finally, note that there are some limitations to upgrading ASDF:
316@itemize
317@item
318Any ASDF extension becomes invalid, and will need to be reloaded.
319This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
320Starting with ASDF 2.014.8 (as far as releases go, 2.015),
321ASDF will actually invalidate all previously loaded systems
322when it is loaded on top of a different ASDF version.
323@item
324Until all implementations provide ASDF 2.015 or later,
325it is safer if you upgrade ASDF and its extensions as a special step
326at the very beginning of whatever script you are running,
327before you start using ASDF to load anything else.
328@item
329Until all implementations provide ASDF 2.015 or later,
330it is unsafe to upgrade ASDF as part of loading a system
331that depends on a more recent version of ASDF,
332since the new one might shadow the old one while the old one is running,
333and the running old one will be confused
334when extensions are loaded into the new one.
335In the meantime, we recommend that your systems should @emph{not} specify
336@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
337but instead that they check that a recent enough ASDF is installed,
338with such code as:
339@example
340(unless (or #+asdf2 (asdf:version-satisfies
341                     (asdf:asdf-version) *required-asdf-version*))
342  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
343@end example
344@item
345Until all implementations provide ASDF 2.27 or later,
346it is unsafe for a system to transitively depend on ASDF
347and not directly depend on ASDF;
348if any of the system you use either depends-on asdf,
349system-depends-on asdf, or transitively does,
350you should also do as well.
351@end itemize
352
353
354@section Loading an otherwise installed ASDF
355
356If your implementation doesn't include ASDF,
357if for some reason the upgrade somehow fails,
358does not or cannot apply to your case,
359you will have to install the file @file{asdf.lisp}
360somewhere and load it with:
361
362@lisp
363(load "/path/to/your/installed/asdf.lisp")
364@end lisp
365
366The single file @file{asdf.lisp} is all you normally need to use ASDF.
367
368You can extract this file from latest release tarball on the
369@url{http://common-lisp.net/project/asdf/,ASDF website}.
370If you are daring and willing to report bugs, you can get
371the latest and greatest version of ASDF from its git repository.
372@xref{Getting the latest version}.
373
374For maximum convenience you might want to have ASDF loaded
375whenever you start your Lisp implementation,
376for example by loading it from the startup script or dumping a custom core
377--- check your Lisp implementation's manual for details.
378
379
380@node Configuring ASDF, Using ASDF, Loading ASDF, Top
381@comment  node-name,  next,  previous,  up
382
383@chapter Configuring ASDF
384
385@section Configuring ASDF to find your systems
386
387So it may compile and load your systems, ASDF must be configured to find
388the @file{.asd} files that contain system definitions.
389
390Since ASDF 2, the preferred way to configure where ASDF finds your systems is
391the @code{source-registry} facility,
392fully described in its own chapter of this manual.
393@xref{Controlling where ASDF searches for systems}.
394
395The default location for a user to install Common Lisp software is under
396@file{~/.local/share/common-lisp/source/}.
397If you install software there, you don't need further configuration.
398If you're installing software yourself at a location that isn't standard,
399you have to tell ASDF where you installed it. See below.
400If you're using some tool to install software (e.g. Quicklisp),
401the authors of that tool should already have configured ASDF.
402
403The simplest way to add a path to your search path,
404say @file{/home/luser/.asd-link-farm/}
405is to create the directory
406@file{~/.config/common-lisp/source-registry.conf.d/}
407and there create a file with any name of your choice,
408and with the type @file{conf},
409for instance @file{42-asd-link-farm.conf}
410containing the line:
411
412@kbd{(:directory "/home/luser/.asd-link-farm/")}
413
414If you want all the subdirectories under @file{/home/luser/lisp/}
415to be recursively scanned for @file{.asd} files, instead use:
416
417@kbd{(:tree "/home/luser/lisp/")}
418
419Note that your Operating System distribution or your system administrator
420may already have configured system-managed libraries for you.
421
422The required @file{.conf} extension allows you to have disabled files
423or editor backups (ending in @file{~}), and works portably
424(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
425Excluded are files the name of which start with a @file{.} character.
426It is customary to start the filename with two digits
427that specify the order in which the directories will be scanned.
428
429ASDF will automatically read your configuration
430the first time you try to find a system.
431You can reset the source-registry configuration with:
432
433@lisp
434(asdf:clear-source-registry)
435@end lisp
436
437And you probably should do so before you dump your Lisp image,
438if the configuration may change
439between the machine where you save it at the time you save it
440and the machine you resume it at the time you resume it.
441
442
443@section Configuring ASDF to find your systems --- old style
444
445The old way to configure ASDF to find your systems is by
446@code{push}ing directory pathnames onto the variable
447@code{asdf:*central-registry*}.
448
449You must configure this variable between the time you load ASDF
450and the time you first try to use it.
451Loading and configuring ASDF presumably happen
452as part of some initialization script that builds or starts
453your Common Lisp software system.
454(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
455
456The @code{asdf:*central-registry*} is empty by default in ASDF 2,
457but is still supported for compatibility with ASDF 1.
458When used, it takes precedence over the above source-registry@footnote{
459It is possible to further customize
460the system definition file search.
461That's considered advanced use, and covered later:
462search forward for
463@code{*system-definition-search-functions*}.
464@xref{Defining systems with defsystem}.}.
465
466For instance, if you wanted ASDF to find the @file{.asd} file
467@file{/home/me/src/foo/foo.asd} your initialization script
468could after it loads ASDF with @code{(require "asdf")}
469configure it with:
470
471@lisp
472(push "/home/me/src/foo/" asdf:*central-registry*)
473@end lisp
474
475Note the trailing slash: when searching for a system,
476ASDF will evaluate each entry of the central registry
477and coerce the result to a pathname@footnote{
478ASDF will indeed call @code{EVAL} on each entry.
479It will also skip entries that evaluate to @code{NIL}.
480
481Strings and pathname objects are self-evaluating,
482in which case the @code{EVAL} step does nothing;
483but you may push arbitrary SEXP onto the central registry,
484that will be evaluated to compute e.g. things that depend
485on the value of shell variables or the identity of the user.
486
487The variable @code{asdf:*central-registry*} is thus a list of
488``system directory designators''.
489A @dfn{system directory designator} is a form
490which will be evaluated whenever a system is to be found,
491and must evaluate to a directory to look in.
492By ``directory'' here, we mean
493``designator for a pathname with a supplied DIRECTORY component''.
494}
495at which point the presence of the trailing directory name separator
496is necessary to tell Lisp that you're discussing a directory
497rather than a file.
498
499Typically, however, there are a lot of @file{.asd} files, and
500a common idiom was to have to put
501a bunch of @emph{symbolic links} to @file{.asd} files
502in a common directory
503and push @emph{that} directory (the ``link farm'')
504to the
505@code{asdf:*central-registry*}
506instead of pushing each of the many involved directories
507to the @code{asdf:*central-registry*}.
508ASDF knows how to follow such @emph{symlinks}
509to the actual file location when resolving the paths of system components
510(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
511if you try aliases under MacOS, we are curious to hear about your experience).
512
513For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
514is a member of @code{*central-registry*}, you could set up the
515system @var{foo} for loading with asdf with the following
516commands at the shell:
517
518@example
519$ cd /home/me/cl/systems/
520$ ln -s ~/src/foo/foo.asd .
521@end example
522
523This old style for configuring ASDF is not recommended for new users,
524but it is supported for old users, and for users who want to programmatically
525control what directories are added to the ASDF search path.
526
527
528@section Configuring where ASDF stores object files
529@findex clear-output-translations
530
531ASDF lets you configure where object files will be stored.
532Sensible defaults are provided and
533you shouldn't normally have to worry about it.
534
535This allows the same source code repository may be shared
536between several versions of several Common Lisp implementations,
537between several users using different compilation options
538and without write privileges on shared source directories, etc.
539This also allows to keep source directories uncluttered
540by plenty of object files.
541
542Starting with ASDF 2, the @code{asdf-output-translations} facility
543was added to ASDF itself, that controls where object files will be stored.
544This facility is fully described in a chapter of this manual,
545@ref{Controlling where ASDF saves compiled files}.
546
547The simplest way to add a translation to your search path,
548say from @file{/foo/bar/baz/quux/}
549to @file{/where/i/want/my/fasls/}
550is to create the directory
551@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
552and there create a file with any name of your choice and the type @file{conf},
553for instance @file{42-bazquux.conf}
554containing the line:
555
556@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
557
558To disable output translations for source under a given directory,
559say @file{/toto/tata/}
560you can create a file @file{40-disable-toto.conf}
561with the line:
562
563@kbd{("/toto/tata/")}
564
565To wholly disable output translations for all directories,
566you can create a file @file{00-disable.conf}
567with the line:
568
569@kbd{(t t)}
570
571Note that your Operating System distribution or your system administrator
572may already have configured translations for you.
573In absence of any configuration, the default is to redirect everything
574under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
575@xref{Controlling where ASDF searches for systems}, for full details.
576
577The required @file{.conf} extension allows you to have disabled files
578or editor backups (ending in @file{~}), and works portably
579(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
580Excluded are files the name of which start with a @file{.} character.
581It is customary to start the filename with two digits
582that specify the order in which the directories will be scanned.
583
584ASDF will automatically read your configuration
585the first time you try to find a system.
586You can reset the source-registry configuration with:
587
588@lisp
589(asdf:clear-output-translations)
590@end lisp
591
592And you probably should do so before you dump your Lisp image,
593if the configuration may change
594between the machine where you save it at the time you save it
595and the machine you resume it at the time you resume it.
596
597Finally note that before ASDF 2,
598other ASDF add-ons offered the same functionality,
599each in subtly different and incompatible ways:
600ASDF-Binary-Locations, cl-launch, common-lisp-controller.
601ASDF-Binary-Locations is now not needed anymore and should not be used.
602cl-launch 3.000 and common-lisp-controller 7.2 have been updated
603to just delegate this functionality to ASDF.
604
605@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
606@comment  node-name,  next,  previous,  up
607
608
609@section Resetting Configuration
610
611When you dump and restore an image, or when you tweak your configuration,
612you may want to reset the ASDF configuration.
613For that you may use the following function:
614
615@defun clear-configuration
616   undoes any ASDF configuration,
617   regarding source-registry or output-translations.
618@end defun
619
620If you use SBCL, CMUCL or SCL, you may use this snippet
621so that the ASDF configuration be cleared automatically as you dump an image:
622
623@example
624#+(or cmu sbcl scl)
625(pushnew 'clear-configuration
626         #+(or cmu scl) ext:*before-save-initializations*
627         #+sbcl sb-ext:*save-hooks*)
628@end example
629
630For compatibility with all Lisp implementations, however,
631you might want instead your build script to explicitly call
632@code{(asdf:clear-configuration)} at an appropriate moment before dumping.
633
634
635@chapter Using ASDF
636
637@section Loading a system
638
639The system @var{foo} is loaded (and compiled, if necessary)
640by evaluating the following Lisp form:
641
642@example
643(asdf:load-system :@var{foo})
644@end example
645
646On some implementations (namely recent versions of
647ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP,
648LispWorks, MKCL, SBCL and XCL),
649ASDF hooks into the @code{CL:REQUIRE} facility
650and you can just use:
651
652@example
653(require :@var{foo})
654@end example
655
656In older versions of ASDF, you needed to use
657@code{(asdf:oos 'asdf:load-op :@var{foo})}.
658If your ASDF is too old to provide @code{asdf:load-system} though
659we recommend that you upgrade to ASDF 2.
660@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
661
662Note the name of a system is specified as a string or a symbol,
663typically a keyword.
664If a symbol (including a keyword), its name is taken and lowercased.
665The name must be a suitable value for the @code{:name} initarg
666to @code{make-pathname} in whatever filesystem the system is to be found.
667The lower-casing-symbols behaviour is unconventional,
668but was selected after some consideration.
669Observations suggest that the type of systems we want to support
670either have lowercase as customary case (unix, mac, windows)
671or silently convert lowercase to uppercase (lpns),
672so this makes more sense than attempting to use @code{:case :common},
673which is reported not to work on some implementations
674
675
676@section Other Operations
677
678ASDF provides three commands for the most common system operations:
679@code{load-system}, @code{compile-system} or @code{test-system}.
680It also provides @code{require-system}, a version of @code{load-system}
681that skips trying to update systems that are already loaded.
682
683Because ASDF is an extensible system
684for defining @emph{operations} on @emph{components},
685it also provides a generic function @code{operate}
686(which is usually abbreviated by @code{oos}).
687You'll use @code{oos} whenever you want to do something beyond
688compiling, loading and testing.
689
690Output from ASDF and ASDF extensions are supposed to be sent
691to the CL stream @code{*standard-output*},
692and so rebinding that stream around calls to @code{asdf:operate}
693should redirect all output from ASDF operations.
694
695Reminder: before ASDF can operate on a system, however,
696it must be able to find and load that system's definition.
697@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
698
699For the advanced users, note that
700@code{require-system} calls @code{load-system}
701with keyword arguments @code{:force-not (loaded-systems)}.
702@code{loaded-systems} returns a list of the names of loaded systems.
703@code{load-system} applies @code{operate} with the operation from
704@code{*load-system-operation*}, which by default is @code{load-op},
705the system, and any provided keyword arguments.
706
707@section Summary
708
709To use ASDF:
710
711@itemize
712@item
713Load ASDF itself into your Lisp image, either through
714@code{(require "asdf")} or else through
715@code{(load "/path/to/asdf.lisp")}.
716
717@item
718Make sure ASDF can find system definitions
719thanks to proper source-registry configuration.
720
721@item
722Load a system with @code{(load-system :my-system)}
723or use some other operation on some system of your choice.
724
725@end itemize
726
727@section Moving on
728
729That's all you need to know to use ASDF to load systems written by others.
730The rest of this manual deals with writing system definitions
731for Common Lisp software you write yourself,
732including how to extend ASDF to define new operation and component types.
733
734
735@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
736@comment  node-name,  next,  previous,  up
737@chapter Defining systems with defsystem
738
739This chapter describes how to use asdf to define systems and develop
740software.
741
742
743@menu
744* The defsystem form::
745* A more involved example::
746* The defsystem grammar::
747* Other code in .asd files::
748@end menu
749
750@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
751@comment  node-name,  next,  previous,  up
752@section The defsystem form
753
754Systems can be constructed programmatically
755by instantiating components using @code{make-instance}.
756Most of the time, however, it is much more practical to use
757a static @code{defsystem} form.
758This section begins with an example of a system definition,
759then gives the full grammar of @code{defsystem}.
760
761Let's look at a simple system.
762This is a complete file that would
763usually be saved as @file{hello-lisp.asd}:
764
765@lisp
766(in-package :asdf)
767
768(defsystem "hello-lisp"
769  :description "hello-lisp: a sample Lisp system."
770  :version "0.2.1"
771  :author "Joe User <joe@@example.com>"
772  :licence "Public Domain"
773  :components ((:file "packages")
774               (:file "macros" :depends-on ("packages"))
775               (:file "hello" :depends-on ("macros"))))
776@end lisp
777
778Some notes about this example:
779
780@itemize
781
782@item
783The file starts with an @code{in-package} form
784to use package @code{asdf}.
785You could instead start your definition by using
786a qualified name @code{asdf:defsystem}.
787
788@item
789If in addition to simply using @code{defsystem},
790you are going to define functions,
791create ASDF extension, globally bind symbols, etc.,
792it is recommended that to avoid namespace pollution between systems,
793you should create your own package for that purpose,
794for instance replacing the above @code{(in-package :asdf)} with:
795
796@lisp
797(defpackage :foo-system
798  (:use :cl :asdf))
799
800(in-package :foo-system)
801@end lisp
802
803@item
804The @code{defsystem} form defines a system named @code{hello-lisp}
805that contains three source files:
806@file{packages}, @file{macros} and @file{hello}.
807
808@item
809The file @file{macros} depends on @file{packages}
810(presumably because the package it's in is defined in @file{packages}),
811and the file @file{hello} depends on @file{macros}
812(and hence, transitively on @file{packages}).
813This means that ASDF will compile and load @file{packages} and @file{macros}
814before starting the compilation of file @file{hello}.
815
816@item
817The files are located in the same directory
818as the file with the system definition.
819ASDF resolves symbolic links (or Windows shortcuts)
820before loading the system definition file and
821stores its location in the resulting system@footnote{
822It is possible, though almost never necessary, to override this behaviour.}.
823This is a good thing because the user can move the system sources
824without having to edit the system definition.
825
826@c FIXME: Should have cross-reference to "Version specifiers" in the
827@c defsystem grammar, but the cross-referencing is so broken by
828@c insufficient node breakdown that I have not put one in.
829@item
830Make sure you know how the @code{:version} numbers will be parsed!  They
831are parsed as period-separated lists of integers.  I.e., in the example,
832@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
833In particular, version @code{0.2.1} is interpreted the same as
834@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
835even though the two are the same when interpreted as decimal fractions.
836@cindex version specifiers
837@cindex :version
838
839@end itemize
840
841@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
842@comment  node-name,  next,  previous,  up
843@section A more involved example
844
845Let's illustrate some more involved uses of @code{defsystem} via a
846slightly convoluted example:
847
848@lisp
849(defsystem "foo"
850  :version "1.0.0"
851  :components ((:module "mod"
852                            :components ((:file "bar")
853                                                  (:file"baz")
854                                                  (:file "quux"))
855                            :perform (compile-op :after (op c)
856                                                  (do-something c))
857                            :explain (compile-op :after (op c)
858                                            (explain-something c)))
859                         (:file "blah")))
860@end lisp
861
862The @code{:module} component named @code{"mod"} is a collection of three files,
863which will be located in a subdirectory of the main code directory named
864@file{mod} (this location can be overridden; see the discussion of the
865@code{:pathname} option in @ref{The defsystem grammar}).
866
867The method-form tokens provide a shorthand for defining methods on
868particular components.  This part
869
870@lisp
871                :perform (compile-op :after (op c)
872                          (do-something c))
873                :explain (compile-op :after (op c)
874                          (explain-something c))
875@end lisp
876
877has the effect of
878
879@lisp
880(defmethod perform :after ((op compile-op) (c (eql ...)))
881           (do-something c))
882(defmethod explain :after ((op compile-op) (c (eql ...)))
883           (explain-something c))
884@end lisp
885
886where @code{...} is the component in question.
887In this case @code{...} would expand to something like
888
889@lisp
890(find-component (find-system "foo") "mod")
891@end lisp
892
893For more details on the syntax of such forms, see @ref{The defsystem
894grammar}.
895For more details on what these methods do, @pxref{Operations} in
896@ref{The object model of ASDF}.
897
898@c The following plunge into the weeds is not appropriate in this
899@c location. [2010/10/03:rpg]
900@c note that although this also supports @code{:before} methods,
901@c they may not do what you want them to ---
902@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
903@c will run after all the dependencies and sub-components have been processed,
904@c but before the component in question has been compiled.
905
906@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
907@comment  node-name,  next,  previous,  up
908@section The defsystem grammar
909
910@c FIXME: @var typesetting not consistently used here.  We should either expand
911@c its use to everywhere, or we should kill it everywhere.
912
913
914@example
915system-definition := ( defsystem system-designator @var{system-option}* )
916
917system-option := :defsystem-depends-on system-list
918                 | :weakly-depends-on @var{system-list}
919                 | :class class-name (see discussion below)
920                 | module-option
921                 | option
922
923module-option := :components component-list
924                 | :serial [ t | nil ]
925                 | :if-component-dep-fails component-dep-fail-option
926
927option :=
928        | :pathname pathname-specifier
929        | :default-component-class class-name
930        | :perform method-form
931        | :explain method-form
932        | :output-files method-form
933        | :operation-done-p method-form
934        | :depends-on ( @var{dependency-def}* )
935        | :in-order-to ( @var{dependency}+ )
936
937
938system-list := ( @var{simple-component-name}* )
939
940component-list := ( @var{component-def}* )
941
942component-def  := ( component-type simple-component-name @var{option}* )
943
944component-type := :system | :module | :file | :static-file | other-component-type
945
946other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
947
948dependency-def := simple-component-name
949               | ( :feature name )
950               | ( :version simple-component-name version-specifier)
951
952dependency := (dependent-op @var{requirement}+)
953requirement := (required-op @var{required-component}+)
954             | (feature feature-name)
955dependent-op := operation-name
956required-op := operation-name | feature
957
958simple-component-name := string
959                      |  symbol
960
961pathname-specifier := pathname | string | symbol
962
963method-form := (operation-name qual lambda-list @&rest body)
964qual := method qualifier
965
966component-dep-fail-option := :fail | :try-next | :ignore
967@end example
968
969
970@subsection Component names
971
972Component names (@code{simple-component-name})
973may be either strings or symbols.
974
975@subsection Component types
976
977Component type names, even if expressed as keywords, will be looked up
978by name in the current package and in the asdf package, if not found in
979the current package.  So a component type @code{my-component-type}, in
980the current package @code{my-system-asd} can be specified as
981@code{:my-component-type}, or @code{my-component-type}.
982
983@subsection System class names
984
985A system class name will be looked up in the same way as a Component
986type (see above).  Typically, one will not need to specify a system
987class name, unless using a non-standard system class defined in some
988ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
989see below.  For such class names in the ASDF package, we recommend that
990the @code{:class} option be specified using a keyword symbol, such as
991
992@example
993:class :MY-NEW-SYSTEM-SUBCLASS
994@end example
995
996This practice will ensure that package name conflicts are avoided.
997Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
998the current package @emph{before} it has been exported from the ASDF
999extension loaded by @code{:defsystem-depends-on}, causing a name
1000conflict in the current package.
1001
1002@subsection Defsystem depends on
1003@cindex :defsystem-depends-on
1004
1005The @code{:defsystem-depends-on} option to @code{defsystem} allows the
1006programmer to specify another ASDF-defined system or set of systems that
1007must be loaded @emph{before} the system definition is processed.
1008Typically this is used to load an ASDF extension that is used in the
1009system definition.
1010
1011@subsection Weakly depends on
1012@cindex :weakly-depends-on
1013
1014The @code{:weakly-depends-on} option to @code{defsystem} allows the
1015programmer to specify another ASDF-defined system or set of systems that
1016ASDF should @emph{try} to load, but need not load in order to be
1017successful.  Typically this is used if there are a number of systems
1018that, if present, could provide additional functionality, but which are
1019not necessary for basic function.
1020
1021Currently, although it is specified to be an option only to
1022@code{defsystem}, this option is accepted at any component, but it probably
1023only makes sense at the @code{defsystem} level. Programmers are cautioned not
1024to use this component option except at the @code{defsystem} level, as
1025this anomalous behavior may be removed without warning.
1026
1027@subsection Pathname specifiers
1028@cindex pathname specifiers
1029
1030A pathname specifier (@code{pathname-specifier})
1031may be a pathname, a string or a symbol.
1032When no pathname specifier is given for a component,
1033which is the usual case, the component name itself is used.
1034
1035If a string is given, which is the usual case,
1036the string will be interpreted as a Unix-style pathname
1037where @code{/} characters will be interpreted as directory separators.
1038Usually, Unix-style relative pathnames are used
1039(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1040they are relative to the path of the parent component.
1041Finally, depending on the @code{component-type},
1042the pathname may be interpreted as either a file or a directory,
1043and if it's a file,
1044a file type may be added corresponding to the @code{component-type},
1045or else it will be extracted from the string itself (if applicable).
1046
1047For instance, the @code{component-type} @code{:module}
1048wants a directory pathname, and so a string @code{"foo/bar"}
1049will be interpreted as the pathname @file{#p"foo/bar/"}.
1050On the other hand, the @code{component-type} @code{:file}
1051wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1052will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1053and a string @code{"foo/bar.quux"}
1054will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1055Finally, the @code{component-type} @code{:static-file}
1056wants a file without specifying a type, and so a string @code{"foo/bar"}
1057will be interpreted as the pathname @file{#p"foo/bar"},
1058and a string @code{"foo/bar.quux"}
1059will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1060
1061ASDF does not interpret the string @code{".."} to designate the parent
1062directory.  This string will be passed through to the underlying
1063operating system for interpretation.  We @emph{believe} that this will
1064work on all platforms where ASDF is deployed, but do not guarantee this
1065behavior.  A pathname object with a relative directory component of
1066@code{:up} or @code{:back} is the only guaranteed way to specify a
1067parent directory.
1068
1069If a symbol is given, it will be translated into a string,
1070and downcased in the process.
1071The downcasing of symbols is unconventional,
1072but was selected after some consideration.
1073Observations suggest that the type of systems we want to support
1074either have lowercase as customary case (Unix, Mac, windows)
1075or silently convert lowercase to uppercase (lpns),
1076so this makes more sense than attempting to use @code{:case :common}
1077as argument to @code{make-pathname},
1078which is reported not to work on some implementations.
1079
1080Pathname objects may be given to override the path for a component.
1081Such objects are typically specified using reader macros such as @code{#p}
1082or @code{#.(make-pathname ...)}.
1083Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1084and that the behavior of @code{parse-namestring} is completely non-portable,
1085unless you are using Common Lisp @code{logical-pathname}s
1086(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1087Pathnames made with @code{#.(make-pathname ...)}
1088can usually be done more easily with the string syntax above.
1089The only case that you really need a pathname object is to override
1090the component-type default file type for a given component.
1091Therefore, pathname objects should only rarely be used.
1092Unhappily, ASDF 1 didn't properly support
1093parsing component names as strings specifying paths with directories,
1094and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1095
1096Note that when specifying pathname objects,
1097ASDF does not do any special interpretation of the pathname
1098influenced by the component type, unlike the procedure for
1099pathname-specifying strings.
1100On the one hand, you have to be careful to provide a pathname that correctly
1101fulfills whatever constraints are required from that component type
1102(e.g. naming a directory or a file with appropriate type);
1103on the other hand, you can circumvent the file type that would otherwise
1104be forced upon you if you were specifying a string.
1105
1106@subsection Version specifiers
1107@cindex version specifiers
1108@cindex :version
1109
1110Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1111@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1112In particular, version @code{0.2.1} is interpreted the same as
1113@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1114even though the two are the same when interpreted as decimal fractions.
1115
1116System definers are encouraged to use version identifiers of the form
1117@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1118API) and patch level.
1119
1120@xref{Common attributes of components}.
1121
1122
1123@subsection Using logical pathnames
1124@cindex logical pathnames
1125
1126We do not generally recommend the use of logical pathnames,
1127especially not so to newcomers to Common Lisp.
1128However, we do support the use of logical pathnames by old timers,
1129when such is their preference.
1130
1131To use logical pathnames,
1132you will have to provide a pathname object as a @code{:pathname} specifier
1133to components that use it, using such syntax as
1134@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1135
1136You only have to specify such logical pathname
1137for your system or some top-level component.
1138Sub-components' relative pathnames,
1139specified using the string syntax for names,
1140will be properly merged with the pathnames of their parents.
1141The specification of a logical pathname host however is @emph{not}
1142otherwise directly supported in the ASDF syntax
1143for pathname specifiers as strings.
1144
1145The @code{asdf-output-translation} layer will
1146avoid trying to resolve and translate logical pathnames.
1147The advantage of this is that
1148you can define yourself what translations you want to use
1149with the logical pathname facility.
1150The disadvantage is that if you do not define such translations,
1151any system that uses logical pathnames will behave differently under
1152asdf-output-translations than other systems you use.
1153
1154If you wish to use logical pathnames you will have to configure the
1155translations yourself before they may be used.
1156ASDF currently provides no specific support
1157for defining logical pathname translations.
1158
1159Note that the reasons we do not recommend logical pathnames are that
1160(1) there is no portable way to set up logical pathnames before they are used,
1161(2) logical pathnames are limited to only portably use
1162a single character case, digits and hyphens.
1163While you can solve the first issue on your own,
1164describing how to do it on each of fifteen implementations supported by ASDF
1165is more than we can document.
1166As for the second issue, mind that the limitation is notably enforced on SBCL,
1167and that you therefore can't portably violate the limitations
1168but must instead define some encoding of your own and add individual mappings
1169to name physical pathnames that do not fit the restrictions.
1170This can notably be a problem when your Lisp files are part of a larger project
1171in which it is common to name files or directories in a way that
1172includes the version numbers of supported protocols,
1173or in which files are shared with software written
1174in different programming languages where conventions include the use of
1175underscores, dots or CamelCase in pathnames.
1176
1177
1178@subsection Serial dependencies
1179@cindex serial dependencies
1180
1181If the @code{:serial t} option is specified for a module,
1182ASDF will add dependencies for each child component,
1183on all the children textually preceding it.
1184This is done as if by @code{:depends-on}.
1185
1186@lisp
1187:serial t
1188:components ((:file "a") (:file "b") (:file "c"))
1189@end lisp
1190
1191is equivalent to
1192
1193@lisp
1194:components ((:file "a")
1195             (:file "b" :depends-on ("a"))
1196             (:file "c" :depends-on ("a" "b")))
1197@end lisp
1198
1199
1200@subsection Source location
1201
1202The @code{:pathname} option is optional in all cases for systems
1203defined via @code{defsystem},
1204and in the usual case the user is recommended not to supply it.
1205
1206Instead, ASDF follows a hairy set of rules that are designed so that
1207@enumerate
1208@item
1209@code{find-system}
1210will load a system from disk
1211and have its pathname default to the right place.
1212@item
1213This pathname information will not be overwritten with
1214@code{*default-pathname-defaults*}
1215(which could be somewhere else altogether)
1216if the user loads up the @file{.asd} file into his editor
1217and interactively re-evaluates that form.
1218@end enumerate
1219
1220If a system is being loaded for the first time,
1221its top-level pathname will be set to:
1222
1223@itemize
1224@item
1225The host/device/directory parts of @code{*load-truename*},
1226if it is bound.
1227@item
1228@code{*default-pathname-defaults*}, otherwise.
1229@end itemize
1230
1231If a system is being redefined, the top-level pathname will be
1232
1233@itemize
1234@item
1235changed, if explicitly supplied or obtained from @code{*load-truename*}
1236(so that an updated source location is reflected in the system definition)
1237@item
1238changed if it had previously been set from @code{*default-pathname-defaults*}
1239@item
1240left as before, if it had previously been set from @code{*load-truename*}
1241and @code{*load-truename*} is currently unbound
1242(so that a developer can evaluate a @code{defsystem} form
1243from within an editor without clobbering its source location)
1244@end itemize
1245
1246@subsection if-component-dep-fails option
1247
1248This option is only appropriate for module components (including
1249systems), not individual source files.
1250
1251For more information about this option, @pxref{Pre-defined subclasses of component}.
1252
1253@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1254@section Other code in .asd files
1255
1256Files containing @code{defsystem} forms
1257are regular Lisp files that are executed by @code{load}.
1258Consequently, you can put whatever Lisp code you like into these files
1259(e.g., code that examines the compile-time environment
1260and adds appropriate features to @code{*features*}).
1261However, some conventions should be followed,
1262so that users can control certain details of execution
1263of the Lisp in @file{.asd} files:
1264
1265@itemize
1266@item
1267Any informative output
1268(other than warnings and errors,
1269which are the condition system's to dispose of)
1270should be sent to the standard CL stream @code{*standard-output*},
1271so that users can easily control the disposition
1272of output from ASDF operations.
1273@end itemize
1274
1275
1276@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1277@comment  node-name,  next,  previous,  up
1278@chapter The object model of ASDF
1279
1280ASDF is designed in an object-oriented way from the ground up.
1281Both a system's structure and the operations that can be performed on systems
1282follow a protocol.
1283ASDF is extensible to new operations and to new component types.
1284This allows the addition of behaviours:
1285for example, a new component could be added for Java JAR archives,
1286and methods specialised on @code{compile-op} added for it
1287that would accomplish the relevant actions.
1288
1289This chapter deals with @emph{components}, the building blocks of a system,
1290and @emph{operations}, the actions that can be performed on a system.
1291
1292
1293
1294@menu
1295* Operations::
1296* Components::
1297* Functions::
1298@end menu
1299
1300@node  Operations, Components, The object model of ASDF, The object model of ASDF
1301@comment  node-name,  next,  previous,  up
1302@section Operations
1303@cindex operation
1304
1305An @dfn{operation} object of the appropriate type is instantiated
1306whenever the user wants to do something with a system like
1307
1308@itemize
1309@item compile all its files
1310@item load the files into a running lisp environment
1311@item copy its source files somewhere else
1312@end itemize
1313
1314Operations can be invoked directly, or examined
1315to see what their effects would be without performing them.
1316@emph{FIXME: document how!}
1317There are a bunch of methods specialised on operation and component type
1318that actually do the grunt work.
1319
1320The operation object contains whatever state is relevant for this purpose
1321(perhaps a list of visited nodes, for example)
1322but primarily is a nice thing to specialise operation methods on
1323and easier than having them all be @code{EQL} methods.
1324
1325Operations are invoked on systems via @code{operate}.
1326@anchor{operate}
1327@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs} @&key @code{force} @code{force-not} @code{verbose} @&allow-other-keys
1328@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs} @&key @&allow-other-keys
1329@code{operate} invokes @var{operation} on @var{system}.
1330@code{oos} is a synonym for @code{operate}.
1331
1332@var{operation} is a symbol that is passed, along with the supplied
1333@var{initargs}, to @code{make-instance} to create the operation object.
1334@var{system} is a system designator.
1335
1336The @var{initargs} are passed to the @code{make-instance} call
1337when creating the operation object.
1338Note that dependencies may cause the operation
1339to invoke other operations on the system or its components:
1340the new operations will be created
1341with the same @var{initargs} as the original one.
1342
1343If @var{force} is @code{t}, then all systems
1344are forced to be recompiled even if not modified since last compilation.
1345If @var{force} is a list, then it specifies a list of systems that
1346are forced to be recompiled even if not modified since last compilation.
1347If @var{force-not} is @code{t}, then all systems
1348are forced not to be recompiled even if modified since last compilation.
1349If @var{force-not} is a list, then it specifies a list of systems that
1350are forced not to be recompiled even if modified since last compilation.
1351@var{force} takes precedences over @var{force-not};
1352both of them apply to systems that are dependencies and were already compiled.
1353@end deffn
1354
1355@menu
1356* Predefined operations of ASDF::
1357* Creating new operations::
1358@end menu
1359
1360@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1361@comment  node-name,  next,  previous,  up
1362@subsection Predefined operations of ASDF
1363
1364All the operations described in this section are in the @code{asdf} package.
1365They are invoked via the @code{operate} generic function.
1366
1367@lisp
1368(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1369@end lisp
1370
1371@deffn Operation @code{compile-op} @&key @code{proclamations}
1372
1373This operation compiles the specified component.
1374If proclamations are supplied, they will be proclaimed.
1375This is a good place to specify optimization settings.
1376
1377When creating a new component type,
1378you should provide methods for @code{compile-op}.
1379
1380When @code{compile-op} is invoked,
1381component dependencies often cause some parts of the system
1382to be loaded as well as compiled.
1383Invoking @code{compile-op}
1384does not necessarily load all the parts of the system, though;
1385use @code{load-op} to load a system.
1386@end deffn
1387
1388@deffn Operation @code{load-op} @&key @code{proclamations}
1389
1390This operation loads a system.
1391
1392The default methods for @code{load-op} compile files before loading them.
1393For parity, your own methods on new component types should probably do so too.
1394@end deffn
1395
1396@deffn Operation @code{load-source-op}
1397
1398This operation will load the source for the files in a module
1399even if the source files have been compiled.
1400Systems sometimes have knotty dependencies
1401which require that sources are loaded
1402before they can be compiled.
1403This is how you do that.
1404
1405If you are creating a component type,
1406you need to implement this operation --- at least, where meaningful.
1407@end deffn
1408
1409@anchor{test-op}
1410@deffn Operation @code{test-op}
1411
1412This operation will perform some tests on the module.
1413The default method will do nothing.
1414The default dependency is to require
1415@code{load-op} to be performed on the module first.
1416The default @code{operation-done-p} is that the operation is @emph{never} done
1417---
1418we assume that if you invoke the @code{test-op},
1419you want to test the system, even if you have already done so.
1420
1421The results of this operation are not defined by ASDF.
1422It has proven difficult to define how the test operation
1423should signal its results to the user
1424in a way that is compatible with all of the various test libraries
1425and test techniques in use in the community.
1426@end deffn
1427
1428@c @deffn Operation test-system-version @&key minimum
1429
1430@c Asks the system whether it satisfies a version requirement.
1431
1432@c The default method accepts a string, which is expected to contain of a
1433@c number of integers separated by #\. characters.  The method is not
1434@c recursive.  The component satisfies the version dependency if it has
1435@c the same major number as required and each of its sub-versions is
1436@c greater than or equal to the sub-version number required.
1437
1438@c @lisp
1439@c (defun version-satisfies (x y)
1440@c   (labels ((bigger (x y)
1441@c           (cond ((not y) t)
1442@c                 ((not x) nil)
1443@c                 ((> (car x) (car y)) t)
1444@c                 ((= (car x) (car y))
1445@c                  (bigger (cdr x) (cdr y))))))
1446@c     (and (= (car x) (car y))
1447@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1448@c @end lisp
1449
1450@c If that doesn't work for your system, you can override it.  I hope
1451@c you have as much fun writing the new method as @verb{|#lisp|} did
1452@c reimplementing this one.
1453@c @end deffn
1454
1455@c @deffn Operation feature-dependent-op
1456
1457@c An instance of @code{feature-dependent-op} will ignore any components
1458@c which have a @code{features} attribute, unless the feature combination
1459@c it designates is satisfied by @code{*features*}.  This operation is
1460@c not intended to be instantiated directly, but other operations may
1461@c inherit from it.
1462
1463@c @end deffn
1464
1465@node  Creating new operations,  , Predefined operations of ASDF, Operations
1466@comment  node-name,  next,  previous,  up
1467@subsection Creating new operations
1468
1469ASDF was designed to be extensible in an object-oriented fashion.
1470To teach ASDF new tricks, a programmer can implement the behaviour he wants
1471by creating a subclass of @code{operation}.
1472
1473ASDF's pre-defined operations are in no way ``privileged'',
1474but it is requested that developers never use the @code{asdf} package
1475for operations they develop themselves.
1476The rationale for this rule is that we don't want to establish a
1477``global asdf operation name registry'',
1478but also want to avoid name clashes.
1479
1480An operation must provide methods for the following generic functions
1481when invoked with an object of type @code{source-file}:
1482@emph{FIXME describe this better}
1483
1484@itemize
1485
1486@item @code{output-files}
1487The @code{output-files} method determines where the method will put its files.
1488It returns two values, a list of pathnames, and a boolean.
1489If the boolean is @code{T} then the pathnames are marked
1490not be translated by enclosing @code{:around} methods.
1491If the boolean is @code{NIL} then enclosing @code{:around} methods
1492may translate these pathnames, e.g. to ensure object files
1493are somehow stored in some implementation-dependent cache.
1494@item @code{perform}
1495The @code{perform} method must call @code{output-files}
1496to find out where to put its files,
1497because the user is allowed to override.
1498@item @code{output-files}
1499for local policy @code{explain}
1500@item @code{operation-done-p},
1501if you don't like the default one
1502
1503@end itemize
1504
1505Operations that print output should send that output to the standard
1506CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1507
1508@node Components, Functions, Operations, The object model of ASDF
1509@comment  node-name,  next,  previous,  up
1510@section Components
1511@cindex component
1512@cindex system
1513@cindex system designator
1514@vindex *system-definition-search-functions*
1515
1516A @dfn{component} represents a source file or
1517(recursively) a collection of components.
1518A @dfn{system} is (roughly speaking) a top-level component
1519that can be found via @code{find-system}.
1520
1521A @dfn{system designator} is a string or symbol
1522and behaves just like any other component name
1523(including with regard to the case conversion rules for component names).
1524
1525
1526@defun find-system system-designator &optional (error-p t)
1527
1528Given a system designator, @code{find-system} finds and returns a system.
1529If no system is found, an error of type
1530@code{missing-component} is thrown,
1531or @code{nil} is returned if @code{error-p} is false.
1532
1533To find and update systems, @code{find-system} funcalls each element
1534in the @code{*system-definition-search-functions*} list,
1535expecting a pathname to be returned, or a system object,
1536from which a pathname may be extracted, and that will be registered.
1537The resulting pathname (if any) is loaded
1538if one of the following conditions is true:
1539
1540@itemize
1541@item
1542there is no system of that name in memory
1543@item
1544the pathname is different from that which was previously loaded
1545@item
1546the file's @code{last-modified} time exceeds the @code{last-modified} time
1547of the system in memory
1548@end itemize
1549
1550When system definitions are loaded from @file{.asd} files,
1551a new scratch package is created for them to load into,
1552so that different systems do not overwrite each others operations.
1553The user may also wish to (and is recommended to)
1554include @code{defpackage} and @code{in-package} forms
1555in his system definition files, however,
1556so that they can be loaded manually if need be.
1557
1558The default value of @code{*system-definition-search-functions*}
1559is a list of two functions.
1560The first function looks in each of the directories given
1561by evaluating members of @code{*central-registry*}
1562for a file whose name is the name of the system and whose type is @file{asd}.
1563The first such file is returned,
1564whether or not it turns out to actually define the appropriate system.
1565The second function does something similar,
1566for the directories specified in the @code{source-registry}.
1567Hence, it is strongly advised to define a system
1568@var{foo} in the corresponding file @var{foo.asd}.
1569@end defun
1570
1571
1572@menu
1573* Common attributes of components::
1574* Pre-defined subclasses of component::
1575* Creating new component types::
1576@end menu
1577
1578@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1579@comment  node-name,  next,  previous,  up
1580@subsection Common attributes of components
1581
1582All components, regardless of type, have the following attributes.
1583All attributes except @code{name} are optional.
1584
1585@subsubsection Name
1586
1587A component name is a string or a symbol.
1588If a symbol, its name is taken and lowercased.
1589
1590Unless overridden by a @code{:pathname} attribute,
1591the name will be interpreted as a pathname specifier according
1592to a Unix-style syntax.
1593@xref{The defsystem grammar,,Pathname specifiers}.
1594
1595@subsubsection Version identifier
1596@findex version-satisfies
1597@cindex :version
1598
1599This optional attribute is used by the generic function
1600@code{version-satisfies}, which tests to see if @code{:version}
1601dependencies are satisfied.
1602the version should be a string of integers separated by dots,
1603for example @samp{1.0.11}.
1604For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1605
1606@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1607@c @xref{Predefined operations of ASDF}.
1608@c @emph{Nota Bene}:
1609@c This operation, planned for ASDF 1,
1610@c is still not implemented yet as of ASDF 2.
1611@c Don't hold your breath.
1612
1613
1614
1615@subsubsection Required features
1616
1617@emph{FIXME: This subsection seems to contradict the
1618@code{defsystem} grammar subsection,
1619which doesn't provide any obvious way to specify required features.
1620Furthermore, in 2009, discussions on the
1621@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1622suggested that the specification of required features may be broken,
1623and that no one may have been using them for a while.
1624Please contact the
1625@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1626if you are interested in getting this features feature fixed.}
1627
1628Traditionally defsystem users have used reader conditionals
1629to include or exclude specific per-implementation files.
1630This means that any single implementation cannot read the entire system,
1631which becomes a problem if it doesn't wish to compile it,
1632but instead for example to create an archive file containing all the sources,
1633as it will omit to process the system-dependent sources for other systems.
1634
1635Each component in an asdf system may therefore specify features using
1636the same syntax as @code{#+} does, and it will (somehow) be ignored for
1637certain operations unless the feature conditional is a member of
1638@code{*features*}.
1639
1640
1641@subsubsection Dependencies
1642
1643This attribute specifies dependencies of the component on its siblings.
1644It is optional but often necessary.
1645
1646There is an excitingly complicated relationship between the initarg
1647and the method that you use to ask about dependencies
1648
1649Dependencies are between (operation component) pairs.
1650In your initargs for the component, you can say
1651
1652@lisp
1653:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1654              (load-op (load-op "foo")))
1655@end lisp
1656
1657This means the following things:
1658@itemize
1659@item
1660before performing compile-op on this component, we must perform
1661load-op on @var{a} and @var{b}, and compile-op on @var{c},
1662@item
1663before performing @code{load-op}, we have to load @var{foo}
1664@end itemize
1665
1666The syntax is approximately
1667
1668@verbatim
1669(this-op {(other-op required-components)}+)
1670
1671simple-component-name := string
1672                      |  symbol
1673
1674required-components := simple-component-name
1675                     | (required-components required-components)
1676
1677component-name := simple-component-name
1678                | (:version simple-component-name minimum-version-object)
1679@end verbatim
1680
1681Side note:
1682
1683This is on a par with what ACL defsystem does.
1684mk-defsystem is less general: it has an implied dependency
1685
1686@verbatim
1687  for all x, (load x) depends on (compile x)
1688@end verbatim
1689
1690and using a @code{:depends-on} argument to say that @var{b} depends on
1691@var{a} @emph{actually} means that
1692
1693@verbatim
1694  (compile b) depends on (load a)
1695@end verbatim
1696
1697This is insufficient for e.g. the McCLIM system, which requires that
1698all the files are loaded before any of them can be compiled ]
1699
1700End side note
1701
1702In ASDF, the dependency information for a given component and operation
1703can be queried using @code{(component-depends-on operation component)},
1704which returns a list
1705
1706@lisp
1707((load-op "a") (load-op "b") (compile-op "c") ...)
1708@end lisp
1709
1710@code{component-depends-on} can be subclassed for more specific
1711component/operation types: these need to @code{(call-next-method)}
1712and append the answer to their dependency, unless
1713they have a good reason for completely overriding the default dependencies.
1714
1715If it weren't for CLISP, we'd be using @code{LIST} method
1716combination to do this transparently.
1717But, we need to support CLISP.
1718If you have the time for some CLISP hacking,
1719I'm sure they'd welcome your fixes.
1720@c Doesn't CLISP now support LIST method combination?
1721
1722See the discussion of the semantics of @code{:version} in the defsystem
1723grammar.
1724
1725@c FIXME: Should have cross-reference to "Version specifiers" in the
1726@c defsystem grammar, but the cross-referencing is so broken by
1727@c insufficient node breakdown that I have not put one in.
1728
1729
1730@subsubsection pathname
1731
1732This attribute is optional and if absent (which is the usual case),
1733the component name will be used.
1734
1735@xref{The defsystem grammar,,Pathname specifiers},
1736for an explanation of how this attribute is interpreted.
1737
1738Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1739does additional processing to set the filesystem location of
1740the top component in that system.
1741This is detailed elsewhere. @xref{Defining systems with defsystem}.
1742
1743
1744@subsubsection properties
1745
1746This attribute is optional.
1747
1748Packaging systems often require information about files or systems
1749in addition to that specified by ASDF's pre-defined component attributes.
1750Programs that create vendor packages out of ASDF systems therefore
1751have to create ``placeholder'' information to satisfy these systems.
1752Sometimes the creator of an ASDF system may know the additional
1753information and wish to provide it directly.
1754
1755@code{(component-property component property-name)} and
1756associated @code{setf} method will allow
1757the programmatic update of this information.
1758Property names are compared as if by @code{EQL},
1759so use symbols or keywords or something.
1760
1761@menu
1762* Pre-defined subclasses of component::
1763* Creating new component types::
1764@end menu
1765
1766@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1767@comment  node-name,  next,  previous,  up
1768@subsection Pre-defined subclasses of component
1769
1770@deffn Component source-file
1771
1772A source file is any file that the system does not know how to
1773generate from other components of the system.
1774
1775Note that this is not necessarily the same thing as
1776``a file containing data that is typically fed to a compiler''.
1777If a file is generated by some pre-processor stage
1778(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1779then it is not, by this definition, a source file.
1780Conversely, we might have a graphic file
1781that cannot be automatically regenerated,
1782or a proprietary shared library that we received as a binary:
1783these do count as source files for our purposes.
1784
1785Subclasses of source-file exist for various languages.
1786@emph{FIXME: describe these.}
1787@end deffn
1788
1789@deffn Component module
1790
1791A module is a collection of sub-components.
1792
1793A module component has the following extra initargs:
1794
1795@itemize
1796@item
1797@code{:components} the components contained in this module
1798
1799@item
1800@code{:default-component-class}
1801All children components which don't specify their class explicitly
1802are inferred to be of this type.
1803
1804@item
1805@code{:if-component-dep-fails}
1806This attribute takes one of the values
1807@code{:fail}, @code{:try-next}, @code{:ignore},
1808its default value is @code{:fail}.
1809The other values can be used for implementing conditional compilation
1810based on implementation @code{*features*},
1811for the case where it is not necessary for all files in a module to be
1812compiled.
1813@emph{FIXME: such conditional compilation has been reported
1814to be broken in 2009.}
1815
1816@item
1817@code{:serial} When this attribute is set,
1818each subcomponent of this component is assumed to depend on all subcomponents
1819before it in the list given to @code{:components}, i.e.
1820all of them are loaded before a compile or load operation is performed on it.
1821
1822@end itemize
1823
1824The default operation knows how to traverse a module, so
1825most operations will not need to provide methods specialised on modules.
1826
1827@code{module} may be subclassed to represent components such as
1828foreign-language linked libraries or archive files.
1829@end deffn
1830
1831@deffn Component system
1832
1833@code{system} is a subclass of @code{module}.
1834
1835A system is a module with a few extra attributes for documentation
1836purposes; these are given elsewhere.
1837@xref{The defsystem grammar}.
1838
1839Users can create new classes for their systems:
1840the default @code{defsystem} macro takes a @code{:class} keyword argument.
1841@end deffn
1842
1843@node  Creating new component types,  , Pre-defined subclasses of component, Components
1844@comment  node-name,  next,  previous,  up
1845@subsection Creating new component types
1846
1847New component types are defined by subclassing one of the existing
1848component classes and specializing methods on the new component class.
1849
1850@emph{FIXME: this should perhaps be explained more throughly,
1851not only by example ...}
1852
1853As an example, suppose we have some implementation-dependent
1854functionality that we want to isolate
1855in one subdirectory per Lisp implementation our system supports.
1856We create a subclass of
1857@code{cl-source-file}:
1858
1859@lisp
1860(defclass unportable-cl-source-file (cl-source-file)
1861  ())
1862@end lisp
1863
1864Function @code{asdf:implementation-type} (exported since 2.014.14)
1865gives us the name of the subdirectory.
1866All that's left is to define how to calculate the pathname
1867of an @code{unportable-cl-source-file}.
1868
1869@lisp
1870(defmethod component-pathname ((component unportable-cl-source-file))
1871  (merge-pathnames*
1872   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1873   (call-next-method)))
1874@end lisp
1875
1876The new component type is used in a @code{defsystem} form in this way:
1877
1878@lisp
1879(defsystem :foo
1880    :components
1881    ((:file "packages")
1882     ...
1883     (:unportable-cl-source-file "threads"
1884      :depends-on ("packages" ...))
1885     ...
1886    )
1887@end lisp
1888
1889@node Functions,  , Components, The object model of ASDF
1890@comment  node-name,  next,  previous,  up
1891@section Functions
1892@findex version-satisfies
1893
1894@deffn version-satisfies @var{version} @var{version-spec}
1895Does @var{version} satisfy the @var{version-spec}.  A generic function.
1896ASDF provides built-in methods for @var{version} being a
1897@code{component} or @code{string}.  @var{version-spec} should be a
1898string.
1899
1900In the wild, we typically see version numbering only on components of
1901type @code{system}.
1902
1903For more information about how @code{version-satisfies} interprets
1904version strings and specifications, @pxref{The defsystem grammar} and
1905@ref{Common attributes of components}.
1906@end deffn
1907
1908@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1909@comment  node-name,  next,  previous,  up
1910@chapter Controlling where ASDF searches for systems
1911
1912@section Configurations
1913
1914Configurations specify paths where to find system files.
1915
1916@enumerate
1917
1918@item
1919The search registry may use some hardcoded wrapping registry specification.
1920This allows some implementations (notably SBCL) to specify where to find
1921some special implementation-provided systems that
1922need to precisely match the version of the implementation itself.
1923
1924@item
1925An application may explicitly initialize the source-registry configuration
1926using the configuration API
1927(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1928in which case this takes precedence.
1929It may itself compute this configuration from the command-line,
1930from a script, from its own configuration file, etc.
1931
1932@item
1933The source registry will be configured from
1934the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1935
1936@item
1937The source registry will be configured from
1938user configuration file
1939@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1940(which defaults to
1941@file{~/.config/common-lisp/source-registry.conf})
1942if it exists.
1943
1944@item
1945The source registry will be configured from
1946user configuration directory
1947@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1948(which defaults to
1949@file{~/.config/common-lisp/source-registry.conf.d/})
1950if it exists.
1951
1952@item
1953The source registry will be configured from
1954system configuration file
1955@file{/etc/common-lisp/source-registry.conf}
1956if it exists/
1957
1958@item
1959The source registry will be configured from
1960system configuration directory
1961@file{/etc/common-lisp/source-registry.conf.d/}
1962if it exists.
1963
1964@item
1965The source registry will be configured from a default configuration.
1966This configuration may allow for implementation-specific systems
1967to be found, for systems to be found the current directory
1968(at the time that the configuration is initialized) as well as
1969@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1970@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1971For instance, SBCL will include directories for its contribs
1972when it can find them; it will look for them where SBCL was installed,
1973or at the location specified by the @code{SBCL_HOME} environment variable.
1974
1975@end enumerate
1976
1977Each of these configurations is specified as an s-expression
1978in a trivial domain-specific language (defined below).
1979Additionally, a more shell-friendly syntax is available
1980for the environment variable (defined yet below).
1981
1982Each of these configurations is only used if the previous
1983configuration explicitly or implicitly specifies that it
1984includes its inherited configuration.
1985
1986Additionally, some implementation-specific directories
1987may be automatically prepended to whatever directories are specified
1988in configuration files, no matter if the last one inherits or not.
1989
1990@section XDG base directory
1991
1992Note that we purport to respect the XDG base directory specification
1993as to where configuration files are located,
1994where data files are located,
1995where output file caches are located.
1996Mentions of XDG variables refer to that document.
1997
1998@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1999
2000This specification allows the user to specify some environment variables
2001to customize how applications behave to his preferences.
2002
2003On Windows platforms, when not using Cygwin,
2004instead of the XDG base directory specification,
2005we try to use folder configuration from the registry regarding
2006@code{Common AppData} and similar directories.
2007However, support for querying the Windows registry is limited as of ASDF 2,
2008and on many implementations, we may fall back to always using the defaults
2009without consulting the registry.
2010Patches welcome.
2011
2012@section Backward Compatibility
2013
2014For backward compatibility as well as to provide a practical backdoor for hackers,
2015ASDF will first search for @code{.asd} files in the directories specified in
2016@code{asdf:*central-registry*}
2017before it searches in the source registry above.
2018
2019@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
2020
2021By default, @code{asdf:*central-registry*} will be empty.
2022
2023This old mechanism will therefore not affect you if you don't use it,
2024but will take precedence over the new mechanism if you do use it.
2025
2026@section Configuration DSL
2027
2028Here is the grammar of the s-expression (SEXP) DSL for source-registry
2029configuration:
2030
2031@c FIXME: This is too wide for happy compilation into pdf.
2032
2033@example
2034;; A configuration is a single SEXP starting with keyword :source-registry
2035;; followed by a list of directives.
2036CONFIGURATION := (:source-registry DIRECTIVE ...)
2037
2038;; A directive is one of the following:
2039DIRECTIVE :=
2040    ;; INHERITANCE DIRECTIVE:
2041    ;; Your configuration expression MUST contain
2042    ;; exactly one of either of these:
2043    :inherit-configuration | ; splices inherited configuration (often specified last)
2044    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2045
2046    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2047    ;; you want to use new configuration features but have to bootstrap a
2048    ;; the newer required ASDF from an older release that doesn't sport said features:
2049    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2050
2051    ;; add a single directory to be scanned (no recursion)
2052    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
2053
2054    ;; add a directory hierarchy, recursing but excluding specified patterns
2055    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
2056
2057    ;; override the defaults for exclusion patterns
2058    (:exclude EXCLUSION-PATTERN ...) |
2059    ;; augment the defaults for exclusion patterns
2060    (:also-exclude EXCLUSION-PATTERN ...) |
2061    ;; Note that the scope of a an exclude pattern specification is
2062    ;; the rest of the current configuration expression or file.
2063
2064    ;; splice the parsed contents of another config file
2065    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
2066
2067    ;; This directive specifies that some default must be spliced.
2068    :default-registry
2069
2070REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
2071DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2072
2073PATHNAME-DESIGNATOR :=
2074    NIL | ;; Special: skip this entry.
2075    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
2076
2077EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2078  against the name of a any subdirectory in the directory component
2079        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2080@end example
2081
2082Pathnames are designated using another DSL,
2083shared with the output-translations configuration DSL below.
2084The DSL is resolved by the function @code{asdf::resolve-location},
2085to be documented and exported at some point in the future.
2086
2087@example
2088ABSOLUTE-COMPONENT-DESIGNATOR :=
2089    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2090    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2091             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2092             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2093             ;; Note that none of the above applies to strings used in *central-registry*,
2094             ;; which doesn't use this DSL: they are processed as normal namestrings.
2095             ;; however, you can compute what you put in the *central-registry*
2096             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2097    PATHNAME | ;; pathname (better be an absolute path, or bust)
2098               ;; In output-translations, unless followed by relative components,
2099               ;; it better have appropriate wildcards, as in **/*.*.*
2100    :HOME | ;; designates the user-homedir-pathname ~/
2101    :USER-CACHE | ;; designates the default location for the user cache
2102    :HERE | ;; designates the location of the configuration file
2103            ;; (or *default-pathname-defaults*, if invoked interactively)
2104    :ROOT ;; magic, for output-translations source only: paths that are relative
2105          ;; to the root of the source host and device
2106    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2107
2108RELATIVE-COMPONENT-DESIGNATOR :=
2109    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2110    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
2111             ;; In output translations, if last component, **/*.*.* is added
2112    PATHNAME | ;; pathname; unless last component, directory is assumed.
2113    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
2114    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2115    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2116    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2117    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2118    :*.*.* | ;; any file (since ASDF 2.011.4)
2119    ;; Not supported (anymore): :UID and :USERNAME
2120@end example
2121
2122For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2123which is the default place ASDF looks for this configuration, once contained:
2124@example
2125(:source-registry
2126  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2127  :inherit-configuration)
2128@end example
2129
2130@section Configuration Directories
2131
2132Configuration directories consist in files each containing
2133a list of directives without any enclosing @code{(:source-registry ...)} form.
2134The files will be sorted by namestring as if by @code{string<} and
2135the lists of directives of these files with be concatenated in order.
2136An implicit @code{:inherit-configuration} will be included
2137at the @emph{end} of the list.
2138
2139This allows for packaging software that has file granularity
2140(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2141to easily include configuration information about distributed software.
2142
2143The convention is that, for sorting purposes,
2144the names of files in such a directory begin with two digits
2145that determine the order in which these entries will be read.
2146Also, the type of these files is conventionally @code{"conf"}
2147and as a limitation to some implementations (e.g. GNU clisp),
2148the type cannot be @code{NIL}.
2149
2150Directories may be included by specifying a directory pathname
2151or namestring in an @code{:include} directive, e.g.:
2152
2153@example
2154  (:include "/foo/bar/")
2155@end example
2156
2157Hence, to achieve the same effect as
2158my example @file{~/.config/common-lisp/source-registry.conf} above,
2159I could simply create a file
2160@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2161alone in its directory with the following contents:
2162@example
2163(:tree "/home/fare/cl/")
2164@end example
2165
2166@subsection The :here directive
2167
2168The @code{:here} directive is an absolute pathname designator that
2169refers to the directory containing the configuration file currently
2170being processed.
2171
2172The @code{:here} directive is intended to simplify the delivery of
2173complex CL systems, and for easy configuration of projects shared through
2174revision control systems, in accordance with our design principle that
2175each participant should be able to provide all and only the information
2176available to him or her.
2177
2178Consider a person X who has set up the source code repository for a
2179complex project with a master directory @file{dir/}.  Ordinarily, one
2180might simply have the user add a directive that would look something
2181like this:
2182@example
2183   (:tree  "path/to/dir")
2184@end example
2185But what if X knows that there are very large subtrees
2186under dir that are filled with, e.g., Java source code, image files for
2187icons, etc.?  All of the asdf system definitions are contained in the
2188subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2189these are the only directories that should be searched.
2190
2191In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2192contains the following:
2193@example
2194(:source-registry
2195   (:tree (:here "src/lisp/"))
2196   (:tree (:here "extlib/lisp"))
2197   (:directory (:here "outlier/")))
2198@end example
2199
2200Then when someone else (call her Y) checks out a copy of this
2201repository, she need only add
2202@example
2203(:include "/path/to/my/checkout/directory/asdf.conf")
2204@end example
2205to one of her previously-existing asdf source location configuration
2206files, or invoke @code{initialize-source-registry} with a configuration
2207form containing that s-expression.  ASDF will find the .conf file that X
2208has provided, and then set up source locations within the working
2209directory according to X's (relative) instructions.
2210
2211@section Shell-friendly syntax for configuration
2212
2213When considering environment variable @code{CL_SOURCE_REGISTRY}
2214ASDF will skip to next configuration if it's an empty string.
2215It will @code{READ} the string as a SEXP in the DSL
2216if it begins with a paren @code{(}
2217and it will be interpreted much like @code{TEXINPUTS}
2218list of paths, where
2219
2220  * paths are separated
2221   by a @code{:} (colon) on Unix platforms (including cygwin),
2222   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2223
2224  * each entry is a directory to add to the search path.
2225
2226  * if the entry ends with a double slash @code{//}
2227    then it instead indicates a tree in the subdirectories
2228    of which to recurse.
2229
2230  * if the entry is the empty string (which may only appear once),
2231    then it indicates that the inherited configuration should be
2232    spliced there.
2233
2234
2235@section Search Algorithm
2236@vindex *default-source-registry-exclusions*
2237
2238In case that isn't clear, the semantics of the configuration is that
2239when searching for a system of a given name,
2240directives are processed in order.
2241
2242When looking in a directory, if the system is found, the search succeeds,
2243otherwise it continues.
2244
2245When looking in a tree, if one system is found, the search succeeds.
2246If multiple systems are found, the consequences are unspecified:
2247the search may succeed with any of the found systems,
2248or an error may be raised.
2249ASDF currently returns the first system found,
2250XCVB currently raised an error.
2251If none is found, the search continues.
2252
2253Exclude statements specify patterns of subdirectories
2254the systems from which to ignore.
2255Typically you don't want to use copies of files kept by such
2256version control systems as Darcs.
2257Exclude statements are not propagated to further included or inherited
2258configuration files or expressions;
2259instead the defaults are reset around every configuration statement
2260to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2261
2262Include statements cause the search to recurse with the path specifications
2263from the file specified.
2264
2265An inherit-configuration statement cause the search to recurse with the path
2266specifications from the next configuration
2267(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2268
2269
2270@section Caching Results
2271
2272The implementation is allowed to either eagerly compute the information
2273from the configurations and file system, or to lazily re-compute it
2274every time, or to cache any part of it as it goes.
2275To explicitly flush any information cached by the system, use the API below.
2276
2277
2278@section Configuration API
2279
2280The specified functions are exported from your build system's package.
2281Thus for ASDF the corresponding functions are in package ASDF,
2282and for XCVB the corresponding functions are in package XCVB.
2283
2284@defun initialize-source-registry @&optional PARAMETER
2285   will read the configuration and initialize all internal variables.
2286   You may extend or override configuration
2287   from the environment and configuration files
2288   with the given @var{PARAMETER}, which can be
2289   @code{NIL} (no configuration override),
2290   or a SEXP (in the SEXP DSL),
2291   a string (as in the string DSL),
2292   a pathname (of a file or directory with configuration),
2293   or a symbol (fbound to function that when called returns one of the above).
2294@end defun
2295
2296@defun clear-source-registry
2297   undoes any source registry configuration
2298   and clears any cache for the search algorithm.
2299   You might want to call this function
2300   (or better, @code{clear-configuration})
2301   before you dump an image that would be resumed
2302   with a different configuration,
2303   and return an empty configuration.
2304   Note that this does not include clearing information about
2305   systems defined in the current image, only about
2306   where to look for systems not yet defined.
2307@end defun
2308
2309@defun ensure-source-registry @&optional PARAMETER
2310   checks whether a source registry has been initialized.
2311   If not, initialize it with the given @var{PARAMETER}.
2312@end defun
2313
2314Every time you use ASDF's @code{find-system}, or
2315anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2316@code{ensure-source-registry} is called with parameter NIL,
2317which the first time around causes your configuration to be read.
2318If you change a configuration file,
2319you need to explicitly @code{initialize-source-registry} again,
2320or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2321which will cause the initialization to happen next time around.
2322
2323
2324@section Future
2325
2326If this mechanism is successful, in the future, we may declare
2327@code{asdf:*central-registry*} obsolete and eventually remove it.
2328Any hook into implementation-specific search mechanisms will by then
2329have been integrated in the @code{:default-configuration} which everyone
2330should either explicitly use or implicit inherit. Some shell syntax
2331for it should probably be added somehow.
2332
2333But we're not there yet. For now, let's see how practical this new
2334source-registry is.
2335
2336
2337@section Rejected ideas
2338
2339Alternatives I considered and rejected included:
2340
2341@enumerate
2342@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2343   and somehow the configuration parser expands the new configuration
2344   language into a expanded series of directories of subdirectories to
2345   lookup, pre-recursing through specified hierarchies. This is kludgy,
2346   and leaves little space of future cleanups and extensions.
2347
2348@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2349   in completely new ways, so that new kinds of entries may be implemented
2350   as a recursive search, etc. This seems somewhat backwards.
2351
2352@item Completely remove @code{asdf:*central-registry*}
2353   and break backwards compatibility.
2354   Hopefully this will happen in a few years after everyone migrate to
2355   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2356
2357@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2358   when you dereference it or setf it. Only the new variable with new
2359   semantics is handled by the new search procedure.
2360   Complex and still introduces subtle semantic issues.
2361@end enumerate
2362
2363
2364I've been suggested the below features, but have rejected them,
2365for the sake of keeping ASDF no more complex than strictly necessary.
2366
2367@itemize
2368@item
2369  More syntactic sugar: synonyms for the configuration directives, such as
2370  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2371  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2372
2373@item
2374   The possibility to register individual files instead of directories.
2375
2376@item
2377  Integrate Xach Beane's tilde expander into the parser,
2378  or something similar that is shell-friendly or shell-compatible.
2379  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2380  minimal by removing the need for evaluated entries that ASDF has?
2381  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2382  Hopefully, these are already superseded by the @code{:default-registry}
2383
2384@item
2385  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2386  down a filesystem tree in the environment variable.
2387  It isn't that Lisp friendly either.
2388@end itemize
2389
2390@section TODO
2391
2392@itemize
2393@item Add examples
2394@end itemize
2395
2396
2397@section Credits for the source-registry
2398
2399Thanks a lot to Stelian Ionescu for the initial idea.
2400
2401Thanks to Rommel Martinez for the initial implementation attempt.
2402
2403All bad design ideas and implementation bugs are to mine, not theirs.
2404But so are good design ideas and elegant implementation tricks.
2405
2406 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2407
2408
2409
2410@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2411@comment  node-name,  next,  previous,  up
2412@chapter Controlling where ASDF saves compiled files
2413@cindex asdf-output-translations
2414@vindex ASDF_OUTPUT_TRANSLATIONS
2415
2416Each Common Lisp implementation has its own format
2417for compiled files (fasls for short, short for ``fast loading'').
2418If you use multiple implementations
2419(or multiple versions of the same implementation),
2420you'll soon find your source directories
2421littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2422Worse yet, some implementations use the same file extension
2423while changing formats from version to version (or platform to platform)
2424which means that you'll have to recompile binaries
2425as you switch from one implementation to the next.
2426
2427ASDF 2 includes the @code{asdf-output-translations} facility
2428to mitigate the problem.
2429
2430@section Configurations
2431
2432Configurations specify mappings from input locations to output locations.
2433Once again we rely on the XDG base directory specification for configuration.
2434@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2435
2436@enumerate
2437
2438@item
2439Some hardcoded wrapping output translations configuration may be used.
2440This allows special output translations (or usually, invariant directories)
2441to be specified corresponding to the similar special entries in the source registry.
2442
2443@item
2444An application may explicitly initialize the output-translations
2445configuration using the Configuration API
2446in which case this takes precedence.
2447(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2448It may itself compute this configuration from the command-line,
2449from a script, from its own configuration file, etc.
2450
2451@item
2452The source registry will be configured from
2453the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2454
2455@item
2456The source registry will be configured from
2457user configuration file
2458@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2459(which defaults to
2460@file{~/.config/common-lisp/asdf-output-translations.conf})
2461if it exists.
2462
2463@item
2464The source registry will be configured from
2465user configuration directory
2466@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2467(which defaults to
2468@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2469if it exists.
2470
2471@item
2472The source registry will be configured from
2473system configuration file
2474@file{/etc/common-lisp/asdf-output-translations.conf}
2475if it exists.
2476
2477@item
2478The source registry will be configured from
2479system configuration directory
2480@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2481if it exists.
2482
2483@end enumerate
2484
2485Each of these configurations is specified as a SEXP
2486in a trival domain-specific language (defined below).
2487Additionally, a more shell-friendly syntax is available
2488for the environment variable (defined yet below).
2489
2490Each of these configurations is only used if the previous
2491configuration explicitly or implicitly specifies that it
2492includes its inherited configuration.
2493
2494Note that by default, a per-user cache is used for output files.
2495This allows the seamless use of shared installations of software
2496between several users, and takes files out of the way of the developers
2497when they browse source code,
2498at the expense of taking a small toll when developers have to clean up
2499output files and find they need to get familiar with output-translations first.
2500
2501
2502@section Backward Compatibility
2503@cindex ASDF-BINARY-LOCATIONS compatibility
2504
2505
2506We purposefully do NOT provide backward compatibility with earlier versions of
2507@code{ASDF-Binary-Locations} (8 Sept 2009),
2508@code{common-lisp-controller} (7.0) or
2509@code{cl-launch} (2.35),
2510each of which had similar general capabilities.
2511The previous APIs of these programs were not designed
2512for configuration by the end-user
2513in an easy way with configuration files.
2514Recent versions of same packages use
2515the new @code{asdf-output-translations} API as defined below:
2516@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2517@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2518
2519This incompatibility shouldn't inconvenience many people.
2520Indeed, few people use and customize these packages;
2521these few people are experts who can trivially adapt to the new configuration.
2522Most people are not experts, could not properly configure these features
2523(except inasmuch as the default configuration of
2524@code{common-lisp-controller} and/or @code{cl-launch}
2525might have been doing the right thing for some users),
2526and yet will experience software that ``just works'',
2527as configured by the system distributor, or by default.
2528
2529Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2530we provide a limited emulation mode:
2531
2532@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2533This function will initialize the new @code{asdf-output-translations} facility in a way
2534that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2535Where you would previously set global variables
2536@var{*centralize-lisp-binaries*},
2537@var{*default-toplevel-directory*},
2538@var{*include-per-user-information*},
2539@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2540you will now have to pass the same values as keyword arguments to this function.
2541Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2542will accept any valid pathname designator for @code{asdf-output-translations}
2543instead of just strings and pathnames.
2544@end defun
2545
2546If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2547(the one available as an extension to load of top of ASDF,
2548not the one built into a few old versions of ASDF),
2549but first you must disable @code{asdf-output-translations}
2550with @code{(asdf:disable-output-translations)},
2551or you might experience ``interesting'' issues.
2552
2553Also, note that output translation is enabled by default.
2554To disable it, use @code{(asdf:disable-output-translations)}.
2555
2556
2557@section Configuration DSL
2558
2559Here is the grammar of the SEXP DSL
2560for @code{asdf-output-translations} configuration:
2561
2562@verbatim
2563;; A configuration is single SEXP starting with keyword :source-registry
2564;; followed by a list of directives.
2565CONFIGURATION := (:output-translations DIRECTIVE ...)
2566
2567;; A directive is one of the following:
2568DIRECTIVE :=
2569    ;; INHERITANCE DIRECTIVE:
2570    ;; Your configuration expression MUST contain
2571    ;; exactly one of either of these:
2572    :inherit-configuration | ; splices inherited configuration (often specified last)
2573    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2574
2575    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2576    ;; you want to use new configuration features but have to bootstrap a
2577    ;; the newer required ASDF from an older release that doesn't sport said features:
2578    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2579
2580    ;; include a configuration file or directory
2581    (:include PATHNAME-DESIGNATOR) |
2582
2583    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2584    :enable-user-cache |
2585    ;; Disable global cache. Map / to /
2586    :disable-cache |
2587
2588    ;; add a single directory to be scanned (no recursion)
2589    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2590
2591    ;; use a function to return the translation of a directory designator
2592    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2593
2594DIRECTORY-DESIGNATOR :=
2595    NIL | ;; As source: skip this entry. As destination: same as source
2596    T | ;; as source matches anything, as destination leaves pathname unmapped.
2597    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2598
2599TRANSLATION-FUNCTION :=
2600    SYMBOL | ;; symbol of a function that takes two arguments,
2601             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2602    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2603             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2604
2605@end verbatim
2606
2607Relative components better be either relative
2608or subdirectories of the path before them, or bust.
2609
2610The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2611You can specify more fine-grained patterns
2612by using a pathname object as the last component
2613e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2614
2615You may use @code{#+features} to customize the configuration file.
2616
2617The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2618to anything but themselves (same as if the second designator was the same as the first).
2619
2620When the first designator is @code{t},
2621the mapping always matches.
2622When the first designator starts with @code{:root},
2623the mapping matches any host and device.
2624In either of these cases, if the second designator
2625isn't @code{t} and doesn't start with @code{:root},
2626then strings indicating the host and pathname are somehow copied
2627in the beginning of the directory component of the source pathname
2628before it is translated.
2629
2630When the second designator is @code{t}, the mapping is the identity.
2631When the second designator starts with @code{:root},
2632the mapping preserves the host and device of the original pathname.
2633Notably, this allows you to map files
2634to a subdirectory of the whichever directory the file is in.
2635Though the syntax is not quite as easy to use as we'd like,
2636you can have an (source destination) mapping entry such as follows
2637in your configuration file,
2638or you may use @code{enable-asdf-binary-locations-compatibility}
2639with @code{:centralize-lisp-binaries nil}
2640which will do the same thing internally for you:
2641@verbatim
2642  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2643          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2644     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2645       (:root ,wild-subdir :implementation ,wild-file)))
2646@end verbatim
2647Starting with ASDF 2.011.4, you can use the simpler:
2648  @code{`(:root (:root :**/ :implementation :*.*.*))}
2649
2650
2651
2652@code{:include} statements cause the search to recurse with the path specifications
2653from the file specified.
2654
2655If the @code{translate-pathname} mechanism cannot achieve a desired
2656translation, the user may provide a function which provides the
2657required algorithim.  Such a translation function is specified by
2658supplying a list as the second @code{directory-designator}
2659the first element of which is the keyword @code{:function},
2660and the second element of which is
2661either a symbol which designates a function or a lambda expression.
2662The function designated by the second argument must take two arguments,
2663the first being the pathname of the source file,
2664the second being the wildcard that was matched.
2665The result of the function invocation should be the translated pathname.
2666
2667An @code{:inherit-configuration} statement cause the search to recurse with the path
2668specifications from the next configuration.
2669@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2670
2671@itemize
2672@item
2673@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2674@item
2675@code{:disable-cache} is the same as @code{(t t)}.
2676@item
2677@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2678which by default is the same as using
2679@code{(:home ".cache" "common-lisp" :implementation)}.
2680@item
2681@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2682which by default is the same as using
2683@code{("/var/cache/common-lisp" :uid :implementation-type)}
2684(on Unix and cygwin), or something semi-sensible on Windows.
2685@end itemize
2686
2687
2688@section Configuration Directories
2689
2690Configuration directories consist in files each contains
2691a list of directives without any enclosing
2692@code{(:output-translations ...)} form.
2693The files will be sorted by namestring as if by @code{string<} and
2694the lists of directives of these files with be concatenated in order.
2695An implicit @code{:inherit-configuration} will be included
2696at the @emph{end} of the list.
2697
2698This allows for packaging software that has file granularity
2699(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2700to easily include configuration information about software being distributed.
2701
2702The convention is that, for sorting purposes,
2703the names of files in such a directory begin with two digits
2704that determine the order in which these entries will be read.
2705Also, the type of these files is conventionally @code{"conf"}
2706and as a limitation of some implementations, the type cannot be @code{NIL}.
2707
2708Directories may be included by specifying a directory pathname
2709or namestring in an @code{:include} directive, e.g.:
2710@verbatim
2711  (:include "/foo/bar/")
2712@end verbatim
2713
2714@section Shell-friendly syntax for configuration
2715
2716When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2717ASDF will skip to next configuration if it's an empty string.
2718It will @code{READ} the string as an SEXP in the DSL
2719if it begins with a paren @code{(}
2720and it will be interpreted as a list of directories.
2721Directories should come by pairs, indicating a mapping directive.
2722Entries are separated
2723by a @code{:} (colon) on Unix platforms (including cygwin),
2724by a @code{;} (semicolon) on other platforms (mainly, Windows).
2725
2726The magic empty entry,
2727if it comes in what would otherwise be the first entry in a pair,
2728indicates the splicing of inherited configuration.
2729If it comes as the second entry in a pair,
2730it indicates that the directory specified first is to be left untranslated
2731(which has the same effect as if the directory had been repeated).
2732
2733
2734@section Semantics of Output Translations
2735
2736From the specified configuration,
2737a list of mappings is extracted in a straightforward way:
2738mappings are collected in order, recursing through
2739included or inherited configuration as specified.
2740To this list is prepended some implementation-specific mappings,
2741and is appended a global default.
2742
2743The list is then compiled to a mapping table as follows:
2744for each entry, in order, resolve the first designated directory
2745into an actual directory pathname for source locations.
2746If no mapping was specified yet for that location,
2747resolve the second designated directory to an output location directory
2748add a mapping to the table mapping the source location to the output location,
2749and add another mapping from the output location to itself
2750(unless a mapping already exists for the output location).
2751
2752Based on the table, a mapping function is defined,
2753mapping source pathnames to output pathnames:
2754given a source pathname, locate the longest matching prefix
2755in the source column of the mapping table.
2756Replace that prefix by the corresponding output column
2757in the same row of the table, and return the result.
2758If no match is found, return the source pathname.
2759(A global default mapping the filesystem root to itself
2760may ensure that there will always be a match,
2761with same fall-through semantics).
2762
2763@section Caching Results
2764
2765The implementation is allowed to either eagerly compute the information
2766from the configurations and file system, or to lazily re-compute it
2767every time, or to cache any part of it as it goes.
2768To explicitly flush any information cached by the system, use the API below.
2769
2770
2771@section Output location API
2772
2773The specified functions are exported from package ASDF.
2774
2775@defun initialize-output-translations @&optional PARAMETER
2776   will read the configuration and initialize all internal variables.
2777   You may extend or override configuration
2778   from the environment and configuration files
2779   with the given @var{PARAMETER}, which can be
2780   @code{NIL} (no configuration override),
2781   or a SEXP (in the SEXP DSL),
2782   a string (as in the string DSL),
2783   a pathname (of a file or directory with configuration),
2784   or a symbol (fbound to function that when called returns one of the above).
2785@end defun
2786
2787@defun disable-output-translations
2788   will initialize output translations in a way
2789   that maps every pathname to itself,
2790   effectively disabling the output translation facility.
2791@end defun
2792
2793@defun clear-output-translations
2794   undoes any output translation configuration
2795   and clears any cache for the mapping algorithm.
2796   You might want to call this function
2797   (or better, @code{clear-configuration})
2798   before you dump an image that would be resumed
2799   with a different configuration,
2800   and return an empty configuration.
2801   Note that this does not include clearing information about
2802   systems defined in the current image, only about
2803   where to look for systems not yet defined.
2804@end defun
2805
2806@defun ensure-output-translations @&optional PARAMETER
2807   checks whether output translations have been initialized.
2808   If not, initialize them with the given @var{PARAMETER}.
2809   This function will be called before any attempt to operate on a system.
2810@end defun
2811
2812@defun apply-output-translations PATHNAME
2813   Applies the configured output location translations to @var{PATHNAME}
2814   (calls @code{ensure-output-translations} for the translations).
2815@end defun
2816
2817Every time you use ASDF's @code{output-files}, or
2818anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2819@code{ensure-output-translations} is called with parameter NIL,
2820which the first time around causes your configuration to be read.
2821If you change a configuration file,
2822you need to explicitly @code{initialize-output-translations} again,
2823or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2824which will cause the initialization to happen next time around.
2825
2826
2827@section Credits for output translations
2828
2829Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2830and to Peter van Eynde for @code{Common Lisp Controller}.
2831
2832All bad design ideas and implementation bugs are to mine, not theirs.
2833But so are good design ideas and elegant implementation tricks.
2834
2835 --- Francois-Rene Rideau @email{fare@@tunes.org}
2836
2837@c @section Default locations
2838@c @findex output-files-for-system-and-operation
2839
2840@c The default binary location for each Lisp implementation
2841@c is a subdirectory of each source directory.
2842@c To account for different Lisps, Operating Systems, Implementation versions,
2843@c and so on, ASDF borrows code from SLIME
2844@c to create reasonable custom directory names.
2845@c Here are some examples:
2846
2847@c @itemize
2848@c @item
2849@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2850
2851@c @item
2852@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2853
2854@c @item
2855@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2856@c @end itemize
2857
2858@c By default, all output file pathnames will be relocated
2859@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2860
2861@c See the document @file{README.asdf-output-translations}
2862@c for a full specification on how to configure @code{asdf-output-translations}.
2863
2864@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2865@comment  node-name,  next,  previous,  up
2866@chapter Error handling
2867@findex SYSTEM-DEFINITION-ERROR
2868@findex OPERATION-ERROR
2869
2870@section ASDF errors
2871
2872If ASDF detects an incorrect system definition, it will signal a generalised instance of
2873@code{SYSTEM-DEFINITION-ERROR}.
2874
2875Operations may go wrong (for example when source files contain errors).
2876These are signalled using generalised instances of
2877@code{OPERATION-ERROR}.
2878
2879@section Compilation error and warning handling
2880@vindex *compile-file-warnings-behaviour*
2881@vindex *compile-file-errors-behavior*
2882
2883ASDF checks for warnings and errors when a file is compiled.
2884The variables @var{*compile-file-warnings-behaviour*} and
2885@var{*compile-file-errors-behavior*}
2886control the handling of any such events.
2887The valid values for these variables are
2888@code{:error}, @code{:warn}, and @code{:ignore}.
2889
2890@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2891@comment  node-name,  next,  previous,  up
2892@chapter Miscellaneous additional functionality
2893
2894ASDF includes several additional features that are generally
2895useful for system definition and development.
2896
2897@section Controlling file compilation
2898
2899When declaring a component (system, module, file),
2900you can specify a keyword argument @code{:around-compile function}.
2901If left unspecified,
2902the value will be inherited from the parent component if any,
2903or with a default of @code{nil}
2904if no value is specified in any transitive parent.
2905
2906The argument must be a either @code{nil}, a fbound symbol,
2907a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
2908a function object (e.g. using @code{#.#'} but that's discouraged
2909because it prevents the introspection done by e.g. asdf-dependency-grovel),
2910or a string that when read yields a symbol or a lambda-expression.
2911@code{nil} means the normal compile-file function will be called.
2912A non-nil value designates a function of one argument
2913that will be called with a function that
2914calls the @code{*compile-op-compile-file-function*}
2915(usually @code{compile-file*})
2916with proper arguments;
2917the around-compile hook may supply additional arguments
2918to pass to that @code{*compile-op-compile-file-function*}.
2919
2920One notable argument that is heeded by @code{compile-file*} is
2921@code{:compile-check},
2922a function called when the compilation was otherwise a success,
2923with the same arguments as @code{compile-file};
2924the function shall return true if the compilation
2925and its resulting compiled file respected all system-specific invariants,
2926and false (NIL) if it broke any of those invariants;
2927it may issue warnings or errors before it returns NIL.
2928(NB: The ability to pass such extra flags
2929is only available starting with ASDF 2.22.3.)
2930This feature is notably exercised by asdf-finalizers.
2931
2932By using a string, you may reference
2933a function, symbol and/or package
2934that will only be created later during the build, but
2935isn't yet present at the time the defsystem form is evaluated.
2936However, if your entire system is using such a hook, you may have to
2937explicitly override the hook with @code{nil} for all the modules and files
2938that are compiled before the hook is defined.
2939
2940Using this hook, you may achieve such effects as:
2941locally renaming packages,
2942binding @var{*readtables*} and other syntax-controlling variables,
2943handling warnings and other conditions,
2944proclaiming consistent optimization settings,
2945saving code coverage information,
2946maintaining meta-data about compilation timings,
2947setting gensym counters and PRNG seeds and other sources of non-determinism,
2948overriding the source-location and/or timestamping systems,
2949checking that some compile-time side-effects were properly balanced,
2950etc.
2951
2952Note that there is no around-load hook. This is on purpose.
2953Some implementations such as ECL, GCL or MKCL link object files,
2954which allows for no such hook.
2955Other implementations allow for concatenating FASL files,
2956which doesn't allow for such a hook either.
2957We aim to discourage something that's not portable,
2958and has some dubious impact on performance and semantics
2959even when it is possible.
2960Things you might want to do with an around-load hook
2961are better done around-compile,
2962though it may at times require some creativity
2963(see e.g. the @code{package-renaming} system).
2964
2965
2966@section Controlling source file character encoding
2967
2968Starting with ASDF 2.21, components accept a @code{:encoding} option
2969so authors may specify which character encoding should be used
2970to read and evaluate their source code.
2971When left unspecified, the encoding is inherited
2972from the parent module or system;
2973if no encoding is specified at any point,
2974the default @code{:autodetect} is assumed.
2975By default, only @code{:default}, @code{:utf-8}
2976and @code{:autodetect} are accepted.
2977@code{:autodetect}, the default, calls
2978@code{*encoding-detection-hook*} which by default always returns
2979@code{*default-encoding*} which itself defaults to @code{:default}.
2980
2981In other words, there now are plenty of extension hooks, but
2982by default ASDF follows the backwards compatible behavior
2983of using whichever @code{:default} encoding your implementation uses,
2984which itself may or may not vary based on environment variables
2985and other locale settings.
2986In practice this means that only source code that only uses ASCII
2987is guaranteed to be read the same on all implementations
2988independently from any user setting.
2989
2990Additionally, for backward-compatibility with older versions of ASDF
2991and/or with implementations that do not support unicode and its many encodings,
2992you may want to use
2993the reader conditionals @code{#+asdf-unicode #+asdf-unicode}
2994to protect any @code{:encoding @emph{encoding}} statement
2995as @code{:asdf-unicode} will be present in @code{*features*}
2996only if you're using a recent ASDF
2997on an implementation that supports unicode.
2998We recommend that you avoid using unprotected @code{:encoding} specifications
2999until after ASDF 2.21 or later becomes widespread, hopefully by the end of 2012.
3000
3001While it offers plenty of hooks for extension,
3002and one such extension is being developed (see below),
3003ASDF itself only recognizes one encoding beside @code{:default},
3004and that is @code{:utf-8}, which is the @emph{de facto} standard,
3005already used by the vast majority of libraries that use more than ASCII.
3006On implementations that do not support unicode,
3007the feature @code{:asdf-unicode} is absent, and
3008the @code{:default} external-format is used
3009to read even source files declared as @code{:utf-8}.
3010On these implementations, non-ASCII characters
3011intended to be read as one CL character
3012may thus end up being read as multiple CL characters.
3013In most cases, this shouldn't affect the software's semantics:
3014comments will be skipped just the same, strings with be read and printed
3015with slightly different lengths, symbol names will be accordingly longer,
3016but none of it should matter.
3017But a few systems that actually depend on unicode characters
3018may fail to work properly, or may work in a subtly different way.
3019See for instance @code{lambda-reader}.
3020
3021We invite you to embrace UTF-8
3022as the encoding for non-ASCII characters starting today,
3023even without any explicit specification in your @code{.asd} files.
3024Indeed, on some implementations and configurations,
3025UTF-8 is already the @code{:default},
3026and loading your code may cause errors if it is encoded in anything but UTF-8.
3027Therefore, even with the legacy behavior,
3028non-UTF-8 is guaranteed to break for some users,
3029whereas UTF-8 is pretty much guaranteed not to break anywhere
3030(provided you do @emph{not} use a BOM),
3031although it might be read incorrectly on some implementations.
3032In the future, we intend to make @code{:utf-8}
3033the default value of @code{*default-encoding*},
3034to be enforced everywhere, so at least the code is guaranteed
3035to be read correctly everywhere it can be.
3036
3037If you need non-standard character encodings for your source code,
3038use the extension system @code{asdf-encodings}, by specifying
3039@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
3040This extension system will register support for more encodings using the
3041@code{*encoding-external-format-hook*} facility,
3042so you can explicitly specify @code{:encoding :latin1}
3043in your @code{.asd} file.
3044Using the @code{*encoding-detection-hook*} it will also
3045eventually implement some autodetection of a file's encoding
3046from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
3047or otherwise based on an analysis of octet patterns in the file.
3048At this point, asdf-encoding only supports the encodings
3049that are supported as part of your implementation.
3050Since the list varies depending on implementations,
3051we once again recommend you use @code{:utf-8} everywhere,
3052which is the most portable (next is @code{:latin1}).
3053
3054If you're not using a version of Quicklisp that has it,
3055you may get the source for @code{asdf-encodings} using git:
3056@kbd{git clone git://common-lisp.net/projects/asdf/asdf-encodings.git}
3057or
3058@kbd{git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git}.
3059You can also browse the repository on
3060@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git}.
3061
3062In the future, we intend to change the default @code{*default-encoding*}
3063to @code{:utf-8}, which is already the de facto standard
3064for most libraries that use non-ASCII characters:
3065utf-8 works everywhere and was backhandedly enforced by
3066a lot of people using SBCL and utf-8 and sending reports to authors
3067so they make their packages compatible.
3068A survey showed only about a handful few libraries
3069are incompatible with non-UTF-8, and then, only in comments,
3070and we believe that authors will adopt UTF-8 when prompted.
3071See the April 2012 discussion on the asdf-devel mailing-list.
3072For backwards compatibility with users who insist on a non-UTF-8 encoding,
3073but cannot immediately transition to using @code{asdf-encodings}
3074(maybe because it isn't ready), it will still be possible to use
3075the @code{:encoding :default} option in your @code{defsystem} form
3076to restore the behavior of ASDF 2.20 and earlier.
3077This shouldn't be required in libraries,
3078because user pressure as mentioned above will already have pushed
3079library authors towards using UTF-8;
3080but authors of end-user programs might care.
3081
3082When you use @code{asdf-encodings}, any further loaded @code{.asd} file
3083will use the autodetection algorithm to determine its encoding;
3084yet if you depend on this detection happening,
3085you may want to explicitly load @code{asdf-encodings} early in your build,
3086for by the time you can use @code{:defsystem-depends-on},
3087it is already too late to load it.
3088In practice, this means that the @code{*default-encoding*}
3089is usually used for @code{.asd} files.
3090Currently, this defaults to @code{:default} for backwards compatibility,
3091and that means that you shouldn't rely on non-ASCII characters in a .asd file.
3092Since component (path)names are the only real data in these files,
3093and non-ASCII characters are not very portable for file names,
3094this isn't too much of an issue.
3095We still encourage you to use either plain ASCII or UTF-8
3096in @code{.asd} files,
3097as we intend to make @code{:utf-8} the default encoding in the future.
3098This might matter, for instance, in meta-data about author's names.
3099
3100
3101@section Miscellaneous Exported Functions
3102
3103@defun coerce-pathname name @&key type defaults
3104
3105This function (available starting with ASDF 2.012.11)
3106takes an argument, and portably interprets it as a pathname.
3107If the argument @var{name} is a pathname or @code{nil}, it is passed through;
3108if it's a symbol, it's interpreted as a string by downcasing it;
3109if it's a string, it is first separated using @code{/} into substrings;
3110the leading substrings denote subdirectories of a relative pathname.
3111If @var{type} is @code{:directory} or the string ends with @code{/},
3112the last substring is also a subdirectory;
3113if @var{type} is a string, it is used as the type of the pathname, and
3114the last substring is the name component of the pathname;
3115if @var{type} is @code{nil}, the last substring specifies both name and type components
3116of the pathname, with the last @code{.} separating them, or only the name component
3117if there's no last @code{.} or if there is only one dot and it's the first character.
3118The host, device and version components come from @var{defaults}, which defaults to
3119@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
3120
3121@end defun
3122
3123@defun merge-pathnames* @&key specified defaults
3124
3125This function is a replacement for @code{merge-pathnames} that uses the host and device
3126from the @var{defaults} rather than the @var{specified} pathname when the latter
3127is a relative pathname. This allows ASDF and its users to create and use relative pathnames
3128without having to know beforehand what are the host and device
3129of the absolute pathnames they are relative to.
3130
3131@end defun
3132
3133@defun system-relative-pathname system name @&key type
3134
3135It's often handy to locate a file relative to some system.
3136The @code{system-relative-pathname} function meets this need.
3137
3138It takes two mandatory arguments @var{system} and @var{name}
3139and a keyword argument @var{type}:
3140@var{system} is name of a system, whereas @var{name} and optionally @var{type}
3141specify a relative pathname, interpreted like a component pathname specifier
3142by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
3143
3144It returns a pathname built from the location of the system's
3145source directory and the relative pathname. For example:
3146
3147@lisp
3148> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
3149#P"/repository/other/cl-ppcre/regex.data"
3150@end lisp
3151
3152@end defun
3153
3154@defun system-source-directory system-designator
3155
3156ASDF does not provide a turnkey solution for locating
3157data (or other miscellaneous) files
3158that are distributed together with the source code of a system.
3159Programmers can use @code{system-source-directory} to find such files.
3160Returns a pathname object.
3161The @var{system-designator} may be a string, symbol, or ASDF system object.
3162@end defun
3163
3164@defun clear-system system-designator
3165
3166It is sometimes useful to force recompilation of a previously loaded system.
3167In these cases, it may be useful to @code{(asdf:clear-system :foo)}
3168to remove the system from the table of currently loaded systems;
3169the next time the system @code{foo} or one that depends on it is re-loaded,
3170@code{foo} will then be loaded again.
3171Alternatively, you could touch @code{foo.asd} or
3172remove the corresponding fasls from the output file cache.
3173(It was once conceived that one should provide
3174a list of systems the recompilation of which to force
3175as the @code{:force} keyword argument to @code{load-system};
3176but this has never worked, and though the feature was fixed in ASDF 2.000,
3177it remains @code{cerror}'ed out as nobody ever used it.)
3178
3179Note that this does not and cannot by itself undo the previous loading
3180of the system. Common Lisp has no provision for such an operation,
3181and its reliance on irreversible side-effects to global datastructures
3182makes such a thing impossible in the general case.
3183If the software being re-loaded is not conceived with hot upgrade in mind,
3184this re-loading may cause many errors, warnings or subtle silent problems,
3185as packages, generic function signatures, structures, types, macros, constants, etc.
3186are being redefined incompatibly.
3187It is up to the user to make sure that reloading is possible and has the desired effect.
3188In some cases, extreme measures such as recursively deleting packages,
3189unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
3190and much more are necessary for reloading to happen smoothly.
3191ASDF itself goes through notable pains to make such a hot upgrade possible
3192with respect to its own code, and what it does is ridiculously complex;
3193look at the beginning of @file{asdf.lisp} to see what it does.
3194@end defun
3195
3196@defun run-shell-command
3197
3198This function is obsolete and present only for the sake of backwards-compatibility:
3199``If it's not backwards, it's not compatible''. We strongly discourage its use.
3200Its current behavior is only well-defined on Unix platforms
3201(which include MacOS X and cygwin). On Windows, anything goes.
3202
3203Instead we recommend the use of such a function as
3204@code{xcvb-driver:run-program/}
3205from the @code{xcvb-driver} system that is distributed with XCVB:
3206@url{http://common-lisp.net/project/xcvb}.
3207It's the only alternative that supports
3208as many implementations and operating systems as ASDF does,
3209and provides well-defined behavior outside Unix (i.e. on Windows).
3210(The only unsupported exception is Genera, since on it
3211@code{run-shell-command} doesn't make sense anyway on that platform).
3212
3213@code{run-shell-command} takes as arguments a @code{format} control-string
3214and arguments to be passed to @code{format} after this control-string
3215to produce a string.
3216This string is a command that will be evaluated with a POSIX shell if possible;
3217yet, on Windows, some implementations will use CMD.EXE,
3218while others (like SBCL) will make an attempt at invoking a POSIX shell
3219(and fail if it is not present).
3220@end defun
3221
3222
3223@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
3224@comment  node-name,  next,  previous,  up
3225@chapter Getting the latest version
3226
3227Decide which version you want.
3228The @code{master} branch is where development happens;
3229its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
3230but an occasional regression may happen despite our (limited) test suite.
3231
3232The @code{release} branch is what cautious people should be using;
3233it has usually been tested more, and releases are cut at a point
3234where there isn't any known unresolved issue.
3235
3236You may get the ASDF source repository using git:
3237@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
3238
3239You will find the above referenced tags in this repository.
3240You can also browse the repository on
3241@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
3242
3243Discussion of ASDF development is conducted on the
3244mailing list
3245@kbd{asdf-devel@@common-lisp.net}.
3246@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
3247
3248
3249@node FAQ, TODO list, Getting the latest version, Top
3250@comment  node-name,  next,  previous,  up
3251@chapter FAQ
3252
3253@section  ``Where do I report a bug?''
3254
3255ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
3256
3257If you're unsure about whether something is a bug, or for general discussion,
3258use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
3259
3260
3261@section ``What has changed between ASDF 1 and ASDF 2?''
3262
3263@subsection What are ASDF 1 and ASDF 2?
3264
3265On May 31st 2010, we have released ASDF 2.
3266ASDF 2 refers to release 2.000 and later.
3267(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
3268ASDF 1 to any release earlier than 1.369 or so.
3269If your ASDF doesn't sport a version, it's an old ASDF 1.
3270
3271ASDF 2 and its release candidates push
3272@code{:asdf2} onto @code{*features*} so that if you are writing
3273ASDF-dependent code you may check for this feature
3274to see if the new API is present.
3275@emph{All} versions of ASDF should have the @code{:asdf} feature.
3276
3277Additionally, all versions of ASDF 2
3278define a function @code{(asdf:asdf-version)} you may use to query the version;
3279and the source code of recent versions of ASDF 2 features the version number
3280prominently on the second line of its source code.
3281
3282If you are experiencing problems or limitations of any sort with ASDF 1,
3283we recommend that you should upgrade to ASDF 2,
3284or whatever is the latest release.
3285
3286
3287@subsection ASDF can portably name files in subdirectories
3288
3289Common Lisp namestrings are not portable,
3290except maybe for logical pathnamestrings,
3291that themselves have various limitations and require a lot of setup
3292that is itself ultimately non-portable.
3293
3294In ASDF 1, the only portable ways to refer to pathnames inside systems and components
3295were very awkward, using @code{#.(make-pathname ...)} and
3296@code{#.(merge-pathnames ...)}.
3297Even the above were themselves were inadequate in the general case
3298due to host and device issues, unless horribly complex patterns were used.
3299Plenty of simple cases that looked portable actually weren't,
3300leading to much confusion and greavance.
3301
3302ASDF 2 implements its own portable syntax for strings as pathname specifiers.
3303Naming files within a system definition becomes easy and portable again.
3304@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
3305@code{merge-pathnames*},
3306@code{coerce-pathname}.
3307
3308On the other hand, there are places where systems used to accept namestrings
3309where you must now use an explicit pathname object:
3310@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3311must now be written with the @code{#p} syntax:
3312@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3313
3314@xref{The defsystem grammar,,Pathname specifiers}.
3315
3316
3317@subsection Output translations
3318
3319A popular feature added to ASDF was output pathname translation:
3320@code{asdf-binary-locations}, @code{common-lisp-controller},
3321@code{cl-launch} and other hacks were all implementing it in ways
3322both mutually incompatible and difficult to configure.
3323
3324Output pathname translation is essential to share
3325source directories of portable systems across multiple implementations
3326or variants thereof,
3327or source directories of shared installations of systems across multiple users,
3328or combinations of the above.
3329
3330In ASDF 2, a standard mechanism is provided for that,
3331@code{asdf-output-translations},
3332with sensible defaults, adequate configuration languages,
3333a coherent set of configuration files and hooks,
3334and support for non-Unix platforms.
3335
3336@xref{Controlling where ASDF saves compiled files}.
3337
3338@subsection Source Registry Configuration
3339
3340Configuring ASDF used to require special magic
3341to be applied just at the right moment,
3342between the moment ASDF is loaded and the moment it is used,
3343in a way that is specific to the user,
3344the implementation he is using and the application he is building.
3345
3346This made for awkward configuration files and startup scripts
3347that could not be shared between users, managed by administrators
3348or packaged by distributions.
3349
3350ASDF 2 provides a well-documented way to configure ASDF,
3351with sensible defaults, adequate configuration languages,
3352and a coherent set of configuration files and hooks.
3353
3354We believe it's a vast improvement because it decouples
3355application distribution from library distribution.
3356The application writer can avoid thinking where the libraries are,
3357and the library distributor (dpkg, clbuild, advanced user, etc.)
3358can configure them once and for every application.
3359Yet settings can be easily overridden where needed,
3360so whoever needs control has exactly as much as required.
3361
3362At the same time, ASDF 2 remains compatible
3363with the old magic you may have in your build scripts
3364(using @code{*central-registry*} and
3365@code{*system-definition-search-functions*})
3366to tailor the ASDF configuration to your build automation needs,
3367and also allows for new magic, simpler and more powerful magic.
3368
3369@xref{Controlling where ASDF searches for systems}.
3370
3371
3372@subsection Usual operations are made easier to the user
3373
3374In ASDF 1, you had to use the awkward syntax
3375@code{(asdf:oos 'asdf:load-op :foo)}
3376to load a system,
3377and similarly for @code{compile-op}, @code{test-op}.
3378
3379In ASDF 2, you can use shortcuts for the usual operations:
3380@code{(asdf:load-system :foo)}, and
3381similarly for @code{compile-system}, @code{test-system}.
3382
3383
3384@subsection Many bugs have been fixed
3385
3386The following issues and many others have been fixed:
3387
3388@itemize
3389@item
3390The infamous TRAVERSE function has been revamped significantly,
3391with many bugs squashed.
3392In particular, dependencies were not correctly propagated
3393across submodules within a system but now are.
3394The :version and :feature features and
3395the :force (system1 .. systemN) feature have been fixed.
3396
3397@item
3398Performance has been notably improved for large systems
3399(say with thousands of components) by using
3400hash-tables instead of linear search,
3401and linear-time list accumulation
3402instead of quadratic-time recursive appends.
3403
3404@item
3405Many features used to not be portable,
3406especially where pathnames were involved.
3407Windows support was notably quirky because of such non-portability.
3408
3409@item
3410The internal test suite used to massively fail on many implementations.
3411While still incomplete, it now fully passes
3412on all implementations supported by the test suite,
3413except for GCL (due to GCL bugs).
3414
3415@item
3416Support was lacking for some implementations.
3417ABCL and GCL were notably wholly broken.
3418ECL extensions were not integrated in the ASDF release.
3419
3420@item
3421The documentation was grossly out of date.
3422
3423@end itemize
3424
3425
3426@subsection ASDF itself is versioned
3427
3428Between new features, old bugs fixed, and new bugs introduced,
3429there were various releases of ASDF in the wild,
3430and no simple way to check which release had which feature set.
3431People using or writing systems had to either make worst-case assumptions
3432as to what features were available and worked,
3433or take great pains to have the correct version of ASDF installed.
3434
3435With ASDF 2, we provide a new stable set of working features
3436that everyone can rely on from now on.
3437Use @code{#+asdf2} to detect presence of ASDF 2,
3438@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3439to check the availability of a version no earlier than required.
3440
3441
3442@subsection ASDF can be upgraded
3443
3444When an old version of ASDF was loaded,
3445it was very hard to upgrade ASDF in your current image
3446without breaking everything.
3447Instead you had to exit the Lisp process and
3448somehow arrange to start a new one from a simpler image.
3449Something that can't be done from within Lisp,
3450making automation of it difficult,
3451which compounded with difficulty in configuration,
3452made the task quite hard.
3453Yet as we saw before, the task would have been required
3454to not have to live with the worst case or non-portable
3455subset of ASDF features.
3456
3457With ASDF 2, it is easy to upgrade
3458from ASDF 2 to later versions from within Lisp,
3459and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3460We support hot upgrade of ASDF and any breakage is a bug
3461that we will do our best to fix.
3462There are still limitations on upgrade, though,
3463most notably the fact that after you upgrade ASDF,
3464you must also reload or upgrade all ASDF extensions.
3465
3466@subsection Decoupled release cycle
3467
3468When vendors were releasing their Lisp implementations with ASDF,
3469they had to basically never change version
3470because neither upgrade nor downgrade was possible
3471without breaking something for someone,
3472and no obvious upgrade path was visible and recommendable.
3473
3474With ASDF 2, upgrade is possible, easy and can be recommended.
3475This means that vendors can safely ship a recent version of ASDF,
3476confident that if a user isn't fully satisfied,
3477he can easily upgrade ASDF and deal
3478with a supported recent version of it.
3479This means that release cycles will be causally decoupled,
3480the practical consequence of which will mean faster convergence
3481towards the latest version for everyone.
3482
3483
3484@subsection Pitfalls of the transition to ASDF 2
3485
3486The main pitfalls in upgrading to ASDF 2 seem to be related
3487to the output translation mechanism.
3488
3489@itemize
3490
3491@item
3492Output translations is enabled by default. This may surprise some users,
3493most of them in pleasant way (we hope), a few of them in an unpleasant way.
3494It is trivial to disable output translations.
3495@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3496
3497@item
3498Some systems in the large have been known
3499not to play well with output translations.
3500They were relatively easy to fix.
3501Once again, it is also easy to disable output translations,
3502or to override its configuration.
3503
3504@item
3505The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3506They replace A-B-L, and there is compatibility mode to emulate
3507your previous A-B-L configuration.
3508See @code{enable-asdf-binary-locations-compatibility} in
3509@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3510But thou shalt not load ABL on top of ASDF 2.
3511
3512@end itemize
3513
3514Other issues include the following:
3515
3516@itemize
3517
3518@item
3519ASDF pathname designators are now specified
3520in places where they were unspecified,
3521and a few small adjustments have to be made to some non-portable defsystems.
3522Notably, in the @code{:pathname} argument
3523to a @code{defsystem} and its components,
3524a logical pathname (or implementation-dependent hierarchical pathname)
3525must now be specified with @code{#p} syntax
3526where the namestring might have previously sufficed;
3527moreover when evaluation is desired @code{#.} must be used,
3528where it wasn't necessary in the toplevel @code{:pathname} argument
3529(but necessary in other @code{:pathname} arguments).
3530
3531@item
3532There is a slight performance bug, notably on SBCL,
3533when initially searching for @file{asd} files,
3534the implicit @code{(directory "/configured/path/**/*.asd")}
3535for every configured path @code{(:tree "/configured/path/")}
3536in your @code{source-registry} configuration can cause a slight pause.
3537Try to @code{(time (asdf:initialize-source-registry))}
3538to see how bad it is or isn't on your system.
3539If you insist on not having this pause,
3540you can avoid the pause by overriding the default source-registry configuration
3541and not use any deep @code{:tree} entry but only @code{:directory} entries
3542or shallow @code{:tree} entries.
3543Or you can fix your implementation to not be quite that slow
3544when recursing through directories.
3545@emph{Update}: This performance bug fixed the hard way in 2.010.
3546
3547@item
3548On Windows, only LispWorks supports proper default configuration pathnames
3549based on the Windows registry.
3550Other implementations make do with environment variables,
3551that you may have to define yourself
3552if you're using an older version of Windows.
3553Windows support is somewhat less tested than Unix support.
3554Please help report and fix bugs.
3555@emph{Update}: As of ASDF 2.21, all implementations
3556should now use the same proper default configuration pathnames
3557and they should actually work, though they haven't all been tested.
3558
3559@item
3560The mechanism by which one customizes a system so that Lisp files
3561may use a different extension from the default @file{.lisp} has changed.
3562Previously, the pathname for a component
3563was lazily computed when operating on a system,
3564and you would
3565@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3566  (declare (ignorable component system)) "lis")}.
3567Now, the pathname for a component is eagerly computed when defining the system,
3568and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3569and use @code{:default-component-class cl-source-file.lis}
3570as argument to @code{defsystem},
3571as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3572
3573@findex source-file-type
3574
3575
3576@end itemize
3577
3578
3579@section Issues with installing the proper version of ASDF
3580
3581@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3582
3583We recommend you upgrade ASDF.
3584@xref{Loading ASDF,,Upgrading ASDF}.
3585
3586If this does not work, it is a bug, and you should report it.
3587@xref{FAQ, report-bugs, Where do I report a bug}.
3588In the meantime, you can load @file{asdf.lisp} directly.
3589@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3590
3591
3592@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3593
3594Starting with current candidate releases of ASDF 2,
3595it should always be a good time to upgrade to a recent ASDF.
3596You may consult with the maintainer for which specific version they recommend,
3597but the latest @code{release} should be correct.
3598We trust you to thoroughly test it with your implementation
3599before you release it.
3600If there are any issues with the current release,
3601it's a bug that you should report upstream and that we will fix ASAP.
3602
3603As to how to include ASDF, we recommend the following:
3604
3605@itemize
3606@item
3607If ASDF isn't loaded yet, then @code{(require "asdf")}
3608should load the version of ASDF that is bundled with your system.
3609If possible so should @code{(require "ASDF")}.
3610You may have it load some other version configured by the user,
3611if you allow such configuration.
3612
3613@item
3614If your system provides a mechanism to hook into @code{CL:REQUIRE},
3615then it would be nice to add ASDF to this hook the same way that
3616ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3617Please send us appropriate code to this end.
3618
3619@item
3620You may, like SBCL, have ASDF be implicitly used to require systems
3621that are bundled with your Lisp distribution.
3622If you do have a few magic systems that come with your implementation
3623in a precompiled way such that one should only use the binary version
3624that goes with your distribution, like SBCL does,
3625then you should add them in the beginning of @code{wrapping-source-registry}.
3626
3627@item
3628If you have magic systems as above, like SBCL does,
3629then we explicitly ask you to @emph{NOT} distribute
3630@file{asdf.asd} as part of those magic systems.
3631You should still include the file @file{asdf.lisp} in your source distribution
3632and precompile it in your binary distribution,
3633but @file{asdf.asd} if included at all,
3634should be secluded from the magic systems,
3635in a separate file hierarchy;
3636alternatively, you may provide the system
3637after renaming it and its @file{.asd} file to e.g.
3638@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3639@code{sb-asdf} and @file{sb-asdf.asd}.
3640Indeed, if you made @file{asdf.asd} a magic system,
3641then users would no longer be able to upgrade ASDF using ASDF itself
3642to some version of their preference that
3643they maintain independently from your Lisp distribution.
3644
3645@item
3646If you do not have any such magic systems, or have other non-magic systems
3647that you want to bundle with your implementation,
3648then you may add them to the @code{default-source-registry},
3649and you are welcome to include @file{asdf.asd} amongst them.
3650
3651@item
3652Please send us upstream any patches you make to ASDF itself,
3653so we can merge them back in for the benefit of your users
3654when they upgrade to the upstream version.
3655
3656@end itemize
3657
3658
3659
3660@section Issues with configuring ASDF
3661
3662@subsection ``How can I customize where fasl files are stored?''
3663
3664@xref{Controlling where ASDF saves compiled files}.
3665
3666Note that in the past there was an add-on to ASDF called
3667@code{ASDF-binary-locations}, developed by Gary King.
3668That add-on has been merged into ASDF proper,
3669then superseded by the @code{asdf-output-translations} facility.
3670
3671Note that use of @code{asdf-output-translations}
3672can interfere with one aspect of your systems
3673--- if your system uses @code{*load-truename*} to find files
3674(e.g., if you have some data files stored with your program),
3675then the relocation that this ASDF customization performs
3676is likely to interfere.
3677Use @code{asdf:system-relative-pathname} to locate a file
3678in the source directory of some system, and
3679use @code{asdf:apply-output-translations} to locate a file
3680whose pathname has been translated by the facility.
3681
3682@subsection ``How can I wholly disable the compiler output cache?''
3683
3684To permanently disable the compiler output cache
3685for all future runs of ASDF, you can:
3686
3687@example
3688mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3689echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3690@end example
3691
3692This assumes that you didn't otherwise configure the ASDF files
3693(if you did, edit them again),
3694and don't somehow override the configuration at runtime
3695with a shell variable (see below) or some other runtime command
3696(e.g. some call to @code{asdf:initialize-output-translations}).
3697
3698To disable the compiler output cache in Lisp processes
3699run by your current shell, try (assuming @code{bash} or @code{zsh})
3700(on Unix and cygwin only):
3701
3702@example
3703export ASDF_OUTPUT_TRANSLATIONS=/:
3704@end example
3705
3706To disable the compiler output cache just in the current Lisp process,
3707use (after loading ASDF but before using it):
3708
3709@example
3710(asdf:disable-output-translations)
3711@end example
3712
3713@section Issues with using and extending ASDF to define systems
3714
3715@subsection ``How can I cater for unit-testing in my system?''
3716
3717ASDF provides a predefined test operation, @code{test-op}.
3718@xref{Predefined operations of ASDF, test-op}.
3719The test operation, however, is largely left to the system definer to specify.
3720@code{test-op} has been
3721a topic of considerable discussion on the
3722@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3723and on the
3724@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3725
3726Here are some guidelines:
3727
3728@itemize
3729@item
3730For a given system, @var{foo}, you will want to define a corresponding
3731test system, such as @var{foo-test}.  The reason that you will want this
3732separate system is that ASDF does not out of the box supply components
3733that are conditionally loaded.  So if you want to have source files
3734(with the test definitions) that will not be loaded except when testing,
3735they should be put elsewhere.
3736
3737@item
3738The @var{foo-test} system can be defined in an asd file of its own or
3739together with @var{foo}.  An aesthetic preference against cluttering up
3740the filesystem with extra asd files should be balanced against the
3741question of whether one might want to directly load @var{foo-test}.
3742Typically one would not want to do this except in early stages of
3743debugging.
3744
3745@item
3746Record that testing is implemented by @var{foo-test}.  For example:
3747@example
3748(defsystem @var{foo}
3749   :in-order-to ((test-op (test-op @var{foo-test})))
3750   ....)
3751
3752(defsystem @var{foo-test}
3753   :depends-on (@var{foo} @var{my-test-library} ...)
3754   ....)
3755@end example
3756@end itemize
3757
3758This procedure will allow you to support users who do not wish to
3759install your test framework.
3760
3761One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3762does not return a value.  So in current versions of ASDF there is no
3763reliable programmatic means of determining whether or not a set of tests
3764has passed, or which tests have failed.  The user must simply read the
3765console output.  This limitation has been the subject of much
3766discussion.
3767
3768@subsection ``How can I cater for documentation generation in my system?''
3769
3770The ASDF developers are currently working to add a @code{doc-op}
3771to the set of predefined ASDF operations.
3772@xref{Predefined operations of ASDF}.
3773See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3774
3775
3776
3777@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3778
3779See @code{cffi}'s @code{cffi-grovel}.
3780
3781@anchor{report-bugs}
3782
3783
3784@subsection ``I want to put my module's files at the top level.  How do I do this?''
3785
3786By default, the files contained in an asdf module go
3787in a subdirectory with the same name as the module.
3788However, this can be overridden by adding a @code{:pathname ""} argument
3789to the module description.
3790For example, here is how it could be done
3791in the spatial-trees ASDF system definition for ASDF 2:
3792
3793@example
3794(asdf:defsystem :spatial-trees
3795  :components
3796  ((:module base
3797            :pathname ""
3798            :components
3799            ((:file "package")
3800             (:file "basedefs" :depends-on ("package"))
3801             (:file "rectangles" :depends-on ("package"))))
3802   (:module tree-impls
3803            :depends-on (base)
3804            :pathname ""
3805            :components
3806            ((:file "r-trees")
3807             (:file "greene-trees" :depends-on ("r-trees"))
3808             (:file "rstar-trees" :depends-on ("r-trees"))
3809             (:file "rplus-trees" :depends-on ("r-trees"))
3810             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3811   (:module viz
3812            :depends-on (base)
3813            :pathname ""
3814            :components
3815            ((:static-file "spatial-tree-viz.lisp")))
3816   (:module tests
3817            :depends-on (base)
3818            :pathname ""
3819            :components
3820            ((:static-file "spatial-tree-test.lisp")))
3821   (:static-file "LICENCE")
3822   (:static-file "TODO")))
3823@end example
3824
3825All of the files in the @code{tree-impls} module are at the top level,
3826instead of in a @file{tree-impls/} subdirectory.
3827
3828Note that the argument to @code{:pathname} can be either a pathname object or a string.
3829A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3830but this is discouraged because the results of parsing a namestring are not portable.
3831A pathname can only be portably constructed with such syntax as
3832@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3833and similarly the current directory can only be portably specified as
3834@code{#.(make-pathname :directory '(:relative))}.
3835However, as of ASDF 2, you can portably use a string to denote a pathname.
3836The string will be parsed as a @code{/}-separated path from the current directory,
3837such that the empty string @code{""} denotes the current directory, and
3838@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3839portably denotes the same subdirectory as above.
3840When files are specified, the last @code{/}-separated component is interpreted
3841either as the name component of a pathname
3842(if the component class specifies a pathname type),
3843or as a name component plus optional dot-separated type component
3844(if the component class doesn't specifies a pathname type).
3845
3846@subsection How do I create a system definition where all the source files have a .cl extension?
3847
3848Starting with ASDF 2.014.14, you may just pass
3849the builtin class @code{cl-source-file.cl} as
3850the @code{:default-component-class} argument to @code{defsystem}:
3851
3852@lisp
3853(defsystem my-cl-system
3854  :default-component-class cl-source-file.cl
3855  ...)
3856@end lisp
3857
3858Another builtin class @code{cl-source-file.lsp} is offered
3859for files ending in @file{.lsp}.
3860
3861If you want to use a different extension
3862for which ASDF doesn't provide builtin support,
3863or want to support versions of ASDF
3864earlier than 2.014.14 (but later than 2.000),
3865you can define a class as follows:
3866
3867@lisp
3868;; Prologue: make sure we're using a sane package.
3869(defpackage :my-asdf-extension
3870   (:use :asdf :common-lisp)
3871   (:export #:cl-source-file.lis))
3872(in-package :my-asdf-extension)
3873
3874(defclass cl-source-file.lis (cl-source-file)
3875   ((type :initform "lis")))
3876@end lisp
3877
3878Then you can use it as follows:
3879@lisp
3880(defsystem my-cl-system
3881  :default-component-class my-asdf-extension:cl-source-file.lis
3882  ...)
3883@end lisp
3884
3885Of course, if you're in the same package, e.g. in the same file,
3886you won't need to use the package qualifier before @code{cl-source-file.lis}.
3887Actually, if all you're doing is defining this class
3888and using it in the same file without other fancy definitions,
3889you might skip package complications:
3890
3891@lisp
3892(in-package :asdf)
3893(defclass cl-source-file.lis (cl-source-file)
3894   ((type :initform "lis")))
3895(defsystem my-cl-system
3896  :default-component-class cl-source-file.lis
3897  ...)
3898@end lisp
3899
3900It is possible to achieve the same effect
3901in a way that supports both ASDF 1 and ASDF 2,
3902but really, friends don't let friends use ASDF 1.
3903Please upgrade to ASDF 2.
3904In short, though: do same as above, but
3905@emph{before} you use the class in a @code{defsystem},
3906you also define the following method:
3907
3908@lisp
3909(defmethod source-file-type ((f cl-source-file.lis) (m module))
3910  (declare (ignorable f m))
3911  "lis")
3912@end lisp
3913
3914@comment FIXME: Add a FAQ about how to use a new system class...
3915
3916
3917@node  TODO list, Inspiration, FAQ, Top
3918@comment  node-name,  next,  previous,  up
3919@chapter TODO list
3920
3921Here is an old list of things to do,
3922in addition to the bugs that are now tracked on launchpad:
3923@url{https://launchpad.net/asdf}.
3924
3925@section Outstanding spec questions, things to add
3926
3927** packaging systems
3928
3929*** manual page component?
3930
3931** style guide for .asd files
3932
3933You should either use keywords or be careful
3934with the package that you evaluate defsystem forms in.
3935Otherwise @code{(defsystem partition ...)}
3936being read in the @code{cl-user} package
3937will intern a @code{cl-user:partition} symbol,
3938which will then collide with the @code{partition:partition} symbol.
3939
3940Actually there's a hairier packages problem to think about too.
3941@code{in-order-to} is not a keyword:
3942if you read @code{defsystem} forms in a package that doesn't use ASDF,
3943odd things might happen.
3944
3945
3946** extending defsystem with new options
3947
3948You might not want to write a whole parser,
3949but just to add options to the existing syntax.
3950Reinstate @code{parse-option} or something akin.
3951
3952
3953** Diagnostics
3954
3955A ``dry run'' of an operation can be made with the following form:
3956
3957@lisp
3958(let ((asdf::*verbose-out* *standard-output*))
3959  (loop :for (op . comp) :in
3960    (asdf::traverse (make-instance '<operation-name> :force t)
3961                    (asdf:find-system <system-name>))
3962    :do (asdf:explain op comp)))
3963@end lisp
3964
3965This uses unexported symbols.
3966What would be a nice interface for this functionality?
3967
3968@section Missing bits in implementation
3969
3970** reuse the same scratch package whenever a system is reloaded from disk
3971
3972Have a package ASDF-USER instead of all these temporary packages?
3973
3974** proclamations probably aren't
3975
3976** (stuff that might happen later)
3977
3978*** Propagation of the @code{:force} option.
3979
3980``I notice that
3981
3982        @code{(asdf:compile-system :araneida :force t)}
3983
3984also forces compilation of every other system
3985the @code{:araneida} system depends on.
3986This is rarely useful to me;
3987usually, when I want to force recompilation
3988of something more than a single source file,
3989I want to recompile only one system.
3990So it would be more useful to have @code{make-sub-operation}
3991refuse to propagate @code{:force t} to other systems, and
3992propagate only something like @code{:force :recursively}.
3993
3994Ideally what we actually want is some kind of criterion that says
3995to which systems (and which operations) a @code{:force} switch will propagate.
3996
3997The problem is perhaps that ``force'' is a pretty meaningless concept.
3998How obvious is it that @code{load :force t} should force @emph{compilation}?
3999But we don't really have the right dependency setup
4000for the user to compile @code{:force t} and expect it to work
4001(files will not be loaded after compilation, so the compile
4002environment for subsequent files will be emptier than it needs to be)
4003
4004What does the user actually want to do when he forces?
4005Usually, for me, update for use with a new version of the Lisp compiler.
4006Perhaps for recovery when he suspects that something has gone wrong.
4007Or else when he's changed compilation options or configuration
4008in some way that's not reflected in the dependency graph.
4009
4010Other possible interface: have a ``revert'' function akin to @code{make clean}.
4011
4012@lisp
4013(asdf:revert 'asdf:compile-op 'araneida)
4014@end lisp
4015
4016would delete any files produced by @code{(compile-system :araneida)}.
4017Of course, it wouldn't be able to do much about stuff in the image itself.
4018
4019How would this work?
4020
4021@code{traverse}
4022
4023There's a difference between a module's dependencies (peers)
4024and its components (children).
4025Perhaps there's a similar difference in operations?
4026For example, @code{(load "use") depends-on (load "macros")} is a peer,
4027whereas @code{(load "use") depends-on (compile "use")}
4028is more of a ``subservient'' relationship.
4029
4030@node  Inspiration, Concept Index, TODO list, Top
4031@comment  node-name,  next,  previous,  up
4032@chapter Inspiration
4033
4034@section mk-defsystem (defsystem-3.x)
4035
4036We aim to solve basically the same problems as @code{mk-defsystem} does.
4037However, our architecture for extensibility
4038better exploits CL language features (and is documented),
4039and we intend to be portable rather than just widely-ported.
4040No slight on the @code{mk-defsystem} authors and maintainers is intended here;
4041that implementation has the unenviable task
4042of supporting pre-ANSI implementations, which is no longer necessary.
4043
4044The surface defsystem syntax of asdf is more-or-less compatible with
4045@code{mk-defsystem}, except that we do not support
4046the @code{source-foo} and @code{binary-foo} prefixes
4047for separating source and binary files, and
4048we advise the removal of all options to specify pathnames.
4049
4050The @code{mk-defsystem} code for topologically sorting
4051a module's dependency list was very useful.
4052
4053@section defsystem-4 proposal
4054
4055Marco and Peter's proposal for defsystem 4 served as the driver for
4056many of the features in here.  Notable differences are:
4057
4058@itemize
4059@item
4060We don't specify output files or output file extensions
4061as part of the system.
4062
4063If you want to find out what files an operation would create,
4064ask the operation.
4065
4066@item
4067We don't deal with CL packages
4068
4069If you want to compile in a particular package, use an @code{in-package} form
4070in that file (ilisp / SLIME will like you more if you do this anyway)
4071
4072@item
4073There is no proposal here that @code{defsystem} does version control.
4074
4075A system has a given version which can be used to check dependencies,
4076but that's all.
4077@end itemize
4078
4079The defsystem 4 proposal tends to look more at the external features,
4080whereas this one centres on a protocol for system introspection.
4081
4082@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
4083
4084Available in updated-for-CL form on the web at
4085@url{http://nhplace.com/kent/Papers/Large-Systems.html}
4086
4087In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
4088and concept to deal with creating component trees
4089from @code{defsystem} surface syntax.
4090[ this is not true right now, though it used to be and
4091probably will be again soon ]
4092
4093
4094@c -------------------
4095
4096
4097@node Concept Index, Function and Class Index, Inspiration, Top
4098@unnumbered Concept Index
4099
4100@printindex cp
4101
4102@node Function and Class Index, Variable Index, Concept Index, Top
4103@unnumbered Function and Class Index
4104
4105@printindex fn
4106
4107@node Variable Index,  , Function and Class Index, Top
4108@unnumbered Variable Index
4109
4110@printindex vr
4111
4112@bye
Note: See TracBrowser for help on using the repository browser.