source: branches/1.0.x/abcl/doc/asdf/asdf.texinfo @ 13656

Last change on this file since 13656 was 13656, checked in by Mark Evenson, 10 years ago

Upgrade to ASDF 2.0.17.022

File size: 141.3 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2011 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex compile-system
201@findex test-system
202@cindex system directory designator
203@findex operate
204@findex oos
205
206@c @menu
207@c * Installing ASDF::
208@c @end menu
209
210
211@section Loading a pre-installed ASDF
212
213Many Lisp implementations include a copy of ASDF.
214You can usually load this copy using Common Lisp's @code{require} function:
215
216@lisp
217(require "asdf")
218@end lisp
219
220As of the writing of this manual,
221the following implementations provide ASDF 2 this way:
222abcl allegro ccl clisp cmucl ecl sbcl xcl.
223The following implementations don't provide it yet but will in a future release:
224lispworks scl.
225The following implementations are obsolete and most probably will never bundle it:
226cormancl gcl genera mcl.
227
228If the implementation you are using doesn't provide ASDF 2,
229see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
230If that implementation is still actively maintained,
231you may also send a bug report to your Lisp vendor and complain
232about their failing to provide ASDF.
233
234NB: all implementations except clisp also accept
235@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
236For portability's sake, you probably want to use @code{(require "asdf")}.
237
238
239@section Checking whether ASDF is loaded
240
241To check whether ASDF is properly loaded in your current Lisp image,
242you can run this form:
243
244@lisp
245(asdf:asdf-version)
246@end lisp
247
248If it returns a string,
249that is the version of ASDF that is currently installed.
250
251If it raises an error,
252then either ASDF is not loaded, or
253you are using an old version of ASDF.
254
255You can check whether an old version is loaded
256by checking if the ASDF package is present.
257The form below will allow you to programmatically determine
258whether a recent version is loaded, an old version is loaded,
259or none at all:
260
261@lisp
262(or #+asdf2 (asdf:asdf-version) #+asdf :old)
263@end lisp
264
265If it returns a version number, that's the version of ASDF installed.
266If it returns the keyword @code{:OLD},
267then you're using an old version of ASDF (from before 1.635).
268If it returns @code{NIL} then ASDF is not installed.
269
270If you are experiencing problems with ASDF,
271please try upgrading to the latest released version,
272using the method below,
273before you contact us and raise an issue.
274
275
276@section Upgrading ASDF
277
278If your implementation does provide ASDF 2 or later,
279and you want to upgrade to a more recent version,
280just install ASDF like any other package
281(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
282configure ASDF as usual (see @pxref{Configuring ASDF} below),
283and upgrade with:
284
285@lisp
286(require "asdf")
287(asdf:load-system :asdf)
288@end lisp
289
290If on the other hand, your implementation only provides an old ASDF,
291you will require a special configuration step and an old-style loading:
292
293@lisp
294(require "asdf")
295(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
296(asdf:oos 'asdf:load-op :asdf)
297@end lisp
298
299Don't forget the trailing @code{/} at the end of your pathname.
300
301Also, note that older versions of ASDF won't redirect their output,
302or at least won't do it according to your usual ASDF 2 configuration.
303You therefore need write access on the directory
304where you install the new ASDF,
305and make sure you're not using it
306for multiple mutually incompatible implementations.
307At worst, you may have to have multiple copies of the new ASDF,
308e.g. one per implementation installation, to avoid clashes.
309Note that to our knowledge all implementations that provide ASDF
310provide ASDF 2 in their latest release, so
311you may want to upgrade your implementation rather than go through that hoop.
312
313Finally, note that there are some limitations to upgrading ASDF:
314@itemize
315@item
316Any ASDF extension becomes invalid, and will need to be reloaded.
317This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
318Starting with ASDF 2.014.8, ASDF will actually invalidate
319all previously loaded systems when it is loaded on top of
320a different ASDF version.
321@item
322Until all implementations provide ASDF 2.015 or later,
323it is safer if you upgrade ASDF and its extensions as a special step
324at the very beginning of whatever script you are running,
325before you start using ASDF to load anything else.
326@item
327Until all implementations provide ASDF 2.015 or later,
328it is unsafe to upgrade ASDF as part of loading a system
329that depends on a more recent version of ASDF,
330since the new one might shadow the old one while the old one is running,
331and the running old one will be confused
332when extensions are loaded into the new one.
333In the meantime, we recommend that your systems should @emph{not} specify
334@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
335but instead that they check that a recent enough ASDF is installed,
336with such code as:
337@example
338(unless (or #+asdf2 (asdf:version-satisfies
339                     (asdf:asdf-version) *required-asdf-version*))
340  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
341@end example
342@end itemize
343
344
345@section Loading an otherwise installed ASDF
346
347If your implementation doesn't include ASDF,
348if for some reason the upgrade somehow fails,
349does not or cannot apply to your case,
350you will have to install the file @file{asdf.lisp}
351somewhere and load it with:
352
353@lisp
354(load "/path/to/your/installed/asdf.lisp")
355@end lisp
356
357The single file @file{asdf.lisp} is all you normally need to use ASDF.
358
359You can extract this file from latest release tarball on the
360@url{http://common-lisp.net/project/asdf/,ASDF website}.
361If you are daring and willing to report bugs, you can get
362the latest and greatest version of ASDF from its git repository.
363@xref{Getting the latest version}.
364
365For maximum convenience you might want to have ASDF loaded
366whenever you start your Lisp implementation,
367for example by loading it from the startup script or dumping a custom core
368--- check your Lisp implementation's manual for details.
369
370
371@node Configuring ASDF, Using ASDF, Loading ASDF, Top
372@comment  node-name,  next,  previous,  up
373
374@chapter Configuring ASDF
375
376@section Configuring ASDF to find your systems
377
378So it may compile and load your systems, ASDF must be configured to find
379the @file{.asd} files that contain system definitions.
380
381Since ASDF 2, the preferred way to configure where ASDF finds your systems is
382the @code{source-registry} facility,
383fully described in its own chapter of this manual.
384@xref{Controlling where ASDF searches for systems}.
385
386The default location for a user to install Common Lisp software is under
387@file{~/.local/share/common-lisp/source/}.
388If you install software there, you don't need further configuration.
389If you're installing software yourself at a location that isn't standard,
390you have to tell ASDF where you installed it. See below.
391If you're using some tool to install software (e.g. Quicklisp),
392the authors of that tool should already have configured ASDF.
393
394The simplest way to add a path to your search path,
395say @file{/home/luser/.asd-link-farm/}
396is to create the directory
397@file{~/.config/common-lisp/source-registry.conf.d/}
398and there create a file with any name of your choice,
399and with the type @file{conf},
400for instance @file{42-asd-link-farm.conf}
401containing the line:
402
403@kbd{(:directory "/home/luser/.asd-link-farm/")}
404
405If you want all the subdirectories under @file{/home/luser/lisp/}
406to be recursively scanned for @file{.asd} files, instead use:
407
408@kbd{(:tree "/home/luser/lisp/")}
409
410Note that your Operating System distribution or your system administrator
411may already have configured system-managed libraries for you.
412
413The required @file{.conf} extension allows you to have disabled files
414or editor backups (ending in @file{~}), and works portably
415(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
416Excluded are files the name of which start with a @file{.} character.
417It is customary to start the filename with two digits
418that specify the order in which the directories will be scanned.
419
420ASDF will automatically read your configuration
421the first time you try to find a system.
422You can reset the source-registry configuration with:
423
424@lisp
425(asdf:clear-source-registry)
426@end lisp
427
428And you probably should do so before you dump your Lisp image,
429if the configuration may change
430between the machine where you save it at the time you save it
431and the machine you resume it at the time you resume it.
432
433
434@section Configuring ASDF to find your systems --- old style
435
436The old way to configure ASDF to find your systems is by
437@code{push}ing directory pathnames onto the variable
438@code{asdf:*central-registry*}.
439
440You must configure this variable between the time you load ASDF
441and the time you first try to use it.
442Loading and configuring ASDF presumably happen
443as part of some initialization script that builds or starts
444your Common Lisp software system.
445(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
446
447The @code{asdf:*central-registry*} is empty by default in ASDF 2,
448but is still supported for compatibility with ASDF 1.
449When used, it takes precedence over the above source-registry@footnote{
450It is possible to further customize
451the system definition file search.
452That's considered advanced use, and covered later:
453search forward for
454@code{*system-definition-search-functions*}.
455@xref{Defining systems with defsystem}.}.
456
457For instance, if you wanted ASDF to find the @file{.asd} file
458@file{/home/me/src/foo/foo.asd} your initialization script
459could after it loads ASDF with @code{(require "asdf")}
460configure it with:
461
462@lisp
463(push "/home/me/src/foo/" asdf:*central-registry*)
464@end lisp
465
466Note the trailing slash: when searching for a system,
467ASDF will evaluate each entry of the central registry
468and coerce the result to a pathname@footnote{
469ASDF will indeed call @code{EVAL} on each entry.
470It will also skip entries that evaluate to @code{NIL}.
471
472Strings and pathname objects are self-evaluating,
473in which case the @code{EVAL} step does nothing;
474but you may push arbitrary SEXP onto the central registry,
475that will be evaluated to compute e.g. things that depend
476on the value of shell variables or the identity of the user.
477
478The variable @code{asdf:*central-registry*} is thus a list of
479``system directory designators''.
480A @dfn{system directory designator} is a form
481which will be evaluated whenever a system is to be found,
482and must evaluate to a directory to look in.
483By ``directory'' here, we mean
484``designator for a pathname with a supplied DIRECTORY component''.
485}
486at which point the presence of the trailing directory name separator
487is necessary to tell Lisp that you're discussing a directory
488rather than a file.
489
490Typically, however, there are a lot of @file{.asd} files, and
491a common idiom was to have to put
492a bunch of @emph{symbolic links} to @file{.asd} files
493in a common directory
494and push @emph{that} directory (the ``link farm'')
495to the
496@code{asdf:*central-registry*}
497instead of pushing each of the many involved directories
498to the @code{asdf:*central-registry*}.
499ASDF knows how to follow such @emph{symlinks}
500to the actual file location when resolving the paths of system components
501(on Windows, you can use Windows shortcuts instead of POSIX symlinks;
502if you try aliases under MacOS, we are curious to hear about your experience).
503
504For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
505is a member of @code{*central-registry*}, you could set up the
506system @var{foo} for loading with asdf with the following
507commands at the shell:
508
509@example
510$ cd /home/me/cl/systems/
511$ ln -s ~/src/foo/foo.asd .
512@end example
513
514This old style for configuring ASDF is not recommended for new users,
515but it is supported for old users, and for users who want to programmatically
516control what directories are added to the ASDF search path.
517
518
519@section Configuring where ASDF stores object files
520@findex clear-output-translations
521
522ASDF lets you configure where object files will be stored.
523Sensible defaults are provided and
524you shouldn't normally have to worry about it.
525
526This allows the same source code repository may be shared
527between several versions of several Common Lisp implementations,
528between several users using different compilation options
529and without write privileges on shared source directories, etc.
530This also allows to keep source directories uncluttered
531by plenty of object files.
532
533Starting with ASDF 2, the @code{asdf-output-translations} facility
534was added to ASDF itself, that controls where object files will be stored.
535This facility is fully described in a chapter of this manual,
536@ref{Controlling where ASDF saves compiled files}.
537
538The simplest way to add a translation to your search path,
539say from @file{/foo/bar/baz/quux/}
540to @file{/where/i/want/my/fasls/}
541is to create the directory
542@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
543and there create a file with any name of your choice and the type @file{conf},
544for instance @file{42-bazquux.conf}
545containing the line:
546
547@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
548
549To disable output translations for source under a given directory,
550say @file{/toto/tata/}
551you can create a file @file{40-disable-toto.conf}
552with the line:
553
554@kbd{("/toto/tata/")}
555
556To wholly disable output translations for all directories,
557you can create a file @file{00-disable.conf}
558with the line:
559
560@kbd{(t t)}
561
562Note that your Operating System distribution or your system administrator
563may already have configured translations for you.
564In absence of any configuration, the default is to redirect everything
565under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
566@xref{Controlling where ASDF searches for systems}, for full details.
567
568The required @file{.conf} extension allows you to have disabled files
569or editor backups (ending in @file{~}), and works portably
570(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
571Excluded are files the name of which start with a @file{.} character.
572It is customary to start the filename with two digits
573that specify the order in which the directories will be scanned.
574
575ASDF will automatically read your configuration
576the first time you try to find a system.
577You can reset the source-registry configuration with:
578
579@lisp
580(asdf:clear-output-translations)
581@end lisp
582
583And you probably should do so before you dump your Lisp image,
584if the configuration may change
585between the machine where you save it at the time you save it
586and the machine you resume it at the time you resume it.
587
588Finally note that before ASDF 2,
589other ASDF add-ons offered the same functionality,
590each in subtly different and incompatible ways:
591ASDF-Binary-Locations, cl-launch, common-lisp-controller.
592ASDF-Binary-Locations is now not needed anymore and should not be used.
593cl-launch 3.000 and common-lisp-controller 7.2 have been updated
594to just delegate this functionality to ASDF.
595
596@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
597@comment  node-name,  next,  previous,  up
598
599
600@section Resetting Configuration
601
602When you dump and restore an image, or when you tweak your configuration,
603you may want to reset the ASDF configuration.
604For that you may use the following function:
605
606@defun clear-configuration
607   undoes any ASDF configuration,
608   regarding source-registry or output-translations.
609@end defun
610
611If you use SBCL, CMUCL or SCL, you may use this snippet
612so that the ASDF configuration be cleared automatically as you dump an image:
613
614@example
615#+(or cmu sbcl scl)
616(pushnew 'clear-configuration
617         #+(or cmu scl) ext:*before-save-initializations*
618         #+sbcl sb-ext:*save-hooks*)
619@end example
620
621For compatibility with all Lisp implementations, however,
622you might want instead your build script to explicitly call
623@code{(asdf:clear-configuration)} at an appropriate moment before dumping.
624
625
626@chapter Using ASDF
627
628@section Loading a system
629
630The system @var{foo} is loaded (and compiled, if necessary)
631by evaluating the following Lisp form:
632
633@example
634(asdf:load-system :@var{foo})
635@end example
636
637On some implementations (namely recent versions of
638ABCL, Clozure CL, CLISP, CMUCL, ECL, SBCL and SCL),
639ASDF hooks into the @code{CL:REQUIRE} facility
640and you can just use:
641
642@example
643(require :@var{foo})
644@end example
645
646In older versions of ASDF, you needed to use
647@code{(asdf:oos 'asdf:load-op :@var{foo})}.
648If your ASDF is too old to provide @code{asdf:load-system} though
649we recommend that you upgrade to ASDF 2.
650@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
651
652Note the name of a system is specified as a string or a symbol,
653typically a keyword.
654If a symbol (including a keyword), its name is taken and lowercased.
655The name must be a suitable value for the @code{:name} initarg
656to @code{make-pathname} in whatever filesystem the system is to be found.
657The lower-casing-symbols behaviour is unconventional,
658but was selected after some consideration.
659Observations suggest that the type of systems we want to support
660either have lowercase as customary case (unix, mac, windows)
661or silently convert lowercase to uppercase (lpns),
662so this makes more sense than attempting to use @code{:case :common},
663which is reported not to work on some implementations
664
665
666@section Other Operations
667
668ASDF provides three commands for the most common system operations:
669@code{load-system}, @code{compile-system} or @code{test-system}.
670
671Because ASDF is an extensible system
672for defining @emph{operations} on @emph{components},
673it also provides a generic function @code{operate}
674(which is usually abbreviated by @code{oos}).
675You'll use @code{oos} whenever you want to do something beyond
676compiling, loading and testing.
677
678Output from ASDF and ASDF extensions are supposed to be sent
679to the CL stream @code{*standard-output*},
680and so rebinding that stream around calls to @code{asdf:operate}
681should redirect all output from ASDF operations.
682
683Reminder: before ASDF can operate on a system, however,
684it must be able to find and load that system's definition.
685@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
686
687
688@section Summary
689
690To use ASDF:
691
692@itemize
693@item
694Load ASDF itself into your Lisp image, either through
695@code{(require "asdf")} or else through
696@code{(load "/path/to/asdf.lisp")}.
697
698@item
699Make sure ASDF can find system definitions
700thanks to proper source-registry configuration.
701
702@item
703Load a system with @code{(load-system :my-system)}
704or use some other operation on some system of your choice.
705
706@end itemize
707
708@section Moving on
709
710That's all you need to know to use ASDF to load systems written by others.
711The rest of this manual deals with writing system definitions
712for Common Lisp software you write yourself,
713including how to extend ASDF to define new operation and component types.
714
715
716@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
717@comment  node-name,  next,  previous,  up
718@chapter Defining systems with defsystem
719
720This chapter describes how to use asdf to define systems and develop
721software.
722
723
724@menu
725* The defsystem form::
726* A more involved example::
727* The defsystem grammar::
728* Other code in .asd files::
729@end menu
730
731@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
732@comment  node-name,  next,  previous,  up
733@section The defsystem form
734
735Systems can be constructed programmatically
736by instantiating components using @code{make-instance}.
737Most of the time, however, it is much more practical to use
738a static @code{defsystem} form.
739This section begins with an example of a system definition,
740then gives the full grammar of @code{defsystem}.
741
742Let's look at a simple system.
743This is a complete file that would
744usually be saved as @file{hello-lisp.asd}:
745
746@lisp
747(in-package :asdf)
748
749(defsystem "hello-lisp"
750  :description "hello-lisp: a sample Lisp system."
751  :version "0.2.1"
752  :author "Joe User <joe@@example.com>"
753  :licence "Public Domain"
754  :components ((:file "packages")
755               (:file "macros" :depends-on ("packages"))
756               (:file "hello" :depends-on ("macros"))))
757@end lisp
758
759Some notes about this example:
760
761@itemize
762
763@item
764The file starts with an @code{in-package} form
765to use package @code{asdf}.
766You could instead start your definition by using
767a qualified name @code{asdf:defsystem}.
768
769@item
770If in addition to simply using @code{defsystem},
771you are going to define functions,
772create ASDF extension, globally bind symbols, etc.,
773it is recommended that to avoid namespace pollution between systems,
774you should create your own package for that purpose,
775for instance replacing the above @code{(in-package :asdf)} with:
776
777@lisp
778(defpackage :foo-system
779  (:use :cl :asdf))
780
781(in-package :foo-system)
782@end lisp
783
784@item
785The @code{defsystem} form defines a system named @code{hello-lisp}
786that contains three source files:
787@file{packages}, @file{macros} and @file{hello}.
788
789@item
790The file @file{macros} depends on @file{packages}
791(presumably because the package it's in is defined in @file{packages}),
792and the file @file{hello} depends on @file{macros}
793(and hence, transitively on @file{packages}).
794This means that ASDF will compile and load @file{packages} and @file{macros}
795before starting the compilation of file @file{hello}.
796
797@item
798The files are located in the same directory
799as the file with the system definition.
800ASDF resolves symbolic links (or Windows shortcuts)
801before loading the system definition file and
802stores its location in the resulting system@footnote{
803It is possible, though almost never necessary, to override this behaviour.}.
804This is a good thing because the user can move the system sources
805without having to edit the system definition.
806
807@c FIXME: Should have cross-reference to "Version specifiers" in the
808@c defsystem grammar, but the cross-referencing is so broken by
809@c insufficient node breakdown that I have not put one in.
810@item
811Make sure you know how the @code{:version} numbers will be parsed!  They
812are parsed as period-separated lists of integers.  I.e., in the example,
813@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
814In particular, version @code{0.2.1} is interpreted the same as
815@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
816even though the two are the same when interpreted as decimal fractions.
817@cindex version specifiers
818@cindex :version
819
820@end itemize
821
822@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
823@comment  node-name,  next,  previous,  up
824@section A more involved example
825
826Let's illustrate some more involved uses of @code{defsystem} via a
827slightly convoluted example:
828
829@lisp
830(defsystem "foo"
831  :version "1.0.0"
832  :components ((:module "mod"
833                            :components ((:file "bar")
834                                                  (:file"baz")
835                                                  (:file "quux"))
836                            :perform (compile-op :after (op c)
837                                                  (do-something c))
838                            :explain (compile-op :after (op c)
839                                            (explain-something c)))
840                         (:file "blah")))
841@end lisp
842
843The @code{:module} component named @code{"mod"} is a collection of three files,
844which will be located in a subdirectory of the main code directory named
845@file{mod} (this location can be overridden; see the discussion of the
846@code{:pathname} option in @ref{The defsystem grammar}).
847
848The method-form tokens provide a shorthand for defining methods on
849particular components.  This part
850
851@lisp
852                :perform (compile-op :after (op c)
853                          (do-something c))
854                :explain (compile-op :after (op c)
855                          (explain-something c))
856@end lisp
857
858has the effect of
859
860@lisp
861(defmethod perform :after ((op compile-op) (c (eql ...)))
862           (do-something c))
863(defmethod explain :after ((op compile-op) (c (eql ...)))
864           (explain-something c))
865@end lisp
866
867where @code{...} is the component in question.
868In this case @code{...} would expand to something like
869
870@lisp
871(find-component (find-system "foo") "mod")
872@end lisp
873
874For more details on the syntax of such forms, see @ref{The defsystem
875grammar}.
876For more details on what these methods do, @pxref{Operations} in
877@ref{The object model of ASDF}.
878
879@c The following plunge into the weeds is not appropriate in this
880@c location. [2010/10/03:rpg]
881@c note that although this also supports @code{:before} methods,
882@c they may not do what you want them to ---
883@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
884@c will run after all the dependencies and sub-components have been processed,
885@c but before the component in question has been compiled.
886
887@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
888@comment  node-name,  next,  previous,  up
889@section The defsystem grammar
890
891@c FIXME: @var typesetting not consistently used here.  We should either expand
892@c its use to everywhere, or we should kill it everywhere.
893
894
895@example
896system-definition := ( defsystem system-designator @var{system-option}* )
897
898system-option := :defsystem-depends-on system-list
899                 | module-option
900                 | option
901
902module-option := :components component-list
903                 | :serial [ t | nil ]
904                 | :if-component-dep-fails component-dep-fail-option
905
906option :=
907        | :pathname pathname-specifier
908        | :default-component-class class-name
909        | :perform method-form
910        | :explain method-form
911        | :output-files method-form
912        | :operation-done-p method-form
913        | :depends-on ( @var{dependency-def}* )
914        | :in-order-to ( @var{dependency}+ )
915
916
917system-list := ( @var{simple-component-name}* )
918
919component-list := ( @var{component-def}* )
920
921component-def  := ( component-type simple-component-name @var{option}* )
922
923component-type := :system | :module | :file | :static-file | other-component-type
924
925other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
926
927dependency-def := simple-component-name
928               | ( :feature name )
929               | ( :version simple-component-name version-specifier)
930
931dependency := (dependent-op @var{requirement}+)
932requirement := (required-op @var{required-component}+)
933             | (feature feature-name)
934dependent-op := operation-name
935required-op := operation-name | feature
936
937simple-component-name := string
938                      |  symbol
939
940pathname-specifier := pathname | string | symbol
941
942method-form := (operation-name qual lambda-list @&rest body)
943qual := method qualifier
944
945component-dep-fail-option := :fail | :try-next | :ignore
946@end example
947
948
949@subsection Component names
950
951Component names (@code{simple-component-name})
952may be either strings or symbols.
953
954@subsection Component types
955
956Component type names, even if expressed as keywords, will be looked up
957by name in the current package and in the asdf package, if not found in
958the current package.  So a component type @code{my-component-type}, in
959the current package @code{my-system-asd} can be specified as
960@code{:my-component-type}, or @code{my-component-type}.
961
962@subsection Defsystem depends on
963
964The @code{:defsystem-depends-on} option to @code{defsystem} allows the
965programmer to specify another ASDF-defined system or set of systems that
966must be loaded @emph{before} the system definition is processed.
967Typically this is used to load an ASDF extension that is used in the
968system definition.
969
970@subsection Pathname specifiers
971@cindex pathname specifiers
972
973A pathname specifier (@code{pathname-specifier})
974may be a pathname, a string or a symbol.
975When no pathname specifier is given for a component,
976which is the usual case, the component name itself is used.
977
978If a string is given, which is the usual case,
979the string will be interpreted as a Unix-style pathname
980where @code{/} characters will be interpreted as directory separators.
981Usually, Unix-style relative pathnames are used
982(i.e. not starting with @code{/}, as opposed to absolute pathnames);
983they are relative to the path of the parent component.
984Finally, depending on the @code{component-type},
985the pathname may be interpreted as either a file or a directory,
986and if it's a file,
987a file type may be added corresponding to the @code{component-type},
988or else it will be extracted from the string itself (if applicable).
989
990For instance, the @code{component-type} @code{:module}
991wants a directory pathname, and so a string @code{"foo/bar"}
992will be interpreted as the pathname @file{#p"foo/bar/"}.
993On the other hand, the @code{component-type} @code{:file}
994wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
995will be interpreted as the pathname @file{#p"foo/bar.lisp"},
996and a string @code{"foo/bar.quux"}
997will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
998Finally, the @code{component-type} @code{:static-file}
999wants a file without specifying a type, and so a string @code{"foo/bar"}
1000will be interpreted as the pathname @file{#p"foo/bar"},
1001and a string @code{"foo/bar.quux"}
1002will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1003
1004ASDF does not interpret the string @code{".."} to designate the parent
1005directory.  This string will be passed through to the underlying
1006operating system for interpretation.  We @emph{believe} that this will
1007work on all platforms where ASDF is deployed, but do not guarantee this
1008behavior.  A pathname object with a relative directory component of
1009@code{:up} or @code{:back} is the only guaranteed way to specify a
1010parent directory.
1011
1012If a symbol is given, it will be translated into a string,
1013and downcased in the process.
1014The downcasing of symbols is unconventional,
1015but was selected after some consideration.
1016Observations suggest that the type of systems we want to support
1017either have lowercase as customary case (Unix, Mac, windows)
1018or silently convert lowercase to uppercase (lpns),
1019so this makes more sense than attempting to use @code{:case :common}
1020as argument to @code{make-pathname},
1021which is reported not to work on some implementations.
1022
1023Pathname objects may be given to override the path for a component.
1024Such objects are typically specified using reader macros such as @code{#p}
1025or @code{#.(make-pathname ...)}.
1026Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
1027and that the behavior of @code{parse-namestring} is completely non-portable,
1028unless you are using Common Lisp @code{logical-pathname}s
1029(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1030Pathnames made with @code{#.(make-pathname ...)}
1031can usually be done more easily with the string syntax above.
1032The only case that you really need a pathname object is to override
1033the component-type default file type for a given component.
1034Therefore, pathname objects should only rarely be used.
1035Unhappily, ASDF 1 didn't properly support
1036parsing component names as strings specifying paths with directories,
1037and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1038
1039Note that when specifying pathname objects,
1040ASDF does not do any special interpretation of the pathname
1041influenced by the component type, unlike the procedure for
1042pathname-specifying strings.
1043On the one hand, you have to be careful to provide a pathname that correctly
1044fulfills whatever constraints are required from that component type
1045(e.g. naming a directory or a file with appropriate type);
1046on the other hand, you can circumvent the file type that would otherwise
1047be forced upon you if you were specifying a string.
1048
1049@subsection Version specifiers
1050@cindex version specifiers
1051@cindex :version
1052
1053Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1054@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1055In particular, version @code{0.2.1} is interpreted the same as
1056@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1057even though the two are the same when interpreted as decimal fractions.
1058
1059System definers are encouraged to use version identifiers of the form
1060@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1061API) and patch level.
1062
1063@xref{Common attributes of components}.
1064
1065
1066@subsection Using logical pathnames
1067@cindex logical pathnames
1068
1069We do not generally recommend the use of logical pathnames,
1070especially not so to newcomers to Common Lisp.
1071However, we do support the use of logical pathnames by old timers,
1072when such is their preference.
1073
1074To use logical pathnames,
1075you will have to provide a pathname object as a @code{:pathname} specifier
1076to components that use it, using such syntax as
1077@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1078
1079You only have to specify such logical pathname
1080for your system or some top-level component.
1081Sub-components' relative pathnames,
1082specified using the string syntax for names,
1083will be properly merged with the pathnames of their parents.
1084The specification of a logical pathname host however is @emph{not}
1085otherwise directly supported in the ASDF syntax
1086for pathname specifiers as strings.
1087
1088The @code{asdf-output-translation} layer will
1089avoid trying to resolve and translate logical pathnames.
1090The advantage of this is that
1091you can define yourself what translations you want to use
1092with the logical pathname facility.
1093The disadvantage is that if you do not define such translations,
1094any system that uses logical pathnames will behave differently under
1095asdf-output-translations than other systems you use.
1096
1097If you wish to use logical pathnames you will have to configure the
1098translations yourself before they may be used.
1099ASDF currently provides no specific support
1100for defining logical pathname translations.
1101
1102Note that the reasons we do not recommend logical pathnames are that
1103(1) there is no portable way to set up logical pathnames before they are used,
1104(2) logical pathnames are limited to only portably use
1105a single character case, digits and hyphens.
1106While you can solve the first issue on your own,
1107describing how to do it on each of fifteen implementations supported by ASDF
1108is more than we can document.
1109As for the second issue, mind that the limitation is notably enforced on SBCL,
1110and that you therefore can't portably violate the limitations
1111but must instead define some encoding of your own and add individual mappings
1112to name physical pathnames that do not fit the restrictions.
1113This can notably be a problem when your Lisp files are part of a larger project
1114in which it is common to name files or directories in a way that
1115includes the version numbers of supported protocols,
1116or in which files are shared with software written
1117in different programming languages where conventions include the use of
1118underscores, dots or CamelCase in pathnames.
1119
1120
1121@subsection Serial dependencies
1122@cindex serial dependencies
1123
1124If the @code{:serial t} option is specified for a module,
1125ASDF will add dependencies for each child component,
1126on all the children textually preceding it.
1127This is done as if by @code{:depends-on}.
1128
1129@lisp
1130:serial t
1131:components ((:file "a") (:file "b") (:file "c"))
1132@end lisp
1133
1134is equivalent to
1135
1136@lisp
1137:components ((:file "a")
1138             (:file "b" :depends-on ("a"))
1139             (:file "c" :depends-on ("a" "b")))
1140@end lisp
1141
1142
1143@subsection Source location
1144
1145The @code{:pathname} option is optional in all cases for systems
1146defined via @code{defsystem},
1147and in the usual case the user is recommended not to supply it.
1148
1149Instead, ASDF follows a hairy set of rules that are designed so that
1150@enumerate
1151@item
1152@code{find-system}
1153will load a system from disk
1154and have its pathname default to the right place.
1155@item
1156This pathname information will not be overwritten with
1157@code{*default-pathname-defaults*}
1158(which could be somewhere else altogether)
1159if the user loads up the @file{.asd} file into his editor
1160and interactively re-evaluates that form.
1161@end enumerate
1162
1163If a system is being loaded for the first time,
1164its top-level pathname will be set to:
1165
1166@itemize
1167@item
1168The host/device/directory parts of @code{*load-truename*},
1169if it is bound.
1170@item
1171@code{*default-pathname-defaults*}, otherwise.
1172@end itemize
1173
1174If a system is being redefined, the top-level pathname will be
1175
1176@itemize
1177@item
1178changed, if explicitly supplied or obtained from @code{*load-truename*}
1179(so that an updated source location is reflected in the system definition)
1180@item
1181changed if it had previously been set from @code{*default-pathname-defaults*}
1182@item
1183left as before, if it had previously been set from @code{*load-truename*}
1184and @code{*load-truename*} is currently unbound
1185(so that a developer can evaluate a @code{defsystem} form
1186from within an editor without clobbering its source location)
1187@end itemize
1188
1189@subsection if-component-dep-fails option
1190
1191This option is only appropriate for module components (including
1192systems), not individual source files.
1193
1194For more information about this option, @pxref{Pre-defined subclasses of component}.
1195
1196@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1197@section Other code in .asd files
1198
1199Files containing @code{defsystem} forms
1200are regular Lisp files that are executed by @code{load}.
1201Consequently, you can put whatever Lisp code you like into these files
1202(e.g., code that examines the compile-time environment
1203and adds appropriate features to @code{*features*}).
1204However, some conventions should be followed,
1205so that users can control certain details of execution
1206of the Lisp in @file{.asd} files:
1207
1208@itemize
1209@item
1210Any informative output
1211(other than warnings and errors,
1212which are the condition system's to dispose of)
1213should be sent to the standard CL stream @code{*standard-output*},
1214so that users can easily control the disposition
1215of output from ASDF operations.
1216@end itemize
1217
1218
1219@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1220@comment  node-name,  next,  previous,  up
1221@chapter The object model of ASDF
1222
1223ASDF is designed in an object-oriented way from the ground up.
1224Both a system's structure and the operations that can be performed on systems
1225follow a protocol.
1226ASDF is extensible to new operations and to new component types.
1227This allows the addition of behaviours:
1228for example, a new component could be added for Java JAR archives,
1229and methods specialised on @code{compile-op} added for it
1230that would accomplish the relevant actions.
1231
1232This chapter deals with @emph{components}, the building blocks of a system,
1233and @emph{operations}, the actions that can be performed on a system.
1234
1235
1236
1237@menu
1238* Operations::
1239* Components::
1240* Functions::
1241@end menu
1242
1243@node  Operations, Components, The object model of ASDF, The object model of ASDF
1244@comment  node-name,  next,  previous,  up
1245@section Operations
1246@cindex operation
1247
1248An @dfn{operation} object of the appropriate type is instantiated
1249whenever the user wants to do something with a system like
1250
1251@itemize
1252@item compile all its files
1253@item load the files into a running lisp environment
1254@item copy its source files somewhere else
1255@end itemize
1256
1257Operations can be invoked directly, or examined
1258to see what their effects would be without performing them.
1259@emph{FIXME: document how!}
1260There are a bunch of methods specialised on operation and component type
1261that actually do the grunt work.
1262
1263The operation object contains whatever state is relevant for this purpose
1264(perhaps a list of visited nodes, for example)
1265but primarily is a nice thing to specialise operation methods on
1266and easier than having them all be @code{EQL} methods.
1267
1268Operations are invoked on systems via @code{operate}.
1269@anchor{operate}
1270@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1271@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1272@code{operate} invokes @var{operation} on @var{system}.
1273@code{oos} is a synonym for @code{operate}.
1274
1275@var{operation} is a symbol that is passed, along with the supplied
1276@var{initargs}, to @code{make-instance} to create the operation object.
1277@var{system} is a system designator.
1278
1279The @var{initargs} are passed to the @code{make-instance} call
1280when creating the operation object.
1281Note that dependencies may cause the operation
1282to invoke other operations on the system or its components:
1283the new operations will be created
1284with the same @var{initargs} as the original one.
1285
1286@end deffn
1287
1288@menu
1289* Predefined operations of ASDF::
1290* Creating new operations::
1291@end menu
1292
1293@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1294@comment  node-name,  next,  previous,  up
1295@subsection Predefined operations of ASDF
1296
1297All the operations described in this section are in the @code{asdf} package.
1298They are invoked via the @code{operate} generic function.
1299
1300@lisp
1301(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1302@end lisp
1303
1304@deffn Operation @code{compile-op} @&key @code{proclamations}
1305
1306This operation compiles the specified component.
1307If proclamations are supplied, they will be proclaimed.
1308This is a good place to specify optimization settings.
1309
1310When creating a new component type,
1311you should provide methods for @code{compile-op}.
1312
1313When @code{compile-op} is invoked,
1314component dependencies often cause some parts of the system
1315to be loaded as well as compiled.
1316Invoking @code{compile-op}
1317does not necessarily load all the parts of the system, though;
1318use @code{load-op} to load a system.
1319@end deffn
1320
1321@deffn Operation @code{load-op} @&key @code{proclamations}
1322
1323This operation loads a system.
1324
1325The default methods for @code{load-op} compile files before loading them.
1326For parity, your own methods on new component types should probably do so too.
1327@end deffn
1328
1329@deffn Operation @code{load-source-op}
1330
1331This operation will load the source for the files in a module
1332even if the source files have been compiled.
1333Systems sometimes have knotty dependencies
1334which require that sources are loaded
1335before they can be compiled.
1336This is how you do that.
1337
1338If you are creating a component type,
1339you need to implement this operation --- at least, where meaningful.
1340@end deffn
1341
1342@anchor{test-op}
1343@deffn Operation @code{test-op}
1344
1345This operation will perform some tests on the module.
1346The default method will do nothing.
1347The default dependency is to require
1348@code{load-op} to be performed on the module first.
1349The default @code{operation-done-p} is that the operation is @emph{never} done
1350---
1351we assume that if you invoke the @code{test-op},
1352you want to test the system, even if you have already done so.
1353
1354The results of this operation are not defined by ASDF.
1355It has proven difficult to define how the test operation
1356should signal its results to the user
1357in a way that is compatible with all of the various test libraries
1358and test techniques in use in the community.
1359@end deffn
1360
1361@c @deffn Operation test-system-version @&key minimum
1362
1363@c Asks the system whether it satisfies a version requirement.
1364
1365@c The default method accepts a string, which is expected to contain of a
1366@c number of integers separated by #\. characters.  The method is not
1367@c recursive.  The component satisfies the version dependency if it has
1368@c the same major number as required and each of its sub-versions is
1369@c greater than or equal to the sub-version number required.
1370
1371@c @lisp
1372@c (defun version-satisfies (x y)
1373@c   (labels ((bigger (x y)
1374@c           (cond ((not y) t)
1375@c                 ((not x) nil)
1376@c                 ((> (car x) (car y)) t)
1377@c                 ((= (car x) (car y))
1378@c                  (bigger (cdr x) (cdr y))))))
1379@c     (and (= (car x) (car y))
1380@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1381@c @end lisp
1382
1383@c If that doesn't work for your system, you can override it.  I hope
1384@c you have as much fun writing the new method as @verb{|#lisp|} did
1385@c reimplementing this one.
1386@c @end deffn
1387
1388@c @deffn Operation feature-dependent-op
1389
1390@c An instance of @code{feature-dependent-op} will ignore any components
1391@c which have a @code{features} attribute, unless the feature combination
1392@c it designates is satisfied by @code{*features*}.  This operation is
1393@c not intended to be instantiated directly, but other operations may
1394@c inherit from it.
1395
1396@c @end deffn
1397
1398@node  Creating new operations,  , Predefined operations of ASDF, Operations
1399@comment  node-name,  next,  previous,  up
1400@subsection Creating new operations
1401
1402ASDF was designed to be extensible in an object-oriented fashion.
1403To teach ASDF new tricks, a programmer can implement the behaviour he wants
1404by creating a subclass of @code{operation}.
1405
1406ASDF's pre-defined operations are in no way ``privileged'',
1407but it is requested that developers never use the @code{asdf} package
1408for operations they develop themselves.
1409The rationale for this rule is that we don't want to establish a
1410``global asdf operation name registry'',
1411but also want to avoid name clashes.
1412
1413An operation must provide methods for the following generic functions
1414when invoked with an object of type @code{source-file}:
1415@emph{FIXME describe this better}
1416
1417@itemize
1418
1419@item @code{output-files}
1420The @code{output-files} method determines where the method will put its files.
1421It returns two values, a list of pathnames, and a boolean.
1422If the boolean is @code{T} then the pathnames are marked
1423not be translated by enclosing @code{:around} methods.
1424If the boolean is @code{NIL} then enclosing @code{:around} methods
1425may translate these pathnames, e.g. to ensure object files
1426are somehow stored in some implementation-dependent cache.
1427@item @code{perform}
1428The @code{perform} method must call @code{output-files}
1429to find out where to put its files,
1430because the user is allowed to override.
1431@item @code{output-files}
1432for local policy @code{explain}
1433@item @code{operation-done-p},
1434if you don't like the default one
1435
1436@end itemize
1437
1438Operations that print output should send that output to the standard
1439CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1440
1441@node Components, Functions, Operations, The object model of ASDF
1442@comment  node-name,  next,  previous,  up
1443@section Components
1444@cindex component
1445@cindex system
1446@cindex system designator
1447@vindex *system-definition-search-functions*
1448
1449A @dfn{component} represents a source file or
1450(recursively) a collection of components.
1451A @dfn{system} is (roughly speaking) a top-level component
1452that can be found via @code{find-system}.
1453
1454A @dfn{system designator} is a string or symbol
1455and behaves just like any other component name
1456(including with regard to the case conversion rules for component names).
1457
1458
1459@defun find-system system-designator &optional (error-p t)
1460
1461Given a system designator, @code{find-system} finds and returns a system.
1462If no system is found, an error of type
1463@code{missing-component} is thrown,
1464or @code{nil} is returned if @code{error-p} is false.
1465
1466To find and update systems, @code{find-system} funcalls each element
1467in the @code{*system-definition-search-functions*} list,
1468expecting a pathname to be returned, or a system object,
1469from which a pathname may be extracted, and that will be registered.
1470The resulting pathname (if any) is loaded
1471if one of the following conditions is true:
1472
1473@itemize
1474@item
1475there is no system of that name in memory
1476@item
1477the pathname is different from that which was previously loaded
1478@item
1479the file's @code{last-modified} time exceeds the @code{last-modified} time
1480of the system in memory
1481@end itemize
1482
1483When system definitions are loaded from @file{.asd} files,
1484a new scratch package is created for them to load into,
1485so that different systems do not overwrite each others operations.
1486The user may also wish to (and is recommended to)
1487include @code{defpackage} and @code{in-package} forms
1488in his system definition files, however,
1489so that they can be loaded manually if need be.
1490
1491The default value of @code{*system-definition-search-functions*}
1492is a list of two functions.
1493The first function looks in each of the directories given
1494by evaluating members of @code{*central-registry*}
1495for a file whose name is the name of the system and whose type is @file{asd}.
1496The first such file is returned,
1497whether or not it turns out to actually define the appropriate system.
1498The second function does something similar,
1499for the directories specified in the @code{source-registry}.
1500Hence, it is strongly advised to define a system
1501@var{foo} in the corresponding file @var{foo.asd}.
1502@end defun
1503
1504
1505@menu
1506* Common attributes of components::
1507* Pre-defined subclasses of component::
1508* Creating new component types::
1509@end menu
1510
1511@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1512@comment  node-name,  next,  previous,  up
1513@subsection Common attributes of components
1514
1515All components, regardless of type, have the following attributes.
1516All attributes except @code{name} are optional.
1517
1518@subsubsection Name
1519
1520A component name is a string or a symbol.
1521If a symbol, its name is taken and lowercased.
1522
1523Unless overridden by a @code{:pathname} attribute,
1524the name will be interpreted as a pathname specifier according
1525to a Unix-style syntax.
1526@xref{The defsystem grammar,,Pathname specifiers}.
1527
1528@subsubsection Version identifier
1529@findex version-satisfies
1530@cindex :version
1531
1532This optional attribute is used by the generic function
1533@code{version-satisfies}, which tests to see if @code{:version}
1534dependencies are satisfied.
1535the version should be a string of integers separated by dots,
1536for example @samp{1.0.11}.
1537For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1538
1539@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1540@c @xref{Predefined operations of ASDF}.
1541@c @emph{Nota Bene}:
1542@c This operation, planned for ASDF 1,
1543@c is still not implemented yet as of ASDF 2.
1544@c Don't hold your breath.
1545
1546
1547
1548@subsubsection Required features
1549
1550@emph{FIXME: This subsection seems to contradict the
1551@code{defsystem} grammar subsection,
1552which doesn't provide any obvious way to specify required features.
1553Furthermore, in 2009, discussions on the
1554@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1555suggested that the specification of required features may be broken,
1556and that no one may have been using them for a while.
1557Please contact the
1558@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1559if you are interested in getting this features feature fixed.}
1560
1561Traditionally defsystem users have used reader conditionals
1562to include or exclude specific per-implementation files.
1563This means that any single implementation cannot read the entire system,
1564which becomes a problem if it doesn't wish to compile it,
1565but instead for example to create an archive file containing all the sources,
1566as it will omit to process the system-dependent sources for other systems.
1567
1568Each component in an asdf system may therefore specify features using
1569the same syntax as @code{#+} does, and it will (somehow) be ignored for
1570certain operations unless the feature conditional is a member of
1571@code{*features*}.
1572
1573
1574@subsubsection Dependencies
1575
1576This attribute specifies dependencies of the component on its siblings.
1577It is optional but often necessary.
1578
1579There is an excitingly complicated relationship between the initarg
1580and the method that you use to ask about dependencies
1581
1582Dependencies are between (operation component) pairs.
1583In your initargs for the component, you can say
1584
1585@lisp
1586:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1587              (load-op (load-op "foo")))
1588@end lisp
1589
1590This means the following things:
1591@itemize
1592@item
1593before performing compile-op on this component, we must perform
1594load-op on @var{a} and @var{b}, and compile-op on @var{c},
1595@item
1596before performing @code{load-op}, we have to load @var{foo}
1597@end itemize
1598
1599The syntax is approximately
1600
1601@verbatim
1602(this-op {(other-op required-components)}+)
1603
1604simple-component-name := string
1605                      |  symbol
1606
1607required-components := simple-component-name
1608                     | (required-components required-components)
1609
1610component-name := simple-component-name
1611                | (:version simple-component-name minimum-version-object)
1612@end verbatim
1613
1614Side note:
1615
1616This is on a par with what ACL defsystem does.
1617mk-defsystem is less general: it has an implied dependency
1618
1619@verbatim
1620  for all x, (load x) depends on (compile x)
1621@end verbatim
1622
1623and using a @code{:depends-on} argument to say that @var{b} depends on
1624@var{a} @emph{actually} means that
1625
1626@verbatim
1627  (compile b) depends on (load a)
1628@end verbatim
1629
1630This is insufficient for e.g. the McCLIM system, which requires that
1631all the files are loaded before any of them can be compiled ]
1632
1633End side note
1634
1635In ASDF, the dependency information for a given component and operation
1636can be queried using @code{(component-depends-on operation component)},
1637which returns a list
1638
1639@lisp
1640((load-op "a") (load-op "b") (compile-op "c") ...)
1641@end lisp
1642
1643@code{component-depends-on} can be subclassed for more specific
1644component/operation types: these need to @code{(call-next-method)}
1645and append the answer to their dependency, unless
1646they have a good reason for completely overriding the default dependencies.
1647
1648If it weren't for CLISP, we'd be using @code{LIST} method
1649combination to do this transparently.
1650But, we need to support CLISP.
1651If you have the time for some CLISP hacking,
1652I'm sure they'd welcome your fixes.
1653@c Doesn't CLISP now support LIST method combination?
1654
1655See the discussion of the semantics of @code{:version} in the defsystem
1656grammar.
1657
1658@c FIXME: Should have cross-reference to "Version specifiers" in the
1659@c defsystem grammar, but the cross-referencing is so broken by
1660@c insufficient node breakdown that I have not put one in.
1661
1662
1663@subsubsection pathname
1664
1665This attribute is optional and if absent (which is the usual case),
1666the component name will be used.
1667
1668@xref{The defsystem grammar,,Pathname specifiers},
1669for an explanation of how this attribute is interpreted.
1670
1671Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1672does additional processing to set the filesystem location of
1673the top component in that system.
1674This is detailed elsewhere. @xref{Defining systems with defsystem}.
1675
1676
1677@subsubsection properties
1678
1679This attribute is optional.
1680
1681Packaging systems often require information about files or systems
1682in addition to that specified by ASDF's pre-defined component attributes.
1683Programs that create vendor packages out of ASDF systems therefore
1684have to create ``placeholder'' information to satisfy these systems.
1685Sometimes the creator of an ASDF system may know the additional
1686information and wish to provide it directly.
1687
1688@code{(component-property component property-name)} and
1689associated @code{setf} method will allow
1690the programmatic update of this information.
1691Property names are compared as if by @code{EQL},
1692so use symbols or keywords or something.
1693
1694@menu
1695* Pre-defined subclasses of component::
1696* Creating new component types::
1697@end menu
1698
1699@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1700@comment  node-name,  next,  previous,  up
1701@subsection Pre-defined subclasses of component
1702
1703@deffn Component source-file
1704
1705A source file is any file that the system does not know how to
1706generate from other components of the system.
1707
1708Note that this is not necessarily the same thing as
1709``a file containing data that is typically fed to a compiler''.
1710If a file is generated by some pre-processor stage
1711(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1712then it is not, by this definition, a source file.
1713Conversely, we might have a graphic file
1714that cannot be automatically regenerated,
1715or a proprietary shared library that we received as a binary:
1716these do count as source files for our purposes.
1717
1718Subclasses of source-file exist for various languages.
1719@emph{FIXME: describe these.}
1720@end deffn
1721
1722@deffn Component module
1723
1724A module is a collection of sub-components.
1725
1726A module component has the following extra initargs:
1727
1728@itemize
1729@item
1730@code{:components} the components contained in this module
1731
1732@item
1733@code{:default-component-class}
1734All children components which don't specify their class explicitly
1735are inferred to be of this type.
1736
1737@item
1738@code{:if-component-dep-fails}
1739This attribute takes one of the values
1740@code{:fail}, @code{:try-next}, @code{:ignore},
1741its default value is @code{:fail}.
1742The other values can be used for implementing conditional compilation
1743based on implementation @code{*features*},
1744for the case where it is not necessary for all files in a module to be
1745compiled.
1746@emph{FIXME: such conditional compilation has been reported
1747to be broken in 2009.}
1748
1749@item
1750@code{:serial} When this attribute is set,
1751each subcomponent of this component is assumed to depend on all subcomponents
1752before it in the list given to @code{:components}, i.e.
1753all of them are loaded before a compile or load operation is performed on it.
1754
1755@end itemize
1756
1757The default operation knows how to traverse a module, so
1758most operations will not need to provide methods specialised on modules.
1759
1760@code{module} may be subclassed to represent components such as
1761foreign-language linked libraries or archive files.
1762@end deffn
1763
1764@deffn Component system
1765
1766@code{system} is a subclass of @code{module}.
1767
1768A system is a module with a few extra attributes for documentation
1769purposes; these are given elsewhere.
1770@xref{The defsystem grammar}.
1771
1772Users can create new classes for their systems:
1773the default @code{defsystem} macro takes a @code{:class} keyword argument.
1774@end deffn
1775
1776@node  Creating new component types,  , Pre-defined subclasses of component, Components
1777@comment  node-name,  next,  previous,  up
1778@subsection Creating new component types
1779
1780New component types are defined by subclassing one of the existing
1781component classes and specializing methods on the new component class.
1782
1783@emph{FIXME: this should perhaps be explained more throughly,
1784not only by example ...}
1785
1786As an example, suppose we have some implementation-dependent
1787functionality that we want to isolate
1788in one subdirectory per Lisp implementation our system supports.
1789We create a subclass of
1790@code{cl-source-file}:
1791
1792@lisp
1793(defclass unportable-cl-source-file (cl-source-file)
1794  ())
1795@end lisp
1796
1797Function @code{asdf:implementation-type} (exported since 2.014.14)
1798gives us the name of the subdirectory.
1799All that's left is to define how to calculate the pathname
1800of an @code{unportable-cl-source-file}.
1801
1802@lisp
1803(defmethod component-pathname ((component unportable-cl-source-file))
1804  (merge-pathnames*
1805   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1806   (call-next-method)))
1807@end lisp
1808
1809The new component type is used in a @code{defsystem} form in this way:
1810
1811@lisp
1812(defsystem :foo
1813    :components
1814    ((:file "packages")
1815     ...
1816     (:unportable-cl-source-file "threads"
1817      :depends-on ("packages" ...))
1818     ...
1819    )
1820@end lisp
1821
1822@node Functions,  , Components, The object model of ASDF
1823@comment  node-name,  next,  previous,  up
1824@section Functions
1825@findex version-satisfies
1826
1827@deffn version-satisfies @var{version} @var{version-spec}
1828Does @var{version} satisfy the @var{version-spec}.  A generic function.
1829ASDF provides built-in methods for @var{version} being a
1830@code{component} or @code{string}.  @var{version-spec} should be a
1831string.
1832
1833In the wild, we typically see version numbering only on components of
1834type @code{system}.
1835
1836For more information about how @code{version-satisfies} interprets
1837version strings and specifications, @pxref{The defsystem grammar} and
1838@ref{Common attributes of components}.
1839@end deffn
1840
1841@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1842@comment  node-name,  next,  previous,  up
1843@chapter Controlling where ASDF searches for systems
1844
1845@section Configurations
1846
1847Configurations specify paths where to find system files.
1848
1849@enumerate
1850
1851@item
1852The search registry may use some hardcoded wrapping registry specification.
1853This allows some implementations (notably SBCL) to specify where to find
1854some special implementation-provided systems that
1855need to precisely match the version of the implementation itself.
1856
1857@item
1858An application may explicitly initialize the source-registry configuration
1859using the configuration API
1860(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1861in which case this takes precedence.
1862It may itself compute this configuration from the command-line,
1863from a script, from its own configuration file, etc.
1864
1865@item
1866The source registry will be configured from
1867the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1868
1869@item
1870The source registry will be configured from
1871user configuration file
1872@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1873(which defaults to
1874@file{~/.config/common-lisp/source-registry.conf})
1875if it exists.
1876
1877@item
1878The source registry will be configured from
1879user configuration directory
1880@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1881(which defaults to
1882@file{~/.config/common-lisp/source-registry.conf.d/})
1883if it exists.
1884
1885@item
1886The source registry will be configured from
1887system configuration file
1888@file{/etc/common-lisp/source-registry.conf}
1889if it exists/
1890
1891@item
1892The source registry will be configured from
1893system configuration directory
1894@file{/etc/common-lisp/source-registry.conf.d/}
1895if it exists.
1896
1897@item
1898The source registry will be configured from a default configuration.
1899This configuration may allow for implementation-specific systems
1900to be found, for systems to be found the current directory
1901(at the time that the configuration is initialized) as well as
1902@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1903@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1904For instance, SBCL will include directories for its contribs
1905when it can find them; it will look for them where SBCL was installed,
1906or at the location specified by the @code{SBCL_HOME} environment variable.
1907
1908@end enumerate
1909
1910Each of these configurations is specified as an s-expression
1911in a trivial domain-specific language (defined below).
1912Additionally, a more shell-friendly syntax is available
1913for the environment variable (defined yet below).
1914
1915Each of these configurations is only used if the previous
1916configuration explicitly or implicitly specifies that it
1917includes its inherited configuration.
1918
1919Additionally, some implementation-specific directories
1920may be automatically prepended to whatever directories are specified
1921in configuration files, no matter if the last one inherits or not.
1922
1923@section XDG base directory
1924
1925Note that we purport to respect the XDG base directory specification
1926as to where configuration files are located,
1927where data files are located,
1928where output file caches are located.
1929Mentions of XDG variables refer to that document.
1930
1931@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1932
1933This specification allows the user to specify some environment variables
1934to customize how applications behave to his preferences.
1935
1936On Windows platforms, when not using Cygwin,
1937instead of the XDG base directory specification,
1938we try to use folder configuration from the registry regarding
1939@code{Common AppData} and similar directories.
1940However, support for querying the Windows registry is limited as of ASDF 2,
1941and on many implementations, we may fall back to always using the defaults
1942without consulting the registry.
1943Patches welcome.
1944
1945@section Backward Compatibility
1946
1947For backward compatibility as well as to provide a practical backdoor for hackers,
1948ASDF will first search for @code{.asd} files in the directories specified in
1949@code{asdf:*central-registry*}
1950before it searches in the source registry above.
1951
1952@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
1953
1954By default, @code{asdf:*central-registry*} will be empty.
1955
1956This old mechanism will therefore not affect you if you don't use it,
1957but will take precedence over the new mechanism if you do use it.
1958
1959@section Configuration DSL
1960
1961Here is the grammar of the s-expression (SEXP) DSL for source-registry
1962configuration:
1963
1964@c FIXME: This is too wide for happy compilation into pdf.
1965
1966@example
1967;; A configuration is a single SEXP starting with keyword :source-registry
1968;; followed by a list of directives.
1969CONFIGURATION := (:source-registry DIRECTIVE ...)
1970
1971;; A directive is one of the following:
1972DIRECTIVE :=
1973    ;; INHERITANCE DIRECTIVE:
1974    ;; Your configuration expression MUST contain
1975    ;; exactly one of either of these:
1976    :inherit-configuration | ; splices inherited configuration (often specified last)
1977    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1978
1979    ;; forward compatibility directive (since ASDF 2.011.4), useful when
1980    ;; you want to use new configuration features but have to bootstrap a
1981    ;; the newer required ASDF from an older release that doesn't sport said features:
1982    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
1983
1984    ;; add a single directory to be scanned (no recursion)
1985    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1986
1987    ;; add a directory hierarchy, recursing but excluding specified patterns
1988    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1989
1990    ;; override the defaults for exclusion patterns
1991    (:exclude EXCLUSION-PATTERN ...) |
1992    ;; augment the defaults for exclusion patterns
1993    (:also-exclude EXCLUSION-PATTERN ...) |
1994    ;; Note that the scope of a an exclude pattern specification is
1995    ;; the rest of the current configuration expression or file.
1996
1997    ;; splice the parsed contents of another config file
1998    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1999
2000    ;; This directive specifies that some default must be spliced.
2001    :default-registry
2002
2003REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
2004DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2005
2006PATHNAME-DESIGNATOR :=
2007    NIL | ;; Special: skip this entry.
2008    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
2009
2010EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2011  against the name of a any subdirectory in the directory component
2012        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2013@end example
2014
2015Pathnames are designated using another DSL,
2016shared with the output-translations configuration DSL below.
2017The DSL is resolved by the function @code{asdf::resolve-location},
2018to be documented and exported at some point in the future.
2019
2020@example
2021ABSOLUTE-COMPONENT-DESIGNATOR :=
2022    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2023    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2024             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2025             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2026             ;; Note that none of the above applies to strings used in *central-registry*,
2027             ;; which doesn't use this DSL: they are processed as normal namestrings.
2028             ;; however, you can compute what you put in the *central-registry*
2029             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2030    PATHNAME | ;; pathname (better be an absolute path, or bust)
2031               ;; In output-translations, unless followed by relative components,
2032               ;; it better have appropriate wildcards, as in **/*.*.*
2033    :HOME | ;; designates the user-homedir-pathname ~/
2034    :USER-CACHE | ;; designates the default location for the user cache
2035    :HERE | ;; designates the location of the configuration file
2036            ;; (or *default-pathname-defaults*, if invoked interactively)
2037    :ROOT ;; magic, for output-translations source only: paths that are relative
2038          ;; to the root of the source host and device
2039    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2040
2041RELATIVE-COMPONENT-DESIGNATOR :=
2042    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
2043    STRING | ;; relative directory pathname as interpreted by coerce-pathname.
2044             ;; In output translations, if last component, **/*.*.* is added
2045    PATHNAME | ;; pathname; unless last component, directory is assumed.
2046    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.49-linux-x64
2047    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2048    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2049    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2050    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2051    :*.*.* | ;; any file (since ASDF 2.011.4)
2052    ;; Not supported (anymore): :UID and :USERNAME
2053@end example
2054
2055For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2056which is the default place ASDF looks for this configuration, once contained:
2057@example
2058(:source-registry
2059  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2060  :inherit-configuration)
2061@end example
2062
2063@section Configuration Directories
2064
2065Configuration directories consist in files each containing
2066a list of directives without any enclosing @code{(:source-registry ...)} form.
2067The files will be sorted by namestring as if by @code{string<} and
2068the lists of directives of these files with be concatenated in order.
2069An implicit @code{:inherit-configuration} will be included
2070at the @emph{end} of the list.
2071
2072This allows for packaging software that has file granularity
2073(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2074to easily include configuration information about distributed software.
2075
2076The convention is that, for sorting purposes,
2077the names of files in such a directory begin with two digits
2078that determine the order in which these entries will be read.
2079Also, the type of these files is conventionally @code{"conf"}
2080and as a limitation to some implementations (e.g. GNU clisp),
2081the type cannot be @code{NIL}.
2082
2083Directories may be included by specifying a directory pathname
2084or namestring in an @code{:include} directive, e.g.:
2085
2086@example
2087  (:include "/foo/bar/")
2088@end example
2089
2090Hence, to achieve the same effect as
2091my example @file{~/.config/common-lisp/source-registry.conf} above,
2092I could simply create a file
2093@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2094alone in its directory with the following contents:
2095@example
2096(:tree "/home/fare/cl/")
2097@end example
2098
2099@subsection The :here directive
2100
2101The @code{:here} directive is an absolute pathname designator that
2102refers to the directory containing the configuration file currently
2103being processed.
2104
2105The @code{:here} directive is intended to simplify the delivery of
2106complex CL systems, and for easy configuration of projects shared through
2107revision control systems, in accordance with our design principle that
2108each participant should be able to provide all and only the information
2109available to him or her.
2110
2111Consider a person X who has set up the source code repository for a
2112complex project with a master directory @file{dir/}.  Ordinarily, one
2113might simply have the user add a directive that would look something
2114like this:
2115@example
2116   (:tree  "path/to/dir")
2117@end example
2118But what if X knows that there are very large subtrees
2119under dir that are filled with, e.g., Java source code, image files for
2120icons, etc.?  All of the asdf system definitions are contained in the
2121subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2122these are the only directories that should be searched.
2123
2124In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2125contains the following:
2126@example
2127(:source-registry
2128   (:tree (:here "src/lisp/"))
2129   (:tree (:here "extlib/lisp"))
2130   (:directory (:here "outlier/")))
2131@end example
2132
2133Then when someone else (call her Y) checks out a copy of this
2134repository, she need only add
2135@example
2136(:include "/path/to/my/checkout/directory/asdf.conf")
2137@end example
2138to one of her previously-existing asdf source location configuration
2139files, or invoke @code{initialize-source-registry} with a configuration
2140form containing that s-expression.  ASDF will find the .conf file that X
2141has provided, and then set up source locations within the working
2142directory according to X's (relative) instructions.
2143
2144@section Shell-friendly syntax for configuration
2145
2146When considering environment variable @code{CL_SOURCE_REGISTRY}
2147ASDF will skip to next configuration if it's an empty string.
2148It will @code{READ} the string as a SEXP in the DSL
2149if it begins with a paren @code{(}
2150and it will be interpreted much like @code{TEXINPUTS}
2151list of paths, where
2152
2153  * paths are separated
2154   by a @code{:} (colon) on Unix platforms (including cygwin),
2155   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2156
2157  * each entry is a directory to add to the search path.
2158
2159  * if the entry ends with a double slash @code{//}
2160    then it instead indicates a tree in the subdirectories
2161    of which to recurse.
2162
2163  * if the entry is the empty string (which may only appear once),
2164    then it indicates that the inherited configuration should be
2165    spliced there.
2166
2167
2168@section Search Algorithm
2169@vindex *default-source-registry-exclusions*
2170
2171In case that isn't clear, the semantics of the configuration is that
2172when searching for a system of a given name,
2173directives are processed in order.
2174
2175When looking in a directory, if the system is found, the search succeeds,
2176otherwise it continues.
2177
2178When looking in a tree, if one system is found, the search succeeds.
2179If multiple systems are found, the consequences are unspecified:
2180the search may succeed with any of the found systems,
2181or an error may be raised.
2182ASDF currently returns the first system found,
2183XCVB currently raised an error.
2184If none is found, the search continues.
2185
2186Exclude statements specify patterns of subdirectories
2187the systems from which to ignore.
2188Typically you don't want to use copies of files kept by such
2189version control systems as Darcs.
2190Exclude statements are not propagated to further included or inherited
2191configuration files or expressions;
2192instead the defaults are reset around every configuration statement
2193to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2194
2195Include statements cause the search to recurse with the path specifications
2196from the file specified.
2197
2198An inherit-configuration statement cause the search to recurse with the path
2199specifications from the next configuration
2200(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2201
2202
2203@section Caching Results
2204
2205The implementation is allowed to either eagerly compute the information
2206from the configurations and file system, or to lazily re-compute it
2207every time, or to cache any part of it as it goes.
2208To explicitly flush any information cached by the system, use the API below.
2209
2210
2211@section Configuration API
2212
2213The specified functions are exported from your build system's package.
2214Thus for ASDF the corresponding functions are in package ASDF,
2215and for XCVB the corresponding functions are in package XCVB.
2216
2217@defun initialize-source-registry @&optional PARAMETER
2218   will read the configuration and initialize all internal variables.
2219   You may extend or override configuration
2220   from the environment and configuration files
2221   with the given @var{PARAMETER}, which can be
2222   @code{NIL} (no configuration override),
2223   or a SEXP (in the SEXP DSL),
2224   a string (as in the string DSL),
2225   a pathname (of a file or directory with configuration),
2226   or a symbol (fbound to function that when called returns one of the above).
2227@end defun
2228
2229@defun clear-source-registry
2230   undoes any source registry configuration
2231   and clears any cache for the search algorithm.
2232   You might want to call this function
2233   (or better, @code{clear-configuration})
2234   before you dump an image that would be resumed
2235   with a different configuration,
2236   and return an empty configuration.
2237   Note that this does not include clearing information about
2238   systems defined in the current image, only about
2239   where to look for systems not yet defined.
2240@end defun
2241
2242@defun ensure-source-registry @&optional PARAMETER
2243   checks whether a source registry has been initialized.
2244   If not, initialize it with the given @var{PARAMETER}.
2245@end defun
2246
2247Every time you use ASDF's @code{find-system}, or
2248anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2249@code{ensure-source-registry} is called with parameter NIL,
2250which the first time around causes your configuration to be read.
2251If you change a configuration file,
2252you need to explicitly @code{initialize-source-registry} again,
2253or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2254which will cause the initialization to happen next time around.
2255
2256
2257@section Future
2258
2259If this mechanism is successful, in the future, we may declare
2260@code{asdf:*central-registry*} obsolete and eventually remove it.
2261Any hook into implementation-specific search mechanisms will by then
2262have been integrated in the @code{:default-configuration} which everyone
2263should either explicitly use or implicit inherit. Some shell syntax
2264for it should probably be added somehow.
2265
2266But we're not there yet. For now, let's see how practical this new
2267source-registry is.
2268
2269
2270@section Rejected ideas
2271
2272Alternatives I considered and rejected included:
2273
2274@enumerate
2275@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2276   and somehow the configuration parser expands the new configuration
2277   language into a expanded series of directories of subdirectories to
2278   lookup, pre-recursing through specified hierarchies. This is kludgy,
2279   and leaves little space of future cleanups and extensions.
2280
2281@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2282   in completely new ways, so that new kinds of entries may be implemented
2283   as a recursive search, etc. This seems somewhat backwards.
2284
2285@item Completely remove @code{asdf:*central-registry*}
2286   and break backwards compatibility.
2287   Hopefully this will happen in a few years after everyone migrate to
2288   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2289
2290@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2291   when you dereference it or setf it. Only the new variable with new
2292   semantics is handled by the new search procedure.
2293   Complex and still introduces subtle semantic issues.
2294@end enumerate
2295
2296
2297I've been suggested the below features, but have rejected them,
2298for the sake of keeping ASDF no more complex than strictly necessary.
2299
2300@itemize
2301@item
2302  More syntactic sugar: synonyms for the configuration directives, such as
2303  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2304  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2305
2306@item
2307   The possibility to register individual files instead of directories.
2308
2309@item
2310  Integrate Xach Beane's tilde expander into the parser,
2311  or something similar that is shell-friendly or shell-compatible.
2312  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2313  minimal by removing the need for evaluated entries that ASDF has?
2314  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2315  Hopefully, these are already superseded by the @code{:default-registry}
2316
2317@item
2318  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2319  down a filesystem tree in the environment variable.
2320  It isn't that Lisp friendly either.
2321@end itemize
2322
2323@section TODO
2324
2325@itemize
2326@item Add examples
2327@end itemize
2328
2329
2330@section Credits for the source-registry
2331
2332Thanks a lot to Stelian Ionescu for the initial idea.
2333
2334Thanks to Rommel Martinez for the initial implementation attempt.
2335
2336All bad design ideas and implementation bugs are to mine, not theirs.
2337But so are good design ideas and elegant implementation tricks.
2338
2339 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2340
2341
2342
2343@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2344@comment  node-name,  next,  previous,  up
2345@chapter Controlling where ASDF saves compiled files
2346@cindex asdf-output-translations
2347@vindex ASDF_OUTPUT_TRANSLATIONS
2348
2349Each Common Lisp implementation has its own format
2350for compiled files (fasls for short, short for ``fast loading'').
2351If you use multiple implementations
2352(or multiple versions of the same implementation),
2353you'll soon find your source directories
2354littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2355Worse yet, some implementations use the same file extension
2356while changing formats from version to version (or platform to platform)
2357which means that you'll have to recompile binaries
2358as you switch from one implementation to the next.
2359
2360ASDF 2 includes the @code{asdf-output-translations} facility
2361to mitigate the problem.
2362
2363@section Configurations
2364
2365Configurations specify mappings from input locations to output locations.
2366Once again we rely on the XDG base directory specification for configuration.
2367@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2368
2369@enumerate
2370
2371@item
2372Some hardcoded wrapping output translations configuration may be used.
2373This allows special output translations (or usually, invariant directories)
2374to be specified corresponding to the similar special entries in the source registry.
2375
2376@item
2377An application may explicitly initialize the output-translations
2378configuration using the Configuration API
2379in which case this takes precedence.
2380(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2381It may itself compute this configuration from the command-line,
2382from a script, from its own configuration file, etc.
2383
2384@item
2385The source registry will be configured from
2386the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2387
2388@item
2389The source registry will be configured from
2390user configuration file
2391@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2392(which defaults to
2393@file{~/.config/common-lisp/asdf-output-translations.conf})
2394if it exists.
2395
2396@item
2397The source registry will be configured from
2398user configuration directory
2399@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2400(which defaults to
2401@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2402if it exists.
2403
2404@item
2405The source registry will be configured from
2406system configuration file
2407@file{/etc/common-lisp/asdf-output-translations.conf}
2408if it exists.
2409
2410@item
2411The source registry will be configured from
2412system configuration directory
2413@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2414if it exists.
2415
2416@end enumerate
2417
2418Each of these configurations is specified as a SEXP
2419in a trival domain-specific language (defined below).
2420Additionally, a more shell-friendly syntax is available
2421for the environment variable (defined yet below).
2422
2423Each of these configurations is only used if the previous
2424configuration explicitly or implicitly specifies that it
2425includes its inherited configuration.
2426
2427Note that by default, a per-user cache is used for output files.
2428This allows the seamless use of shared installations of software
2429between several users, and takes files out of the way of the developers
2430when they browse source code,
2431at the expense of taking a small toll when developers have to clean up
2432output files and find they need to get familiar with output-translations first.
2433
2434
2435@section Backward Compatibility
2436@cindex ASDF-BINARY-LOCATIONS compatibility
2437
2438
2439We purposefully do NOT provide backward compatibility with earlier versions of
2440@code{ASDF-Binary-Locations} (8 Sept 2009),
2441@code{common-lisp-controller} (7.0) or
2442@code{cl-launch} (2.35),
2443each of which had similar general capabilities.
2444The previous APIs of these programs were not designed
2445for configuration by the end-user
2446in an easy way with configuration files.
2447Recent versions of same packages use
2448the new @code{asdf-output-translations} API as defined below:
2449@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2450@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2451
2452This incompatibility shouldn't inconvenience many people.
2453Indeed, few people use and customize these packages;
2454these few people are experts who can trivially adapt to the new configuration.
2455Most people are not experts, could not properly configure these features
2456(except inasmuch as the default configuration of
2457@code{common-lisp-controller} and/or @code{cl-launch}
2458might have been doing the right thing for some users),
2459and yet will experience software that ``just works'',
2460as configured by the system distributor, or by default.
2461
2462Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2463we provide a limited emulation mode:
2464
2465@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2466This function will initialize the new @code{asdf-output-translations} facility in a way
2467that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2468Where you would previously set global variables
2469@var{*centralize-lisp-binaries*},
2470@var{*default-toplevel-directory*},
2471@var{*include-per-user-information*},
2472@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2473you will now have to pass the same values as keyword arguments to this function.
2474Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2475will accept any valid pathname designator for @code{asdf-output-translations}
2476instead of just strings and pathnames.
2477@end defun
2478
2479If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2480(the one available as an extension to load of top of ASDF,
2481not the one built into a few old versions of ASDF),
2482but first you must disable @code{asdf-output-translations}
2483with @code{(asdf:disable-output-translations)},
2484or you might experience ``interesting'' issues.
2485
2486Also, note that output translation is enabled by default.
2487To disable it, use @code{(asdf:disable-output-translations)}.
2488
2489
2490@section Configuration DSL
2491
2492Here is the grammar of the SEXP DSL
2493for @code{asdf-output-translations} configuration:
2494
2495@verbatim
2496;; A configuration is single SEXP starting with keyword :source-registry
2497;; followed by a list of directives.
2498CONFIGURATION := (:output-translations DIRECTIVE ...)
2499
2500;; A directive is one of the following:
2501DIRECTIVE :=
2502    ;; INHERITANCE DIRECTIVE:
2503    ;; Your configuration expression MUST contain
2504    ;; exactly one of either of these:
2505    :inherit-configuration | ; splices inherited configuration (often specified last)
2506    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2507
2508    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2509    ;; you want to use new configuration features but have to bootstrap a
2510    ;; the newer required ASDF from an older release that doesn't sport said features:
2511    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2512
2513    ;; include a configuration file or directory
2514    (:include PATHNAME-DESIGNATOR) |
2515
2516    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2517    :enable-user-cache |
2518    ;; Disable global cache. Map / to /
2519    :disable-cache |
2520
2521    ;; add a single directory to be scanned (no recursion)
2522    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2523
2524    ;; use a function to return the translation of a directory designator
2525    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2526
2527DIRECTORY-DESIGNATOR :=
2528    NIL | ;; As source: skip this entry. As destination: same as source
2529    T | ;; as source matches anything, as destination leaves pathname unmapped.
2530    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
2531
2532TRANSLATION-FUNCTION :=
2533    SYMBOL | ;; symbol of a function that takes two arguments,
2534             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2535    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2536             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2537
2538@end verbatim
2539
2540Relative components better be either relative
2541or subdirectories of the path before them, or bust.
2542
2543The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2544You can specify more fine-grained patterns
2545by using a pathname object as the last component
2546e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2547
2548You may use @code{#+features} to customize the configuration file.
2549
2550The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2551to anything but themselves (same as if the second designator was the same as the first).
2552
2553When the first designator is @code{t},
2554the mapping always matches.
2555When the first designator starts with @code{:root},
2556the mapping matches any host and device.
2557In either of these cases, if the second designator
2558isn't @code{t} and doesn't start with @code{:root},
2559then strings indicating the host and pathname are somehow copied
2560in the beginning of the directory component of the source pathname
2561before it is translated.
2562
2563When the second designator is @code{t}, the mapping is the identity.
2564When the second designator starts with @code{:root},
2565the mapping preserves the host and device of the original pathname.
2566Notably, this allows you to map files
2567to a subdirectory of the whichever directory the file is in.
2568Though the syntax is not quite as easy to use as we'd like,
2569you can have an (source destination) mapping entry such as follows
2570in your configuration file,
2571or you may use @code{enable-asdf-binary-locations-compatibility}
2572with @code{:centralize-lisp-binaries nil}
2573which will do the same thing internally for you:
2574@verbatim
2575  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2576          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2577     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2578       (:root ,wild-subdir :implementation ,wild-file)))
2579@end verbatim
2580Starting with ASDF 2.011.4, you can use the simpler:
2581  @code{`(:root (:root :**/ :implementation :*.*.*))}
2582
2583
2584
2585@code{:include} statements cause the search to recurse with the path specifications
2586from the file specified.
2587
2588If the @code{translate-pathname} mechanism cannot achieve a desired
2589translation, the user may provide a function which provides the
2590required algorithim.  Such a translation function is specified by
2591supplying a list as the second @code{directory-designator}
2592the first element of which is the keyword @code{:function},
2593and the second element of which is
2594either a symbol which designates a function or a lambda expression.
2595The function designated by the second argument must take two arguments,
2596the first being the pathname of the source file,
2597the second being the wildcard that was matched.
2598The result of the function invocation should be the translated pathname.
2599
2600An @code{:inherit-configuration} statement cause the search to recurse with the path
2601specifications from the next configuration.
2602@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2603
2604@itemize
2605@item
2606@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2607@item
2608@code{:disable-cache} is the same as @code{(t t)}.
2609@item
2610@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2611which by default is the same as using
2612@code{(:home ".cache" "common-lisp" :implementation)}.
2613@item
2614@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2615which by default is the same as using
2616@code{("/var/cache/common-lisp" :uid :implementation-type)}
2617(on Unix and cygwin), or something semi-sensible on Windows.
2618@end itemize
2619
2620
2621@section Configuration Directories
2622
2623Configuration directories consist in files each contains
2624a list of directives without any enclosing
2625@code{(:output-translations ...)} form.
2626The files will be sorted by namestring as if by @code{string<} and
2627the lists of directives of these files with be concatenated in order.
2628An implicit @code{:inherit-configuration} will be included
2629at the @emph{end} of the list.
2630
2631This allows for packaging software that has file granularity
2632(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2633to easily include configuration information about software being distributed.
2634
2635The convention is that, for sorting purposes,
2636the names of files in such a directory begin with two digits
2637that determine the order in which these entries will be read.
2638Also, the type of these files is conventionally @code{"conf"}
2639and as a limitation of some implementations, the type cannot be @code{NIL}.
2640
2641Directories may be included by specifying a directory pathname
2642or namestring in an @code{:include} directive, e.g.:
2643@verbatim
2644  (:include "/foo/bar/")
2645@end verbatim
2646
2647@section Shell-friendly syntax for configuration
2648
2649When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2650ASDF will skip to next configuration if it's an empty string.
2651It will @code{READ} the string as an SEXP in the DSL
2652if it begins with a paren @code{(}
2653and it will be interpreted as a list of directories.
2654Directories should come by pairs, indicating a mapping directive.
2655Entries are separated
2656by a @code{:} (colon) on Unix platforms (including cygwin),
2657by a @code{;} (semicolon) on other platforms (mainly, Windows).
2658
2659The magic empty entry,
2660if it comes in what would otherwise be the first entry in a pair,
2661indicates the splicing of inherited configuration.
2662If it comes as the second entry in a pair,
2663it indicates that the directory specified first is to be left untranslated
2664(which has the same effect as if the directory had been repeated).
2665
2666
2667@section Semantics of Output Translations
2668
2669From the specified configuration,
2670a list of mappings is extracted in a straightforward way:
2671mappings are collected in order, recursing through
2672included or inherited configuration as specified.
2673To this list is prepended some implementation-specific mappings,
2674and is appended a global default.
2675
2676The list is then compiled to a mapping table as follows:
2677for each entry, in order, resolve the first designated directory
2678into an actual directory pathname for source locations.
2679If no mapping was specified yet for that location,
2680resolve the second designated directory to an output location directory
2681add a mapping to the table mapping the source location to the output location,
2682and add another mapping from the output location to itself
2683(unless a mapping already exists for the output location).
2684
2685Based on the table, a mapping function is defined,
2686mapping source pathnames to output pathnames:
2687given a source pathname, locate the longest matching prefix
2688in the source column of the mapping table.
2689Replace that prefix by the corresponding output column
2690in the same row of the table, and return the result.
2691If no match is found, return the source pathname.
2692(A global default mapping the filesystem root to itself
2693may ensure that there will always be a match,
2694with same fall-through semantics).
2695
2696@section Caching Results
2697
2698The implementation is allowed to either eagerly compute the information
2699from the configurations and file system, or to lazily re-compute it
2700every time, or to cache any part of it as it goes.
2701To explicitly flush any information cached by the system, use the API below.
2702
2703
2704@section Output location API
2705
2706The specified functions are exported from package ASDF.
2707
2708@defun initialize-output-translations @&optional PARAMETER
2709   will read the configuration and initialize all internal variables.
2710   You may extend or override configuration
2711   from the environment and configuration files
2712   with the given @var{PARAMETER}, which can be
2713   @code{NIL} (no configuration override),
2714   or a SEXP (in the SEXP DSL),
2715   a string (as in the string DSL),
2716   a pathname (of a file or directory with configuration),
2717   or a symbol (fbound to function that when called returns one of the above).
2718@end defun
2719
2720@defun disable-output-translations
2721   will initialize output translations in a way
2722   that maps every pathname to itself,
2723   effectively disabling the output translation facility.
2724@end defun
2725
2726@defun clear-output-translations
2727   undoes any output translation configuration
2728   and clears any cache for the mapping algorithm.
2729   You might want to call this function
2730   (or better, @code{clear-configuration})
2731   before you dump an image that would be resumed
2732   with a different configuration,
2733   and return an empty configuration.
2734   Note that this does not include clearing information about
2735   systems defined in the current image, only about
2736   where to look for systems not yet defined.
2737@end defun
2738
2739@defun ensure-output-translations @&optional PARAMETER
2740   checks whether output translations have been initialized.
2741   If not, initialize them with the given @var{PARAMETER}.
2742   This function will be called before any attempt to operate on a system.
2743@end defun
2744
2745@defun apply-output-translations PATHNAME
2746   Applies the configured output location translations to @var{PATHNAME}
2747   (calls @code{ensure-output-translations} for the translations).
2748@end defun
2749
2750Every time you use ASDF's @code{output-files}, or
2751anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2752@code{ensure-output-translations} is called with parameter NIL,
2753which the first time around causes your configuration to be read.
2754If you change a configuration file,
2755you need to explicitly @code{initialize-output-translations} again,
2756or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2757which will cause the initialization to happen next time around.
2758
2759
2760@section Credits for output translations
2761
2762Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2763and to Peter van Eynde for @code{Common Lisp Controller}.
2764
2765All bad design ideas and implementation bugs are to mine, not theirs.
2766But so are good design ideas and elegant implementation tricks.
2767
2768 --- Francois-Rene Rideau @email{fare@@tunes.org}
2769
2770@c @section Default locations
2771@c @findex output-files-for-system-and-operation
2772
2773@c The default binary location for each Lisp implementation
2774@c is a subdirectory of each source directory.
2775@c To account for different Lisps, Operating Systems, Implementation versions,
2776@c and so on, ASDF borrows code from SLIME
2777@c to create reasonable custom directory names.
2778@c Here are some examples:
2779
2780@c @itemize
2781@c @item
2782@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2783
2784@c @item
2785@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2786
2787@c @item
2788@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2789@c @end itemize
2790
2791@c By default, all output file pathnames will be relocated
2792@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2793
2794@c See the document @file{README.asdf-output-translations}
2795@c for a full specification on how to configure @code{asdf-output-translations}.
2796
2797@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2798@comment  node-name,  next,  previous,  up
2799@chapter Error handling
2800@findex SYSTEM-DEFINITION-ERROR
2801@findex OPERATION-ERROR
2802
2803@section ASDF errors
2804
2805If ASDF detects an incorrect system definition, it will signal a generalised instance of
2806@code{SYSTEM-DEFINITION-ERROR}.
2807
2808Operations may go wrong (for example when source files contain errors).
2809These are signalled using generalised instances of
2810@code{OPERATION-ERROR}.
2811
2812@section Compilation error and warning handling
2813@vindex *compile-file-warnings-behaviour*
2814@vindex *compile-file-errors-behavior*
2815
2816ASDF checks for warnings and errors when a file is compiled.
2817The variables @var{*compile-file-warnings-behaviour*} and
2818@var{*compile-file-errors-behavior*}
2819control the handling of any such events.
2820The valid values for these variables are
2821@code{:error}, @code{:warn}, and @code{:ignore}.
2822
2823@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2824@comment  node-name,  next,  previous,  up
2825@chapter Miscellaneous additional functionality
2826
2827ASDF includes several additional features that are generally
2828useful for system definition and development.
2829
2830@section Controlling file compilation
2831
2832When declaring a component (system, module, file),
2833you can specify a keyword argument @code{:around-compile some-symbol}.
2834If left unspecified, the value will be inherited from the parent component if any,
2835or with a default of @code{nil} if no value is specified in any transitive parent.
2836
2837The argument must be a either fbound symbol or @code{nil}.
2838@code{nil} means the normal compile-file function will be called.
2839A symbol means the function fbound to it will be called with a single argument,
2840a thunk that calls the compile-file function;
2841the function you specify must then funcall that thunk
2842inside whatever wrapping you want.
2843
2844Using this hook, you may achieve such effects as:
2845locally renaming packages,
2846binding @var{*readtables*} and other syntax-controlling variables,
2847handling warnings and other conditions,
2848proclaiming consistent optimization settings,
2849saving code coverage information,
2850maintaining meta-data about compilation timings,
2851setting gensym counters and PRNG seeds and other sources of non-determinism,
2852overriding the source-location and/or timestamping systems,
2853checking that some compile-time side-effects were properly balanced,
2854etc.
2855
2856
2857@section Miscellaneous Exported Functions
2858
2859@defun coerce-pathname name @&key type defaults
2860
2861This function (available starting with ASDF 2.012.11)
2862takes an argument, and portably interprets it as a pathname.
2863If the argument @var{name} is a pathname or @code{nil}, it is passed through;
2864if it's a symbol, it's interpreted as a string by downcasing it;
2865if it's a string, it is first separated using @code{/} into substrings;
2866the leading substrings denote subdirectories of a relative pathname.
2867If @var{type} is @code{:directory} or the string ends with @code{/},
2868the last substring is also a subdirectory;
2869if @var{type} is a string, it is used as the type of the pathname, and
2870the last substring is the name component of the pathname;
2871if @var{type} is @code{nil}, the last substring specifies both name and type components
2872of the pathname, with the last @code{.} separating them, or only the name component
2873if there's no last @code{.} or if there is only one dot and it's the first character.
2874The host, device and version components come from @var{defaults}, which defaults to
2875@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
2876
2877@end defun
2878
2879@defun merge-pathnames* @&key specified defaults
2880
2881This function is a replacement for @code{merge-pathnames} that uses the host and device
2882from the @var{defaults} rather than the @var{specified} pathname when the latter
2883is a relative pathname. This allows ASDF and its users to create and use relative pathnames
2884without having to know beforehand what are the host and device
2885of the absolute pathnames they are relative to.
2886
2887@end defun
2888
2889@defun system-relative-pathname system name @&key type
2890
2891It's often handy to locate a file relative to some system.
2892The @code{system-relative-pathname} function meets this need.
2893
2894It takes two mandatory arguments @var{system} and @var{name}
2895and a keyword argument @var{type}:
2896@var{system} is name of a system, whereas @var{name} and optionally @var{type}
2897specify a relative pathname, interpreted like a component pathname specifier
2898by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
2899
2900It returns a pathname built from the location of the system's
2901source directory and the relative pathname. For example:
2902
2903@lisp
2904> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
2905#P"/repository/other/cl-ppcre/regex.data"
2906@end lisp
2907
2908@end defun
2909
2910@defun system-source-directory system-designator
2911
2912ASDF does not provide a turnkey solution for locating
2913data (or other miscellaneous) files
2914that are distributed together with the source code of a system.
2915Programmers can use @code{system-source-directory} to find such files.
2916Returns a pathname object.
2917The @var{system-designator} may be a string, symbol, or ASDF system object.
2918@end defun
2919
2920@defun clear-system system-designator
2921
2922It is sometimes useful to force recompilation of a previously loaded system.
2923In these cases, it may be useful to @code{(asdf:clear-system :foo)}
2924to remove the system from the table of currently loaded systems;
2925the next time the system @code{foo} or one that depends on it is re-loaded,
2926@code{foo} will then be loaded again.
2927Alternatively, you could touch @code{foo.asd} or
2928remove the corresponding fasls from the output file cache.
2929(It was once conceived that one should provide
2930a list of systems the recompilation of which to force
2931as the @code{:force} keyword argument to @code{load-system};
2932but this has never worked, and though the feature was fixed in ASDF 2.000,
2933it remains @code{cerror}'ed out as nobody ever used it.)
2934
2935Note that this does not and cannot by itself undo the previous loading
2936of the system. Common Lisp has no provision for such an operation,
2937and its reliance on irreversible side-effects to global datastructures
2938makes such a thing impossible in the general case.
2939If the software being re-loaded is not conceived with hot upgrade in mind,
2940this re-loading may cause many errors, warnings or subtle silent problems,
2941as packages, generic function signatures, structures, types, macros, constants, etc.
2942are being redefined incompatibly.
2943It is up to the user to make sure that reloading is possible and has the desired effect.
2944In some cases, extreme measures such as recursively deleting packages,
2945unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
2946and much more are necessary for reloading to happen smoothly.
2947ASDF itself goes through notable pains to make such a hot upgrade possible
2948with respect to its own code, and what it does is ridiculously complex;
2949look at the beginning of @file{asdf.lisp} to see what it does.
2950@end defun
2951
2952@defun run-shell-command
2953
2954This function is obsolete and present only for the sake of backwards-compatibility:
2955``If it's not backwards, it's not compatible''. We strongly discourage its use.
2956Its current behavior is only well-defined on Unix platforms
2957(which includes MacOS X and cygwin). On Windows, anything goes.
2958
2959Instead we recommend the use of such a function as
2960@code{xcvb-driver:run-program/process-output-stream}
2961from the @code{xcvb-driver} system that is distributed with XCVB:
2962@url{http://common-lisp.net/project/xcvb}.
2963It's only alternative that supports
2964as many implementations and operating systems as ASDF does,
2965and provides well-defined behavior outside Unix (i.e. on Windows).
2966(The only unsupported exception is Genera, since on it
2967@code{run-shell-command} doesn't make sense anyway on that platform).
2968
2969This function takes as arguments a @code{format} control-string
2970and arguments to be passed to @code{format} after this control-string
2971to produce a string.
2972This string is a command that will be evaluated with a POSIX shell if possible;
2973yet, on Windows, some implementations will use CMD.EXE,
2974while others (like SBCL) will make an attempt at invoking a POSIX shell
2975(and fail if it is not present).
2976@end defun
2977
2978
2979@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2980@comment  node-name,  next,  previous,  up
2981@chapter Getting the latest version
2982
2983Decide which version you want.
2984The @code{master} branch is where development happens;
2985its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
2986but an occasional regression may happen despite our (limited) test suite.
2987
2988The @code{release} branch is what cautious people should be using;
2989it has usually been tested more, and releases are cut at a point
2990where there isn't any known unresolved issue.
2991
2992You may get the ASDF source repository using git:
2993@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2994
2995You will find the above referenced tags in this repository.
2996You can also browse the repository on
2997@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2998
2999Discussion of ASDF development is conducted on the
3000mailing list
3001@kbd{asdf-devel@@common-lisp.net}.
3002@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
3003
3004
3005@node FAQ, TODO list, Getting the latest version, Top
3006@comment  node-name,  next,  previous,  up
3007@chapter FAQ
3008
3009@section  ``Where do I report a bug?''
3010
3011ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
3012
3013If you're unsure about whether something is a bug, or for general discussion,
3014use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
3015
3016
3017@section ``What has changed between ASDF 1 and ASDF 2?''
3018
3019@subsection What are ASDF 1 and ASDF 2?
3020
3021On May 31st 2010, we have released ASDF 2.
3022ASDF 2 refers to release 2.000 and later.
3023(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
3024ASDF 1 to any release earlier than 1.369 or so.
3025If your ASDF doesn't sport a version, it's an old ASDF 1.
3026
3027ASDF 2 and its release candidates push
3028@code{:asdf2} onto @code{*features*} so that if you are writing
3029ASDF-dependent code you may check for this feature
3030to see if the new API is present.
3031@emph{All} versions of ASDF should have the @code{:asdf} feature.
3032
3033If you are experiencing problems or limitations of any sort with ASDF 1,
3034we recommend that you should upgrade to ASDF 2,
3035or whatever is the latest release.
3036
3037
3038@subsection ASDF can portably name files in subdirectories
3039
3040Common Lisp namestrings are not portable,
3041except maybe for logical pathnamestrings,
3042that themselves have various limitations and require a lot of setup
3043that is itself ultimately non-portable.
3044
3045In ASDF 1, the only portable ways to refer to pathnames inside systems and components
3046were very awkward, using @code{#.(make-pathname ...)} and
3047@code{#.(merge-pathnames ...)}.
3048Even the above were themselves were inadequate in the general case
3049due to host and device issues, unless horribly complex patterns were used.
3050Plenty of simple cases that looked portable actually weren't,
3051leading to much confusion and greavance.
3052
3053ASDF 2 implements its own portable syntax for strings as pathname specifiers.
3054Naming files within a system definition becomes easy and portable again.
3055@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
3056@code{merge-pathnames*},
3057@code{coerce-pathname}.
3058
3059On the other hand, there are places where systems used to accept namestrings
3060where you must now use an explicit pathname object:
3061@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3062must now be written with the @code{#p} syntax:
3063@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
3064
3065@xref{The defsystem grammar,,Pathname specifiers}.
3066
3067
3068@subsection Output translations
3069
3070A popular feature added to ASDF was output pathname translation:
3071@code{asdf-binary-locations}, @code{common-lisp-controller},
3072@code{cl-launch} and other hacks were all implementing it in ways
3073both mutually incompatible and difficult to configure.
3074
3075Output pathname translation is essential to share
3076source directories of portable systems across multiple implementations
3077or variants thereof,
3078or source directories of shared installations of systems across multiple users,
3079or combinations of the above.
3080
3081In ASDF 2, a standard mechanism is provided for that,
3082@code{asdf-output-translations},
3083with sensible defaults, adequate configuration languages,
3084a coherent set of configuration files and hooks,
3085and support for non-Unix platforms.
3086
3087@xref{Controlling where ASDF saves compiled files}.
3088
3089@subsection Source Registry Configuration
3090
3091Configuring ASDF used to require special magic
3092to be applied just at the right moment,
3093between the moment ASDF is loaded and the moment it is used,
3094in a way that is specific to the user,
3095the implementation he is using and the application he is building.
3096
3097This made for awkward configuration files and startup scripts
3098that could not be shared between users, managed by administrators
3099or packaged by distributions.
3100
3101ASDF 2 provides a well-documented way to configure ASDF,
3102with sensible defaults, adequate configuration languages,
3103and a coherent set of configuration files and hooks.
3104
3105We believe it's a vast improvement because it decouples
3106application distribution from library distribution.
3107The application writer can avoid thinking where the libraries are,
3108and the library distributor (dpkg, clbuild, advanced user, etc.)
3109can configure them once and for every application.
3110Yet settings can be easily overridden where needed,
3111so whoever needs control has exactly as much as required.
3112
3113At the same time, ASDF 2 remains compatible
3114with the old magic you may have in your build scripts
3115(using @code{*central-registry*} and
3116@code{*system-definition-search-functions*})
3117to tailor the ASDF configuration to your build automation needs,
3118and also allows for new magic, simpler and more powerful magic.
3119
3120@xref{Controlling where ASDF searches for systems}.
3121
3122
3123@subsection Usual operations are made easier to the user
3124
3125In ASDF 1, you had to use the awkward syntax
3126@code{(asdf:oos 'asdf:load-op :foo)}
3127to load a system,
3128and similarly for @code{compile-op}, @code{test-op}.
3129
3130In ASDF 2, you can use shortcuts for the usual operations:
3131@code{(asdf:load-system :foo)}, and
3132similarly for @code{compile-system}, @code{test-system}.
3133
3134
3135@subsection Many bugs have been fixed
3136
3137The following issues and many others have been fixed:
3138
3139@itemize
3140@item
3141The infamous TRAVERSE function has been revamped significantly,
3142with many bugs squashed.
3143In particular, dependencies were not correctly propagated
3144across submodules within a system but now are.
3145The :version and :feature features and
3146the :force (system1 .. systemN) feature have been fixed.
3147
3148@item
3149Performance has been notably improved for large systems
3150(say with thousands of components) by using
3151hash-tables instead of linear search,
3152and linear-time list accumulation
3153instead of quadratic-time recursive appends.
3154
3155@item
3156Many features used to not be portable,
3157especially where pathnames were involved.
3158Windows support was notably quirky because of such non-portability.
3159
3160@item
3161The internal test suite used to massively fail on many implementations.
3162While still incomplete, it now fully passes
3163on all implementations supported by the test suite,
3164except for GCL (due to GCL bugs).
3165
3166@item
3167Support was lacking for some implementations.
3168ABCL and GCL were notably wholly broken.
3169ECL extensions were not integrated in the ASDF release.
3170
3171@item
3172The documentation was grossly out of date.
3173
3174@end itemize
3175
3176
3177@subsection ASDF itself is versioned
3178
3179Between new features, old bugs fixed, and new bugs introduced,
3180there were various releases of ASDF in the wild,
3181and no simple way to check which release had which feature set.
3182People using or writing systems had to either make worst-case assumptions
3183as to what features were available and worked,
3184or take great pains to have the correct version of ASDF installed.
3185
3186With ASDF 2, we provide a new stable set of working features
3187that everyone can rely on from now on.
3188Use @code{#+asdf2} to detect presence of ASDF 2,
3189@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3190to check the availability of a version no earlier than required.
3191
3192
3193@subsection ASDF can be upgraded
3194
3195When an old version of ASDF was loaded,
3196it was very hard to upgrade ASDF in your current image
3197without breaking everything.
3198Instead you had to exit the Lisp process and
3199somehow arrange to start a new one from a simpler image.
3200Something that can't be done from within Lisp,
3201making automation of it difficult,
3202which compounded with difficulty in configuration,
3203made the task quite hard.
3204Yet as we saw before, the task would have been required
3205to not have to live with the worst case or non-portable
3206subset of ASDF features.
3207
3208With ASDF 2, it is easy to upgrade
3209from ASDF 2 to later versions from within Lisp,
3210and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3211We support hot upgrade of ASDF and any breakage is a bug
3212that we will do our best to fix.
3213There are still limitations on upgrade, though,
3214most notably the fact that after you upgrade ASDF,
3215you must also reload or upgrade all ASDF extensions.
3216
3217@subsection Decoupled release cycle
3218
3219When vendors were releasing their Lisp implementations with ASDF,
3220they had to basically never change version
3221because neither upgrade nor downgrade was possible
3222without breaking something for someone,
3223and no obvious upgrade path was visible and recommendable.
3224
3225With ASDF 2, upgrade is possible, easy and can be recommended.
3226This means that vendors can safely ship a recent version of ASDF,
3227confident that if a user isn't fully satisfied,
3228he can easily upgrade ASDF and deal
3229with a supported recent version of it.
3230This means that release cycles will be causally decoupled,
3231the practical consequence of which will mean faster convergence
3232towards the latest version for everyone.
3233
3234
3235@subsection Pitfalls of the transition to ASDF 2
3236
3237The main pitfalls in upgrading to ASDF 2 seem to be related
3238to the output translation mechanism.
3239
3240@itemize
3241
3242@item
3243Output translations is enabled by default. This may surprise some users,
3244most of them in pleasant way (we hope), a few of them in an unpleasant way.
3245It is trivial to disable output translations.
3246@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3247
3248@item
3249Some systems in the large have been known not to play well with output translations.
3250They were relatively easy to fix.
3251Once again, it is also easy to disable output translations,
3252or to override its configuration.
3253
3254@item
3255The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3256They replace A-B-L, and there is compatibility mode to emulate
3257your previous A-B-L configuration.
3258See @code{enable-asdf-binary-locations-compatibility} in
3259@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3260But thou shall not load ABL on top of ASDF 2.
3261
3262@end itemize
3263
3264Other issues include the following:
3265
3266@itemize
3267
3268@item
3269ASDF pathname designators are now specified in places where they were unspecified,
3270and a few small adjustments have to be made to some non-portable defsystems.
3271Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
3272a logical pathname (or implementation-dependent hierarchical pathname)
3273must now be specified with @code{#p} syntax
3274where the namestring might have previously sufficed;
3275moreover when evaluation is desired @code{#.} must be used,
3276where it wasn't necessary in the toplevel @code{:pathname} argument.
3277
3278@item
3279There is a slight performance bug, notably on SBCL,
3280when initially searching for @file{asd} files,
3281the implicit @code{(directory "/configured/path/**/*.asd")}
3282for every configured path @code{(:tree "/configured/path/")}
3283in your @code{source-registry} configuration can cause a slight pause.
3284Try to @code{(time (asdf:initialize-source-registry))}
3285to see how bad it is or isn't on your system.
3286If you insist on not having this pause,
3287you can avoid the pause by overriding the default source-registry configuration
3288and not use any deep @code{:tree} entry but only @code{:directory} entries
3289or shallow @code{:tree} entries.
3290Or you can fix your implementation to not be quite that slow
3291when recursing through directories.
3292@emph{Update}: performance bug fixed the hard way in 2.010.
3293
3294@item
3295On Windows, only LispWorks supports proper default configuration pathnames
3296based on the Windows registry.
3297Other implementations make do with environment variables,
3298that you may have to define yourself if you're using an older version of Windows.
3299Windows support is somewhat less tested than Unix support.
3300Please help report and fix bugs.
3301
3302@item
3303The mechanism by which one customizes a system so that Lisp files
3304may use a different extension from the default @file{.lisp} has changed.
3305Previously, the pathname for a component was lazily computed when operating on a system,
3306and you would
3307@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3308  (declare (ignorable component system)) "lis")}.
3309Now, the pathname for a component is eagerly computed when defining the system,
3310and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3311and use @code{:default-component-class cl-source-file.lis} as argument to @code{defsystem},
3312as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3313
3314@findex source-file-type
3315
3316
3317@end itemize
3318
3319
3320@section Issues with installing the proper version of ASDF
3321
3322@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3323
3324We recommend you upgrade ASDF.
3325@xref{Loading ASDF,,Upgrading ASDF}.
3326
3327If this does not work, it is a bug, and you should report it.
3328@xref{FAQ, report-bugs, Where do I report a bug}.
3329In the meantime, you can load @file{asdf.lisp} directly.
3330@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3331
3332
3333@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3334
3335Starting with current candidate releases of ASDF 2,
3336it should always be a good time to upgrade to a recent ASDF.
3337You may consult with the maintainer for which specific version they recommend,
3338but the latest @code{release} should be correct.
3339We trust you to thoroughly test it with your implementation before you release it.
3340If there are any issues with the current release,
3341it's a bug that you should report upstream and that we will fix ASAP.
3342
3343As to how to include ASDF, we recommend the following:
3344
3345@itemize
3346@item
3347If ASDF isn't loaded yet, then @code{(require "asdf")}
3348should load the version of ASDF that is bundled with your system.
3349You may have it load some other version configured by the user,
3350if you allow such configuration.
3351
3352@item
3353If your system provides a mechanism to hook into @code{CL:REQUIRE},
3354then it would be nice to add ASDF to this hook the same way that
3355ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3356
3357@item
3358You may, like SBCL, have ASDF be implicitly used to require systems
3359that are bundled with your Lisp distribution.
3360If you do have a few magic systems that come with your implementation
3361in a precompiled way such that one should only use the binary version
3362that goes with your distribution, like SBCL does,
3363then you should add them in the beginning of @code{wrapping-source-registry}.
3364
3365@item
3366If you have magic systems as above, like SBCL does,
3367then we explicitly ask you to @emph{NOT} distribute
3368@file{asdf.asd} as part of those magic systems.
3369You should still include the file @file{asdf.lisp} in your source distribution
3370and precompile it in your binary distribution,
3371but @file{asdf.asd} if included at all,
3372should be secluded from the magic systems,
3373in a separate file hierarchy;
3374alternatively, you may provide the system
3375after renaming it and its @file{.asd} file to e.g.
3376@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3377@code{sb-asdf} and @file{sb-asdf.asd}.
3378Indeed, if you made @file{asdf.asd} a magic system,
3379then users would no longer be able to upgrade ASDF using ASDF itself
3380to some version of their preference that
3381they maintain independently from your Lisp distribution.
3382
3383@item
3384If you do not have any such magic systems, or have other non-magic systems
3385that you want to bundle with your implementation,
3386then you may add them to the @code{default-source-registry},
3387and you are welcome to include @file{asdf.asd} amongst them.
3388
3389@item
3390Please send us upstream any patches you make to ASDF itself,
3391so we can merge them back in for the benefit of your users
3392when they upgrade to the upstream version.
3393
3394@end itemize
3395
3396
3397
3398@section Issues with configuring ASDF
3399
3400@subsection ``How can I customize where fasl files are stored?''
3401
3402@xref{Controlling where ASDF saves compiled files}.
3403
3404Note that in the past there was an add-on to ASDF called
3405@code{ASDF-binary-locations}, developed by Gary King.
3406That add-on has been merged into ASDF proper,
3407then superseded by the @code{asdf-output-translations} facility.
3408
3409Note that use of @code{asdf-output-translations}
3410can interfere with one aspect of your systems
3411--- if your system uses @code{*load-truename*} to find files
3412(e.g., if you have some data files stored with your program),
3413then the relocation that this ASDF customization performs
3414is likely to interfere.
3415Use @code{asdf:system-relative-pathname} to locate a file
3416in the source directory of some system, and
3417use @code{asdf:apply-output-translations} to locate a file
3418whose pathname has been translated by the facility.
3419
3420@subsection ``How can I wholly disable the compiler output cache?''
3421
3422To permanently disable the compiler output cache
3423for all future runs of ASDF, you can:
3424
3425@example
3426mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3427echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3428@end example
3429
3430This assumes that you didn't otherwise configure the ASDF files
3431(if you did, edit them again),
3432and don't somehow override the configuration at runtime
3433with a shell variable (see below) or some other runtime command
3434(e.g. some call to @code{asdf:initialize-output-translations}).
3435
3436To disable the compiler output cache in Lisp processes
3437run by your current shell, try (assuming @code{bash} or @code{zsh})
3438(on Unix and cygwin only):
3439
3440@example
3441export ASDF_OUTPUT_TRANSLATIONS=/:
3442@end example
3443
3444To disable the compiler output cache just in the current Lisp process,
3445use (after loading ASDF but before using it):
3446
3447@example
3448(asdf:disable-output-translations)
3449@end example
3450
3451@section Issues with using and extending ASDF to define systems
3452
3453@subsection ``How can I cater for unit-testing in my system?''
3454
3455ASDF provides a predefined test operation, @code{test-op}.
3456@xref{Predefined operations of ASDF, test-op}.
3457The test operation, however, is largely left to the system definer to specify.
3458@code{test-op} has been
3459a topic of considerable discussion on the
3460@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3461and on the
3462@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3463
3464Here are some guidelines:
3465
3466@itemize
3467@item
3468For a given system, @var{foo}, you will want to define a corresponding
3469test system, such as @var{foo-test}.  The reason that you will want this
3470separate system is that ASDF does not out of the box supply components
3471that are conditionally loaded.  So if you want to have source files
3472(with the test definitions) that will not be loaded except when testing,
3473they should be put elsewhere.
3474
3475@item
3476The @var{foo-test} system can be defined in an asd file of its own or
3477together with @var{foo}.  An aesthetic preference against cluttering up
3478the filesystem with extra asd files should be balanced against the
3479question of whether one might want to directly load @var{foo-test}.
3480Typically one would not want to do this except in early stages of
3481debugging.
3482
3483@item
3484Record that testing is implemented by @var{foo-test}.  For example:
3485@example
3486(defsystem @var{foo}
3487   :in-order-to ((test-op (test-op @var{foo-test})))
3488   ....)
3489
3490(defsystem @var{foo-test}
3491   :depends-on (@var{foo} @var{my-test-library} ...)
3492   ....)
3493@end example
3494@end itemize
3495
3496This procedure will allow you to support users who do not wish to
3497install your test framework.
3498
3499One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3500does not return a value.  So in current versions of ASDF there is no
3501reliable programmatic means of determining whether or not a set of tests
3502has passed, or which tests have failed.  The user must simply read the
3503console output.  This limitation has been the subject of much
3504discussion.
3505
3506@subsection ``How can I cater for documentation generation in my system?''
3507
3508The ASDF developers are currently working to add a @code{doc-op}
3509to the set of predefined ASDF operations.
3510@xref{Predefined operations of ASDF}.
3511See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3512
3513
3514
3515@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3516
3517See @code{cffi}'s @code{cffi-grovel}.
3518
3519@anchor{report-bugs}
3520
3521
3522@subsection ``I want to put my module's files at the top level.  How do I do this?''
3523
3524By default, the files contained in an asdf module go
3525in a subdirectory with the same name as the module.
3526However, this can be overridden by adding a @code{:pathname ""} argument
3527to the module description.
3528For example, here is how it could be done
3529in the spatial-trees ASDF system definition for ASDF 2:
3530
3531@example
3532(asdf:defsystem :spatial-trees
3533  :components
3534  ((:module base
3535            :pathname ""
3536            :components
3537            ((:file "package")
3538             (:file "basedefs" :depends-on ("package"))
3539             (:file "rectangles" :depends-on ("package"))))
3540   (:module tree-impls
3541            :depends-on (base)
3542            :pathname ""
3543            :components
3544            ((:file "r-trees")
3545             (:file "greene-trees" :depends-on ("r-trees"))
3546             (:file "rstar-trees" :depends-on ("r-trees"))
3547             (:file "rplus-trees" :depends-on ("r-trees"))
3548             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3549   (:module viz
3550            :depends-on (base)
3551            :pathname ""
3552            :components
3553            ((:static-file "spatial-tree-viz.lisp")))
3554   (:module tests
3555            :depends-on (base)
3556            :pathname ""
3557            :components
3558            ((:static-file "spatial-tree-test.lisp")))
3559   (:static-file "LICENCE")
3560   (:static-file "TODO")))
3561@end example
3562
3563All of the files in the @code{tree-impls} module are at the top level,
3564instead of in a @file{tree-impls/} subdirectory.
3565
3566Note that the argument to @code{:pathname} can be either a pathname object or a string.
3567A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3568but this is discouraged because the results of parsing a namestring are not portable.
3569A pathname can only be portably constructed with such syntax as
3570@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3571and similarly the current directory can only be portably specified as
3572@code{#.(make-pathname :directory '(:relative))}.
3573However, as of ASDF 2, you can portably use a string to denote a pathname.
3574The string will be parsed as a @code{/}-separated path from the current directory,
3575such that the empty string @code{""} denotes the current directory, and
3576@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3577portably denotes the same subdirectory as above.
3578When files are specified, the last @code{/}-separated component is interpreted
3579either as the name component of a pathname
3580(if the component class specifies a pathname type),
3581or as a name component plus optional dot-separated type component
3582(if the component class doesn't specifies a pathname type).
3583
3584@subsection How do I create a system definition where all the source files have a .cl extension?
3585
3586Starting with ASDF 2.014.14, you may just pass
3587the builtin class @code{cl-source-file.cl} as
3588the @code{:default-component-class} argument to @code{defsystem}:
3589
3590@lisp
3591(defsystem my-cl-system
3592  :default-component-class cl-source-file.cl
3593  ...)
3594@end lisp
3595
3596Another builtin class @code{cl-source-file.lsp} is offered
3597for files ending in @file{.lsp}.
3598
3599If you want to use a different extension
3600for which ASDF doesn't provide builtin support,
3601or want to support versions of ASDF
3602earlier than 2.014.14 (but later than 2.000),
3603you can define a class as follows:
3604
3605@lisp
3606;; Prologue: make sure we're using a sane package.
3607(defpackage :my-asdf-extension
3608   (:use :asdf :common-lisp)
3609   (:export #:cl-source-file.lis))
3610(in-package :my-asdf-extension)
3611
3612(defclass cl-source-file.lis (cl-source-file)
3613   ((type :initform "lis")))
3614@end lisp
3615
3616Then you can use it as follows:
3617@lisp
3618(defsystem my-cl-system
3619  :default-component-class my-asdf-extension:cl-source-file.lis
3620  ...)
3621@end lisp
3622
3623Of course, if you're in the same package, e.g. in the same file,
3624you won't need to use the package qualifier before @code{cl-source-file.lis}.
3625Actually, if all you're doing is defining this class
3626and using it in the same file without other fancy definitions,
3627you might skip package complications:
3628
3629@lisp
3630(in-package :asdf)
3631(defclass cl-source-file.lis (cl-source-file)
3632   ((type :initform "lis")))
3633(defsystem my-cl-system
3634  :default-component-class cl-source-file.lis
3635  ...)
3636@end lisp
3637
3638It is possible to achieve the same effect
3639in a way that supports both ASDF 1 and ASDF 2,
3640but really, friends don't let friends use ASDF 1.
3641Please upgrade to ASDF 2.
3642In short, though: do same as above, but
3643@emph{before} you use the class in a @code{defsystem},
3644you also define the following method:
3645
3646@lisp
3647(defmethod source-file-type ((f cl-source-file.lis) (m module))
3648  (declare (ignorable f m))
3649  "lis")
3650@end lisp
3651
3652
3653@node  TODO list, Inspiration, FAQ, Top
3654@comment  node-name,  next,  previous,  up
3655@chapter TODO list
3656
3657Here is an old list of things to do,
3658in addition to the bugs that are now tracked on launchpad:
3659@url{https://launchpad.net/asdf}.
3660
3661@section Outstanding spec questions, things to add
3662
3663** packaging systems
3664
3665*** manual page component?
3666
3667** style guide for .asd files
3668
3669You should either use keywords or be careful
3670with the package that you evaluate defsystem forms in.
3671Otherwise @code{(defsystem partition ...)}
3672being read in the @code{cl-user} package
3673will intern a @code{cl-user:partition} symbol,
3674which will then collide with the @code{partition:partition} symbol.
3675
3676Actually there's a hairier packages problem to think about too.
3677@code{in-order-to} is not a keyword:
3678if you read @code{defsystem} forms in a package that doesn't use ASDF,
3679odd things might happen.
3680
3681
3682** extending defsystem with new options
3683
3684You might not want to write a whole parser,
3685but just to add options to the existing syntax.
3686Reinstate @code{parse-option} or something akin.
3687
3688
3689** document all the error classes
3690
3691** what to do with compile-file failure
3692
3693Should check the primary return value from compile-file and see if
3694that gets us any closer to a sensible error handling strategy
3695
3696** foreign files
3697
3698lift unix-dso stuff from db-sockets
3699
3700** Diagnostics
3701
3702A ``dry run'' of an operation can be made with the following form:
3703
3704@lisp
3705(traverse (make-instance '<operation-name>)
3706          (find-system <system-name>)
3707          'explain)
3708@end lisp
3709
3710This uses unexported symbols.
3711What would be a nice interface for this functionality?
3712
3713@section Missing bits in implementation
3714
3715** reuse the same scratch package whenever a system is reloaded from disk
3716
3717** proclamations probably aren't
3718
3719** when a system is reloaded with fewer components than it previously had, odd things happen
3720
3721We should do something inventive when processing a @code{defsystem} form,
3722like take the list of kids and @code{setf} the slot to @code{nil},
3723then transfer children from old to new list as they're found.
3724
3725** (stuff that might happen later)
3726
3727*** Propagation of the @code{:force} option.
3728
3729``I notice that
3730
3731        @code{(asdf:compile-system :araneida :force t)}
3732
3733also forces compilation of every other system the @code{:araneida} system depends on.
3734This is rarely useful to me;
3735usually, when I want to force recompilation of something more than a single source file,
3736I want to recompile only one system.
3737So it would be more useful to have @code{make-sub-operation}
3738refuse to propagate @code{:force t} to other systems, and
3739propagate only something like @code{:force :recursively}.
3740
3741Ideally what we actually want is some kind of criterion that says
3742to which systems (and which operations) a @code{:force} switch will propagate.
3743
3744The problem is perhaps that ``force'' is a pretty meaningless concept.
3745How obvious is it that @code{load :force t} should force @emph{compilation}?
3746But we don't really have the right dependency setup
3747for the user to compile @code{:force t} and expect it to work
3748(files will not be loaded after compilation, so the compile
3749environment for subsequent files will be emptier than it needs to be)
3750
3751What does the user actually want to do when he forces?
3752Usually, for me, update for use with a new version of the Lisp compiler.
3753Perhaps for recovery when he suspects that something has gone wrong.
3754Or else when he's changed compilation options or configuration
3755in some way that's not reflected in the dependency graph.
3756
3757Other possible interface: have a ``revert'' function akin to @code{make clean}.
3758
3759@lisp
3760(asdf:revert 'asdf:compile-op 'araneida)
3761@end lisp
3762
3763would delete any files produced by @code{(compile-system :araneida)}.
3764Of course, it wouldn't be able to do much about stuff in the image itself.
3765
3766How would this work?
3767
3768@code{traverse}
3769
3770There's a difference between a module's dependencies (peers)
3771and its components (children).
3772Perhaps there's a similar difference in operations?
3773For example, @code{(load "use") depends-on (load "macros")} is a peer,
3774whereas @code{(load "use") depends-on (compile "use")}
3775is more of a ``subservient'' relationship.
3776
3777@node  Inspiration, Concept Index, TODO list, Top
3778@comment  node-name,  next,  previous,  up
3779@chapter Inspiration
3780
3781@section mk-defsystem (defsystem-3.x)
3782
3783We aim to solve basically the same problems as @code{mk-defsystem} does.
3784However, our architecture for extensibility
3785better exploits CL language features (and is documented),
3786and we intend to be portable rather than just widely-ported.
3787No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3788that implementation has the unenviable task
3789of supporting pre-ANSI implementations, which is no longer necessary.
3790
3791The surface defsystem syntax of asdf is more-or-less compatible with
3792@code{mk-defsystem}, except that we do not support
3793the @code{source-foo} and @code{binary-foo} prefixes
3794for separating source and binary files, and
3795we advise the removal of all options to specify pathnames.
3796
3797The @code{mk-defsystem} code for topologically sorting
3798a module's dependency list was very useful.
3799
3800@section defsystem-4 proposal
3801
3802Marco and Peter's proposal for defsystem 4 served as the driver for
3803many of the features in here.  Notable differences are:
3804
3805@itemize
3806@item
3807We don't specify output files or output file extensions
3808as part of the system.
3809
3810If you want to find out what files an operation would create,
3811ask the operation.
3812
3813@item
3814We don't deal with CL packages
3815
3816If you want to compile in a particular package, use an @code{in-package} form
3817in that file (ilisp / SLIME will like you more if you do this anyway)
3818
3819@item
3820There is no proposal here that @code{defsystem} does version control.
3821
3822A system has a given version which can be used to check dependencies,
3823but that's all.
3824@end itemize
3825
3826The defsystem 4 proposal tends to look more at the external features,
3827whereas this one centres on a protocol for system introspection.
3828
3829@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
3830
3831Available in updated-for-CL form on the web at
3832@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3833
3834In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3835and concept to deal with creating component trees
3836from @code{defsystem} surface syntax.
3837[ this is not true right now, though it used to be and
3838probably will be again soon ]
3839
3840
3841@c -------------------
3842
3843
3844@node Concept Index, Function and Class Index, Inspiration, Top
3845@unnumbered Concept Index
3846
3847@printindex cp
3848
3849@node Function and Class Index, Variable Index, Concept Index, Top
3850@unnumbered Function and Class Index
3851
3852@printindex fn
3853
3854@node Variable Index,  , Function and Class Index, Top
3855@unnumbered Variable Index
3856
3857@printindex vr
3858
3859@bye
Note: See TracBrowser for help on using the repository browser.