source: trunk/abcl/doc/asdf/asdf.texinfo @ 13311

Last change on this file since 13311 was 13311, checked in by Mark Evenson, 10 years ago

Update to asdf-2.016.

File size: 137.0 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro &allow-other-keys
10&allow-other-keys
11@end macro
12@macro &optional
13&optional
14@end macro
15@macro &rest
16&rest
17@end macro
18@macro &key
19&key
20@end macro
21@macro &body
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2011 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2011 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top asdf: another system definition facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Loading ASDF::
91* Configuring ASDF::
92* Using ASDF::
93* Defining systems with defsystem::
94* The object model of ASDF::
95* Controlling where ASDF searches for systems::
96* Controlling where ASDF saves compiled files::
97* Error handling::
98* Miscellaneous additional functionality::
99* Getting the latest version::
100* FAQ::
101* TODO list::
102* Inspiration::
103* Concept Index::
104* Function and Class Index::
105* Variable Index::
106
107@c @detailmenu
108@c  --- The Detailed Node Listing ---
109
110@c Defining systems with defsystem
111
112@c * The defsystem form::
113@c * A more involved example::
114@c * The defsystem grammar::
115@c * Other code in .asd files::
116
117@c The object model of ASDF
118
119@c * Operations::
120@c * Components::
121
122@c Operations
123
124@c * Predefined operations of ASDF::
125@c * Creating new operations::
126
127@c Components
128
129@c * Common attributes of components::
130@c * Pre-defined subclasses of component::
131@c * Creating new component types::
132
133@c properties
134
135@c * Pre-defined subclasses of component::
136@c * Creating new component types::
137
138@c @end detailmenu
139@end menu
140
141@end ifnottex
142
143@c -------------------
144
145@node Introduction, Loading ASDF, Top, Top
146@comment  node-name,  next,  previous,  up
147@chapter Introduction
148@cindex ASDF-related features
149@vindex *features*
150@cindex Testing for ASDF
151@cindex ASDF versions
152@cindex :asdf
153@cindex :asdf2
154
155ASDF is Another System Definition Facility:
156a tool for specifying how systems of Common Lisp software
157are comprised of components (sub-systems and files),
158and how to operate on these components in the right order
159so that they can be compiled, loaded, tested, etc.
160
161ASDF presents three faces:
162one for users of Common Lisp software who want to reuse other people's code,
163one for writers of Common Lisp software who want to specify how to build their systems,
164one for implementers of Common Lisp extensions who want to extend the build system.
165@xref{Using ASDF,,Loading a system},
166to learn how to use ASDF to load a system.
167@xref{Defining systems with defsystem},
168to learn how to define a system of your own.
169@xref{The object model of ASDF}, for a description of
170the ASDF internals and how to extend ASDF.
171
172@emph{Nota Bene}:
173We have released ASDF 2.000 on May 31st 2010.
174Subsequent releases of ASDF 2 have since then been included
175in all actively maintained CL implementations that used to bundle ASDF 1,
176plus some implementations that didn't use to,
177and has been made to work with all actively used CL implementations and a few more.
178@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
179Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 on the fly.
180For this reason, we have stopped supporting ASDF 1;
181if you are using ASDF 1 and are experiencing any kind of issues or limitations,
182we recommend you upgrade to ASDF 2
183--- and we explain how to do that. @xref{Loading ASDF}.
184
185Also note that ASDF is not to be confused with ASDF-Install.
186ASDF-Install is not part of ASDF, but a separate piece of software.
187ASDF-Install is also unmaintained and obsolete.
188We recommend you use Quicklisp instead,
189which works great and is being actively maintained.
190If you want to download software from version control instead of tarballs,
191so you may more easily modify it, we recommend clbuild.
192
193
194@node Loading ASDF, Configuring ASDF, Introduction, Top
195@comment  node-name,  next,  previous,  up
196@chapter Loading ASDF
197@vindex *central-registry*
198@cindex link farm
199@findex load-system
200@findex compile-system
201@findex test-system
202@cindex system directory designator
203@findex operate
204@findex oos
205
206@c @menu
207@c * Installing ASDF::
208@c @end menu
209
210
211@section Loading a pre-installed ASDF
212
213Many Lisp implementations include a copy of ASDF.
214You can usually load this copy using Common Lisp's @code{require} function:
215
216@lisp
217(require "asdf")
218@end lisp
219
220As of the writing of this manual,
221the following implementations provide ASDF 2 this way:
222abcl allegro ccl clisp cmucl ecl sbcl xcl.
223The following implementations don't provide it yet but will in a future release:
224lispworks scl.
225The following implementations are obsolete and most probably will never bundle it:
226cormancl gcl genera mcl.
227
228If the implementation you are using doesn't provide ASDF 2,
229see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
230If that implementation is still actively maintained,
231you may also send a bug report to your Lisp vendor and complain
232about their failing to provide ASDF.
233
234NB: all implementations except clisp also accept
235@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
236For portability's sake, you probably want to use @code{(require "asdf")}.
237
238
239@section Checking whether ASDF is loaded
240
241To check whether ASDF is properly loaded in your current Lisp image,
242you can run this form:
243
244@lisp
245(asdf:asdf-version)
246@end lisp
247
248If it returns a string,
249that is the version of ASDF that is currently installed.
250
251If it raises an error,
252then either ASDF is not loaded, or
253you are using an old version of ASDF.
254
255You can check whether an old version is loaded
256by checking if the ASDF package is present.
257The form below will allow you to programmatically determine
258whether a recent version is loaded, an old version is loaded,
259or none at all:
260
261@lisp
262(or #+asdf2 (asdf:asdf-version) #+asdf :old)
263@end lisp
264
265If it returns a version number, that's the version of ASDF installed.
266If it returns the keyword @code{:OLD},
267then you're using an old version of ASDF (from before 1.635).
268If it returns @code{NIL} then ASDF is not installed.
269
270If you are experiencing problems with ASDF,
271please try upgrading to the latest released version,
272using the method below,
273before you contact us and raise an issue.
274
275
276@section Upgrading ASDF
277
278If your implementation does provide ASDF 2 or later,
279and you want to upgrade to a more recent version,
280just install ASDF like any other package
281(see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below),
282configure ASDF as usual (see @pxref{Configuring ASDF} below),
283and upgrade with:
284
285@lisp
286(require "asdf")
287(asdf:load-system :asdf)
288@end lisp
289
290If on the other hand, your implementation only provides an old ASDF,
291you will require a special configuration step and an old-style loading:
292
293@lisp
294(require "asdf")
295(push #p"@var{/path/to/new/asdf/}" asdf:*central-registry*)
296(asdf:oos 'asdf:load-op :asdf)
297@end lisp
298
299Don't forget the trailing @code{/} at the end of your pathname.
300
301Also, note that older versions of ASDF won't redirect their output,
302or at least won't do it according to your usual ASDF 2 configuration.
303You therefore need write access on the directory
304where you install the new ASDF,
305and make sure you're not using it
306for multiple mutually incompatible implementations.
307At worst, you may have to have multiple copies of the new ASDF,
308e.g. one per implementation installation, to avoid clashes.
309Note that to our knowledge all implementations that provide ASDF
310provide ASDF 2 in their latest release, so
311you may want to upgrade your implementation rather than go through that hoop.
312
313Finally, note that there are some limitations to upgrading ASDF:
314@itemize
315@item
316Any ASDF extension becomes invalid, and will need to be reloaded.
317This applies to e.g. CFFI-Grovel, or to hacks used by ironclad, etc.
318Starting with ASDF 2.014.8, ASDF will actually invalidate
319all previously loaded systems when it is loaded on top of
320a different ASDF version.
321@item
322Until all implementations provide ASDF 2.015 or later,
323it is safer if you upgrade ASDF and its extensions as a special step
324at the very beginning of whatever script you are running,
325before you start using ASDF to load anything else.
326@item
327Until all implementations provide ASDF 2.015 or later,
328it is unsafe to upgrade ASDF as part of loading a system
329that depends on a more recent version of ASDF,
330since the new one might shadow the old one while the old one is running,
331and the running old one will be confused
332when extensions are loaded into the new one.
333In the meantime, we recommend that your systems should @emph{not} specify
334@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "2.010"))},
335but instead that they check that a recent enough ASDF is installed,
336with such code as:
337@example
338(unless (or #+asdf2 (asdf:version-satisfies
339                     (asdf:asdf-version) *required-asdf-version*))
340  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
341@end example
342@end itemize
343
344
345@section Loading an otherwise installed ASDF
346
347If your implementation doesn't include ASDF,
348if for some reason the upgrade somehow fails,
349does not or cannot apply to your case,
350you will have to install the file @file{asdf.lisp}
351somewhere and load it with:
352
353@lisp
354(load "/path/to/your/installed/asdf.lisp")
355@end lisp
356
357The single file @file{asdf.lisp} is all you normally need to use ASDF.
358
359You can extract this file from latest release tarball on the
360@url{http://common-lisp.net/project/asdf/,ASDF website}.
361If you are daring and willing to report bugs, you can get
362the latest and greatest version of ASDF from its git repository.
363@xref{Getting the latest version}.
364
365For maximum convenience you might want to have ASDF loaded
366whenever you start your Lisp implementation,
367for example by loading it from the startup script or dumping a custom core
368--- check your Lisp implementation's manual for details.
369
370
371@node Configuring ASDF, Using ASDF, Loading ASDF, Top
372@comment  node-name,  next,  previous,  up
373
374@chapter Configuring ASDF
375
376@section Configuring ASDF to find your systems
377
378So it may compile and load your systems, ASDF must be configured to find
379the @file{.asd} files that contain system definitions.
380
381Since ASDF 2, the preferred way to configure where ASDF finds your systems is
382the @code{source-registry} facility,
383fully described in its own chapter of this manual.
384@xref{Controlling where ASDF searches for systems}.
385
386The default location for a user to install Common Lisp software is under
387@file{~/.local/share/common-lisp/source/}.
388If you install software there, you don't need further configuration.
389If you're installing software yourself at a location that isn't standard,
390you have to tell ASDF where you installed it. See below.
391If you're using some tool to install software (e.g. Quicklisp),
392the authors of that tool should already have configured ASDF.
393
394The simplest way to add a path to your search path,
395say @file{/home/luser/.asd-link-farm/}
396is to create the directory
397@file{~/.config/common-lisp/source-registry.conf.d/}
398and there create a file with any name of your choice,
399and with the type @file{conf},
400for instance @file{42-asd-link-farm.conf}
401containing the line:
402
403@kbd{(:directory "/home/luser/.asd-link-farm/")}
404
405If you want all the subdirectories under @file{/home/luser/lisp/}
406to be recursively scanned for @file{.asd} files, instead use:
407
408@kbd{(:tree "/home/luser/lisp/")}
409
410Note that your Operating System distribution or your system administrator
411may already have configured system-managed libraries for you.
412
413The required @file{.conf} extension allows you to have disabled files
414or editor backups (ending in @file{~}), and works portably
415(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
416Excluded are files the name of which start with a @file{.} character.
417It is customary to start the filename with two digits
418that specify the order in which the directories will be scanned.
419
420ASDF will automatically read your configuration
421the first time you try to find a system.
422You can reset the source-registry configuration with:
423
424@lisp
425(asdf:clear-source-registry)
426@end lisp
427
428And you probably should do so before you dump your Lisp image,
429if the configuration may change
430between the machine where you save it at the time you save it
431and the machine you resume it at the time you resume it.
432
433
434@section Configuring ASDF to find your systems -- old style
435
436The old way to configure ASDF to find your systems is by
437@code{push}ing directory pathnames onto the variable
438@code{asdf:*central-registry*}.
439
440You must configure this variable between the time you load ASDF
441and the time you first try to use it.
442Loading and configuring ASDF presumably happen
443as part of some initialization script that builds or starts
444your Common Lisp software system.
445(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
446
447The @code{asdf:*central-registry*} is empty by default in ASDF 2,
448but is still supported for compatibility with ASDF 1.
449When used, it takes precedence over the above source-registry@footnote{
450It is possible to further customize
451the system definition file search.
452That's considered advanced use, and covered later:
453search forward for
454@code{*system-definition-search-functions*}.
455@xref{Defining systems with defsystem}.}.
456
457For instance, if you wanted ASDF to find the @file{.asd} file
458@file{/home/me/src/foo/foo.asd} your initialization script
459could after it loads ASDF with @code{(require "asdf")}
460configure it with:
461
462@lisp
463(push "/home/me/src/foo/" asdf:*central-registry*)
464@end lisp
465
466Note the trailing slash: when searching for a system,
467ASDF will evaluate each entry of the central registry
468and coerce the result to a pathname@footnote{
469ASDF will indeed call @code{EVAL} on each entry.
470It will also skip entries that evaluate to @code{NIL}.
471
472Strings and pathname objects are self-evaluating,
473in which case the @code{EVAL} step does nothing;
474but you may push arbitrary SEXP onto the central registry,
475that will be evaluated to compute e.g. things that depend
476on the value of shell variables or the identity of the user.
477
478The variable @code{asdf:*central-registry*} is thus a list of
479``system directory designators''.
480A @dfn{system directory designator} is a form
481which will be evaluated whenever a system is to be found,
482and must evaluate to a directory to look in.
483By ``directory'' here, we mean
484``designator for a pathname with a supplied DIRECTORY component''.
485}
486at which point the presence of the trailing directory name separator
487is necessary to tell Lisp that you're discussing a directory
488rather than a file.
489
490Typically, however, there are a lot of @file{.asd} files, and
491a common idiom was to have to put
492a bunch of @emph{symbolic links} to @file{.asd} files
493in a common directory
494and push @emph{that} directory (the ``link farm'')
495to the
496@code{asdf:*central-registry*}
497instead of pushing each of the many involved directories
498to the @code{asdf:*central-registry*}.
499ASDF knows how to follow such @emph{symlinks}
500to the actual file location when resolving the paths of system components
501(on Windows, you can use Windows shortcuts instead of POSIX symlinks).
502
503For example, if @code{#p"/home/me/cl/systems/"} (note the trailing slash)
504is a member of @code{*central-registry*}, you could set up the
505system @var{foo} for loading with asdf with the following
506commands at the shell:
507
508@example
509$ cd /home/me/cl/systems/
510$ ln -s ~/src/foo/foo.asd .
511@end example
512
513This old style for configuring ASDF is not recommended for new users,
514but it is supported for old users, and for users who want to programmatically
515control what directories are added to the ASDF search path.
516
517
518@section Configuring where ASDF stores object files
519@findex clear-output-locations
520
521ASDF lets you configure where object files will be stored.
522Sensible defaults are provided and
523you shouldn't normally have to worry about it.
524
525This allows the same source code repository may be shared
526between several versions of several Common Lisp implementations,
527between several users using different compilation options
528and without write privileges on shared source directories, etc.
529This also allows to keep source directories uncluttered
530by plenty of object files.
531
532Starting with ASDF 2, the @code{asdf-output-translations} facility
533was added to ASDF itself, that controls where object files will be stored.
534This facility is fully described in a chapter of this manual,
535@ref{Controlling where ASDF saves compiled files}.
536
537The simplest way to add a translation to your search path,
538say from @file{/foo/bar/baz/quux/}
539to @file{/where/i/want/my/fasls/}
540is to create the directory
541@file{~/.config/common-lisp/asdf-output-translations.conf.d/}
542and there create a file with any name of your choice and the type @file{conf},
543for instance @file{42-bazquux.conf}
544containing the line:
545
546@kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
547
548To disable output translations for source under a given directory,
549say @file{/toto/tata/}
550you can create a file @file{40-disable-toto.conf}
551with the line:
552
553@kbd{("/toto/tata/")}
554
555To wholly disable output translations for all directories,
556you can create a file @file{00-disable.conf}
557with the line:
558
559@kbd{(t t)}
560
561Note that your Operating System distribution or your system administrator
562may already have configured translations for you.
563In absence of any configuration, the default is to redirect everything
564under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
565@xref{Controlling where ASDF searches for systems}, for full details.
566
567The required @file{.conf} extension allows you to have disabled files
568or editor backups (ending in @file{~}), and works portably
569(for instance, it is a pain to allow both empty and non-empty extension on CLISP).
570Excluded are files the name of which start with a @file{.} character.
571It is customary to start the filename with two digits
572that specify the order in which the directories will be scanned.
573
574ASDF will automatically read your configuration
575the first time you try to find a system.
576You can reset the source-registry configuration with:
577
578@lisp
579(asdf:clear-output-translations)
580@end lisp
581
582And you probably should do so before you dump your Lisp image,
583if the configuration may change
584between the machine where you save it at the time you save it
585and the machine you resume it at the time you resume it.
586
587Finally note that before ASDF 2,
588other ASDF add-ons offered the same functionality,
589each in subtly different and incompatible ways:
590ASDF-Binary-Locations, cl-launch, common-lisp-controller.
591ASDF-Binary-Locations is now not needed anymore and should not be used.
592cl-launch 3.000 and common-lisp-controller 7.2 have been updated
593to just delegate this functionality to ASDF.
594
595@node Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
596@comment  node-name,  next,  previous,  up
597
598@chapter Using ASDF
599
600@section Loading a system
601
602The system @var{foo} is loaded (and compiled, if necessary)
603by evaluating the following Lisp form:
604
605@example
606(asdf:load-system :@var{foo})
607@end example
608
609On some implementations (namely recent versions of
610ABCL, Clozure CL, CLISP, CMUCL, ECL, SBCL and SCL),
611ASDF hooks into the @code{CL:REQUIRE} facility
612and you can just use:
613
614@example
615(require :@var{foo})
616@end example
617
618In older versions of ASDF, you needed to use
619@code{(asdf:oos 'asdf:load-op :@var{foo})}.
620If your ASDF is too old to provide @code{asdf:load-system} though
621we recommend that you upgrade to ASDF 2.
622@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
623
624Note the name of a system is specified as a string or a symbol,
625typically a keyword.
626If a symbol (including a keyword), its name is taken and lowercased.
627The name must be a suitable value for the @code{:name} initarg
628to @code{make-pathname} in whatever filesystem the system is to be found.
629The lower-casing-symbols behaviour is unconventional,
630but was selected after some consideration.
631Observations suggest that the type of systems we want to support
632either have lowercase as customary case (unix, mac, windows)
633or silently convert lowercase to uppercase (lpns),
634so this makes more sense than attempting to use @code{:case :common},
635which is reported not to work on some implementations
636
637
638@section Other Operations
639
640ASDF provides three commands for the most common system operations:
641@code{load-system}, @code{compile-system} or @code{test-system}.
642
643Because ASDF is an extensible system
644for defining @emph{operations} on @emph{components},
645it also provides a generic function @code{operate}
646(which is usually abbreviated by @code{oos}).
647You'll use @code{oos} whenever you want to do something beyond
648compiling, loading and testing.
649
650Output from ASDF and ASDF extensions are supposed to be sent
651to the CL stream @code{*standard-output*},
652and so rebinding that stream around calls to @code{asdf:operate}
653should redirect all output from ASDF operations.
654
655Reminder: before ASDF can operate on a system, however,
656it must be able to find and load that system's definition.
657@xref{Configuring ASDF,,Configuring ASDF to find your systems}.
658
659
660@section Summary
661
662To use ASDF:
663
664@itemize
665@item
666Load ASDF itself into your Lisp image, either through
667@code{(require "asdf")} or else through
668@code{(load "/path/to/asdf.lisp")}.
669
670@item
671Make sure ASDF can find system definitions
672thanks to proper source-registry configuration.
673
674@item
675Load a system with @code{(load-system :my-system)}
676or use some other operation on some system of your choice.
677
678@end itemize
679
680@section Moving on
681
682That's all you need to know to use ASDF to load systems written by others.
683The rest of this manual deals with writing system definitions
684for Common Lisp software you write yourself,
685including how to extend ASDF to define new operation and component types.
686
687
688@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
689@comment  node-name,  next,  previous,  up
690@chapter Defining systems with defsystem
691
692This chapter describes how to use asdf to define systems and develop
693software.
694
695
696@menu
697* The defsystem form::
698* A more involved example::
699* The defsystem grammar::
700* Other code in .asd files::
701@end menu
702
703@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
704@comment  node-name,  next,  previous,  up
705@section The defsystem form
706
707Systems can be constructed programmatically
708by instantiating components using @code{make-instance}.
709Most of the time, however, it is much more practical to use
710a static @code{defsystem} form.
711This section begins with an example of a system definition,
712then gives the full grammar of @code{defsystem}.
713
714Let's look at a simple system.
715This is a complete file that would
716usually be saved as @file{hello-lisp.asd}:
717
718@lisp
719(in-package :asdf)
720
721(defsystem "hello-lisp"
722  :description "hello-lisp: a sample Lisp system."
723  :version "0.2.1"
724  :author "Joe User <joe@@example.com>"
725  :licence "Public Domain"
726  :components ((:file "packages")
727               (:file "macros" :depends-on ("packages"))
728               (:file "hello" :depends-on ("macros"))))
729@end lisp
730
731Some notes about this example:
732
733@itemize
734
735@item
736The file starts with an @code{in-package} form
737to use package @code{asdf}.
738You could instead start your definition by using
739a qualified name @code{asdf:defsystem}.
740
741@item
742If in addition to simply using @code{defsystem},
743you are going to define functions,
744create ASDF extension, globally bind symbols, etc.,
745it is recommended that to avoid namespace pollution between systems,
746you should create your own package for that purpose,
747for instance replacing the above @code{(in-package :asdf)} with:
748
749@lisp
750(defpackage :foo-system
751  (:use :cl :asdf))
752
753(in-package :foo-system)
754@end lisp
755
756@item
757The @code{defsystem} form defines a system named @code{hello-lisp}
758that contains three source files:
759@file{packages}, @file{macros} and @file{hello}.
760
761@item
762The file @file{macros} depends on @file{packages}
763(presumably because the package it's in is defined in @file{packages}),
764and the file @file{hello} depends on @file{macros}
765(and hence, transitively on @file{packages}).
766This means that ASDF will compile and load @file{packages} and @file{macros}
767before starting the compilation of file @file{hello}.
768
769@item
770The files are located in the same directory
771as the file with the system definition.
772ASDF resolves symbolic links (or Windows shortcuts)
773before loading the system definition file and
774stores its location in the resulting system@footnote{
775It is possible, though almost never necessary, to override this behaviour.}.
776This is a good thing because the user can move the system sources
777without having to edit the system definition.
778
779@c FIXME: Should have cross-reference to "Version specifiers" in the
780@c defsystem grammar, but the cross-referencing is so broken by
781@c insufficient node breakdown that I have not put one in.
782@item
783Make sure you know how the @code{:version} numbers will be parsed!  They
784are parsed as period-separated lists of integers.  I.e., in the example,
785@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
786In particular, version @code{0.2.1} is interpreted the same as
787@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
788even though the two are the same when interpreted as decimal fractions.
789@cindex version specifiers
790@cindex :version
791
792@end itemize
793
794@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
795@comment  node-name,  next,  previous,  up
796@section A more involved example
797
798Let's illustrate some more involved uses of @code{defsystem} via a
799slightly convoluted example:
800
801@lisp
802(defsystem "foo"
803  :version "1.0.0"
804  :components ((:module "mod"
805                            :components ((:file "bar")
806                                                  (:file"baz")
807                                                  (:file "quux"))
808                            :perform (compile-op :after (op c)
809                                                  (do-something c))
810                            :explain (compile-op :after (op c)
811                                            (explain-something c)))
812                         (:file "blah")))
813@end lisp
814
815The @code{:module} component named @code{"mod"} is a collection of three files,
816which will be located in a subdirectory of the main code directory named
817@file{mod} (this location can be overridden; see the discussion of the
818@code{:pathname} option in @ref{The defsystem grammar}).
819
820The method-form tokens provide a shorthand for defining methods on
821particular components.  This part
822
823@lisp
824                :perform (compile-op :after (op c)
825                          (do-something c))
826                :explain (compile-op :after (op c)
827                          (explain-something c))
828@end lisp
829
830has the effect of
831
832@lisp
833(defmethod perform :after ((op compile-op) (c (eql ...)))
834           (do-something c))
835(defmethod explain :after ((op compile-op) (c (eql ...)))
836           (explain-something c))
837@end lisp
838
839where @code{...} is the component in question.
840In this case @code{...} would expand to something like
841
842@lisp
843(find-component (find-system "foo") "mod")
844@end lisp
845
846For more details on the syntax of such forms, see @ref{The defsystem
847grammar}.
848For more details on what these methods do, @pxref{Operations} in
849@ref{The object model of ASDF}.
850
851@c The following plunge into the weeds is not appropriate in this
852@c location. [2010/10/03:rpg]
853@c note that although this also supports @code{:before} methods,
854@c they may not do what you want them to ---
855@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
856@c will run after all the dependencies and sub-components have been processed,
857@c but before the component in question has been compiled.
858
859@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
860@comment  node-name,  next,  previous,  up
861@section The defsystem grammar
862
863@c FIXME: @var typesetting not consistently used here.  We should either expand
864@c its use to everywhere, or we should kill it everywhere.
865
866
867@example
868system-definition := ( defsystem system-designator @var{system-option}* )
869
870system-option := :defsystem-depends-on system-list
871                 | module-option
872                 | option
873
874module-option := :components component-list
875                 | :serial [ t | nil ]
876                 | :if-component-dep-fails component-dep-fail-option
877
878option :=
879        | :pathname pathname-specifier
880        | :default-component-class class-name
881        | :perform method-form
882        | :explain method-form
883        | :output-files method-form
884        | :operation-done-p method-form
885        | :depends-on ( @var{dependency-def}* )
886        | :in-order-to ( @var{dependency}+ )
887
888
889system-list := ( @var{simple-component-name}* )
890
891component-list := ( @var{component-def}* )
892
893component-def  := ( component-type simple-component-name @var{option}* )
894
895component-type := :system | :module | :file | :static-file | other-component-type
896
897other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
898
899dependency-def := simple-component-name
900               | ( :feature name )
901               | ( :version simple-component-name version-specifier)
902
903dependency := (dependent-op @var{requirement}+)
904requirement := (required-op @var{required-component}+)
905             | (feature feature-name)
906dependent-op := operation-name
907required-op := operation-name | feature
908
909simple-component-name := string
910                      |  symbol
911
912pathname-specifier := pathname | string | symbol
913
914method-form := (operation-name qual lambda-list @&rest body)
915qual := method qualifier
916
917component-dep-fail-option := :fail | :try-next | :ignore
918@end example
919
920
921@subsection Component names
922
923Component names (@code{simple-component-name})
924may be either strings or symbols.
925
926@subsection Component types
927
928Component type names, even if expressed as keywords, will be looked up
929by name in the current package and in the asdf package, if not found in
930the current package.  So a component type @code{my-component-type}, in
931the current package @code{my-system-asd} can be specified as
932@code{:my-component-type}, or @code{my-component-type}.
933
934@subsection Defsystem depends on
935
936The @code{:defsystem-depends-on} option to @code{defsystem} allows the
937programmer to specify another ASDF-defined system or set of systems that
938must be loaded @emph{before} the system definition is processed.
939Typically this is used to load an ASDF extension that is used in the
940system definition.
941
942@subsection Pathname specifiers
943@cindex pathname specifiers
944
945A pathname specifier (@code{pathname-specifier})
946may be a pathname, a string or a symbol.
947When no pathname specifier is given for a component,
948which is the usual case, the component name itself is used.
949
950If a string is given, which is the usual case,
951the string will be interpreted as a Unix-style pathname
952where @code{/} characters will be interpreted as directory separators.
953Usually, Unix-style relative pathnames are used
954(i.e. not starting with @code{/}, as opposed to absolute pathnames);
955they are relative to the path of the parent component.
956Finally, depending on the @code{component-type},
957the pathname may be interpreted as either a file or a directory,
958and if it's a file,
959a file type may be added corresponding to the @code{component-type},
960or else it will be extracted from the string itself (if applicable).
961
962For instance, the @code{component-type} @code{:module}
963wants a directory pathname, and so a string @code{"foo/bar"}
964will be interpreted as the pathname @file{#p"foo/bar/"}.
965On the other hand, the @code{component-type} @code{:file}
966wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
967will be interpreted as the pathname @file{#p"foo/bar.lisp"},
968and a string @code{"foo/bar.quux"}
969will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
970Finally, the @code{component-type} @code{:static-file}
971wants a file without specifying a type, and so a string @code{"foo/bar"}
972will be interpreted as the pathname @file{#p"foo/bar"},
973and a string @code{"foo/bar.quux"}
974will be interpreted as the pathname @file{#p"foo/bar.quux"}.
975
976ASDF does not interpret the string @code{".."} to designate the parent
977directory.  This string will be passed through to the underlying
978operating system for interpretation.  We @emph{believe} that this will
979work on all platforms where ASDF is deployed, but do not guarantee this
980behavior.  A pathname object with a relative directory component of
981@code{:up} or @code{:back} is the only guaranteed way to specify a
982parent directory.
983
984If a symbol is given, it will be translated into a string,
985and downcased in the process.
986The downcasing of symbols is unconventional,
987but was selected after some consideration.
988Observations suggest that the type of systems we want to support
989either have lowercase as customary case (Unix, Mac, windows)
990or silently convert lowercase to uppercase (lpns),
991so this makes more sense than attempting to use @code{:case :common}
992as argument to @code{make-pathname},
993which is reported not to work on some implementations.
994
995Pathname objects may be given to override the path for a component.
996Such objects are typically specified using reader macros such as @code{#p}
997or @code{#.(make-pathname ...)}.
998Note however, that @code{#p...} is a shorthand for @code{#.(parse-namestring ...)}
999and that the behavior of @code{parse-namestring} is completely non-portable,
1000unless you are using Common Lisp @code{logical-pathname}s
1001(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1002Pathnames made with @code{#.(make-pathname ...)}
1003can usually be done more easily with the string syntax above.
1004The only case that you really need a pathname object is to override
1005the component-type default file type for a given component.
1006Therefore, pathname objects should only rarely be used.
1007Unhappily, ASDF 1 didn't properly support
1008parsing component names as strings specifying paths with directories,
1009and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1010
1011Note that when specifying pathname objects,
1012ASDF does not do any special interpretation of the pathname
1013influenced by the component type, unlike the procedure for
1014pathname-specifying strings.
1015On the one hand, you have to be careful to provide a pathname that correctly
1016fulfills whatever constraints are required from that component type
1017(e.g. naming a directory or a file with appropriate type);
1018on the other hand, you can circumvent the file type that would otherwise
1019be forced upon you if you were specifying a string.
1020
1021@subsection Version specifiers
1022@cindex version specifiers
1023@cindex :version
1024
1025Version specifiers are parsed as period-separated lists of integers.  I.e., in the example,
1026@code{0.2.1} is to be interpreted, roughly speaking, as @code{(0 2 1)}.
1027In particular, version @code{0.2.1} is interpreted the same as
1028@code{0.0002.1} and is strictly version-less-than version @code{0.20.1},
1029even though the two are the same when interpreted as decimal fractions.
1030
1031System definers are encouraged to use version identifiers of the form
1032@var{x}.@var{y}.@var{z} for major version, minor version (compatible
1033API) and patch level.
1034
1035@xref{Common attributes of components}.
1036
1037
1038@subsection Using logical pathnames
1039@cindex logical pathnames
1040
1041We do not generally recommend the use of logical pathnames,
1042especially not so to newcomers to Common Lisp.
1043However, we do support the use of logical pathnames by old timers,
1044when such is their preference.
1045
1046To use logical pathnames,
1047you will have to provide a pathname object as a @code{:pathname} specifier
1048to components that use it, using such syntax as
1049@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1050
1051You only have to specify such logical pathname
1052for your system or some top-level component.
1053Sub-components' relative pathnames,
1054specified using the string syntax for names,
1055will be properly merged with the pathnames of their parents.
1056The specification of a logical pathname host however is @emph{not}
1057otherwise directly supported in the ASDF syntax
1058for pathname specifiers as strings.
1059
1060The @code{asdf-output-translation} layer will
1061avoid trying to resolve and translate logical pathnames.
1062The advantage of this is that
1063you can define yourself what translations you want to use
1064with the logical pathname facility.
1065The disadvantage is that if you do not define such translations,
1066any system that uses logical pathnames will behave differently under
1067asdf-output-translations than other systems you use.
1068
1069If you wish to use logical pathnames you will have to configure the
1070translations yourself before they may be used.
1071ASDF currently provides no specific support
1072for defining logical pathname translations.
1073
1074Note that the reasons we do not recommend logical pathnames are that
1075(1) there is no portable way to set up logical pathnames before they are used,
1076(2) logical pathnames are limited to only portably use
1077a single character case, digits and hyphens.
1078While you can solve the first issue on your own,
1079describing how to do it on each of fifteen implementations supported by ASDF
1080is more than we can document.
1081As for the second issue, mind that the limitation is notably enforced on SBCL,
1082and that you therefore can't portably violate the limitations
1083but must instead define some encoding of your own and add individual mappings
1084to name physical pathnames that do not fit the restrictions.
1085This can notably be a problem when your Lisp files are part of a larger project
1086in which it is common to name files or directories in a way that
1087includes the version numbers of supported protocols,
1088or in which files are shared with software written
1089in different programming languages where conventions include the use of
1090underscores, dots or CamelCase in pathnames.
1091
1092
1093@subsection Serial dependencies
1094@cindex serial dependencies
1095
1096If the @code{:serial t} option is specified for a module,
1097ASDF will add dependencies for each child component,
1098on all the children textually preceding it.
1099This is done as if by @code{:depends-on}.
1100
1101@lisp
1102:serial t
1103:components ((:file "a") (:file "b") (:file "c"))
1104@end lisp
1105
1106is equivalent to
1107
1108@lisp
1109:components ((:file "a")
1110             (:file "b" :depends-on ("a"))
1111             (:file "c" :depends-on ("a" "b")))
1112@end lisp
1113
1114
1115@subsection Source location
1116
1117The @code{:pathname} option is optional in all cases for systems
1118defined via @code{defsystem},
1119and in the usual case the user is recommended not to supply it.
1120
1121Instead, ASDF follows a hairy set of rules that are designed so that
1122@enumerate
1123@item
1124@code{find-system}
1125will load a system from disk
1126and have its pathname default to the right place.
1127@item
1128This pathname information will not be overwritten with
1129@code{*default-pathname-defaults*}
1130(which could be somewhere else altogether)
1131if the user loads up the @file{.asd} file into his editor
1132and interactively re-evaluates that form.
1133@end enumerate
1134
1135If a system is being loaded for the first time,
1136its top-level pathname will be set to:
1137
1138@itemize
1139@item
1140The host/device/directory parts of @code{*load-truename*},
1141if it is bound.
1142@item
1143@code{*default-pathname-defaults*}, otherwise.
1144@end itemize
1145
1146If a system is being redefined, the top-level pathname will be
1147
1148@itemize
1149@item
1150changed, if explicitly supplied or obtained from @code{*load-truename*}
1151(so that an updated source location is reflected in the system definition)
1152@item
1153changed if it had previously been set from @code{*default-pathname-defaults*}
1154@item
1155left as before, if it had previously been set from @code{*load-truename*}
1156and @code{*load-truename*} is currently unbound
1157(so that a developer can evaluate a @code{defsystem} form
1158from within an editor without clobbering its source location)
1159@end itemize
1160
1161@subsection if-component-dep-fails option
1162
1163This option is only appropriate for module components (including
1164systems), not individual source files.
1165
1166For more information about this option, @pxref{Pre-defined subclasses of component}.
1167
1168@node Other code in .asd files,  , The defsystem grammar, Defining systems with defsystem
1169@section Other code in .asd files
1170
1171Files containing @code{defsystem} forms
1172are regular Lisp files that are executed by @code{load}.
1173Consequently, you can put whatever Lisp code you like into these files
1174(e.g., code that examines the compile-time environment
1175and adds appropriate features to @code{*features*}).
1176However, some conventions should be followed,
1177so that users can control certain details of execution
1178of the Lisp in @file{.asd} files:
1179
1180@itemize
1181@item
1182Any informative output
1183(other than warnings and errors,
1184which are the condition system's to dispose of)
1185should be sent to the standard CL stream @code{*standard-output*},
1186so that users can easily control the disposition
1187of output from ASDF operations.
1188@end itemize
1189
1190
1191@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1192@comment  node-name,  next,  previous,  up
1193@chapter The object model of ASDF
1194
1195ASDF is designed in an object-oriented way from the ground up.
1196Both a system's structure and the operations that can be performed on systems
1197follow a protocol.
1198ASDF is extensible to new operations and to new component types.
1199This allows the addition of behaviours:
1200for example, a new component could be added for Java JAR archives,
1201and methods specialised on @code{compile-op} added for it
1202that would accomplish the relevant actions.
1203
1204This chapter deals with @emph{components}, the building blocks of a system,
1205and @emph{operations}, the actions that can be performed on a system.
1206
1207
1208
1209@menu
1210* Operations::
1211* Components::
1212@end menu
1213
1214@node  Operations, Components, The object model of ASDF, The object model of ASDF
1215@comment  node-name,  next,  previous,  up
1216@section Operations
1217@cindex operation
1218
1219An @dfn{operation} object of the appropriate type is instantiated
1220whenever the user wants to do something with a system like
1221
1222@itemize
1223@item compile all its files
1224@item load the files into a running lisp environment
1225@item copy its source files somewhere else
1226@end itemize
1227
1228Operations can be invoked directly, or examined
1229to see what their effects would be without performing them.
1230@emph{FIXME: document how!}
1231There are a bunch of methods specialised on operation and component type
1232that actually do the grunt work.
1233
1234The operation object contains whatever state is relevant for this purpose
1235(perhaps a list of visited nodes, for example)
1236but primarily is a nice thing to specialise operation methods on
1237and easier than having them all be @code{EQL} methods.
1238
1239Operations are invoked on systems via @code{operate}.
1240@anchor{operate}
1241@deffn {Generic function} @code{operate} @var{operation} @var{system} @&rest @var{initargs}
1242@deffnx {Generic function} @code{oos} @var{operation} @var{system} @&rest @var{initargs}
1243@code{operate} invokes @var{operation} on @var{system}.
1244@code{oos} is a synonym for @code{operate}.
1245
1246@var{operation} is a symbol that is passed, along with the supplied
1247@var{initargs}, to @code{make-instance} to create the operation object.
1248@var{system} is a system designator.
1249
1250The @var{initargs} are passed to the @code{make-instance} call
1251when creating the operation object.
1252Note that dependencies may cause the operation
1253to invoke other operations on the system or its components:
1254the new operations will be created
1255with the same @var{initargs} as the original one.
1256
1257@end deffn
1258
1259@menu
1260* Predefined operations of ASDF::
1261* Creating new operations::
1262@end menu
1263
1264@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1265@comment  node-name,  next,  previous,  up
1266@subsection Predefined operations of ASDF
1267
1268All the operations described in this section are in the @code{asdf} package.
1269They are invoked via the @code{operate} generic function.
1270
1271@lisp
1272(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1273@end lisp
1274
1275@deffn Operation @code{compile-op} @&key @code{proclamations}
1276
1277This operation compiles the specified component.
1278If proclamations are supplied, they will be proclaimed.
1279This is a good place to specify optimization settings.
1280
1281When creating a new component type,
1282you should provide methods for @code{compile-op}.
1283
1284When @code{compile-op} is invoked,
1285component dependencies often cause some parts of the system
1286to be loaded as well as compiled.
1287Invoking @code{compile-op}
1288does not necessarily load all the parts of the system, though;
1289use @code{load-op} to load a system.
1290@end deffn
1291
1292@deffn Operation @code{load-op} @&key @code{proclamations}
1293
1294This operation loads a system.
1295
1296The default methods for @code{load-op} compile files before loading them.
1297For parity, your own methods on new component types should probably do so too.
1298@end deffn
1299
1300@deffn Operation @code{load-source-op}
1301
1302This operation will load the source for the files in a module
1303even if the source files have been compiled.
1304Systems sometimes have knotty dependencies
1305which require that sources are loaded
1306before they can be compiled.
1307This is how you do that.
1308
1309If you are creating a component type,
1310you need to implement this operation --- at least, where meaningful.
1311@end deffn
1312
1313@anchor{test-op}
1314@deffn Operation @code{test-op}
1315
1316This operation will perform some tests on the module.
1317The default method will do nothing.
1318The default dependency is to require
1319@code{load-op} to be performed on the module first.
1320The default @code{operation-done-p} is that the operation is @emph{never} done
1321---
1322we assume that if you invoke the @code{test-op},
1323you want to test the system, even if you have already done so.
1324
1325The results of this operation are not defined by ASDF.
1326It has proven difficult to define how the test operation
1327should signal its results to the user
1328in a way that is compatible with all of the various test libraries
1329and test techniques in use in the community.
1330@end deffn
1331
1332@c @deffn Operation test-system-version @&key minimum
1333
1334@c Asks the system whether it satisfies a version requirement.
1335
1336@c The default method accepts a string, which is expected to contain of a
1337@c number of integers separated by #\. characters.  The method is not
1338@c recursive.  The component satisfies the version dependency if it has
1339@c the same major number as required and each of its sub-versions is
1340@c greater than or equal to the sub-version number required.
1341
1342@c @lisp
1343@c (defun version-satisfies (x y)
1344@c   (labels ((bigger (x y)
1345@c           (cond ((not y) t)
1346@c                 ((not x) nil)
1347@c                 ((> (car x) (car y)) t)
1348@c                 ((= (car x) (car y))
1349@c                  (bigger (cdr x) (cdr y))))))
1350@c     (and (= (car x) (car y))
1351@c       (or (not (cdr y)) (bigger (cdr x) (cdr y))))))
1352@c @end lisp
1353
1354@c If that doesn't work for your system, you can override it.  I hope
1355@c you have as much fun writing the new method as @verb{|#lisp|} did
1356@c reimplementing this one.
1357@c @end deffn
1358
1359@c @deffn Operation feature-dependent-op
1360
1361@c An instance of @code{feature-dependent-op} will ignore any components
1362@c which have a @code{features} attribute, unless the feature combination
1363@c it designates is satisfied by @code{*features*}.  This operation is
1364@c not intended to be instantiated directly, but other operations may
1365@c inherit from it.
1366
1367@c @end deffn
1368
1369@node  Creating new operations,  , Predefined operations of ASDF, Operations
1370@comment  node-name,  next,  previous,  up
1371@subsection Creating new operations
1372
1373ASDF was designed to be extensible in an object-oriented fashion.
1374To teach ASDF new tricks, a programmer can implement the behaviour he wants
1375by creating a subclass of @code{operation}.
1376
1377ASDF's pre-defined operations are in no way ``privileged'',
1378but it is requested that developers never use the @code{asdf} package
1379for operations they develop themselves.
1380The rationale for this rule is that we don't want to establish a
1381``global asdf operation name registry'',
1382but also want to avoid name clashes.
1383
1384An operation must provide methods for the following generic functions
1385when invoked with an object of type @code{source-file}:
1386@emph{FIXME describe this better}
1387
1388@itemize
1389
1390@item @code{output-files}
1391The @code{output-files} method determines where the method will put its files.
1392It returns two values, a list of pathnames, and a boolean.
1393If the boolean is @code{T} then the pathnames are marked
1394not be translated by enclosing @code{:around} methods.
1395If the boolean is @code{NIL} then enclosing @code{:around} methods
1396may translate these pathnames, e.g. to ensure object files
1397are somehow stored in some implementation-dependent cache.
1398@item @code{perform}
1399The @code{perform} method must call @code{output-files}
1400to find out where to put its files,
1401because the user is allowed to override.
1402@item @code{output-files}
1403for local policy @code{explain}
1404@item @code{operation-done-p},
1405if you don't like the default one
1406
1407@end itemize
1408
1409Operations that print output should send that output to the standard
1410CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
1411
1412@node Components,  , Operations, The object model of ASDF
1413@comment  node-name,  next,  previous,  up
1414@section Components
1415@cindex component
1416@cindex system
1417@cindex system designator
1418@vindex *system-definition-search-functions*
1419
1420A @dfn{component} represents a source file or
1421(recursively) a collection of components.
1422A @dfn{system} is (roughly speaking) a top-level component
1423that can be found via @code{find-system}.
1424
1425A @dfn{system designator} is a string or symbol
1426and behaves just like any other component name
1427(including with regard to the case conversion rules for component names).
1428
1429
1430@defun find-system system-designator &optional (error-p t)
1431
1432Given a system designator, @code{find-system} finds and returns a system.
1433If no system is found, an error of type
1434@code{missing-component} is thrown,
1435or @code{nil} is returned if @code{error-p} is false.
1436
1437To find and update systems, @code{find-system} funcalls each element
1438in the @code{*system-definition-search-functions*} list,
1439expecting a pathname to be returned, or a system object,
1440from which a pathname may be extracted, and that will be registered.
1441The resulting pathname (if any) is loaded
1442if one of the following conditions is true:
1443
1444@itemize
1445@item
1446there is no system of that name in memory
1447@item
1448the pathname is different from that which was previously loaded
1449@item
1450the file's @code{last-modified} time exceeds the @code{last-modified} time
1451of the system in memory
1452@end itemize
1453
1454When system definitions are loaded from @file{.asd} files,
1455a new scratch package is created for them to load into,
1456so that different systems do not overwrite each others operations.
1457The user may also wish to (and is recommended to)
1458include @code{defpackage} and @code{in-package} forms
1459in his system definition files, however,
1460so that they can be loaded manually if need be.
1461
1462The default value of @code{*system-definition-search-functions*}
1463is a list of two functions.
1464The first function looks in each of the directories given
1465by evaluating members of @code{*central-registry*}
1466for a file whose name is the name of the system and whose type is @file{asd}.
1467The first such file is returned,
1468whether or not it turns out to actually define the appropriate system.
1469The second function does something similar,
1470for the directories specified in the @code{source-registry}.
1471Hence, it is strongly advised to define a system
1472@var{foo} in the corresponding file @var{foo.asd}.
1473@end defun
1474
1475
1476@menu
1477* Common attributes of components::
1478* Pre-defined subclasses of component::
1479* Creating new component types::
1480@end menu
1481
1482@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
1483@comment  node-name,  next,  previous,  up
1484@subsection Common attributes of components
1485
1486All components, regardless of type, have the following attributes.
1487All attributes except @code{name} are optional.
1488
1489@subsubsection Name
1490
1491A component name is a string or a symbol.
1492If a symbol, its name is taken and lowercased.
1493
1494Unless overridden by a @code{:pathname} attribute,
1495the name will be interpreted as a pathname specifier according
1496to a Unix-style syntax.
1497@xref{The defsystem grammar,,Pathname specifiers}.
1498
1499@subsubsection Version identifier
1500@findex version-satisfies
1501@cindex :version
1502
1503This optional attribute is used by the generic function
1504@code{version-satisfies}, which tests to see if @code{:version}
1505dependencies are satisfied.
1506the version should be a string of integers separated by dots,
1507for example @samp{1.0.11}.
1508For more information on the semantics of version specifiers, see @ref{The defsystem grammar}.
1509
1510@c This optional attribute is intended to be used by the @code{test-system-version} operation.
1511@c @xref{Predefined operations of ASDF}.
1512@c @emph{Nota Bene}:
1513@c This operation, planned for ASDF 1,
1514@c is still not implemented yet as of ASDF 2.
1515@c Don't hold your breath.
1516
1517
1518
1519@subsubsection Required features
1520
1521@emph{FIXME: This subsection seems to contradict the
1522@code{defsystem} grammar subsection,
1523which doesn't provide any obvious way to specify required features.
1524Furthermore, in 2009, discussions on the
1525@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1526suggested that the specification of required features may be broken,
1527and that no one may have been using them for a while.
1528Please contact the
1529@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
1530if you are interested in getting this features feature fixed.}
1531
1532Traditionally defsystem users have used reader conditionals
1533to include or exclude specific per-implementation files.
1534This means that any single implementation cannot read the entire system,
1535which becomes a problem if it doesn't wish to compile it,
1536but instead for example to create an archive file containing all the sources,
1537as it will omit to process the system-dependent sources for other systems.
1538
1539Each component in an asdf system may therefore specify features using
1540the same syntax as @code{#+} does, and it will (somehow) be ignored for
1541certain operations unless the feature conditional is a member of
1542@code{*features*}.
1543
1544
1545@subsubsection Dependencies
1546
1547This attribute specifies dependencies of the component on its siblings.
1548It is optional but often necessary.
1549
1550There is an excitingly complicated relationship between the initarg
1551and the method that you use to ask about dependencies
1552
1553Dependencies are between (operation component) pairs.
1554In your initargs for the component, you can say
1555
1556@lisp
1557:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
1558              (load-op (load-op "foo")))
1559@end lisp
1560
1561This means the following things:
1562@itemize
1563@item
1564before performing compile-op on this component, we must perform
1565load-op on @var{a} and @var{b}, and compile-op on @var{c},
1566@item
1567before performing @code{load-op}, we have to load @var{foo}
1568@end itemize
1569
1570The syntax is approximately
1571
1572@verbatim
1573(this-op {(other-op required-components)}+)
1574
1575required-components := component-name
1576                     | (required-components required-components)
1577
1578component-name := string
1579                | (:version string minimum-version-object)
1580@end verbatim
1581
1582Side note:
1583
1584This is on a par with what ACL defsystem does.
1585mk-defsystem is less general: it has an implied dependency
1586
1587@verbatim
1588  for all x, (load x) depends on (compile x)
1589@end verbatim
1590
1591and using a @code{:depends-on} argument to say that @var{b} depends on
1592@var{a} @emph{actually} means that
1593
1594@verbatim
1595  (compile b) depends on (load a)
1596@end verbatim
1597
1598This is insufficient for e.g. the McCLIM system, which requires that
1599all the files are loaded before any of them can be compiled ]
1600
1601End side note
1602
1603In ASDF, the dependency information for a given component and operation
1604can be queried using @code{(component-depends-on operation component)},
1605which returns a list
1606
1607@lisp
1608((load-op "a") (load-op "b") (compile-op "c") ...)
1609@end lisp
1610
1611@code{component-depends-on} can be subclassed for more specific
1612component/operation types: these need to @code{(call-next-method)}
1613and append the answer to their dependency, unless
1614they have a good reason for completely overriding the default dependencies.
1615
1616If it weren't for CLISP, we'd be using @code{LIST} method
1617combination to do this transparently.
1618But, we need to support CLISP.
1619If you have the time for some CLISP hacking,
1620I'm sure they'd welcome your fixes.
1621@c Doesn't CLISP now support LIST method combination?
1622
1623See the discussion of the semantics of @code{:version} in the defsystem
1624grammar.
1625
1626@c FIXME: Should have cross-reference to "Version specifiers" in the
1627@c defsystem grammar, but the cross-referencing is so broken by
1628@c insufficient node breakdown that I have not put one in.
1629
1630
1631@subsubsection pathname
1632
1633This attribute is optional and if absent (which is the usual case),
1634the component name will be used.
1635
1636@xref{The defsystem grammar,,Pathname specifiers},
1637for an explanation of how this attribute is interpreted.
1638
1639Note that the @code{defsystem} macro (used to create a ``top-level'' system)
1640does additional processing to set the filesystem location of
1641the top component in that system.
1642This is detailed elsewhere. @xref{Defining systems with defsystem}.
1643
1644
1645@subsubsection properties
1646
1647This attribute is optional.
1648
1649Packaging systems often require information about files or systems
1650in addition to that specified by ASDF's pre-defined component attributes.
1651Programs that create vendor packages out of ASDF systems therefore
1652have to create ``placeholder'' information to satisfy these systems.
1653Sometimes the creator of an ASDF system may know the additional
1654information and wish to provide it directly.
1655
1656@code{(component-property component property-name)} and
1657associated @code{setf} method will allow
1658the programmatic update of this information.
1659Property names are compared as if by @code{EQL},
1660so use symbols or keywords or something.
1661
1662@menu
1663* Pre-defined subclasses of component::
1664* Creating new component types::
1665@end menu
1666
1667@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
1668@comment  node-name,  next,  previous,  up
1669@subsection Pre-defined subclasses of component
1670
1671@deffn Component source-file
1672
1673A source file is any file that the system does not know how to
1674generate from other components of the system.
1675
1676Note that this is not necessarily the same thing as
1677``a file containing data that is typically fed to a compiler''.
1678If a file is generated by some pre-processor stage
1679(e.g. a @file{.h} file from @file{.h.in} by autoconf)
1680then it is not, by this definition, a source file.
1681Conversely, we might have a graphic file
1682that cannot be automatically regenerated,
1683or a proprietary shared library that we received as a binary:
1684these do count as source files for our purposes.
1685
1686Subclasses of source-file exist for various languages.
1687@emph{FIXME: describe these.}
1688@end deffn
1689
1690@deffn Component module
1691
1692A module is a collection of sub-components.
1693
1694A module component has the following extra initargs:
1695
1696@itemize
1697@item
1698@code{:components} the components contained in this module
1699
1700@item
1701@code{:default-component-class}
1702All children components which don't specify their class explicitly
1703are inferred to be of this type.
1704
1705@item
1706@code{:if-component-dep-fails}
1707This attribute takes one of the values
1708@code{:fail}, @code{:try-next}, @code{:ignore},
1709its default value is @code{:fail}.
1710The other values can be used for implementing conditional compilation
1711based on implementation @code{*features*},
1712for the case where it is not necessary for all files in a module to be
1713compiled.
1714@emph{FIXME: such conditional compilation has been reported
1715to be broken in 2009.}
1716
1717@item
1718@code{:serial} When this attribute is set,
1719each subcomponent of this component is assumed to depend on all subcomponents
1720before it in the list given to @code{:components}, i.e.
1721all of them are loaded before a compile or load operation is performed on it.
1722
1723@end itemize
1724
1725The default operation knows how to traverse a module, so
1726most operations will not need to provide methods specialised on modules.
1727
1728@code{module} may be subclassed to represent components such as
1729foreign-language linked libraries or archive files.
1730@end deffn
1731
1732@deffn Component system
1733
1734@code{system} is a subclass of @code{module}.
1735
1736A system is a module with a few extra attributes for documentation
1737purposes; these are given elsewhere.
1738@xref{The defsystem grammar}.
1739
1740Users can create new classes for their systems:
1741the default @code{defsystem} macro takes a @code{:class} keyword argument.
1742@end deffn
1743
1744@node  Creating new component types,  , Pre-defined subclasses of component, Components
1745@comment  node-name,  next,  previous,  up
1746@subsection Creating new component types
1747
1748New component types are defined by subclassing one of the existing
1749component classes and specializing methods on the new component class.
1750
1751@emph{FIXME: this should perhaps be explained more throughly,
1752not only by example ...}
1753
1754As an example, suppose we have some implementation-dependent
1755functionality that we want to isolate
1756in one subdirectory per Lisp implementation our system supports.
1757We create a subclass of
1758@code{cl-source-file}:
1759
1760@lisp
1761(defclass unportable-cl-source-file (cl-source-file)
1762  ())
1763@end lisp
1764
1765Function @code{asdf:implementation-type} (exported since 2.014.14)
1766gives us the name of the subdirectory.
1767All that's left is to define how to calculate the pathname
1768of an @code{unportable-cl-source-file}.
1769
1770@lisp
1771(defmethod component-pathname ((component unportable-cl-source-file))
1772  (merge-pathnames*
1773   (coerce-pathname (format nil "~(~A~)/" (asdf:implementation-type)))
1774   (call-next-method)))
1775@end lisp
1776
1777The new component type is used in a @code{defsystem} form in this way:
1778
1779@lisp
1780(defsystem :foo
1781    :components
1782    ((:file "packages")
1783     ...
1784     (:unportable-cl-source-file "threads"
1785      :depends-on ("packages" ...))
1786     ...
1787    )
1788@end lisp
1789
1790@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
1791@comment  node-name,  next,  previous,  up
1792@chapter Controlling where ASDF searches for systems
1793
1794@section Configurations
1795
1796Configurations specify paths where to find system files.
1797
1798@enumerate
1799
1800@item
1801The search registry may use some hardcoded wrapping registry specification.
1802This allows some implementations (notably SBCL) to specify where to find
1803some special implementation-provided systems that
1804need to precisely match the version of the implementation itself.
1805
1806@item
1807An application may explicitly initialize the source-registry configuration
1808using the configuration API
1809(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
1810in which case this takes precedence.
1811It may itself compute this configuration from the command-line,
1812from a script, from its own configuration file, etc.
1813
1814@item
1815The source registry will be configured from
1816the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
1817
1818@item
1819The source registry will be configured from
1820user configuration file
1821@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
1822(which defaults to
1823@file{~/.config/common-lisp/source-registry.conf})
1824if it exists.
1825
1826@item
1827The source registry will be configured from
1828user configuration directory
1829@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
1830(which defaults to
1831@file{~/.config/common-lisp/source-registry.conf.d/})
1832if it exists.
1833
1834@item
1835The source registry will be configured from
1836system configuration file
1837@file{/etc/common-lisp/source-registry.conf}
1838if it exists/
1839
1840@item
1841The source registry will be configured from
1842system configuration directory
1843@file{/etc/common-lisp/source-registry.conf.d/}
1844if it exists.
1845
1846@item
1847The source registry will be configured from a default configuration.
1848This configuration may allow for implementation-specific systems
1849to be found, for systems to be found the current directory
1850(at the time that the configuration is initialized) as well as
1851@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
1852@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
1853For instance, SBCL will include directories for its contribs
1854when it can find them; it will look for them where SBCL was installed,
1855or at the location specified by the @code{SBCL_HOME} environment variable.
1856
1857@end enumerate
1858
1859Each of these configurations is specified as an s-expression
1860in a trivial domain-specific language (defined below).
1861Additionally, a more shell-friendly syntax is available
1862for the environment variable (defined yet below).
1863
1864Each of these configurations is only used if the previous
1865configuration explicitly or implicitly specifies that it
1866includes its inherited configuration.
1867
1868Additionally, some implementation-specific directories
1869may be automatically prepended to whatever directories are specified
1870in configuration files, no matter if the last one inherits or not.
1871
1872@section XDG base directory
1873
1874Note that we purport to respect the XDG base directory specification
1875as to where configuration files are located,
1876where data files are located,
1877where output file caches are located.
1878Mentions of XDG variables refer to that document.
1879
1880@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
1881
1882This specification allows the user to specify some environment variables
1883to customize how applications behave to his preferences.
1884
1885On Windows platforms, when not using Cygwin,
1886instead of the XDG base directory specification,
1887we try to use folder configuration from the registry regarding
1888@code{Common AppData} and similar directories.
1889However, support for querying the Windows registry is limited as of ASDF 2,
1890and on many implementations, we may fall back to always using the defaults
1891without consulting the registry.
1892Patches welcome.
1893
1894@section Backward Compatibility
1895
1896For backward compatibility as well as to provide a practical backdoor for hackers,
1897ASDF will first search for @code{.asd} files in the directories specified in
1898@code{asdf:*central-registry*}
1899before it searches in the source registry above.
1900
1901@xref{Configuring ASDF,,Configuring ASDF to find your systems -- old style}.
1902
1903By default, @code{asdf:*central-registry*} will be empty.
1904
1905This old mechanism will therefore not affect you if you don't use it,
1906but will take precedence over the new mechanism if you do use it.
1907
1908@section Configuration DSL
1909
1910Here is the grammar of the s-expression (SEXP) DSL for source-registry
1911configuration:
1912
1913@c FIXME: This is too wide for happy compilation into pdf.
1914
1915@example
1916;; A configuration is a single SEXP starting with keyword :source-registry
1917;; followed by a list of directives.
1918CONFIGURATION := (:source-registry DIRECTIVE ...)
1919
1920;; A directive is one of the following:
1921DIRECTIVE :=
1922    ;; INHERITANCE DIRECTIVE:
1923    ;; Your configuration expression MUST contain
1924    ;; exactly one of either of these:
1925    :inherit-configuration | ; splices inherited configuration (often specified last)
1926    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
1927
1928    ;; forward compatibility directive (since ASDF 2.011.4), useful when
1929    ;; you want to use new configuration features but have to bootstrap a
1930    ;; the newer required ASDF from an older release that doesn't sport said features:
1931    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
1932
1933    ;; add a single directory to be scanned (no recursion)
1934    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
1935
1936    ;; add a directory hierarchy, recursing but excluding specified patterns
1937    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
1938
1939    ;; override the defaults for exclusion patterns
1940    (:exclude PATTERN ...) |
1941    ;; augment the defaults for exclusion patterns
1942    (:also-exclude PATTERN ...) |
1943    ;; Note that the scope of a an exclude pattern specification is
1944    ;; the rest of the current configuration expression or file.
1945
1946    ;; splice the parsed contents of another config file
1947    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
1948
1949    ;; This directive specifies that some default must be spliced.
1950    :default-registry
1951
1952REGULAR-FILE-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a file
1953DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
1954
1955PATHNAME-DESIGNATOR :=
1956    NULL | ;; Special: skip this entry.
1957    ABSOLUTE-COMPONENT-DESIGNATOR |
1958    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
1959
1960ABSOLUTE-COMPONENT-DESIGNATOR :=
1961    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable)
1962    PATHNAME | ;; pathname (better be an absolute path, or bust)
1963    :HOME | ;; designates the user-homedir-pathname ~/
1964    :USER-CACHE | ;; designates the default location for the user cache
1965    :SYSTEM-CACHE | ;; designates the default location for the system cache
1966    :HERE  ;; designates the location of the configuration file
1967           ;; (or *default-pathname-defaults*, if invoked interactively)
1968
1969RELATIVE-COMPONENT-DESIGNATOR :=
1970    STRING | ;; namestring (directory assumed where applicable)
1971    PATHNAME | ;; pathname
1972    :IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.45-linux-amd64
1973    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
1974    :UID | ;; current UID -- not available on Windows
1975    :USER ;; current USER name -- NOT IMPLEMENTED(!)
1976
1977PATTERN := a string without wildcards, that will be matched exactly
1978  against the name of a any subdirectory in the directory component
1979        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
1980@end example
1981
1982For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
1983which is the default place ASDF looks for this configuration,
1984once contained:
1985@example
1986(:source-registry
1987  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
1988  :inherit-configuration)
1989@end example
1990
1991@section Configuration Directories
1992
1993Configuration directories consist in files each containing
1994a list of directives without any enclosing @code{(:source-registry ...)} form.
1995The files will be sorted by namestring as if by @code{string<} and
1996the lists of directives of these files with be concatenated in order.
1997An implicit @code{:inherit-configuration} will be included
1998at the @emph{end} of the list.
1999
2000This allows for packaging software that has file granularity
2001(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2002to easily include configuration information about distributed software.
2003
2004The convention is that, for sorting purposes,
2005the names of files in such a directory begin with two digits
2006that determine the order in which these entries will be read.
2007Also, the type of these files is conventionally @code{"conf"}
2008and as a limitation to some implementations (e.g. GNU clisp),
2009the type cannot be @code{NIL}.
2010
2011Directories may be included by specifying a directory pathname
2012or namestring in an @code{:include} directive, e.g.:
2013
2014@example
2015  (:include "/foo/bar/")
2016@end example
2017
2018Hence, to achieve the same effect as
2019my example @file{~/.config/common-lisp/source-registry.conf} above,
2020I could simply create a file
2021@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
2022alone in its directory with the following contents:
2023@example
2024(:tree "/home/fare/cl/")
2025@end example
2026
2027@subsection The :here directive
2028
2029The @code{:here} directive is an absolute pathname designator that
2030refers to the directory containing the configuration file currently
2031being processed.
2032
2033The @code{:here} directive is intended to simplify the delivery of
2034complex CL systems, and for easy configuration of projects shared through
2035revision control systems, in accordance with our design principle that
2036each participant should be able to provide all and only the information
2037available to him or her.
2038
2039Consider a person X who has set up the source code repository for a
2040complex project with a master directory @file{dir/}.  Ordinarily, one
2041might simply have the user add a directive that would look something
2042like this:
2043@example
2044   (:tree  "path/to/dir")
2045@end example
2046But what if X knows that there are very large subtrees
2047under dir that are filled with, e.g., Java source code, image files for
2048icons, etc.?  All of the asdf system definitions are contained in the
2049subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2050these are the only directories that should be searched.
2051
2052In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2053contains the following:
2054@example
2055(:source-registry
2056   (:tree (:here "src/lisp/"))
2057   (:tree (:here "extlib/lisp"))
2058   (:directory (:here "outlier/")))
2059@end example
2060
2061Then when someone else (call her Y) checks out a copy of this
2062repository, she need only add
2063@example
2064(:include "/path/to/my/checkout/directory/asdf.conf")
2065@end example
2066to one of her previously-existing asdf source location configuration
2067files, or invoke @code{initialize-source-registry} with a configuration
2068form containing that s-expression.  ASDF will find the .conf file that X
2069has provided, and then set up source locations within the working
2070directory according to X's (relative) instructions.
2071
2072@section Shell-friendly syntax for configuration
2073
2074When considering environment variable @code{CL_SOURCE_REGISTRY}
2075ASDF will skip to next configuration if it's an empty string.
2076It will @code{READ} the string as a SEXP in the DSL
2077if it begins with a paren @code{(}
2078and it will be interpreted much like @code{TEXINPUTS}
2079list of paths, where
2080
2081  * paths are separated
2082   by a @code{:} (colon) on Unix platforms (including cygwin),
2083   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2084
2085  * each entry is a directory to add to the search path.
2086
2087  * if the entry ends with a double slash @code{//}
2088    then it instead indicates a tree in the subdirectories
2089    of which to recurse.
2090
2091  * if the entry is the empty string (which may only appear once),
2092    then it indicates that the inherited configuration should be
2093    spliced there.
2094
2095
2096@section Search Algorithm
2097@vindex *default-source-registry-exclusions*
2098
2099In case that isn't clear, the semantics of the configuration is that
2100when searching for a system of a given name,
2101directives are processed in order.
2102
2103When looking in a directory, if the system is found, the search succeeds,
2104otherwise it continues.
2105
2106When looking in a tree, if one system is found, the search succeeds.
2107If multiple systems are found, the consequences are unspecified:
2108the search may succeed with any of the found systems,
2109or an error may be raised.
2110ASDF currently returns the first system found,
2111XCVB currently raised an error.
2112If none is found, the search continues.
2113
2114Exclude statements specify patterns of subdirectories
2115the systems from which to ignore.
2116Typically you don't want to use copies of files kept by such
2117version control systems as Darcs.
2118Exclude statements are not propagated to further included or inherited
2119configuration files or expressions;
2120instead the defaults are reset around every configuration statement
2121to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2122
2123Include statements cause the search to recurse with the path specifications
2124from the file specified.
2125
2126An inherit-configuration statement cause the search to recurse with the path
2127specifications from the next configuration
2128(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2129
2130
2131@section Caching Results
2132
2133The implementation is allowed to either eagerly compute the information
2134from the configurations and file system, or to lazily re-compute it
2135every time, or to cache any part of it as it goes.
2136To explicitly flush any information cached by the system, use the API below.
2137
2138
2139@section Configuration API
2140
2141The specified functions are exported from your build system's package.
2142Thus for ASDF the corresponding functions are in package ASDF,
2143and for XCVB the corresponding functions are in package XCVB.
2144
2145@defun initialize-source-registry @&optional PARAMETER
2146   will read the configuration and initialize all internal variables.
2147   You may extend or override configuration
2148   from the environment and configuration files
2149   with the given @var{PARAMETER}, which can be
2150   @code{NIL} (no configuration override),
2151   or a SEXP (in the SEXP DSL),
2152   a string (as in the string DSL),
2153   a pathname (of a file or directory with configuration),
2154   or a symbol (fbound to function that when called returns one of the above).
2155@end defun
2156
2157@defun clear-source-registry
2158   undoes any source registry configuration
2159   and clears any cache for the search algorithm.
2160   You might want to call this function
2161   (or better, @code{clear-configuration})
2162   before you dump an image that would be resumed
2163   with a different configuration,
2164   and return an empty configuration.
2165   Note that this does not include clearing information about
2166   systems defined in the current image, only about
2167   where to look for systems not yet defined.
2168@end defun
2169
2170@defun ensure-source-registry @&optional PARAMETER
2171   checks whether a source registry has been initialized.
2172   If not, initialize it with the given @var{PARAMETER}.
2173@end defun
2174
2175Every time you use ASDF's @code{find-system}, or
2176anything that uses it (such as @code{operate}, @code{load-system}, etc.),
2177@code{ensure-source-registry} is called with parameter NIL,
2178which the first time around causes your configuration to be read.
2179If you change a configuration file,
2180you need to explicitly @code{initialize-source-registry} again,
2181or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
2182which will cause the initialization to happen next time around.
2183
2184
2185@section Future
2186
2187If this mechanism is successful, in the future, we may declare
2188@code{asdf:*central-registry*} obsolete and eventually remove it.
2189Any hook into implementation-specific search mechanisms will by then
2190have been integrated in the @code{:default-configuration} which everyone
2191should either explicitly use or implicit inherit. Some shell syntax
2192for it should probably be added somehow.
2193
2194But we're not there yet. For now, let's see how practical this new
2195source-registry is.
2196
2197
2198@section Rejected ideas
2199
2200Alternatives I considered and rejected included:
2201
2202@enumerate
2203@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
2204   and somehow the configuration parser expands the new configuration
2205   language into a expanded series of directories of subdirectories to
2206   lookup, pre-recursing through specified hierarchies. This is kludgy,
2207   and leaves little space of future cleanups and extensions.
2208
2209@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
2210   in completely new ways, so that new kinds of entries may be implemented
2211   as a recursive search, etc. This seems somewhat backwards.
2212
2213@item Completely remove @code{asdf:*central-registry*}
2214   and break backwards compatibility.
2215   Hopefully this will happen in a few years after everyone migrate to
2216   a better ASDF and/or to XCVB, but it would be very bad to do it now.
2217
2218@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
2219   when you dereference it or setf it. Only the new variable with new
2220   semantics is handled by the new search procedure.
2221   Complex and still introduces subtle semantic issues.
2222@end enumerate
2223
2224
2225I've been suggested the below features, but have rejected them,
2226for the sake of keeping ASDF no more complex than strictly necessary.
2227
2228@itemize
2229@item
2230  More syntactic sugar: synonyms for the configuration directives, such as
2231  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
2232  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
2233
2234@item
2235   The possibility to register individual files instead of directories.
2236
2237@item
2238  Integrate Xach Beane's tilde expander into the parser,
2239  or something similar that is shell-friendly or shell-compatible.
2240  I'd rather keep ASDF minimal. But maybe this precisely keeps it
2241  minimal by removing the need for evaluated entries that ASDF has?
2242  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
2243  Hopefully, these are already superseded by the @code{:default-registry}
2244
2245@item
2246  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
2247  down a filesystem tree in the environment variable.
2248  It isn't that Lisp friendly either.
2249@end itemize
2250
2251@section TODO
2252
2253@itemize
2254@item Add examples
2255@end itemize
2256
2257
2258@section Credits for the source-registry
2259
2260Thanks a lot to Stelian Ionescu for the initial idea.
2261
2262Thanks to Rommel Martinez for the initial implementation attempt.
2263
2264All bad design ideas and implementation bugs are to mine, not theirs.
2265But so are good design ideas and elegant implementation tricks.
2266
2267 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
2268
2269
2270
2271@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
2272@comment  node-name,  next,  previous,  up
2273@chapter Controlling where ASDF saves compiled files
2274@cindex asdf-output-translations
2275@vindex ASDF_OUTPUT_TRANSLATIONS
2276
2277Each Common Lisp implementation has its own format
2278for compiled files (fasls for short, short for ``fast loading'').
2279If you use multiple implementations
2280(or multiple versions of the same implementation),
2281you'll soon find your source directories
2282littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
2283Worse yet, some implementations use the same file extension
2284while changing formats from version to version (or platform to platform)
2285which means that you'll have to recompile binaries
2286as you switch from one implementation to the next.
2287
2288ASDF 2 includes the @code{asdf-output-translations} facility
2289to mitigate the problem.
2290
2291@section Configurations
2292
2293Configurations specify mappings from input locations to output locations.
2294Once again we rely on the XDG base directory specification for configuration.
2295@xref{Controlling where ASDF searches for systems,,XDG base directory}.
2296
2297@enumerate
2298
2299@item
2300Some hardcoded wrapping output translations configuration may be used.
2301This allows special output translations (or usually, invariant directories)
2302to be specified corresponding to the similar special entries in the source registry.
2303
2304@item
2305An application may explicitly initialize the output-translations
2306configuration using the Configuration API
2307in which case this takes precedence.
2308(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
2309It may itself compute this configuration from the command-line,
2310from a script, from its own configuration file, etc.
2311
2312@item
2313The source registry will be configured from
2314the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
2315
2316@item
2317The source registry will be configured from
2318user configuration file
2319@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
2320(which defaults to
2321@file{~/.config/common-lisp/asdf-output-translations.conf})
2322if it exists.
2323
2324@item
2325The source registry will be configured from
2326user configuration directory
2327@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
2328(which defaults to
2329@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
2330if it exists.
2331
2332@item
2333The source registry will be configured from
2334system configuration file
2335@file{/etc/common-lisp/asdf-output-translations.conf}
2336if it exists.
2337
2338@item
2339The source registry will be configured from
2340system configuration directory
2341@file{/etc/common-lisp/asdf-output-translations.conf.d/}
2342if it exists.
2343
2344@end enumerate
2345
2346Each of these configurations is specified as a SEXP
2347in a trival domain-specific language (defined below).
2348Additionally, a more shell-friendly syntax is available
2349for the environment variable (defined yet below).
2350
2351Each of these configurations is only used if the previous
2352configuration explicitly or implicitly specifies that it
2353includes its inherited configuration.
2354
2355Note that by default, a per-user cache is used for output files.
2356This allows the seamless use of shared installations of software
2357between several users, and takes files out of the way of the developers
2358when they browse source code,
2359at the expense of taking a small toll when developers have to clean up
2360output files and find they need to get familiar with output-translations first.
2361
2362
2363@section Backward Compatibility
2364@cindex ASDF-BINARY-LOCATIONS compatibility
2365
2366
2367We purposefully do NOT provide backward compatibility with earlier versions of
2368@code{ASDF-Binary-Locations} (8 Sept 2009),
2369@code{common-lisp-controller} (7.0) or
2370@code{cl-launch} (2.35),
2371each of which had similar general capabilities.
2372The previous APIs of these programs were not designed
2373for configuration by the end-user
2374in an easy way with configuration files.
2375Recent versions of same packages use
2376the new @code{asdf-output-translations} API as defined below:
2377@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
2378@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
2379
2380This incompatibility shouldn't inconvenience many people.
2381Indeed, few people use and customize these packages;
2382these few people are experts who can trivially adapt to the new configuration.
2383Most people are not experts, could not properly configure these features
2384(except inasmuch as the default configuration of
2385@code{common-lisp-controller} and/or @code{cl-launch}
2386might have been doing the right thing for some users),
2387and yet will experience software that ``just works'',
2388as configured by the system distributor, or by default.
2389
2390Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
2391we provide a limited emulation mode:
2392
2393@defun enable-asdf-binary-locations-compatibility @&key centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
2394This function will initialize the new @code{asdf-output-translations} facility in a way
2395that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
2396Where you would previously set global variables
2397@var{*centralize-lisp-binaries*},
2398@var{*default-toplevel-directory*},
2399@var{*include-per-user-information*},
2400@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
2401you will now have to pass the same values as keyword arguments to this function.
2402Note however that as an extension the @code{:source-to-target-mappings} keyword argument
2403will accept any valid pathname designator for @code{asdf-output-translations}
2404instead of just strings and pathnames.
2405@end defun
2406
2407If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
2408(the one available as an extension to load of top of ASDF,
2409not the one built into a few old versions of ASDF),
2410but first you must disable @code{asdf-output-translations}
2411with @code{(asdf:disable-output-translations)},
2412or you might experience ``interesting'' issues.
2413
2414Also, note that output translation is enabled by default.
2415To disable it, use @code{(asdf:disable-output-translations)}.
2416
2417
2418@section Configuration DSL
2419
2420Here is the grammar of the SEXP DSL
2421for @code{asdf-output-translations} configuration:
2422
2423@verbatim
2424;; A configuration is single SEXP starting with keyword :source-registry
2425;; followed by a list of directives.
2426CONFIGURATION := (:output-translations DIRECTIVE ...)
2427
2428;; A directive is one of the following:
2429DIRECTIVE :=
2430    ;; INHERITANCE DIRECTIVE:
2431    ;; Your configuration expression MUST contain
2432    ;; exactly one of either of these:
2433    :inherit-configuration | ; splices inherited configuration (often specified last)
2434    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2435
2436    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2437    ;; you want to use new configuration features but have to bootstrap a
2438    ;; the newer required ASDF from an older release that doesn't sport said features:
2439    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2440
2441    ;; include a configuration file or directory
2442    (:include PATHNAME-DESIGNATOR) |
2443
2444    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
2445    :enable-user-cache |
2446    ;; Disable global cache. Map / to /
2447    :disable-cache |
2448
2449    ;; add a single directory to be scanned (no recursion)
2450    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
2451
2452    ;; use a function to return the translation of a directory designator
2453    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
2454
2455DIRECTORY-DESIGNATOR :=
2456    T | ;; as source matches anything, as destination leaves pathname unmapped.
2457    ABSOLUTE-COMPONENT-DESIGNATOR |
2458    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...)
2459
2460ABSOLUTE-COMPONENT-DESIGNATOR :=
2461    NULL | ;; As source: skip this entry. As destination: same as source
2462    :ROOT | ;; magic: paths that are relative to the root of the source host and device
2463    STRING | ;; namestring (directory is assumed, better be absolute or bust, ``/**/*.*'' added)
2464    PATHNAME | ;; pathname (better be an absolute directory or bust)
2465    :HOME | ;; designates the user-homedir-pathname ~/
2466    :USER-CACHE | ;; designates the default location for the user cache
2467    :SYSTEM-CACHE ;; designates the default location for the system cache
2468
2469RELATIVE-COMPONENT-DESIGNATOR :=
2470    STRING | ;; namestring, directory is assumed. If the last component, /**/*.* is added
2471    PATHNAME | ;; pathname; unless last component, directory is assumed.
2472    :IMPLEMENTATION | ;; a directory based on implementation, e.g. sbcl-1.0.45-linux-amd64
2473    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2474    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2475    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2476    :*.*.* | ;; any file (since ASDF 2.011.4)
2477    :UID | ;; current UID -- not available on Windows
2478    :USER ;; current USER name -- NOT IMPLEMENTED(!)
2479
2480TRANSLATION-FUNCTION :=
2481    SYMBOL | ;; symbol of a function that takes two arguments,
2482             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2483    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
2484             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
2485
2486@end verbatim
2487
2488Relative components better be either relative
2489or subdirectories of the path before them, or bust.
2490
2491The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
2492You can specify more fine-grained patterns
2493by using a pathname object as the last component
2494e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
2495
2496You may use @code{#+features} to customize the configuration file.
2497
2498The second designator of a mapping may be @code{NIL}, indicating that files are not mapped
2499to anything but themselves (same as if the second designator was the same as the first).
2500
2501When the first designator is @code{t},
2502the mapping always matches.
2503When the first designator starts with @code{:root},
2504the mapping matches any host and device.
2505In either of these cases, if the second designator
2506isn't @code{t} and doesn't start with @code{:root},
2507then strings indicating the host and pathname are somehow copied
2508in the beginning of the directory component of the source pathname
2509before it is translated.
2510
2511When the second designator is @code{t}, the mapping is the identity.
2512When the second designator starts with @code{:root},
2513the mapping preserves the host and device of the original pathname.
2514Notably, this allows you to map files
2515to a subdirectory of the whichever directory the file is in.
2516Though the syntax is not quite as easy to use as we'd like,
2517you can have an (source destination) mapping entry such as follows
2518in your configuration file,
2519or you may use @code{enable-asdf-binary-locations-compatibility}
2520with @code{:centralize-lisp-binaries nil}
2521which will do the same thing internally for you:
2522@verbatim
2523  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
2524          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
2525     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
2526       (:root ,wild-subdir :implementation ,wild-file)))
2527@end verbatim
2528Starting with ASDF 2.011.4, you can use the simpler:
2529  @code{`(:root (:root :**/ :implementation :*.*.*))}
2530
2531
2532
2533@code{:include} statements cause the search to recurse with the path specifications
2534from the file specified.
2535
2536If the @code{translate-pathname} mechanism cannot achieve a desired
2537translation, the user may provide a function which provides the
2538required algorithim.  Such a translation function is specified by
2539supplying a list as the second @code{directory-designator}
2540the first element of which is the keyword @code{:function},
2541and the second element of which is
2542either a symbol which designates a function or a lambda expression.
2543The function designated by the second argument must take two arguments,
2544the first being the pathname of the source file,
2545the second being the wildcard that was matched.
2546The result of the function invocation should be the translated pathname.
2547
2548An @code{:inherit-configuration} statement cause the search to recurse with the path
2549specifications from the next configuration.
2550@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
2551
2552@itemize
2553@item
2554@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
2555@item
2556@code{:disable-cache} is the same as @code{(t t)}.
2557@item
2558@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
2559which by default is the same as using
2560@code{(:home ".cache" "common-lisp" :implementation)}.
2561@item
2562@code{:system-cache} uses the contents of variable @code{asdf::*system-cache*}
2563which by default is the same as using
2564@code{("/var/cache/common-lisp" :uid :implementation-type)}
2565(on Unix and cygwin), or something semi-sensible on Windows.
2566@end itemize
2567
2568
2569@section Configuration Directories
2570
2571Configuration directories consist in files each contains
2572a list of directives without any enclosing
2573@code{(:output-translations ...)} form.
2574The files will be sorted by namestring as if by @code{string<} and
2575the lists of directives of these files with be concatenated in order.
2576An implicit @code{:inherit-configuration} will be included
2577at the @emph{end} of the list.
2578
2579This allows for packaging software that has file granularity
2580(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
2581to easily include configuration information about software being distributed.
2582
2583The convention is that, for sorting purposes,
2584the names of files in such a directory begin with two digits
2585that determine the order in which these entries will be read.
2586Also, the type of these files is conventionally @code{"conf"}
2587and as a limitation of some implementations, the type cannot be @code{NIL}.
2588
2589Directories may be included by specifying a directory pathname
2590or namestring in an @code{:include} directive, e.g.:
2591@verbatim
2592  (:include "/foo/bar/")
2593@end verbatim
2594
2595@section Shell-friendly syntax for configuration
2596
2597When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
2598ASDF will skip to next configuration if it's an empty string.
2599It will @code{READ} the string as an SEXP in the DSL
2600if it begins with a paren @code{(}
2601and it will be interpreted as a list of directories.
2602Directories should come by pairs, indicating a mapping directive.
2603Entries are separated
2604by a @code{:} (colon) on Unix platforms (including cygwin),
2605by a @code{;} (semicolon) on other platforms (mainly, Windows).
2606
2607The magic empty entry,
2608if it comes in what would otherwise be the first entry in a pair,
2609indicates the splicing of inherited configuration.
2610If it comes as the second entry in a pair,
2611it indicates that the directory specified first is to be left untranslated
2612(which has the same effect as if the directory had been repeated).
2613
2614
2615@section Semantics of Output Translations
2616
2617From the specified configuration,
2618a list of mappings is extracted in a straightforward way:
2619mappings are collected in order, recursing through
2620included or inherited configuration as specified.
2621To this list is prepended some implementation-specific mappings,
2622and is appended a global default.
2623
2624The list is then compiled to a mapping table as follows:
2625for each entry, in order, resolve the first designated directory
2626into an actual directory pathname for source locations.
2627If no mapping was specified yet for that location,
2628resolve the second designated directory to an output location directory
2629add a mapping to the table mapping the source location to the output location,
2630and add another mapping from the output location to itself
2631(unless a mapping already exists for the output location).
2632
2633Based on the table, a mapping function is defined,
2634mapping source pathnames to output pathnames:
2635given a source pathname, locate the longest matching prefix
2636in the source column of the mapping table.
2637Replace that prefix by the corresponding output column
2638in the same row of the table, and return the result.
2639If no match is found, return the source pathname.
2640(A global default mapping the filesystem root to itself
2641may ensure that there will always be a match,
2642with same fall-through semantics).
2643
2644@section Caching Results
2645
2646The implementation is allowed to either eagerly compute the information
2647from the configurations and file system, or to lazily re-compute it
2648every time, or to cache any part of it as it goes.
2649To explicitly flush any information cached by the system, use the API below.
2650
2651
2652@section Output location API
2653
2654The specified functions are exported from package ASDF.
2655
2656@defun initialize-output-translations @&optional PARAMETER
2657   will read the configuration and initialize all internal variables.
2658   You may extend or override configuration
2659   from the environment and configuration files
2660   with the given @var{PARAMETER}, which can be
2661   @code{NIL} (no configuration override),
2662   or a SEXP (in the SEXP DSL),
2663   a string (as in the string DSL),
2664   a pathname (of a file or directory with configuration),
2665   or a symbol (fbound to function that when called returns one of the above).
2666@end defun
2667
2668@defun disable-output-translations
2669   will initialize output translations in a way
2670   that maps every pathname to itself,
2671   effectively disabling the output translation facility.
2672@end defun
2673
2674@defun clear-output-translations
2675   undoes any output translation configuration
2676   and clears any cache for the mapping algorithm.
2677   You might want to call this function
2678   (or better, @code{clear-configuration})
2679   before you dump an image that would be resumed
2680   with a different configuration,
2681   and return an empty configuration.
2682   Note that this does not include clearing information about
2683   systems defined in the current image, only about
2684   where to look for systems not yet defined.
2685@end defun
2686
2687@defun ensure-output-translations @&optional PARAMETER
2688   checks whether output translations have been initialized.
2689   If not, initialize them with the given @var{PARAMETER}.
2690   This function will be called before any attempt to operate on a system.
2691@end defun
2692
2693@defun apply-output-translations PATHNAME
2694   Applies the configured output location translations to @var{PATHNAME}
2695   (calls @code{ensure-output-translations} for the translations).
2696@end defun
2697
2698Every time you use ASDF's @code{output-files}, or
2699anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
2700@code{ensure-output-translations} is called with parameter NIL,
2701which the first time around causes your configuration to be read.
2702If you change a configuration file,
2703you need to explicitly @code{initialize-output-translations} again,
2704or maybe @code{clear-output-translations} (or @code{clear-configuration}),
2705which will cause the initialization to happen next time around.
2706
2707
2708@section Credits for output translations
2709
2710Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
2711and to Peter van Eynde for @code{Common Lisp Controller}.
2712
2713All bad design ideas and implementation bugs are to mine, not theirs.
2714But so are good design ideas and elegant implementation tricks.
2715
2716 --- Francois-Rene Rideau @email{fare@@tunes.org}
2717
2718@c @section Default locations
2719@c @findex output-files-for-system-and-operation
2720
2721@c The default binary location for each Lisp implementation
2722@c is a subdirectory of each source directory.
2723@c To account for different Lisps, Operating Systems, Implementation versions,
2724@c and so on, ASDF borrows code from SLIME
2725@c to create reasonable custom directory names.
2726@c Here are some examples:
2727
2728@c @itemize
2729@c @item
2730@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
2731
2732@c @item
2733@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
2734
2735@c @item
2736@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
2737@c @end itemize
2738
2739@c By default, all output file pathnames will be relocated
2740@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
2741
2742@c See the document @file{README.asdf-output-translations}
2743@c for a full specification on how to configure @code{asdf-output-translations}.
2744
2745@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
2746@comment  node-name,  next,  previous,  up
2747@chapter Error handling
2748@findex SYSTEM-DEFINITION-ERROR
2749@findex OPERATION-ERROR
2750
2751@section ASDF errors
2752
2753If ASDF detects an incorrect system definition, it will signal a generalised instance of
2754@code{SYSTEM-DEFINITION-ERROR}.
2755
2756Operations may go wrong (for example when source files contain errors).
2757These are signalled using generalised instances of
2758@code{OPERATION-ERROR}.
2759
2760@section Compilation error and warning handling
2761@vindex *compile-file-warnings-behaviour*
2762@vindex *compile-file-errors-behavior*
2763
2764ASDF checks for warnings and errors when a file is compiled.
2765The variables @var{*compile-file-warnings-behaviour*} and
2766@var{*compile-file-errors-behavior*}
2767control the handling of any such events.
2768The valid values for these variables are
2769@code{:error}, @code{:warn}, and @code{:ignore}.
2770
2771@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
2772@comment  node-name,  next,  previous,  up
2773@chapter Miscellaneous additional functionality
2774
2775@emph{FIXME:  Add discussion of @code{run-shell-command}?  Others?}
2776
2777ASDF includes several additional features that are generally
2778useful for system definition and development. These include:
2779
2780@defun coerce-pathname name @&key type defaults
2781
2782This function (available starting with ASDF 2.012.11)
2783takes an argument, and portably interprets it as a pathname.
2784If the argument @var{name} is a pathname or @code{nil}, it is passed through;
2785if it's a symbol, it's interpreted as a string by downcasing it;
2786if it's a string, it is first separated using @code{/} into substrings;
2787the leading substrings denote subdirectories of a relative pathname.
2788If @var{type} is @code{:directory} or the string ends with @code{/},
2789the last substring is also a subdirectory;
2790if @var{type} is a string, it is used as the type of the pathname, and
2791the last substring is the name component of the pathname;
2792if @var{type} is @code{nil}, the last substring specifies both name and type components
2793of the pathname, with the last @code{.} separating them, or only the name component
2794if there's no last @code{.} or if there is only one dot and it's the first character.
2795The host, device and version components come from @var{defaults}, which defaults to
2796@var{*default-pathname-defaults*}; but that shouldn't matter if you use @code{merge-pathnames*}.
2797
2798@end defun
2799
2800@defun merge-pathnames* @&key specified defaults
2801
2802This function is a replacement for @code{merge-pathnames} that uses the host and device
2803from the @var{defaults} rather than the @var{specified} pathname when the latter
2804is a relative pathname. This allows ASDF and its users to create and use relative pathnames
2805without having to know beforehand what are the host and device
2806of the absolute pathnames they are relative to.
2807
2808@end defun
2809
2810@defun system-relative-pathname system name @&key type
2811
2812It's often handy to locate a file relative to some system.
2813The @code{system-relative-pathname} function meets this need.
2814
2815It takes two mandatory arguments @var{system} and @var{name}
2816and a keyword argument @var{type}:
2817@var{system} is name of a system, whereas @var{name} and optionally @var{type}
2818specify a relative pathname, interpreted like a component pathname specifier
2819by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
2820
2821It returns a pathname built from the location of the system's
2822source directory and the relative pathname. For example:
2823
2824@lisp
2825> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
2826#P"/repository/other/cl-ppcre/regex.data"
2827@end lisp
2828
2829@end defun
2830
2831@defun system-source-directory system-designator
2832
2833ASDF does not provide a turnkey solution for locating
2834data (or other miscellaneous) files
2835that are distributed together with the source code of a system.
2836Programmers can use @code{system-source-directory} to find such files.
2837Returns a pathname object.
2838The @var{system-designator} may be a string, symbol, or ASDF system object.
2839@end defun
2840
2841@defun clear-system system-designator
2842
2843It is sometimes useful to force recompilation of a previously loaded system.
2844In these cases, it may be useful to @code{(asdf:clear-system :foo)}
2845to remove the system from the table of currently loaded systems;
2846the next time the system @code{foo} or one that depends on it is re-loaded,
2847@code{foo} will then be loaded again.
2848Alternatively, you could touch @code{foo.asd} or
2849remove the corresponding fasls from the output file cache.
2850(It was once conceived that one should provide
2851a list of systems the recompilation of which to force
2852as the @code{:force} keyword argument to @code{load-system};
2853but this has never worked, and though the feature was fixed in ASDF 2.000,
2854it remains @code{cerror}'ed out as nobody ever used it.)
2855
2856Note that this does not and cannot by itself undo the previous loading
2857of the system. Common Lisp has no provision for such an operation,
2858and its reliance on irreversible side-effects to global datastructures
2859makes such a thing impossible in the general case.
2860If the software being re-loaded is not conceived with hot upgrade in mind,
2861this re-loading may cause many errors, warnings or subtle silent problems,
2862as packages, generic function signatures, structures, types, macros, constants, etc.
2863are being redefined incompatibly.
2864It is up to the user to make sure that reloading is possible and has the desired effect.
2865In some cases, extreme measures such as recursively deleting packages,
2866unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
2867and much more are necessary for reloading to happen smoothly.
2868ASDF itself goes through notable pains to make such a hot upgrade possible
2869with respect to its own code, and what it does is ridiculously complex;
2870look at the beginning of @file{asdf.lisp} to see what it does.
2871@end defun
2872
2873@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
2874@comment  node-name,  next,  previous,  up
2875@chapter Getting the latest version
2876
2877Decide which version you want.
2878The @code{master} branch is where development happens;
2879its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
2880but an occasional regression may happen despite our (limited) test suite.
2881
2882The @code{release} branch is what cautious people should be using;
2883it has usually been tested more, and releases are cut at a point
2884where there isn't any known unresolved issue.
2885
2886You may get the ASDF source repository using git:
2887@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
2888
2889You will find the above referenced tags in this repository.
2890You can also browse the repository on
2891@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
2892
2893Discussion of ASDF development is conducted on the
2894mailing list
2895@kbd{asdf-devel@@common-lisp.net}.
2896@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
2897
2898
2899@node FAQ, TODO list, Getting the latest version, Top
2900@comment  node-name,  next,  previous,  up
2901@chapter FAQ
2902
2903@section  ``Where do I report a bug?''
2904
2905ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
2906
2907If you're unsure about whether something is a bug, or for general discussion,
2908use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
2909
2910
2911@section ``What has changed between ASDF 1 and ASDF 2?''
2912
2913@subsection What are ASDF 1 and ASDF 2?
2914
2915On May 31st 2010, we have released ASDF 2.
2916ASDF 2 refers to release 2.000 and later.
2917(Releases between 1.656 and 1.728 were development releases for ASDF 2.)
2918ASDF 1 to any release earlier than 1.369 or so.
2919If your ASDF doesn't sport a version, it's an old ASDF 1.
2920
2921ASDF 2 and its release candidates push
2922@code{:asdf2} onto @code{*features*} so that if you are writing
2923ASDF-dependent code you may check for this feature
2924to see if the new API is present.
2925@emph{All} versions of ASDF should have the @code{:asdf} feature.
2926
2927If you are experiencing problems or limitations of any sort with ASDF 1,
2928we recommend that you should upgrade to ASDF 2,
2929or whatever is the latest release.
2930
2931
2932@subsection ASDF can portably name files in subdirectories
2933
2934Common Lisp namestrings are not portable,
2935except maybe for logical pathnamestrings,
2936that themselves have various limitations and require a lot of setup
2937that is itself ultimately non-portable.
2938
2939In ASDF 1, the only portable ways to refer to pathnames inside systems and components
2940were very awkward, using @code{#.(make-pathname ...)} and
2941@code{#.(merge-pathnames ...)}.
2942Even the above were themselves were inadequate in the general case
2943due to host and device issues, unless horribly complex patterns were used.
2944Plenty of simple cases that looked portable actually weren't,
2945leading to much confusion and greavance.
2946
2947ASDF 2 implements its own portable syntax for strings as pathname specifiers.
2948Naming files within a system definition becomes easy and portable again.
2949@xref{Miscellaneous additional functionality,asdf:system-relative-pathname},
2950@code{merge-pathnames*},
2951@code{coerce-pathname}.
2952
2953On the other hand, there are places where systems used to accept namestrings
2954where you must now use an explicit pathname object:
2955@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2956must now be written with the @code{#p} syntax:
2957@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
2958
2959@xref{The defsystem grammar,,Pathname specifiers}.
2960
2961
2962@subsection Output translations
2963
2964A popular feature added to ASDF was output pathname translation:
2965@code{asdf-binary-locations}, @code{common-lisp-controller},
2966@code{cl-launch} and other hacks were all implementing it in ways
2967both mutually incompatible and difficult to configure.
2968
2969Output pathname translation is essential to share
2970source directories of portable systems across multiple implementations
2971or variants thereof,
2972or source directories of shared installations of systems across multiple users,
2973or combinations of the above.
2974
2975In ASDF 2, a standard mechanism is provided for that,
2976@code{asdf-output-translations},
2977with sensible defaults, adequate configuration languages,
2978a coherent set of configuration files and hooks,
2979and support for non-Unix platforms.
2980
2981@xref{Controlling where ASDF saves compiled files}.
2982
2983@subsection Source Registry Configuration
2984
2985Configuring ASDF used to require special magic
2986to be applied just at the right moment,
2987between the moment ASDF is loaded and the moment it is used,
2988in a way that is specific to the user,
2989the implementation he is using and the application he is building.
2990
2991This made for awkward configuration files and startup scripts
2992that could not be shared between users, managed by administrators
2993or packaged by distributions.
2994
2995ASDF 2 provides a well-documented way to configure ASDF,
2996with sensible defaults, adequate configuration languages,
2997and a coherent set of configuration files and hooks.
2998
2999We believe it's a vast improvement because it decouples
3000application distribution from library distribution.
3001The application writer can avoid thinking where the libraries are,
3002and the library distributor (dpkg, clbuild, advanced user, etc.)
3003can configure them once and for every application.
3004Yet settings can be easily overridden where needed,
3005so whoever needs control has exactly as much as required.
3006
3007At the same time, ASDF 2 remains compatible
3008with the old magic you may have in your build scripts
3009(using @code{*central-registry*} and
3010@code{*system-definition-search-functions*})
3011to tailor the ASDF configuration to your build automation needs,
3012and also allows for new magic, simpler and more powerful magic.
3013
3014@xref{Controlling where ASDF searches for systems}.
3015
3016
3017@subsection Usual operations are made easier to the user
3018
3019In ASDF 1, you had to use the awkward syntax
3020@code{(asdf:oos 'asdf:load-op :foo)}
3021to load a system,
3022and similarly for @code{compile-op}, @code{test-op}.
3023
3024In ASDF 2, you can use shortcuts for the usual operations:
3025@code{(asdf:load-system :foo)}, and
3026similarly for @code{compile-system}, @code{test-system}.
3027
3028
3029@subsection Many bugs have been fixed
3030
3031The following issues and many others have been fixed:
3032
3033@itemize
3034@item
3035The infamous TRAVERSE function has been revamped significantly,
3036with many bugs squashed.
3037In particular, dependencies were not correctly propagated
3038across submodules within a system but now are.
3039The :version and :feature features and
3040the :force (system1 .. systemN) feature have been fixed.
3041
3042@item
3043Performance has been notably improved for large systems
3044(say with thousands of components) by using
3045hash-tables instead of linear search,
3046and linear-time list accumulation
3047instead of quadratic-time recursive appends.
3048
3049@item
3050Many features used to not be portable,
3051especially where pathnames were involved.
3052Windows support was notably quirky because of such non-portability.
3053
3054@item
3055The internal test suite used to massively fail on many implementations.
3056While still incomplete, it now fully passes
3057on all implementations supported by the test suite,
3058except for GCL (due to GCL bugs).
3059
3060@item
3061Support was lacking for some implementations.
3062ABCL and GCL were notably wholly broken.
3063ECL extensions were not integrated in the ASDF release.
3064
3065@item
3066The documentation was grossly out of date.
3067
3068@end itemize
3069
3070
3071@subsection ASDF itself is versioned
3072
3073Between new features, old bugs fixed, and new bugs introduced,
3074there were various releases of ASDF in the wild,
3075and no simple way to check which release had which feature set.
3076People using or writing systems had to either make worst-case assumptions
3077as to what features were available and worked,
3078or take great pains to have the correct version of ASDF installed.
3079
3080With ASDF 2, we provide a new stable set of working features
3081that everyone can rely on from now on.
3082Use @code{#+asdf2} to detect presence of ASDF 2,
3083@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
3084to check the availability of a version no earlier than required.
3085
3086
3087@subsection ASDF can be upgraded
3088
3089When an old version of ASDF was loaded,
3090it was very hard to upgrade ASDF in your current image
3091without breaking everything.
3092Instead you had to exit the Lisp process and
3093somehow arrange to start a new one from a simpler image.
3094Something that can't be done from within Lisp,
3095making automation of it difficult,
3096which compounded with difficulty in configuration,
3097made the task quite hard.
3098Yet as we saw before, the task would have been required
3099to not have to live with the worst case or non-portable
3100subset of ASDF features.
3101
3102With ASDF 2, it is easy to upgrade
3103from ASDF 2 to later versions from within Lisp,
3104and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
3105We support hot upgrade of ASDF and any breakage is a bug
3106that we will do our best to fix.
3107There are still limitations on upgrade, though,
3108most notably the fact that after you upgrade ASDF,
3109you must also reload or upgrade all ASDF extensions.
3110
3111@subsection Decoupled release cycle
3112
3113When vendors were releasing their Lisp implementations with ASDF,
3114they had to basically never change version
3115because neither upgrade nor downgrade was possible
3116without breaking something for someone,
3117and no obvious upgrade path was visible and recommendable.
3118
3119With ASDF 2, upgrade is possible, easy and can be recommended.
3120This means that vendors can safely ship a recent version of ASDF,
3121confident that if a user isn't fully satisfied,
3122he can easily upgrade ASDF and deal
3123with a supported recent version of it.
3124This means that release cycles will be causally decoupled,
3125the practical consequence of which will mean faster convergence
3126towards the latest version for everyone.
3127
3128
3129@subsection Pitfalls of the transition to ASDF 2
3130
3131The main pitfalls in upgrading to ASDF 2 seem to be related
3132to the output translation mechanism.
3133
3134@itemize
3135
3136@item
3137Output translations is enabled by default. This may surprise some users,
3138most of them in pleasant way (we hope), a few of them in an unpleasant way.
3139It is trivial to disable output translations.
3140@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
3141
3142@item
3143Some systems in the large have been known not to play well with output translations.
3144They were relatively easy to fix.
3145Once again, it is also easy to disable output translations,
3146or to override its configuration.
3147
3148@item
3149The new ASDF output translations are incompatible with ASDF-Binary-Locations.
3150They replace A-B-L, and there is compatibility mode to emulate
3151your previous A-B-L configuration.
3152See @code{enable-asdf-binary-locations-compatibility} in
3153@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
3154But thou shall not load ABL on top of ASDF 2.
3155
3156@end itemize
3157
3158Other issues include the following:
3159
3160@itemize
3161
3162@item
3163ASDF pathname designators are now specified in places where they were unspecified,
3164and a few small adjustments have to be made to some non-portable defsystems.
3165Notably, in the @code{:pathname} argument to a @code{defsystem} and its components,
3166a logical pathname (or implementation-dependent hierarchical pathname)
3167must now be specified with @code{#p} syntax
3168where the namestring might have previously sufficed;
3169moreover when evaluation is desired @code{#.} must be used,
3170where it wasn't necessary in the toplevel @code{:pathname} argument.
3171
3172@item
3173There is a slight performance bug, notably on SBCL,
3174when initially searching for @file{asd} files,
3175the implicit @code{(directory "/configured/path/**/*.asd")}
3176for every configured path @code{(:tree "/configured/path/")}
3177in your @code{source-registry} configuration can cause a slight pause.
3178Try to @code{(time (asdf:initialize-source-registry))}
3179to see how bad it is or isn't on your system.
3180If you insist on not having this pause,
3181you can avoid the pause by overriding the default source-registry configuration
3182and not use any deep @code{:tree} entry but only @code{:directory} entries
3183or shallow @code{:tree} entries.
3184Or you can fix your implementation to not be quite that slow
3185when recursing through directories.
3186@underline{Update}: performance bug fixed the hard way in 2.010.
3187
3188@item
3189On Windows, only LispWorks supports proper default configuration pathnames
3190based on the Windows registry.
3191Other implementations make do with environment variables,
3192that you may have to define yourself if you're using an older version of Windows.
3193Windows support is somewhat less tested than Unix support.
3194Please help report and fix bugs.
3195
3196@item
3197The mechanism by which one customizes a system so that Lisp files
3198may use a different extension from the default @file{.lisp} has changed.
3199Previously, the pathname for a component was lazily computed when operating on a system,
3200and you would
3201@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
3202  (declare (ignorable component system)) "lis")}.
3203Now, the pathname for a component is eagerly computed when defining the system,
3204and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
3205and use @code{:default-component-class cl-source-file.lis} as argument to @code{defsystem},
3206as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
3207
3208@findex source-file-type
3209
3210
3211@end itemize
3212
3213
3214@section Issues with installing the proper version of ASDF
3215
3216@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
3217
3218We recommend you upgrade ASDF.
3219@xref{Loading ASDF,,Upgrading ASDF}.
3220
3221If this does not work, it is a bug, and you should report it.
3222@xref{FAQ, report-bugs, Where do I report a bug}.
3223In the meantime, you can load @file{asdf.lisp} directly.
3224@xref{Loading ASDF,Loading an otherwise installed ASDF}.
3225
3226
3227@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
3228
3229Starting with current candidate releases of ASDF 2,
3230it should always be a good time to upgrade to a recent ASDF.
3231You may consult with the maintainer for which specific version they recommend,
3232but the latest @code{release} should be correct.
3233We trust you to thoroughly test it with your implementation before you release it.
3234If there are any issues with the current release,
3235it's a bug that you should report upstream and that we will fix ASAP.
3236
3237As to how to include ASDF, we recommend the following:
3238
3239@itemize
3240@item
3241If ASDF isn't loaded yet, then @code{(require "asdf")}
3242should load the version of ASDF that is bundled with your system.
3243You may have it load some other version configured by the user,
3244if you allow such configuration.
3245
3246@item
3247If your system provides a mechanism to hook into @code{CL:REQUIRE},
3248then it would be nice to add ASDF to this hook the same way that
3249ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
3250
3251@item
3252You may, like SBCL, have ASDF be implicitly used to require systems
3253that are bundled with your Lisp distribution.
3254If you do have a few magic systems that come with your implementation
3255in a precompiled way such that one should only use the binary version
3256that goes with your distribution, like SBCL does,
3257then you should add them in the beginning of @code{wrapping-source-registry}.
3258
3259@item
3260If you have magic systems as above, like SBCL does,
3261then we explicitly ask you to @emph{NOT} distribute
3262@file{asdf.asd} as part of those magic systems.
3263You should still include the file @file{asdf.lisp} in your source distribution
3264and precompile it in your binary distribution,
3265but @file{asdf.asd} if included at all,
3266should be secluded from the magic systems,
3267in a separate file hierarchy;
3268alternatively, you may provide the system
3269after renaming it and its @file{.asd} file to e.g.
3270@code{asdf-ecl} and @file{asdf-ecl.asd}, or
3271@code{sb-asdf} and @file{sb-asdf.asd}.
3272Indeed, if you made @file{asdf.asd} a magic system,
3273then users would no longer be able to upgrade ASDF using ASDF itself
3274to some version of their preference that
3275they maintain independently from your Lisp distribution.
3276
3277@item
3278If you do not have any such magic systems, or have other non-magic systems
3279that you want to bundle with your implementation,
3280then you may add them to the @code{default-source-registry},
3281and you are welcome to include @file{asdf.asd} amongst them.
3282
3283@item
3284Please send us upstream any patches you make to ASDF itself,
3285so we can merge them back in for the benefit of your users
3286when they upgrade to the upstream version.
3287
3288@end itemize
3289
3290
3291
3292@section Issues with configuring ASDF
3293
3294@subsection ``How can I customize where fasl files are stored?''
3295
3296@xref{Controlling where ASDF saves compiled files}.
3297
3298Note that in the past there was an add-on to ASDF called
3299@code{ASDF-binary-locations}, developed by Gary King.
3300That add-on has been merged into ASDF proper,
3301then superseded by the @code{asdf-output-translations} facility.
3302
3303Note that use of @code{asdf-output-translations}
3304can interfere with one aspect of your systems
3305--- if your system uses @code{*load-truename*} to find files
3306(e.g., if you have some data files stored with your program),
3307then the relocation that this ASDF customization performs
3308is likely to interfere.
3309Use @code{asdf:system-relative-pathname} to locate a file
3310in the source directory of some system, and
3311use @code{asdf:apply-output-translations} to locate a file
3312whose pathname has been translated by the facility.
3313
3314@subsection ``How can I wholly disable the compiler output cache?''
3315
3316To permanently disable the compiler output cache
3317for all future runs of ASDF, you can:
3318
3319@example
3320mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
3321echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
3322@end example
3323
3324This assumes that you didn't otherwise configure the ASDF files
3325(if you did, edit them again),
3326and don't somehow override the configuration at runtime
3327with a shell variable (see below) or some other runtime command
3328(e.g. some call to @code{asdf:initialize-output-translations}).
3329
3330To disable the compiler output cache in Lisp processes
3331run by your current shell, try (assuming @code{bash} or @code{zsh})
3332(on Unix and cygwin only):
3333
3334@example
3335export ASDF_OUTPUT_TRANSLATIONS=/:
3336@end example
3337
3338To disable the compiler output cache just in the current Lisp process,
3339use (after loading ASDF but before using it):
3340
3341@example
3342(asdf:disable-output-translations)
3343@end example
3344
3345@section Issues with using and extending ASDF to define systems
3346
3347@subsection ``How can I cater for unit-testing in my system?''
3348
3349ASDF provides a predefined test operation, @code{test-op}.
3350@xref{Predefined operations of ASDF, test-op}.
3351The test operation, however, is largely left to the system definer to specify.
3352@code{test-op} has been
3353a topic of considerable discussion on the
3354@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
3355and on the
3356@url{https://launchpad.net/asdf,launchpad bug-tracker}.
3357
3358Here are some guidelines:
3359
3360@itemize
3361@item
3362For a given system, @var{foo}, you will want to define a corresponding
3363test system, such as @var{foo-test}.  The reason that you will want this
3364separate system is that ASDF does not out of the box supply components
3365that are conditionally loaded.  So if you want to have source files
3366(with the test definitions) that will not be loaded except when testing,
3367they should be put elsewhere.
3368
3369@item
3370The @var{foo-test} system can be defined in an asd file of its own or
3371together with @var{foo}.  An aesthetic preference against cluttering up
3372the filesystem with extra asd files should be balanced against the
3373question of whether one might want to directly load @var{foo-test}.
3374Typically one would not want to do this except in early stages of
3375debugging.
3376
3377@item
3378Record that testing is implemented by @var{foo-test}.  For example:
3379@example
3380(defsystem @var{foo}
3381   :in-order-to ((test-op (test-op @var{foo-test})))
3382   ....)
3383
3384(defsystem @var{foo-test}
3385   :depends-on (@var{foo} @var{my-test-library} ...)
3386   ....)
3387@end example
3388@end itemize
3389
3390This procedure will allow you to support users who do not wish to
3391install your test framework.
3392
3393One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
3394does not return a value.  So in current versions of ASDF there is no
3395reliable programmatic means of determining whether or not a set of tests
3396has passed, or which tests have failed.  The user must simply read the
3397console output.  This limitation has been the subject of much
3398discussion.
3399
3400@subsection ``How can I cater for documentation generation in my system?''
3401
3402The ASDF developers are currently working to add a @code{doc-op}
3403to the set of predefined ASDF operations.
3404@xref{Predefined operations of ASDF}.
3405See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
3406
3407
3408
3409@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
3410
3411See @code{cffi}'s @code{cffi-grovel}.
3412
3413@anchor{report-bugs}
3414
3415
3416@subsection ``I want to put my module's files at the top level.  How do I do this?''
3417
3418By default, the files contained in an asdf module go
3419in a subdirectory with the same name as the module.
3420However, this can be overridden by adding a @code{:pathname ""} argument
3421to the module description.
3422For example, here is how it could be done
3423in the spatial-trees ASDF system definition for ASDF 2:
3424
3425@example
3426(asdf:defsystem :spatial-trees
3427  :components
3428  ((:module base
3429            :pathname ""
3430            :components
3431            ((:file "package")
3432             (:file "basedefs" :depends-on ("package"))
3433             (:file "rectangles" :depends-on ("package"))))
3434   (:module tree-impls
3435            :depends-on (base)
3436            :pathname ""
3437            :components
3438            ((:file "r-trees")
3439             (:file "greene-trees" :depends-on ("r-trees"))
3440             (:file "rstar-trees" :depends-on ("r-trees"))
3441             (:file "rplus-trees" :depends-on ("r-trees"))
3442             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
3443   (:module viz
3444            :depends-on (base)
3445            :pathname ""
3446            :components
3447            ((:static-file "spatial-tree-viz.lisp")))
3448   (:module tests
3449            :depends-on (base)
3450            :pathname ""
3451            :components
3452            ((:static-file "spatial-tree-test.lisp")))
3453   (:static-file "LICENCE")
3454   (:static-file "TODO")))
3455@end example
3456
3457All of the files in the @code{tree-impls} module are at the top level,
3458instead of in a @file{tree-impls/} subdirectory.
3459
3460Note that the argument to @code{:pathname} can be either a pathname object or a string.
3461A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
3462but this is discouraged because the results of parsing a namestring are not portable.
3463A pathname can only be portably constructed with such syntax as
3464@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
3465and similarly the current directory can only be portably specified as
3466@code{#.(make-pathname :directory '(:relative))}.
3467However, as of ASDF 2, you can portably use a string to denote a pathname.
3468The string will be parsed as a @code{/}-separated path from the current directory,
3469such that the empty string @code{""} denotes the current directory, and
3470@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
3471portably denotes the same subdirectory as above.
3472When files are specified, the last @code{/}-separated component is interpreted
3473either as the name component of a pathname
3474(if the component class specifies a pathname type),
3475or as a name component plus optional dot-separated type component
3476(if the component class doesn't specifies a pathname type).
3477
3478@subsection How do I create a system definition where all the source files have a .cl extension?
3479
3480Starting with ASDF 2.014.14, you may just pass
3481the builtin class @code{cl-source-file.cl} as
3482the @code{:default-component-class} argument to @code{defsystem}:
3483
3484@lisp
3485(defsystem my-cl-system
3486  :default-component-class cl-source-file.cl
3487  ...)
3488@end lisp
3489
3490Another builtin class @code{cl-source-file.lsp} is offered
3491for files ending in @file{.lsp}.
3492
3493If you want to use a different extension
3494for which ASDF doesn't provide builtin support,
3495or want to support versions of ASDF
3496earlier than 2.014.14 (but later than 2.000),
3497you can define a class as follows:
3498
3499@lisp
3500;; Prologue: make sure we're using a sane package.
3501(defpackage :my-asdf-extension
3502   (:use :asdf :common-lisp)
3503   (:export #:cl-source-file.lis))
3504(in-package :my-asdf-extension)
3505
3506(defclass cl-source-file.lis (cl-source-file)
3507   ((type :initform "lis")))
3508@end lisp
3509
3510Then you can use it as follows:
3511@lisp
3512(defsystem my-cl-system
3513  :default-component-class my-asdf-extension:cl-source-file.lis
3514  ...)
3515@end lisp
3516
3517Of course, if you're in the same package, e.g. in the same file,
3518you won't need to use the package qualifier before @code{cl-source-file.lis}.
3519Actually, if all you're doing is defining this class
3520and using it in the same file without other fancy definitions,
3521you might skip package complications:
3522
3523@lisp
3524(in-package :asdf)
3525(defclass cl-source-file.lis (cl-source-file)
3526   ((type :initform "lis")))
3527(defsystem my-cl-system
3528  :default-component-class cl-source-file.lis
3529  ...)
3530@end lisp
3531
3532It is possible to achieve the same effect
3533in a way that supports both ASDF 1 and ASDF 2,
3534but really, friends don't let friends use ASDF 1.
3535Please upgrade to ASDF 2.
3536In short, though: do same as above, but
3537@emph{before} you use the class in a @code{defsystem},
3538you also define the following method:
3539
3540@lisp
3541(defmethod source-file-type ((f cl-source-file.lis) (m module))
3542  (declare (ignorable f m))
3543  "lis")
3544@end lisp
3545
3546
3547@node  TODO list, Inspiration, FAQ, Top
3548@comment  node-name,  next,  previous,  up
3549@chapter TODO list
3550
3551Here is an old list of things to do,
3552in addition to the bugs that are now tracked on launchpad:
3553@url{https://launchpad.net/asdf}.
3554
3555@section Outstanding spec questions, things to add
3556
3557** packaging systems
3558
3559*** manual page component?
3560
3561** style guide for .asd files
3562
3563You should either use keywords or be careful
3564with the package that you evaluate defsystem forms in.
3565Otherwise @code{(defsystem partition ...)}
3566being read in the @code{cl-user} package
3567will intern a @code{cl-user:partition} symbol,
3568which will then collide with the @code{partition:partition} symbol.
3569
3570Actually there's a hairier packages problem to think about too.
3571@code{in-order-to} is not a keyword:
3572if you read @code{defsystem} forms in a package that doesn't use ASDF,
3573odd things might happen.
3574
3575
3576** extending defsystem with new options
3577
3578You might not want to write a whole parser,
3579but just to add options to the existing syntax.
3580Reinstate @code{parse-option} or something akin.
3581
3582
3583** document all the error classes
3584
3585** what to do with compile-file failure
3586
3587Should check the primary return value from compile-file and see if
3588that gets us any closer to a sensible error handling strategy
3589
3590** foreign files
3591
3592lift unix-dso stuff from db-sockets
3593
3594** Diagnostics
3595
3596A ``dry run'' of an operation can be made with the following form:
3597
3598@lisp
3599(traverse (make-instance '<operation-name>)
3600          (find-system <system-name>)
3601          'explain)
3602@end lisp
3603
3604This uses unexported symbols.
3605What would be a nice interface for this functionality?
3606
3607@section Missing bits in implementation
3608
3609** reuse the same scratch package whenever a system is reloaded from disk
3610
3611** proclamations probably aren't
3612
3613** when a system is reloaded with fewer components than it previously had, odd things happen
3614
3615We should do something inventive when processing a @code{defsystem} form,
3616like take the list of kids and @code{setf} the slot to @code{nil},
3617then transfer children from old to new list as they're found.
3618
3619** (stuff that might happen later)
3620
3621*** Propagation of the @code{:force} option.
3622
3623``I notice that
3624
3625        @code{(asdf:compile-system :araneida :force t)}
3626
3627also forces compilation of every other system the @code{:araneida} system depends on.
3628This is rarely useful to me;
3629usually, when I want to force recompilation of something more than a single source file,
3630I want to recompile only one system.
3631So it would be more useful to have @code{make-sub-operation}
3632refuse to propagate @code{:force t} to other systems, and
3633propagate only something like @code{:force :recursively}.
3634
3635Ideally what we actually want is some kind of criterion that says
3636to which systems (and which operations) a @code{:force} switch will propagate.
3637
3638The problem is perhaps that ``force'' is a pretty meaningless concept.
3639How obvious is it that @code{load :force t} should force @emph{compilation}?
3640But we don't really have the right dependency setup
3641for the user to compile @code{:force t} and expect it to work
3642(files will not be loaded after compilation, so the compile
3643environment for subsequent files will be emptier than it needs to be)
3644
3645What does the user actually want to do when he forces?
3646Usually, for me, update for use with a new version of the Lisp compiler.
3647Perhaps for recovery when he suspects that something has gone wrong.
3648Or else when he's changed compilation options or configuration
3649in some way that's not reflected in the dependency graph.
3650
3651Other possible interface: have a ``revert'' function akin to @code{make clean}.
3652
3653@lisp
3654(asdf:revert 'asdf:compile-op 'araneida)
3655@end lisp
3656
3657would delete any files produced by @code{(compile-system :araneida)}.
3658Of course, it wouldn't be able to do much about stuff in the image itself.
3659
3660How would this work?
3661
3662@code{traverse}
3663
3664There's a difference between a module's dependencies (peers)
3665and its components (children).
3666Perhaps there's a similar difference in operations?
3667For example, @code{(load "use") depends-on (load "macros")} is a peer,
3668whereas @code{(load "use") depends-on (compile "use")}
3669is more of a ``subservient'' relationship.
3670
3671@node  Inspiration, Concept Index, TODO list, Top
3672@comment  node-name,  next,  previous,  up
3673@chapter Inspiration
3674
3675@section mk-defsystem (defsystem-3.x)
3676
3677We aim to solve basically the same problems as @code{mk-defsystem} does.
3678However, our architecture for extensibility
3679better exploits CL language features (and is documented),
3680and we intend to be portable rather than just widely-ported.
3681No slight on the @code{mk-defsystem} authors and maintainers is intended here;
3682that implementation has the unenviable task
3683of supporting pre-ANSI implementations, which is no longer necessary.
3684
3685The surface defsystem syntax of asdf is more-or-less compatible with
3686@code{mk-defsystem}, except that we do not support
3687the @code{source-foo} and @code{binary-foo} prefixes
3688for separating source and binary files, and
3689we advise the removal of all options to specify pathnames.
3690
3691The @code{mk-defsystem} code for topologically sorting
3692a module's dependency list was very useful.
3693
3694@section defsystem-4 proposal
3695
3696Marco and Peter's proposal for defsystem 4 served as the driver for
3697many of the features in here.  Notable differences are:
3698
3699@itemize
3700@item
3701We don't specify output files or output file extensions
3702as part of the system.
3703
3704If you want to find out what files an operation would create,
3705ask the operation.
3706
3707@item
3708We don't deal with CL packages
3709
3710If you want to compile in a particular package, use an @code{in-package} form
3711in that file (ilisp / SLIME will like you more if you do this anyway)
3712
3713@item
3714There is no proposal here that @code{defsystem} does version control.
3715
3716A system has a given version which can be used to check dependencies,
3717but that's all.
3718@end itemize
3719
3720The defsystem 4 proposal tends to look more at the external features,
3721whereas this one centres on a protocol for system introspection.
3722
3723@section kmp's ``The Description of Large Systems'', MIT AI Memo 801
3724
3725Available in updated-for-CL form on the web at
3726@url{http://nhplace.com/kent/Papers/Large-Systems.html}
3727
3728In our implementation we borrow kmp's overall @code{PROCESS-OPTIONS}
3729and concept to deal with creating component trees
3730from @code{defsystem} surface syntax.
3731[ this is not true right now, though it used to be and
3732probably will be again soon ]
3733
3734
3735@c -------------------
3736
3737
3738@node Concept Index, Function and Class Index, Inspiration, Top
3739@unnumbered Concept Index
3740
3741@printindex cp
3742
3743@node Function and Class Index, Variable Index, Concept Index, Top
3744@unnumbered Function and Class Index
3745
3746@printindex fn
3747
3748@node Variable Index,  , Function and Class Index, Top
3749@unnumbered Variable Index
3750
3751@printindex vr
3752
3753@bye
Note: See TracBrowser for help on using the repository browser.