source: trunk/abcl/doc/asdf/asdf.texinfo @ 14714

Last change on this file since 14714 was 14714, checked in by Mark Evenson, 8 years ago

ASDF 3.1.2.9

changeset: 2488:0a1ded36af37
bookmark: master
tag: default/master
tag: tip
user: Francois-Rene Rideau <tunes@…>
summary: Tweak the debian changelog.

File size: 230.6 KB
Line 
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
3@setfilename asdf.info
4@settitle ASDF Manual
5@c %**end of header
6
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro AallowOtherKeys
10&allow-other-keys
11@end macro
12@macro Aoptional
13&optional
14@end macro
15@macro Arest
16&rest
17@end macro
18@macro Akey
19&key
20@end macro
21@macro Abody
22&body
23@end macro
24
25@c for install-info
26@dircategory Software development
27@direntry
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
30
31@copying
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
34
35You can find the latest version of this manual at
36@url{http://common-lisp.net/project/asdf/asdf.html}.
37
38ASDF Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
39
40This manual Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
41
42This manual revised @copyright{} 2009-2014 Robert P. Goldman and Francois-Rene Rideau.
43
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
51
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
54
55THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
56EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
57MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
58NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
59LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
60OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
61WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
62
63@end copying
64
65
66
67@titlepage
68@title ASDF: Another System Definition Facility
69
70@c The following two commands start the copyright page.
71@page
72@vskip 0pt plus 1filll
73@insertcopying
74@end titlepage
75
76@c Output the table of contents at the beginning.
77@contents
78
79@c -------------------
80
81@ifnottex
82
83@node Top, Introduction, (dir), (dir)
84@top ASDF: Another System Definition Facility
85
86@insertcopying
87
88@menu
89* Introduction::
90* Quick start summary::
91* Loading ASDF::
92* Configuring ASDF::
93* Using ASDF::
94* Defining systems with defsystem::
95* The object model of ASDF::
96* Controlling where ASDF searches for systems::
97* Controlling where ASDF saves compiled files::
98* Error handling::
99* Miscellaneous additional functionality::
100* Getting the latest version::
101* FAQ::
102* Ongoing Work::
103* Bibliography::
104* Concept Index::
105* Function and Class Index::
106* Variable Index::              @c @detailmenu
107@c
108
109@detailmenu
110 --- The Detailed Node Listing ---
111
112Loading ASDF
113
114* Loading a pre-installed ASDF::
115* Checking whether ASDF is loaded::
116* Upgrading ASDF::
117* Loading ASDF from source::
118
119Upgrading ASDF
120
121* Upgrading your implementation's ASDF::
122* Issues with upgrading ASDF::
123
124Configuring ASDF
125
126* Configuring ASDF to find your systems::
127* Configuring ASDF to find your systems --- old style::
128* Configuring where ASDF stores object files::
129* Resetting the ASDF configuration::
130
131Using ASDF
132
133* Loading a system::
134* Other Operations::
135* Moving on::
136
137Defining systems with defsystem
138
139* The defsystem form::
140* A more involved example::
141* The defsystem grammar::
142* Other code in .asd files::
143* The package-inferred-system extension::
144
145The Object model of ASDF
146
147* Operations::
148* Components::
149* Dependencies::
150* Functions::
151
152Operations
153
154* Predefined operations of ASDF::
155* Creating new operations::
156
157Components
158
159* Common attributes of components::
160* Pre-defined subclasses of component::
161* Creating new component types::
162
163properties
164
165* Pre-defined subclasses of component::
166* Creating new component types::
167
168Controlling where ASDF searches for systems
169
170* Configurations::
171* Truenames and other dangers::
172* XDG base directory::
173* Backward Compatibility::
174* Configuration DSL::
175* Configuration Directories::
176* Shell-friendly syntax for configuration::
177* Search Algorithm::
178* Caching Results::
179* Configuration API::
180* Introspection::
181* Status::
182* Rejected ideas::
183* TODO::
184* Credits for the source-registry::
185
186Configuration Directories
187
188* The here directive::
189
190Introspection
191
192* *source-registry-parameter* variable::
193* Information about system dependencies::
194
195Controlling where ASDF saves compiled files
196
197* Output Configurations::
198* Output Backward Compatibility::
199* Output Configuration DSL::
200* Output Configuration Directories::
201* Output Shell-friendly syntax for configuration::
202* Semantics of Output Translations::
203* Output Caching Results::
204* Output location API::
205* Credits for output translations::
206
207Miscellaneous additional functionality
208
209* Controlling file compilation::
210* Controlling source file character encoding::
211* Some Utility Functions::
212
213FAQ
214
215* Where do I report a bug?::
216* What has changed between ASDF 1 and ASDF 2?::
217* Issues with installing the proper version of ASDF::
218* Issues with configuring ASDF::
219* Issues with using and extending ASDF to define systems::
220* ASDF development FAQs::
221
222``What has changed between ASDF 1, ASDF 2 and ASDF 3?''
223
224* What are ASDF 1 2 3?::
225* How do I detect the ASDF version?::
226* ASDF can portably name files in subdirectories::
227* Output translations::
228* Source Registry Configuration::
229* Usual operations are made easier to the user::
230* Many bugs have been fixed::
231* ASDF itself is versioned::
232* ASDF can be upgraded::
233* Decoupled release cycle::
234* Pitfalls of the transition to ASDF 2::
235
236Issues with installing the proper version of ASDF
237
238* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
239* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
240
241Issues with configuring ASDF
242
243* How can I customize where fasl files are stored?::
244* How can I wholly disable the compiler output cache?::
245
246Issues with using and extending ASDF to define systems
247
248* How can I cater for unit-testing in my system?::
249* How can I cater for documentation generation in my system?::
250* How can I maintain non-Lisp (e.g. C) source files?::
251* I want to put my module's files at the top level.  How do I do this?::
252* How do I create a system definition where all the source files have a .cl extension?::
253* How do I mark a source file to be loaded only and not compiled?::
254* How do I work with readtables?::
255
256ASDF development FAQs
257
258* How do run the tests interactively in a REPL?::
259
260@end detailmenu
261@end menu
262
263@end ifnottex
264
265@c -------------------
266
267@node Introduction, Quick start summary, Top, Top
268@comment  node-name,  next,  previous,  up
269@chapter Introduction
270@cindex ASDF-related features
271@vindex *features*
272@cindex Testing for ASDF
273@cindex ASDF versions
274@cindex :asdf
275@cindex :asdf2
276@cindex :asdf3
277
278ASDF is Another System Definition Facility:
279a tool for specifying how systems of Common Lisp software
280are made up of components (sub-systems and files),
281and how to operate on these components in the right order
282so that they can be compiled, loaded, tested, etc.
283If you are new to ASDF, @pxref{Quick start summary,,the quick start
284guide}.
285
286ASDF presents three faces:
287one for users of Common Lisp software who want to reuse other people's code,
288one for writers of Common Lisp software who want to specify how to build their systems,
289and one for implementers of Common Lisp extensions who want to extend
290the build system.
291For more specifics,
292@pxref{Using ASDF,,Loading a system},
293to learn how to use ASDF to load a system.
294@xref{Defining systems with defsystem},
295to learn how to define a system of your own.
296@xref{The object model of ASDF}, for a description of
297the ASDF internals and how to extend ASDF.
298
299Note that
300ASDF is @emph{not} a tool for library and system @emph{installation}; it
301plays a role like @t{make} or @t{ant}, not like a package manager.
302In particular, ASDF should not to be confused with ASDF-Install, which attempts to find and
303download ASDF systems for you.
304Despite the name, ASDF-Install is not part of ASDF, but a separate piece of software.
305ASDF-Install is also unmaintained and obsolete.
306We recommend you use Quicklisp
307(@uref{http://www.quicklisp.org}) instead,
308a Common Lisp package manager which works well and is being actively maintained.
309If you want to download software from version control instead of tarballs,
310so you may more easily modify it, we recommend clbuild (@uref{http://common-lisp.net/project/clbuild/}).
311We recommend @file{~/common-lisp/}
312as a place into which to install Common Lisp software;
313starting with ASDF 3.1.2, it is included in the default source-registry configuration.
314
315@node  Quick start summary, Loading ASDF, Introduction, Top
316@chapter Quick start summary
317
318@itemize
319
320@item To load an ASDF system:
321
322@itemize
323@item
324Load ASDF itself into your Lisp image, either through
325@code{(require "asdf")} (if it's supplied by your lisp implementation)
326or else through
327@code{(load "/path/to/asdf.lisp")}.  For more details, @ref{Loading ASDF}.
328
329@item
330Make sure ASDF can find system definitions
331through proper source-registry configuration.
332For more details, @xref{Configuring ASDF to find your systems}.
333The simplest way is simply to put all your lisp code in subdirectories of
334@file{~/common-lisp/} (starting with ASDF 3.1.2),
335or @file{~/.local/share/common-lisp/source/}
336(for ASDF 2 and later, or if you want to keep source in a hidden directory).
337Such code will automatically be found.
338
339@item
340Load a system with @code{(asdf:load-system :system)}. @xref{Using ASDF}.
341
342@end itemize
343
344@item To make your own ASDF system:
345
346@itemize
347@item
348As above, load and configure ASDF.
349
350@item
351Make a new directory for your system, @code{my-system/} in a location
352where ASDF can find it (@pxref{Configuring ASDF to find your systems}).
353All else being equal, the easiest location is probably
354@file{~/common-lisp/my-system/}.
355
356
357@item
358Create an ASDF system definition listing the dependencies of
359your system, its components, and their interdependencies,
360and put it in @file{my-system.asd}.
361This file must have the same name as your system.
362@xref{Defining systems with defsystem}.
363
364@item
365Use @code{(asdf:load-system :my-system)}
366to make sure it's all working properly. @xref{Using ASDF}.
367
368@end itemize
369@end itemize
370
371@c FIXME: (1) add a sample project that the user can cut and paste to
372@c get started.  (2) discuss the option of starting with Quicklisp.
373
374
375
376
377
378@node Loading ASDF, Configuring ASDF, Quick start summary, Top
379@comment  node-name,  next,  previous,  up
380@chapter Loading ASDF
381
382@menu
383* Loading a pre-installed ASDF::
384* Checking whether ASDF is loaded::
385* Upgrading ASDF::
386* Loading ASDF from source::
387@end menu
388
389@node  Loading a pre-installed ASDF, Checking whether ASDF is loaded, Loading ASDF, Loading ASDF
390@section Loading a pre-installed ASDF
391
392Most recent Lisp implementations include a copy of ASDF 3,
393or at least ASDF 2.
394You can usually load this copy using Common Lisp's @code{require} function.@footnote{
395NB: all implementations except GNU CLISP also accept
396@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
397For portability's sake, you should use @code{(require "asdf")}.
398}
399
400@lisp
401(require "asdf")
402@end lisp
403
404As of the writing of this manual,
405the following implementations provide ASDF 3 this way:
406ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, MKCL, SBCL.
407The following implementations only provide ASDF 2:
408LispWorks, mocl, XCL.
409The following implementations don't provide ASDF:
410Corman CL, GCL, Genera, MCL, SCL.
411The latter implementations are not actively maintained;
412if some of them are ever released again, they probably will include ASDF 3.
413
414If the implementation you are using doesn't provide ASDF 2 or ASDF 3,
415see @pxref{Loading ASDF,,Loading ASDF from source} below.
416If that implementation is still actively maintained,
417you may also send a bug report to your Lisp vendor and complain
418about their failing to provide ASDF.
419
420@node Checking whether ASDF is loaded, Upgrading ASDF, Loading a pre-installed ASDF, Loading ASDF
421@section Checking whether ASDF is loaded
422
423To check whether ASDF is properly loaded in your current Lisp image,
424you can run this form:
425
426@lisp
427(asdf:asdf-version)
428@end lisp
429
430If it returns a string,
431that is the version of ASDF that is currently installed.
432
433If it raises an error,
434then either ASDF is not loaded, or
435you are using a very old version of ASDF,
436and need to install ASDF 3.
437
438You can check whether an old version is loaded
439by checking if the ASDF package is present.
440The form below will allow you to programmatically determine
441whether a recent version is loaded, an old version is loaded,
442or none at all:
443
444@lisp
445(when (find-package :asdf)
446  (let ((ver (symbol-value
447                   (or (find-symbol (string :*asdf-version*) :asdf)
448                       (find-symbol (string :*asdf-revision*) :asdf)))))
449    (etypecase ver
450      (string ver)
451      (cons (with-output-to-string (s)
452              (loop for (n . m) on ver
453                    do (princ n s)
454                       (when m (princ "." s)))))
455      (null "1.0"))))
456@end lisp
457
458If it returns @code{nil} then ASDF is not installed.
459Otherwise it should return a string.
460If it returns @code{"1.0"}, then it can actually be
461any version before 1.77 or so, or some buggy variant of 1.x.
462
463If you are experiencing problems with ASDF,
464please try upgrading to the latest released version,
465using the method below,
466before you contact us and raise an issue.
467
468@node  Upgrading ASDF, Loading ASDF from source, Checking whether ASDF is loaded, Loading ASDF
469@section Upgrading ASDF
470@c FIXME: tighten this up a bit -- there's a lot of stuff here that
471@c doesn't matter to almost anyone.  Move discussion of updating antique
472@c versions of ASDF down, or encapsulate it.
473
474If you want to upgrade to a more recent ASDF version,
475you need to install and configure your ASDF just like any other system
476(@pxref{Configuring ASDF to find your systems}).
477
478If your implementation provides ASDF 3 or later,
479you only need to @code{(require "asdf")}:
480ASDF will automatically look whether an updated version of itself is available
481amongst the regularly configured systems, before it compiles anything else.
482
483@menu
484* Upgrading your implementation's ASDF::
485* Issues with upgrading ASDF::
486@end menu
487
488@node Upgrading your implementation's ASDF, Issues with upgrading ASDF, Upgrading ASDF, Upgrading ASDF
489@subsection Upgrading your implementation's ASDF
490
491Most implementations provide a recent ASDF 3 in their latest release.
492If yours doesn't, we recommend upgrading your implementation.
493If the latest version of your implementation still doesn't provide ASDF,
494or provides an old version, we recommend installing a recent ASDF so your implementation provides it,
495as explained below.
496If all fails, we recommend you load ASDF from source
497@pxref{Loading ASDF,,Loading ASDF from source}.
498
499The ASDF source repository contains a script
500@file{bin/install-asdf-as-module} that can help you upgrade your implementation's ASDF.
501It works on
502Allegro CL, Clozure CL, CMU CL, ECL, GNU CLISP, LispWorks, MKCL, SBCL, SCL, XCL.
503That's all known implementations except ABCL, Corman CL, GCL, Genera, MCL, MOCL.
504Happily, ABCL is usually pretty up to date and shouldn't need that script.
505GCL would be supported, except that so far is still lacking usable support for @code{require}.
506Corman CL, Genera, MCL are obsolete anyway.
507MOCL is under development.
508On an old version of an implementation that does not provide ASDF,
509you may have to load ASDF 3 from source before you load that script.
510
511The script relies on @code{cl-launch} 4 for command-line invocation,
512which may depend on ASDF being checked out in @file{~/common-lisp/asdf/}
513(which we recommend anyway)
514if your implementation doesn't even have an ASDF 2.
515If you don't have @code{cl-launch},
516you can instead @code{(load "bin/install-asdf-as-module")}
517from your implementation's REPL after loading ASDF from source.
518
519Finally, if your implementation only provides ASDF 2,
520and you can't or won't upgrade it or override its ASDF module,
521you may simply configure ASDF to find a proper upgrade;
522however, to avoid issues with a self-upgrade in mid-build,
523you @emph{must} make sure to upgrade ASDF immediately
524after requiring the builtin ASDF 2:
525
526@lisp
527(require "asdf")
528;; <--- insert programmatic configuration here if needed
529(asdf:load-system :asdf)
530@end lisp
531
532@node Issues with upgrading ASDF,  , Upgrading your implementation's ASDF, Upgrading ASDF
533@subsection Issues with upgrading ASDF
534
535Note that there are some limitations to upgrading ASDF:
536@itemize
537@item
538Previously loaded ASDF extensions become invalid, and will need to be reloaded.
539Examples include CFFI-Grovel, hacks used by ironclad, etc.
540Since it isn't possible to automatically detect what extensions
541need to be invalidated,
542ASDF will invalidate @emph{all} previously loaded systems
543when it is loaded on top of a forward-incompatible ASDF version.
544@footnote{
545@vindex *oldest-forward-compatible-asdf-version*
546Forward incompatibility can be determined using the variable
547@code{asdf/upgrade::*oldest-forward-compatible-asdf-version*},
548which is 2.33 at the time of this writing.}
549
550Starting with ASDF 3 (2.27 or later),
551this self-upgrade will be automatically attempted as the first step
552to any system operation, to avoid any possibility of
553a catastrophic attempt to self-upgrade in mid-build.
554
555@c FIXME: Fix grammar below.
556@item
557For this and many other reasons,
558you should load, configure and upgrade ASDF
559as one of the very first things done by your build and startup scripts.
560It is safer if you upgrade ASDF and its extensions as a special step
561at the very beginning of whatever script you are running,
562before you start using ASDF to load anything else.
563
564@item
565Until all implementations provide ASDF 3 or later,
566it is unsafe to upgrade ASDF as part of loading a system
567that depends on a more recent version of ASDF,
568since the new one might shadow the old one while the old one is running,
569and the running old one will be confused
570when extensions are loaded into the new one.
571In the meantime, we recommend that your systems should @emph{not} specify
572@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "3.0.1"))},
573but instead that they check that a recent enough ASDF is installed,
574with such code as:
575@example
576(unless (or #+asdf2 (asdf:version-satisfies
577                     (asdf:asdf-version) *required-asdf-version*))
578  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
579@end example
580@item
581Until all implementations provide ASDF 3 or later,
582it is unsafe for a system to transitively depend on ASDF
583and not directly depend on ASDF;
584if any of the system you use either depends-on asdf,
585system-depends-on asdf, or transitively does,
586you should also do as well.
587@end itemize
588
589@node Loading ASDF from source,  , Upgrading ASDF, Loading ASDF
590@section Loading ASDF from source
591
592If your implementation doesn't include ASDF,
593if for some reason the upgrade somehow fails,
594does not or cannot apply to your case,
595you will have to install the file @file{asdf.lisp}
596somewhere and load it with:
597
598@lisp
599(load "/path/to/your/installed/asdf.lisp")
600@end lisp
601
602The single file @file{asdf.lisp} is all you normally need to use ASDF.
603
604You can extract this file from latest release tarball on the
605@url{http://common-lisp.net/project/asdf/,ASDF website}.
606If you are daring and willing to report bugs, you can get
607the latest and greatest version of ASDF from its git repository.
608@xref{Getting the latest version}.
609
610For maximum convenience you might want to have ASDF loaded
611whenever you start your Lisp implementation,
612for example by loading it from the startup script or dumping a custom core
613--- check your Lisp implementation's manual for details.
614
615
616@node Configuring ASDF, Using ASDF, Loading ASDF, Top
617@comment  node-name,  next,  previous,  up
618@chapter Configuring ASDF
619
620For standard use cases, ASDF should work pretty much out of the box.
621We recommend you skim the sections on configuring ASDF to find your systems
622and choose the method of installing Lisp software that works best for you.
623Then skip directly to @xref{Using ASDF}. That will probably be enough.
624You are unlikely to have to worry about the way ASDF stores object files,
625and resetting the ASDF configuration is usually only needed in corner cases.
626
627
628@menu
629* Configuring ASDF to find your systems::
630* Configuring ASDF to find your systems --- old style::
631* Configuring where ASDF stores object files::
632* Resetting the ASDF configuration::
633@end menu
634
635@node Configuring ASDF to find your systems, Configuring ASDF to find your systems --- old style, Configuring ASDF, Configuring ASDF
636@section Configuring ASDF to find your systems
637
638In order to compile and load your systems, ASDF must be configured to find
639the @file{.asd} files that contain system definitions.
640
641There are a number of different techniques for setting yourself up with
642ASDF, starting from easiest to the most complex:
643
644@itemize @bullet
645
646@item
647Put all of your systems in one of the standard locations, subdirectories
648of
649@itemize
650@item
651@file{~/common-lisp/} or
652@item
653@file{~/.local/share/common-lisp/source/}.
654@end itemize
655If you install software there, you don't need further
656configuration.@footnote{@file{~/common-lisp/} is only included in
657the default configuration
658starting with ASDF 3.1.2 or later.}
659
660@item
661If you're using some tool to install software (e.g. Quicklisp),
662the authors of that tool should already have configured ASDF.
663
664@item
665If you have more specific desires about how to lay out your software on
666disk, the preferred way to configure where ASDF finds your systems is
667the @code{source-registry} facility,
668fully described in its own chapter of this manual.
669@xref{Controlling where ASDF searches for systems}.  Here is a quick
670recipe for getting started:
671
672The simplest way to add a path to your search path,
673say @file{/home/luser/.asd-link-farm/}
674is to create the directory
675@file{~/.config/common-lisp/source-registry.conf.d/}
676and there create a file with any name of your choice,
677and with the type @file{conf}@footnote{By requiring the @file{.conf}
678extension, and ignoring other files, ASDF allows you to have disabled files,
679editor backups, etc. in the same directory with your active
680configuration files.
681
682ASDF will also ignore files whose names start with a @file{.} character.
683
684It is customary to start the filename with two digits, to control the
685sorting of the @code{conf} files in the source registry directory, and
686thus the order in which the directories will be scanned.},
687for instance @file{42-asd-link-farm.conf},
688containing the line:
689
690@kbd{(:directory "/home/luser/.asd-link-farm/")}
691
692If you want all the subdirectories under @file{/home/luser/lisp/}
693to be recursively scanned for @file{.asd} files, instead use:
694
695@kbd{(:tree "/home/luser/lisp/")}
696
697ASDF will automatically read your configuration
698the first time you try to find a system.
699If necessary, you can reset the source-registry configuration with:
700
701@lisp
702(asdf:clear-source-registry)
703@end lisp
704
705@item
706In earlier versions of ASDF, the system source registry was configured
707using a global variable, @code{asdf:*central-registry*}.
708For more details about this, see the following section,
709@ref{Configuring ASDF to find your systems --- old style}.
710Unless you need to understand this,
711skip directly to @ref{Configuring where ASDF stores object files}.
712
713@end itemize
714
715Note that your Operating System distribution or your system administrator
716may already have configured system-managed libraries for you.
717
718
719
720@node Configuring ASDF to find your systems --- old style, Configuring where ASDF stores object files, Configuring ASDF to find your systems, Configuring ASDF
721@section Configuring ASDF to find your systems --- old style
722
723
724@c FIXME: this section should be moved elsewhere.  The novice user
725@c should not be burdened with it. [2014/02/27:rpg]
726
727
728The old way to configure ASDF to find your systems is by
729@code{push}ing directory pathnames onto the variable
730@code{asdf:*central-registry*}.
731
732You must configure this variable between the time you load ASDF
733and the time you first try to use it.
734Loading and configuring ASDF presumably happen
735as part of some initialization script that builds or starts
736your Common Lisp software system.
737(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
738
739The @code{asdf:*central-registry*} is empty by default in ASDF 2 or ASDF 3,
740but is still supported for compatibility with ASDF 1.
741When used, it takes precedence over the above source-registry.@footnote{
742It is possible to further customize
743the system definition file search.
744That's considered advanced use, and covered later:
745search forward for
746@code{*system-definition-search-functions*}.
747@xref{Defining systems with defsystem}.}
748
749For example, let's say you want ASDF to find the @file{.asd} file
750@file{/home/me/src/foo/foo.asd}.
751In your lisp initialization file, you could have the following:
752
753@lisp
754(require "asdf")
755(push "/home/me/src/foo/" asdf:*central-registry*)
756@end lisp
757
758Note the trailing slash: when searching for a system,
759ASDF will evaluate each entry of the central registry
760and coerce the result to a pathname.@footnote{
761ASDF will indeed call @code{eval} on each entry.
762It will skip entries that evaluate to @code{nil}.
763
764Strings and pathname objects are self-evaluating,
765in which case the @code{eval} step does nothing;
766but you may push arbitrary s-expressions onto the central registry.
767These s-expressions may be evaluated to compute context-dependent
768entries, e.g. things that depend
769on the value of shell variables or the identity of the user.
770
771The variable @code{asdf:*central-registry*} is thus a list of
772``system directory designators''.
773A @dfn{system directory designator} is a form
774which will be evaluated whenever a system is to be found,
775and must evaluate to a directory to look in (or @code{NIL}).
776By ``directory'', we mean
777``designator for a pathname with a non-empty DIRECTORY component''.
778}
779The trailing directory name separator
780is necessary to tell Lisp that you're discussing a directory
781rather than a file.  If you leave it out, ASDF is likely to look in
782@code{/home/me/src/} instead of @code{/home/me/src/foo/} as you
783intended, and fail to find your system definition.
784
785Typically there are a lot of @file{.asd} files, and
786a common idiom was to put
787@emph{symbolic links} to all of one's @file{.asd} files
788in a common directory
789and push @emph{that} directory (the ``link farm'')
790onto
791@code{asdf:*central-registry*},
792instead of pushing each individual system directory.
793
794ASDF knows to follow @emph{symlinks}
795to the actual location of the systems.@footnote{
796On Windows, you can use Windows shortcuts instead of POSIX symlinks.
797if you try aliases under MacOS, we are curious to hear about your experience.}
798
799For example, if @code{#p"/home/me/cl/systems/"}
800is an element of @code{*central-registry*}, you could set up the
801system @var{foo} as follows:
802
803@example
804$ cd /home/me/cl/systems/
805$ ln -s ~/src/foo/foo.asd .
806@end example
807
808This old style for configuring ASDF is not recommended for new users,
809but it is supported for old users, and for users who want to programmatically
810control what directories are added to the ASDF search path.
811
812
813@node Configuring where ASDF stores object files, Resetting the ASDF configuration, Configuring ASDF to find your systems --- old style, Configuring ASDF
814@section Configuring where ASDF stores object files
815@findex clear-output-translations
816
817ASDF lets you configure where object files will be stored.
818Sensible defaults are provided and
819you shouldn't normally have to worry about it.
820
821This allows the same source code repository to be shared
822between several versions of several Common Lisp implementations,
823between several users using different compilation options,
824with users who lack write privileges on shared source directories, etc.
825This also keeps source directories from being cluttered
826with object/fasl files.
827
828Starting with ASDF 2, the @code{asdf-output-translations} facility
829was added to ASDF itself.  This facility controls where object files will be stored.
830This facility is fully described in a chapter of this manual,
831@ref{Controlling where ASDF saves compiled files}.
832
833@c FIXME: possibly this should be moved elsewhere.  It's redundant here,
834@c and makes this section of the manual too long and daunting for the
835@c new user. [2014/02/27:rpg]
836@c The simplest way to add a translation to your search path,
837@c say from @file{/foo/bar/baz/quux/}
838@c to @file{/where/i/want/my/fasls/}
839@c is to create the directory
840@c @file{~/.config/common-lisp/asdf-output-translations.conf.d/}
841@c and there create a file with any name of your choice and the type @file{conf},
842@c for instance @file{42-bazquux.conf}
843@c containing the line:
844
845@c @kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
846
847@c To disable output translations for source under a given directory,
848@c say @file{/toto/tata/}
849@c you can create a file @file{40-disable-toto.conf}
850@c with the line:
851
852@c @kbd{("/toto/tata/")}
853
854@c To wholly disable output translations for all directories,
855@c you can create a file @file{00-disable.conf}
856@c with the line:
857
858@c @kbd{(t t)}
859
860@c Note that your Operating System distribution or your system administrator
861@c may already have configured translations for you.
862@c In absence of any configuration, the default is to redirect everything
863@c under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
864@c @xref{Controlling where ASDF searches for systems}, for full details.
865
866@c The required @file{.conf} extension allows you to have disabled files
867@c or editor backups (ending in @file{~}), and works portably
868@c (for instance, it is a pain to allow both empty and non-empty extension on CLISP).
869@c Excluded are files the name of which start with a @file{.} character.
870@c It is customary to start the filename with two digits
871@c that specify the order in which the directories will be scanned.
872
873@c ASDF will automatically read your configuration
874@c the first time you try to find a system.
875@c You can reset the source-registry configuration with:
876
877@c @lisp
878@c (asdf:clear-output-translations)
879@c @end lisp
880
881@c And you probably should do so before you dump your Lisp image,
882@c if the configuration may change
883@c between the machine where you save it at the time you save it
884@c and the machine you resume it at the time you resume it.
885@c (Once again, you should use @code{(asdf:clear-configuration)}
886@c before you dump your Lisp image, which includes the above.)
887
888Note that before ASDF 2,
889other ASDF add-ons offered the same functionality,
890each in subtly different and incompatible ways:
891ASDF-Binary-Locations, cl-launch, common-lisp-controller.
892ASDF-Binary-Locations is now not needed anymore and should not be used.
893cl-launch 3.000 and common-lisp-controller 7.2 have been updated
894to delegate object file placement to ASDF.
895
896@node Resetting the ASDF configuration,  , Configuring where ASDF stores object files, Configuring ASDF
897@section Resetting the ASDF configuration
898
899@c FIXME: this should probably be moved out of the "quickstart" part of
900@c the manual. [2014/02/27:rpg]
901
902
903When you dump and restore an image, or when you tweak your configuration,
904you may want to reset the ASDF configuration.
905For that you may use the following function:
906
907@defun clear-configuration
908   Undoes any ASDF configuration
909   regarding source-registry or output-translations.
910@end defun
911
912@vindex *image-dump-hook*
913This function is pushed onto the @code{uiop:*image-dump-hook*} by default,
914which means that if you save an image using @code{uiop:dump-image},
915or via @code{asdf:image-op} and @code{asdf:program-op},
916it will be automatically called to clear your configuration.
917If for some reason you prefer to call your implementation's underlying functionality,
918be sure to call @code{clear-configuration} manually,
919or push it into your implementation's equivalent of @code{uiop:*image-dump-hook*},
920e.g. @code{sb-ext:*save-hooks*} on SBCL, or @code{ext:*before-save-initializations*}
921on CMUCL and SCL, etc.
922
923@node  Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
924@chapter Using ASDF
925
926@menu
927* Loading a system::
928* Other Operations::
929* Moving on::
930@end menu
931
932@node Loading a system, Other Operations, Using ASDF, Using ASDF
933@section Loading a system
934
935The system @var{foo} is loaded (and compiled, if necessary)
936by evaluating the following Lisp form:
937
938@example
939(asdf:load-system :@var{foo})
940@end example
941
942On some implementations (namely recent versions of
943ABCL, Clozure CL, CMUCL, ECL, GNU CLISP, MKCL and SBCL),
944ASDF hooks into the @code{CL:REQUIRE} facility
945and you can just use:
946
947@example
948(require :@var{foo})
949@end example
950
951In older versions of ASDF, you needed to use
952@code{(asdf:oos 'asdf:load-op :@var{foo})}.
953If your ASDF is too old to provide @code{asdf:load-system} though
954we recommend that you upgrade to ASDF 3.
955@xref{Loading ASDF,,Loading ASDF from source}.
956
957Note the name of a system is specified as a string or a symbol.
958If a symbol (including a keyword), its name is taken and lowercased.
959The name must be a suitable value for the @code{:name} initarg
960to @code{make-pathname} in whatever filesystem the system is to be
961found.
962
963The lower-casing-symbols behaviour is unconventional,
964but was selected after some consideration.
965The type of systems we want to support
966either have lowercase as customary case (Unix, Mac, Windows)
967or silently convert lowercase to uppercase (lpns).
968@c so this makes more sense than attempting to use @code{:case :common},
969@c which is reported not to work on some implementations
970
971@node Other Operations, Moving on, Loading a system, Using ASDF
972@section Other Operations
973
974@findex load-system
975@findex compile-system
976@findex test-system
977@findex requrie-system
978
979ASDF provides three commands for the most common system operations:
980@code{load-system}, @code{compile-system}, and @code{test-system}.
981It also provides @code{require-system}, a version of @code{load-system}
982that skips trying to update systems that are already loaded.
983
984@c FIXME: We seem to export @findex bundle-system also.
985
986@findex operate
987@findex oos
988
989Because ASDF is an extensible system
990for defining @emph{operations} on @emph{components},
991it also provides a generic function @code{operate}
992(which is usually abbreviated by @code{oos},
993which stands for operate-on-system).
994You'll use @code{oos} whenever you want to do something beyond
995compiling, loading and testing.
996
997Output from ASDF and ASDF extensions are sent
998to the CL stream @code{*standard-output*},
999so rebinding that stream around calls to @code{asdf:operate}
1000should redirect all output from ASDF operations.
1001
1002@c Reminder: before ASDF can operate on a system, however,
1003@c it must be able to find and load that system's definition.
1004@c @xref{Configuring ASDF,,Configuring ASDF to find your systems}.
1005
1006@c FIXME: the following is too complicated for here, especially since
1007@c :force hasn't been defined yet.  Move it. [2014/02/27:rpg]
1008
1009@findex already-loaded-systems
1010@findex require-system
1011@findex load-system
1012@vindex *load-system-operation*
1013For advanced users, note that
1014@code{require-system} calls @code{load-system}
1015with keyword arguments @code{:force-not (already-loaded-systems)}.
1016@code{already-loaded-systems} returns a list of the names of loaded systems.
1017@code{load-system} applies @code{operate} with the operation from
1018@code{*load-system-operation*} (which by default is @code{load-op}),
1019the system, and any provided keyword arguments.
1020
1021
1022@node Moving on,  , Other Operations, Using ASDF
1023@section Moving on
1024
1025That's all you need to know to use ASDF to load systems written by others.
1026The rest of this manual deals with writing system definitions
1027for Common Lisp software you write yourself,
1028including how to extend ASDF to define new operation and component types.
1029
1030
1031@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
1032@comment  node-name,  next,  previous,  up
1033@chapter Defining systems with defsystem
1034
1035This chapter describes how to use ASDF to define systems and develop
1036software.
1037
1038
1039@menu
1040* The defsystem form::
1041* A more involved example::
1042* The defsystem grammar::
1043* Other code in .asd files::
1044* The package-inferred-system extension::
1045@end menu
1046
1047@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
1048@comment  node-name,  next,  previous,  up
1049@section The defsystem form
1050@findex defsystem
1051@cindex asdf-user
1052@findex load-asd
1053
1054This section begins with an example of a system definition,
1055then gives the full grammar of @code{defsystem}.
1056
1057Let's look at a simple system.
1058This is a complete file that should be saved as @file{hello-lisp.asd}
1059(in order that ASDF can find it
1060when ordered to operate on the system named @code{"hello-lisp"}).
1061
1062@c FIXME: the first example should have an outside dependency, e.g.,
1063@c CL-PPCRE.
1064
1065@lisp
1066(in-package :asdf-user)
1067
1068(defsystem "hello-lisp"
1069  :description "hello-lisp: a sample Lisp system."
1070  :version "0.0.1"
1071  :author "Joe User <joe@@example.com>"
1072  :licence "Public Domain"
1073  :components ((:file "packages")
1074               (:file "macros" :depends-on ("packages"))
1075               (:file "hello" :depends-on ("macros"))))
1076@end lisp
1077
1078Some notes about this example:
1079
1080@itemize
1081
1082@item
1083The file starts with an @code{in-package} form
1084for package @code{asdf-user}.  Quick summary: just do this, because it
1085helps make interactive development of @code{defsystem} forms behave in
1086the same was as when these forms are loaded by ASDF.  If that's enough
1087for you, skip the rest of this item.  Otherwise read on for the gory details.
1088
1089If your file is loaded by ASDF 3, it will be loaded into the
1090@code{asdf-user} package.  The @code{in-package} form
1091will ensure that the system definition is read the
1092same as within ASDF when you load it interactively with @code{cl:load}.
1093However, we recommend that you load @file{.asd} files
1094through function @code{asdf::load-asd} rather than through @code{cl:load},
1095in which case this form is unnecessary.
1096Recent versions of SLIME (2013-02 and later) know to do that.
1097
1098@item
1099You can always rely on symbols
1100from both package @code{asdf} and @code{common-lisp} being available in
1101@code{.asd} files --
1102most importantly including @code{defsystem}.
1103
1104@c FIXME: the following should be inserted in a more advanced
1105@c bit of the manual.  For now, it is simply elided.
1106@c Starting with ASDF 3.1,
1107@c @file{.asd} files are read in the package @code{asdf-user}
1108@c that uses @code{asdf}, @code{uiop} and @code{uiop/common-lisp}
1109@c (a variant of @code{common-lisp}
1110@c that has some portability fixes on old implementations).
1111@c ASDF 3 releases before 3.1 also read in package @code{asdf-user}
1112@c but that package don't use the full @code{uiop}, only @code{uiop/package}.
1113@c ASDF 1 and ASDF 2 releases (up until 2.26) instead read @file{.asd} files
1114@c in a temporary package @code{asdf@emph{N}}
1115@c that uses @code{asdf} and @code{common-lisp}.
1116@c You may thus have to package-qualify some symbols with @code{uiop:}
1117@c to support older variants of ASDF 3,
1118@c and/or package-qualify them with @code{asdf::}
1119@c to be compatible with even older variants of ASDF 2
1120@c (and then only use the few already available in ASDF 2).
1121
1122
1123@item
1124The @code{defsystem} form defines a system named @code{hello-lisp}
1125that contains three source files:
1126@file{packages}, @file{macros} and @file{hello}.
1127
1128@c FIXME: The first example system should probably use just :serial T.
1129@item
1130The file @file{macros} depends on @file{packages}
1131(presumably because the package it's in is defined in @file{packages}),
1132and the file @file{hello} depends on @file{macros}
1133(and hence, transitively on @file{packages}).
1134This means that ASDF will compile and load @file{packages} and @file{macros}
1135before starting the compilation of file @file{hello}.
1136
1137@item
1138System source files should be located in the same directory
1139as the @code{.asd} file with the system definition.
1140@c FIXME: the following should live somewhere, but not in the quickstart
1141@c page. [2014/05/03:rpg]
1142@c ASDF resolves symbolic links (or Windows shortcuts)
1143@c before loading the system definition file and
1144@c stores its location in the resulting system@footnote{
1145@c It is possible, though almost never necessary, to override this behaviour.}.
1146@c This is a good thing because the user can move the system sources
1147@c without having to edit the system definition.
1148
1149@c FIXME: Should have cross-reference to "Version specifiers" in the
1150@c defsystem grammar, but the cross-referencing is so broken by
1151@c insufficient node breakdown that I have not put one in.
1152@c FIXME: this is way too detailed for the first example!
1153@c move it!
1154@item
1155Make sure you know how the @code{:version} numbers will be parsed!
1156Only period-separated non-negative integers are accepted.
1157See below Version specifiers in @ref{The defsystem grammar}.
1158@cindex :version
1159
1160@end itemize
1161
1162@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
1163@comment  node-name,  next,  previous,  up
1164@section A more involved example
1165@findex defsystem
1166
1167Let's illustrate some more involved uses of @code{defsystem} via a
1168slightly convoluted example:
1169
1170@lisp
1171(in-package :asdf-user)
1172
1173(defsystem "foo"
1174  :version "1.0.0"
1175  :components ((:module "mod"
1176                :components ((:file "bar")
1177                             (:file"baz")
1178                             (:file "quux"))
1179                :perform (compile-op :after (op c)
1180                           (do-something c))
1181                :explain (compile-op :after (op c)
1182                           (explain-something c)))
1183               (:file "blah")))
1184@end lisp
1185
1186The @code{:module} component named @code{"mod"} is a collection of three files,
1187which will be located in a subdirectory of the main code directory named
1188@file{mod} (this location can be overridden; see the discussion of the
1189@code{:pathname} option in @ref{The defsystem grammar}).
1190
1191The method-form tokens provide a shorthand for defining methods on
1192particular components.  This part
1193
1194@lisp
1195                :perform (compile-op :after (op c)
1196                          (do-something c))
1197                :explain (compile-op :after (op c)
1198                          (explain-something c))
1199@end lisp
1200
1201has the effect of
1202
1203@lisp
1204(defmethod perform :after ((op compile-op) (c (eql ...)))
1205           (do-something c))
1206(defmethod explain :after ((op compile-op) (c (eql ...)))
1207           (explain-something c))
1208@end lisp
1209
1210where @code{...} is the component in question.
1211In this case @code{...} would expand to something like
1212
1213@lisp
1214(find-component "foo" "mod")
1215@end lisp
1216
1217For more details on the syntax of such forms, see @ref{The defsystem
1218grammar}.
1219For more details on what these methods do, @pxref{Operations} in
1220@ref{The object model of ASDF}.
1221
1222@c FIXME: The following plunge into detail weeds is not appropriate in this
1223@c location. [2010/10/03:rpg]
1224@c note that although this also supports @code{:before} methods,
1225@c they may not do what you want them to ---
1226@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
1227@c will run after all the dependencies and sub-components have been processed,
1228@c but before the component in question has been compiled.
1229
1230
1231@c FIXME: There should be YA example that shows definitions of functions
1232@c and classes.  The following material should go there.
1233@c @item
1234@c If in addition to simply using @code{defsystem},
1235@c you are going to define functions,
1236@c create ASDF extension, globally bind symbols, etc.,
1237@c it is recommended that to avoid namespace pollution between systems,
1238@c you should create your own package for that purpose, with:
1239
1240@c @lisp
1241@c (defpackage :hello-lisp-system
1242@c   (:use :cl :asdf))
1243
1244@c (in-package :hello-lisp-system)
1245@c @end lisp
1246
1247
1248@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
1249@comment  node-name,  next,  previous,  up
1250@section The defsystem grammar
1251@findex defsystem
1252@cindex DEFSYSTEM grammar
1253
1254@c FIXME: @var typesetting not consistently used here.  We should either expand
1255@c its use to everywhere, or we should kill it everywhere.
1256
1257
1258@example
1259system-definition := ( defsystem system-designator @var{system-option}* )
1260
1261system-option := :defsystem-depends-on system-list
1262                 | :weakly-depends-on @var{system-list}
1263                 | :class class-name (see discussion below)
1264                 | module-option
1265                 | option
1266
1267module-option := :components component-list
1268                 | :serial [ t | nil ]
1269
1270option :=
1271        | :pathname pathname-specifier
1272        | :default-component-class class-name
1273        | :perform method-form
1274        | :explain method-form
1275        | :output-files method-form
1276        | :operation-done-p method-form
1277        | :if-feature feature-expression
1278        | :depends-on ( @var{dependency-def}* )
1279        | :in-order-to ( @var{dependency}+ )
1280
1281
1282system-list := ( @var{simple-component-name}* )
1283
1284component-list := ( @var{component-def}* )
1285
1286component-def  := ( component-type simple-component-name @var{option}* )
1287
1288component-type := :module | :file | :static-file | other-component-type
1289
1290other-component-type := symbol-by-name
1291                        (@pxref{The defsystem grammar,,Component types})
1292
1293# This is used in :depends-on, as opposed to ``dependency,''
1294# which is used in :in-order-to
1295dependency-def := simple-component-name
1296               | ( :feature @var{feature-expression} dependency-def )
1297               | ( :version simple-component-name version-specifier )
1298               | ( :require module-name )
1299
1300# ``dependency'' is used in :in-order-to, as opposed to
1301# ``dependency-def''
1302dependency := (dependent-op @var{requirement}+)
1303requirement := (required-op @var{required-component}+)
1304dependent-op := operation-name
1305required-op := operation-name
1306
1307simple-component-name := string
1308                      |  symbol
1309
1310pathname-specifier := pathname | string | symbol
1311
1312method-form := (operation-name qual lambda-list @Arest{} body)
1313qual := method qualifier
1314
1315component-dep-fail-option := :fail | :try-next | :ignore
1316
1317feature-expression := keyword
1318                      | (:and @var{feature-expression}*)
1319                      | (:or @var{feature-expression}*)
1320                      | (:not @var{feature-expression})
1321@end example
1322
1323
1324@subsection Component names
1325
1326Component names (@code{simple-component-name})
1327may be either strings or symbols.
1328
1329@subsection Component types
1330
1331Component type names, even if expressed as keywords, will be looked up
1332by name in the current package and in the asdf package, if not found in
1333the current package.  So a component type @code{my-component-type}, in
1334the current package @code{my-system-asd} can be specified as
1335@code{:my-component-type}, or @code{my-component-type}.
1336
1337@code{system} and its subclasses are @emph{not}
1338allowed as component types for such children components.
1339
1340@subsection System class names
1341
1342A system class name will be looked up
1343in the same way as a Component type (see above),
1344except that only @code{system} and its subclasses are allowed.
1345Typically, one will not need to specify a system
1346class name, unless using a non-standard system class defined in some
1347ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
1348see below.  For such class names in the ASDF package, we recommend that
1349the @code{:class} option be specified using a keyword symbol, such as
1350
1351@example
1352:class :MY-NEW-SYSTEM-SUBCLASS
1353@end example
1354
1355This practice will ensure that package name conflicts are avoided.
1356Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
1357the current package @emph{before} it has been exported from the ASDF
1358extension loaded by @code{:defsystem-depends-on}, causing a name
1359conflict in the current package.
1360
1361@subsection Defsystem depends on
1362@cindex :defsystem-depends-on
1363
1364The @code{:defsystem-depends-on} option to @code{defsystem} allows the
1365programmer to specify another ASDF-defined system or set of systems that
1366must be loaded @emph{before} the system definition is processed.
1367Typically this is used to load an ASDF extension that is used in the
1368system definition.
1369
1370@subsection Weakly depends on
1371@cindex :weakly-depends-on
1372
1373We do @emph{NOT} recommend you use this feature.
1374If you are tempted to write a system @var{foo}
1375that weakly-depends-on a system @var{bar},
1376we recommend that you should instead
1377write system @var{foo} in a parametric way,
1378and offer some special variable and/or some hook to specialize its behavior;
1379then you should write a system @var{foo+bar}
1380that does the hooking of things together.
1381
1382The (deprecated) @code{:weakly-depends-on} option to @code{defsystem}
1383allows the programmer to specify another ASDF-defined system or set of systems
1384that ASDF should @emph{try} to load,
1385but need not load in order to be successful.
1386Typically this is used if there are a number of systems
1387that, if present, could provide additional functionality,
1388but which are not necessary for basic function.
1389
1390Currently, although it is specified to be an option only to @code{defsystem},
1391this option is accepted at any component, but it probably
1392only makes sense at the @code{defsystem} level.
1393Programmers are cautioned not
1394to use this component option except at the @code{defsystem} level, as
1395this anomalous behavior may be removed without warning.
1396
1397@c Finally, you might look into the @code{asdf-system-connections} extension,
1398@c that will let you define additional code to be loaded
1399@c when two systems are simultaneously loaded.
1400@c It may or may not be considered good style, but at least it can be used
1401@c in a way that has deterministic behavior independent of load order,
1402@c unlike @code{weakly-depends-on}.
1403
1404
1405@subsection Pathname specifiers
1406@cindex pathname specifiers
1407
1408A pathname specifier (@code{pathname-specifier})
1409may be a pathname, a string or a symbol.
1410When no pathname specifier is given for a component,
1411which is the usual case, the component name itself is used.
1412
1413If a string is given, which is the usual case,
1414the string will be interpreted as a Unix-style pathname
1415where @code{/} characters will be interpreted as directory separators.
1416Usually, Unix-style relative pathnames are used
1417(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1418they are relative to the path of the parent component.
1419Finally, depending on the @code{component-type},
1420the pathname may be interpreted as either a file or a directory,
1421and if it's a file,
1422a file type may be added corresponding to the @code{component-type},
1423or else it will be extracted from the string itself (if applicable).
1424
1425For instance, the @code{component-type} @code{:module}
1426wants a directory pathname, and so a string @code{"foo/bar"}
1427will be interpreted as the pathname @file{#p"foo/bar/"}.
1428On the other hand, the @code{component-type} @code{:file}
1429wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1430will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1431and a string @code{"foo/bar.quux"}
1432will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1433Finally, the @code{component-type} @code{:static-file}
1434wants a file without specifying a type, and so a string @code{"foo/bar"}
1435will be interpreted as the pathname @file{#p"foo/bar"},
1436and a string @code{"foo/bar.quux"}
1437will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1438
1439ASDF interprets the string @code{".."}
1440as the pathname directory component word @code{:back},
1441which when merged, goes back one level in the directory hierarchy.
1442
1443If a symbol is given, it will be translated into a string,
1444and downcased in the process.
1445The downcasing of symbols is unconventional,
1446but was selected after some consideration.
1447Observations suggest that the type of systems we want to support
1448either have lowercase as customary case (Unix, Mac, windows)
1449or silently convert lowercase to uppercase (lpns),
1450so this makes more sense than attempting to use @code{:case :common}
1451as argument to @code{make-pathname},
1452which is reported not to work on some implementations.
1453
1454Pathname objects may be given to override the path for a component.
1455Such objects are typically specified using reader macros such as @code{#p}
1456or @code{#.(make-pathname ...)}.
1457Note however, that @code{#p...} is
1458a shorthand for @code{#.(parse-namestring ...)}
1459and that the behavior of @code{parse-namestring} is completely non-portable,
1460unless you are using Common Lisp @code{logical-pathname}s,
1461which themselves involve other non-portable behavior
1462(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1463Pathnames made with @code{#.(make-pathname ...)}
1464can usually be done more easily with the string syntax above.
1465The only case that you really need a pathname object is to override
1466the component-type default file type for a given component.
1467Therefore, pathname objects should only rarely be used.
1468Unhappily, ASDF 1 used not to properly support
1469parsing component names as strings specifying paths with directories,
1470and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1471An alternative to @code{#.} read-time evaluation is to use
1472@code{(eval `(defsystem ... ,pathname ...))}.
1473
1474Note that when specifying pathname objects,
1475ASDF does not do any special interpretation of the pathname
1476influenced by the component type, unlike the procedure for
1477pathname-specifying strings.
1478On the one hand, you have to be careful to provide a pathname that correctly
1479fulfills whatever constraints are required from that component type
1480(e.g. naming a directory or a file with appropriate type);
1481on the other hand, you can circumvent the file type that would otherwise
1482be forced upon you if you were specifying a string.
1483
1484@subsection Version specifiers
1485@cindex version specifiers
1486@cindex :version
1487
1488Version specifiers are strings to be parsed as period-separated lists of integers.
1489I.e., in the example, @code{"0.2.1"} is to be interpreted,
1490roughly speaking, as @code{(0 2 1)}.
1491In particular, version @code{"0.2.1"} is interpreted the same as @code{"0.0002.1"},
1492though the latter is not canonical and may lead to a warning being issued.
1493Also, @code{"1.3"} and @code{"1.4"} are both strictly @code{uiop:version<} to @code{"1.30"},
1494quite unlike what would have happened
1495had the version strings been interpreted as decimal fractions.
1496
1497Instead of a string representing the version,
1498the @code{:version} argument can be an expression that is resolved to
1499such a string using the following trivial domain-specific language:
1500in addition to being a literal string, it can be an expression of the form
1501@code{(:read-file-form <pathname-or-string> :at <access-at-specifier>)},
1502which will be resolved by reading a form in the specified pathname
1503(read as a subpathname of the current system if relative or a unix-namestring).
1504You may use a @code{uiop:access-at} specifier
1505with the (optional) @code{:at} keyword,
1506by default the specifier is @code{0}, meaning the first form is returned;
1507subforms can also be specified, with e.g. @code{(1 2 2)} specifying
1508``the third subform (index 2) of the third subform (index 2) of the second form (index 1)''
1509in the file (mind the off-by-one error in the English language).
1510
1511System definers are encouraged to use version identifiers of the form
1512@var{x}.@var{y}.@var{z} for
1513major version, minor version and patch level,
1514where significant API incompatibilities are signaled by an increased major number.
1515
1516@xref{Common attributes of components}.
1517
1518@subsection Require
1519@cindex :require dependencies
1520
1521Use the implementation's own @code{require} to load the @var{module-name}.
1522
1523
1524@subsection Using logical pathnames
1525@cindex logical pathnames
1526
1527We do not generally recommend the use of logical pathnames,
1528especially not so to newcomers to Common Lisp.
1529However, we do support the use of logical pathnames by old timers,
1530when such is their preference.
1531
1532To use logical pathnames,
1533you will have to provide a pathname object as a @code{:pathname} specifier
1534to components that use it, using such syntax as
1535@code{#p"LOGICAL-HOST:absolute;path;to;component.lisp"}.
1536
1537You only have to specify such logical pathname
1538for your system or some top-level component.
1539Sub-components' relative pathnames,
1540specified using the string syntax for names,
1541will be properly merged with the pathnames of their parents.
1542The specification of a logical pathname host however is @emph{not}
1543otherwise directly supported in the ASDF syntax
1544for pathname specifiers as strings.
1545
1546The @code{asdf-output-translation} layer will
1547avoid trying to resolve and translate logical pathnames.
1548The advantage of this is that
1549you can define yourself what translations you want to use
1550with the logical pathname facility.
1551The disadvantage is that if you do not define such translations,
1552any system that uses logical pathnames will behave differently under
1553asdf-output-translations than other systems you use.
1554
1555If you wish to use logical pathnames you will have to configure the
1556translations yourself before they may be used.
1557ASDF currently provides no specific support
1558for defining logical pathname translations.
1559
1560Note that the reasons we do not recommend logical pathnames are that
1561(1) there is no portable way to set up logical pathnames before they are used,
1562(2) logical pathnames are limited to only portably use
1563a single character case, digits and hyphens.
1564While you can solve the first issue on your own,
1565describing how to do it on each of fifteen implementations supported by ASDF
1566is more than we can document.
1567As for the second issue, mind that the limitation is notably enforced on SBCL,
1568and that you therefore can't portably violate the limitations
1569but must instead define some encoding of your own and add individual mappings
1570to name physical pathnames that do not fit the restrictions.
1571This can notably be a problem when your Lisp files are part of a larger project
1572in which it is common to name files or directories in a way that
1573includes the version numbers of supported protocols,
1574or in which files are shared with software written
1575in different programming languages where conventions include the use of
1576underscores, dots or CamelCase in pathnames.
1577
1578
1579@subsection Serial dependencies
1580@cindex serial dependencies
1581
1582If the @code{:serial t} option is specified for a module,
1583ASDF will add dependencies for each child component,
1584on all the children textually preceding it.
1585This is done as if by @code{:depends-on}.
1586
1587@lisp
1588:serial t
1589:components ((:file "a") (:file "b") (:file "c"))
1590@end lisp
1591
1592is equivalent to
1593
1594@lisp
1595:components ((:file "a")
1596             (:file "b" :depends-on ("a"))
1597             (:file "c" :depends-on ("a" "b")))
1598@end lisp
1599
1600
1601@subsection Source location (@code{:pathname})
1602
1603The @code{:pathname} option is optional in all cases for systems
1604defined via @code{defsystem}, and generally is unnecessary.  In the
1605simple case, source files will be found in the same directory as the
1606system or, in the case of modules, in a subdirectory with the same name
1607as the module.
1608
1609@c FIXME: This should be moved elsewhere -- it's too much detail for the
1610@c grammar section.
1611
1612More specifically, ASDF follows a hairy set of rules that are designed so that
1613@enumerate
1614@item
1615@code{find-system}
1616will load a system from disk
1617and have its pathname default to the right place.
1618
1619@item
1620This pathname information will not be overwritten with
1621@code{*default-pathname-defaults*}
1622(which could be somewhere else altogether)
1623if the user loads up the @file{.asd} file into his editor
1624and interactively re-evaluates that form.
1625@end enumerate
1626
1627If a system is being loaded for the first time,
1628its top-level pathname will be set to:
1629
1630@itemize
1631@item
1632The host/device/directory parts of @code{*load-truename*},
1633if it is bound.
1634@item
1635@code{*default-pathname-defaults*}, otherwise.
1636@end itemize
1637
1638If a system is being redefined, the top-level pathname will be
1639
1640@itemize
1641@item
1642changed, if explicitly supplied or obtained from @code{*load-truename*}
1643(so that an updated source location is reflected in the system definition)
1644
1645@item
1646changed if it had previously been set from @code{*default-pathname-defaults*}
1647
1648@item
1649left as before, if it had previously been set from @code{*load-truename*}
1650and @code{*load-truename*} is currently unbound
1651(so that a developer can evaluate a @code{defsystem} form
1652from within an editor without clobbering its source location)
1653@end itemize
1654
1655@subsection if-feature option
1656@cindex :if-feature component option
1657@anchor{if-feature-option}      @c redo if this ever becomes a node in
1658@c its own right...
1659
1660This option allows you to specify a feature expression to be evaluated
1661as if by @code{#+} to conditionally include a component in your build.
1662If the expression is false, the component is dropped
1663as well as any dependency pointing to it.
1664As compared to using @code{#+} which is expanded at read-time,
1665this allows you to have an object in your component hierarchy
1666that can be used for manipulations beside building your project, and
1667that is accessible to outside code that wishes to reason about system
1668structure.
1669
1670Programmers should be careful to consider @strong{when} the
1671@code{:if-feature} is evaluated.  Recall that ASDF first computes a
1672build plan, and then executes that plan.  ASDF will check to see whether
1673or not a feature is present @strong{at planning time}, not during the
1674build.  It follows that one cannot use @code{:if-feature} to check
1675features that are set during the course of the build.  It can only be
1676used to check the state of features before any build operations have
1677been performed.
1678
1679This option was added in ASDF 3.  For more information,
1680@xref{required-features, Required features}.
1681
1682@subsection if-component-dep-fails option
1683@cindex :if-component-dep-fails component option
1684This option was removed in ASDF 3.
1685Its semantics was limited in purpose and dubious to explain,
1686and its implementation was breaking a hole into the ASDF object model.
1687Please use the @code{if-feature} option instead.
1688
1689@subsection feature requirement
1690This requirement was removed in ASDF 3.1.  Please do not use it.  In
1691most cases, @code{:if-feature} (@pxref{if-feature-option}) will provide
1692an adequate substitute.
1693
1694The @code{feature} requirement used to ensure that a chain of component
1695dependencies would fail when a key feature was absent.
1696Used in conjunction with @code{:if-component-dep-fails}
1697this provided
1698a roundabout way to express conditional compilation.
1699
1700
1701@node Other code in .asd files, The package-inferred-system extension, The defsystem grammar, Defining systems with defsystem
1702@section Other code in .asd files
1703
1704Files containing @code{defsystem} forms
1705are regular Lisp files that are executed by @code{load}.
1706Consequently, you can put whatever Lisp code you like into these files.
1707However, it is recommended to keep such forms to a minimal,
1708and to instead define @code{defsystem} extensions
1709that you use with @code{:defsystem-depends-on}.
1710
1711If however, you might insist on including code in the @file{.asd} file itself,
1712e.g., to examine and adjust the compile-time environment,
1713possibly adding appropriate features to @code{*features*}.
1714If so, here are some conventions we recommend you follow,
1715so that users can control certain details of execution
1716of the Lisp in @file{.asd} files:
1717
1718@itemize
1719@item
1720Any informative output
1721(other than warnings and errors,
1722which are the condition system's to dispose of)
1723should be sent to the standard CL stream @code{*standard-output*},
1724so that users can easily control the disposition
1725of output from ASDF operations.
1726@end itemize
1727
1728
1729@node The package-inferred-system extension,  , Other code in .asd files, Defining systems with defsystem
1730@section The package-inferred-system extension
1731
1732Starting with release 3.1.2,
1733ASDF supports a one-package-per-file style of programming,
1734whereby each file is its own system,
1735and dependencies are deduced from the @code{defpackage} form
1736(or its variant @code{uiop:define-package}).
1737
1738
1739In this style, packages refer to a system with the same name (downcased);
1740and if a system is defined with @code{:class package-inferred-system},
1741then system names that start with that name
1742(using the slash @code{/} separator)
1743refer to files under the filesystem hierarchy where the system is defined.
1744For instance, if system @code{my-lib} is defined in
1745@file{/foo/bar/my-lib/my-lib.asd}, then system @code{my-lib/src/utility}
1746will be found in file @file{/foo/bar/my-lib/src/utility.lisp}.
1747
1748This style was made popular by @code{faslpath} and @code{quick-build} before,
1749and at the cost of a stricter package discipline,
1750seems to make for more maintainable code.
1751It is used by ASDF itself (starting with ASDF 3) and by @code{lisp-interface-library}.
1752
1753To use this style, choose a toplevel system name, e.g. @code{my-lib},
1754and create a file @file{my-lib.asd}
1755with the @code{:class :package-inferred-system} option in its @code{defsystem}.
1756For instance:
1757@example
1758#-asdf (error "my-lib requires ASDF 3")
1759(defsystem my-lib
1760  :class :package-inferred-system
1761  :defsystem-depends-on (:asdf-package-system)
1762  :depends-on (:lil/interface/all
1763               :lil/pure/all
1764               :lil/stateful/all
1765               :lil/transform/all)
1766  :in-order-to ((test-op (load-op :lil/test/all)))
1767  :perform (test-op (o c) (symbol-call :lil/test/all :test-suite)))
1768
1769(defsystem :lil/test :depends-on (:lil/test/all))
1770
1771(register-system-packages :lil/interface/all '(:interface))
1772(register-system-packages :lil/pure/all '(:pure))
1773(register-system-packages :lil/stateful/all '(:stateful))
1774(register-system-packages :lil/transform/classy '(:classy))
1775(register-system-packages :lil/transform/posh '(:posh))
1776(register-system-packages :lil/test/all '(:lil/test))
1777
1778(register-system-packages
1779 :closer-mop
1780 '(:c2mop :closer-common-lisp :c2cl :closer-common-lisp-user :c2cl-user))
1781@end example
1782
1783In the code above, the
1784@code{:defsystem-depends-on (:asdf-package-system)} is
1785for compatibility with older versions of ASDF 3 (ASDF 2 is not supported),
1786and requires the @code{asdf-package-system} library to be present
1787(it is implicitly provided by ASDF starting with release 3.1.2,
1788which can be detected with the feature @code{:asdf3.1}).
1789
1790The function @code{register-system-packages} has to be called to register
1791packages used or provided by your system and its components
1792where the name of the system that provides the package
1793is not the downcase of the package name.
1794
1795Then, file @file{interface/order.lisp} under the @code{lil} hierarchy,
1796that defines abstract interfaces for order comparisons,
1797starts with the following form,
1798dependencies being trivially computed from the @code{:use} and @code{:mix} clauses:
1799
1800@example
1801(uiop:define-package :lil/interface/order
1802  (:use :closer-common-lisp
1803   :lil/interface/definition
1804   :lil/interface/base
1805   :lil/interface/eq :lil/interface/group)
1806  (:mix :fare-utils :uiop :alexandria)
1807  (:export ...))
1808@end example
1809
1810ASDF can tell that this file depends on system @code{closer-mop} (registered above),
1811@code{lil/interface/definition}, @code{lil/interface/base},
1812@code{lil/interface/eq}, and @code{lil/interface/group}
1813(package and system names match, and they will be looked up hierarchically).
1814
1815ASDF also detects dependencies from @code{:import-from} clauses.
1816To depend on a system without using a package or importing any symbol from it
1817(because you'll fully qualify them when used),
1818you may thus use an @code{:import-from} clause with an empty list of symbols, as in:
1819
1820@example
1821(defpackage :foo/bar
1822  (:use :cl)
1823  (:import-from :foo/baz #:sym1 #:sym2)
1824  (:import-from :foo/quux)
1825  (:export ...))
1826@end example
1827
1828The form @code{uiop:define-package} is supported as well as @code{defpackage},
1829and has many options that prove useful in this context,
1830such as @code{:use-reexport} and @code{:mix-reexport}
1831that allow for ``inheritance'' of symbols being exported.
1832
1833@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1834@comment  node-name,  next,  previous,  up
1835@chapter The Object model of ASDF
1836@tindex component
1837@tindex operation
1838
1839ASDF is designed in an object-oriented way from the ground up.
1840Both a system's structure and the operations that can be performed on systems
1841follow a extensible protocol, allowing programmers to add new behaviors to ASDF.
1842For example, @code{cffi} adds support for special FFI description files
1843that interface with C libraries and for wrapper files that embed C code in Lisp.
1844@code{abcl-jar} supports creating Java JAR archives in ABCL.
1845@code{poiu} supports compiling code in parallel using background processes.
1846
1847The key classes in ASDF are @code{component} and @code{operation}.
1848A @code{component} represents an individual source file or a group of source files,
1849and the products (e.g., fasl files) produced from it.
1850An @code{operation} represents a transformation that can be performed on a component,
1851turning them from source files to intermediate results to final outputs.
1852Components are related by @emph{dependencies}, specified in system
1853definitions.
1854
1855When ordered to @code{operate} with some operation on a component (usually a system),
1856ASDF will first compute a @emph{plan}
1857by traversing the dependency graph using function @code{make-plan}.@footnote{
1858  Historically, the function that built a plan was
1859  called @code{traverse}, and returned a list of actions;
1860  it was deprecated in favor of @code{make-plan} (that returns a plan object)
1861  when the @code{plan} objects were introduced;
1862  the old function is kept for backward compatibility and debugging purposes only.
1863}
1864The resulting plan object contains an ordered list of @emph{actions}.
1865An action is a pair of an @code{operation} and a @code{component},
1866representing  a particular build step to be @code{perform}ed.
1867The ordering of the plan ensures that no action is performed before
1868all its dependencies have been fulfilled.@footnote{
1869  The term @emph{action}
1870  was used by Kent Pitman in his article, ``The Description of Large Systems,''
1871  (@pxref{Bibliography}).
1872  Although the term was only used by ASDF hackers starting with ASDF 2,
1873  the concept was there since the very beginning of ASDF 1,
1874  just not clearly articulated.
1875}
1876
1877In this chapter, we describe ASDF's object-oriented protocol,
1878the classes that make it up, and the generic functions on those classes.
1879These generic functions often take
1880both an operation and a component as arguments:
1881much of the power and configurability of ASDF is provided by
1882this use of CLOS's multiple dispatch.
1883We will describe the built-in component and operation classes, and
1884explain how to extend the ASDF protocol by defining new classes and
1885methods for ASDF's generic functions.
1886We will also describe the many @emph{hooks} that can be configured to
1887customize the behavior of existing @emph{functions}.
1888
1889@c FIXME: Swap operations and components.
1890@c FIXME: Possibly add a description of the PLAN object.
1891@c Not critical, since the user isn't expected to interact with it.
1892@menu
1893* Operations::
1894* Components::
1895* Dependencies::
1896* Functions::
1897@end menu
1898
1899@node  Operations, Components, The object model of ASDF, The object model of ASDF
1900@comment  node-name,  next,  previous,  up
1901@section Operations
1902@cindex operation
1903
1904An @dfn{operation} object of the appropriate type is instantiated
1905whenever the user wants to do something with a system like
1906
1907@itemize
1908@item compile all its files
1909@item load the files into a running lisp environment
1910@item copy its source files somewhere else
1911@end itemize
1912
1913Operations can be invoked directly, or examined
1914to see what their effects would be without performing them.
1915There are a bunch of methods specialised on operation and component type
1916that actually do the grunt work.
1917Operations are invoked on systems via @code{operate} (@pxref{operate}).
1918
1919ASDF contains a number of pre-defined @t{operation} classes for common,
1920and even fairly uncommon tasks that you might want to do with it.
1921In addition, ASDF contains ``abstract'' @t{operation} classes that
1922programmers can use as building blocks to define ASDF extensions.  We
1923discuss these in turn below.
1924
1925@c The operation object contains whatever state is relevant for this purpose
1926@c (perhaps a list of visited nodes, for example)
1927@c but primarily is a nice thing to specialise operation methods on
1928@c and easier than having them all be @code{EQL} methods.
1929
1930@menu
1931* Predefined operations of ASDF::
1932* Creating new operations::
1933@end menu
1934
1935Operations are invoked on systems via @code{operate}.
1936@anchor{operate}
1937@deffn {Generic function} @code{operate} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @code{force} @code{force-not} @code{verbose} @AallowOtherKeys
1938@deffnx {Generic function} @code{oos} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @AallowOtherKeys{}
1939@code{operate} invokes @var{operation} on @var{system}.
1940@code{oos} is a synonym for @code{operate} (it stands for operate-on-system).
1941
1942@var{operation} is a symbol that is passed,
1943along with the supplied @var{initargs},
1944to @code{make-operation} (which will call @code{make-instance})
1945to create the operation object.
1946@var{component} is a component designator,
1947usually a string or symbol that designates a system,
1948sometimes a list of strings or symbols that designate a subcomponent of a system.
1949
1950The @var{initargs} are passed to the @code{make-instance} call
1951when creating the operation object.
1952@c We probably want to deprecate that, because
1953@c (1) there is a mix of flags for operate, for the operation-class, for the plan-class, etc.
1954@c (2) flags to operations have never been well-supported, anyway.
1955@c The future solution probably involves having an explicit :operation-options keyword or some such
1956@c (if operation options are not wholly eliminated), a separate :plan-options, etc.
1957Note that dependencies may cause the operation
1958to invoke other operations on the system or its components:
1959the new operations will be created
1960with the same @var{initargs} as the original one.
1961
1962If @var{force} is @code{:all}, then all systems
1963are forced to be recompiled even if not modified since last compilation.
1964If @var{force} is @code{t}, then only the system being loaded
1965is forced to be recompiled even if not modified since last compilation,
1966but other systems are not affected.
1967If @var{force} is a list, then it specifies a list of systems that
1968are forced to be recompiled even if not modified since last compilation.
1969If @var{force-not} is @code{:all}, then all systems
1970are forced not to be recompiled even if modified since last compilation.
1971If @var{force-not} is @code{t}, then all systems but the system being loaded
1972are forced not to be recompiled even if modified since last compilation
1973(note: this was changed in ASDF 3.1.2).
1974If @var{force-not} is a list, then it specifies a list of systems that
1975are forced not to be recompiled even if modified since last compilation.
1976
1977Both @var{force} and @var{force-not} apply to systems that are dependencies and were already compiled.
1978@var{force-not} takes precedences over @var{force},
1979as it should, really, but unhappily only since ASDF 3.1.2.
1980Moreover, systems the name of which is member of the set @var{*immutable-systems*}
1981(represented as an equal hash-table) are always considered @var{forced-not}, and
1982even their @file{.asd} is not refreshed from the filesystem.
1983
1984To see what @code{operate} would do, you can use:
1985@example
1986(asdf:traverse operation-class system-name)
1987@end example
1988
1989@end deffn
1990
1991
1992
1993@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1994@comment  node-name,  next,  previous,  up
1995@subsection Predefined operations of ASDF
1996@c FIXME: All these deffn's should be replaced with deftyp.  Also, we
1997@c should set up an appropriate index.
1998
1999All the operations described in this section are in the @code{asdf} package.
2000They are invoked via the @code{operate} generic function.
2001
2002@lisp
2003(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
2004@end lisp
2005
2006@deffn Operation @code{compile-op}
2007
2008This operation compiles the specified component.
2009A @code{cl-source-file} will be @code{compile-file}'d.
2010All the children and dependencies of a system or module
2011will be recursively compiled by @code{compile-op}.
2012
2013@code{compile-op} depends on @code{prepare-op} which
2014itself depends on a @code{load-op} of all of a component's dependencies,
2015as well as of its parent's dependencies.
2016When @code{operate} is called on @code{compile-op},
2017all these dependencies will be loaded as well as compiled;
2018yet, some parts of the system main remain unloaded,
2019because nothing depends on them.
2020Use @code{load-op} to load a system.
2021@end deffn
2022
2023@deffn Operation @code{load-op}
2024
2025This operation loads the compiled code for a specified component.
2026A @code{cl-source-file} will have its compiled fasl @code{load}ed,
2027which fasl is the output of @code{compile-op} that @code{load-op} depends on.
2028
2029@code{load-op} will recursively load all the children of a system or module.
2030
2031@code{load-op} also depends on @code{prepare-op} which
2032itself depends on a @code{load-op} of all of a component's dependencies,
2033as well as of its parent's dependencies.
2034@end deffn
2035
2036@deffn Operation @code{prepare-op}
2037
2038This operation ensures that the dependencies of a component
2039and its recursive parents are loaded (as per @code{load-op}),
2040as a prerequisite before @code{compile-op} and @code{load-op} operations
2041may be performed on a given component.
2042@end deffn
2043
2044@deffn Operation @code{load-source-op}, @code{prepare-source-op}
2045
2046@code{load-source-op} will load the source for the files in a module
2047rather than the compiled fasl output.
2048It has a @code{prepare-source-op} analog to @code{prepare-op},
2049that ensures the dependencies are themselves loaded via @code{load-source-op}.
2050
2051@end deffn
2052
2053@anchor{test-op}
2054@deffn Operation @code{test-op}
2055
2056This operation will perform some tests on the module.
2057The default method will do nothing.
2058The default dependency is to require
2059@code{load-op} to be performed on the module first.
2060Its @code{operation-done-p} method returns @code{nil},
2061which means that the operation is @emph{never} done
2062--
2063we assume that if you invoke the @code{test-op},
2064you want to test the system, even if you have already done so.
2065
2066The results of this operation are not defined by ASDF.
2067It has proven difficult to define how the test operation
2068should signal its results to the user
2069in a way that is compatible with all of the various test libraries
2070and test techniques in use in the community, and
2071given the fact that ASDF operations do not return a value indicating
2072success or failure.
2073For those willing to go to the effort, we suggest defining conditions to
2074signal when a @code{test-op} fails, and storing in those conditions
2075information that describes which tests fail.
2076
2077People typically define a separate test @emph{system} to hold the tests.
2078Doing this avoids unnecessarily adding a test framework as a dependency
2079on a library.  For example, one might have
2080@lisp
2081(defsystem foo
2082  :in-order-to ((test-op (test-op "foo/test")))
2083 ...)
2084
2085(defsystem foo/test
2086  :depends-on (foo fiveam) ; fiveam is a test framework library
2087  ...)
2088@end lisp
2089
2090Then one defines @code{perform} methods on
2091@code{test-op} such as the following:
2092@lisp
2093(defsystem foo/test
2094  :depends-on (foo fiveam) ; fiveam is a test framework library
2095  :perform (test-op (o s)
2096                    (uiop:symbol-call :fiveam  '#:run!
2097                       (uiop:find-symbol* '#:foo-test-suite
2098                                            :foo-tests)))
2099  ...)
2100@end lisp
2101
2102@end deffn
2103
2104
2105
2106@deffn Operation @code{compile-bundle-op}, @code{monolithic-compile-bundle-op}, @code{load-bundle-op}, @code{monolithic-load-bundle-op}, @code{deliver-asd-op}, @code{monolithic-deliver-asd-op}, @code{lib-op}, @code{monolithic-lib-op}, @code{dll-op}, @code{monolithic-dll-op}, @code{image-op}, @code{program-op}
2107
2108These are ``bundle'' operations, that can create a single-file ``bundle''
2109for all the contents of each system in an application,
2110or for the entire application.
2111
2112@code{compile-bundle-op} will create a single fasl file for each of the systems needed,
2113grouping all its many fasls in one,
2114so you can deliver each system as a single fasl
2115@code{monolithic-compile-bundle-op} will create a single fasl file for the target system
2116and all its dependencies,
2117so you can deliver your entire application as a single fasl.
2118@code{load-bundle-op} will load the output of @code{compile-bundle-op}.
2119Note that if it the output is not up-to-date,
2120@code{compile-bundle-op} may load the intermediate fasls as a side-effect.
2121Bundling fasls together matters a lot on ECL,
2122where the dynamic linking involved in loading tens of individual fasls
2123can be noticeably more expensive than loading a single one.
2124
2125NB: @code{compile-bundle-op}, @code{monolithic-compile-bundle-op}, @code{load-bundle-op}, @code{monolithic-load-bundle-op}, @code{deliver-asd-op}, @code{monolithic-deliver-asd-op} were respectively called
2126@code{fasl-op}, @code{monolithic-fasl-op}, @code{load-fasl-op}, @code{monolithic-load-fasl-op}, @code{binary-op}, @code{monolithic-binary-op} before ASDF 3.1.
2127The old names still exist for backward compatibility,
2128though they poorly label what is going on.
2129
2130Once you have created a fasl with @code{compile-bundle-op},
2131you can use @code{precompiled-system} to deliver it in a way
2132that is compatible with clients having dependencies on your system,
2133whether it is distributed as source or as a single binary;
2134the @file{.asd} file to be delivered with the fasl will look like this:
2135@example
2136(defsystem :mysystem :class :precompiled-system
2137  :fasl (some expression that will evaluate to a pathname))
2138@end example
2139Or you can use @code{deliver-asd-op} to let ASDF create such a system for you
2140as well as the @code{compile-bundle-op} output,
2141or @code{monolithic-deliver-asd-op}.
2142This allows you to deliver code for your systems or applications
2143as a single file.
2144Of course, if you want to test the result in the current image,
2145@emph{before} you try to use any newly created @file{.asd} files,
2146you should not forget to @code{(asdf:clear-configuration)}
2147or at least @code{(asdf:clear-source-registry)},
2148so it re-populates the source-registry from the filesystem.
2149
2150The @code{program-op} operation will create an executable program
2151from the specified system and its dependencies.
2152You can use UIOP for its pre-image-dump hooks, its post-image-restore hooks,
2153and its access to command-line arguments.
2154And you can specify an entry point @code{my-app:main}
2155by specifying in your @code{defsystem}
2156the option @code{:entry-point "my-app:main"}.
2157Depending on your implementation,
2158running @code{(asdf:operate 'asdf:program-op :my-app)}
2159may quit the current Lisp image upon completion.
2160See the example in
2161@file{test/hello-world-example.asd} and @file{test/hello.lisp},
2162as built and tested by
2163@file{test/test-program.script} and @file{test/make-hello-world.lisp}.
2164@code{image-op} will dump an image that may not be standalone
2165and does not start its own function,
2166but follows the usual execution convention of the underlying Lisp,
2167just with more code pre-loaded,
2168for use as an intermediate build result or with a wrapper invocation script.
2169
2170There is also @code{lib-op}
2171for building a linkable @file{.a} file (Windows: @file{.lib})
2172from all linkable object dependencies (FFI files, and on ECL, Lisp files too),
2173and its monolithic equivalent @code{monolithic-lib-op}.
2174And there is also @code{dll-op}
2175(respectively its monolithic equivalent @code{monolithic-lib-op})
2176for building a linkable @file{.so} file
2177(Windows: @file{.dll}, MacOS X: @file{.dynlib})
2178to create a single dynamic library
2179for all the extra FFI code to be linked into each of your systems
2180(respectively your entire application).
2181
2182All these ``bundle'' operations are available since ASDF 3
2183on all actively supported Lisp implementations,
2184but may be unavailable on unmaintained legacy implementations.
2185This functionality was previously available for select implementations,
2186as part of a separate system @code{asdf-bundle},
2187itself descended from the ECL-only @code{asdf-ecl}.
2188
2189The pathname of the output of bundle operations
2190is subject to output-translation as usual,
2191unless the operation is equal to
2192the @code{:build-operation} argument to @code{defsystem}.
2193This behavior is not very satisfactory and may change in the future.
2194Maybe you have suggestions on how to better configure it?
2195@end deffn
2196
2197@deffn Operation @code{concatenate-source-op}, @code{monolithic-concatenate-source-op}, @code{load-concatenated-source-op}, @code{compile-concatenated-source-op}, @code{load-compiled-concatenated-source-op}, @code{monolithic-load-concatenated-source-op}, @code{monolithic-compile-concatenated-source-op}, @code{monolithic-load-compiled-concatenated-source-op}
2198
2199These operations, as their respective names indicate,
2200will concatenate all the @code{cl-source-file} source files in a system
2201(or in a system and all its dependencies, if monolithic),
2202in the order defined by dependencies,
2203then load the result, or compile and then load the result.
2204
2205These operations are useful to deliver a system or application
2206as a single source file,
2207and for testing that said file loads properly, or compiles and then loads properly.
2208
2209ASDF itself is delivered as a single source file this way,
2210using @code{monolithic-concatenate-source-op},
2211prepending a prelude and the @code{uiop} library
2212before the @code{asdf/defsystem} system itself.
2213@end deffn
2214
2215
2216@node  Creating new operations,  , Predefined operations of ASDF, Operations
2217@comment  node-name,  next,  previous,  up
2218@subsection Creating new operations
2219
2220ASDF was designed to be extensible in an object-oriented fashion.
2221To teach ASDF new tricks, a programmer can implement the behaviour he wants
2222by creating a subclass of @code{operation}.
2223
2224ASDF's pre-defined operations are in no way ``privileged'',
2225but it is requested that developers never use the @code{asdf} package
2226for operations they develop themselves.
2227The rationale for this rule is that we don't want to establish a
2228``global asdf operation name registry'',
2229but also want to avoid name clashes.
2230
2231Your operation @emph{must} usually provide methods
2232for one or more of the following generic functions:
2233
2234@itemize
2235
2236@findex perform
2237@item @code{perform}
2238Unless your operation, like @code{prepare-op},
2239is for dependency propagation only,
2240the most important function for which to define a method
2241is usually @code{perform},
2242which will be called to perform the operation on a specified component,
2243after all dependencies have been performed.
2244
2245The @code{perform} method must call @code{input-files} and @code{output-files} (see below)
2246to locate its inputs and outputs,
2247because the user is allowed to override the method
2248or tweak the output-translation mechanism.
2249Perform should only use the primary value returned by @code{output-files}.
2250If one and only one output file is expected,
2251it can call @code{output-file} that checks that this is the case
2252and returns the first and only list element.
2253
2254@findex output-files
2255@item @code{output-files}
2256If your perform method has any output,
2257you must define a method for this function.
2258for ASDF to determine where the outputs of performing operation lie.
2259
2260Your method may return two values, a list of pathnames, and a boolean.
2261If the boolean is @code{nil} (or you fail to return multiple values),
2262then enclosing @code{:around} methods may translate these pathnames,
2263e.g. to ensure object files are somehow stored
2264in some implementation-dependent cache.
2265If the boolean is @code{t} then the pathnames are marked
2266not be translated by the enclosing @code{:around} method.
2267
2268@findex component-depends-on
2269@item @code{component-depends-on}
2270If the action of performing the operation on a component has dependencies,
2271you must define a method on @code{component-depends-on}.
2272
2273Your method will take as specialized arguments
2274an operation and a component which together identify an action,
2275and return a list of entries describing actions that this action depends on.
2276The format of entries is described below.
2277
2278It is @emph{strongly} advised that
2279you should always append the results of @code{(call-next-method)}
2280to the results of your method,
2281or ``interesting'' failures will likely occur,
2282unless you're a true specialist of ASDF internals.
2283It is unhappily too late to compatibly use the @code{append} method combination,
2284but conceptually that's the protocol that is being manually implemented.
2285
2286Each entry returned by @code{component-depends-on} is itself a list.
2287
2288The first element of an entry is an operation designator:
2289either an operation object designating itself, or
2290a symbol that names an operation class
2291(that ASDF will instantiate using @code{make-operation}).
2292For instance, @code{load-op}, @code{compile-op} and @code{prepare-op}
2293are common such names, denoting the respective operations.
2294
2295@c FIXME COERCE-NAME is referenced, but not defined.
2296@findex coerce-name
2297@findex find-component
2298The rest of each entry is a list of component designators:
2299either a component object designating itself,
2300or an identifier to be used with @code{find-component}.
2301@code{find-component} will be called with the current component's parent as parent,
2302and the identifier as second argument.
2303The identifier is typically a string,
2304a symbol (to be downcased as per @code{coerce-name}),
2305or a list of strings or symbols.
2306In particular, the empty list @code{nil} denotes the parent itself.
2307
2308@end itemize
2309
2310An operation @emph{may} provide methods for the following generic functions:
2311
2312@itemize
2313
2314@item @code{input-files}
2315@findex input-files
2316A method for this function is often not needed,
2317since ASDF has a pretty clever default @code{input-files} mechanism.
2318You only need create a method if there are multiple ultimate input files,
2319and/or the bottom one doesn't depend
2320on the @code{component-pathname} of the component.
2321
2322@item @code{operation-done-p}
2323@findex operation-done-p
2324You only need to define a method on that function
2325if you can detect conditions that invalidate previous runs of the operation,
2326even though no filesystem timestamp has changed,
2327in which case you return @code{nil} (the default is @code{t}).
2328
2329For instance, the method for @code{test-op} always returns @code{nil},
2330so that tests are always run afresh.
2331Of course, the @code{test-op} for your system could depend
2332on a deterministically repeatable @code{test-report-op},
2333and just read the results from the report files,
2334in which case you could have this method return @code{t}.
2335
2336@end itemize
2337
2338Operations that print output should send that output to the standard
2339CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
2340
2341@node Components, Dependencies, Operations, The object model of ASDF
2342@comment  node-name,  next,  previous,  up
2343@section Components
2344@cindex component
2345@cindex system
2346@cindex system designator
2347@cindex component designator
2348@vindex *system-definition-search-functions*
2349
2350A @code{component} represents an individual source file or a group of source files,
2351and the things that get transformed into.
2352A @code{system} is a component at the top level of the component hierarchy,
2353that can be found via @code{find-system}.
2354A @code{source-file} is a component representing a single source-file
2355and the successive output files into which it is transformed.
2356A @code{module} is an intermediate component itself grouping several other components,
2357themselves source-files or further modules.
2358
2359A @dfn{system designator} is a system itself,
2360or a string or symbol that behaves just like any other component name
2361(including with regard to the case conversion rules for component names).
2362
2363A @dfn{component designator}, relative to a base component,
2364is either a component itself,
2365or a string or symbol,
2366or a list of designators.
2367
2368@defun find-system system-designator @Aoptional{} (error-p t)
2369
2370Given a system designator, @code{find-system} finds and returns a system.
2371If no system is found, an error of type
2372@code{missing-component} is thrown,
2373or @code{nil} is returned if @code{error-p} is false.
2374
2375To find and update systems, @code{find-system} funcalls each element
2376in the @code{*system-definition-search-functions*} list,
2377expecting a pathname to be returned, or a system object,
2378from which a pathname may be extracted, and that will be registered.
2379The resulting pathname (if any) is loaded
2380if one of the following conditions is true:
2381
2382@itemize
2383@item
2384there is no system of that name in memory
2385@item
2386the pathname is different from that which was previously loaded
2387@item
2388the file's @code{last-modified} time exceeds the @code{last-modified} time
2389of the system in memory
2390@end itemize
2391
2392@cindex ASDF-USER package
2393When system definitions are loaded from @file{.asd} files,
2394they are implicitly loaded into the @code{ASDF-USER} package,
2395which uses @code{ASDF}, @code{UIOP} and @code{UIOP/COMMON-LISP}@footnote{
2396Note that between releases 2.27 and 3.0.3, only @code{UIOP/PACKAGE},
2397not all of @code{UIOP}, was used; if you want your code to work
2398with releases earlier than 3.1.2, you may have to explicitly define a package
2399that uses @code{UIOP}, or use proper package prefix to your symbols, as in
2400@code{uiop:version<}.}
2401Programmers who do anything non-trivial in a @file{.asd} file,
2402such as defining new variables, functions or classes,
2403should include @code{defpackage} and @code{in-package} forms in this file,
2404so they will not overwrite each others' extensions.
2405Such forms might also help the files behave identically
2406if loaded manually with @code{cl:load} for development or debugging,
2407though we recommend you use the function @code{asdf::load-asd} instead,
2408which the @code{slime-asdf} contrib knows about.
2409
2410The default value of @code{*system-definition-search-functions*}
2411is a list of three functions.
2412The first function looks in each of the directories given
2413by evaluating members of @code{*central-registry*}
2414for a file whose name is the name of the system and whose type is @file{asd};
2415the first such file is returned,
2416whether or not it turns out to actually define the appropriate system.
2417The second function does something similar,
2418for the directories specified in the @code{source-registry},
2419but searches the filesystem only once and caches its results.
2420The third function makes the @code{package-inferred-system} extension work,
2421@pxref{The package-inferred-system extension}.
2422
2423Because of the way these search functions are defined,
2424you should put the definition for a system
2425@var{foo} in a file named @file{foo.asd},
2426in a directory that is
2427in the central registry or
2428which can be found using the
2429source registry configuration.
2430
2431@c FIXME: Move this discussion to the system definition grammar, or somewhere else.
2432@anchor{System names}
2433@cindex System names
2434@cindex Primary system name
2435@findex primary-system-name
2436It is often useful to define multiple systems in a same file,
2437but ASDF can only locate a system's definition file based on the system
2438name.
2439For this reason,
2440ASDF 3's system search algorithm has been extended to
2441allow a file @file{foo.asd} to contain
2442secondary systems named @var{foo/bar}, @var{foo/baz}, @var{foo/quux}, etc.,
2443in addition to the primary system named @var{foo}.
2444The first component of a system name,
2445separated by the slash character, @code{/},
2446is called the primary name of a system.
2447The primary name may be
2448extracted by function @code{asdf::primary-system-name};
2449when ASDF 3 is told to find a system whose name has a slash,
2450it will first attempt to load the corresponding primary system,
2451and will thus see any such definitions, and/or any
2452definition of a @code{package-inferred-system}.@footnote{
2453ASDF 2.26 and earlier versions
2454do not support this primary system name convention.
2455With these versions of ASDF
2456you must explicitly load @file{foo.asd}
2457before you can use system @var{foo/bar} defined therein,
2458e.g. using @code{(asdf:find-system "foo")}.
2459We do not support ASDF 2, and recommend that you should upgrade to ASDF 3.
2460}
2461If your file @file{foo.asd} also defines systems
2462that do not follow this convention, e.g., a system named @var{foo-test},
2463ASDF will not be able to automatically locate a definition for these systems,
2464and will only see their definition
2465if you explicitly find or load the primary system
2466using e.g. @code{(asdf:find-system "foo")} before you try to use them.
2467We strongly recommend against this practice,
2468though it is currently supported for backward compatibility.
2469
2470@end defun
2471
2472@defun primary-system-name name
2473
2474Internal (not exported) function, @code{asdf::primary-system-name}.
2475Returns the primary system name (the portion before
2476the slash, @code{/}, in a secondary system name) from @var{name}.
2477
2478@end defun
2479
2480@defun locate-system name
2481
2482This function should typically @emph{not} be invoked directly.  It is
2483exported as part of the API only for programmers who wish to provide
2484their own @code{*system-definition-search-functions*}.
2485
2486Given a system @var{name} designator,
2487try to locate where to load the system definition from.
2488@c (This does not include the loading of the system definition,
2489@c which is done by @code{find-system},
2490@c or the loading of the system itself, which is done by @code{load-system};
2491@c however, for systems the definition of which has already been loaded,
2492@c @code{locate-system} may return an object of class @code{system}.)
2493Returns five values: @var{foundp}, @var{found-system}, @var{pathname},
2494@var{previous}, and @var{previous-time}.
2495@var{foundp} is true when a system was found,
2496either a new as yet unregistered one, or a previously registered one.
2497The @var{found-system} return value
2498will be a @code{system} object, if a system definition is found in your
2499source registry.
2500@c This system may be registered (by @code{register-system}) or may not, if
2501@c it's preloaded code.  Fare writes:
2502@c In the case of preloaded code, as for "asdf", "uiop", etc.,
2503@c themselves, the system objects are not registered until after they are
2504@c initially located by sysdef-preloaded-system-search as a fallback when
2505@c no source code was found.
2506The system definition will @emph{not} be
2507loaded if it hasn't been loaded already.
2508@var{pathname} when not null is a path from which to load the system,
2509either associated with @var{found-system}, or with the @var{previous} system.
2510If @var{previous} is not null, it will be a @emph{previously loaded}
2511@code{system} object of the same name (note that the system
2512@emph{definition} is previously-loaded: the system itself may or may not be).
2513@var{previous-time} when not null is
2514the timestamp of the previous system definition file, at the
2515time when the @var{previous} system definition was loaded.
2516
2517For example, if your current registry has @file{foo.asd} in
2518@file{/current/path/to/foo.asd},
2519but system @code{foo} was previously loaded from @file{/previous/path/to/foo.asd}
2520then @var{locate-system} will return the following values:
2521@enumerate
2522@item
2523@var{foundp} will be @code{T},
2524@item
2525@var{found-system} will be @code{NIL},
2526@item
2527@var{pathname} will be @code{#p"/current/path/to/foo.asd"},
2528@item
2529@var{previous} will be an object of type @code{SYSTEM} with
2530@code{system-source-file} slot value of
2531@code{#p"/previous/path/to/foo.asd"}
2532@item
2533@var{previous-time} will be the timestamp of
2534@code{#p"/previous/path/to/foo.asd"} at the time it was loaded.
2535@end enumerate
2536@end defun
2537
2538@defun find-component base path
2539
2540Given a @var{base} component (or designator for such),
2541and a @var{path}, find the component designated by the @var{path}
2542starting from the @var{base}.
2543
2544If @var{path} is a component object, it designates itself,
2545independently from the base.
2546
2547@findex coerce-name
2548If @var{path} is a string, or symbol denoting a string via @code{coerce-name},
2549then @var{base} is resolved to a component object,
2550which must be a system or module,
2551and the designated component is the child named by the @var{path}.
2552
2553If @var{path} is a @code{cons} cell,
2554@code{find-component} with the base and the @code{car} of the @var{path},
2555and the resulting object is used as the base for a tail call
2556to @code{find-component} with the @code{car} of the @var{path}.
2557
2558If @var{base} is a component object, it designates itself.
2559
2560If @var{base} is null, then @var{path} is used as the base, with @code{nil} as the path.
2561
2562If @var{base} is a string, or symbol denoting a string via @code{coerce-name},
2563it designates a system as per @code{find-system}.
2564
2565If @var{base} is a @code{cons} cell, it designates the component found by
2566@code{find-component} with its @code{car} as base and @code{cdr} as path.
2567@end defun
2568
2569
2570@menu
2571* Common attributes of components::
2572* Pre-defined subclasses of component::
2573* Creating new component types::
2574@end menu
2575
2576@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
2577@comment  node-name,  next,  previous,  up
2578@subsection Common attributes of components
2579
2580All components, regardless of type, have the following attributes.
2581All attributes except @code{name} are optional.
2582
2583@subsubsection Name
2584@findex coerce-name
2585A component name is a string or a symbol.
2586If a symbol, its name is taken and lowercased.  This translation is
2587performed by the exported function @code{coerce-name}.
2588
2589Unless overridden by a @code{:pathname} attribute,
2590the name will be interpreted as a pathname specifier according
2591to a Unix-style syntax.
2592@xref{The defsystem grammar,,Pathname specifiers}.
2593
2594@subsubsection Version identifier
2595@findex version-satisfies
2596@cindex :version
2597
2598This optional attribute specifies a version for the current component.
2599The version should typically be a string of integers separated by dots,
2600for example @samp{1.0.11}.
2601For more information on version specifiers, see @ref{The defsystem grammar}.
2602
2603A version may then be queried by the generic function @code{version-satisfies},
2604to see if @code{:version} dependencies are satisfied,
2605and when specifying dependencies, a constraint of minimal version to satisfy
2606can be specified using e.g. @code{(:version "mydepname" "1.0.11")}.
2607
2608Note that in the wild, we typically see version numbering
2609only on components of type @code{system}.
2610Presumably it is much less useful within a given system,
2611wherein the library author is responsible to keep the various files in synch.
2612
2613@subsubsection Required features
2614@anchor{required-features}
2615
2616Traditionally defsystem users have used @code{#+} reader conditionals
2617to include or exclude specific per-implementation files.
2618For example, CFFI, the portable C foreign function interface contained
2619lines like:
2620@lisp
2621     #+sbcl       (:file "cffi-sbcl")
2622@end lisp
2623An unfortunate side effect of this approach is that no single
2624implementation can read the entire system.
2625This causes problems if, for example, one wished to design an @code{archive-op}
2626that would create an archive file containing all the sources, since
2627for example the file @code{cffi-sbcl.lisp} above would be invisible when
2628running the @code{archive-op} on any implementation other than SBCL.
2629
2630Starting with ASDF 3,
2631components may therefore have an @code{:if-feature} option.
2632The value of this option should be
2633a feature expression using the same syntax as @code{#+} does.
2634If that feature expression evaluates to false, any reference to the component will be ignored
2635during compilation, loading and/or linking.
2636Since the expression is read by the normal reader,
2637you must explicitly prefix your symbols with @code{:} so they be read as keywords;
2638this is as contrasted with the @code{#+} syntax
2639that implicitly reads symbols in the keyword package by default.
2640
2641For instance, @code{:if-feature (:and :x86 (:or :sbcl :cmu :scl))} specifies that
2642the given component is only to be compiled and loaded
2643when the implementation is SBCL, CMUCL or Scieneer CL on an x86 machine.
2644You cannot write it as @code{:if-feature (and x86 (or sbcl cmu scl))}
2645since the symbols would not be read as keywords.
2646
2647@xref{if-feature-option}.
2648
2649@subsubsection Dependencies
2650
2651This attribute specifies dependencies of the component on its siblings.
2652It is optional but often necessary.
2653
2654There is an excitingly complicated relationship between the initarg
2655and the method that you use to ask about dependencies
2656
2657Dependencies are between (operation component) pairs.
2658In your initargs for the component, you can say
2659
2660@lisp
2661:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
2662              (load-op (load-op "foo")))
2663@end lisp
2664
2665This means the following things:
2666@itemize
2667@item
2668before performing compile-op on this component, we must perform
2669load-op on @var{a} and @var{b}, and compile-op on @var{c},
2670@item
2671before performing @code{load-op}, we have to load @var{foo}
2672@end itemize
2673
2674The syntax is approximately
2675
2676@verbatim
2677(this-op @{(other-op required-components)@}+)
2678
2679simple-component-name := string
2680                      |  symbol
2681
2682required-components := simple-component-name
2683                     | (required-components required-components)
2684
2685component-name := simple-component-name
2686                | (:version simple-component-name minimum-version-object)
2687@end verbatim
2688
2689Side note:
2690
2691This is on a par with what ACL defsystem does.
2692mk-defsystem is less general: it has an implied dependency
2693
2694@verbatim
2695  for all source file x, (load x) depends on (compile x)
2696@end verbatim
2697
2698and using a @code{:depends-on} argument to say that @var{b} depends on
2699@var{a} @emph{actually} means that
2700
2701@verbatim
2702  (compile b) depends on (load a)
2703@end verbatim
2704
2705This is insufficient for e.g. the McCLIM system, which requires that
2706all the files are loaded before any of them can be compiled ]
2707
2708End side note
2709
2710In ASDF, the dependency information for a given component and operation
2711can be queried using @code{(component-depends-on operation component)},
2712which returns a list
2713
2714@lisp
2715((load-op "a") (load-op "b") (compile-op "c") ...)
2716@end lisp
2717
2718@code{component-depends-on} can be subclassed for more specific
2719component/operation types: these need to @code{(call-next-method)}
2720and append the answer to their dependency, unless
2721they have a good reason for completely overriding the default dependencies.
2722
2723If it weren't for CLISP, we'd be using @code{LIST} method
2724combination to do this transparently.
2725But, we need to support CLISP.
2726If you have the time for some CLISP hacking,
2727I'm sure they'd welcome your fixes.
2728@c Doesn't CLISP now support LIST method combination?
2729
2730A minimal version can be specified for a component you depend on
2731(typically another system), by specifying @code{(:version "other-system" "1.2.3")}
2732instead of simply @code{"other-system"} as the dependency.
2733See the discussion of the semantics of @code{:version}
2734in the defsystem grammar.
2735
2736@c FIXME: Should have cross-reference to "Version specifiers" in the
2737@c defsystem grammar, but the cross-referencing is so broken by
2738@c insufficient node breakdown that I have not put one in.
2739
2740
2741@subsubsection pathname
2742
2743This attribute is optional and if absent (which is the usual case),
2744the component name will be used.
2745
2746@xref{The defsystem grammar,,Pathname specifiers},
2747for an explanation of how this attribute is interpreted.
2748
2749Note that the @code{defsystem} macro (used to create a ``top-level'' system)
2750does additional processing to set the filesystem location of
2751the top component in that system.
2752This is detailed elsewhere. @xref{Defining systems with defsystem}.
2753
2754
2755@subsubsection properties
2756
2757This attribute is optional.
2758
2759Packaging systems often require information about files or systems
2760in addition to that specified by ASDF's pre-defined component attributes.
2761Programs that create vendor packages out of ASDF systems therefore
2762have to create ``placeholder'' information to satisfy these systems.
2763Sometimes the creator of an ASDF system may know the additional
2764information and wish to provide it directly.
2765
2766@code{(component-property component property-name)} and
2767associated @code{setf} method will allow
2768the programmatic update of this information.
2769Property names are compared as if by @code{EQL},
2770so use symbols or keywords or something.
2771
2772@menu
2773* Pre-defined subclasses of component::
2774* Creating new component types::
2775@end menu
2776
2777@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
2778@comment  node-name,  next,  previous,  up
2779@subsection Pre-defined subclasses of component
2780
2781@deffn Component source-file
2782
2783A source file is any file that the system does not know how to
2784generate from other components of the system.
2785
2786Note that this is not necessarily the same thing as
2787``a file containing data that is typically fed to a compiler''.
2788If a file is generated by some pre-processor stage
2789(e.g. a @file{.h} file from @file{.h.in} by autoconf)
2790then it is not, by this definition, a source file.
2791Conversely, we might have a graphic file
2792that cannot be automatically regenerated,
2793or a proprietary shared library that we received as a binary:
2794these do count as source files for our purposes.
2795
2796Subclasses of source-file exist for various languages.
2797@emph{FIXME: describe these.}
2798@end deffn
2799
2800@deffn Component module
2801
2802A module is a collection of sub-components.
2803
2804A module component has the following extra initargs:
2805
2806@itemize
2807@item
2808@code{:components} the components contained in this module
2809
2810@item
2811@code{:default-component-class}
2812All children components which don't specify their class explicitly
2813are inferred to be of this type.
2814
2815@item
2816@code{:if-component-dep-fails}
2817This attribute was removed in ASDF 3. Do not use it.
2818Use @code{:if-feature} instead (@pxref{required-features}, and @pxref{if-feature-option}).
2819
2820@item
2821@code{:serial} When this attribute is set,
2822each subcomponent of this component is assumed to depend on all subcomponents
2823before it in the list given to @code{:components}, i.e.
2824all of them are loaded before a compile or load operation is performed on it.
2825
2826@end itemize
2827
2828The default operation knows how to traverse a module, so
2829most operations will not need to provide methods specialised on modules.
2830
2831@code{module} may be subclassed to represent components such as
2832foreign-language linked libraries or archive files.
2833@end deffn
2834
2835@deffn Component system
2836
2837@code{system} is a subclass of @code{module}.
2838
2839A system is a module with a few extra attributes for documentation
2840purposes; these are given elsewhere.
2841@xref{The defsystem grammar}.
2842
2843Users can create new classes for their systems:
2844the default @code{defsystem} macro takes a @code{:class} keyword argument.
2845@end deffn
2846
2847@node  Creating new component types,  , Pre-defined subclasses of component, Components
2848@comment  node-name,  next,  previous,  up
2849@subsection Creating new component types
2850
2851New component types are defined by subclassing one of the existing
2852component classes and specializing methods on the new component class.
2853
2854@c FIXME: this should perhaps be explained more throughly,
2855@c not only by example ...
2856
2857As an example, suppose we have some implementation-dependent
2858functionality that we want to isolate
2859in one subdirectory per Lisp implementation our system supports.
2860We create a subclass of
2861@code{cl-source-file}:
2862
2863@lisp
2864(defclass unportable-cl-source-file (cl-source-file)
2865  ())
2866@end lisp
2867
2868Function @code{asdf:implementation-type} (exported since 2.014.14)
2869gives us the name of the subdirectory.
2870All that's left is to define how to calculate the pathname
2871of an @code{unportable-cl-source-file}.
2872
2873@lisp
2874(defmethod component-pathname ((component unportable-cl-source-file))
2875  (merge-pathnames*
2876   (parse-unix-namestring (format nil "~(~A~)/" (asdf:implementation-type)))
2877   (call-next-method)))
2878@end lisp
2879
2880The new component type is used in a @code{defsystem} form in this way:
2881
2882@lisp
2883(defsystem :foo
2884    :components
2885    ((:file "packages")
2886     ...
2887     (:unportable-cl-source-file "threads"
2888      :depends-on ("packages" ...))
2889     ...
2890    )
2891@end lisp
2892
2893@node Dependencies, Functions, Components, The object model of ASDF
2894@section Dependencies
2895@c FIXME: Moved this material here, but it isn't very comfortable
2896@c here....  Also needs to be revised to be coherent.
2897
2898To be successfully build-able, this graph of actions must be acyclic.
2899If, as a user, extender or implementer of ASDF, you introduce
2900a cycle into the dependency graph,
2901ASDF will fail loudly.
2902To clearly distinguish the direction of dependencies,
2903ASDF 3 uses the words @emph{requiring} and @emph{required}
2904as applied to an action depending on the other:
2905the requiring action @code{depends-on} the completion of all required actions
2906before it may itself be @code{perform}ed.
2907
2908Using the @code{defsystem} syntax, users may easily express
2909direct dependencies along the graph of the object hierarchy:
2910between a component and its parent, its children, and its siblings.
2911By defining custom CLOS methods, you can express more elaborate dependencies as you wish.
2912Most common operations, such as @code{load-op}, @code{compile-op} or @code{load-source-op}
2913are automatically propagate ``downward'' the component hierarchy and are ``covariant'' with it:
2914to act the operation on the parent module, you must first act it on all the children components,
2915with the action on the parent being parent of the action on each child.
2916Other operations, such as @code{prepare-op} and @code{prepare-source-op}
2917(introduced in ASDF 3) are automatically propagated ``upward'' the component hierarchy
2918and are ``contravariant'' with it:
2919to perform the operation of preparing for compilation of a child component,
2920you must perform the operation of preparing for compilation of its parent component, and so on,
2921ensuring that all the parent's dependencies are (compiled and) loaded
2922before the child component may be compiled and loaded.
2923Yet other operations, such as @code{test-op} or @code{load-bundle-op}
2924remain at the system level, and are not propagated along the hierarchy,
2925but instead do something global on the system.
2926
2927
2928@node Functions,  , Dependencies, The object model of ASDF
2929@comment  node-name,  next,  previous,  up
2930@section Functions
2931
2932@c FIXME: this does not belong here....
2933@defun version-satisfies @var{version} @var{version-spec}
2934Does @var{version} satisfy the @var{version-spec}.  A generic function.
2935ASDF provides built-in methods for @var{version} being a @code{component} or @code{string}.
2936@var{version-spec} should be a string.
2937If it's a component, its version is extracted as a string before further processing.
2938
2939A version string satisfies the version-spec if after parsing,
2940the former is no older than the latter.
2941Therefore @code{"1.9.1"}, @code{"1.9.2"} and @code{"1.10"} all satisfy @code{"1.9.1"},
2942but @code{"1.8.4"} or @code{"1.9"} do not.
2943For more information about how @code{version-satisfies} parses and interprets
2944version strings and specifications,
2945@pxref{The defsystem grammar} (version specifiers) and
2946@ref{Common attributes of components}.
2947
2948Note that in versions of ASDF prior to 3.0.1,
2949including the entire ASDF 1 and ASDF 2 series,
2950@code{version-satisfies} would also require that the version and the version-spec
2951have the same major version number (the first integer in the list);
2952if the major version differed, the version would be considered as not matching the spec.
2953But that feature was not documented, therefore presumably not relied upon,
2954whereas it was a nuisance to several users.
2955Starting with ASDF 3.0.1,
2956@code{version-satisfies} does not treat the major version number specially,
2957and returns T simply if the first argument designates a version that isn't older
2958than the one specified as a second argument.
2959If needs be, the @code{(:version ...)} syntax for specifying dependencies
2960could be in the future extended to specify an exclusive upper bound for compatible versions
2961as well as an inclusive lower bound.
2962@end defun
2963
2964@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
2965@comment  node-name,  next,  previous,  up
2966@chapter Controlling where ASDF searches for systems
2967
2968
2969
2970@menu
2971* Configurations::
2972* Truenames and other dangers::
2973* XDG base directory::
2974* Backward Compatibility::
2975* Configuration DSL::
2976* Configuration Directories::
2977* Shell-friendly syntax for configuration::
2978* Search Algorithm::
2979* Caching Results::
2980* Configuration API::
2981* Introspection::
2982* Status::
2983* Rejected ideas::
2984* TODO::
2985* Credits for the source-registry::
2986@end menu
2987
2988@node  Configurations, Truenames and other dangers, Controlling where ASDF searches for systems, Controlling where ASDF searches for systems
2989@section Configurations
2990
2991Configurations specify paths where to find system files.
2992
2993@enumerate
2994
2995@item
2996The search registry may use some hardcoded wrapping registry specification.
2997This allows some implementations (notably SBCL) to specify where to find
2998some special implementation-provided systems that
2999need to precisely match the version of the implementation itself.
3000
3001@item
3002An application may explicitly initialize the source-registry configuration
3003using the configuration API
3004(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
3005in which case this takes precedence.
3006It may itself compute this configuration from the command-line,
3007from a script, from its own configuration file, etc.
3008
3009@item
3010The source registry will be configured from
3011the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
3012
3013@item
3014The source registry will be configured from
3015user configuration file
3016@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf}
3017(which defaults to
3018@file{~/.config/common-lisp/source-registry.conf})
3019if it exists.
3020
3021@item
3022The source registry will be configured from
3023user configuration directory
3024@file{$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/}
3025(which defaults to
3026@file{~/.config/common-lisp/source-registry.conf.d/})
3027if it exists.
3028
3029@item
3030The source registry will be configured from
3031default user configuration trees
3032@file{~/common-lisp/} (since ASDF 3.1.2 only),
3033@file{~/.sbcl/systems/} (on SBCL only),
3034@file{$XDG_DATA_HOME/common-lisp/systems/} (no recursion, link farm)
3035@file{$XDG_DATA_HOME/common-lisp/source/}.
3036The @code{XDG_DATA_HOME} directory defaults to @file{~/.local/share/}.
3037On Windows, the @code{local-appdata} and @code{appdata} directories are used instead.
3038
3039@item
3040The source registry will be configured from
3041system configuration file
3042@file{/etc/common-lisp/source-registry.conf}
3043if it exists.
3044
3045@item
3046The source registry will be configured from
3047system configuration directory
3048@file{/etc/common-lisp/source-registry.conf.d/}
3049if it exists.
3050
3051@item
3052The source registry will be configured from a default configuration.
3053This configuration may allow for implementation-specific systems
3054to be found, for systems to be found the current directory
3055(at the time that the configuration is initialized) as well as
3056@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
3057@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/},
3058where @code{XDG_DATA_DIRS} defaults to @file{/usr/local/share} and @file{/usr/share} on Unix,
3059and the @code{common-appdata} directory on Windows.
3060
3061@item
3062The source registry may include implementation-dependent directories
3063that correspond to implementation-provided extensions.
3064
3065@end enumerate
3066
3067Each of these configurations is specified as an s-expression
3068in a trivial domain-specific language (defined below).
3069Additionally, a more shell-friendly syntax is available
3070for the environment variable (defined yet below).
3071
3072Each of these configurations is only used if the previous
3073configuration explicitly or implicitly specifies that it
3074includes its inherited configuration.
3075
3076Additionally, some implementation-specific directories
3077may be automatically prepended to whatever directories are specified
3078in configuration files, no matter if the last one inherits or not.
3079
3080@node Truenames and other dangers, XDG base directory, Configurations, Controlling where ASDF searches for systems
3081@section Truenames and other dangers
3082
3083One great innovation of the original ASDF was its ability to leverage
3084@code{CL:TRUENAME} to locate where your source code was and where to build it,
3085allowing for symlink farms as a simple but effective configuration mechanism
3086that is easy to control programmatically.
3087ASDF 3 still supports this configuration style, and it is enabled by default;
3088however we recommend you instead use
3089our source-registry configuration mechanism described below,
3090because it is easier to setup in a portable way across users and implementations.
3091
3092Additionally, some people dislike truename,
3093either because it is very slow on their system, or
3094because they are using content-addressed storage where the truename of a file
3095is related to a digest of its individual contents,
3096and not to other files in the same intended project.
3097For these people, ASDF 3 allows to eschew the @code{TRUENAME} mechanism,
3098by setting the variable @var{asdf:*resolve-symlinks*} to @code{nil}.
3099
3100PS: Yes, if you haven't read Vernor Vinge's short but great classic
3101``True Names... and Other Dangers'' then you're in for a treat.
3102
3103@node XDG base directory, Backward Compatibility, Truenames and other dangers, Controlling where ASDF searches for systems
3104@section XDG base directory
3105
3106Note that we purport to respect the XDG base directory specification
3107as to where configuration files are located,
3108where data files are located,
3109where output file caches are located.
3110Mentions of XDG variables refer to that document.
3111
3112@url{http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html}
3113
3114This specification allows the user to specify some environment variables
3115to customize how applications behave to his preferences.
3116
3117On Windows platforms, when not using Cygwin,
3118instead of the XDG base directory specification,
3119we try to use folder configuration from the registry regarding
3120@code{Common AppData} and similar directories.
3121Since support for querying the Windows registry
3122is not possible to do in reasonable amounts of portable Common Lisp code,
3123ASDF 3 relies on the environment variables that Windows usually exports.
3124
3125@node Backward Compatibility, Configuration DSL, XDG base directory, Controlling where ASDF searches for systems
3126@section Backward Compatibility
3127
3128For backward compatibility as well as to provide a practical backdoor for hackers,
3129ASDF will first search for @file{.asd} files in the directories specified in
3130@code{asdf:*central-registry*}
3131before it searches in the source registry above.
3132
3133@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
3134
3135By default, @code{asdf:*central-registry*} will be empty.
3136
3137This old mechanism will therefore not affect you if you don't use it,
3138but will take precedence over the new mechanism if you do use it.
3139
3140@node Configuration DSL, Configuration Directories, Backward Compatibility, Controlling where ASDF searches for systems
3141@section Configuration DSL
3142@cindex :inherit-configuration source config directive
3143@cindex inherit-configuration source config directive
3144@cindex :ignore-invalid-entries source config directive
3145@cindex ignore-invalid-entries source config directive
3146@cindex :directory source config directive
3147@cindex directory source config directive
3148@cindex :tree source config directive
3149@cindex tree source config directive
3150@cindex :exclude source config directive
3151@cindex exclude source config directive
3152@cindex :also-exclude source config directive
3153@cindex also-exclude source config directive
3154@cindex :include source config directive
3155@cindex include source config directive
3156@cindex :default-registry source config directive
3157@cindex default-registry source config directive
3158
3159Here is the grammar of the s-expression (SEXP) DSL for source-registry
3160configuration:
3161
3162@c FIXME: This is too wide for happy compilation into pdf.
3163
3164@example
3165;; A configuration is a single SEXP starting with the keyword
3166;; :source-registry followed by a list of directives.
3167CONFIGURATION := (:source-registry DIRECTIVE ...)
3168
3169;; A directive is one of the following:
3170DIRECTIVE :=
3171    ;; INHERITANCE DIRECTIVE:
3172    ;; Your configuration expression MUST contain
3173    ;; exactly one of the following:
3174    :inherit-configuration |
3175    ;; splices inherited configuration (often specified last) or
3176    :ignore-inherited-configuration |
3177    ;; drop inherited configuration (specified anywhere)
3178
3179    ;; forward compatibility directive (since ASDF 2.011.4), useful when
3180    ;; you want to use new configuration features but have to bootstrap
3181    ;; the newer required ASDF from an older release that doesn't
3182    ;; support said features:
3183    :ignore-invalid-entries |
3184
3185    ;; add a single directory to be scanned (no recursion)
3186    (:directory DIRECTORY-PATHNAME-DESIGNATOR) |
3187
3188    ;; add a directory hierarchy, recursing but
3189    ;; excluding specified patterns
3190    (:tree DIRECTORY-PATHNAME-DESIGNATOR) |
3191
3192    ;; override the defaults for exclusion patterns
3193    (:exclude EXCLUSION-PATTERN ...) |
3194    ;; augment the defaults for exclusion patterns
3195    (:also-exclude EXCLUSION-PATTERN ...) |
3196    ;; Note that the scope of a an exclude pattern specification is
3197    ;; the rest of the current configuration expression or file.
3198
3199    ;; splice the parsed contents of another config file
3200    (:include REGULAR-FILE-PATHNAME-DESIGNATOR) |
3201
3202    ;; This directive specifies that some default must be spliced.
3203    :default-registry
3204
3205REGULAR-FILE-PATHNAME-DESIGNATOR
3206    := PATHNAME-DESIGNATOR ; interpreted as a file
3207DIRECTORY-PATHNAME-DESIGNATOR
3208    := PATHNAME-DESIGNATOR ; interpreted as a directory
3209
3210PATHNAME-DESIGNATOR :=
3211    NIL | ;; Special: skip this entry.
3212    ABSOLUTE-COMPONENT-DESIGNATOR ;; see pathname DSL
3213
3214EXCLUSION-PATTERN := a string without wildcards, that will be matched
3215    exactly against the name of a any subdirectory in the directory
3216    component of a path. e.g. @code{"_darcs"} will match
3217    @file{#p"/foo/bar/_darcs/src/bar.asd"}
3218@end example
3219
3220Pathnames are designated using another DSL,
3221shared with the output-translations configuration DSL below.
3222The DSL is resolved by the function @code{asdf::resolve-location},
3223to be documented and exported at some point in the future.
3224
3225@example
3226ABSOLUTE-COMPONENT-DESIGNATOR :=
3227    (ABSOLUTE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
3228    STRING |
3229    ;; namestring (better be absolute or bust, directory assumed where
3230    ;; applicable).  In output-translations, directory is assumed and
3231    ;; **/*.*.* added if it's last.  On MCL, a MacOSX-style POSIX
3232    ;; namestring (for MacOS9 style, use #p"..."); Note that none of the
3233    ;; above applies to strings used in *central-registry*, which
3234    ;; doesn't use this DSL: they are processed as normal namestrings.
3235    ;; however, you can compute what you put in the *central-registry*
3236    ;; based on the results of say
3237    ;; (asdf::resolve-location "/Users/fare/cl/cl-foo/")
3238    PATHNAME |
3239    ;; pathname (better be an absolute path, or bust)
3240    ;; In output-translations, unless followed by relative components,
3241    ;; it better have appropriate wildcards, as in **/*.*.*
3242    :HOME | ; designates the user-homedir-pathname ~/
3243    :USER-CACHE | ; designates the default location for the user cache
3244    :HERE |
3245    ;; designates the location of the configuration file
3246    ;; (or *default-pathname-defaults*, if invoked interactively)
3247    :ROOT
3248    ;; magic, for output-translations source only: paths that are relative
3249    ;; to the root of the source host and device
3250
3251They keyword :SYSTEM-CACHE is not accepted in ASDF 3.1 and beyond: it
3252was a security hazard.
3253
3254RELATIVE-COMPONENT-DESIGNATOR :=
3255    (RELATIVE-COMPONENT-DESIGNATOR RELATIVE-COMPONENT-DESIGNATOR ...) |
3256    STRING |
3257      ;; relative directory pathname as interpreted by
3258      ;; parse-unix-namestring.
3259      ;; In output translations, if last component, **/*.*.* is added
3260    PATHNAME | ; pathname; unless last component, directory is assumed.
3261    :IMPLEMENTATION |
3262       ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
3263    :IMPLEMENTATION-TYPE |
3264       ;; a directory based on lisp-implementation-type only, e.g. sbcl
3265    :DEFAULT-DIRECTORY |
3266       ;; a relativized version of the default directory
3267    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
3268    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
3269    :*.*.* | ;; any file (since ASDF 2.011.4)
3270
3271The keywords :UID and :USERNAME are no longer supported.
3272@end example
3273
3274For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
3275which is the default place ASDF looks for this configuration, once contained:
3276@example
3277(:source-registry
3278  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
3279  :inherit-configuration)
3280@end example
3281
3282@node Configuration Directories, Shell-friendly syntax for configuration, Configuration DSL, Controlling where ASDF searches for systems
3283@section Configuration Directories
3284
3285Configuration directories consist in files each containing
3286a list of directives without any enclosing @code{(:source-registry ...)} form.
3287The files will be sorted by namestring as if by @code{string<} and
3288the lists of directives of these files with be concatenated in order.
3289An implicit @code{:inherit-configuration} will be included
3290at the @emph{end} of the list.
3291
3292System-wide or per-user Common Lisp software distributions
3293such as Debian packages or some future version of @code{clbuild}
3294may then include files such as
3295@file{/etc/common-lisp/source-registry.conf.d/10-foo.conf} or
3296@file{~/.config/common-lisp/source-registry.conf.d/10-foo.conf}
3297to easily and modularly register configuration information
3298about software being distributed.
3299
3300The convention is that, for sorting purposes,
3301the names of files in such a directory begin with two digits
3302that determine the order in which these entries will be read.
3303Also, the type of these files must be @file{.conf},
3304which not only simplifies the implementation by allowing
3305for more portable techniques in finding those files,
3306but also makes it trivial to disable a file, by renaming it to a different file type.
3307
3308Directories may be included by specifying a directory pathname
3309or namestring in an @code{:include} directive, e.g.:
3310
3311@example
3312  (:include "/foo/bar/")
3313@end example
3314
3315Hence, to achieve the same effect as
3316my example @file{~/.config/common-lisp/source-registry.conf} above,
3317I could simply create a file
3318@file{~/.config/common-lisp/source-registry.conf.d/33-home-fare-cl.conf}
3319alone in its directory with the following contents:
3320@example
3321(:tree "/home/fare/cl/")
3322@end example
3323
3324@menu
3325* The here directive::
3326@end menu
3327
3328@node The here directive,  , Configuration Directories, Configuration Directories
3329@subsection The :here directive
3330
3331The @code{:here} directive is an absolute pathname designator that
3332refers to the directory containing the configuration file currently
3333being processed.
3334
3335The @code{:here} directive is intended to simplify the delivery of
3336complex CL systems, and for easy configuration of projects shared through
3337revision control systems, in accordance with our design principle that
3338each participant should be able to provide all and only the information
3339available to him or her.
3340
3341Consider a person X who has set up the source code repository for a
3342complex project with a master directory @file{dir/}.  Ordinarily, one
3343might simply have the user add a directive that would look something
3344like this:
3345@example
3346   (:tree "path/to/dir")
3347@end example
3348But what if X knows that there are very large subtrees
3349under dir that are filled with, e.g., Java source code, image files for
3350icons, etc.?  All of the asdf system definitions are contained in the
3351subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
3352these are the only directories that should be searched.
3353
3354In this case, X can put into @file{dir/} a file @file{asdf.conf} that
3355contains the following:
3356@example
3357(:source-registry
3358   (:tree (:here "src/lisp/"))
3359   (:tree (:here "extlib/lisp"))
3360   (:directory (:here "outlier/")))
3361@end example
3362
3363Then when someone else (call her Y) checks out a copy of this
3364repository, she need only add
3365@example
3366(:include "/path/to/my/checkout/directory/asdf.conf")
3367@end example
3368to one of her previously-existing asdf source location configuration
3369files, or invoke @code{initialize-source-registry} with a configuration
3370form containing that s-expression.  ASDF will find the .conf file that X
3371has provided, and then set up source locations within the working
3372directory according to X's (relative) instructions.
3373
3374@node Shell-friendly syntax for configuration, Search Algorithm, Configuration Directories, Controlling where ASDF searches for systems
3375@section Shell-friendly syntax for configuration
3376
3377When considering environment variable @code{CL_SOURCE_REGISTRY}
3378ASDF will skip to next configuration if it's an empty string.
3379It will @code{READ} the string as a SEXP in the DSL
3380if it begins with a paren @code{(},
3381otherwise it will be interpreted much like @code{TEXINPUTS},
3382as a list of paths, where
3383
3384  * paths are separated
3385   by a @code{:} (colon) on Unix platforms (including cygwin),
3386   by a @code{;} (semicolon) on other platforms (mainly, Windows).
3387
3388  * each entry is a directory to add to the search path.
3389
3390  * if the entry ends with a double slash @code{//}
3391    then it instead indicates a tree in the subdirectories
3392    of which to recurse.
3393
3394  * if the entry is the empty string (which may only appear once),
3395    then it indicates that the inherited configuration should be
3396    spliced there.
3397
3398@node Search Algorithm, Caching Results, Shell-friendly syntax for configuration, Controlling where ASDF searches for systems
3399@section Search Algorithm
3400@vindex *default-source-registry-exclusions*
3401
3402In case that isn't clear, the semantics of the configuration is that
3403when searching for a system of a given name,
3404directives are processed in order.
3405
3406When looking in a directory, if the system is found, the search succeeds,
3407otherwise it continues.
3408
3409When looking in a tree, if one system is found, the search succeeds.
3410If multiple systems are found, the consequences are unspecified:
3411the search may succeed with any of the found systems,
3412or an error may be raised.
3413ASDF currently returns the first system found,
3414XCVB currently raised an error.
3415If none is found, the search continues.
3416
3417Exclude statements specify patterns of subdirectories
3418the systems from which to ignore.
3419Typically you don't want to use copies of files kept by such
3420version control systems as Darcs.
3421Exclude statements are not propagated to further included or inherited
3422configuration files or expressions;
3423instead the defaults are reset around every configuration statement
3424to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
3425
3426Include statements cause the search to recurse with the path specifications
3427from the file specified.
3428
3429An inherit-configuration statement cause the search to recurse with the path
3430specifications from the next configuration
3431(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
3432
3433
3434@node Caching Results, Configuration API, Search Algorithm, Controlling where ASDF searches for systems
3435@section Caching Results
3436
3437The implementation is allowed to either eagerly compute the information
3438from the configurations and file system, or to lazily re-compute it
3439every time, or to cache any part of it as it goes.
3440To explicitly flush any information cached by the system, use the API below.
3441
3442@node Configuration API, Introspection, Caching Results, Controlling where ASDF searches for systems
3443@section Configuration API
3444
3445The specified functions are exported from your build system's package.
3446Thus for ASDF the corresponding functions are in package ASDF,
3447and for XCVB the corresponding functions are in package XCVB.
3448
3449@defun initialize-source-registry @Aoptional{} PARAMETER
3450   will read the configuration and initialize all internal variables.
3451   You may extend or override configuration
3452   from the environment and configuration files
3453   with the given @var{PARAMETER}, which can be
3454   @code{nil} (no configuration override),
3455   or a SEXP (in the SEXP DSL),
3456   a string (as in the string DSL),
3457   a pathname (of a file or directory with configuration),
3458   or a symbol (fbound to function that when called returns one of the above).
3459@end defun
3460
3461@defun clear-source-registry
3462   undoes any source registry configuration
3463   and clears any cache for the search algorithm.
3464   You might want to call this function
3465   (or better, @code{clear-configuration})
3466   before you dump an image that would be resumed
3467   with a different configuration,
3468   and return an empty configuration.
3469   Note that this does not include clearing information about
3470   systems defined in the current image, only about
3471   where to look for systems not yet defined.
3472@end defun
3473
3474@defun ensure-source-registry @Aoptional{} PARAMETER
3475   checks whether a source registry has been initialized.
3476   If not, initialize it with the given @var{PARAMETER}.
3477@end defun
3478
3479Every time you use ASDF's @code{find-system}, or
3480anything that uses it (such as @code{operate}, @code{load-system}, etc.),
3481@code{ensure-source-registry} is called with parameter @code{nil},
3482which the first time around causes your configuration to be read.
3483If you change a configuration file,
3484you need to explicitly @code{initialize-source-registry} again,
3485or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
3486which will cause the initialization to happen next time around.
3487
3488@node Introspection, Status, Configuration API, Controlling where ASDF searches for systems
3489@section Introspection
3490
3491@menu
3492* *source-registry-parameter* variable::
3493* Information about system dependencies::
3494@end menu
3495
3496@node *source-registry-parameter* variable, Information about system dependencies, Introspection, Introspection
3497@subsection *source-registry-parameter* variable
3498@vindex *source-registry-parameter*
3499
3500We have made available the variable @code{*source-registry-parameter*}
3501that can be used by code that wishes to introspect about the (past)
3502configuration of ASDF's source registry.  @strong{This variable should
3503never be set!}  It will be set as a side-effect of calling
3504@code{initialize-source-registry}; user code should treat it as
3505read-only.
3506
3507@node Information about system dependencies,  , *source-registry-parameter* variable, Introspection
3508@subsection Information about system dependencies
3509
3510ASDF makes available three functions to read system interdependencies.
3511These are intended to aid programmers who wish to perform dependency
3512analyses.
3513
3514@defun system-defsystem-depends-on system
3515@end defun
3516
3517@defun system-depends-on system
3518@end defun
3519
3520@defun system-weakly-depends-on system
3521Returns a list of names of systems that are weakly depended on by
3522@var{system}.  Weakly depended on systems are optionally loaded only if
3523ASDF can find them; failure to find such systems does @emph{not} cause an
3524error in loading.
3525
3526Note that the return value for @code{system-weakly-depends-on} is simpler
3527than the return values of the other two system dependency introspection
3528functions.
3529@end defun
3530
3531@node Status, Rejected ideas, Introspection, Controlling where ASDF searches for systems
3532@section Status
3533
3534This mechanism is vastly successful, and we have declared
3535that @code{asdf:*central-registry*} is not recommended anymore,
3536though we will continue to support it.
3537All hooks into implementation-specific search mechanisms
3538have been integrated in the @code{wrapping-source-registry}
3539that everyone uses implicitly.
3540
3541@node Rejected ideas, TODO, Status, Controlling where ASDF searches for systems
3542@section Rejected ideas
3543
3544Alternatives I (FRR) considered and rejected while developing ASDF 2 included:
3545
3546@enumerate
3547@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
3548   and somehow the configuration parser expands the new configuration
3549   language into a expanded series of directories of subdirectories to
3550   lookup, pre-recursing through specified hierarchies. This is kludgy,
3551   and leaves little space of future cleanups and extensions.
3552
3553@item Keep @code{asdf:*central-registry*} as the master but extend its semantics
3554   in completely new ways, so that new kinds of entries may be implemented
3555   as a recursive search, etc. This seems somewhat backwards.
3556
3557@item Completely remove @code{asdf:*central-registry*}
3558   and break backwards compatibility.
3559   Hopefully this will happen in a few years after everyone migrate to
3560   a better ASDF and/or to XCVB, but it would be very bad to do it now.
3561
3562@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
3563   when you dereference it or setf it. Only the new variable with new
3564   semantics is handled by the new search procedure.
3565   Complex and still introduces subtle semantic issues.
3566@end enumerate
3567
3568
3569I've been suggested the below features, but have rejected them,
3570for the sake of keeping ASDF no more complex than strictly necessary.
3571
3572@itemize
3573@item
3574  More syntactic sugar: synonyms for the configuration directives, such as
3575  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
3576  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
3577
3578@item
3579   The possibility to register individual files instead of directories.
3580
3581@item
3582  Integrate Xach Beane's tilde expander into the parser,
3583  or something similar that is shell-friendly or shell-compatible.
3584  I'd rather keep ASDF minimal. But maybe this precisely keeps it
3585  minimal by removing the need for evaluated entries that ASDF has?
3586  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
3587  Hopefully, these are already superseded by the @code{:default-registry}
3588
3589@item
3590  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
3591  down a filesystem tree in the environment variable.
3592  It isn't that Lisp friendly either.
3593@end itemize
3594
3595@node TODO, Credits for the source-registry, Rejected ideas, Controlling where ASDF searches for systems
3596@section TODO
3597
3598@itemize
3599@item Add examples
3600@end itemize
3601
3602@node Credits for the source-registry,  , TODO, Controlling where ASDF searches for systems
3603@section Credits for the source-registry
3604
3605Thanks a lot to Stelian Ionescu for the initial idea.
3606
3607Thanks to Rommel Martinez for the initial implementation attempt.
3608
3609All bad design ideas and implementation bugs are mine, not theirs.
3610But so are good design ideas and elegant implementation tricks.
3611
3612 --- Francois-Rene Rideau @email{fare@@tunes.org}, Mon, 22 Feb 2010 00:07:33 -0500
3613
3614
3615
3616@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
3617@comment  node-name,  next,  previous,  up
3618@chapter Controlling where ASDF saves compiled files
3619@cindex asdf-output-translations
3620@vindex ASDF_OUTPUT_TRANSLATIONS
3621
3622Each Common Lisp implementation has its own format
3623for compiled files or fasls.@footnote{``FASL'' is short for ``FASt Loading.''}
3624If you use multiple implementations
3625(or multiple versions of the same implementation),
3626you'll soon find your source directories
3627littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so
3628on.
3629Worse yet, multiple implementations use the same file extension and
3630some implementations maintain the same file extension
3631while changing formats from version to version (or platform to
3632platform).
3633This can lead to many errors and much confusion
3634as you switch from one implementation to the next.
3635
3636Since ASDF 2, ASDF includes the @code{asdf-output-translations} facility
3637to mitigate the problem.
3638
3639@menu
3640* Output Configurations::
3641* Output Backward Compatibility::
3642* Output Configuration DSL::
3643* Output Configuration Directories::
3644* Output Shell-friendly syntax for configuration::
3645* Semantics of Output Translations::
3646* Output Caching Results::
3647* Output location API::
3648* Credits for output translations::
3649@end menu
3650
3651@node Output Configurations, Output Backward Compatibility, Controlling where ASDF saves compiled files, Controlling where ASDF saves compiled files
3652@section Configurations
3653
3654@c FIXME: Explain how configurations work: can't expect reader will have
3655@c looked at previous chapter. Probably cut and paste will do.
3656
3657
3658Configurations specify mappings from input locations to output locations.
3659Once again we rely on the XDG base directory specification for configuration.
3660@xref{Controlling where ASDF searches for systems,,XDG base directory}.
3661
3662@enumerate
3663
3664@item
3665Some hardcoded wrapping output translations configuration may be used.
3666This allows special output translations (or usually, invariant directories)
3667to be specified corresponding to the similar special entries in the source registry.
3668
3669@item
3670An application may explicitly initialize the output-translations
3671configuration using the Configuration API
3672in which case this takes precedence.
3673(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
3674It may itself compute this configuration from the command-line,
3675from a script, from its own configuration file, etc.
3676
3677@item
3678The source registry will be configured from
3679the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
3680
3681@item
3682The source registry will be configured from
3683user configuration file
3684@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf}
3685(which defaults to
3686@file{~/.config/common-lisp/asdf-output-translations.conf})
3687if it exists.
3688
3689@item
3690The source registry will be configured from
3691user configuration directory
3692@file{$XDG_CONFIG_DIRS/common-lisp/asdf-output-translations.conf.d/}
3693(which defaults to
3694@file{~/.config/common-lisp/asdf-output-translations.conf.d/})
3695if it exists.
3696
3697@item
3698The source registry will be configured from
3699system configuration file
3700@file{/etc/common-lisp/asdf-output-translations.conf}
3701if it exists.
3702
3703@item
3704The source registry will be configured from
3705system configuration directory
3706@file{/etc/common-lisp/asdf-output-translations.conf.d/}
3707if it exists.
3708
3709@end enumerate
3710
3711Each of these configurations is specified as a SEXP
3712in a trivial domain-specific language (@pxref{Configuration DSL}).
3713Additionally, a more shell-friendly syntax is available
3714for the environment variable (@pxref{Shell-friendly syntax for configuration}).
3715
3716When processing an entry in the above list of configuration methods,
3717ASDF will stop unless that entry
3718explicitly or implicitly specifies that it
3719includes its inherited configuration.
3720
3721Note that by default, a per-user cache is used for output files.
3722This allows the seamless use of shared installations of software
3723between several users, and takes files out of the way of the developers
3724when they browse source code,
3725at the expense of taking a small toll when developers have to clean up
3726output files and find they need to get familiar with output-translations
3727first.@footnote{A @code{CLEAN-OP} would be a partial solution to this problem.}
3728
3729
3730@node Output Backward Compatibility, Output Configuration DSL, Output Configurations, Controlling where ASDF saves compiled files
3731@section Backward Compatibility
3732@cindex ASDF-BINARY-LOCATIONS compatibility
3733@c FIXME: Demote this section -- the typical reader doesn't care about
3734@c backwards compatibility.
3735
3736
3737We purposely do @emph{not} provide backward compatibility with earlier versions of
3738@code{ASDF-Binary-Locations} (8 Sept 2009),
3739@code{common-lisp-controller} (7.0) or
3740@code{cl-launch} (2.35),
3741each of which had similar general capabilities.
3742The APIs of these programs were not designed
3743for easy user configuration
3744through configuration files.
3745Recent versions of @code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000)
3746use the new @code{asdf-output-translations} API as defined below.
3747@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
3748
3749This incompatibility shouldn't inconvenience many people.
3750Indeed, few people use and customize these packages;
3751these few people are experts who can trivially adapt to the new configuration.
3752Most people are not experts, could not properly configure these features
3753(except inasmuch as the default configuration of
3754@code{common-lisp-controller} and/or @code{cl-launch}
3755might have been doing the right thing for some users),
3756and yet will experience software that ``just works'',
3757as configured by the system distributor, or by default.
3758
3759Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
3760we provide a limited emulation mode:
3761
3762@defun enable-asdf-binary-locations-compatibility @Akey{} centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
3763This function will initialize the new @code{asdf-output-translations} facility in a way
3764that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
3765Where you would previously set global variables
3766@var{*centralize-lisp-binaries*},
3767@var{*default-toplevel-directory*},
3768@var{*include-per-user-information*},
3769@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
3770you will now have to pass the same values as keyword arguments to this function.
3771Note however that as an extension the @code{:source-to-target-mappings} keyword argument
3772will accept any valid pathname designator for @code{asdf-output-translations}
3773instead of just strings and pathnames.
3774@end defun
3775
3776If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
3777(the one available as an extension to load of top of ASDF,
3778not the one built into a few old versions of ASDF),
3779but first you must disable @code{asdf-output-translations}
3780with @code{(asdf:disable-output-translations)},
3781or you might experience ``interesting'' issues.
3782
3783Also, note that output translation is enabled by default.
3784To disable it, use @code{(asdf:disable-output-translations)}.
3785
3786@node Output Configuration DSL, Output Configuration Directories, Output Backward Compatibility, Controlling where ASDF saves compiled files
3787@section Configuration DSL
3788
3789Here is the grammar of the SEXP DSL
3790for @code{asdf-output-translations} configuration:
3791
3792@verbatim
3793;; A configuration is single SEXP starting with keyword :source-registry
3794;; followed by a list of directives.
3795CONFIGURATION := (:output-translations DIRECTIVE ...)
3796
3797;; A directive is one of the following:
3798DIRECTIVE :=
3799    ;; INHERITANCE DIRECTIVE:
3800    ;; Your configuration expression MUST contain
3801    ;; exactly one of either of these:
3802    :inherit-configuration |
3803      ;; splices inherited configuration (often specified last)
3804    :ignore-inherited-configuration |
3805      ;; drop inherited configuration (specified anywhere)
3806
3807    ;; forward compatibility directive (since ASDF 2.011.4), useful when
3808    ;; you want to use new configuration features but have to bootstrap a
3809    ;; the newer required ASDF from an older release that doesn't have
3810    ;; said features:
3811    :ignore-invalid-entries |
3812
3813    ;; include a configuration file or directory
3814    (:include PATHNAME-DESIGNATOR) |
3815
3816    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/
3817    ;; or something.
3818    :enable-user-cache |
3819    ;; Disable global cache. Map / to /
3820    :disable-cache |
3821
3822    ;; add a single directory to be scanned (no recursion)
3823    (DIRECTORY-DESIGNATOR DIRECTORY-DESIGNATOR)
3824
3825    ;; use a function to return the translation of a directory designator
3826    (DIRECTORY-DESIGNATOR (:function TRANSLATION-FUNCTION))
3827
3828DIRECTORY-DESIGNATOR :=
3829    NIL | ; As source: skip this entry. As destination: same as source
3830    T | ; as source matches anything, as destination
3831        ; maps pathname to itself.
3832    ABSOLUTE-COMPONENT-DESIGNATOR ; same as in the source-registry language
3833
3834TRANSLATION-FUNCTION :=
3835    SYMBOL | ;; symbol naming a function that takes two arguments:
3836             ;; the pathname to be translated and the matching
3837             ;; DIRECTORY-DESIGNATOR
3838    LAMBDA   ;; A form which evaluates to a function taking two arguments:
3839             ;; the pathname to be translated and the matching
3840             ;; DIRECTORY-DESIGNATOR
3841
3842@end verbatim
3843
3844Relative components better be either relative
3845or subdirectories of the path before them, or bust.
3846
3847@c FIXME: the following assumes that the reader is familiar with the use
3848@c of this pattern in logical pathnames, which may not be a reasonable
3849@c assumption.  Expand.
3850The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
3851You can specify more fine-grained patterns
3852by using a pathname object as the last component
3853e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
3854
3855You may use @code{#+features} to customize the configuration file.
3856
3857The second designator of a mapping may be @code{nil}, indicating that files are not mapped
3858to anything but themselves (same as if the second designator was the same as the first).
3859
3860When the first designator is @code{t},
3861the mapping always matches.
3862When the first designator starts with @code{:root},
3863the mapping matches any host and device.
3864In either of these cases, if the second designator
3865isn't @code{t} and doesn't start with @code{:root},
3866then strings indicating the host and pathname are somehow copied
3867in the beginning of the directory component of the source pathname
3868before it is translated.
3869
3870When the second designator is @code{t}, the mapping is the identity.
3871When the second designator starts with @code{:root},
3872the mapping preserves the host and device of the original pathname.
3873Notably, this allows you to map files
3874to a subdirectory of the whichever directory the file is in.
3875Though the syntax is not quite as easy to use as we'd like,
3876you can have an (source destination) mapping entry such as follows
3877in your configuration file,
3878or you may use @code{enable-asdf-binary-locations-compatibility}
3879with @code{:centralize-lisp-binaries nil}
3880which will do the same thing internally for you:
3881@lisp
3882#.(let ((wild-subdir
3883          (make-pathname :directory '(:relative :wild-inferiors)))
3884        (wild-file
3885          (make-pathname :name :wild :version :wild :type :wild)))
3886   `((:root ,wild-subdir ,wild-file)
3887     (:root ,wild-subdir :implementation ,wild-file)))
3888@end lisp
3889Starting with ASDF 2.011.4, you can use the simpler:
3890  @code{`(:root (:root :**/ :implementation :*.*.*))}
3891
3892
3893
3894@code{:include} statements cause the search to recurse with the path specifications
3895from the file specified.
3896
3897If the @code{translate-pathname} mechanism cannot achieve a desired
3898translation, the user may provide a function which provides the
3899required algorithm.  Such a translation function is specified by
3900supplying a list as the second @code{directory-designator}
3901the first element of which is the keyword @code{:function},
3902and the second element of which is
3903either a symbol which designates a function or a lambda expression.
3904The function designated by the second argument must take two arguments,
3905the first being the pathname of the source file,
3906the second being the wildcard that was matched.
3907When invoked, the function should return the translated pathname.
3908
3909An @code{:inherit-configuration} statement causes the search to recurse with the path
3910specifications from the next configuration in the bulleted list.
3911@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
3912
3913@vindex @code{asdf::*user-cache*}
3914@itemize
3915@item
3916@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
3917@item
3918@code{:disable-cache} is the same as @code{(t t)}.
3919@item
3920@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
3921which by default is the same as using
3922@code{(:home ".cache" "common-lisp" :implementation)}.
3923@end itemize
3924
3925
3926@node Output Configuration Directories, Output Shell-friendly syntax for configuration, Output Configuration DSL, Controlling where ASDF saves compiled files
3927@section Configuration Directories
3928
3929Configuration directories consist of files, each of which contains
3930a list of directives without any enclosing
3931@code{(:output-translations ...)} form.
3932The files will be sorted by namestring as if by @code{string<} and
3933the lists of directives of these files with be concatenated in order.
3934An implicit @code{:inherit-configuration} will be included
3935at the @emph{end} of the list.
3936
3937System-wide or per-user Common Lisp software distributions
3938such as Debian packages or some future version of @code{clbuild}
3939may then include files such as
3940@file{/etc/common-lisp/asdf-output-translations.conf.d/10-foo.conf} or
3941@file{~/.config/common-lisp/asdf-output-translations.conf.d/10-foo.conf}
3942to easily and modularly register configuration information
3943about software being distributed.
3944
3945The convention is that, for sorting purposes,
3946the names of files in such a directory begin with two digits
3947that determine the order in which these entries will be read.
3948Also, the type of these files must be @file{.conf},
3949which not only simplifies the implementation by allowing
3950for more portable techniques in finding those files,
3951but also makes it trivial to disable a file, by renaming it to a different file type.
3952
3953Directories may be included by specifying a directory pathname
3954or namestring in an @code{:include} directive, e.g.:
3955
3956@verbatim
3957  (:include "/foo/bar/")
3958@end verbatim
3959
3960@node Output Shell-friendly syntax for configuration, Semantics of Output Translations, Output Configuration Directories, Controlling where ASDF saves compiled files
3961@section Shell-friendly syntax for configuration
3962
3963When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
3964ASDF will skip to the next configuration if it's an empty string.
3965It will @code{READ} the string as an SEXP in the DSL
3966if it begins with a paren @code{(}
3967and it will be interpreted as a list of directories.
3968Directories should come by pairs, indicating a mapping directive.
3969Entries are separated
3970by a @code{:} (colon) on Unix platforms (including cygwin),
3971by a @code{;} (semicolon) on other platforms (mainly, Windows).
3972
3973The magic empty entry,
3974if it comes in what would otherwise be the first entry in a pair,
3975indicates the splicing of inherited configuration.
3976If it comes as the second entry in a pair,
3977it indicates that the directory specified first is to be left untranslated
3978(which has the same effect as if the directory had been repeated).
3979Thus @code{"/foo:/bar::/baz:"} means that
3980things under directory @file{/foo/}
3981are translated to be under @file{/bar/},
3982then include the inherited configuration,
3983then specify that things under directory @file{/baz/} are not translated.
3984
3985@node Semantics of Output Translations, Output Caching Results, Output Shell-friendly syntax for configuration, Controlling where ASDF saves compiled files
3986@section Semantics of Output Translations
3987
3988From the specified configuration,
3989a list of mappings is extracted in a straightforward way:
3990mappings are collected in order, recursing through
3991included or inherited configuration as specified.
3992To this list is prepended some implementation-specific mappings,
3993and is appended a global default.
3994
3995The list is then compiled to a mapping table as follows:
3996for each entry, in order, resolve the first designated directory
3997into an actual directory pathname for source locations.
3998If no mapping was specified yet for that location,
3999resolve the second designated directory to an output location directory
4000add a mapping to the table mapping the source location to the output location,
4001and add another mapping from the output location to itself
4002(unless a mapping already exists for the output location).
4003
4004Based on the table, a mapping function is defined,
4005mapping source pathnames to output pathnames:
4006given a source pathname, locate the longest matching prefix
4007in the source column of the mapping table.
4008Replace that prefix by the corresponding output column
4009in the same row of the table, and return the result.
4010If no match is found, return the source pathname.
4011(A global default mapping the filesystem root to itself
4012may ensure that there will always be a match,
4013with same fall-through semantics).
4014
4015@node Output Caching Results, Output location API, Semantics of Output Translations, Controlling where ASDF saves compiled files
4016@section Caching Results
4017
4018The implementation is allowed to either eagerly compute the information
4019from the configurations and file system, or to lazily re-compute it
4020every time, or to cache any part of it as it goes.
4021To explicitly flush any information cached by the system, use the API below.
4022
4023
4024@node Output location API, Credits for output translations, Output Caching Results, Controlling where ASDF saves compiled files
4025@section Output location API
4026
4027The specified functions are exported from package ASDF.
4028
4029@defun initialize-output-translations @Aoptional{} PARAMETER
4030   will read the configuration and initialize all internal variables.
4031   You may extend or override configuration
4032   from the environment and configuration files
4033   with the given @var{PARAMETER}, which can be
4034   @code{nil} (no configuration override),
4035   or a SEXP (in the SEXP DSL),
4036   a string (as in the string DSL),
4037   a pathname (of a file or directory with configuration),
4038   or a symbol (fbound to function that when called returns one of the above).
4039@end defun
4040
4041@defun disable-output-translations
4042   will initialize output translations in a way
4043   that maps every pathname to itself,
4044   effectively disabling the output translation facility.
4045@end defun
4046
4047@defun clear-output-translations
4048   undoes any output translation configuration
4049   and clears any cache for the mapping algorithm.
4050   You might want to call this function
4051   (or better, @code{clear-configuration})
4052   before you dump an image that would be resumed
4053   with a different configuration,
4054   and return an empty configuration.
4055   Note that this does not include clearing information about
4056   systems defined in the current image, only about
4057   where to look for systems not yet defined.
4058@end defun
4059
4060@defun ensure-output-translations @Aoptional{} PARAMETER
4061   checks whether output translations have been initialized.
4062   If not, initialize them with the given @var{PARAMETER}.
4063   This function will be called before any attempt to operate on a system.
4064@end defun
4065
4066@defun apply-output-translations PATHNAME
4067   Applies the configured output location translations to @var{PATHNAME}
4068   (calls @code{ensure-output-translations} for the translations).
4069@end defun
4070
4071Every time you use ASDF's @code{output-files}, or
4072anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
4073@code{ensure-output-translations} is called with parameter @code{nil},
4074which the first time around causes your configuration to be read.
4075If you change a configuration file,
4076you need to explicitly @code{initialize-output-translations} again,
4077or maybe @code{clear-output-translations} (or @code{clear-configuration}),
4078which will cause the initialization to happen next time around.
4079
4080
4081@node Credits for output translations,  , Output location API, Controlling where ASDF saves compiled files
4082@section Credits for output translations
4083
4084Thanks a lot to Peter van Eynde for @code{Common Lisp Controller}
4085and to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations}.
4086
4087All bad design ideas and implementation bugs are to mine, not theirs.
4088But so are good design ideas and elegant implementation tricks.
4089
4090 --- Francois-Rene Rideau @email{fare@@tunes.org}
4091
4092@c @section Default locations
4093@c @findex output-files-for-system-and-operation
4094
4095@c The default binary location for each Lisp implementation
4096@c is a subdirectory of each source directory.
4097@c To account for different Lisps, Operating Systems, Implementation versions,
4098@c and so on, ASDF borrows code from SLIME
4099@c to create reasonable custom directory names.
4100@c Here are some examples:
4101
4102@c @itemize
4103@c @item
4104@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
4105
4106@c @item
4107@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
4108
4109@c @item
4110@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
4111@c @end itemize
4112
4113@c By default, all output file pathnames will be relocated
4114@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
4115
4116@c See the document @file{README.asdf-output-translations}
4117@c for a full specification on how to configure @code{asdf-output-translations}.
4118
4119@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
4120@comment  node-name,  next,  previous,  up
4121@chapter Error handling
4122@findex SYSTEM-DEFINITION-ERROR
4123@findex OPERATION-ERROR
4124
4125@section ASDF errors
4126
4127If ASDF detects an incorrect system definition, it will signal a generalised instance of
4128@code{SYSTEM-DEFINITION-ERROR}.
4129
4130Operations may go wrong (for example when source files contain errors).
4131These are signalled using generalised instances of
4132@code{OPERATION-ERROR}.
4133
4134@section Compilation error and warning handling
4135@vindex *compile-file-warnings-behaviour*
4136@vindex *compile-file-errors-behavior*
4137
4138ASDF checks for warnings and errors when a file is compiled.
4139The variables @var{*compile-file-warnings-behaviour*} and
4140@var{*compile-file-errors-behavior*}
4141control the handling of any such events.
4142The valid values for these variables are
4143@code{:error}, @code{:warn}, and @code{:ignore}.
4144
4145@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
4146@comment  node-name,  next,  previous,  up
4147@chapter Miscellaneous additional functionality
4148
4149ASDF includes several additional features that are generally
4150useful for system definition and development.
4151
4152@menu
4153* Controlling file compilation::
4154* Controlling source file character encoding::
4155* Some Utility Functions::
4156@end menu
4157
4158@node Controlling file compilation, Controlling source file character encoding, Miscellaneous additional functionality, Miscellaneous additional functionality
4159@section Controlling file compilation
4160@cindex :around-compile
4161@cindex around-compile keyword
4162@cindex compile-check keyword
4163@cindex :compile-check
4164@findex compile-file*
4165
4166@c FIXME: Needs rewrite.  Start with motivation -- why are we doing
4167@c this?  (there is some, but it's buried).  Also, all of a sudden in
4168@c the middle of the discussion we start talking about a "hook," which
4169@c is confusing.
4170
4171When declaring a component (system, module, file),
4172you can specify a keyword argument @code{:around-compile function}.
4173If left unspecified (and therefore unbound),
4174the value will be inherited from the parent component if any,
4175or with a default of @code{nil}
4176if no value is specified in any transitive parent.
4177
4178The argument must be either @code{nil}, an fbound symbol,
4179a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
4180a function object (e.g. using @code{#.#'} but that's discouraged
4181because it prevents the introspection done by e.g. asdf-dependency-grovel),
4182or a string that when @code{read} yields a symbol or a lambda-expression.
4183@code{nil} means the normal compile-file function will be called.
4184A non-nil value designates a function of one argument
4185that will be called with a function that will
4186invoke @code{compile-file*} with various arguments;
4187the around-compile hook may supply additional keyword arguments
4188to pass to that call to @code{compile-file*}.
4189
4190One notable argument that is heeded by @code{compile-file*} is
4191@code{:compile-check},
4192a function called when the compilation was otherwise a success,
4193with the same arguments as @code{compile-file};
4194the function shall return true if the compilation
4195and its resulting compiled file respected all system-specific invariants,
4196and false (@code{nil}) if it broke any of those invariants;
4197it may issue warnings or errors before it returns @code{nil}.
4198(NB: The ability to pass such extra flags
4199is only available starting with ASDF 2.22.3.)
4200This feature is notably exercised by asdf-finalizers.
4201
4202By using a string, you may reference
4203a function, symbol and/or package
4204that will only be created later during the build, but
4205isn't yet present at the time the defsystem form is evaluated.
4206However, if your entire system is using such a hook, you may have to
4207explicitly override the hook with @code{nil} for all the modules and files
4208that are compiled before the hook is defined.
4209
4210Using this hook, you may achieve such effects as:
4211locally renaming packages,
4212binding @var{*readtables*} and other syntax-controlling variables,
4213handling warnings and other conditions,
4214proclaiming consistent optimization settings,
4215saving code coverage information,
4216maintaining meta-data about compilation timings,
4217setting gensym counters and PRNG seeds and other sources of non-determinism,
4218overriding the source-location and/or timestamping systems,
4219checking that some compile-time side-effects were properly balanced,
4220etc.
4221
4222Note that there is no around-load hook. This is on purpose.
4223Some implementations such as ECL, GCL or MKCL link object files,
4224which allows for no such hook.
4225Other implementations allow for concatenating FASL files,
4226which doesn't allow for such a hook either.
4227We aim to discourage something that's not portable,
4228and has some dubious impact on performance and semantics
4229even when it is possible.
4230Things you might want to do with an around-load hook
4231are better done around-compile,
4232though it may at times require some creativity
4233(see e.g. the @code{package-renaming} system).
4234
4235
4236@node Controlling source file character encoding, Some Utility Functions, Controlling file compilation, Miscellaneous additional functionality
4237@section Controlling source file character encoding
4238
4239Starting with ASDF 2.21, components accept a @code{:encoding} option
4240so authors may specify which character encoding should be used
4241to read and evaluate their source code.
4242When left unspecified, the encoding is inherited
4243from the parent module or system;
4244if no encoding is specified at any point,
4245or if @code{nil} is explicitly specified,
4246an extensible protocol described below is followed,
4247that ultimately defaults to @code{:utf-8} since ASDF 3.
4248
4249The protocol to determine the encoding is
4250to call the function @code{detect-encoding},
4251which itself, if provided a valid file,
4252calls the function specified by @var{*encoding-detection-hook*},
4253or else defaults to the @var{*default-encoding*}.
4254The @var{*encoding-detection-hook*} is by default bound
4255to function @code{always-default-encoding},
4256that always returns the contents of @var{*default-encoding*}.
4257@var{*default-encoding*} is bound to @code{:utf-8} by default
4258(before ASDF 3, the default was @code{:default}).
4259
4260Whichever encoding is returned must be a portable keyword,
4261that will be translated to an implementation-specific external-format designator
4262by function @code{encoding-external-format},
4263which itself simply calls the function specified @var{*encoding-external-format-hook*};
4264that function by default is @code{default-encoding-external-format},
4265that only recognizes @code{:utf-8} and @code{:default},
4266and translates the former to the implementation-dependent @var{*utf-8-external-format*},
4267and the latter to itself (that itself is portable but has an implementation-dependent meaning).
4268
4269In other words, there now are plenty of extension hooks, but
4270by default ASDF enforces the previous @emph{de facto} standard behavior
4271of using @code{:utf-8}, independently from
4272whatever configuration the user may be using.
4273Thus, system authors can now rely on @code{:utf-8}
4274being used while compiling their files,
4275even if the user is currently using @code{:koi8-r} or @code{:euc-jp}
4276as their interactive encoding.
4277(Before ASDF 3, there was no such guarantee, @code{:default} was used,
4278and only plain ASCII was safe to include in source code.)
4279
4280Some legacy implementations only support 8-bit characters,
4281and some implementations provide 8-bit only variants.
4282On these implementations, the @var{*utf-8-external-format*}
4283gracefully falls back to @code{:default},
4284and Unicode characters will be read as multi-character mojibake.
4285To detect such situations, UIOP will push the @code{:asdf-unicode} feature
4286on implementations that support Unicode, and you can use reader-conditionalization
4287to protect any @code{:encoding @emph{encoding}} statement, as in
4288@code{#+asdf-unicode :encoding #+asdf-unicode :utf-8}.
4289We recommend that you avoid using unprotected @code{:encoding} specifications
4290until after ASDF 2.21 or later becomes widespread
4291(in April 2014, only LispWorks lags with ASDF 2.019,
4292and is scheduled to be updated later this year).
4293
4294While it offers plenty of hooks for extension,
4295and one such extension is available (see @code{asdf-encodings} below),
4296ASDF itself only recognizes one encoding beside @code{:default},
4297and that is @code{:utf-8}, which is the @emph{de facto} standard,
4298already used by the vast majority of libraries that use more than ASCII.
4299On implementations that do not support unicode,
4300the feature @code{:asdf-unicode} is absent, and
4301the @code{:default} external-format is used
4302to read even source files declared as @code{:utf-8}.
4303On these implementations, non-ASCII characters
4304intended to be read as one CL character
4305may thus end up being read as multiple CL characters.
4306In most cases, this shouldn't affect the software's semantics:
4307comments will be skipped just the same, strings with be read and printed
4308with slightly different lengths, symbol names will be accordingly longer,
4309but none of it should matter.
4310But a few systems that actually depend on unicode characters
4311may fail to work properly, or may work in a subtly different way.
4312See for instance @code{lambda-reader}.
4313
4314We invite you to embrace UTF-8
4315as the encoding for non-ASCII characters starting today,
4316even without any explicit specification in your @file{.asd} files.
4317Indeed, on some implementations and configurations,
4318UTF-8 is already the @code{:default},
4319and loading your code may cause errors if it is encoded in anything but UTF-8.
4320Therefore, even with the legacy behavior,
4321non-UTF-8 is guaranteed to break for some users,
4322whereas UTF-8 is pretty much guaranteed not to break anywhere
4323(provided you do @emph{not} use a BOM),
4324although it might be read incorrectly on some implementations.
4325@code{:utf-8} has been the default value of @code{*default-encoding*} since ASDF 3.
4326
4327If you need non-standard character encodings for your source code,
4328use the extension system @code{asdf-encodings}, by specifying
4329@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
4330This extension system will register support for more encodings using the
4331@code{*encoding-external-format-hook*} facility,
4332so you can explicitly specify @code{:encoding :latin1}
4333in your @file{.asd} file.
4334Using the @code{*encoding-detection-hook*} it will also
4335eventually implement some autodetection of a file's encoding
4336from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
4337or otherwise based on an analysis of octet patterns in the file.
4338At this point, @code{asdf-encoding} only supports the encodings
4339that are supported as part of your implementation.
4340Since the list varies depending on implementations,
4341we still recommend you use @code{:utf-8} everywhere,
4342which is the most portable (next to it is @code{:latin1}).
4343
4344Recent versions of Quicklisp include @code{asdf-encodings};
4345if you're not using it, you may get this extension using git:
4346@kbd{git clone git://common-lisp.net/projects/asdf/asdf-encodings.git}
4347or
4348@kbd{git clone ssh://common-lisp.net/project/asdf/git/asdf-encodings.git}.
4349You can also browse the repository on
4350@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git}.
4351
4352When you use @code{asdf-encodings},
4353any @file{.asd} file loaded
4354will use the autodetection algorithm to determine its encoding.
4355If you depend on this detection happening,
4356you should explicitly load @code{asdf-encodings} early in your build.
4357Note that @code{:defsystem-depends-on} cannot be used here: by the time
4358the @code{:defsystem-depends-on} is loaded, the enclosing
4359@code{defsystem} form has already been read.
4360
4361In practice, this means that the @code{*default-encoding*}
4362is usually used for @file{.asd} files.
4363Currently, this defaults to @code{:utf-8}, and
4364you should be safe using Unicode characters in those files.
4365This might matter, for instance, in meta-data about author's names.
4366Otherwise, the main data in these files is component (path)names,
4367and we don't recommend using non-ASCII characters for these,
4368for the result probably isn't very portable.
4369
4370@section Miscellaneous Functions
4371
4372These functions are exported by ASDF for your convenience.
4373
4374@anchor{system-relative-pathname}
4375@defun system-relative-pathname system name @Akey{} type
4376
4377It's often handy to locate a file relative to some system.
4378The @code{system-relative-pathname} function meets this need.
4379
4380It takes two mandatory arguments @var{system} and @var{name}
4381and a keyword argument @var{type}:
4382@var{system} is name of a system, whereas @var{name} and optionally @var{type}
4383specify a relative pathname, interpreted like a component pathname specifier
4384by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
4385
4386It returns a pathname built from the location of the system's
4387source directory and the relative pathname. For example:
4388
4389@lisp
4390> (asdf:system-relative-pathname 'cl-ppcre "regex.data")
4391#P"/repository/other/cl-ppcre/regex.data"
4392@end lisp
4393
4394@end defun
4395
4396@defun system-source-directory system-designator
4397
4398ASDF does not provide a turnkey solution for locating
4399data (or other miscellaneous) files
4400that are distributed together with the source code of a system.
4401Programmers can use @code{system-source-directory} to find such files.
4402Returns a pathname object.
4403The @var{system-designator} may be a string, symbol, or ASDF system object.
4404@end defun
4405
4406@defun clear-system system-designator
4407
4408It is sometimes useful to force recompilation of a previously loaded system.
4409For these cases, @code{(asdf:clear-system :foo)}
4410will remove the system from the table of currently loaded systems:
4411the next time the system @code{foo} or one that depends on it is re-loaded,
4412@code{foo} will be loaded again.@footnote{Alternatively, you could touch @code{foo.asd} or
4413remove the corresponding fasls from the output file cache.}
4414
4415Note that this does not and cannot undo
4416the previous loading of the system.
4417Common Lisp has no provision for such an operation,
4418and its reliance on irreversible side-effects to global data structures
4419makes such a thing impossible in the general case.
4420If the software being re-loaded is not conceived with hot upgrade in mind,
4421re-loading may cause many errors, warnings or subtle silent problems,
4422as packages, generic function signatures, structures, types, macros, constants, etc.
4423are being redefined incompatibly.
4424It is up to the user to make sure that reloading is possible and has the desired effect.
4425In some cases, extreme measures such as recursively deleting packages,
4426unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
4427and much more are necessary for reloading to happen smoothly.
4428ASDF itself goes to extensive effort to make a hot upgrade possible
4429with respect to its own code.
4430If you want, you can reuse some of its utilities such as
4431@code{uiop:define-package} and @code{uiop:with-upgradability},
4432and get inspiration (or disinspiration)
4433from what it does in @file{header.lisp} and @file{upgrade.lisp}.
4434@end defun
4435
4436@defun register-preloaded-system name @Arest{} keys
4437A system with name @var{name},
4438created by @code{make-instance} with extra keys @var{keys}
4439(e.g. @code{:version}),
4440is registered as @emph{preloaded}.
4441That is, its code has already been loaded into the current image,
4442and if at some point some other system @code{:depends-on} it yet no source code is found,
4443it is considered as already provided,
4444and ASDF will not raise a @code{missing-component} error.
4445
4446This function is particularly useful if you distribute your code
4447as fasls with either @code{compile-bundle-op} or @code{monolithic-compile-bundle-op},
4448and want to register systems so that dependencies will work uniformly
4449whether you're using your software from source or from fasl.
4450@end defun
4451
4452@defun run-shell-command control-string @Arest{} args
4453
4454This function is obsolete and present only for the sake of backwards-compatibility:
4455``If it's not backwards, it's not compatible''. We @emph{strongly} discourage its use.
4456Its current behavior is only well-defined on Unix platforms
4457(which include MacOS X and cygwin). On Windows, anything goes.
4458The following documentation is only for the purpose of your migrating away from it
4459in a way that preserves semantics.
4460
4461Instead we recommend the use @code{run-program}, described in the next section, and
4462available as part of ASDF since ASDF 3.
4463
4464@code{run-shell-command} takes as arguments a format @code{control-string}
4465and arguments to be passed to @code{format} after this control-string
4466to produce a string.
4467This string is a command that will be evaluated with a POSIX shell if possible;
4468yet, on Windows, some implementations will use CMD.EXE,
4469while others (like SBCL) will make an attempt at invoking a POSIX shell
4470(and fail if it is not present).
4471@end defun
4472
4473@node Some Utility Functions,  , Controlling source file character encoding, Miscellaneous additional functionality
4474@section Some Utility Functions
4475
4476The below functions are not exported by ASDF itself, but by UIOP, available since ASDF 3.
4477Some of them have precursors in ASDF 2, but we recommend
4478you rely on ASDF 3 for active developments.
4479UIOP provides many, many more utility functions, and we recommend
4480you read its README and sources for more information.
4481
4482
4483@defun parse-unix-namestring name @Akey{} type defaults dot-dot ensure-directory @AallowOtherKeys
4484Coerce NAME into a PATHNAME using standard Unix syntax.
4485
4486Unix syntax is used whether or not the underlying system is Unix;
4487on non-Unix systems it is only usable for relative pathnames.
4488In order to manipulate relative pathnames portably, it is crucial
4489to possess a portable pathname syntax independent of the underlying OS.
4490This is what @code{parse-unix-namestring} provides, and why we use it in ASDF.
4491
4492When given a @code{pathname} object, just return it untouched.
4493When given @code{nil}, just return @code{nil}.
4494When given a non-null @code{symbol}, first downcase its name and treat it as a string.
4495When given a @code{string}, portably decompose it into a pathname as below.
4496
4497@code{#\/} separates directory components.
4498
4499The last @code{#\/}-separated substring is interpreted as follows:
45001- If @var{type} is @code{:directory} or @var{ensure-directory} is true,
4501 the string is made the last directory component, and its @code{name} and @code{type} are @code{nil}.
4502 if the string is empty, it's the empty pathname with all slots @code{nil}.
45032- If @var{type} is @code{nil}, the substring is a file-namestring,
4504 and its @code{name} and @code{type} are separated by @code{split-name-type}.
45053- If @var{type} is a string, it is the given @code{type}, and the whole string is the @code{name}.
4506
4507Directory components with an empty name the name @code{.} are removed.
4508Any directory named @code{..} is read as @var{dot-dot},
4509which must be one of @code{:back} or @code{:up} and defaults to @code{:back}.
4510
4511@vindex *nil-pathname*
4512@code{host}, @code{device} and @code{version} components are taken from @var{defaults},
4513which itself defaults to @code{*nil-pathname*}.
4514@code{*nil-pathname*} is also used if @var{defaults} is @code{nil}.
4515No host or device can be specified in the string itself,
4516which makes it unsuitable for absolute pathnames outside Unix.
4517
4518For relative pathnames, these components (and hence the defaults) won't matter
4519if you use @code{merge-pathnames*} but will matter if you use @code{merge-pathnames},
4520which is an important reason to always use @code{merge-pathnames*}.
4521
4522Arbitrary keys are accepted, and the parse result is passed to @code{ensure-pathname}
4523with those keys, removing @var{type}, @var{defaults} and @var{dot-dot}.
4524When you're manipulating pathnames that are supposed to make sense portably
4525even though the OS may not be Unixish, we recommend you use @code{:want-relative t}
4526so that @code{parse-unix-namestring} will throw an error if the pathname is absolute.
4527@end defun
4528
4529@defun merge-pathnames* specified @Aoptional{} defaults
4530
4531This function is a replacement for @code{merge-pathnames} that uses the host and device
4532from the @var{defaults} rather than the @var{specified} pathname when the latter
4533is a relative pathname. This allows ASDF and its users to create and use relative pathnames
4534without having to know beforehand what are the host and device
4535of the absolute pathnames they are relative to.
4536
4537@end defun
4538
4539@defun subpathname pathname subpath @Akey{} type
4540
4541This function takes a @var{pathname} and a @var{subpath} and a @var{type}.
4542If @var{subpath} is already a @code{pathname} object (not namestring),
4543and is an absolute pathname at that, it is returned unchanged;
4544otherwise, @var{subpath} is turned into a relative pathname with given @var{type}
4545as per @code{parse-unix-namestring} with @code{:want-relative t :type }@var{type},
4546then it is merged with the @code{pathname-directory-pathname} of @var{pathname},
4547as per @code{merge-pathnames*}.
4548
4549We strongly encourage the use of this function
4550for portably resolving relative pathnames in your code base.
4551@end defun
4552
4553@defun subpathname* pathname subpath @Akey{} type
4554
4555This function returns @code{nil} if the base @var{pathname} is @code{nil},
4556otherwise acts like @code{subpathname}.
4557@end defun
4558
4559@defun run-program command @Akey{} ignore-error-status force-shell input output @
4560error-output if-input-does-not-exist if-output-exists if-error-output-exists @
4561element-type external-format @AallowOtherKeys
4562
4563@code{run-program} takes a @var{command} argument that is either
4564a list of a program name or path and its arguments,
4565or a string to be executed by a shell.
4566It spawns the command, waits for it to return,
4567verifies that it exited cleanly (unless told not too below),
4568and optionally captures and processes its output.
4569It accepts many keyword arguments to configure its behavior.
4570
4571@code{run-program} returns three values: the first for the output,
4572the second for the error-output, and the third for the return value.
4573(Beware that before ASDF 3.0.2.11, it didn't handle input or error-output,
4574and returned only one value,
4575the one for the output if any handler was specified, or else the exit code;
4576please upgrade ASDF, or at least UIOP, to rely on the new enhanced behavior.)
4577
4578@var{output} is its most important argument;
4579it specifies how the output is captured and processed.
4580If it is @code{nil}, then the output is redirected to the null device,
4581that will discard it.
4582If it is @code{:interactive}, then it is inherited from the current process
4583(beware: this may be different from your @var{*standard-output*},
4584and under SLIME will be on your @code{*inferior-lisp*} buffer).
4585If it is @code{t}, output goes to your current @var{*standard-output*} stream.
4586Otherwise, @var{output} should be a value that is a suitable first argument to
4587@code{slurp-input-stream} (see below), or
4588a list of such a value and keyword arguments.
4589In this case, @code{run-program} will
4590create a temporary stream for the program output;
4591the program output, in that stream,
4592will be processed by a call to @code{slurp-input-stream},
4593using @var{output} as the first argument
4594(or if it's a list the first element of @var{output} and the rest as keywords).
4595The primary value resulting from that call
4596(or @code{nil} if no call was needed)
4597will be the first value returned by @code{run-program}.
4598E.g., using @code{:output :string}
4599will have it return the entire output stream as a string.
4600And using @code{:output '(:string :stripped t)}
4601will have it return the same string stripped of any ending newline.
4602
4603@var{error-output} is similar to @var{output}, except that
4604the resulting value is returned as the second value of @code{run-program}.
4605@code{t} designates the @var{*error-output*}.
4606Also @code{:output} means redirecting the error output to the output stream,
4607in which case @code{nil} is returned.
4608
4609@var{input} is similar to @var{output}, except that
4610@code{vomit-output-stream} is used, no value is returned,
4611and @code{t} designates the @var{*standard-input*}.
4612
4613@code{element-type} and @code{external-format} are passed on
4614to your Lisp implementation, when applicable, for creation of the output stream.
4615
4616One and only one of the stream slurping or vomiting may or may not happen
4617in parallel in parallel with the subprocess,
4618depending on options and implementation,
4619and with priority being given to output processing.
4620Other streams are completely produced or consumed
4621before or after the subprocess is spawned, using temporary files.
4622
4623@code{force-shell} forces evaluation of the command through a shell,
4624even if it was passed as a list rather than a string.
4625If a shell is used, it is @file{/bin/sh} on Unix or @file{CMD.EXE} on Windows,
4626except on implementations that (erroneously, IMNSHO)
4627insist on consulting @code{$SHELL} like clisp.
4628
4629@code{ignore-error-status} causes @code{run-program}
4630to not raise an error if the spawned program exits in error.
4631Following POSIX convention, an error is anything but
4632a normal exit with status code zero.
4633By default, an error of type @code{subprocess-error} is raised in this case.
4634
4635@code{run-program} works on all platforms supported by ASDF, except Genera.
4636See the source code for more documentation.
4637
4638@end defun
4639
4640@defun slurp-input-stream processor input-stream @Akey{}
4641
4642@code{slurp-input-stream} is a generic function of two arguments, a target object and an input stream,
4643and accepting keyword arguments.
4644Predefined methods based on the target object are as follows:
4645
4646@itemize
4647@item
4648If the object is a function, the function is called with the stream as argument.
4649
4650@item If the object is a cons, its first element is applied to its rest appended by
4651a list of the input stream.
4652
4653@item If the object is an output stream, the contents of the input stream are copied to it.
4654If the @var{linewise} keyword argument is provided, copying happens line by line,
4655and an optional @var{prefix} is printed before each line.
4656Otherwise, copying happen based on a buffer of size @var{buffer-size},
4657using the specified @var{element-type}.
4658
4659@item If the object is @code{'string} or @code{:string}, the content is captured into a string.
4660Accepted keywords include the @var{element-type} and a flag @var{stripped},
4661which when true causes any single line ending to be removed as per @code{uiop:stripln}.
4662
4663@item If the object is @code{:lines}, the content is captured as a list of strings,
4664one per line, without line ending. If the @var{count} keyword argument is provided,
4665it is a maximum count of lines to be read.
4666
4667@item If the object is @code{:line}, the content is captured as with @code{:lines} above,
4668and then its sub-object is extracted with the @var{at} argument,
4669which defaults to @code{0}, extracting the first line.
4670A number will extract the corresponding line.
4671See the documentation for @code{uiop:access-at}.
4672
4673@item If the object is @code{:forms}, the content is captured as a list of S-expressions,
4674as read by the Lisp reader.
4675If the @var{count} argument is provided,
4676it is a maximum count of lines to be read.
4677We recommend you control the syntax with such macro as
4678@code{uiop:with-safe-io-syntax}.
4679
4680@item If the object is @code{:form}, the content is captured as with @code{:forms} above,
4681and then its sub-object is extracted with the @var{at} argument,
4682which defaults to @code{0}, extracting the first form.
4683A number will extract the corresponding form.
4684See the documentation for @code{uiop:access-at}.
4685We recommend you control the syntax with such macro as
4686@code{uiop:with-safe-io-syntax}.
4687@end itemize
4688@end defun
4689
4690
4691@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
4692@comment  node-name,  next,  previous,  up
4693@chapter Getting the latest version
4694
4695Decide which version you want.
4696The @code{master} branch is where development happens;
4697its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
4698but an occasional regression may happen despite our (limited) test suite.
4699
4700The @code{release} branch is what cautious people should be using;
4701it has usually been tested more, and releases are cut at a point
4702where there isn't any known unresolved issue.
4703
4704You may get the ASDF source repository using git:
4705@kbd{git clone git://common-lisp.net/projects/asdf/asdf.git}
4706
4707You will find the above referenced tags in this repository.
4708You can also browse the repository on
4709@url{http://common-lisp.net/gitweb?p=projects/asdf/asdf.git}.
4710
4711Discussion of ASDF development is conducted on the
4712mailing list
4713@kbd{asdf-devel@@common-lisp.net}.
4714@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel}
4715
4716
4717@node FAQ, Ongoing Work, Getting the latest version, Top
4718@comment  node-name,  next,  previous,  up
4719@chapter FAQ
4720
4721@menu
4722* Where do I report a bug?::
4723* What has changed between ASDF 1 and ASDF 2?::
4724* Issues with installing the proper version of ASDF::
4725* Issues with configuring ASDF::
4726* Issues with using and extending ASDF to define systems::
4727* ASDF development FAQs::
4728@end menu
4729
4730@node Where do I report a bug?, What has changed between ASDF 1 and ASDF 2?, FAQ, FAQ
4731@section  ``Where do I report a bug?''
4732
4733ASDF bugs are tracked on launchpad: @url{https://launchpad.net/asdf}.
4734
4735If you're unsure about whether something is a bug, or for general discussion,
4736use the @url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list}
4737
4738
4739@node What has changed between ASDF 1 and ASDF 2?, Issues with installing the proper version of ASDF, Where do I report a bug?, FAQ
4740@section ``What has changed between ASDF 1, ASDF 2 and ASDF 3?''
4741
4742We released ASDF 2.000 on May 31st 2010,
4743and ASDF 3.0.0 on May 15th 2013.
4744Releases of ASDF 2 and later have since then been included
4745in all actively maintained CL implementations that used to bundle ASDF 1,
4746plus some implementations that previously did not.
4747ASDF has been made to work with all actively maintained CL
4748implementations and even a few implementations that are @emph{not}
4749actively maintained.
4750@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
4751Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly
4752(though we recommend instead upgrading your implementation or its ASDF module).
4753For this reason, we have stopped supporting ASDF 1 and ASDF 2.
4754If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
4755we recommend you upgrade to ASDF 3
4756--- and we explain how to do that. @xref{Loading ASDF}.
4757(In the context of compatibility requirements,
4758ASDF 2.27, released on Feb 1st 2013, and further 2.x releases up to 2.33,
4759count as pre-releases of ASDF 3, and define the @code{:asdf3} feature;
4760still, please use the latest release).
4761Release ASDF 3.1.2 and later also define the @code{:asdf3.1} feature.
4762
4763
4764@menu
4765* What are ASDF 1 2 3?::
4766* How do I detect the ASDF version?::
4767* ASDF can portably name files in subdirectories::
4768* Output translations::
4769* Source Registry Configuration::
4770* Usual operations are made easier to the user::
4771* Many bugs have been fixed::
4772* ASDF itself is versioned::
4773* ASDF can be upgraded::
4774* Decoupled release cycle::
4775* Pitfalls of the transition to ASDF 2::
4776@end menu
4777
4778@node What are ASDF 1 2 3?, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?, What has changed between ASDF 1 and ASDF 2?
4779@subsection What are ASDF 1, ASDF 2, and ASDF 3?
4780
4781ASDF 1 refers to any release earlier than 1.369 or so (from August 2001 to October 2009),
4782and to any development revision earlier than 2.000 (May 2010).
4783If your copy of ASDF doesn't even contain version information, it's an old ASDF 1.
4784Revisions between 1.656 and 1.728 may count as development releases for ASDF 2.
4785
4786ASDF 2 refers to releases from 2.000 (May 31st 2010) to 2.26 (Oct 30 2012),
4787and any development revision newer than ASDF 1 and older than 2.27 (Feb 1 2013).
4788
4789ASDF 3 refers to releases from 2.27 (Feb 1 2013) to 2.33 and 3.0.0 onward (May 15 2013).
47902.27 to 2.33 count as pre-releases to ASDF 3.
4791
4792@node How do I detect the ASDF version?, ASDF can portably name files in subdirectories, What are ASDF 1 2 3?, What has changed between ASDF 1 and ASDF 2?
4793@subsection How do I detect the ASDF version?
4794@findex asdf-version
4795@cindex *features*
4796
4797All releases of ASDF
4798push @code{:asdf} onto @code{*features*}.
4799Releases starting with ASDF 2
4800push @code{:asdf2} onto @code{*features*}.
4801Releases starting with ASDF 3 (including 2.27 and later pre-releases)
4802push @code{:asdf3} onto @code{*features*}.
4803Furthermore, releases starting with ASDF 3.1.2 (May 2014),
4804though they count as ASDF 3, include enough progress that they
4805push @code{:asdf3.1} onto @code{*features*}.
4806You may depend on the presence or absence of these features
4807to write code that takes advantage of recent ASDF functionality
4808but still works on older versions, or at least detects the old version and signals an error.
4809
4810Additionally, all releases starting with ASDF 2
4811define a function @code{(asdf:asdf-version)} you may use to query the version.
4812All releases starting with 2.013 display the version number prominently
4813on the second line of the @file{asdf.lisp} source file.
4814
4815If you are experiencing problems or limitations of any sort with ASDF 1 or ASDF 2,
4816we recommend that you should upgrade to the latest release, be it ASDF 3 or other.
4817
4818
4819@node ASDF can portably name files in subdirectories, Output translations, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?
4820@subsection ASDF can portably name files in subdirectories
4821
4822Common Lisp namestrings are not portable,
4823except maybe for logical pathname namestrings,
4824that themselves have various limitations and require a lot of setup
4825that is itself ultimately non-portable.
4826
4827In ASDF 1, the only portable ways to refer to pathnames inside systems and components
4828were very awkward, using @code{#.(make-pathname ...)} and
4829@code{#.(merge-pathnames ...)}.
4830Even the above were themselves were inadequate in the general case
4831due to host and device issues, unless horribly complex patterns were used.
4832Plenty of simple cases that looked portable actually weren't,
4833leading to much confusion and greavance.
4834
4835ASDF 2 implements its own portable syntax for strings as pathname specifiers.
4836Naming files within a system definition becomes easy and portable again.
4837@xref{Miscellaneous additional functionality,system-relative-pathname},
4838@code{merge-pathnames*},
4839@code{coerce-pathname}.
4840
4841On the other hand, there are places where systems used to accept namestrings
4842where you must now use an explicit pathname object:
4843@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4844must now be written with the @code{#p} syntax:
4845@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4846
4847@xref{The defsystem grammar,,Pathname specifiers}.
4848
4849
4850@node Output translations, Source Registry Configuration, ASDF can portably name files in subdirectories, What has changed between ASDF 1 and ASDF 2?
4851@subsection Output translations
4852
4853A popular feature added to ASDF was output pathname translation:
4854@code{asdf-binary-locations}, @code{common-lisp-controller},
4855@code{cl-launch} and other hacks were all implementing it in ways
4856both mutually incompatible and difficult to configure.
4857
4858Output pathname translation is essential to share
4859source directories of portable systems across multiple implementations
4860or variants thereof,
4861or source directories of shared installations of systems across multiple users,
4862or combinations of the above.
4863
4864In ASDF 2, a standard mechanism is provided for that,
4865@code{asdf-output-translations},
4866with sensible defaults, adequate configuration languages,
4867a coherent set of configuration files and hooks,
4868and support for non-Unix platforms.
4869
4870@xref{Controlling where ASDF saves compiled files}.
4871
4872@node Source Registry Configuration, Usual operations are made easier to the user, Output translations, What has changed between ASDF 1 and ASDF 2?
4873@subsection Source Registry Configuration
4874
4875Configuring ASDF used to require special magic
4876to be applied just at the right moment,
4877between the moment ASDF is loaded and the moment it is used,
4878in a way that is specific to the user,
4879the implementation he is using and the application he is building.
4880
4881This made for awkward configuration files and startup scripts
4882that could not be shared between users, managed by administrators
4883or packaged by distributions.
4884
4885ASDF 2 provides a well-documented way to configure ASDF,
4886with sensible defaults, adequate configuration languages,
4887and a coherent set of configuration files and hooks.
4888
4889We believe it's a vast improvement because it decouples
4890application distribution from library distribution.
4891The application writer can avoid thinking where the libraries are,
4892and the library distributor (dpkg, clbuild, advanced user, etc.)
4893can configure them once and for every application.
4894Yet settings can be easily overridden where needed,
4895so whoever needs control has exactly as much as required.
4896
4897At the same time, ASDF 2 remains compatible
4898with the old magic you may have in your build scripts
4899(using @code{*central-registry*} and
4900@code{*system-definition-search-functions*})
4901to tailor the ASDF configuration to your build automation needs,
4902and also allows for new magic, simpler and more powerful magic.
4903
4904@xref{Controlling where ASDF searches for systems}.
4905
4906
4907@node Usual operations are made easier to the user, Many bugs have been fixed, Source Registry Configuration, What has changed between ASDF 1 and ASDF 2?
4908@subsection Usual operations are made easier to the user
4909
4910In ASDF 1, you had to use the awkward syntax
4911@code{(asdf:oos 'asdf:load-op :foo)}
4912to load a system,
4913and similarly for @code{compile-op}, @code{test-op}.
4914
4915In ASDF 2, you can use shortcuts for the usual operations:
4916@code{(asdf:load-system :foo)}, and
4917similarly for @code{compile-system}, @code{test-system}.
4918
4919
4920@node Many bugs have been fixed, ASDF itself is versioned, Usual operations are made easier to the user, What has changed between ASDF 1 and ASDF 2?
4921@subsection Many bugs have been fixed
4922
4923The following issues and many others have been fixed:
4924
4925@itemize
4926@item
4927The infamous TRAVERSE function has been revamped completely
4928between ASDF 1 and ASDF 2, with many bugs squashed.
4929In particular, dependencies were not correctly propagated
4930across modules but now are.
4931It has been completely rewritten many times over
4932between ASDF 2.000 and ASDF 3,
4933with fundamental issues in the original model being fixed.
4934Timestamps were not propagated at all, and now are.
4935The internal model of how actions depend on each other
4936is now both consistent and complete.
4937The @code{:version} and
4938the @code{:force (system1 .. systemN)} feature have been fixed.
4939
4940@item
4941Performance has been notably improved for large systems
4942(say with thousands of components) by using
4943hash-tables instead of linear search,
4944and linear-time list accumulation instead of cubic time recursive append,
4945for an overall @emph{O(n)} complexity vs @emph{O(n^4)}.
4946
4947@item
4948Many features used to not be portable,
4949especially where pathnames were involved.
4950Windows support was notably quirky because of such non-portability.
4951
4952@item
4953The internal test suite used to massively fail on many implementations.
4954While still incomplete, it now fully passes
4955on all implementations supported by the test suite,
4956though some tests are commented out on a few implementations.
4957
4958@item
4959Support was lacking for some implementations.
4960ABCL and GCL were notably wholly broken.
4961ECL extensions were not integrated with ASDF release.
4962
4963@item
4964The documentation was grossly out of date.
4965
4966@end itemize
4967
4968
4969@node ASDF itself is versioned, ASDF can be upgraded, Many bugs have been fixed, What has changed between ASDF 1 and ASDF 2?
4970@subsection ASDF itself is versioned
4971
4972Between new features, old bugs fixed, and new bugs introduced,
4973there were various releases of ASDF in the wild,
4974and no simple way to check which release had which feature set.
4975People using or writing systems had to either make worst-case assumptions
4976as to what features were available and worked,
4977or take great pains to have the correct version of ASDF installed.
4978
4979With ASDF 2, we provide a new stable set of working features
4980that everyone can rely on from now on.
4981Use @code{#+asdf2} to detect presence of ASDF 2,
4982@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
4983to check the availability of a version no earlier than required.
4984
4985
4986@node ASDF can be upgraded, Decoupled release cycle, ASDF itself is versioned, What has changed between ASDF 1 and ASDF 2?
4987@subsection ASDF can be upgraded
4988
4989When an old version of ASDF was loaded,
4990it was very hard to upgrade ASDF in your current image
4991without breaking everything.
4992Instead you had to exit the Lisp process and
4993somehow arrange to start a new one from a simpler image.
4994Something that can't be done from within Lisp,
4995making automation of it difficult,
4996which compounded with difficulty in configuration,
4997made the task quite hard.
4998Yet as we saw before, the task would have been required
4999to not have to live with the worst case or non-portable
5000subset of ASDF features.
5001
5002With ASDF 2, it is easy to upgrade
5003from ASDF 2 to later versions from within Lisp,
5004and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
5005We support hot upgrade of ASDF and any breakage is a bug
5006that we will do our best to fix.
5007There are still limitations on upgrade, though,
5008most notably the fact that after you upgrade ASDF,
5009you must also reload or upgrade all ASDF extensions.
5010
5011@node Decoupled release cycle, Pitfalls of the transition to ASDF 2, ASDF can be upgraded, What has changed between ASDF 1 and ASDF 2?
5012@subsection Decoupled release cycle
5013
5014When vendors were releasing their Lisp implementations with ASDF,
5015they had to basically never change version
5016because neither upgrade nor downgrade was possible
5017without breaking something for someone,
5018and no obvious upgrade path was visible and recommendable.
5019
5020With ASDF 2, upgrade is possible, easy and can be recommended.
5021This means that vendors can safely ship a recent version of ASDF,
5022confident that if a user isn't fully satisfied,
5023he can easily upgrade ASDF and deal
5024with a supported recent version of it.
5025This means that release cycles will be causally decoupled,
5026the practical consequence of which will mean faster convergence
5027towards the latest version for everyone.
5028
5029
5030@node Pitfalls of the transition to ASDF 2,  , Decoupled release cycle, What has changed between ASDF 1 and ASDF 2?
5031@subsection Pitfalls of the transition to ASDF 2
5032
5033The main pitfalls in upgrading to ASDF 2 seem to be related
5034to the output translation mechanism.
5035
5036@itemize
5037
5038@item
5039Output translations is enabled by default. This may surprise some users,
5040most of them in pleasant way (we hope), a few of them in an unpleasant way.
5041It is trivial to disable output translations.
5042@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
5043
5044@item
5045Some systems in the large have been known
5046not to play well with output translations.
5047They were relatively easy to fix.
5048Once again, it is also easy to disable output translations,
5049or to override its configuration.
5050
5051@item
5052The new ASDF output translations are incompatible with ASDF-Binary-Locations.
5053They replace A-B-L, and there is compatibility mode to emulate
5054your previous A-B-L configuration.
5055See @code{enable-asdf-binary-locations-compatibility} in
5056@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
5057But thou shalt not load ABL on top of ASDF 2.
5058
5059@end itemize
5060
5061Other issues include the following:
5062
5063@itemize
5064
5065@item
5066ASDF pathname designators are now specified
5067in places where they were unspecified,
5068and a few small adjustments have to be made to some non-portable defsystems.
5069Notably, in the @code{:pathname} argument
5070to a @code{defsystem} and its components,
5071a logical pathname (or implementation-dependent hierarchical pathname)
5072must now be specified with @code{#p} syntax
5073where the namestring might have previously sufficed;
5074moreover when evaluation is desired @code{#.} must be used,
5075where it wasn't necessary in the toplevel @code{:pathname} argument
5076(but necessary in other @code{:pathname} arguments).
5077
5078@item
5079There is a slight performance bug, notably on SBCL,
5080when initially searching for @file{asd} files,
5081the implicit @code{(directory "/configured/path/**/*.asd")}
5082for every configured path @code{(:tree "/configured/path/")}
5083in your @code{source-registry} configuration can cause a slight pause.
5084Try to @code{(time (asdf:initialize-source-registry))}
5085to see how bad it is or isn't on your system.
5086If you insist on not having this pause,
5087you can avoid the pause by overriding the default source-registry configuration
5088and not use any deep @code{:tree} entry but only @code{:directory} entries
5089or shallow @code{:tree} entries.
5090Or you can fix your implementation to not be quite that slow
5091when recursing through directories.
5092@emph{Update}: This performance bug fixed the hard way in 2.010.
5093
5094@item
5095On Windows, only LispWorks supports proper default configuration pathnames
5096based on the Windows registry.
5097Other implementations make do with environment variables,
5098that you may have to define yourself
5099if you're using an older version of Windows.
5100Windows support is somewhat less tested than Unix support.
5101Please help report and fix bugs.
5102@emph{Update}: As of ASDF 2.21, all implementations
5103should now use the same proper default configuration pathnames
5104and they should actually work, though they haven't all been tested.
5105
5106@item
5107The mechanism by which one customizes a system so that Lisp files
5108may use a different extension from the default @file{.lisp} has changed.
5109Previously, the pathname for a component
5110was lazily computed when operating on a system,
5111and you would
5112@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
5113  (declare (ignorable component system)) "lis")}.
5114Now, the pathname for a component is eagerly computed when defining the system,
5115and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
5116and use @code{:default-component-class cl-source-file.lis}
5117as argument to @code{defsystem},
5118as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
5119
5120@findex source-file-type
5121
5122
5123@end itemize
5124
5125
5126@node Issues with installing the proper version of ASDF, Issues with configuring ASDF, What has changed between ASDF 1 and ASDF 2?, FAQ
5127@section Issues with installing the proper version of ASDF
5128
5129@menu
5130* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
5131* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
5132@end menu
5133
5134@node My Common Lisp implementation comes with an outdated version of ASDF. What to do?, I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?, Issues with installing the proper version of ASDF, Issues with installing the proper version of ASDF
5135@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
5136
5137We recommend you upgrade ASDF.
5138@xref{Loading ASDF,,Upgrading ASDF}.
5139
5140If this does not work, it is a bug, and you should report it.
5141@xref{FAQ, report-bugs, Where do I report a bug}.
5142In the meantime, you can load @file{asdf.lisp} directly.
5143@xref{Loading ASDF,Loading an otherwise installed ASDF}.
5144
5145
5146@node I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?,  , My Common Lisp implementation comes with an outdated version of ASDF. What to do?, Issues with installing the proper version of ASDF
5147@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
5148
5149Since ASDF 2,
5150it should always be a good time to upgrade to a recent version of ASDF.
5151You may consult with the maintainer for which specific version they recommend,
5152but the latest @code{release} should be correct.
5153Though we do try to test ASDF releases against all implementations that we can,
5154we may not be testing against all variants of your implementation,
5155and we may not be running enough tests;
5156we trust you to thoroughly test it with your own implementation
5157before you release it.
5158If there are any issues with the current release,
5159it's a bug that you should report upstream and that we will fix ASAP.
5160
5161As to how to include ASDF, we recommend the following:
5162
5163@itemize
5164@item
5165If ASDF isn't loaded yet, then @code{(require "asdf")}
5166should load the version of ASDF that is bundled with your system.
5167If possible so should @code{(require "ASDF")}.
5168You may have it load some other version configured by the user,
5169if you allow such configuration.
5170
5171@item
5172If your system provides a mechanism to hook into @code{CL:REQUIRE},
5173then it would be nice to add ASDF to this hook the same way that
5174ABCL, CCL, CLISP, CMUCL, ECL, SBCL and SCL do it.
5175Please send us appropriate code to this end.
5176
5177@item
5178You may, like SBCL since 1.1.13 or MKCL since 1.1.9,
5179have ASDF create bundle FASLs
5180that are provided as modules by your Lisp distribution.
5181You may also, but we don't recommend that anymore,
5182have ASDF like SBCL up until 1.1.12 be implicitly used
5183when requiring modules that are provided by your Lisp distribution;
5184if you do, you should add them in the beginning of both
5185@code{wrapping-source-registry} and @code{wrapping-output-translations}.
5186
5187@item
5188If you have magic systems as above, like SBCL used to do,
5189then we explicitly ask you to @emph{NOT} distribute
5190@file{asdf.asd} as part of those magic systems.
5191You should still include the file @file{asdf.lisp} in your source distribution
5192and precompile it in your binary distribution,
5193but @file{asdf.asd} if included at all,
5194should be secluded from the magic systems,
5195in a separate file hierarchy.
5196Alternatively, you may provide the system
5197after renaming it and its @file{.asd} file to e.g.
5198@code{asdf-ecl} and @file{asdf-ecl.asd}, or
5199@code{sb-asdf} and @file{sb-asdf.asd}.
5200Indeed, if you made @file{asdf.asd} a magic system,
5201then users would no longer be able to upgrade ASDF using ASDF itself
5202to some version of their preference that
5203they maintain independently from your Lisp distribution.
5204
5205@item
5206If you do not have any such magic systems, or have other non-magic systems
5207that you want to bundle with your implementation,
5208then you may add them to the @code{wrapping-source-registry},
5209and you are welcome to include @file{asdf.asd} amongst them.
5210Non-magic systems should be at the back of the @code{wrapping-source-registry}
5211while magic systems are at the front.
5212If they are precompiled,
5213they should also be in the @code{wrapping-output-translations}.
5214
5215@item
5216Since ASDF 3, the library UIOP comes transcluded in ASDF.
5217But if you want to be nice to users who care for UIOP but not for ASDF,
5218you may package UIOP separately,
5219so that one may @code{(require "uiop")} and not load ASDF,
5220or one may @code{(require "asdf")}
5221which would implicitly require and load the former.
5222
5223@item
5224Please send us upstream any patches you make to ASDF itself,
5225so we can merge them back in for the benefit of your users
5226when they upgrade to the upstream version.
5227
5228@end itemize
5229
5230
5231
5232@node Issues with configuring ASDF, Issues with using and extending ASDF to define systems, Issues with installing the proper version of ASDF, FAQ
5233@section Issues with configuring ASDF
5234
5235@menu
5236* How can I customize where fasl files are stored?::
5237* How can I wholly disable the compiler output cache?::
5238@end menu
5239
5240@node How can I customize where fasl files are stored?, How can I wholly disable the compiler output cache?, Issues with configuring ASDF, Issues with configuring ASDF
5241@subsection ``How can I customize where fasl files are stored?''
5242
5243@xref{Controlling where ASDF saves compiled files}.
5244
5245Note that in the past there was an add-on to ASDF called
5246@code{ASDF-binary-locations}, developed by Gary King.
5247That add-on has been merged into ASDF proper,
5248then superseded by the @code{asdf-output-translations} facility.
5249
5250Note that use of @code{asdf-output-translations}
5251can interfere with one aspect of your systems
5252--- if your system uses @code{*load-truename*} to find files
5253(e.g., if you have some data files stored with your program),
5254then the relocation that this ASDF customization performs
5255is likely to interfere.
5256Use @code{asdf:system-relative-pathname} to locate a file
5257in the source directory of some system, and
5258use @code{asdf:apply-output-translations} to locate a file
5259whose pathname has been translated by the facility.
5260
5261@node How can I wholly disable the compiler output cache?,  , How can I customize where fasl files are stored?, Issues with configuring ASDF
5262@subsection ``How can I wholly disable the compiler output cache?''
5263
5264To permanently disable the compiler output cache
5265for all future runs of ASDF, you can:
5266
5267@example
5268mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
5269echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
5270@end example
5271
5272This assumes that you didn't otherwise configure the ASDF files
5273(if you did, edit them again),
5274and don't somehow override the configuration at runtime
5275with a shell variable (see below) or some other runtime command
5276(e.g. some call to @code{asdf:initialize-output-translations}).
5277
5278To disable the compiler output cache in Lisp processes
5279run by your current shell, try (assuming @code{bash} or @code{zsh})
5280(on Unix and cygwin only):
5281
5282@example
5283export ASDF_OUTPUT_TRANSLATIONS=/:
5284@end example
5285
5286To disable the compiler output cache just in the current Lisp process,
5287use (after loading ASDF but before using it):
5288
5289@example
5290(asdf:disable-output-translations)
5291@end example
5292
5293Note that this does @emph{NOT} belong in a @file{.asd} file.
5294Please do not tamper with ASDF configuration from a @file{.asd} file,
5295and only do this from your personal configuration or build scripts.
5296
5297@node Issues with using and extending ASDF to define systems, ASDF development FAQs, Issues with configuring ASDF, FAQ
5298@section Issues with using and extending ASDF to define systems
5299
5300@menu
5301* How can I cater for unit-testing in my system?::
5302* How can I cater for documentation generation in my system?::
5303* How can I maintain non-Lisp (e.g. C) source files?::
5304* I want to put my module's files at the top level.  How do I do this?::
5305* How do I create a system definition where all the source files have a .cl extension?::
5306* How do I mark a source file to be loaded only and not compiled?::
5307* How do I work with readtables?::
5308@end menu
5309
5310@node How can I cater for unit-testing in my system?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems, Issues with using and extending ASDF to define systems
5311@subsection ``How can I cater for unit-testing in my system?''
5312
5313ASDF provides a predefined test operation, @code{test-op}.
5314@xref{Predefined operations of ASDF, test-op}.
5315The test operation, however, is largely left to the system definer to specify.
5316@code{test-op} has been
5317a topic of considerable discussion on the
5318@url{http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel,asdf-devel mailing list},
5319and on the
5320@url{https://launchpad.net/asdf,launchpad bug-tracker}.
5321We provide some guidelines in the discussion of @code{test-op}.
5322
5323@c cut the following because it's discussed in the discussion of test-op.
5324@c Here are some guidelines:
5325
5326@c @itemize
5327@c @item
5328@c For a given system, @var{foo}, you will want to define a corresponding
5329@c test system, such as @var{foo-test}.  The reason that you will want this
5330@c separate system is that ASDF does not out of the box supply components
5331@c that are conditionally loaded.  So if you want to have source files
5332@c (with the test definitions) that will not be loaded except when testing,
5333@c they should be put elsewhere.
5334
5335@c @item
5336@c The @var{foo-test} system can be defined in an asd file of its own or
5337@c together with @var{foo}.  An aesthetic preference against cluttering up
5338@c the filesystem with extra asd files should be balanced against the
5339@c question of whether one might want to directly load @var{foo-test}.
5340@c Typically one would not want to do this except in early stages of
5341@c debugging.
5342
5343@c @item
5344@c Record that testing is implemented by @var{foo-test}.  For example:
5345@c @example
5346@c (defsystem @var{foo}
5347@c    :in-order-to ((test-op (test-op @var{foo-test})))
5348@c    ....)
5349
5350@c (defsystem @var{foo-test}
5351@c    :depends-on (@var{foo} @var{my-test-library} ...)
5352@c    ....)
5353@c @end example
5354@c @end itemize
5355
5356@c This procedure will allow you to support users who do not wish to
5357@c install your test framework.
5358
5359@c One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
5360@c does not return a value.  So in current versions of ASDF there is no
5361@c reliable programmatic means of determining whether or not a set of tests
5362@c has passed, or which tests have failed.  The user must simply read the
5363@c console output.  This limitation has been the subject of much
5364@c discussion.
5365
5366@node How can I cater for documentation generation in my system?, How can I maintain non-Lisp (e.g. C) source files?, How can I cater for unit-testing in my system?, Issues with using and extending ASDF to define systems
5367@subsection ``How can I cater for documentation generation in my system?''
5368
5369Various ASDF extensions provide some kind of @code{doc-op} operation.
5370See also @url{https://bugs.launchpad.net/asdf/+bug/479470}.
5371
5372
5373@node How can I maintain non-Lisp (e.g. C) source files?, I want to put my module's files at the top level.  How do I do this?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems
5374@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
5375
5376See @code{cffi}'s @code{cffi-grovel}.
5377
5378@anchor{report-bugs}
5379
5380
5381@node I want to put my module's files at the top level.  How do I do this?, How do I create a system definition where all the source files have a .cl extension?, How can I maintain non-Lisp (e.g. C) source files?, Issues with using and extending ASDF to define systems
5382@subsection ``I want to put my module's files at the top level.  How do I do this?''
5383
5384By default, the files contained in an asdf module go
5385in a subdirectory with the same name as the module.
5386However, this can be overridden by adding a @code{:pathname ""} argument
5387to the module description.
5388For example, here is how it could be done
5389in the spatial-trees ASDF system definition for ASDF 2:
5390
5391@example
5392(asdf:defsystem :spatial-trees
5393  :components
5394  ((:module base
5395            :pathname ""
5396            :components
5397            ((:file "package")
5398             (:file "basedefs" :depends-on ("package"))
5399             (:file "rectangles" :depends-on ("package"))))
5400   (:module tree-impls
5401            :depends-on (base)
5402            :pathname ""
5403            :components
5404            ((:file "r-trees")
5405             (:file "greene-trees" :depends-on ("r-trees"))
5406             (:file "rstar-trees" :depends-on ("r-trees"))
5407             (:file "rplus-trees" :depends-on ("r-trees"))
5408             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
5409   (:module viz
5410            :depends-on (base)
5411            :pathname ""
5412            :components
5413            ((:static-file "spatial-tree-viz.lisp")))
5414   (:module tests
5415            :depends-on (base)
5416            :pathname ""
5417            :components
5418            ((:static-file "spatial-tree-test.lisp")))
5419   (:static-file "LICENCE")
5420   (:static-file "TODO")))
5421@end example
5422
5423All of the files in the @code{tree-impls} module are at the top level,
5424instead of in a @file{tree-impls/} subdirectory.
5425
5426Note that the argument to @code{:pathname} can be either a pathname object or a string.
5427A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
5428but this is discouraged because the results of parsing a namestring are not portable.
5429A pathname can only be portably constructed with such syntax as
5430@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
5431and similarly the current directory can only be portably specified as
5432@code{#.(make-pathname :directory '(:relative))}.
5433However, as of ASDF 2, you can portably use a string to denote a pathname.
5434The string will be parsed as a @code{/}-separated path from the current directory,
5435such that the empty string @code{""} denotes the current directory, and
5436@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
5437portably denotes the same subdirectory as above.
5438When files are specified, the last @code{/}-separated component is interpreted
5439either as the name component of a pathname
5440(if the component class specifies a pathname type),
5441or as a name component plus optional dot-separated type component
5442(if the component class doesn't specifies a pathname type).
5443
5444@node How do I create a system definition where all the source files have a .cl extension?, How do I mark a source file to be loaded only and not compiled?, I want to put my module's files at the top level.  How do I do this?, Issues with using and extending ASDF to define systems
5445@subsection How do I create a system definition where all the source files have a .cl extension?
5446
5447Starting with ASDF 2.014.14, you may just pass
5448the builtin class @code{cl-source-file.cl} as
5449the @code{:default-component-class} argument to @code{defsystem}:
5450
5451@lisp
5452(defsystem my-cl-system
5453  :default-component-class cl-source-file.cl
5454  ...)
5455@end lisp
5456
5457Another builtin class @code{cl-source-file.lsp} is offered
5458for files ending in @file{.lsp}.
5459
5460If you want to use a different extension
5461for which ASDF doesn't provide builtin support,
5462or want to support versions of ASDF
5463earlier than 2.014.14 (but later than 2.000),
5464you can define a class as follows:
5465
5466@lisp
5467;; Prologue: make sure we're using a sane package.
5468(defpackage :my-asdf-extension
5469   (:use :asdf :common-lisp)
5470   (:export #:cl-source-file.lis))
5471(in-package :my-asdf-extension)
5472
5473(defclass cl-source-file.lis (cl-source-file)
5474  ((type :initform "lis")))
5475@end lisp
5476
5477Then you can use it as follows:
5478@lisp
5479(defsystem my-cl-system
5480  :default-component-class my-asdf-extension:cl-source-file.lis
5481  ...)
5482@end lisp
5483
5484Of course, if you're in the same package, e.g. in the same file,
5485you won't need to use the package qualifier before @code{cl-source-file.lis}.
5486Actually, if all you're doing is defining this class
5487and using it in the same file without other fancy definitions,
5488you might skip package complications:
5489
5490@lisp
5491(in-package :asdf)
5492(defclass cl-source-file.lis (cl-source-file)
5493   ((type :initform "lis")))
5494(defsystem my-cl-system
5495  :default-component-class cl-source-file.lis
5496  ...)
5497@end lisp
5498
5499It is possible to achieve the same effect
5500in a way that supports both ASDF 1 and ASDF 2,
5501but really, friends don't let friends use ASDF 1.
5502Please upgrade to ASDF 3.
5503In short, though: do same as above, but
5504@emph{before} you use the class in a @code{defsystem},
5505you also define the following method:
5506
5507@lisp
5508(defmethod source-file-type ((f cl-source-file.lis) (s system))
5509  (declare (ignorable f s))
5510  "lis")
5511@end lisp
5512
5513@node How do I mark a source file to be loaded only and not compiled?, How do I work with readtables?, How do I create a system definition where all the source files have a .cl extension?, Issues with using and extending ASDF to define systems
5514@subsection How do I mark a source file to be loaded only and not compiled?
5515
5516There is no provision in ASDF for ensuring that
5517some components are always loaded as source, while others are always
5518compiled.
5519There is @code{load-source-op} (@pxref{Predefined operations of
5520ASDF,load-source-op}), but that is an operation to be applied to a
5521system as a whole, not to one or another specific source files.
5522While this idea often comes up in discussions,
5523it doesn't play well with either the linking model of ECL
5524or with various bundle operations.
5525In addition, the dependency model of ASDF would have to be modified incompatibly
5526to allow for such a trick.
5527@c If your code doesn't compile cleanly, fix it.
5528@c If compilation makes it slow, use @code{declaim} or @code{eval-when}
5529@c to adjust your compiler settings,
5530@c or eschew compilation by @code{eval}uating a quoted source form at load-time.
5531
5532@node How do I work with readtables?,  , How do I mark a source file to be loaded only and not compiled?, Issues with using and extending ASDF to define systems
5533@subsection How do I work with readtables?
5534
5535@cindex readtables
5536
5537It is possible to configure the lisp syntax by modifying the currently-active readtable.
5538However, this same readtable is shared globally by all software being compiled by ASDF,
5539especially since @code{load} and @code{compile-file} both bind @var{*readtable*},
5540so that its value is the same across the build at the start of every file
5541(unless overridden by some @code{perform :around} method),
5542even if a file locally binds it to a different readtable during the build.
5543
5544Therefore, the following hygiene restrictions apply. If you don't abide by these restrictions,
5545there will be situations where your output files will be corrupted during an incremental build.
5546We are not trying to prescribe new restrictions for the sake of good style:
5547these restrictions have always applied implicitly, and
5548we are simply describing what they have always been.
5549
5550@itemize
5551@item It is forbidden to modifying any standard character or standard macro dispatch defined in the CLHS.
5552@item No two dependencies may assign different meanings to the same non-standard character.
5553@item Using any non-standard character while expecting the implementation to treat some way
5554    counts as such an assignment of meaning.
5555@item libraries need to document these assignments of meaning to non-standard characters.
5556@item free software libraries will register these changes on:
5557        @url{http://www.cliki.net/Macro%20Characters}
5558@end itemize
5559
5560If you want to use readtable modifications that cannot abide by those restrictions,
5561you @emph{must} create a different readtable object and set @var{*readtable*}
5562to temporarily bind it to your new readtable (which will be undone after processing the file).
5563
5564For that, we recommend you use system @code{named-readtables}
5565to define or combine such readtables using @code{named-readtables:defreadtable}
5566and use them using @code{named-readtables:in-readtable}.
5567Equivalently, you can use system @code{cl-syntax},
5568that itself uses @code{named-readtables},
5569but may someday do more with, e.g. @var{*print-pprint-dispatch*}.
5570
5571For even more advanced syntax modification beyond what a readtable can express,
5572you may consider either:
5573@itemize
5574@item a @code{perform} method that compiles a constant file that contains a single form
5575  @code{#.*code-read-with-alternate-reader*} in an environment where this special variable
5576  was bound to the code read by your alternate reader, or
5577@item using the system @code{reader-interception}.
5578@end itemize
5579
5580Beware that @c unless and until the @code{syntax-control} branch is merged,
5581it is unsafe to use ASDF from the REPL to compile or load systems
5582while the readtable isn't the shared readtable previously used to build software.
5583You @emph{must} manually undo any binding of @var{*readtable*} at the REPL
5584and restore its initial value whenever you call @code{operate}
5585(via e.g. @code{load-system}, @code{test-system} or @code{require})
5586from a REPL that is using a different readtable.
5587
5588@subsubsection How should my system use a readtable exported by another system?
5589
5590Use from the @code{named-readtables} system the macro @code{named-readtables:in-readtable}.
5591
5592If the other system fails to use @code{named-readtables}, fix it and send a patch upstream.
5593In the day and age of Quicklisp and clbuild, there is little reason
5594to eschew using such an important library anymore.
5595
5596@subsubsection How should my library make a readtable available to other systems?
5597
5598Use from the @code{named-readtables} system the macro @code{named-readtables:defreadtable}.
5599
5600@node ASDF development FAQs,  , Issues with using and extending ASDF to define systems, FAQ
5601@section ASDF development FAQs
5602
5603@menu
5604* How do run the tests interactively in a REPL?::
5605@end menu
5606
5607@node How do run the tests interactively in a REPL?,  , ASDF development FAQs, ASDF development FAQs
5608@subsection How do run the tests interactively in a REPL?
5609
5610This not-so-frequently asked question is primarily for ASDF developers,
5611but those who encounter an unexpected error in some test may be
5612interested, too.
5613
5614Here's the procedure for experimenting with tests in a REPL:
5615@example
5616;; BEWARE! Some tests expect you to be in the .../asdf/test directory
5617;; If your REPL is not there yet, change your current directory:
5618;; under SLIME, you may: ,change-directory ~/common-lisp/asdf/test/
5619;; otherwise you may evaluate something like:
5620(require "asdf") (asdf:upgrade-asdf) ;load UIOP & update asdf.lisp
5621(uiop:chdir (asdf:system-relative-pathname :asdf "test/"))
5622(setf *default-pathname-defaults* (uiop:getcwd))
5623
5624;; Load the test script support.
5625(load "script-support.lisp")
5626
5627;; Initialize the script support.
5628;; This will also change your *package* to asdf-test.
5629;; NB: this function is also available from package cl-user,
5630;; and also available with the shorter name da in both packages.
5631(asdf-test::debug-asdf)
5632
5633;; In case you modified ASDF since you last tested it,
5634;; you need to update asdf.lisp itself by evaluating 'make' in a shell,
5635;; or (require "asdf") (asdf:load-system :asdf) in another CL REPL,
5636;; if not done in this REPL above.
5637;; *Then*, in this REPL, you need to evaluate:
5638;(asdf-test::compile-load-asdf)
5639
5640;; Now, you may experiment with test code from a .script file.
5641;; See the instructions given at the end of your failing test
5642;; to identify which form is needed, e.g.
5643(frob-packages)
5644(asdf::with-asdf-cache () (load "test-utilities.script"))
5645@end example
5646
5647
5648@comment FIXME: Add a FAQ about how to use a new system class...
5649
5650@comment  node-name,  next,  previous,  up
5651@node  Ongoing Work, Bibliography, FAQ, Top
5652@unnumbered Ongoing Work
5653For an active list of things to be done,
5654see the @file{TODO} file in the source repository.
5655
5656Also, bugs are now tracked on launchpad:
5657@url{https://launchpad.net/asdf}.
5658
5659@node Bibliography, Concept Index, Ongoing Work, Top
5660@unnumbered Bibliography
5661
5662@itemize
5663@item Francois-Rene Rideau:
5664  ``ASDF 3, or Why Lisp is Now an Acceptable Scripting Language'', 2014.
5665  This article describes the innovations in ASDF 3 and 3.1,
5666  as well as historical information on previous versions.
5667  @url{http://github.com/fare/asdf3-2013}
5668@item Alastair Bridgewater:
5669  ``Quick-build'' (private communication), 2012.
5670  @code{quick-build} is a simple and robust one file, one package build system,
5671  similar to @code{faslpath}, in 182 lines of code
5672  (117 of which are not blank, not comments, not docstrings).
5673  Unhappily, it remains unpublished and its IP status is unclear as of April 2014.
5674  @code{asdf/package-system} is mostly compatible with it,
5675  modulo a different setup for toplevel hierarchies.
5676@item Zach Beane:
5677  ``Quicklisp'', 2011.
5678  The Quicklisp blog and Xach's livejournal contain information on Quicklisp.
5679  @url{http://blog.quicklisp.org/}
5680  @url{http://xach.livejournal.com/}
5681@item Francois-Rene Rideau and Robert Goldman:
5682  ``Evolving ASDF: More Cooperation, Less Coordination'', 2010.
5683  This article describes the main issues solved by ASDF 2.
5684  @url{http://common-lisp.net/project/asdf/doc/ilc2010draft.pdf}
5685  @url{http://www.common-lisp.org/gitweb?p=projects/asdf/ilc2010.git}
5686@item Francois-Rene Rideau and Spencer Brody:
5687  ``XCVB: an eXtensible Component Verifier and Builder for Common Lisp'', 2009.
5688  This article describes XCVB, a proposed competitor for ASDF,
5689  many ideas of which have been incorporated into ASDF 2 and 3,
5690  though many other of which still haven't.
5691  @url{http://common-lisp.net/projects/xcvb/}
5692@item Peter von Etter:
5693  ``faslpath'', 2009.
5694  @code{faslpath} is similar to the latter @code{quick-build}
5695  and our letter @code{asdf/package-system} extension,
5696  except that it uses the dot @code{.} rather than the slash @code{/} as a separator.
5697  @url{https://code.google.com/p/faslpath/}
5698@item Drew McDermott:
5699  ``A Framework for Maintaining the Coherence of a Running Lisp,''
5700  International Lisp Conference, 2005, available in pre-print form at
5701  @url{http://www.cs.yale.edu/homes/dvm/papers/lisp05.pdf}
5702@item Dan Barlow: ``ASDF Manual'', 2004.
5703  Older versions of this document from the days of ASDF 1;
5704  they include ideas laid down by Dan Barlow,
5705  and comparisons with older defsystems (@code{mk-defsystem})
5706  and defsystem (@code{defsystem-4}, kmp's Memo 801).
5707@item Marco Antoniotti and Peter Van Eynde:
5708 ``@code{DEFSYSTEM}: A @code{make} for Common Lisp, A Thoughtful Re-Implementation of an Old Idea'', 2002.
5709  The @file{defsystem-4} proposal available in the CLOCC repository.
5710@item Mark Kantrovitz: ``Defsystem: A Portable Make Facility for Common Lisp'', 1990.
5711  The classic @file{mk-defsystem}, later variants of which
5712  are available in the CLOCC repository as @code{defsystem-3.x}.
5713@item Richard Elliot Robbins:
5714  ``BUILD: A Tool for Maintaining Consistency in Modular Systems'', MIT AI TR 874, 1985.
5715  @url{ftp://publications.ai.mit.edu/ai-publications/pdf/AITR-874.pdf}
5716@item Kent M. Pitman (kmp): ``The Description of Large Systems'', MIT AI Memo 801, 1984.
5717  Available in updated-for-CL form on the web at
5718  @url{http://nhplace.com/kent/Papers/Large-Systems.html}
5719@item Dan Weinreb and David Moon:
5720  ``Lisp Machine Manual'', MIT, 1981.
5721  The famous CHINE NUAL describes one of the earliest variants of DEFSYSTEM.
5722  @url{https://bitsavers.trailing-edge.com/pdf/mit/cadr/chinual_4thEd_Jul81.pdf}
5723@end itemize
5724
5725
5726@node Concept Index, Function and Class Index, Bibliography, Top
5727@unnumbered Concept Index
5728
5729@printindex cp
5730
5731@node Function and Class Index, Variable Index, Concept Index, Top
5732@unnumbered Function and Class Index
5733
5734@printindex fn
5735
5736@node Variable Index,  , Function and Class Index, Top
5737@unnumbered Variable Index
5738
5739@printindex vr
5740
5741@bye
5742
5743@c  LocalWords:  clbuild tarballs defsystem Quicklisp initarg uiop fasl
5744@c  LocalWords:  namestring initargs fasls
Note: See TracBrowser for help on using the repository browser.