source: trunk/abcl/doc/asdf/asdf.texinfo @ 14713

Last change on this file since 14713 was 14713, checked in by Mark Evenson, 8 years ago

Update to ASDF version.

File size: 225.0 KB
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
4@settitle ASDF Manual
5@c %**end of header
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro AallowOtherKeys
11@end macro
12@macro Aoptional
14@end macro
15@macro Arest
17@end macro
18@macro Akey
20@end macro
21@macro Abody
23@end macro
25@c for install-info
26@dircategory Software development
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
35You can find the latest version of this manual at
38ASDF Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
40This manual Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
42This manual revised @copyright{} 2009-2014 Robert P. Goldman and Francois-Rene Rideau.
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
63@end copying
68@title ASDF: Another System Definition Facility
70@c The following two commands start the copyright page.
72@vskip 0pt plus 1filll
74@end titlepage
76@c Output the table of contents at the beginning.
79@c -------------------
83@node Top, Introduction, (dir), (dir)
84@top ASDF: Another System Definition Facility
89* Introduction::
90* Quick start summary::
91* Loading ASDF::
92* Configuring ASDF::
93* Using ASDF::
94* Defining systems with defsystem::
95* The object model of ASDF::
96* Controlling where ASDF searches for systems::
97* Controlling where ASDF saves compiled files::
98* Error handling::
99* Miscellaneous additional functionality::
100* Getting the latest version::
101* FAQ::
102* Ongoing Work::
103* Bibliography::
104* Concept Index::
105* Function and Class Index::
106* Variable Index::              @c @detailmenu
110 --- The Detailed Node Listing ---
112Loading ASDF
114* Loading a pre-installed ASDF::
115* Checking whether ASDF is loaded::
116* Upgrading ASDF::
117* Loading ASDF from source::
119Upgrading ASDF
121* Upgrading your implementation's ASDF::
122* Issues with upgrading ASDF::
124Configuring ASDF
126* Configuring ASDF to find your systems::
127* Configuring ASDF to find your systems --- old style::
128* Configuring where ASDF stores object files::
129* Resetting the ASDF configuration::
131Using ASDF
133* Loading a system::
134* Other Operations::
135* Moving on::
137Defining systems with defsystem
139* The defsystem form::
140* A more involved example::
141* The defsystem grammar::
142* Other code in .asd files::
143* The package-inferred-system extension::
145The Object model of ASDF
147* Operations::
148* Components::
149* Dependencies::
150* Functions::
154* Predefined operations of ASDF::
155* Creating new operations::
159* Common attributes of components::
160* Pre-defined subclasses of component::
161* Creating new component types::
165* Pre-defined subclasses of component::
166* Creating new component types::
168Controlling where ASDF searches for systems
170* Configurations::
171* Truenames and other dangers::
172* XDG base directory::
173* Backward Compatibility::
174* Configuration DSL::
175* Configuration Directories::
176* Shell-friendly syntax for configuration::
177* Search Algorithm::
178* Caching Results::
179* Configuration API::
180* Introspection::
181* Status::
182* Rejected ideas::
183* TODO::
184* Credits for the source-registry::
186Configuration Directories
188* The here directive::
192* *source-registry-parameter* variable::
193* Information about system dependencies::
195Controlling where ASDF saves compiled files
197* Output Configurations::
198* Output Backward Compatibility::
199* Output Configuration DSL::
200* Output Configuration Directories::
201* Output Shell-friendly syntax for configuration::
202* Semantics of Output Translations::
203* Output Caching Results::
204* Output location API::
205* Credits for output translations::
207Miscellaneous additional functionality
209* Controlling file compilation::
210* Controlling source file character encoding::
211* Some Utility Functions::
215* Where do I report a bug?::
216* What has changed between ASDF 1 and ASDF 2?::
217* Issues with installing the proper version of ASDF::
218* Issues with configuring ASDF::
219* Issues with using and extending ASDF to define systems::
220* ASDF development FAQs::
222``What has changed between ASDF 1, ASDF 2 and ASDF 3?''
224* What are ASDF 1 2 3?::
225* How do I detect the ASDF version?::
226* ASDF can portably name files in subdirectories::
227* Output translations::
228* Source Registry Configuration::
229* Usual operations are made easier to the user::
230* Many bugs have been fixed::
231* ASDF itself is versioned::
232* ASDF can be upgraded::
233* Decoupled release cycle::
234* Pitfalls of the transition to ASDF 2::
236Issues with installing the proper version of ASDF
238* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
239* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
241Issues with configuring ASDF
243* How can I customize where fasl files are stored?::
244* How can I wholly disable the compiler output cache?::
246Issues with using and extending ASDF to define systems
248* How can I cater for unit-testing in my system?::
249* How can I cater for documentation generation in my system?::
250* How can I maintain non-Lisp (e.g. C) source files?::
251* I want to put my module's files at the top level.  How do I do this?::
252* How do I create a system definition where all the source files have a .cl extension?::
253* How do I mark a source file to be loaded only and not compiled?::
254* How do I work with readtables?::
256ASDF development FAQs
258* How do run the tests interactively in a REPL?::
260@end detailmenu
261@end menu
263@end ifnottex
265@c -------------------
267@node Introduction, Quick start summary, Top, Top
268@comment  node-name,  next,  previous,  up
269@chapter Introduction
270@cindex ASDF-related features
271@vindex *features*
272@cindex Testing for ASDF
273@cindex ASDF versions
274@cindex :asdf
275@cindex :asdf2
276@cindex :asdf3
278ASDF is Another System Definition Facility:
279a tool for specifying how systems of Common Lisp software
280are made up of components (sub-systems and files),
281and how to operate on these components in the right order
282so that they can be compiled, loaded, tested, etc.
283If you are new to ASDF, @pxref{Quick start summary,,the quick start
286ASDF presents three faces:
287one for users of Common Lisp software who want to reuse other people's code,
288one for writers of Common Lisp software who want to specify how to build their systems,
289and one for implementers of Common Lisp extensions who want to extend
290the build system.
291For more specifics,
292@pxref{Using ASDF,,Loading a system},
293to learn how to use ASDF to load a system.
294@xref{Defining systems with defsystem},
295to learn how to define a system of your own.
296@xref{The object model of ASDF}, for a description of
297the ASDF internals and how to extend ASDF.
299Note that
300ASDF is @emph{not} a tool for library and system @emph{installation}; it
301plays a role like @t{make} or @t{ant}, not like a package manager.
302In particular, ASDF should not to be confused with ASDF-Install, which attempts to find and
303download ASDF systems for you.
304Despite the name, ASDF-Install is not part of ASDF, but a separate piece of software.
305ASDF-Install is also unmaintained and obsolete.
306We recommend you use Quicklisp
307(@uref{}) instead,
308a Common Lisp package manager which works well and is being actively maintained.
309If you want to download software from version control instead of tarballs,
310so you may more easily modify it, we recommend clbuild (@uref{}).
311We recommend @file{~/common-lisp/}
312as a place into which to install Common Lisp software;
313starting with ASDF 3.1.2, it is included in the default source-registry configuration.
315@node  Quick start summary, Loading ASDF, Introduction, Top
316@chapter Quick start summary
320@item To load an ASDF system:
324Load ASDF itself into your Lisp image, either through
325@code{(require "asdf")} (if it's supplied by your lisp implementation)
326or else through
327@code{(load "/path/to/asdf.lisp")}.  For more details, @ref{Loading ASDF}.
330Make sure ASDF can find system definitions
331through proper source-registry configuration.
332For more details, @xref{Configuring ASDF to find your systems}.
333The simplest way is simply to put all your lisp code in subdirectories of
334@file{~/common-lisp/} (starting with ASDF 3.1.2),
335or @file{~/.local/share/common-lisp/source/}
336(for ASDF 2 and later, or if you want to keep source in a hidden directory).
337Such code will automatically be found.
340Load a system with @code{(asdf:load-system :system)}. @xref{Using ASDF}.
342@end itemize
344@item To make your own ASDF system:
348As above, load and configure ASDF.
351Make a new directory for your system, @code{my-system/} in a location
352where ASDF can find it (@pxref{Configuring ASDF to find your systems}).
353All else being equal, the easiest location is probably
358Create an ASDF system definition listing the dependencies of
359your system, its components, and their interdependencies,
360and put it in @file{my-system.asd}.
361This file must have the same name as your system.
362@xref{Defining systems with defsystem}.
365Use @code{(asdf:load-system :my-system)}
366to make sure it's all working properly. @xref{Using ASDF}.
368@end itemize
369@end itemize
371@c FIXME: (1) add a sample project that the user can cut and paste to
372@c get started.  (2) discuss the option of starting with Quicklisp.
378@node Loading ASDF, Configuring ASDF, Quick start summary, Top
379@comment  node-name,  next,  previous,  up
380@chapter Loading ASDF
383* Loading a pre-installed ASDF::
384* Checking whether ASDF is loaded::
385* Upgrading ASDF::
386* Loading ASDF from source::
387@end menu
389@node  Loading a pre-installed ASDF, Checking whether ASDF is loaded, Loading ASDF, Loading ASDF
390@section Loading a pre-installed ASDF
392Most recent Lisp implementations include a copy of ASDF 3,
393or at least ASDF 2.
394You can usually load this copy using Common Lisp's @code{require} function.@footnote{
395NB: all implementations except GNU CLISP also accept
396@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
397For portability's sake, you should use @code{(require "asdf")}.
401(require "asdf")
402@end lisp
404As of the writing of this manual,
405the following implementations provide ASDF 3 this way:
407The following implementations only provide ASDF 2:
408LispWorks, mocl, XCL.
409The following implementations don't provide ASDF:
410Corman CL, GCL, Genera, MCL, SCL.
411The latter implementations are not actively maintained;
412if some of them are ever released again, they probably will include ASDF 3.
414If the implementation you are using doesn't provide ASDF 2 or ASDF 3,
415see @pxref{Loading ASDF,,Loading ASDF from source} below.
416If that implementation is still actively maintained,
417you may also send a bug report to your Lisp vendor and complain
418about their failing to provide ASDF.
420@node Checking whether ASDF is loaded, Upgrading ASDF, Loading a pre-installed ASDF, Loading ASDF
421@section Checking whether ASDF is loaded
423To check whether ASDF is properly loaded in your current Lisp image,
424you can run this form:
428@end lisp
430If it returns a string,
431that is the version of ASDF that is currently installed.
433If it raises an error,
434then either ASDF is not loaded, or
435you are using a very old version of ASDF,
436and need to install ASDF 3.
438You can check whether an old version is loaded
439by checking if the ASDF package is present.
440The form below will allow you to programmatically determine
441whether a recent version is loaded, an old version is loaded,
442or none at all:
445(when (find-package :asdf)
446  (let ((ver (symbol-value
447                   (or (find-symbol (string :*asdf-version*) :asdf)
448                       (find-symbol (string :*asdf-revision*) :asdf)))))
449    (etypecase ver
450      (string ver)
451      (cons (with-output-to-string (s)
452              (loop for (n . m) on ver
453                    do (princ n s)
454                       (when m (princ "." s)))))
455      (null "1.0"))))
456@end lisp
458If it returns @code{nil} then ASDF is not installed.
459Otherwise it should return a string.
460If it returns @code{"1.0"}, then it can actually be
461any version before 1.77 or so, or some buggy variant of 1.x.
463If you are experiencing problems with ASDF,
464please try upgrading to the latest released version,
465using the method below,
466before you contact us and raise an issue.
468@node  Upgrading ASDF, Loading ASDF from source, Checking whether ASDF is loaded, Loading ASDF
469@section Upgrading ASDF
470@c FIXME: tighten this up a bit -- there's a lot of stuff here that
471@c doesn't matter to almost anyone.  Move discussion of updating antique
472@c versions of ASDF down, or encapsulate it.
474If you want to upgrade to a more recent ASDF version,
475you need to install and configure your ASDF just like any other system
476(@pxref{Configuring ASDF to find your systems}).
478If your implementation provides ASDF 3 or later,
479you only need to @code{(require "asdf")}:
480ASDF will automatically look whether an updated version of itself is available
481amongst the regularly configured systems, before it compiles anything else.
484* Upgrading your implementation's ASDF::
485* Issues with upgrading ASDF::
486@end menu
488@node Upgrading your implementation's ASDF, Issues with upgrading ASDF, Upgrading ASDF, Upgrading ASDF
489@subsection Upgrading your implementation's ASDF
491Most implementations provide a recent ASDF 3 in their latest release.
492If yours doesn't, we recommend upgrading your implementation.
493If the latest version of your implementation still doesn't provide ASDF,
494or provides an old version, we recommend installing a recent ASDF so your implementation provides it,
495as explained below.
496If all fails, we recommend you load ASDF from source
497@pxref{Loading ASDF,,Loading ASDF from source}.
499The ASDF source repository contains a script
500@file{bin/install-asdf-as-module} that can help you upgrade your implementation's ASDF.
501It works on
502Allegro CL, Clozure CL, CMU CL, ECL, GNU CLISP, LispWorks, MKCL, SBCL, SCL, XCL.
503That's all known implementations except ABCL, Corman CL, GCL, Genera, MCL, MOCL.
504Happily, ABCL is usually pretty up to date and shouldn't need that script.
505GCL would be supported, except that so far is still lacking usable support for @code{require}.
506Corman CL, Genera, MCL are obsolete anyway.
507MOCL is under development.
508On an old version of an implementation that does not provide ASDF,
509you may have to load ASDF 3 from source before you load that script.
511The script relies on @code{cl-launch} 4 for command-line invocation,
512which may depend on ASDF being checked out in @file{~/common-lisp/asdf/}
513(which we recommend anyway)
514if your implementation doesn't even have an ASDF 2.
515If you don't have @code{cl-launch},
516you can instead @code{(load "bin/install-asdf-as-module")}
517from your implementation's REPL after loading ASDF from source.
519Finally, if your implementation only provides ASDF 2,
520and you can't or won't upgrade it or override its ASDF module,
521you may simply configure ASDF to find a proper upgrade;
522however, to avoid issues with a self-upgrade in mid-build,
523you @emph{must} make sure to upgrade ASDF immediately
524after requiring the builtin ASDF 2:
527(require "asdf")
528;; <--- insert programmatic configuration here if needed
529(asdf:load-system :asdf)
530@end lisp
532@node Issues with upgrading ASDF,  , Upgrading your implementation's ASDF, Upgrading ASDF
533@subsection Issues with upgrading ASDF
535Note that there are some limitations to upgrading ASDF:
538Previously loaded ASDF extensions become invalid, and will need to be reloaded.
539Examples include CFFI-Grovel, hacks used by ironclad, etc.
540Since it isn't possible to automatically detect what extensions
541need to be invalidated,
542ASDF will invalidate @emph{all} previously loaded systems
543when it is loaded on top of a forward-incompatible ASDF version.
545@vindex *oldest-forward-compatible-asdf-version*
546Forward incompatibility can be determined using the variable
548which is 2.33 at the time of this writing.}
550Starting with ASDF 3 (2.27 or later),
551this self-upgrade will be automatically attempted as the first step
552to any system operation, to avoid any possibility of
553a catastrophic attempt to self-upgrade in mid-build.
555@c FIXME: Fix grammar below.
557For this and many other reasons,
558you should load, configure and upgrade ASDF
559as one of the very first things done by your build and startup scripts.
560It is safer if you upgrade ASDF and its extensions as a special step
561at the very beginning of whatever script you are running,
562before you start using ASDF to load anything else.
565Until all implementations provide ASDF 3 or later,
566it is unsafe to upgrade ASDF as part of loading a system
567that depends on a more recent version of ASDF,
568since the new one might shadow the old one while the old one is running,
569and the running old one will be confused
570when extensions are loaded into the new one.
571In the meantime, we recommend that your systems should @emph{not} specify
572@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "3.0.1"))},
573but instead that they check that a recent enough ASDF is installed,
574with such code as:
576(unless (or #+asdf2 (asdf:version-satisfies
577                     (asdf:asdf-version) *required-asdf-version*))
578  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
579@end example
581Until all implementations provide ASDF 3 or later,
582it is unsafe for a system to transitively depend on ASDF
583and not directly depend on ASDF;
584if any of the system you use either depends-on asdf,
585system-depends-on asdf, or transitively does,
586you should also do as well.
587@end itemize
589@node Loading ASDF from source,  , Upgrading ASDF, Loading ASDF
590@section Loading ASDF from source
592If your implementation doesn't include ASDF,
593if for some reason the upgrade somehow fails,
594does not or cannot apply to your case,
595you will have to install the file @file{asdf.lisp}
596somewhere and load it with:
599(load "/path/to/your/installed/asdf.lisp")
600@end lisp
602The single file @file{asdf.lisp} is all you normally need to use ASDF.
604You can extract this file from latest release tarball on the
605@url{,ASDF website}.
606If you are daring and willing to report bugs, you can get
607the latest and greatest version of ASDF from its git repository.
608@xref{Getting the latest version}.
610For maximum convenience you might want to have ASDF loaded
611whenever you start your Lisp implementation,
612for example by loading it from the startup script or dumping a custom core
613--- check your Lisp implementation's manual for details.
616@node Configuring ASDF, Using ASDF, Loading ASDF, Top
617@comment  node-name,  next,  previous,  up
618@chapter Configuring ASDF
620For standard use cases, ASDF should work pretty much out of the box.
621We recommend you skim the sections on configuring ASDF to find your systems
622and choose the method of installing Lisp software that works best for you.
623Then skip directly to @xref{Using ASDF}. That will probably be enough.
624You are unlikely to have to worry about the way ASDF stores object files,
625and resetting the ASDF configuration is usually only needed in corner cases.
629* Configuring ASDF to find your systems::
630* Configuring ASDF to find your systems --- old style::
631* Configuring where ASDF stores object files::
632* Resetting the ASDF configuration::
633@end menu
635@node Configuring ASDF to find your systems, Configuring ASDF to find your systems --- old style, Configuring ASDF, Configuring ASDF
636@section Configuring ASDF to find your systems
638In order to compile and load your systems, ASDF must be configured to find
639the @file{.asd} files that contain system definitions.
641There are a number of different techniques for setting yourself up with
642ASDF, starting from easiest to the most complex:
644@itemize @bullet
647Put all of your systems in one of the standard locations, subdirectories
651@file{~/common-lisp/} or
654@end itemize
655If you install software there, you don't need further
656configuration.@footnote{@file{~/common-lisp/} is only included in
657the default configuration
658starting with ASDF 3.1.2 or later.}
661If you're using some tool to install software (e.g. Quicklisp),
662the authors of that tool should already have configured ASDF.
665If you have more specific desires about how to lay out your software on
666disk, the preferred way to configure where ASDF finds your systems is
667the @code{source-registry} facility,
668fully described in its own chapter of this manual.
669@xref{Controlling where ASDF searches for systems}.  Here is a quick
670recipe for getting started:
672The simplest way to add a path to your search path,
673say @file{/home/luser/.asd-link-farm/}
674is to create the directory
676and there create a file with any name of your choice,
677and with the type @file{conf}@footnote{By requiring the @file{.conf}
678extension, and ignoring other files, ASDF allows you to have disabled files,
679editor backups, etc. in the same directory with your active
680configuration files.
682ASDF will also ignore files whose names start with a @file{.} character.
684It is customary to start the filename with two digits, to control the
685sorting of the @code{conf} files in the source registry directory, and
686thus the order in which the directories will be scanned.},
687for instance @file{42-asd-link-farm.conf},
688containing the line:
690@kbd{(:directory "/home/luser/.asd-link-farm/")}
692If you want all the subdirectories under @file{/home/luser/lisp/}
693to be recursively scanned for @file{.asd} files, instead use:
695@kbd{(:tree "/home/luser/lisp/")}
697ASDF will automatically read your configuration
698the first time you try to find a system.
699If necessary, you can reset the source-registry configuration with:
703@end lisp
706In earlier versions of ASDF, the system source registry was configured
707using a global variable, @code{asdf:*central-registry*}.
708For more details about this, see the following section,
709@ref{Configuring ASDF to find your systems --- old style}.
710Unless you need to understand this,
711skip directly to @ref{Configuring where ASDF stores object files}.
713@end itemize
715Note that your Operating System distribution or your system administrator
716may already have configured system-managed libraries for you.
720@node Configuring ASDF to find your systems --- old style, Configuring where ASDF stores object files, Configuring ASDF to find your systems, Configuring ASDF
721@section Configuring ASDF to find your systems --- old style
724@c FIXME: this section should be moved elsewhere.  The novice user
725@c should not be burdened with it. [2014/02/27:rpg]
728The old way to configure ASDF to find your systems is by
729@code{push}ing directory pathnames onto the variable
732You must configure this variable between the time you load ASDF
733and the time you first try to use it.
734Loading and configuring ASDF presumably happen
735as part of some initialization script that builds or starts
736your Common Lisp software system.
737(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
739The @code{asdf:*central-registry*} is empty by default in ASDF 2 or ASDF 3,
740but is still supported for compatibility with ASDF 1.
741When used, it takes precedence over the above source-registry.@footnote{
742It is possible to further customize
743the system definition file search.
744That's considered advanced use, and covered later:
745search forward for
747@xref{Defining systems with defsystem}.}
749For example, let's say you want ASDF to find the @file{.asd} file
751In your lisp initialization file, you could have the following:
754(require "asdf")
755(push "/home/me/src/foo/" asdf:*central-registry*)
756@end lisp
758Note the trailing slash: when searching for a system,
759ASDF will evaluate each entry of the central registry
760and coerce the result to a pathname.@footnote{
761ASDF will indeed call @code{eval} on each entry.
762It will skip entries that evaluate to @code{nil}.
764Strings and pathname objects are self-evaluating,
765in which case the @code{eval} step does nothing;
766but you may push arbitrary s-expressions onto the central registry.
767These s-expressions may be evaluated to compute context-dependent
768entries, e.g. things that depend
769on the value of shell variables or the identity of the user.
771The variable @code{asdf:*central-registry*} is thus a list of
772``system directory designators''.
773A @dfn{system directory designator} is a form
774which will be evaluated whenever a system is to be found,
775and must evaluate to a directory to look in (or @code{NIL}).
776By ``directory'', we mean
777``designator for a pathname with a non-empty DIRECTORY component''.
779The trailing directory name separator
780is necessary to tell Lisp that you're discussing a directory
781rather than a file.  If you leave it out, ASDF is likely to look in
782@code{/home/me/src/} instead of @code{/home/me/src/foo/} as you
783intended, and fail to find your system definition.
785Typically there are a lot of @file{.asd} files, and
786a common idiom was to put
787@emph{symbolic links} to all of one's @file{.asd} files
788in a common directory
789and push @emph{that} directory (the ``link farm'')
792instead of pushing each individual system directory.
794ASDF knows to follow @emph{symlinks}
795to the actual location of the systems.@footnote{
796On Windows, you can use Windows shortcuts instead of POSIX symlinks.
797if you try aliases under MacOS, we are curious to hear about your experience.}
799For example, if @code{#p"/home/me/cl/systems/"}
800is an element of @code{*central-registry*}, you could set up the
801system @var{foo} as follows:
804$ cd /home/me/cl/systems/
805$ ln -s ~/src/foo/foo.asd .
806@end example
808This old style for configuring ASDF is not recommended for new users,
809but it is supported for old users, and for users who want to programmatically
810control what directories are added to the ASDF search path.
813@node Configuring where ASDF stores object files, Resetting the ASDF configuration, Configuring ASDF to find your systems --- old style, Configuring ASDF
814@section Configuring where ASDF stores object files
815@findex clear-output-translations
817ASDF lets you configure where object files will be stored.
818Sensible defaults are provided and
819you shouldn't normally have to worry about it.
821This allows the same source code repository to be shared
822between several versions of several Common Lisp implementations,
823between several users using different compilation options,
824with users who lack write privileges on shared source directories, etc.
825This also keeps source directories from being cluttered
826with object/fasl files.
828Starting with ASDF 2, the @code{asdf-output-translations} facility
829was added to ASDF itself.  This facility controls where object files will be stored.
830This facility is fully described in a chapter of this manual,
831@ref{Controlling where ASDF saves compiled files}.
833@c FIXME: possibly this should be moved elsewhere.  It's redundant here,
834@c and makes this section of the manual too long and daunting for the
835@c new user. [2014/02/27:rpg]
836@c The simplest way to add a translation to your search path,
837@c say from @file{/foo/bar/baz/quux/}
838@c to @file{/where/i/want/my/fasls/}
839@c is to create the directory
840@c @file{~/.config/common-lisp/asdf-output-translations.conf.d/}
841@c and there create a file with any name of your choice and the type @file{conf},
842@c for instance @file{42-bazquux.conf}
843@c containing the line:
845@c @kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
847@c To disable output translations for source under a given directory,
848@c say @file{/toto/tata/}
849@c you can create a file @file{40-disable-toto.conf}
850@c with the line:
852@c @kbd{("/toto/tata/")}
854@c To wholly disable output translations for all directories,
855@c you can create a file @file{00-disable.conf}
856@c with the line:
858@c @kbd{(t t)}
860@c Note that your Operating System distribution or your system administrator
861@c may already have configured translations for you.
862@c In absence of any configuration, the default is to redirect everything
863@c under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
864@c @xref{Controlling where ASDF searches for systems}, for full details.
866@c The required @file{.conf} extension allows you to have disabled files
867@c or editor backups (ending in @file{~}), and works portably
868@c (for instance, it is a pain to allow both empty and non-empty extension on CLISP).
869@c Excluded are files the name of which start with a @file{.} character.
870@c It is customary to start the filename with two digits
871@c that specify the order in which the directories will be scanned.
873@c ASDF will automatically read your configuration
874@c the first time you try to find a system.
875@c You can reset the source-registry configuration with:
877@c @lisp
878@c (asdf:clear-output-translations)
879@c @end lisp
881@c And you probably should do so before you dump your Lisp image,
882@c if the configuration may change
883@c between the machine where you save it at the time you save it
884@c and the machine you resume it at the time you resume it.
885@c (Once again, you should use @code{(asdf:clear-configuration)}
886@c before you dump your Lisp image, which includes the above.)
888Note that before ASDF 2,
889other ASDF add-ons offered the same functionality,
890each in subtly different and incompatible ways:
891ASDF-Binary-Locations, cl-launch, common-lisp-controller.
892ASDF-Binary-Locations is now not needed anymore and should not be used.
893cl-launch 3.000 and common-lisp-controller 7.2 have been updated
894to delegate object file placement to ASDF.
896@node Resetting the ASDF configuration,  , Configuring where ASDF stores object files, Configuring ASDF
897@section Resetting the ASDF configuration
899@c FIXME: this should probably be moved out of the "quickstart" part of
900@c the manual. [2014/02/27:rpg]
903When you dump and restore an image, or when you tweak your configuration,
904you may want to reset the ASDF configuration.
905For that you may use the following function:
907@defun clear-configuration
908   Undoes any ASDF configuration
909   regarding source-registry or output-translations.
910@end defun
912This function is pushed onto the @code{uiop:*image-dump-hook*} by default,
913which means that if you save an image using @code{uiop:dump-image},
914or via @code{asdf:image-op} and @code{asdf:program-op},
915it will be automatically called to clear your configuration.
916If for some reason you prefer to call your implementation's underlying functionality,
917be sure to call @code{clear-configuration} manually,
918or push it into your implementation's equivalent of @code{uiop:*image-dump-hook*},
919e.g. @code{sb-ext:*save-hooks*} on SBCL, or @code{ext:*before-save-initializations*}
920on CMUCL and SCL, etc.
922@node  Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
923@chapter Using ASDF
926* Loading a system::
927* Other Operations::
928* Moving on::
929@end menu
931@node Loading a system, Other Operations, Using ASDF, Using ASDF
932@section Loading a system
934The system @var{foo} is loaded (and compiled, if necessary)
935by evaluating the following Lisp form:
938(asdf:load-system :@var{foo})
939@end example
941On some implementations (namely recent versions of
943ASDF hooks into the @code{CL:REQUIRE} facility
944and you can just use:
947(require :@var{foo})
948@end example
950In older versions of ASDF, you needed to use
951@code{(asdf:oos 'asdf:load-op :@var{foo})}.
952If your ASDF is too old to provide @code{asdf:load-system} though
953we recommend that you upgrade to ASDF 3.
954@xref{Loading ASDF,,Loading ASDF from source}.
956Note the name of a system is specified as a string or a symbol.
957If a symbol (including a keyword), its name is taken and lowercased.
958The name must be a suitable value for the @code{:name} initarg
959to @code{make-pathname} in whatever filesystem the system is to be
962The lower-casing-symbols behaviour is unconventional,
963but was selected after some consideration.
964The type of systems we want to support
965either have lowercase as customary case (Unix, Mac, Windows)
966or silently convert lowercase to uppercase (lpns).
967@c so this makes more sense than attempting to use @code{:case :common},
968@c which is reported not to work on some implementations
970@node Other Operations, Moving on, Loading a system, Using ASDF
971@section Other Operations
973@findex load-system
974@findex compile-system
975@findex test-system
976@findex requrie-system
978ASDF provides three commands for the most common system operations:
979@code{load-system}, @code{compile-system}, and @code{test-system}.
980It also provides @code{require-system}, a version of @code{load-system}
981that skips trying to update systems that are already loaded.
983@c FIXME: We seem to export @findex bundle-system also.
985@findex operate
986@findex oos
988Because ASDF is an extensible system
989for defining @emph{operations} on @emph{components},
990it also provides a generic function @code{operate}
991(which is usually abbreviated by @code{oos},
992which stands for operate-on-system).
993You'll use @code{oos} whenever you want to do something beyond
994compiling, loading and testing.
996Output from ASDF and ASDF extensions are sent
997to the CL stream @code{*standard-output*},
998so rebinding that stream around calls to @code{asdf:operate}
999should redirect all output from ASDF operations.
1001@c Reminder: before ASDF can operate on a system, however,
1002@c it must be able to find and load that system's definition.
1003@c @xref{Configuring ASDF,,Configuring ASDF to find your systems}.
1005@c FIXME: the following is too complicated for here, especially since
1006@c :force hasn't been defined yet.  Move it. [2014/02/27:rpg]
1008@findex already-loaded-systems
1009@findex require-system
1010@findex load-system
1011@vindex *load-system-operation*
1012For advanced users, note that
1013@code{require-system} calls @code{load-system}
1014with keyword arguments @code{:force-not (already-loaded-systems)}.
1015@code{already-loaded-systems} returns a list of the names of loaded systems.
1016@code{load-system} applies @code{operate} with the operation from
1017@code{*load-system-operation*} (which by default is @code{load-op}),
1018the system, and any provided keyword arguments.
1021@node Moving on,  , Other Operations, Using ASDF
1022@section Moving on
1024That's all you need to know to use ASDF to load systems written by others.
1025The rest of this manual deals with writing system definitions
1026for Common Lisp software you write yourself,
1027including how to extend ASDF to define new operation and component types.
1030@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
1031@comment  node-name,  next,  previous,  up
1032@chapter Defining systems with defsystem
1034This chapter describes how to use ASDF to define systems and develop
1039* The defsystem form::
1040* A more involved example::
1041* The defsystem grammar::
1042* Other code in .asd files::
1043* The package-inferred-system extension::
1044@end menu
1046@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
1047@comment  node-name,  next,  previous,  up
1048@section The defsystem form
1049@findex defsystem
1050@cindex asdf-user
1051@findex load-asd
1053This section begins with an example of a system definition,
1054then gives the full grammar of @code{defsystem}.
1056Let's look at a simple system.
1057This is a complete file that should be saved as @file{hello-lisp.asd}
1058(in order that ASDF can find it
1059when ordered to operate on the system named @code{"hello-lisp"}).
1061@c FIXME: the first example should have an outside dependency, e.g.,
1062@c CL-PPCRE.
1065(in-package :asdf-user)
1067(defsystem "hello-lisp"
1068  :description "hello-lisp: a sample Lisp system."
1069  :version "0.0.1"
1070  :author "Joe User <>"
1071  :licence "Public Domain"
1072  :components ((:file "packages")
1073               (:file "macros" :depends-on ("packages"))
1074               (:file "hello" :depends-on ("macros"))))
1075@end lisp
1077Some notes about this example:
1082The file starts with an @code{in-package} form
1083for package @code{asdf-user}.  Quick summary: just do this, because it
1084helps make interactive development of @code{defsystem} forms behave in
1085the same was as when these forms are loaded by ASDF.  If that's enough
1086for you, skip the rest of this item.  Otherwise read on for the gory details.
1088If your file is loaded by ASDF 3, it will be loaded into the
1089@code{asdf-user} package.  The @code{in-package} form
1090will ensure that the system definition is read the
1091same as within ASDF when you load it interactively with @code{cl:load}.
1092However, we recommend that you load @file{.asd} files
1093through function @code{asdf::load-asd} rather than through @code{cl:load},
1094in which case this form is unnecessary.
1095Recent versions of SLIME (2013-02 and later) know to do that.
1098You can always rely on symbols
1099from both package @code{asdf} and @code{common-lisp} being available in
1100@code{.asd} files --
1101most importantly including @code{defsystem}.
1103@c FIXME: the following should be inserted in a more advanced
1104@c bit of the manual.  For now, it is simply elided.
1105@c Starting with ASDF 3.1,
1106@c @file{.asd} files are read in the package @code{asdf-user}
1107@c that uses @code{asdf}, @code{uiop} and @code{uiop/common-lisp}
1108@c (a variant of @code{common-lisp}
1109@c that has some portability fixes on old implementations).
1110@c ASDF 3 releases before 3.1 also read in package @code{asdf-user}
1111@c but that package don't use the full @code{uiop}, only @code{uiop/package}.
1112@c ASDF 1 and ASDF 2 releases (up until 2.26) instead read @file{.asd} files
1113@c in a temporary package @code{asdf@emph{N}}
1114@c that uses @code{asdf} and @code{common-lisp}.
1115@c You may thus have to package-qualify some symbols with @code{uiop:}
1116@c to support older variants of ASDF 3,
1117@c and/or package-qualify them with @code{asdf::}
1118@c to be compatible with even older variants of ASDF 2
1119@c (and then only use the few already available in ASDF 2).
1123The @code{defsystem} form defines a system named @code{hello-lisp}
1124that contains three source files:
1125@file{packages}, @file{macros} and @file{hello}.
1127@c FIXME: The first example system should probably use just :serial T.
1129The file @file{macros} depends on @file{packages}
1130(presumably because the package it's in is defined in @file{packages}),
1131and the file @file{hello} depends on @file{macros}
1132(and hence, transitively on @file{packages}).
1133This means that ASDF will compile and load @file{packages} and @file{macros}
1134before starting the compilation of file @file{hello}.
1137System source files should be located in the same directory
1138as the @code{.asd} file with the system definition.
1139@c FIXME: the following should live somewhere, but not in the quickstart
1140@c page. [2014/05/03:rpg]
1141@c ASDF resolves symbolic links (or Windows shortcuts)
1142@c before loading the system definition file and
1143@c stores its location in the resulting system@footnote{
1144@c It is possible, though almost never necessary, to override this behaviour.}.
1145@c This is a good thing because the user can move the system sources
1146@c without having to edit the system definition.
1148@c FIXME: Should have cross-reference to "Version specifiers" in the
1149@c defsystem grammar, but the cross-referencing is so broken by
1150@c insufficient node breakdown that I have not put one in.
1151@c FIXME: this is way too detailed for the first example!
1152@c move it!
1154Make sure you know how the @code{:version} numbers will be parsed!
1155Only period-separated non-negative integers are accepted.
1156See below Version specifiers in @ref{The defsystem grammar}.
1157@cindex :version
1159@end itemize
1161@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
1162@comment  node-name,  next,  previous,  up
1163@section A more involved example
1164@findex defsystem
1166Let's illustrate some more involved uses of @code{defsystem} via a
1167slightly convoluted example:
1170(in-package :asdf-user)
1172(defsystem "foo"
1173  :version "1.0.0"
1174  :components ((:module "mod"
1175                :components ((:file "bar")
1176                             (:file"baz")
1177                             (:file "quux"))
1178                :perform (compile-op :after (op c)
1179                           (do-something c))
1180                :explain (compile-op :after (op c)
1181                           (explain-something c)))
1182               (:file "blah")))
1183@end lisp
1185The @code{:module} component named @code{"mod"} is a collection of three files,
1186which will be located in a subdirectory of the main code directory named
1187@file{mod} (this location can be overridden; see the discussion of the
1188@code{:pathname} option in @ref{The defsystem grammar}).
1190The method-form tokens provide a shorthand for defining methods on
1191particular components.  This part
1194                :perform (compile-op :after (op c)
1195                          (do-something c))
1196                :explain (compile-op :after (op c)
1197                          (explain-something c))
1198@end lisp
1200has the effect of
1203(defmethod perform :after ((op compile-op) (c (eql ...)))
1204           (do-something c))
1205(defmethod explain :after ((op compile-op) (c (eql ...)))
1206           (explain-something c))
1207@end lisp
1209where @code{...} is the component in question.
1210In this case @code{...} would expand to something like
1213(find-component "foo" "mod")
1214@end lisp
1216For more details on the syntax of such forms, see @ref{The defsystem
1218For more details on what these methods do, @pxref{Operations} in
1219@ref{The object model of ASDF}.
1221@c FIXME: The following plunge into detail weeds is not appropriate in this
1222@c location. [2010/10/03:rpg]
1223@c note that although this also supports @code{:before} methods,
1224@c they may not do what you want them to ---
1225@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
1226@c will run after all the dependencies and sub-components have been processed,
1227@c but before the component in question has been compiled.
1230@c FIXME: There should be YA example that shows definitions of functions
1231@c and classes.  The following material should go there.
1232@c @item
1233@c If in addition to simply using @code{defsystem},
1234@c you are going to define functions,
1235@c create ASDF extension, globally bind symbols, etc.,
1236@c it is recommended that to avoid namespace pollution between systems,
1237@c you should create your own package for that purpose, with:
1239@c @lisp
1240@c (defpackage :hello-lisp-system
1241@c   (:use :cl :asdf))
1243@c (in-package :hello-lisp-system)
1244@c @end lisp
1247@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
1248@comment  node-name,  next,  previous,  up
1249@section The defsystem grammar
1250@findex defsystem
1251@cindex DEFSYSTEM grammar
1253@c FIXME: @var typesetting not consistently used here.  We should either expand
1254@c its use to everywhere, or we should kill it everywhere.
1258system-definition := ( defsystem system-designator @var{system-option}* )
1260system-option := :defsystem-depends-on system-list
1261                 | :weakly-depends-on @var{system-list}
1262                 | :class class-name (see discussion below)
1263                 | module-option
1264                 | option
1266module-option := :components component-list
1267                 | :serial [ t | nil ]
1269option :=
1270        | :pathname pathname-specifier
1271        | :default-component-class class-name
1272        | :perform method-form
1273        | :explain method-form
1274        | :output-files method-form
1275        | :operation-done-p method-form
1276        | :if-feature feature-expression
1277        | :depends-on ( @var{dependency-def}* )
1278        | :in-order-to ( @var{dependency}+ )
1281system-list := ( @var{simple-component-name}* )
1283component-list := ( @var{component-def}* )
1285component-def  := ( component-type simple-component-name @var{option}* )
1287component-type := :module | :file | :static-file | other-component-type
1289other-component-type := symbol-by-name
1290                        (@pxref{The defsystem grammar,,Component types})
1292# This is used in :depends-on, as opposed to ``dependency,''
1293# which is used in :in-order-to
1294dependency-def := simple-component-name
1295               | ( :feature @var{feature-expression} dependency-def )
1296               | ( :version simple-component-name version-specifier )
1297               | ( :require module-name )
1299# ``dependency'' is used in :in-order-to, as opposed to
1300# ``dependency-def''
1301dependency := (dependent-op @var{requirement}+)
1302requirement := (required-op @var{required-component}+)
1303dependent-op := operation-name
1304required-op := operation-name
1306simple-component-name := string
1307                      |  symbol
1309pathname-specifier := pathname | string | symbol
1311method-form := (operation-name qual lambda-list @Arest{} body)
1312qual := method qualifier
1314component-dep-fail-option := :fail | :try-next | :ignore
1316feature-expression := keyword
1317                      | (:and @var{feature-expression}*)
1318                      | (:or @var{feature-expression}*)
1319                      | (:not @var{feature-expression})
1320@end example
1323@subsection Component names
1325Component names (@code{simple-component-name})
1326may be either strings or symbols.
1328@subsection Component types
1330Component type names, even if expressed as keywords, will be looked up
1331by name in the current package and in the asdf package, if not found in
1332the current package.  So a component type @code{my-component-type}, in
1333the current package @code{my-system-asd} can be specified as
1334@code{:my-component-type}, or @code{my-component-type}.
1336@code{system} and its subclasses are @emph{not}
1337allowed as component types for such children components.
1339@subsection System class names
1341A system class name will be looked up
1342in the same way as a Component type (see above),
1343except that only @code{system} and its subclasses are allowed.
1344Typically, one will not need to specify a system
1345class name, unless using a non-standard system class defined in some
1346ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
1347see below.  For such class names in the ASDF package, we recommend that
1348the @code{:class} option be specified using a keyword symbol, such as
1352@end example
1354This practice will ensure that package name conflicts are avoided.
1355Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
1356the current package @emph{before} it has been exported from the ASDF
1357extension loaded by @code{:defsystem-depends-on}, causing a name
1358conflict in the current package.
1360@subsection Defsystem depends on
1361@cindex :defsystem-depends-on
1363The @code{:defsystem-depends-on} option to @code{defsystem} allows the
1364programmer to specify another ASDF-defined system or set of systems that
1365must be loaded @emph{before} the system definition is processed.
1366Typically this is used to load an ASDF extension that is used in the
1367system definition.
1369@subsection Weakly depends on
1370@cindex :weakly-depends-on
1372We do @emph{NOT} recommend you use this feature.
1373If you are tempted to write a system @var{foo}
1374that weakly-depends-on a system @var{bar},
1375we recommend that you should instead
1376write system @var{foo} in a parametric way,
1377and offer some special variable and/or some hook to specialize its behavior;
1378then you should write a system @var{foo+bar}
1379that does the hooking of things together.
1381The (deprecated) @code{:weakly-depends-on} option to @code{defsystem}
1382allows the programmer to specify another ASDF-defined system or set of systems
1383that ASDF should @emph{try} to load,
1384but need not load in order to be successful.
1385Typically this is used if there are a number of systems
1386that, if present, could provide additional functionality,
1387but which are not necessary for basic function.
1389Currently, although it is specified to be an option only to @code{defsystem},
1390this option is accepted at any component, but it probably
1391only makes sense at the @code{defsystem} level.
1392Programmers are cautioned not
1393to use this component option except at the @code{defsystem} level, as
1394this anomalous behavior may be removed without warning.
1396@c Finally, you might look into the @code{asdf-system-connections} extension,
1397@c that will let you define additional code to be loaded
1398@c when two systems are simultaneously loaded.
1399@c It may or may not be considered good style, but at least it can be used
1400@c in a way that has deterministic behavior independent of load order,
1401@c unlike @code{weakly-depends-on}.
1404@subsection Pathname specifiers
1405@cindex pathname specifiers
1407A pathname specifier (@code{pathname-specifier})
1408may be a pathname, a string or a symbol.
1409When no pathname specifier is given for a component,
1410which is the usual case, the component name itself is used.
1412If a string is given, which is the usual case,
1413the string will be interpreted as a Unix-style pathname
1414where @code{/} characters will be interpreted as directory separators.
1415Usually, Unix-style relative pathnames are used
1416(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1417they are relative to the path of the parent component.
1418Finally, depending on the @code{component-type},
1419the pathname may be interpreted as either a file or a directory,
1420and if it's a file,
1421a file type may be added corresponding to the @code{component-type},
1422or else it will be extracted from the string itself (if applicable).
1424For instance, the @code{component-type} @code{:module}
1425wants a directory pathname, and so a string @code{"foo/bar"}
1426will be interpreted as the pathname @file{#p"foo/bar/"}.
1427On the other hand, the @code{component-type} @code{:file}
1428wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1429will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1430and a string @code{"foo/bar.quux"}
1431will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1432Finally, the @code{component-type} @code{:static-file}
1433wants a file without specifying a type, and so a string @code{"foo/bar"}
1434will be interpreted as the pathname @file{#p"foo/bar"},
1435and a string @code{"foo/bar.quux"}
1436will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1438ASDF interprets the string @code{".."}
1439as the pathname directory component word @code{:back},
1440which when merged, goes back one level in the directory hierarchy.
1442If a symbol is given, it will be translated into a string,
1443and downcased in the process.
1444The downcasing of symbols is unconventional,
1445but was selected after some consideration.
1446Observations suggest that the type of systems we want to support
1447either have lowercase as customary case (Unix, Mac, windows)
1448or silently convert lowercase to uppercase (lpns),
1449so this makes more sense than attempting to use @code{:case :common}
1450as argument to @code{make-pathname},
1451which is reported not to work on some implementations.
1453Pathname objects may be given to override the path for a component.
1454Such objects are typically specified using reader macros such as @code{#p}
1455or @code{#.(make-pathname ...)}.
1456Note however, that @code{#p...} is
1457a shorthand for @code{#.(parse-namestring ...)}
1458and that the behavior of @code{parse-namestring} is completely non-portable,
1459unless you are using Common Lisp @code{logical-pathname}s,
1460which themselves involve other non-portable behavior
1461(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1462Pathnames made with @code{#.(make-pathname ...)}
1463can usually be done more easily with the string syntax above.
1464The only case that you really need a pathname object is to override
1465the component-type default file type for a given component.
1466Therefore, pathname objects should only rarely be used.
1467Unhappily, ASDF 1 used not to properly support
1468parsing component names as strings specifying paths with directories,
1469and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1470An alternative to @code{#.} read-time evaluation is to use
1471@code{(eval `(defsystem ... ,pathname ...))}.
1473Note that when specifying pathname objects,
1474ASDF does not do any special interpretation of the pathname
1475influenced by the component type, unlike the procedure for
1476pathname-specifying strings.
1477On the one hand, you have to be careful to provide a pathname that correctly
1478fulfills whatever constraints are required from that component type
1479(e.g. naming a directory or a file with appropriate type);
1480on the other hand, you can circumvent the file type that would otherwise
1481be forced upon you if you were specifying a string.
1483@subsection Version specifiers
1484@cindex version specifiers
1485@cindex :version
1487Version specifiers are strings to be parsed as period-separated lists of integers.
1488I.e., in the example, @code{"0.2.1"} is to be interpreted,
1489roughly speaking, as @code{(0 2 1)}.
1490In particular, version @code{"0.2.1"} is interpreted the same as @code{"0.0002.1"},
1491though the latter is not canonical and may lead to a warning being issued.
1492Also, @code{"1.3"} and @code{"1.4"} are both strictly @code{uiop:version<} to @code{"1.30"},
1493quite unlike what would have happened
1494had the version strings been interpreted as decimal fractions.
1496Instead of a string representing the version,
1497the @code{:version} argument can be an expression that is resolved to
1498such a string using the following trivial domain-specific language:
1499in addition to being a literal string, it can be an expression of the form
1500@code{(:read-file-form <pathname-or-string> :at <access-at-specifier>)},
1501which will be resolved by reading a form in the specified pathname
1502(read as a subpathname of the current system if relative or a unix-namestring).
1503You may use a @code{uiop:access-at} specifier
1504with the (optional) @code{:at} keyword,
1505by default the specifier is @code{0}, meaning the first form is returned;
1506subforms can also be specified, with e.g. @code{(1 2 2)} specifying
1507``the third subform (index 2) of the third subform (index 2) of the second form (index 1)''
1508in the file (mind the off-by-one error in the English language).
1510System definers are encouraged to use version identifiers of the form
1511@var{x}.@var{y}.@var{z} for
1512major version, minor version and patch level,
1513where significant API incompatibilities are signaled by an increased major number.
1515@xref{Common attributes of components}.
1517@subsection Require
1518@cindex :require dependencies
1520Use the implementation's own @code{require} to load the @var{module-name}.
1523@subsection Using logical pathnames
1524@cindex logical pathnames
1526We do not generally recommend the use of logical pathnames,
1527especially not so to newcomers to Common Lisp.
1528However, we do support the use of logical pathnames by old timers,
1529when such is their preference.
1531To use logical pathnames,
1532you will have to provide a pathname object as a @code{:pathname} specifier
1533to components that use it, using such syntax as
1536You only have to specify such logical pathname
1537for your system or some top-level component.
1538Sub-components' relative pathnames,
1539specified using the string syntax for names,
1540will be properly merged with the pathnames of their parents.
1541The specification of a logical pathname host however is @emph{not}
1542otherwise directly supported in the ASDF syntax
1543for pathname specifiers as strings.
1545The @code{asdf-output-translation} layer will
1546avoid trying to resolve and translate logical pathnames.
1547The advantage of this is that
1548you can define yourself what translations you want to use
1549with the logical pathname facility.
1550The disadvantage is that if you do not define such translations,
1551any system that uses logical pathnames will behave differently under
1552asdf-output-translations than other systems you use.
1554If you wish to use logical pathnames you will have to configure the
1555translations yourself before they may be used.
1556ASDF currently provides no specific support
1557for defining logical pathname translations.
1559Note that the reasons we do not recommend logical pathnames are that
1560(1) there is no portable way to set up logical pathnames before they are used,
1561(2) logical pathnames are limited to only portably use
1562a single character case, digits and hyphens.
1563While you can solve the first issue on your own,
1564describing how to do it on each of fifteen implementations supported by ASDF
1565is more than we can document.
1566As for the second issue, mind that the limitation is notably enforced on SBCL,
1567and that you therefore can't portably violate the limitations
1568but must instead define some encoding of your own and add individual mappings
1569to name physical pathnames that do not fit the restrictions.
1570This can notably be a problem when your Lisp files are part of a larger project
1571in which it is common to name files or directories in a way that
1572includes the version numbers of supported protocols,
1573or in which files are shared with software written
1574in different programming languages where conventions include the use of
1575underscores, dots or CamelCase in pathnames.
1578@subsection Serial dependencies
1579@cindex serial dependencies
1581If the @code{:serial t} option is specified for a module,
1582ASDF will add dependencies for each child component,
1583on all the children textually preceding it.
1584This is done as if by @code{:depends-on}.
1587:serial t
1588:components ((:file "a") (:file "b") (:file "c"))
1589@end lisp
1591is equivalent to
1594:components ((:file "a")
1595             (:file "b" :depends-on ("a"))
1596             (:file "c" :depends-on ("a" "b")))
1597@end lisp
1600@subsection Source location (@code{:pathname})
1602The @code{:pathname} option is optional in all cases for systems
1603defined via @code{defsystem}, and generally is unnecessary.  In the
1604simple case, source files will be found in the same directory as the
1605system or, in the case of modules, in a subdirectory with the same name
1606as the module.
1608@c FIXME: This should be moved elsewhere -- it's too much detail for the
1609@c grammar section.
1611More specifically, ASDF follows a hairy set of rules that are designed so that
1615will load a system from disk
1616and have its pathname default to the right place.
1619This pathname information will not be overwritten with
1621(which could be somewhere else altogether)
1622if the user loads up the @file{.asd} file into his editor
1623and interactively re-evaluates that form.
1624@end enumerate
1626If a system is being loaded for the first time,
1627its top-level pathname will be set to:
1631The host/device/directory parts of @code{*load-truename*},
1632if it is bound.
1634@code{*default-pathname-defaults*}, otherwise.
1635@end itemize
1637If a system is being redefined, the top-level pathname will be
1641changed, if explicitly supplied or obtained from @code{*load-truename*}
1642(so that an updated source location is reflected in the system definition)
1645changed if it had previously been set from @code{*default-pathname-defaults*}
1648left as before, if it had previously been set from @code{*load-truename*}
1649and @code{*load-truename*} is currently unbound
1650(so that a developer can evaluate a @code{defsystem} form
1651from within an editor without clobbering its source location)
1652@end itemize
1654@subsection if-feature option
1655@cindex :if-feature component option
1656@anchor{if-feature-option}      @c redo if this ever becomes a node in
1657@c its own right...
1659This option allows you to specify a feature expression to be evaluated
1660as if by @code{#+} to conditionally include a component in your build.
1661If the expression is false, the component is dropped
1662as well as any dependency pointing to it.
1663As compared to using @code{#+} which is expanded at read-time,
1664this allows you to have an object in your component hierarchy
1665that can be used for manipulations beside building your project, and
1666that is accessible to outside code that wishes to reason about system
1669Programmers should be careful to consider @strong{when} the
1670@code{:if-feature} is evaluated.  Recall that ASDF first computes a
1671build plan, and then executes that plan.  ASDF will check to see whether
1672or not a feature is present @strong{at planning time}, not during the
1673build.  It follows that one cannot use @code{:if-feature} to check
1674features that are set during the course of the build.  It can only be
1675used to check the state of features before any build operations have
1676been performed.
1678This option was added in ASDF 3.  For more information,
1679@xref{required-features, Required features}.
1681@subsection if-component-dep-fails option
1682@cindex :if-component-dep-fails component option
1683This option was removed in ASDF 3.
1684Its semantics was limited in purpose and dubious to explain,
1685and its implementation was breaking a hole into the ASDF object model.
1686Please use the @code{if-feature} option instead.
1688@subsection feature requirement
1689This requirement was removed in ASDF 3.1.  Please do not use it.  In
1690most cases, @code{:if-feature} (@pxref{if-feature-option}) will provide
1691an adequate substitute.
1693The @code{feature} requirement used to ensure that a chain of component
1694dependencies would fail when a key feature was absent.
1695Used in conjunction with @code{:if-component-dep-fails}
1696this provided
1697a roundabout way to express conditional compilation.
1700@node Other code in .asd files, The package-inferred-system extension, The defsystem grammar, Defining systems with defsystem
1701@section Other code in .asd files
1703Files containing @code{defsystem} forms
1704are regular Lisp files that are executed by @code{load}.
1705Consequently, you can put whatever Lisp code you like into these files.
1706However, it is recommended to keep such forms to a minimal,
1707and to instead define @code{defsystem} extensions
1708that you use with @code{:defsystem-depends-on}.
1710If however, you might insist on including code in the @file{.asd} file itself,
1711e.g., to examine and adjust the compile-time environment,
1712possibly adding appropriate features to @code{*features*}.
1713If so, here are some conventions we recommend you follow,
1714so that users can control certain details of execution
1715of the Lisp in @file{.asd} files:
1719Any informative output
1720(other than warnings and errors,
1721which are the condition system's to dispose of)
1722should be sent to the standard CL stream @code{*standard-output*},
1723so that users can easily control the disposition
1724of output from ASDF operations.
1725@end itemize
1728@node The package-inferred-system extension,  , Other code in .asd files, Defining systems with defsystem
1729@section The package-inferred-system extension
1731Starting with release 3.1.2,
1732ASDF supports a one-package-per-file style of programming,
1733whereby each file is its own system,
1734and dependencies are deduced from the @code{defpackage} form
1735(or its variant @code{uiop:define-package}).
1738In this style, packages refer to a system with the same name (downcased);
1739and if a system is defined with @code{:class package-inferred-system},
1740then system names that start with that name
1741(using the slash @code{/} separator)
1742refer to files under the filesystem hierarchy where the system is defined.
1743For instance, if system @code{my-lib} is defined in
1744@file{/foo/bar/my-lib/my-lib.asd}, then system @code{my-lib/src/utility}
1745will be found in file @file{/foo/bar/my-lib/src/utility.lisp}.
1747This style was made popular by @code{faslpath} and @code{quick-build} before,
1748and at the cost of a stricter package discipline,
1749seems to make for more maintainable code.
1750It is used by ASDF itself (starting with ASDF 3) and by @code{lisp-interface-library}.
1752To use this style, choose a toplevel system name, e.g. @code{my-lib},
1753and create a file @file{my-lib.asd}
1754with the @code{:class :package-inferred-system} option in its @code{defsystem}.
1755For instance:
1757#-asdf (error "my-lib requires ASDF 3")
1758(defsystem my-lib
1759  :class :package-inferred-system
1760  :defsystem-depends-on (:asdf-package-system)
1761  :depends-on (:lil/interface/all
1762               :lil/pure/all
1763               :lil/stateful/all
1764               :lil/transform/all)
1765  :in-order-to ((test-op (load-op :lil/test/all)))
1766  :perform (test-op (o c) (symbol-call :lil/test/all :test-suite)))
1768(defsystem :lil/test :depends-on (:lil/test/all))
1770(register-system-packages :lil/interface/all '(:interface))
1771(register-system-packages :lil/pure/all '(:pure))
1772(register-system-packages :lil/stateful/all '(:stateful))
1773(register-system-packages :lil/transform/classy '(:classy))
1774(register-system-packages :lil/transform/posh '(:posh))
1775(register-system-packages :lil/test/all '(:lil/test))
1778 :closer-mop
1779 '(:c2mop :closer-common-lisp :c2cl :closer-common-lisp-user :c2cl-user))
1780@end example
1782In the code above, the
1783@code{:defsystem-depends-on (:asdf-package-system)} is
1784for compatibility with older versions of ASDF 3 (ASDF 2 is not supported),
1785and requires the @code{asdf-package-system} library to be present
1786(it is implicitly provided by ASDF starting with release 3.1.2,
1787which can be detected with the feature @code{:asdf3.1}).
1789The function @code{register-system-packages} has to be called to register
1790packages used or provided by your system and its components
1791where the name of the system that provides the package
1792is not the downcase of the package name.
1794Then, file @file{interface/order.lisp} under the @code{lil} hierarchy,
1795that defines abstract interfaces for order comparisons,
1796starts with the following form,
1797dependencies being trivially computed from the @code{:use} and @code{:mix} clauses:
1800(uiop:define-package :lil/interface/order
1801  (:use :closer-common-lisp
1802   :lil/interface/definition
1803   :lil/interface/base
1804   :lil/interface/eq :lil/interface/group)
1805  (:mix :fare-utils :uiop :alexandria)
1806  (:export ...))
1807@end example
1809ASDF can tell that this file depends on system @code{closer-mop} (registered above),
1810@code{lil/interface/definition}, @code{lil/interface/base},
1811@code{lil/interface/eq}, and @code{lil/interface/group}
1812(package and system names match, and they will be looked up hierarchically).
1814ASDF also detects dependencies from @code{:import-from} clauses.
1815To depend on a system without using a package or importing any symbol from it
1816(because you'll fully qualify them when used),
1817you may thus use an @code{:import-from} clause with an empty list of symbols, as in:
1820(defpackage :foo/bar
1821  (:use :cl)
1822  (:import-from :foo/baz #:sym1 #:sym2)
1823  (:import-from :foo/quux)
1824  (:export ...))
1825@end example
1827The form @code{uiop:define-package} is supported as well as @code{defpackage},
1828and has many options that prove useful in this context,
1829such as @code{:use-reexport} and @code{:mix-reexport}
1830that allow for ``inheritance'' of symbols being exported.
1832@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1833@comment  node-name,  next,  previous,  up
1834@chapter The Object model of ASDF
1835@tindex component
1836@tindex operation
1838ASDF is designed in an object-oriented way from the ground up.
1839Both a system's structure and the operations that can be performed on systems
1840follow a extensible protocol, allowing programmers to add new behaviors to ASDF.
1841For example, @code{cffi} adds support for special FFI description files
1842that interface with C libraries and for wrapper files that embed C code in Lisp.
1843@code{abcl-jar} supports creating Java JAR archives in ABCL.
1844@code{poiu} supports compiling code in parallel using background processes.
1846The key classes in ASDF are @code{component} and @code{operation}.
1847A @code{component} represents an individual source file or a group of source files,
1848and the products (e.g., fasl files) produced from it.
1849An @code{operation} represents a transformation that can be performed on a component,
1850turning them from source files to intermediate results to final outputs.
1851Components are related by @emph{dependencies}, specified in system
1854When ordered to @code{operate} with some operation on a component (usually a system),
1855ASDF will first compute a @emph{plan}
1856by traversing the dependency graph using function @code{make-plan}.@footnote{
1857  Historically, the function that built a plan was
1858  called @code{traverse}, and returned a list of actions;
1859  it was deprecated in favor of @code{make-plan} (that returns a plan object)
1860  when the @code{plan} objects were introduced;
1861  the old function is kept for backward compatibility and debugging purposes only.
1863The resulting plan object contains an ordered list of @emph{actions}.
1864An action is a pair of an @code{operation} and a @code{component},
1865representing  a particular build step to be @code{perform}ed.
1866The ordering of the plan ensures that no action is performed before
1867all its dependencies have been fulfilled.@footnote{
1868  The term @emph{action}
1869  was used by Kent Pitman in his article, ``The Description of Large Systems,''
1870  (@pxref{Bibliography}).
1871  Although the term was only used by ASDF hackers starting with ASDF 2,
1872  the concept was there since the very beginning of ASDF 1,
1873  just not clearly articulated.
1876In this chapter, we describe ASDF's object-oriented protocol,
1877the classes that make it up, and the generic functions on those classes.
1878These generic functions often take
1879both an operation and a component as arguments:
1880much of the power and configurability of ASDF is provided by
1881this use of CLOS's multiple dispatch.
1882We will describe the built-in component and operation classes, and
1883explain how to extend the ASDF protocol by defining new classes and
1884methods for ASDF's generic functions.
1885We will also describe the many @emph{hooks} that can be configured to
1886customize the behavior of existing @emph{functions}.
1888@c FIXME: Swap operations and components.
1889@c FIXME: Possibly add a description of the PLAN object.
1890@c Not critical, since the user isn't expected to interact with it.
1892* Operations::
1893* Components::
1894* Dependencies::
1895* Functions::
1896@end menu
1898@node  Operations, Components, The object model of ASDF, The object model of ASDF
1899@comment  node-name,  next,  previous,  up
1900@section Operations
1901@cindex operation
1903An @dfn{operation} object of the appropriate type is instantiated
1904whenever the user wants to do something with a system like
1907@item compile all its files
1908@item load the files into a running lisp environment
1909@item copy its source files somewhere else
1910@end itemize
1912Operations can be invoked directly, or examined
1913to see what their effects would be without performing them.
1914There are a bunch of methods specialised on operation and component type
1915that actually do the grunt work.
1916Operations are invoked on systems via @code{operate} (@pxref{operate}).
1918ASDF contains a number of pre-defined @t{operation} classes for common,
1919and even fairly uncommon tasks that you might want to do with it.
1920In addition, ASDF contains ``abstract'' @t{operation} classes that
1921programmers can use as building blocks to define ASDF extensions.  We
1922discuss these in turn below.
1924@c The operation object contains whatever state is relevant for this purpose
1925@c (perhaps a list of visited nodes, for example)
1926@c but primarily is a nice thing to specialise operation methods on
1927@c and easier than having them all be @code{EQL} methods.
1930* Predefined operations of ASDF::
1931* Creating new operations::
1932@end menu
1934Operations are invoked on systems via @code{operate}.
1936@deffn {Generic function} @code{operate} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @code{force} @code{force-not} @code{verbose} @AallowOtherKeys
1937@deffnx {Generic function} @code{oos} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @AallowOtherKeys{}
1938@code{operate} invokes @var{operation} on @var{system}.
1939@code{oos} is a synonym for @code{operate} (it stands for operate-on-system).
1941@var{operation} is a symbol that is passed,
1942along with the supplied @var{initargs},
1943to @code{make-operation} (which will call @code{make-instance})
1944to create the operation object.
1945@var{component} is a component designator,
1946usually a string or symbol that designates a system,
1947sometimes a list of strings or symbols that designate a subcomponent of a system.
1949The @var{initargs} are passed to the @code{make-instance} call
1950when creating the operation object.
1951@c We probably want to deprecate that, because
1952@c (1) there is a mix of flags for operate, for the operation-class, for the plan-class, etc.
1953@c (2) flags to operations have never been well-supported, anyway.
1954@c The future solution probably involves having an explicit :operation-options keyword or some such
1955@c (if operation options are not wholly eliminated), a separate :plan-options, etc.
1956Note that dependencies may cause the operation
1957to invoke other operations on the system or its components:
1958the new operations will be created
1959with the same @var{initargs} as the original one.
1961If @var{force} is @code{:all}, then all systems
1962are forced to be recompiled even if not modified since last compilation.
1963If @var{force} is @code{t}, then only the system being loaded
1964is forced to be recompiled even if not modified since last compilation,
1965but other systems are not affected.
1966If @var{force} is a list, then it specifies a list of systems that
1967are forced to be recompiled even if not modified since last compilation.
1968If @var{force-not} is @code{:all}, then all systems
1969are forced not to be recompiled even if modified since last compilation.
1970If @var{force-not} is @code{t}, then all systems but the system being loaded
1971are forced not to be recompiled even if modified since last compilation
1972(note: this was changed in ASDF 3.1.2).
1973If @var{force-not} is a list, then it specifies a list of systems that
1974are forced not to be recompiled even if modified since last compilation.
1976Both @var{force} and @var{force-not} apply to systems that are dependencies and were already compiled.
1977@var{force-not} takes precedences over @var{force},
1978as it should, really, but unhappily only since ASDF 3.1.2.
1979Moreover, systems the name of which is member of the set @var{*immutable-systems*}
1980(represented as an equal hash-table) are always considered @var{forced-not}, and
1981even their @file{.asd} is not refreshed from the filesystem.
1983To see what @code{operate} would do, you can use:
1985(asdf:traverse operation-class system-name)
1986@end example
1988@end deffn
1992@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1993@comment  node-name,  next,  previous,  up
1994@subsection Predefined operations of ASDF
1995@c FIXME: All these deffn's should be replaced with deftyp.  Also, we
1996@c should set up an appropriate index.
1998All the operations described in this section are in the @code{asdf} package.
1999They are invoked via the @code{operate} generic function.
2002(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
2003@end lisp
2005@deffn Operation @code{compile-op}
2007This operation compiles the specified component.
2008A @code{cl-source-file} will be @code{compile-file}'d.
2009All the children and dependencies of a system or module
2010will be recursively compiled by @code{compile-op}.
2012@code{compile-op} depends on @code{prepare-op} which
2013itself depends on a @code{load-op} of all of a component's dependencies,
2014as well as of its parent's dependencies.
2015When @code{operate} is called on @code{compile-op},
2016all these dependencies will be loaded as well as compiled;
2017yet, some parts of the system main remain unloaded,
2018because nothing depends on them.
2019Use @code{load-op} to load a system.
2020@end deffn
2022@deffn Operation @code{load-op}
2024This operation loads the compiled code for a specified component.
2025A @code{cl-source-file} will have its compiled fasl @code{load}ed,
2026which fasl is the output of @code{compile-op} that @code{load-op} depends on.
2028@code{load-op} will recursively load all the children of a system or module.
2030@code{load-op} also depends on @code{prepare-op} which
2031itself depends on a @code{load-op} of all of a component's dependencies,
2032as well as of its parent's dependencies.
2033@end deffn
2035@deffn Operation @code{prepare-op}
2037This operation ensures that the dependencies of a component
2038and its recursive parents are loaded (as per @code{load-op}),
2039as a prerequisite before @code{compile-op} and @code{load-op} operations
2040may be performed on a given component.
2041@end deffn
2043@deffn Operation @code{load-source-op}, @code{prepare-source-op}
2045@code{load-source-op} will load the source for the files in a module
2046rather than the compiled fasl output.
2047It has a @code{prepare-source-op} analog to @code{prepare-op},
2048that ensures the dependencies are themselves loaded via @code{load-source-op}.
2050@end deffn
2053@deffn Operation @code{test-op}
2055This operation will perform some tests on the module.
2056The default method will do nothing.
2057The default dependency is to require
2058@code{load-op} to be performed on the module first.
2059Its @code{operation-done-p} method returns @code{nil},
2060which means that the operation is @emph{never} done
2062we assume that if you invoke the @code{test-op},
2063you want to test the system, even if you have already done so.
2065The results of this operation are not defined by ASDF.
2066It has proven difficult to define how the test operation
2067should signal its results to the user
2068in a way that is compatible with all of the various test libraries
2069and test techniques in use in the community, and
2070given the fact that ASDF operations do not return a value indicating
2071success or failure.
2072For those willing to go to the effort, we suggest defining conditions to
2073signal when a @code{test-op} fails, and storing in those conditions
2074information that describes which tests fail.
2076People typically define a separate test @emph{system} to hold the tests.
2077Doing this avoids unnecessarily adding a test framework as a dependency
2078on a library.  For example, one might have
2080(defsystem foo
2081  :in-order-to ((test-op (test-op "foo/test")))
2082 ...)
2084(defsystem foo/test
2085  :depends-on (foo fiveam) ; fiveam is a test framework library
2086  ...)
2087@end lisp
2089Then one defines @code{perform} methods on
2090@code{test-op} such as the following:
2092(defsystem foo/test
2093  :depends-on (foo fiveam) ; fiveam is a test framework library
2094  :perform (test-op (o s)
2095                    (uiop:symbol-call :fiveam  '#:run!
2096                       (uiop:find-symbol* '#:foo-test-suite
2097                                            :foo-tests)))
2098  ...)
2099@end lisp
2101@end deffn
2105@deffn Operation @code{compile-bundle-op}, @code{monolithic-compile-bundle-op}, @code{load-bundle-op}, @code{monolithic-load-bundle-op}, @code{deliver-asd-op}, @code{monolithic-deliver-asd-op}, @code{lib-op}, @code{monolithic-lib-op}, @code{dll-op}, @code{monolithic-dll-op}, @code{image-op}, @code{program-op}
2107These are ``bundle'' operations, that can create a single-file ``bundle''
2108for all the contents of each system in an application,
2109or for the entire application.
2111@code{compile-bundle-op} will create a single fasl file for each of the systems needed,
2112grouping all its many fasls in one,
2113so you can deliver each system as a single fasl
2114@code{monolithic-compile-bundle-op} will create a single fasl file for the target system
2115and all its dependencies,
2116so you can deliver your entire application as a single fasl.
2117@code{load-bundle-op} will load the output of @code{compile-bundle-op}.
2118Note that if it the output is not up-to-date,
2119@code{compile-bundle-op} may load the intermediate fasls as a side-effect.
2120Bundling fasls together matters a lot on ECL,
2121where the dynamic linking involved in loading tens of individual fasls
2122can be noticeably more expensive than loading a single one.
2124NB: @code{compile-bundle-op}, @code{monolithic-compile-bundle-op}, @code{load-bundle-op}, @code{monolithic-load-bundle-op}, @code{deliver-asd-op}, @code{monolithic-deliver-asd-op} were respectively called
2125@code{fasl-op}, @code{monolithic-fasl-op}, @code{load-fasl-op}, @code{monolithic-load-fasl-op}, @code{binary-op}, @code{monolithic-binary-op} before ASDF 3.1.
2126The old names still exist for backward compatibility,
2127though they poorly label what is going on.
2129Once you have created a fasl with @code{compile-bundle-op},
2130you can use @code{precompiled-system} to deliver it in a way
2131that is compatible with clients having dependencies on your system,
2132whether it is distributed as source or as a single binary;
2133the @file{.asd} file to be delivered with the fasl will look like this:
2135(defsystem :mysystem :class :precompiled-system
2136  :fasl (some expression that will evaluate to a pathname))
2137@end example
2138Or you can use @code{deliver-asd-op} to let ASDF create such a system for you
2139as well as the @code{compile-bundle-op} output,
2140or @code{monolithic-deliver-asd-op}.
2141This allows you to deliver code for your systems or applications
2142as a single file.
2143Of course, if you want to test the result in the current image,
2144@emph{before} you try to use any newly created @file{.asd} files,
2145you should not forget to @code{(asdf:clear-configuration)}
2146or at least @code{(asdf:clear-source-registry)},
2147so it re-populates the source-registry from the filesystem.
2149The @code{program-op} operation will create an executable program
2150from the specified system and its dependencies.
2151You can use UIOP for its pre-image-dump hooks, its post-image-restore hooks,
2152and its access to command-line arguments.
2153And you can specify an entry point @code{my-app:main}
2154by specifying in your @code{defsystem}
2155the option @code{:entry-point "my-app:main"}.
2156Depending on your implementation,
2157running @code{(asdf:operate 'asdf:program-op :my-app)}
2158may quit the current Lisp image upon completion.
2159See the example in
2160@file{test/hello-world-example.asd} and @file{test/hello.lisp},
2161as built and tested by
2162@file{test/test-program.script} and @file{test/make-hello-world.lisp}.
2163@code{image-op} will dump an image that may not be standalone
2164and does not start its own function,
2165but follows the usual execution convention of the underlying Lisp,
2166just with more code pre-loaded,
2167for use as an intermediate build result or with a wrapper invocation script.
2169There is also @code{lib-op}
2170for building a linkable @file{.a} file (Windows: @file{.lib})
2171from all linkable object dependencies (FFI files, and on ECL, Lisp files too),
2172and its monolithic equivalent @code{monolithic-lib-op}.
2173And there is also @code{dll-op}
2174(respectively its monolithic equivalent @code{monolithic-lib-op})
2175for building a linkable @file{.so} file
2176(Windows: @file{.dll}, MacOS X: @file{.dynlib})
2177to create a single dynamic library
2178for all the extra FFI code to be linked into each of your systems
2179(respectively your entire application).
2181All these ``bundle'' operations are available since ASDF 3
2182on all actively supported Lisp implementations,
2183but may be unavailable on unmaintained legacy implementations.
2184This functionality was previously available for select implementations,
2185as part of a separate system @code{asdf-bundle},
2186itself descended from the ECL-only @code{asdf-ecl}.
2188The pathname of the output of bundle operations
2189is subject to output-translation as usual,
2190unless the operation is equal to
2191the @code{:build-operation} argument to @code{defsystem}.
2192This behavior is not very satisfactory and may change in the future.
2193Maybe you have suggestions on how to better configure it?
2194@end deffn
2196@deffn Operation @code{concatenate-source-op}, @code{monolithic-concatenate-source-op}, @code{load-concatenated-source-op}, @code{compile-concatenated-source-op}, @code{load-compiled-concatenated-source-op}, @code{monolithic-load-concatenated-source-op}, @code{monolithic-compile-concatenated-source-op}, @code{monolithic-load-compiled-concatenated-source-op}
2198These operations, as their respective names indicate,
2199will concatenate all the @code{cl-source-file} source files in a system
2200(or in a system and all its dependencies, if monolithic),
2201in the order defined by dependencies,
2202then load the result, or compile and then load the result.
2204These operations are useful to deliver a system or application
2205as a single source file,
2206and for testing that said file loads properly, or compiles and then loads properly.
2208ASDF itself is delivered as a single source file this way,
2209using @code{monolithic-concatenate-source-op},
2210prepending a prelude and the @code{uiop} library
2211before the @code{asdf/defsystem} system itself.
2212@end deffn
2215@node  Creating new operations,  , Predefined operations of ASDF, Operations
2216@comment  node-name,  next,  previous,  up
2217@subsection Creating new operations
2219ASDF was designed to be extensible in an object-oriented fashion.
2220To teach ASDF new tricks, a programmer can implement the behaviour he wants
2221by creating a subclass of @code{operation}.
2223ASDF's pre-defined operations are in no way ``privileged'',
2224but it is requested that developers never use the @code{asdf} package
2225for operations they develop themselves.
2226The rationale for this rule is that we don't want to establish a
2227``global asdf operation name registry'',
2228but also want to avoid name clashes.
2230Your operation @emph{must} usually provide methods
2231for one or more of the following generic functions:
2235@findex perform
2236@item @code{perform}
2237Unless your operation, like @code{prepare-op},
2238is for dependency propagation only,
2239the most important function for which to define a method
2240is usually @code{perform},
2241which will be called to perform the operation on a specified component,
2242after all dependencies have been performed.
2244The @code{perform} method must call @code{input-files} and @code{output-files} (see below)
2245to locate its inputs and outputs,
2246because the user is allowed to override the method
2247or tweak the output-translation mechanism.
2248Perform should only use the primary value returned by @code{output-files}.
2249If one and only one output file is expected,
2250it can call @code{output-file} that checks that this is the case
2251and returns the first and only list element.
2253@findex output-files
2254@item @code{output-files}
2255If your perform method has any output,
2256you must define a method for this function.
2257for ASDF to determine where the outputs of performing operation lie.
2259Your method may return two values, a list of pathnames, and a boolean.
2260If the boolean is @code{nil} (or you fail to return multiple values),
2261then enclosing @code{:around} methods may translate these pathnames,
2262e.g. to ensure object files are somehow stored
2263in some implementation-dependent cache.
2264If the boolean is @code{t} then the pathnames are marked
2265not be translated by the enclosing @code{:around} method.
2267@findex component-depends-on
2268@item @code{component-depends-on}
2269If the action of performing the operation on a component has dependencies,
2270you must define a method on @code{component-depends-on}.
2272Your method will take as specialized arguments
2273an operation and a component which together identify an action,
2274and return a list of entries describing actions that this action depends on.
2275The format of entries is described below.
2277It is @emph{strongly} advised that
2278you should always append the results of @code{(call-next-method)}
2279to the results of your method,
2280or ``interesting'' failures will likely occur,
2281unless you're a true specialist of ASDF internals.
2282It is unhappily too late to compatibly use the @code{append} method combination,
2283but conceptually that's the protocol that is being manually implemented.
2285Each entry returned by @code{component-depends-on} is itself a list.
2287The first element of an entry is an operation designator:
2288either an operation object designating itself, or
2289a symbol that names an operation class
2290(that ASDF will instantiate using @code{make-operation}).
2291For instance, @code{load-op}, @code{compile-op} and @code{prepare-op}
2292are common such names, denoting the respective operations.
2294@c FIXME COERCE-NAME is referenced, but not defined.
2295@findex coerce-name
2296@findex find-component
2297The rest of each entry is a list of component designators:
2298either a component object designating itself,
2299or an identifier to be used with @code{find-component}.
2300@code{find-component} will be called with the current component's parent as parent,
2301and the identifier as second argument.
2302The identifier is typically a string,
2303a symbol (to be downcased as per @code{coerce-name}),
2304or a list of strings or symbols.
2305In particular, the empty list @code{nil} denotes the parent itself.
2307@end itemize
2309An operation @emph{may} provide methods for the following generic functions:
2313@item @code{input-files}
2314@findex input-files
2315A method for this function is often not needed,
2316since ASDF has a pretty clever default @code{input-files} mechanism.
2317You only need create a method if there are multiple ultimate input files,
2318and/or the bottom one doesn't depend
2319on the @code{component-pathname} of the component.
2321@item @code{operation-done-p}
2322@findex operation-done-p
2323You only need to define a method on that function
2324if you can detect conditions that invalidate previous runs of the operation,
2325even though no filesystem timestamp has changed,
2326in which case you return @code{nil} (the default is @code{t}).
2328For instance, the method for @code{test-op} always returns @code{nil},
2329so that tests are always run afresh.
2330Of course, the @code{test-op} for your system could depend
2331on a deterministically repeatable @code{test-report-op},
2332and just read the results from the report files,
2333in which case you could have this method return @code{t}.
2335@end itemize
2337Operations that print output should send that output to the standard
2338CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
2340@node Components, Dependencies, Operations, The object model of ASDF
2341@comment  node-name,  next,  previous,  up
2342@section Components
2343@cindex component
2344@cindex system
2345@cindex system designator
2346@cindex component designator
2347@vindex *system-definition-search-functions*
2349A @code{component} represents an individual source file or a group of source files,
2350and the things that get transformed into.
2351A @code{system} is a component at the top level of the component hierarchy,
2352that can be found via @code{find-system}.
2353A @code{source-file} is a component representing a single source-file
2354and the successive output files into which it is transformed.
2355A @code{module} is an intermediate component itself grouping several other components,
2356themselves source-files or further modules.
2358A @dfn{system designator} is a system itself,
2359or a string or symbol that behaves just like any other component name
2360(including with regard to the case conversion rules for component names).
2362A @dfn{component designator}, relative to a base component,
2363is either a component itself,
2364or a string or symbol,
2365or a list of designators.
2367@defun find-system system-designator @Aoptional{} (error-p t)
2369Given a system designator, @code{find-system} finds and returns a system.
2370If no system is found, an error of type
2371@code{missing-component} is thrown,
2372or @code{nil} is returned if @code{error-p} is false.
2374To find and update systems, @code{find-system} funcalls each element
2375in the @code{*system-definition-search-functions*} list,
2376expecting a pathname to be returned, or a system object,
2377from which a pathname may be extracted, and that will be registered.
2378The resulting pathname (if any) is loaded
2379if one of the following conditions is true:
2383there is no system of that name in memory
2385the pathname is different from that which was previously loaded
2387the file's @code{last-modified} time exceeds the @code{last-modified} time
2388of the system in memory
2389@end itemize
2391When system definitions are loaded from @file{.asd} files,
2392a new scratch package is created for them to load into,
2393so that different systems do not overwrite each others operations.
2394The user may also wish to (and is recommended to)
2395include @code{defpackage} and @code{in-package} forms
2396in his system definition files, however,
2397so that they can be loaded manually if need be.
2399The default value of @code{*system-definition-search-functions*}
2400is a list of two functions.
2401The first function looks in each of the directories given
2402by evaluating members of @code{*central-registry*}
2403for a file whose name is the name of the system and whose type is @file{asd}.
2404The first such file is returned,
2405whether or not it turns out to actually define the appropriate system.
2406The second function does something similar,
2407for the directories specified in the @code{source-registry}.
2408Hence, it is strongly advised to define a system
2409@var{foo} in the corresponding file @var{foo.asd}.
2410@end defun
2412@defun find-component base path
2414Given a @var{base} component (or designator for such),
2415and a @var{path}, find the component designated by the @var{path}
2416starting from the @var{base}.
2418If @var{path} is a component object, it designates itself,
2419independently from the base.
2421@findex coerce-name
2422If @var{path} is a string, or symbol denoting a string via @code{coerce-name},
2423then @var{base} is resolved to a component object,
2424which must be a system or module,
2425and the designated component is the child named by the @var{path}.
2427If @var{path} is a @code{cons} cell,
2428@code{find-component} with the base and the @code{car} of the @var{path},
2429and the resulting object is used as the base for a tail call
2430to @code{find-component} with the @code{car} of the @var{path}.
2432If @var{base} is a component object, it designates itself.
2434If @var{base} is null, then @var{path} is used as the base, with @code{nil} as the path.
2436If @var{base} is a string, or symbol denoting a string via @code{coerce-name},
2437it designates a system as per @code{find-system}.
2439If @var{base} is a @code{cons} cell, it designates the component found by
2440@code{find-component} with its @code{car} as base and @code{cdr} as path.
2441@end defun
2445* Common attributes of components::
2446* Pre-defined subclasses of component::
2447* Creating new component types::
2448@end menu
2450@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
2451@comment  node-name,  next,  previous,  up
2452@subsection Common attributes of components
2454All components, regardless of type, have the following attributes.
2455All attributes except @code{name} are optional.
2457@subsubsection Name
2458@findex coerce-name
2459A component name is a string or a symbol.
2460If a symbol, its name is taken and lowercased.  This translation is
2461performed by the exported function @code{coerce-name}.
2463Unless overridden by a @code{:pathname} attribute,
2464the name will be interpreted as a pathname specifier according
2465to a Unix-style syntax.
2466@xref{The defsystem grammar,,Pathname specifiers}.
2468@subsubsection Version identifier
2469@findex version-satisfies
2470@cindex :version
2472This optional attribute specifies a version for the current component.
2473The version should typically be a string of integers separated by dots,
2474for example @samp{1.0.11}.
2475For more information on version specifiers, see @ref{The defsystem grammar}.
2477A version may then be queried by the generic function @code{version-satisfies},
2478to see if @code{:version} dependencies are satisfied,
2479and when specifying dependencies, a constraint of minimal version to satisfy
2480can be specified using e.g. @code{(:version "mydepname" "1.0.11")}.
2482Note that in the wild, we typically see version numbering
2483only on components of type @code{system}.
2484Presumably it is much less useful within a given system,
2485wherein the library author is responsible to keep the various files in synch.
2487@subsubsection Required features
2490Traditionally defsystem users have used @code{#+} reader conditionals
2491to include or exclude specific per-implementation files.
2492For example, CFFI, the portable C foreign function interface contained
2493lines like:
2495     #+sbcl       (:file "cffi-sbcl")
2496@end lisp
2497An unfortunate side effect of this approach is that no single
2498implementation can read the entire system.
2499This causes problems if, for example, one wished to design an @code{archive-op}
2500that would create an archive file containing all the sources, since
2501for example the file @code{cffi-sbcl.lisp} above would be invisible when
2502running the @code{archive-op} on any implementation other than SBCL.
2504Starting with ASDF 3,
2505components may therefore have an @code{:if-feature} option.
2506The value of this option should be
2507a feature expression using the same syntax as @code{#+} does.
2508If that feature expression evaluates to false, any reference to the component will be ignored
2509during compilation, loading and/or linking.
2510Since the expression is read by the normal reader,
2511you must explicitly prefix your symbols with @code{:} so they be read as keywords;
2512this is as contrasted with the @code{#+} syntax
2513that implicitly reads symbols in the keyword package by default.
2515For instance, @code{:if-feature (:and :x86 (:or :sbcl :cmu :scl))} specifies that
2516the given component is only to be compiled and loaded
2517when the implementation is SBCL, CMUCL or Scieneer CL on an x86 machine.
2518You cannot write it as @code{:if-feature (and x86 (or sbcl cmu scl))}
2519since the symbols would not be read as keywords.
2523@subsubsection Dependencies
2525This attribute specifies dependencies of the component on its siblings.
2526It is optional but often necessary.
2528There is an excitingly complicated relationship between the initarg
2529and the method that you use to ask about dependencies
2531Dependencies are between (operation component) pairs.
2532In your initargs for the component, you can say
2535:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
2536              (load-op (load-op "foo")))
2537@end lisp
2539This means the following things:
2542before performing compile-op on this component, we must perform
2543load-op on @var{a} and @var{b}, and compile-op on @var{c},
2545before performing @code{load-op}, we have to load @var{foo}
2546@end itemize
2548The syntax is approximately
2551(this-op @{(other-op required-components)@}+)
2553simple-component-name := string
2554                      |  symbol
2556required-components := simple-component-name
2557                     | (required-components required-components)
2559component-name := simple-component-name
2560                | (:version simple-component-name minimum-version-object)
2561@end verbatim
2563Side note:
2565This is on a par with what ACL defsystem does.
2566mk-defsystem is less general: it has an implied dependency
2569  for all source file x, (load x) depends on (compile x)
2570@end verbatim
2572and using a @code{:depends-on} argument to say that @var{b} depends on
2573@var{a} @emph{actually} means that
2576  (compile b) depends on (load a)
2577@end verbatim
2579This is insufficient for e.g. the McCLIM system, which requires that
2580all the files are loaded before any of them can be compiled ]
2582End side note
2584In ASDF, the dependency information for a given component and operation
2585can be queried using @code{(component-depends-on operation component)},
2586which returns a list
2589((load-op "a") (load-op "b") (compile-op "c") ...)
2590@end lisp
2592@code{component-depends-on} can be subclassed for more specific
2593component/operation types: these need to @code{(call-next-method)}
2594and append the answer to their dependency, unless
2595they have a good reason for completely overriding the default dependencies.
2597If it weren't for CLISP, we'd be using @code{LIST} method
2598combination to do this transparently.
2599But, we need to support CLISP.
2600If you have the time for some CLISP hacking,
2601I'm sure they'd welcome your fixes.
2602@c Doesn't CLISP now support LIST method combination?
2604A minimal version can be specified for a component you depend on
2605(typically another system), by specifying @code{(:version "other-system" "1.2.3")}
2606instead of simply @code{"other-system"} as the dependency.
2607See the discussion of the semantics of @code{:version}
2608in the defsystem grammar.
2610@c FIXME: Should have cross-reference to "Version specifiers" in the
2611@c defsystem grammar, but the cross-referencing is so broken by
2612@c insufficient node breakdown that I have not put one in.
2615@subsubsection pathname
2617This attribute is optional and if absent (which is the usual case),
2618the component name will be used.
2620@xref{The defsystem grammar,,Pathname specifiers},
2621for an explanation of how this attribute is interpreted.
2623Note that the @code{defsystem} macro (used to create a ``top-level'' system)
2624does additional processing to set the filesystem location of
2625the top component in that system.
2626This is detailed elsewhere. @xref{Defining systems with defsystem}.
2629@subsubsection properties
2631This attribute is optional.
2633Packaging systems often require information about files or systems
2634in addition to that specified by ASDF's pre-defined component attributes.
2635Programs that create vendor packages out of ASDF systems therefore
2636have to create ``placeholder'' information to satisfy these systems.
2637Sometimes the creator of an ASDF system may know the additional
2638information and wish to provide it directly.
2640@code{(component-property component property-name)} and
2641associated @code{setf} method will allow
2642the programmatic update of this information.
2643Property names are compared as if by @code{EQL},
2644so use symbols or keywords or something.
2647* Pre-defined subclasses of component::
2648* Creating new component types::
2649@end menu
2651@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
2652@comment  node-name,  next,  previous,  up
2653@subsection Pre-defined subclasses of component
2655@deffn Component source-file
2657A source file is any file that the system does not know how to
2658generate from other components of the system.
2660Note that this is not necessarily the same thing as
2661``a file containing data that is typically fed to a compiler''.
2662If a file is generated by some pre-processor stage
2663(e.g. a @file{.h} file from @file{} by autoconf)
2664then it is not, by this definition, a source file.
2665Conversely, we might have a graphic file
2666that cannot be automatically regenerated,
2667or a proprietary shared library that we received as a binary:
2668these do count as source files for our purposes.
2670Subclasses of source-file exist for various languages.
2671@emph{FIXME: describe these.}
2672@end deffn
2674@deffn Component module
2676A module is a collection of sub-components.
2678A module component has the following extra initargs:
2682@code{:components} the components contained in this module
2686All children components which don't specify their class explicitly
2687are inferred to be of this type.
2691This attribute was removed in ASDF 3. Do not use it.
2692Use @code{:if-feature} instead (@pxref{required-features}, and @pxref{if-feature-option}).
2695@code{:serial} When this attribute is set,
2696each subcomponent of this component is assumed to depend on all subcomponents
2697before it in the list given to @code{:components}, i.e.
2698all of them are loaded before a compile or load operation is performed on it.
2700@end itemize
2702The default operation knows how to traverse a module, so
2703most operations will not need to provide methods specialised on modules.
2705@code{module} may be subclassed to represent components such as
2706foreign-language linked libraries or archive files.
2707@end deffn
2709@deffn Component system
2711@code{system} is a subclass of @code{module}.
2713A system is a module with a few extra attributes for documentation
2714purposes; these are given elsewhere.
2715@xref{The defsystem grammar}.
2717Users can create new classes for their systems:
2718the default @code{defsystem} macro takes a @code{:class} keyword argument.
2719@end deffn
2721@node  Creating new component types,  , Pre-defined subclasses of component, Components
2722@comment  node-name,  next,  previous,  up
2723@subsection Creating new component types
2725New component types are defined by subclassing one of the existing
2726component classes and specializing methods on the new component class.
2728@emph{FIXME: this should perhaps be explained more throughly,
2729not only by example ...}
2731As an example, suppose we have some implementation-dependent
2732functionality that we want to isolate
2733in one subdirectory per Lisp implementation our system supports.
2734We create a subclass of
2738(defclass unportable-cl-source-file (cl-source-file)
2739  ())
2740@end lisp
2742Function @code{asdf:implementation-type} (exported since 2.014.14)
2743gives us the name of the subdirectory.
2744All that's left is to define how to calculate the pathname
2745of an @code{unportable-cl-source-file}.
2748(defmethod component-pathname ((component unportable-cl-source-file))
2749  (merge-pathnames*
2750   (parse-unix-namestring (format nil "~(~A~)/" (asdf:implementation-type)))
2751   (call-next-method)))
2752@end lisp
2754The new component type is used in a @code{defsystem} form in this way:
2757(defsystem :foo
2758    :components
2759    ((:file "packages")
2760     ...
2761     (:unportable-cl-source-file "threads"
2762      :depends-on ("packages" ...))
2763     ...
2764    )
2765@end lisp
2767@node Dependencies, Functions, Components, The object model of ASDF
2768@section Dependencies
2769@c FIXME: Moved this material here, but it isn't very comfortable
2770@c here....  Also needs to be revised to be coherent.
2772To be successfully buildable, this graph of actions but be acyclic.
2773If, as a user, extender or implementer of ASDF, you fail
2774to keep the dependency graph without cycles,
2775ASDF will fail loudly as it eventually finds one.
2776To clearly distinguish the direction of dependencies,
2777ASDF 3 uses the words @emph{requiring} and @emph{required}
2778as applied to an action depending on the other:
2779the requiring action @code{depends-on} the completion of all required actions
2780before it may itself be @code{perform}ed.
2782Using the @code{defsystem} syntax, users may easily express
2783direct dependencies along the graph of the object hierarchy:
2784between a component and its parent, its children, and its siblings.
2785By defining custom CLOS methods, you can express more elaborate dependencies as you wish.
2786Most common operations, such as @code{load-op}, @code{compile-op} or @code{load-source-op}
2787are automatically propagate ``downward'' the component hierarchy and are ``covariant'' with it:
2788to act the operation on the parent module, you must first act it on all the children components,
2789with the action on the parent being parent of the action on each child.
2790Other operations, such as @code{prepare-op} and @code{prepare-source-op}
2791(introduced in ASDF 3) are automatically propagated ``upward'' the component hierarchy
2792and are ``contravariant'' with it:
2793to perform the operation of preparing for compilation of a child component,
2794you must perform the operation of preparing for compilation of its parent component, and so on,
2795ensuring that all the parent's dependencies are (compiled and) loaded
2796before the child component may be compiled and loaded.
2797Yet other operations, such as @code{test-op} or @code{load-bundle-op}
2798remain at the system level, and are not propagated along the hierarchy,
2799but instead do something global on the system.
2802@node Functions,  , Dependencies, The object model of ASDF
2803@comment  node-name,  next,  previous,  up
2804@section Functions
2806@c FIXME: this does not belong here....
2807@defun version-satisfies @var{version} @var{version-spec}
2808Does @var{version} satisfy the @var{version-spec}.  A generic function.
2809ASDF provides built-in methods for @var{version} being a @code{component} or @code{string}.
2810@var{version-spec} should be a string.
2811If it's a component, its version is extracted as a string before further processing.
2813A version string satisfies the version-spec if after parsing,
2814the former is no older than the latter.
2815Therefore @code{"1.9.1"}, @code{"1.9.2"} and @code{"1.10"} all satisfy @code{"1.9.1"},
2816but @code{"1.8.4"} or @code{"1.9"} do not.
2817For more information about how @code{version-satisfies} parses and interprets
2818version strings and specifications,
2819@pxref{The defsystem grammar} (version specifiers) and
2820@ref{Common attributes of components}.
2822Note that in versions of ASDF prior to 3.0.1,
2823including the entire ASDF 1 and ASDF 2 series,
2824@code{version-satisfies} would also require that the version and the version-spec
2825have the same major version number (the first integer in the list);
2826if the major version differed, the version would be considered as not matching the spec.
2827But that feature was not documented, therefore presumably not relied upon,
2828whereas it was a nuisance to several users.
2829Starting with ASDF 3.0.1,
2830@code{version-satisfies} does not treat the major version number specially,
2831and returns T simply if the first argument designates a version that isn't older
2832than the one specified as a second argument.
2833If needs be, the @code{(:version ...)} syntax for specifying dependencies
2834could be in the future extended to specify an exclusive upper bound for compatible versions
2835as well as an inclusive lower bound.
2836@end defun
2838@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
2839@comment  node-name,  next,  previous,  up
2840@chapter Controlling where ASDF searches for systems
2845* Configurations::
2846* Truenames and other dangers::
2847* XDG base directory::
2848* Backward Compatibility::
2849* Configuration DSL::
2850* Configuration Directories::
2851* Shell-friendly syntax for configuration::
2852* Search Algorithm::
2853* Caching Results::
2854* Configuration API::
2855* Introspection::
2856* Status::
2857* Rejected ideas::
2858* TODO::
2859* Credits for the source-registry::
2860@end menu
2862@node  Configurations, Truenames and other dangers, Controlling where ASDF searches for systems, Controlling where ASDF searches for systems
2863@section Configurations
2865Configurations specify paths where to find system files.
2870The search registry may use some hardcoded wrapping registry specification.
2871This allows some implementations (notably SBCL) to specify where to find
2872some special implementation-provided systems that
2873need to precisely match the version of the implementation itself.
2876An application may explicitly initialize the source-registry configuration
2877using the configuration API
2878(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
2879in which case this takes precedence.
2880It may itself compute this configuration from the command-line,
2881from a script, from its own configuration file, etc.
2884The source registry will be configured from
2885the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
2888The source registry will be configured from
2889user configuration file
2891(which defaults to
2893if it exists.
2896The source registry will be configured from
2897user configuration directory
2899(which defaults to
2901if it exists.
2904The source registry will be configured from
2905default user configuration trees
2906@file{~/common-lisp/} (since ASDF 3.1.2 only),
2907@file{~/.sbcl/systems/} (on SBCL only),
2908@file{$XDG_DATA_HOME/common-lisp/systems/} (no recursion, link farm)
2910The @code{XDG_DATA_HOME} directory defaults to @file{~/.local/share/}.
2911On Windows, the @code{local-appdata} and @code{appdata} directories are used instead.
2914The source registry will be configured from
2915system configuration file
2917if it exists.
2920The source registry will be configured from
2921system configuration directory
2923if it exists.
2926The source registry will be configured from a default configuration.
2927This configuration may allow for implementation-specific systems
2928to be found, for systems to be found the current directory
2929(at the time that the configuration is initialized) as well as
2930@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
2931@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/},
2932where @code{XDG_DATA_DIRS} defaults to @file{/usr/local/share} and @file{/usr/share} on Unix,
2933and the @code{common-appdata} directory on Windows.
2936The source registry may include implementation-dependent directories
2937that correspond to implementation-provided extensions.
2939@end enumerate
2941Each of these configurations is specified as an s-expression
2942in a trivial domain-specific language (defined below).
2943Additionally, a more shell-friendly syntax is available
2944for the environment variable (defined yet below).
2946Each of these configurations is only used if the previous
2947configuration explicitly or implicitly specifies that it
2948includes its inherited configuration.
2950Additionally, some implementation-specific directories
2951may be automatically prepended to whatever directories are specified
2952in configuration files, no matter if the last one inherits or not.
2954@node Truenames and other dangers, XDG base directory, Configurations, Controlling where ASDF searches for systems
2955@section Truenames and other dangers
2957One great innovation of the original ASDF was its ability to leverage
2958@code{CL:TRUENAME} to locate where your source code was and where to build it,
2959allowing for symlink farms as a simple but effective configuration mechanism
2960that is easy to control programmatically.
2961ASDF 3 still supports this configuration style, and it is enabled by default;
2962however we recommend you instead use
2963our source-registry configuration mechanism described below,
2964because it is easier to setup in a portable way across users and implementations.
2966Addtionally, some people dislike truename,
2967either because it is very slow on their system, or
2968because they are using content-addressed storage where the truename of a file
2969is related to a digest of its individual contents,
2970and not to other files in the same intended project.
2971For these people, ASDF 3 allows to eschew the @code{TRUENAME} mechanism,
2972by setting the variable @var{asdf:*resolve-symlinks*} to @code{nil}.
2974PS: Yes, if you haven't read Vernor Vinge's short but great classic
2975``True Names... and Other Dangers'' then you're in for a treat.
2977@node XDG base directory, Backward Compatibility, Truenames and other dangers, Controlling where ASDF searches for systems
2978@section XDG base directory
2980Note that we purport to respect the XDG base directory specification
2981as to where configuration files are located,
2982where data files are located,
2983where output file caches are located.
2984Mentions of XDG variables refer to that document.
2988This specification allows the user to specify some environment variables
2989to customize how applications behave to his preferences.
2991On Windows platforms, when not using Cygwin,
2992instead of the XDG base directory specification,
2993we try to use folder configuration from the registry regarding
2994@code{Common AppData} and similar directories.
2995Since support for querying the Windows registry
2996is not possible to do in reasonable amounts of portable Common Lisp code,
2997ASDF 3 relies on the environment variables that Windows usually exports.
2999@node Backward Compatibility, Configuration DSL, XDG base directory, Controlling where ASDF searches for systems
3000@section Backward Compatibility
3002For backward compatibility as well as to provide a practical backdoor for hackers,
3003ASDF will first search for @file{.asd} files in the directories specified in
3005before it searches in the source registry above.
3007@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
3009By default, @code{asdf:*central-registry*} will be empty.
3011This old mechanism will therefore not affect you if you don't use it,
3012but will take precedence over the new mechanism if you do use it.
3014@node Configuration DSL, Configuration Directories, Backward Compatibility, Controlling where ASDF searches for systems
3015@section Configuration DSL
3016@cindex :inherit-configuration source config directive
3017@cindex inherit-configuration source config directive
3018@cindex :ignore-invalid-entries source config directive
3019@cindex ignore-invalid-entries source config directive
3020@cindex :directory source config directive
3021@cindex directory source config directive
3022@cindex :tree source config directive
3023@cindex tree source config directive
3024@cindex :exclude source config directive
3025@cindex exclude source config directive
3026@cindex :also-exclude source config directive
3027@cindex also-exclude source config directive
3028@cindex :include source config directive
3029@cindex include source config directive
3030@cindex :default-registry source config directive
3031@cindex default-registry source config directive
3033Here is the grammar of the s-expression (SEXP) DSL for source-registry
3036@c FIXME: This is too wide for happy compilation into pdf.
3039;; A configuration is a single SEXP starting with the keyword
3040;; :source-registry followed by a list of directives.
3041CONFIGURATION := (:source-registry DIRECTIVE ...)
3043;; A directive is one of the following:
3046    ;; Your configuration expression MUST contain
3047    ;; exactly one of the following:
3048    :inherit-configuration |
3049    ;; splices inherited configuration (often specified last) or
3050    :ignore-inherited-configuration |
3051    ;; drop inherited configuration (specified anywhere)
3053    ;; forward compatibility directive (since ASDF 2.011.4), useful when
3054    ;; you want to use new configuration features but have to bootstrap
3055    ;; the newer required ASDF from an older release that doesn't
3056    ;; support said features:
3057    :ignore-invalid-entries |
3059    ;; add a single directory to be scanned (no recursion)
3062    ;; add a directory hierarchy, recursing but
3063    ;; excluding specified patterns
3066    ;; override the defaults for exclusion patterns
3067    (:exclude EXCLUSION-PATTERN ...) |
3068    ;; augment the defaults for exclusion patterns
3069    (:also-exclude EXCLUSION-PATTERN ...) |
3070    ;; Note that the scope of a an exclude pattern specification is
3071    ;; the rest of the current configuration expression or file.
3073    ;; splice the parsed contents of another config file
3076    ;; This directive specifies that some default must be spliced.
3077    :default-registry
3080    := PATHNAME-DESIGNATOR ; interpreted as a file
3082    := PATHNAME-DESIGNATOR ; interpreted as a directory
3085    NIL | ;; Special: skip this entry.
3088EXCLUSION-PATTERN := a string without wildcards, that will be matched
3089    exactly against the name of a any subdirectory in the directory
3090    component of a path. e.g. @code{"_darcs"} will match
3091    @file{#p"/foo/bar/_darcs/src/bar.asd"}
3092@end example
3094Pathnames are designated using another DSL,
3095shared with the output-translations configuration DSL below.
3096The DSL is resolved by the function @code{asdf::resolve-location},
3097to be documented and exported at some point in the future.
3102    STRING |
3103    ;; namestring (better be absolute or bust, directory assumed where
3104    ;; applicable).  In output-translations, directory is assumed and
3105    ;; **/*.*.* added if it's last.  On MCL, a MacOSX-style POSIX
3106    ;; namestring (for MacOS9 style, use #p"..."); Note that none of the
3107    ;; above applies to strings used in *central-registry*, which
3108    ;; doesn't use this DSL: they are processed as normal namestrings.
3109    ;; however, you can compute what you put in the *central-registry*
3110    ;; based on the results of say
3111    ;; (asdf::resolve-location "/Users/fare/cl/cl-foo/")
3112    PATHNAME |
3113    ;; pathname (better be an absolute path, or bust)
3114    ;; In output-translations, unless followed by relative components,
3115    ;; it better have appropriate wildcards, as in **/*.*.*
3116    :HOME | ; designates the user-homedir-pathname ~/
3117    :USER-CACHE | ; designates the default location for the user cache
3118    :HERE |
3119    ;; designates the location of the configuration file
3120    ;; (or *default-pathname-defaults*, if invoked interactively)
3121    :ROOT
3122    ;; magic, for output-translations source only: paths that are relative
3123    ;; to the root of the source host and device
3125They keyword :SYSTEM-CACHE is not accepted in ASDF 3.1 and beyond: it
3126was a security hazard.
3130    STRING |
3131      ;; relative directory pathname as interpreted by
3132      ;; parse-unix-namestring.
3133      ;; In output translations, if last component, **/*.*.* is added
3134    PATHNAME | ; pathname; unless last component, directory is assumed.
3136       ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
3138       ;; a directory based on lisp-implementation-type only, e.g. sbcl
3140       ;; a relativized version of the default directory
3141    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
3142    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
3143    :*.*.* | ;; any file (since ASDF 2.011.4)
3145The keywords :UID and :USERNAME are no longer supported.
3146@end example
3148For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
3149which is the default place ASDF looks for this configuration, once contained:
3152  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
3153  :inherit-configuration)
3154@end example
3156@node Configuration Directories, Shell-friendly syntax for configuration, Configuration DSL, Controlling where ASDF searches for systems
3157@section Configuration Directories
3159Configuration directories consist in files each containing
3160a list of directives without any enclosing @code{(:source-registry ...)} form.
3161The files will be sorted by namestring as if by @code{string<} and
3162the lists of directives of these files with be concatenated in order.
3163An implicit @code{:inherit-configuration} will be included
3164at the @emph{end} of the list.
3166System-wide or per-user Common Lisp software distributions
3167such as Debian packages or some future version of @code{clbuild}
3168may then include files such as
3169@file{/etc/common-lisp/source-registry.conf.d/10-foo.conf} or
3171to easily and modularly register configuration information
3172about software being distributed.
3174The convention is that, for sorting purposes,
3175the names of files in such a directory begin with two digits
3176that determine the order in which these entries will be read.
3177Also, the type of these files must be @file{.conf},
3178which not only simplifies the implementation by allowing
3179for more portable techniques in finding those files,
3180but also makes it trivial to disable a file, by renaming it to a different file type.
3182Directories may be included by specifying a directory pathname
3183or namestring in an @code{:include} directive, e.g.:
3186  (:include "/foo/bar/")
3187@end example
3189Hence, to achieve the same effect as
3190my example @file{~/.config/common-lisp/source-registry.conf} above,
3191I could simply create a file
3193alone in its directory with the following contents:
3195(:tree "/home/fare/cl/")
3196@end example
3199* The here directive::
3200@end menu
3202@node The here directive,  , Configuration Directories, Configuration Directories
3203@subsection The :here directive
3205The @code{:here} directive is an absolute pathname designator that
3206refers to the directory containing the configuration file currently
3207being processed.
3209The @code{:here} directive is intended to simplify the delivery of
3210complex CL systems, and for easy configuration of projects shared through
3211revision control systems, in accordance with our design principle that
3212each participant should be able to provide all and only the information
3213available to him or her.
3215Consider a person X who has set up the source code repository for a
3216complex project with a master directory @file{dir/}.  Ordinarily, one
3217might simply have the user add a directive that would look something
3218like this:
3220   (:tree "path/to/dir")
3221@end example
3222But what if X knows that there are very large subtrees
3223under dir that are filled with, e.g., Java source code, image files for
3224icons, etc.?  All of the asdf system definitions are contained in the
3225subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
3226these are the only directories that should be searched.
3228In this case, X can put into @file{dir/} a file @file{asdf.conf} that
3229contains the following:
3232   (:tree (:here "src/lisp/"))
3233   (:tree (:here "extlib/lisp"))
3234   (:directory (:here "outlier/")))
3235@end example
3237Then when someone else (call her Y) checks out a copy of this
3238repository, she need only add
3240(:include "/path/to/my/checkout/directory/asdf.conf")
3241@end example
3242to one of her previously-existing asdf source location configuration
3243files, or invoke @code{initialize-source-registry} with a configuration
3244form containing that s-expression.  ASDF will find the .conf file that X
3245has provided, and then set up source locations within the working
3246directory according to X's (relative) instructions.
3248@node Shell-friendly syntax for configuration, Search Algorithm, Configuration Directories, Controlling where ASDF searches for systems
3249@section Shell-friendly syntax for configuration
3251When considering environment variable @code{CL_SOURCE_REGISTRY}
3252ASDF will skip to next configuration if it's an empty string.
3253It will @code{READ} the string as a SEXP in the DSL
3254if it begins with a paren @code{(}
3255and it will be interpreted much like @code{TEXINPUTS}
3256list of paths, where
3258  * paths are separated
3259   by a @code{:} (colon) on Unix platforms (including cygwin),
3260   by a @code{;} (semicolon) on other platforms (mainly, Windows).
3262  * each entry is a directory to add to the search path.
3264  * if the entry ends with a double slash @code{//}
3265    then it instead indicates a tree in the subdirectories
3266    of which to recurse.
3268  * if the entry is the empty string (which may only appear once),
3269    then it indicates that the inherited configuration should be
3270    spliced there.
3272@node Search Algorithm, Caching Results, Shell-friendly syntax for configuration, Controlling where ASDF searches for systems
3273@section Search Algorithm
3274@vindex *default-source-registry-exclusions*
3276In case that isn't clear, the semantics of the configuration is that
3277when searching for a system of a given name,
3278directives are processed in order.
3280When looking in a directory, if the system is found, the search succeeds,
3281otherwise it continues.
3283When looking in a tree, if one system is found, the search succeeds.
3284If multiple systems are found, the consequences are unspecified:
3285the search may succeed with any of the found systems,
3286or an error may be raised.
3287ASDF currently returns the first system found,
3288XCVB currently raised an error.
3289If none is found, the search continues.
3291Exclude statements specify patterns of subdirectories
3292the systems from which to ignore.
3293Typically you don't want to use copies of files kept by such
3294version control systems as Darcs.
3295Exclude statements are not propagated to further included or inherited
3296configuration files or expressions;
3297instead the defaults are reset around every configuration statement
3298to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
3300Include statements cause the search to recurse with the path specifications
3301from the file specified.
3303An inherit-configuration statement cause the search to recurse with the path
3304specifications from the next configuration
3305(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
3308@node Caching Results, Configuration API, Search Algorithm, Controlling where ASDF searches for systems
3309@section Caching Results
3311The implementation is allowed to either eagerly compute the information
3312from the configurations and file system, or to lazily re-compute it
3313every time, or to cache any part of it as it goes.
3314To explicitly flush any information cached by the system, use the API below.
3316@node Configuration API, Introspection, Caching Results, Controlling where ASDF searches for systems
3317@section Configuration API
3319The specified functions are exported from your build system's package.
3320Thus for ASDF the corresponding functions are in package ASDF,
3321and for XCVB the corresponding functions are in package XCVB.
3323@defun initialize-source-registry @Aoptional{} PARAMETER
3324   will read the configuration and initialize all internal variables.
3325   You may extend or override configuration
3326   from the environment and configuration files
3327   with the given @var{PARAMETER}, which can be
3328   @code{nil} (no configuration override),
3329   or a SEXP (in the SEXP DSL),
3330   a string (as in the string DSL),
3331   a pathname (of a file or directory with configuration),
3332   or a symbol (fbound to function that when called returns one of the above).
3333@end defun
3335@defun clear-source-registry
3336   undoes any source registry configuration
3337   and clears any cache for the search algorithm.
3338   You might want to call this function
3339   (or better, @code{clear-configuration})
3340   before you dump an image that would be resumed
3341   with a different configuration,
3342   and return an empty configuration.
3343   Note that this does not include clearing information about
3344   systems defined in the current image, only about
3345   where to look for systems not yet defined.
3346@end defun
3348@defun ensure-source-registry @Aoptional{} PARAMETER
3349   checks whether a source registry has been initialized.
3350   If not, initialize it with the given @var{PARAMETER}.
3351@end defun
3353Every time you use ASDF's @code{find-system}, or
3354anything that uses it (such as @code{operate}, @code{load-system}, etc.),
3355@code{ensure-source-registry} is called with parameter @code{nil},
3356which the first time around causes your configuration to be read.
3357If you change a configuration file,
3358you need to explicitly @code{initialize-source-registry} again,
3359or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
3360which will cause the initialization to happen next time around.
3362@node Introspection, Status, Configuration API, Controlling where ASDF searches for systems
3363@section Introspection
3366* *source-registry-parameter* variable::
3367* Information about system dependencies::
3368@end menu
3370@node *source-registry-parameter* variable, Information about system dependencies, Introspection, Introspection
3371@subsection *source-registry-parameter* variable
3372@vindex *source-registry-parameter*
3374We have made available the variable @code{*source-registry-parameter*}
3375that can be used by code that wishes to introspect about the (past)
3376configuration of ASDF's source registry.  @strong{This variable should
3377never be set!}  It will be set as a side-effect of calling
3378@code{initialize-source-registry}; user code should treat it as
3381@node Information about system dependencies,  , *source-registry-parameter* variable, Introspection
3382@subsection Information about system dependencies
3384ASDF makes available three functions to read system interdependencies.
3385These are intended to aid programmers who wish to perform dependency
3388@defun system-defsystem-depends-on system
3389@end defun
3391@defun system-depends-on system
3392@end defun
3394@defun system-weakly-depends-on system
3395Returns a list of names of systems that are weakly depended on by
3396@var{system}.  Weakly depended on systems are optionally loaded only if
3397ASDF can find them; failure to find such systems does @emph{not} cause an
3398error in loading.
3400Note that the return value for @code{system-weakly-depends-on} is simpler
3401than the return values of the other two system dependency introspection
3403@end defun
3405@node Status, Rejected ideas, Introspection, Controlling where ASDF searches for systems
3406@section Status
3408This mechanism is vastly successful, and we have declared
3409that @code{asdf:*central-registry*} is not recommended anymore,
3410though we will continue to support it.
3411All hooks into implementation-specific search mechanisms
3412have been integrated in the @code{wrapping-source-registry}
3413that everyone uses implicitly.
3415@node Rejected ideas, TODO, Status, Controlling where ASDF searches for systems
3416@section Rejected ideas
3418Alternatives I (FRR) considered and rejected while developing ASDF 2 included:
3421@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
3422   and somehow the configuration parser expands the new configuration
3423   language into a expanded series of directories of subdirectories to
3424   lookup, pre-recursing through specified hierarchies. This is kludgy,
3425   and leaves little space of future cleanups and extensions.
3427@item Keep @code{asdf:*central-registry*} as the master but extend its semantics
3428   in completely new ways, so that new kinds of entries may be implemented
3429   as a recursive search, etc. This seems somewhat backwards.
3431@item Completely remove @code{asdf:*central-registry*}
3432   and break backwards compatibility.
3433   Hopefully this will happen in a few years after everyone migrate to
3434   a better ASDF and/or to XCVB, but it would be very bad to do it now.
3436@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
3437   when you dereference it or setf it. Only the new variable with new
3438   semantics is handled by the new search procedure.
3439   Complex and still introduces subtle semantic issues.
3440@end enumerate
3443I've been suggested the below features, but have rejected them,
3444for the sake of keeping ASDF no more complex than strictly necessary.
3448  More syntactic sugar: synonyms for the configuration directives, such as
3449  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
3450  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
3453   The possibility to register individual files instead of directories.
3456  Integrate Xach Beane's tilde expander into the parser,
3457  or something similar that is shell-friendly or shell-compatible.
3458  I'd rather keep ASDF minimal. But maybe this precisely keeps it
3459  minimal by removing the need for evaluated entries that ASDF has?
3460  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
3461  Hopefully, these are already superseded by the @code{:default-registry}
3464  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
3465  down a filesystem tree in the environment variable.
3466  It isn't that Lisp friendly either.
3467@end itemize
3469@node TODO, Credits for the source-registry, Rejected ideas, Controlling where ASDF searches for systems
3470@section TODO
3473@item Add examples
3474@end itemize
3476@node Credits for the source-registry,  , TODO, Controlling where ASDF searches for systems
3477@section Credits for the source-registry
3479Thanks a lot to Stelian Ionescu for the initial idea.
3481Thanks to Rommel Martinez for the initial implementation attempt.
3483All bad design ideas and implementation bugs are mine, not theirs.
3484But so are good design ideas and elegant implementation tricks.
3486 --- Francois-Rene Rideau @email{}, Mon, 22 Feb 2010 00:07:33 -0500
3490@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
3491@comment  node-name,  next,  previous,  up
3492@chapter Controlling where ASDF saves compiled files
3493@cindex asdf-output-translations
3496Each Common Lisp implementation has its own format
3497for compiled files or fasls.@footnote{``FASL'' is short for ``FASt Loading.''}
3498If you use multiple implementations
3499(or multiple versions of the same implementation),
3500you'll soon find your source directories
3501littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so
3503Worse yet, multiple implementations use the same file extension and
3504some implementations maintain the same file extension
3505while changing formats from version to version (or platform to
3507This can lead to many errors and much confusion
3508as you switch from one implementation to the next.
3510Since ASDF 2, ASDF includes the @code{asdf-output-translations} facility
3511to mitigate the problem.
3514* Output Configurations::
3515* Output Backward Compatibility::
3516* Output Configuration DSL::
3517* Output Configuration Directories::
3518* Output Shell-friendly syntax for configuration::
3519* Semantics of Output Translations::
3520* Output Caching Results::
3521* Output location API::
3522* Credits for output translations::
3523@end menu
3525@node Output Configurations, Output Backward Compatibility, Controlling where ASDF saves compiled files, Controlling where ASDF saves compiled files
3526@section Configurations
3528@c FIXME: Explain how configurations work: can't expect reader will have
3529@c looked at previous chapter. Probably cut and paste will do.
3532Configurations specify mappings from input locations to output locations.
3533Once again we rely on the XDG base directory specification for configuration.
3534@xref{Controlling where ASDF searches for systems,,XDG base directory}.
3539Some hardcoded wrapping output translations configuration may be used.
3540This allows special output translations (or usually, invariant directories)
3541to be specified corresponding to the similar special entries in the source registry.
3544An application may explicitly initialize the output-translations
3545configuration using the Configuration API
3546in which case this takes precedence.
3547(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
3548It may itself compute this configuration from the command-line,
3549from a script, from its own configuration file, etc.
3552The source registry will be configured from
3553the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
3556The source registry will be configured from
3557user configuration file
3559(which defaults to
3561if it exists.
3564The source registry will be configured from
3565user configuration directory
3567(which defaults to
3569if it exists.
3572The source registry will be configured from
3573system configuration file
3575if it exists.
3578The source registry will be configured from
3579system configuration directory
3581if it exists.
3583@end enumerate
3585Each of these configurations is specified as a SEXP
3586in a trivial domain-specific language (@pxref{Configuration DSL}).
3587Additionally, a more shell-friendly syntax is available
3588for the environment variable (@pxref{Shell-friendly syntax for configuration}).
3590When processing an entry in the above list of configuration methods,
3591ASDF will stop unless that entry
3592explicitly or implicitly specifies that it
3593includes its inherited configuration.
3595Note that by default, a per-user cache is used for output files.
3596This allows the seamless use of shared installations of software
3597between several users, and takes files out of the way of the developers
3598when they browse source code,
3599at the expense of taking a small toll when developers have to clean up
3600output files and find they need to get familiar with output-translations
3601first.@footnote{A @code{CLEAN-OP} would be a partial solution to this problem.}
3604@node Output Backward Compatibility, Output Configuration DSL, Output Configurations, Controlling where ASDF saves compiled files
3605@section Backward Compatibility
3606@cindex ASDF-BINARY-LOCATIONS compatibility
3607@c FIXME: Demote this section -- the typical reader doesn't care about
3608@c backwards compatibility.
3611We purposely do @emph{not} provide backward compatibility with earlier versions of
3612@code{ASDF-Binary-Locations} (8 Sept 2009),
3613@code{common-lisp-controller} (7.0) or
3614@code{cl-launch} (2.35),
3615each of which had similar general capabilities.
3616The APIs of these programs were not designed
3617for easy user configuration
3618through configuration files.
3619Recent versions of @code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000)
3620use the new @code{asdf-output-translations} API as defined below.
3621@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
3623This incompatibility shouldn't inconvenience many people.
3624Indeed, few people use and customize these packages;
3625these few people are experts who can trivially adapt to the new configuration.
3626Most people are not experts, could not properly configure these features
3627(except inasmuch as the default configuration of
3628@code{common-lisp-controller} and/or @code{cl-launch}
3629might have been doing the right thing for some users),
3630and yet will experience software that ``just works'',
3631as configured by the system distributor, or by default.
3633Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
3634we provide a limited emulation mode:
3636@defun enable-asdf-binary-locations-compatibility @Akey{} centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
3637This function will initialize the new @code{asdf-output-translations} facility in a way
3638that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
3639Where you would previously set global variables
3643@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
3644you will now have to pass the same values as keyword arguments to this function.
3645Note however that as an extension the @code{:source-to-target-mappings} keyword argument
3646will accept any valid pathname designator for @code{asdf-output-translations}
3647instead of just strings and pathnames.
3648@end defun
3650If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
3651(the one available as an extension to load of top of ASDF,
3652not the one built into a few old versions of ASDF),
3653but first you must disable @code{asdf-output-translations}
3654with @code{(asdf:disable-output-translations)},
3655or you might experience ``interesting'' issues.
3657Also, note that output translation is enabled by default.
3658To disable it, use @code{(asdf:disable-output-translations)}.
3660@node Output Configuration DSL, Output Configuration Directories, Output Backward Compatibility, Controlling where ASDF saves compiled files
3661@section Configuration DSL
3663Here is the grammar of the SEXP DSL
3664for @code{asdf-output-translations} configuration:
3667;; A configuration is single SEXP starting with keyword :source-registry
3668;; followed by a list of directives.
3669CONFIGURATION := (:output-translations DIRECTIVE ...)
3671;; A directive is one of the following:
3674    ;; Your configuration expression MUST contain
3675    ;; exactly one of either of these:
3676    :inherit-configuration |
3677      ;; splices inherited configuration (often specified last)
3678    :ignore-inherited-configuration |
3679      ;; drop inherited configuration (specified anywhere)
3681    ;; forward compatibility directive (since ASDF 2.011.4), useful when
3682    ;; you want to use new configuration features but have to bootstrap a
3683    ;; the newer required ASDF from an older release that doesn't have
3684    ;; said features:
3685    :ignore-invalid-entries |
3687    ;; include a configuration file or directory
3688    (:include PATHNAME-DESIGNATOR) |
3690    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/
3691    ;; or something.
3692    :enable-user-cache |
3693    ;; Disable global cache. Map / to /
3694    :disable-cache |
3696    ;; add a single directory to be scanned (no recursion)
3699    ;; use a function to return the translation of a directory designator
3703    NIL | ; As source: skip this entry. As destination: same as source
3704    T | ; as source matches anything, as destination
3705        ; maps pathname to itself.
3706    ABSOLUTE-COMPONENT-DESIGNATOR ; same as in the source-registry language
3709    SYMBOL | ;; symbol naming a function that takes two arguments:
3710             ;; the pathname to be translated and the matching
3711             ;; DIRECTORY-DESIGNATOR
3712    LAMBDA   ;; A form which evalutates to a function taking two arguments:
3713             ;; the pathname to be translated and the matching
3714             ;; DIRECTORY-DESIGNATOR
3716@end verbatim
3718Relative components better be either relative
3719or subdirectories of the path before them, or bust.
3721@c FIXME: the following assumes that the reader is familiar with the use
3722@c of this pattern in logical pathnames, which may not be a reasonable
3723@c assumption.  Expand.
3724The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
3725You can specify more fine-grained patterns
3726by using a pathname object as the last component
3727e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
3729You may use @code{#+features} to customize the configuration file.
3731The second designator of a mapping may be @code{nil}, indicating that files are not mapped
3732to anything but themselves (same as if the second designator was the same as the first).
3734When the first designator is @code{t},
3735the mapping always matches.
3736When the first designator starts with @code{:root},
3737the mapping matches any host and device.
3738In either of these cases, if the second designator
3739isn't @code{t} and doesn't start with @code{:root},
3740then strings indicating the host and pathname are somehow copied
3741in the beginning of the directory component of the source pathname
3742before it is translated.
3744When the second designator is @code{t}, the mapping is the identity.
3745When the second designator starts with @code{:root},
3746the mapping preserves the host and device of the original pathname.
3747Notably, this allows you to map files
3748to a subdirectory of the whichever directory the file is in.
3749Though the syntax is not quite as easy to use as we'd like,
3750you can have an (source destination) mapping entry such as follows
3751in your configuration file,
3752or you may use @code{enable-asdf-binary-locations-compatibility}
3753with @code{:centralize-lisp-binaries nil}
3754which will do the same thing internally for you:
3756#.(let ((wild-subdir
3757          (make-pathname :directory '(:relative :wild-inferiors)))
3758        (wild-file
3759          (make-pathname :name :wild :version :wild :type :wild)))
3760   `((:root ,wild-subdir ,wild-file)
3761     (:root ,wild-subdir :implementation ,wild-file)))
3762@end lisp
3763Starting with ASDF 2.011.4, you can use the simpler:
3764  @code{`(:root (:root :**/ :implementation :*.*.*))}
3768@code{:include} statements cause the search to recurse with the path specifications
3769from the file specified.
3771If the @code{translate-pathname} mechanism cannot achieve a desired
3772translation, the user may provide a function which provides the
3773required algorithim.  Such a translation function is specified by
3774supplying a list as the second @code{directory-designator}
3775the first element of which is the keyword @code{:function},
3776and the second element of which is
3777either a symbol which designates a function or a lambda expression.
3778The function designated by the second argument must take two arguments,
3779the first being the pathname of the source file,
3780the second being the wildcard that was matched.
3781When invoked, the function should return the translated pathname.
3783An @code{:inherit-configuration} statement causes the search to recurse with the path
3784specifications from the next configuration in the bulleted list.
3785@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
3787@vindex @code{asdf::*user-cache*}
3790@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
3792@code{:disable-cache} is the same as @code{(t t)}.
3794@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
3795which by default is the same as using
3796@code{(:home ".cache" "common-lisp" :implementation)}.
3797@end itemize
3800@node Output Configuration Directories, Output Shell-friendly syntax for configuration, Output Configuration DSL, Controlling where ASDF saves compiled files
3801@section Configuration Directories
3803Configuration directories consist of files, each of which contains
3804a list of directives without any enclosing
3805@code{(:output-translations ...)} form.
3806The files will be sorted by namestring as if by @code{string<} and
3807the lists of directives of these files with be concatenated in order.
3808An implicit @code{:inherit-configuration} will be included
3809at the @emph{end} of the list.
3811System-wide or per-user Common Lisp software distributions
3812such as Debian packages or some future version of @code{clbuild}
3813may then include files such as
3814@file{/etc/common-lisp/asdf-output-translations.conf.d/10-foo.conf} or
3816to easily and modularly register configuration information
3817about software being distributed.
3819The convention is that, for sorting purposes,
3820the names of files in such a directory begin with two digits
3821that determine the order in which these entries will be read.
3822Also, the type of these files must be @file{.conf},
3823which not only simplifies the implementation by allowing
3824for more portable techniques in finding those files,
3825but also makes it trivial to disable a file, by renaming it to a different file type.
3827Directories may be included by specifying a directory pathname
3828or namestring in an @code{:include} directive, e.g.:
3831  (:include "/foo/bar/")
3832@end verbatim
3834@node Output Shell-friendly syntax for configuration, Semantics of Output Translations, Output Configuration Directories, Controlling where ASDF saves compiled files
3835@section Shell-friendly syntax for configuration
3837When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
3838ASDF will skip to the next configuration if it's an empty string.
3839It will @code{READ} the string as an SEXP in the DSL
3840if it begins with a paren @code{(}
3841and it will be interpreted as a list of directories.
3842Directories should come by pairs, indicating a mapping directive.
3843Entries are separated
3844by a @code{:} (colon) on Unix platforms (including cygwin),
3845by a @code{;} (semicolon) on other platforms (mainly, Windows).
3847The magic empty entry,
3848if it comes in what would otherwise be the first entry in a pair,
3849indicates the splicing of inherited configuration.
3850If it comes as the second entry in a pair,
3851it indicates that the directory specified first is to be left untranslated
3852(which has the same effect as if the directory had been repeated).
3853Thus @code{"/foo:/bar::/baz:"} means that
3854things under directory @file{/foo/}
3855are translated to be under @file{/bar/},
3856then include the inherited configuration,
3857then specify that things under directory @file{/baz/} are not translated.
3859@node Semantics of Output Translations, Output Caching Results, Output Shell-friendly syntax for configuration, Controlling where ASDF saves compiled files
3860@section Semantics of Output Translations
3862From the specified configuration,
3863a list of mappings is extracted in a straightforward way:
3864mappings are collected in order, recursing through
3865included or inherited configuration as specified.
3866To this list is prepended some implementation-specific mappings,
3867and is appended a global default.
3869The list is then compiled to a mapping table as follows:
3870for each entry, in order, resolve the first designated directory
3871into an actual directory pathname for source locations.
3872If no mapping was specified yet for that location,
3873resolve the second designated directory to an output location directory
3874add a mapping to the table mapping the source location to the output location,
3875and add another mapping from the output location to itself
3876(unless a mapping already exists for the output location).
3878Based on the table, a mapping function is defined,
3879mapping source pathnames to output pathnames:
3880given a source pathname, locate the longest matching prefix
3881in the source column of the mapping table.
3882Replace that prefix by the corresponding output column
3883in the same row of the table, and return the result.
3884If no match is found, return the source pathname.
3885(A global default mapping the filesystem root to itself
3886may ensure that there will always be a match,
3887with same fall-through semantics).
3889@node Output Caching Results, Output location API, Semantics of Output Translations, Controlling where ASDF saves compiled files
3890@section Caching Results
3892The implementation is allowed to either eagerly compute the information
3893from the configurations and file system, or to lazily re-compute it
3894every time, or to cache any part of it as it goes.
3895To explicitly flush any information cached by the system, use the API below.
3898@node Output location API, Credits for output translations, Output Caching Results, Controlling where ASDF saves compiled files
3899@section Output location API
3901The specified functions are exported from package ASDF.
3903@defun initialize-output-translations @Aoptional{} PARAMETER
3904   will read the configuration and initialize all internal variables.
3905   You may extend or override configuration
3906   from the environment and configuration files
3907   with the given @var{PARAMETER}, which can be
3908   @code{nil} (no configuration override),
3909   or a SEXP (in the SEXP DSL),
3910   a string (as in the string DSL),
3911   a pathname (of a file or directory with configuration),
3912   or a symbol (fbound to function that when called returns one of the above).
3913@end defun
3915@defun disable-output-translations
3916   will initialize output translations in a way
3917   that maps every pathname to itself,
3918   effectively disabling the output translation facility.
3919@end defun
3921@defun clear-output-translations
3922   undoes any output translation configuration
3923   and clears any cache for the mapping algorithm.
3924   You might want to call this function
3925   (or better, @code{clear-configuration})
3926   before you dump an image that would be resumed
3927   with a different configuration,
3928   and return an empty configuration.
3929   Note that this does not include clearing information about
3930   systems defined in the current image, only about
3931   where to look for systems not yet defined.
3932@end defun
3934@defun ensure-output-translations @Aoptional{} PARAMETER
3935   checks whether output translations have been initialized.
3936   If not, initialize them with the given @var{PARAMETER}.
3937   This function will be called before any attempt to operate on a system.
3938@end defun
3940@defun apply-output-translations PATHNAME
3941   Applies the configured output location translations to @var{PATHNAME}
3942   (calls @code{ensure-output-translations} for the translations).
3943@end defun
3945Every time you use ASDF's @code{output-files}, or
3946anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
3947@code{ensure-output-translations} is called with parameter @code{nil},
3948which the first time around causes your configuration to be read.
3949If you change a configuration file,
3950you need to explicitly @code{initialize-output-translations} again,
3951or maybe @code{clear-output-translations} (or @code{clear-configuration}),
3952which will cause the initialization to happen next time around.
3955@node Credits for output translations,  , Output location API, Controlling where ASDF saves compiled files
3956@section Credits for output translations
3958Thanks a lot to Peter van Eynde for @code{Common Lisp Controller}
3959and to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations}.
3961All bad design ideas and implementation bugs are to mine, not theirs.
3962But so are good design ideas and elegant implementation tricks.
3964 --- Francois-Rene Rideau @email{}
3966@c @section Default locations
3967@c @findex output-files-for-system-and-operation
3969@c The default binary location for each Lisp implementation
3970@c is a subdirectory of each source directory.
3971@c To account for different Lisps, Operating Systems, Implementation versions,
3972@c and so on, ASDF borrows code from SLIME
3973@c to create reasonable custom directory names.
3974@c Here are some examples:
3976@c @itemize
3977@c @item
3978@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
3980@c @item
3981@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
3983@c @item
3984@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
3985@c @end itemize
3987@c By default, all output file pathnames will be relocated
3988@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
3990@c See the document @file{README.asdf-output-translations}
3991@c for a full specification on how to configure @code{asdf-output-translations}.
3993@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
3994@comment  node-name,  next,  previous,  up
3995@chapter Error handling
3999@section ASDF errors
4001If ASDF detects an incorrect system definition, it will signal a generalised instance of
4004Operations may go wrong (for example when source files contain errors).
4005These are signalled using generalised instances of
4008@section Compilation error and warning handling
4009@vindex *compile-file-warnings-behaviour*
4010@vindex *compile-file-errors-behavior*
4012ASDF checks for warnings and errors when a file is compiled.
4013The variables @var{*compile-file-warnings-behaviour*} and
4015control the handling of any such events.
4016The valid values for these variables are
4017@code{:error}, @code{:warn}, and @code{:ignore}.
4019@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
4020@comment  node-name,  next,  previous,  up
4021@chapter Miscellaneous additional functionality
4023ASDF includes several additional features that are generally
4024useful for system definition and development.
4027* Controlling file compilation::
4028* Controlling source file character encoding::
4029* Some Utility Functions::
4030@end menu
4032@node Controlling file compilation, Controlling source file character encoding, Miscellaneous additional functionality, Miscellaneous additional functionality
4033@section Controlling file compilation
4034@cindex :around-compile
4035@cindex around-compile keyword
4036@cindex compile-check keyword
4037@cindex :compile-check
4038@findex compile-file*
4040@c FIXME: Needs rewrite.  Start with motivation -- why are we doing
4041@c this?  (there is some, but it's buried).  Also, all of a sudden in
4042@c the middle of the discussion we start talking about a "hook," which
4043@c is confusing.
4045When declaring a component (system, module, file),
4046you can specify a keyword argument @code{:around-compile function}.
4047If left unspecified (and therefore unbound),
4048the value will be inherited from the parent component if any,
4049or with a default of @code{nil}
4050if no value is specified in any transitive parent.
4052The argument must be either @code{nil}, an fbound symbol,
4053a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
4054a function object (e.g. using @code{#.#'} but that's discouraged
4055because it prevents the introspection done by e.g. asdf-dependency-grovel),
4056or a string that when @code{read} yields a symbol or a lambda-expression.
4057@code{nil} means the normal compile-file function will be called.
4058A non-nil value designates a function of one argument
4059that will be called with a function that will
4060invoke @code{compile-file*} with various arguments;
4061the around-compile hook may supply additional keyword arguments
4062to pass to that call to @code{compile-file*}.
4064One notable argument that is heeded by @code{compile-file*} is
4066a function called when the compilation was otherwise a success,
4067with the same arguments as @code{compile-file};
4068the function shall return true if the compilation
4069and its resulting compiled file respected all system-specific invariants,
4070and false (@code{nil}) if it broke any of those invariants;
4071it may issue warnings or errors before it returns @code{nil}.
4072(NB: The ability to pass such extra flags
4073is only available starting with ASDF 2.22.3.)
4074This feature is notably exercised by asdf-finalizers.
4076By using a string, you may reference
4077a function, symbol and/or package
4078that will only be created later during the build, but
4079isn't yet present at the time the defsystem form is evaluated.
4080However, if your entire system is using such a hook, you may have to
4081explicitly override the hook with @code{nil} for all the modules and files
4082that are compiled before the hook is defined.
4084Using this hook, you may achieve such effects as:
4085locally renaming packages,
4086binding @var{*readtables*} and other syntax-controlling variables,
4087handling warnings and other conditions,
4088proclaiming consistent optimization settings,
4089saving code coverage information,
4090maintaining meta-data about compilation timings,
4091setting gensym counters and PRNG seeds and other sources of non-determinism,
4092overriding the source-location and/or timestamping systems,
4093checking that some compile-time side-effects were properly balanced,
4096Note that there is no around-load hook. This is on purpose.
4097Some implementations such as ECL, GCL or MKCL link object files,
4098which allows for no such hook.
4099Other implementations allow for concatenating FASL files,
4100which doesn't allow for such a hook either.
4101We aim to discourage something that's not portable,
4102and has some dubious impact on performance and semantics
4103even when it is possible.
4104Things you might want to do with an around-load hook
4105are better done around-compile,
4106though it may at times require some creativity
4107(see e.g. the @code{package-renaming} system).
4110@node Controlling source file character encoding, Some Utility Functions, Controlling file compilation, Miscellaneous additional functionality
4111@section Controlling source file character encoding
4113Starting with ASDF 2.21, components accept a @code{:encoding} option
4114so authors may specify which character encoding should be used
4115to read and evaluate their source code.
4116When left unspecified, the encoding is inherited
4117from the parent module or system;
4118if no encoding is specified at any point,
4119or if @code{nil} is explicitly specified,
4120an extensible protocol described below is followed,
4121that ultimately defaults to @code{:utf-8} since ASDF 3.
4123The protocol to determine the encoding is
4124to call the function @code{detect-encoding},
4125which itself, if provided a valid file,
4126calls the function specified by @var{*encoding-detection-hook*},
4127or else defaults to the @var{*default-encoding*}.
4128The @var{*encoding-detection-hook*} is by default bound
4129to function @code{always-default-encoding},
4130that always returns the contents of @var{*default-encoding*}.
4131@var{*default-encoding*} is bound to @code{:utf-8} by default
4132(before ASDF 3, the default was @code{:default}).
4134Whichever encoding is returned must be a portable keyword,
4135that will be translated to an implementation-specific external-format designator
4136by function @code{encoding-external-format},
4137which itself simply calls the function specified @var{*encoding-external-format-hook*};
4138that function by default is @code{default-encoding-external-format},
4139that only recognizes @code{:utf-8} and @code{:default},
4140and translates the former to the implementation-dependent @var{*utf-8-external-format*},
4141and the latter to itself (that itself is portable but has an implementation-dependent meaning).
4143In other words, there now are plenty of extension hooks, but
4144by default ASDF enforces the previous @emph{de facto} standard behavior
4145of using @code{:utf-8}, independently from
4146whatever configuration the user may be using.
4147Thus, system authors can now rely on @code{:utf-8}
4148being used while compiling their files,
4149even if the user is currently using @code{:koi8-r} or @code{:euc-jp}
4150as their interactive encoding.
4151(Before ASDF 3, there was no such guarantee, @code{:default} was used,
4152and only plain ASCII was safe to include in source code.)
4154Some legacy implementations only support 8-bit characters,
4155and some implementations provide 8-bit only variants.
4156On these implementations, the @var{*utf-8-external-format*}
4157gracefully falls back to @code{:default},
4158and Unicode characters will be read as multi-character mojibake.
4159To detect such situations, UIOP will push the @code{:asdf-unicode} feature
4160on implementations that support Unicode, and you can use reader-conditionalization
4161to protect any @code{:encoding @emph{encoding}} statement, as in
4162@code{#+asdf-unicode :encoding #+asdf-unicode :utf-8}.
4163We recommend that you avoid using unprotected @code{:encoding} specifications
4164until after ASDF 2.21 or later becomes widespread
4165(in April 2014, only LispWorks lags with ASDF 2.019,
4166and is scheduled to be updated later this year).
4168While it offers plenty of hooks for extension,
4169and one such extension is available (see @code{asdf-encodings} below),
4170ASDF itself only recognizes one encoding beside @code{:default},
4171and that is @code{:utf-8}, which is the @emph{de facto} standard,
4172already used by the vast majority of libraries that use more than ASCII.
4173On implementations that do not support unicode,
4174the feature @code{:asdf-unicode} is absent, and
4175the @code{:default} external-format is used
4176to read even source files declared as @code{:utf-8}.
4177On these implementations, non-ASCII characters
4178intended to be read as one CL character
4179may thus end up being read as multiple CL characters.
4180In most cases, this shouldn't affect the software's semantics:
4181comments will be skipped just the same, strings with be read and printed
4182with slightly different lengths, symbol names will be accordingly longer,
4183but none of it should matter.
4184But a few systems that actually depend on unicode characters
4185may fail to work properly, or may work in a subtly different way.
4186See for instance @code{lambda-reader}.
4188We invite you to embrace UTF-8
4189as the encoding for non-ASCII characters starting today,
4190even without any explicit specification in your @file{.asd} files.
4191Indeed, on some implementations and configurations,
4192UTF-8 is already the @code{:default},
4193and loading your code may cause errors if it is encoded in anything but UTF-8.
4194Therefore, even with the legacy behavior,
4195non-UTF-8 is guaranteed to break for some users,
4196whereas UTF-8 is pretty much guaranteed not to break anywhere
4197(provided you do @emph{not} use a BOM),
4198although it might be read incorrectly on some implementations.
4199@code{:utf-8} has been the default value of @code{*default-encoding*} since ASDF 3.
4201If you need non-standard character encodings for your source code,
4202use the extension system @code{asdf-encodings}, by specifying
4203@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
4204This extension system will register support for more encodings using the
4205@code{*encoding-external-format-hook*} facility,
4206so you can explicitly specify @code{:encoding :latin1}
4207in your @file{.asd} file.
4208Using the @code{*encoding-detection-hook*} it will also
4209eventually implement some autodetection of a file's encoding
4210from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
4211or otherwise based on an analysis of octet patterns in the file.
4212At this point, @code{asdf-encoding} only supports the encodings
4213that are supported as part of your implementation.
4214Since the list varies depending on implementations,
4215we still recommend you use @code{:utf-8} everywhere,
4216which is the most portable (next to it is @code{:latin1}).
4218Recent versions of Quicklisp include @code{asdf-encodings};
4219if you're not using it, you may get this extension using git:
4220@kbd{git clone git://}
4222@kbd{git clone ssh://}.
4223You can also browse the repository on
4226When you use @code{asdf-encodings},
4227any @file{.asd} file loaded
4228will use the autodetection algorithm to determine its encoding.
4229If you depend on this detection happening,
4230you should explicitly load @code{asdf-encodings} early in your build.
4231Note that @code{:defsystem-depends-on} cannot be used here: by the time
4232the @code{:defsystem-depends-on} is loaded, the enclosing
4233@code{defsystem} form has already been read.
4235In practice, this means that the @code{*default-encoding*}
4236is usually used for @file{.asd} files.
4237Currently, this defaults to @code{:utf-8}, and
4238you should be safe using Unicode characters in those files.
4239This might matter, for instance, in meta-data about author's names.
4240Otherwise, the main data in these files is component (path)names,
4241and we don't recommend using non-ASCII characters for these,
4242for the result probably isn't very portable.
4244@section Miscellaneous Functions
4246These functions are exported by ASDF for your convenience.
4249@defun system-relative-pathname system name @Akey{} type
4251It's often handy to locate a file relative to some system.
4252The @code{system-relative-pathname} function meets this need.
4254It takes two mandatory arguments @var{system} and @var{name}
4255and a keyword argument @var{type}:
4256@var{system} is name of a system, whereas @var{name} and optionally @var{type}
4257specify a relative pathname, interpreted like a component pathname specifier
4258by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
4260It returns a pathname built from the location of the system's
4261source directory and the relative pathname. For example:
4264> (asdf:system-relative-pathname 'cl-ppcre "")
4266@end lisp
4268@end defun
4270@defun system-source-directory system-designator
4272ASDF does not provide a turnkey solution for locating
4273data (or other miscellaneous) files
4274that are distributed together with the source code of a system.
4275Programmers can use @code{system-source-directory} to find such files.
4276Returns a pathname object.
4277The @var{system-designator} may be a string, symbol, or ASDF system object.
4278@end defun
4280@defun clear-system system-designator
4282It is sometimes useful to force recompilation of a previously loaded system.
4283For these cases, @code{(asdf:clear-system :foo)}
4284will remove the system from the table of currently loaded systems:
4285the next time the system @code{foo} or one that depends on it is re-loaded,
4286@code{foo} will be loaded again.@footnote{Alternatively, you could touch @code{foo.asd} or
4287remove the corresponding fasls from the output file cache.}
4289Note that this does not and cannot undo
4290the previous loading of the system.
4291Common Lisp has no provision for such an operation,
4292and its reliance on irreversible side-effects to global data structures
4293makes such a thing impossible in the general case.
4294If the software being re-loaded is not conceived with hot upgrade in mind,
4295re-loading may cause many errors, warnings or subtle silent problems,
4296as packages, generic function signatures, structures, types, macros, constants, etc.
4297are being redefined incompatibly.
4298It is up to the user to make sure that reloading is possible and has the desired effect.
4299In some cases, extreme measures such as recursively deleting packages,
4300unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
4301and much more are necessary for reloading to happen smoothly.
4302ASDF itself goes to extensive effort to make a hot upgrade possible
4303with respect to its own code.
4304If you want, you can reuse some of its utilities such as
4305@code{uiop:define-package} and @code{uiop:with-upgradability},
4306and get inspiration (or disinspiration)
4307from what it does in @file{header.lisp} and @file{upgrade.lisp}.
4308@end defun
4310@defun register-preloaded-system name @Arest{} keys
4311A system with name @var{name},
4312created by @code{make-instance} with extra keys @var{keys}
4313(e.g. @code{:version}),
4314is registered as @emph{preloaded}.
4315That is, its code has already been loaded into the current image,
4316and if at some point some other system @code{:depends-on} it yet no source code is found,
4317it is considered as already provided,
4318and ASDF will not raise a @code{missing-component} error.
4320This function is particularly useful if you distribute your code
4321as fasls with either @code{compile-bundle-op} or @code{monolithic-compile-bundle-op},
4322and want to register systems so that dependencies will work uniformly
4323whether you're using your software from source or from fasl.
4324@end defun
4326@defun run-shell-command control-string @Arest{} args
4328This function is obsolete and present only for the sake of backwards-compatibility:
4329``If it's not backwards, it's not compatible''. We @emph{strongly} discourage its use.
4330Its current behavior is only well-defined on Unix platforms
4331(which include MacOS X and cygwin). On Windows, anything goes.
4332The following documentation is only for the purpose of your migrating away from it
4333in a way that preserves semantics.
4335Instead we recommend the use @code{run-program}, described in the next section, and
4336available as part of ASDF since ASDF 3.
4338@code{run-shell-command} takes as arguments a format @code{control-string}
4339and arguments to be passed to @code{format} after this control-string
4340to produce a string.
4341This string is a command that will be evaluated with a POSIX shell if possible;
4342yet, on Windows, some implementations will use CMD.EXE,
4343while others (like SBCL) will make an attempt at invoking a POSIX shell
4344(and fail if it is not present).
4345@end defun
4347@node Some Utility Functions,  , Controlling source file character encoding, Miscellaneous additional functionality
4348@section Some Utility Functions
4350The below functions are not exported by ASDF itself, but by UIOP, available since ASDF 3.
4351Some of them have precursors in ASDF 2, but we recommend
4352you rely on ASDF 3 for active developments.
4353UIOP provides many, many more utility functions, and we recommend
4354you read its README and sources for more information.
4357@defun parse-unix-namestring name @Akey{} type defaults dot-dot ensure-directory @AallowOtherKeys
4358Coerce NAME into a PATHNAME using standard Unix syntax.
4360Unix syntax is used whether or not the underlying system is Unix;
4361on non-Unix systems it is only usable for relative pathnames.
4362In order to manipulate relative pathnames portably, it is crucial
4363to possess a portable pathname syntax independent of the underlying OS.
4364This is what @code{parse-unix-namestring} provides, and why we use it in ASDF.
4366When given a @code{pathname} object, just return it untouched.
4367When given @code{nil}, just return @code{nil}.
4368When given a non-null @code{symbol}, first downcase its name and treat it as a string.
4369When given a @code{string}, portably decompose it into a pathname as below.
4371@code{#\/} separates directory components.
4373The last @code{#\/}-separated substring is interpreted as follows:
43741- If @var{type} is @code{:directory} or @var{ensure-directory} is true,
4375 the string is made the last directory component, and its @code{name} and @code{type} are @code{nil}.
4376 if the string is empty, it's the empty pathname with all slots @code{nil}.
43772- If @var{type} is @code{nil}, the substring is a file-namestring,
4378 and its @code{name} and @code{type} are separated by @code{split-name-type}.
43793- If @var{type} is a string, it is the given @code{type}, and the whole string is the @code{name}.
4381Directory components with an empty name the name @code{.} are removed.
4382Any directory named @code{..} is read as @var{dot-dot},
4383which must be one of @code{:back} or @code{:up} and defaults to @code{:back}.
4385@vindex *nil-pathname*
4386@code{host}, @code{device} and @code{version} components are taken from @var{defaults},
4387which itself defaults to @code{*nil-pathname*}.
4388@code{*nil-pathname*} is also used if @var{defaults} is @code{nil}.
4389No host or device can be specified in the string itself,
4390which makes it unsuitable for absolute pathnames outside Unix.
4392For relative pathnames, these components (and hence the defaults) won't matter
4393if you use @code{merge-pathnames*} but will matter if you use @code{merge-pathnames},
4394which is an important reason to always use @code{merge-pathnames*}.
4396Arbitrary keys are accepted, and the parse result is passed to @code{ensure-pathname}
4397with those keys, removing @var{type}, @var{defaults} and @var{dot-dot}.
4398When you're manipulating pathnames that are supposed to make sense portably
4399even though the OS may not be Unixish, we recommend you use @code{:want-relative t}
4400so that @code{parse-unix-namestring} will throw an error if the pathname is absolute.
4401@end defun
4403@defun merge-pathnames* specified @Aoptional{} defaults
4405This function is a replacement for @code{merge-pathnames} that uses the host and device
4406from the @var{defaults} rather than the @var{specified} pathname when the latter
4407is a relative pathname. This allows ASDF and its users to create and use relative pathnames
4408without having to know beforehand what are the host and device
4409of the absolute pathnames they are relative to.
4411@end defun
4413@defun subpathname pathname subpath @Akey{} type
4415This function takes a @var{pathname} and a @var{subpath} and a @var{type}.
4416If @var{subpath} is already a @code{pathname} object (not namestring),
4417and is an absolute pathname at that, it is returned unchanged;
4418otherwise, @var{subpath} is turned into a relative pathname with given @var{type}
4419as per @code{parse-unix-namestring} with @code{:want-relative t :type }@var{type},
4420then it is merged with the @code{pathname-directory-pathname} of @var{pathname},
4421as per @code{merge-pathnames*}.
4423We strongly encourage the use of this function
4424for portably resolving relative pathnames in your code base.
4425@end defun
4427@defun subpathname* pathname subpath @Akey{} type
4429This function returns @code{nil} if the base @var{pathname} is @code{nil},
4430otherwise acts like @code{subpathname}.
4431@end defun
4433@defun run-program command @Akey{} ignore-error-status force-shell input output @
4434error-output if-input-does-not-exist if-output-exists if-error-output-exists @
4435element-type external-format @AallowOtherKeys
4437@code{run-program} takes a @var{command} argument that is either
4438a list of a program name or path and its arguments,
4439or a string to be executed by a shell.
4440It spawns the command, waits for it to return,
4441verifies that it exited cleanly (unless told not too below),
4442and optionally captures and processes its output.
4443It accepts many keyword arguments to configure its behavior.
4445@code{run-program} returns three values: the first for the output,
4446the second for the error-output, and the third for the return value.
4447(Beware that before ASDF, it didn't handle input or error-output,
4448and returned only one value,
4449the one for the output if any handler was specified, or else the exit code;
4450please upgrade ASDF, or at least UIOP, to rely on the new enhanced behavior.)
4452@var{output} is its most important argument;
4453it specifies how the output is captured and processed.
4454If it is @code{nil}, then the output is redirected to the null device,
4455that will discard it.
4456If it is @code{:interactive}, then it is inherited from the current process
4457(beware: this may be different from your @var{*standard-output*},
4458and under SLIME will be on your @code{*inferior-lisp*} buffer).
4459If it is @code{t}, output goes to your current @var{*standard-output*} stream.
4460Otherwise, @var{output} should be a value that is a suitable first argument to
4461@code{slurp-input-stream} (see below), or
4462a list of such a value and keyword arguments.
4463In this case, @code{run-program} will
4464create a temporary stream for the program output;
4465the program output, in that stream,
4466will be processed by a call to @code{slurp-input-stream},
4467using @var{output} as the first argument
4468(or if it's a list the first element of @var{output} and the rest as keywords).
4469The primary value resulting from that call
4470(or @code{nil} if no call was needed)
4471will be the first value returned by @code{run-program}.
4472E.g., using @code{:output :string}
4473will have it return the entire output stream as a string.
4474And using @code{:output '(:string :stripped t)}
4475will have it return the same string stripped of any ending newline.
4477@var{error-output} is similar to @var{output}, except that
4478the resulting value is returned as the second value of @code{run-program}.
4479@code{t} designates the @var{*error-output*}.
4480Also @code{:output} means redirecting the error output to the output stream,
4481in which case @code{nil} is returned.
4483@var{input} is similar to @var{output}, except that
4484@code{vomit-output-stream} is used, no value is returned,
4485and @code{t} designates the @var{*standard-input*}.
4487@code{element-type} and @code{external-format} are passed on
4488to your Lisp implementation, when applicable, for creation of the output stream.
4490One and only one of the stream slurping or vomiting may or may not happen
4491in parallel in parallel with the subprocess,
4492depending on options and implementation,
4493and with priority being given to output processing.
4494Other streams are completely produced or consumed
4495before or after the subprocess is spawned, using temporary files.
4497@code{force-shell} forces evaluation of the command through a shell,
4498even if it was passed as a list rather than a string.
4499If a shell is used, it is @file{/bin/sh} on Unix or @file{CMD.EXE} on Windows,
4500except on implementations that (erroneously, IMNSHO)
4501insist on consulting @code{$SHELL} like clisp.
4503@code{ignore-error-status} causes @code{run-program}
4504to not raise an error if the spawned program exits in error.
4505Following POSIX convention, an error is anything but
4506a normal exit with status code zero.
4507By default, an error of type @code{subprocess-error} is raised in this case.
4509@code{run-program} works on all platforms supported by ASDF, except Genera.
4510See the source code for more documentation.
4512@end defun
4514@defun slurp-input-stream processor input-stream @Akey{}
4516@code{slurp-input-stream} is a generic function of two arguments, a target object and an input stream,
4517and accepting keyword arguments.
4518Predefined methods based on the target object are as follows:
4522If the object is a function, the function is called with the stream as argument.
4524@item If the object is a cons, its first element is applied to its rest appended by
4525a list of the input stream.
4527@item If the object is an output stream, the contents of the input stream are copied to it.
4528If the @var{linewise} keyword argument is provided, copying happens line by line,
4529and an optional @var{prefix} is printed before each line.
4530Otherwise, copying happen based on a buffer of size @var{buffer-size},
4531using the specified @var{element-type}.
4533@item If the object is @code{'string} or @code{:string}, the content is captured into a string.
4534Accepted keywords include the @var{element-type} and a flag @var{stripped},
4535which when true causes any single line ending to be removed as per @code{uiop:stripln}.
4537@item If the object is @code{:lines}, the content is captured as a list of strings,
4538one per line, without line ending. If the @var{count} keyword argument is provided,
4539it is a maximum count of lines to be read.
4541@item If the object is @code{:line}, the content is captured as with @code{:lines} above,
4542and then its sub-object is extracted with the @var{at} argument,
4543which defaults to @code{0}, extracting the first line.
4544A number will extract the corresponding line.
4545See the documentation for @code{uiop:access-at}.
4547@item If the object is @code{:forms}, the content is captured as a list of S-expressions,
4548as read by the Lisp reader.
4549If the @var{count} argument is provided,
4550it is a maximum count of lines to be read.
4551We recommend you control the syntax with such macro as
4554@item If the object is @code{:form}, the content is captured as with @code{:forms} above,
4555and then its sub-object is extracted with the @var{at} argument,
4556which defaults to @code{0}, extracting the first form.
4557A number will extract the corresponding form.
4558See the documentation for @code{uiop:access-at}.
4559We recommend you control the syntax with such macro as
4561@end itemize
4562@end defun
4565@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
4566@comment  node-name,  next,  previous,  up
4567@chapter Getting the latest version
4569Decide which version you want.
4570The @code{master} branch is where development happens;
4571its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
4572but an occasional regression may happen despite our (limited) test suite.
4574The @code{release} branch is what cautious people should be using;
4575it has usually been tested more, and releases are cut at a point
4576where there isn't any known unresolved issue.
4578You may get the ASDF source repository using git:
4579@kbd{git clone git://}
4581You will find the above referenced tags in this repository.
4582You can also browse the repository on
4585Discussion of ASDF development is conducted on the
4586mailing list
4591@node FAQ, Ongoing Work, Getting the latest version, Top
4592@comment  node-name,  next,  previous,  up
4593@chapter FAQ
4596* Where do I report a bug?::
4597* What has changed between ASDF 1 and ASDF 2?::
4598* Issues with installing the proper version of ASDF::
4599* Issues with configuring ASDF::
4600* Issues with using and extending ASDF to define systems::
4601* ASDF development FAQs::
4602@end menu
4604@node Where do I report a bug?, What has changed between ASDF 1 and ASDF 2?, FAQ, FAQ
4605@section  ``Where do I report a bug?''
4607ASDF bugs are tracked on launchpad: @url{}.
4609If you're unsure about whether something is a bug, or for general discussion,
4610use the @url{,asdf-devel mailing list}
4613@node What has changed between ASDF 1 and ASDF 2?, Issues with installing the proper version of ASDF, Where do I report a bug?, FAQ
4614@section ``What has changed between ASDF 1, ASDF 2 and ASDF 3?''
4616We released ASDF 2.000 on May 31st 2010,
4617and ASDF 3.0.0 on May 15th 2013.
4618Releases of ASDF 2 and later have since then been included
4619in all actively maintained CL implementations that used to bundle ASDF 1,
4620plus some implementations that previously did not.
4621ASDF has been made to work with all actively maintained CL
4622implementations and even a few implementations that are @emph{not}
4623actively maintained.
4624@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
4625Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly
4626(though we recommend instead upgrading your implementation or its ASDF module).
4627For this reason, we have stopped supporting ASDF 1 and ASDF 2.
4628If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
4629we recommend you upgrade to ASDF 3
4630--- and we explain how to do that. @xref{Loading ASDF}.
4631(In the context of compatibility requirements,
4632ASDF 2.27, released on Feb 1st 2013, and further 2.x releases up to 2.33,
4633count as pre-releases of ASDF 3, and define the @code{:asdf3} feature;
4634still, please use the latest release).
4635Release ASDF 3.1.2 and later also define the @code{:asdf3.1} feature.
4639* What are ASDF 1 2 3?::
4640* How do I detect the ASDF version?::
4641* ASDF can portably name files in subdirectories::
4642* Output translations::
4643* Source Registry Configuration::
4644* Usual operations are made easier to the user::
4645* Many bugs have been fixed::
4646* ASDF itself is versioned::
4647* ASDF can be upgraded::
4648* Decoupled release cycle::
4649* Pitfalls of the transition to ASDF 2::
4650@end menu
4652@node What are ASDF 1 2 3?, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?, What has changed between ASDF 1 and ASDF 2?
4653@subsection What are ASDF 1, ASDF 2, and ASDF 3?
4655ASDF 1 refers to any release earlier than 1.369 or so (from August 2001 to October 2009),
4656and to any development revision earlier than 2.000 (May 2010).
4657If your copy of ASDF doesn't even contain version information, it's an old ASDF 1.
4658Revisions between 1.656 and 1.728 may count as development releases for ASDF 2.
4660ASDF 2 refers to releases from 2.000 (May 31st 2010) to 2.26 (Oct 30 2012),
4661and any development revision newer than ASDF 1 and older than 2.27 (Feb 1 2013).
4663ASDF 3 refers to releases from 2.27 (Feb 1 2013) to 2.33 and 3.0.0 onward (May 15 2013).
46642.27 to 2.33 count as pre-releases to ASDF 3.
4666@node How do I detect the ASDF version?, ASDF can portably name files in subdirectories, What are ASDF 1 2 3?, What has changed between ASDF 1 and ASDF 2?
4667@subsection How do I detect the ASDF version?
4668@findex asdf-version
4669@cindex *features*
4671All releases of ASDF
4672push @code{:asdf} onto @code{*features*}.
4673Releases starting with ASDF 2
4674push @code{:asdf2} onto @code{*features*}.
4675Releases starting with ASDF 3 (including 2.27 and later pre-releases)
4676push @code{:asdf3} onto @code{*features*}.
4677Furthermore, releases starting with ASDF 3.1.2 (May 2014),
4678though they count as ASDF 3, include enough progress that they
4679push @code{:asdf3.1} onto @code{*features*}.
4680You may depend on the presence or absence of these features
4681to write code that takes advantage of recent ASDF functionality
4682but still works on older versions, or at least detects the old version and signals an error.
4684Additionally, all releases starting with ASDF 2
4685define a function @code{(asdf:asdf-version)} you may use to query the version.
4686All releases starting with 2.013 display the version number prominently
4687on the second line of the @file{asdf.lisp} source file.
4689If you are experiencing problems or limitations of any sort with ASDF 1 or ASDF 2,
4690we recommend that you should upgrade to the latest release, be it ASDF 3 or other.
4693@node ASDF can portably name files in subdirectories, Output translations, How do I detect the ASDF version?, What has changed between ASDF 1 and ASDF 2?
4694@subsection ASDF can portably name files in subdirectories
4696Common Lisp namestrings are not portable,
4697except maybe for logical pathname namestrings,
4698that themselves have various limitations and require a lot of setup
4699that is itself ultimately non-portable.
4701In ASDF 1, the only portable ways to refer to pathnames inside systems and components
4702were very awkward, using @code{#.(make-pathname ...)} and
4703@code{#.(merge-pathnames ...)}.
4704Even the above were themselves were inadequate in the general case
4705due to host and device issues, unless horribly complex patterns were used.
4706Plenty of simple cases that looked portable actually weren't,
4707leading to much confusion and greavance.
4709ASDF 2 implements its own portable syntax for strings as pathname specifiers.
4710Naming files within a system definition becomes easy and portable again.
4711@xref{Miscellaneous additional functionality,system-relative-pathname},
4715On the other hand, there are places where systems used to accept namestrings
4716where you must now use an explicit pathname object:
4717@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4718must now be written with the @code{#p} syntax:
4719@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4721@xref{The defsystem grammar,,Pathname specifiers}.
4724@node Output translations, Source Registry Configuration, ASDF can portably name files in subdirectories, What has changed between ASDF 1 and ASDF 2?
4725@subsection Output translations
4727A popular feature added to ASDF was output pathname translation:
4728@code{asdf-binary-locations}, @code{common-lisp-controller},
4729@code{cl-launch} and other hacks were all implementing it in ways
4730both mutually incompatible and difficult to configure.
4732Output pathname translation is essential to share
4733source directories of portable systems across multiple implementations
4734or variants thereof,
4735or source directories of shared installations of systems across multiple users,
4736or combinations of the above.
4738In ASDF 2, a standard mechanism is provided for that,
4740with sensible defaults, adequate configuration languages,
4741a coherent set of configuration files and hooks,
4742and support for non-Unix platforms.
4744@xref{Controlling where ASDF saves compiled files}.
4746@node Source Registry Configuration, Usual operations are made easier to the user, Output translations, What has changed between ASDF 1 and ASDF 2?
4747@subsection Source Registry Configuration
4749Configuring ASDF used to require special magic
4750to be applied just at the right moment,
4751between the moment ASDF is loaded and the moment it is used,
4752in a way that is specific to the user,
4753the implementation he is using and the application he is building.
4755This made for awkward configuration files and startup scripts
4756that could not be shared between users, managed by administrators
4757or packaged by distributions.
4759ASDF 2 provides a well-documented way to configure ASDF,
4760with sensible defaults, adequate configuration languages,
4761and a coherent set of configuration files and hooks.
4763We believe it's a vast improvement because it decouples
4764application distribution from library distribution.
4765The application writer can avoid thinking where the libraries are,
4766and the library distributor (dpkg, clbuild, advanced user, etc.)
4767can configure them once and for every application.
4768Yet settings can be easily overridden where needed,
4769so whoever needs control has exactly as much as required.
4771At the same time, ASDF 2 remains compatible
4772with the old magic you may have in your build scripts
4773(using @code{*central-registry*} and
4775to tailor the ASDF configuration to your build automation needs,
4776and also allows for new magic, simpler and more powerful magic.
4778@xref{Controlling where ASDF searches for systems}.
4781@node Usual operations are made easier to the user, Many bugs have been fixed, Source Registry Configuration, What has changed between ASDF 1 and ASDF 2?
4782@subsection Usual operations are made easier to the user
4784In ASDF 1, you had to use the awkward syntax
4785@code{(asdf:oos 'asdf:load-op :foo)}
4786to load a system,
4787and similarly for @code{compile-op}, @code{test-op}.
4789In ASDF 2, you can use shortcuts for the usual operations:
4790@code{(asdf:load-system :foo)}, and
4791similarly for @code{compile-system}, @code{test-system}.
4794@node Many bugs have been fixed, ASDF itself is versioned, Usual operations are made easier to the user, What has changed between ASDF 1 and ASDF 2?
4795@subsection Many bugs have been fixed
4797The following issues and many others have been fixed:
4801The infamous TRAVERSE function has been revamped completely
4802between ASDF 1 and ASDF 2, with many bugs squashed.
4803In particular, dependencies were not correctly propagated
4804across modules but now are.
4805It has been completely rewritten many times over
4806between ASDF 2.000 and ASDF 3,
4807with fundamental issues in the original model being fixed.
4808Timestamps were not propagated at all, and now are.
4809The internal model of how actions depend on each other
4810is now both consistent and complete.
4811The @code{:version} and
4812the @code{:force (system1 .. systemN)} feature have been fixed.
4815Performance has been notably improved for large systems
4816(say with thousands of components) by using
4817hash-tables instead of linear search,
4818and linear-time list accumulation instead of cubic time recursive append,
4819for an overall @emph{O(n)} complexity vs @emph{O(n^4)}.
4822Many features used to not be portable,
4823especially where pathnames were involved.
4824Windows support was notably quirky because of such non-portability.
4827The internal test suite used to massively fail on many implementations.
4828While still incomplete, it now fully passes
4829on all implementations supported by the test suite,
4830though some tests are commented out on a few implementations.
4833Support was lacking for some implementations.
4834ABCL and GCL were notably wholly broken.
4835ECL extensions were not integrated with ASDF release.
4838The documentation was grossly out of date.
4840@end itemize
4843@node ASDF itself is versioned, ASDF can be upgraded, Many bugs have been fixed, What has changed between ASDF 1 and ASDF 2?
4844@subsection ASDF itself is versioned
4846Between new features, old bugs fixed, and new bugs introduced,
4847there were various releases of ASDF in the wild,
4848and no simple way to check which release had which feature set.
4849People using or writing systems had to either make worst-case assumptions
4850as to what features were available and worked,
4851or take great pains to have the correct version of ASDF installed.
4853With ASDF 2, we provide a new stable set of working features
4854that everyone can rely on from now on.
4855Use @code{#+asdf2} to detect presence of ASDF 2,
4856@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
4857to check the availability of a version no earlier than required.
4860@node ASDF can be upgraded, Decoupled release cycle, ASDF itself is versioned, What has changed between ASDF 1 and ASDF 2?
4861@subsection ASDF can be upgraded
4863When an old version of ASDF was loaded,
4864it was very hard to upgrade ASDF in your current image
4865without breaking everything.
4866Instead you had to exit the Lisp process and
4867somehow arrange to start a new one from a simpler image.
4868Something that can't be done from within Lisp,
4869making automation of it difficult,
4870which compounded with difficulty in configuration,
4871made the task quite hard.
4872Yet as we saw before, the task would have been required
4873to not have to live with the worst case or non-portable
4874subset of ASDF features.
4876With ASDF 2, it is easy to upgrade
4877from ASDF 2 to later versions from within Lisp,
4878and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
4879We support hot upgrade of ASDF and any breakage is a bug
4880that we will do our best to fix.
4881There are still limitations on upgrade, though,
4882most notably the fact that after you upgrade ASDF,
4883you must also reload or upgrade all ASDF extensions.
4885@node Decoupled release cycle, Pitfalls of the transition to ASDF 2, ASDF can be upgraded, What has changed between ASDF 1 and ASDF 2?
4886@subsection Decoupled release cycle
4888When vendors were releasing their Lisp implementations with ASDF,
4889they had to basically never change version
4890because neither upgrade nor downgrade was possible
4891without breaking something for someone,
4892and no obvious upgrade path was visible and recommendable.
4894With ASDF 2, upgrade is possible, easy and can be recommended.
4895This means that vendors can safely ship a recent version of ASDF,
4896confident that if a user isn't fully satisfied,
4897he can easily upgrade ASDF and deal
4898with a supported recent version of it.
4899This means that release cycles will be causally decoupled,
4900the practical consequence of which will mean faster convergence
4901towards the latest version for everyone.
4904@node Pitfalls of the transition to ASDF 2,  , Decoupled release cycle, What has changed between ASDF 1 and ASDF 2?
4905@subsection Pitfalls of the transition to ASDF 2
4907The main pitfalls in upgrading to ASDF 2 seem to be related
4908to the output translation mechanism.
4913Output translations is enabled by default. This may surprise some users,
4914most of them in pleasant way (we hope), a few of them in an unpleasant way.
4915It is trivial to disable output translations.
4916@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
4919Some systems in the large have been known
4920not to play well with output translations.
4921They were relatively easy to fix.
4922Once again, it is also easy to disable output translations,
4923or to override its configuration.
4926The new ASDF output translations are incompatible with ASDF-Binary-Locations.
4927They replace A-B-L, and there is compatibility mode to emulate
4928your previous A-B-L configuration.
4929See @code{enable-asdf-binary-locations-compatibility} in
4930@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
4931But thou shalt not load ABL on top of ASDF 2.
4933@end itemize
4935Other issues include the following:
4940ASDF pathname designators are now specified
4941in places where they were unspecified,
4942and a few small adjustments have to be made to some non-portable defsystems.
4943Notably, in the @code{:pathname} argument
4944to a @code{defsystem} and its components,
4945a logical pathname (or implementation-dependent hierarchical pathname)
4946must now be specified with @code{#p} syntax
4947where the namestring might have previously sufficed;
4948moreover when evaluation is desired @code{#.} must be used,
4949where it wasn't necessary in the toplevel @code{:pathname} argument
4950(but necessary in other @code{:pathname} arguments).
4953There is a slight performance bug, notably on SBCL,
4954when initially searching for @file{asd} files,
4955the implicit @code{(directory "/configured/path/**/*.asd")}
4956for every configured path @code{(:tree "/configured/path/")}
4957in your @code{source-registry} configuration can cause a slight pause.
4958Try to @code{(time (asdf:initialize-source-registry))}
4959to see how bad it is or isn't on your system.
4960If you insist on not having this pause,
4961you can avoid the pause by overriding the default source-registry configuration
4962and not use any deep @code{:tree} entry but only @code{:directory} entries
4963or shallow @code{:tree} entries.
4964Or you can fix your implementation to not be quite that slow
4965when recursing through directories.
4966@emph{Update}: This performance bug fixed the hard way in 2.010.
4969On Windows, only LispWorks supports proper default configuration pathnames
4970based on the Windows registry.
4971Other implementations make do with environment variables,
4972that you may have to define yourself
4973if you're using an older version of Windows.
4974Windows support is somewhat less tested than Unix support.
4975Please help report and fix bugs.
4976@emph{Update}: As of ASDF 2.21, all implementations
4977should now use the same proper default configuration pathnames
4978and they should actually work, though they haven't all been tested.
4981The mechanism by which one customizes a system so that Lisp files
4982may use a different extension from the default @file{.lisp} has changed.
4983Previously, the pathname for a component
4984was lazily computed when operating on a system,
4985and you would
4986@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
4987  (declare (ignorable component system)) "lis")}.
4988Now, the pathname for a component is eagerly computed when defining the system,
4989and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
4990and use @code{:default-component-class cl-source-file.lis}
4991as argument to @code{defsystem},
4992as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
4994@findex source-file-type
4997@end itemize
5000@node Issues with installing the proper version of ASDF, Issues with configuring ASDF, What has changed between ASDF 1 and ASDF 2?, FAQ
5001@section Issues with installing the proper version of ASDF
5004* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
5005* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
5006@end menu
5008@node My Common Lisp implementation comes with an outdated version of ASDF. What to do?, I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?, Issues with installing the proper version of ASDF, Issues with installing the proper version of ASDF
5009@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
5011We recommend you upgrade ASDF.
5012@xref{Loading ASDF,,Upgrading ASDF}.
5014If this does not work, it is a bug, and you should report it.
5015@xref{FAQ, report-bugs, Where do I report a bug}.
5016In the meantime, you can load @file{asdf.lisp} directly.
5017@xref{Loading ASDF,Loading an otherwise installed ASDF}.
5020@node I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?,  , My Common Lisp implementation comes with an outdated version of ASDF. What to do?, Issues with installing the proper version of ASDF
5021@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
5023Since ASDF 2,
5024it should always be a good time to upgrade to a recent version of ASDF.
5025You may consult with the maintainer for which specific version they recommend,
5026but the latest @code{release} should be correct.
5027Though we do try to test ASDF releases against all implementations that we can,
5028we may not be testing against all variants of your implementation,
5029and we may not be running enough tests;
5030we trust you to thoroughly test it with your own implementation
5031before you release it.
5032If there are any issues with the current release,
5033it's a bug that you should report upstream and that we will fix ASAP.
5035As to how to include ASDF, we recommend the following:
5039If ASDF isn't loaded yet, then @code{(require "asdf")}
5040should load the version of ASDF that is bundled with your system.
5041If possible so should @code{(require "ASDF")}.
5042You may have it load some other version configured by the user,
5043if you allow such configuration.
5046If your system provides a mechanism to hook into @code{CL:REQUIRE},
5047then it would be nice to add ASDF to this hook the same way that
5049Please send us appropriate code to this end.
5052You may, like SBCL since 1.1.13 or MKCL since 1.1.9,
5053have ASDF create bundle FASLs
5054that are provided as modules by your Lisp distribution.
5055You may also, but we don't recommend that anymore,
5056have ASDF like SBCL up until 1.1.12 be implicitly used
5057when requiring modules that are provided by your Lisp distribution;
5058if you do, you should add them in the beginning of both
5059@code{wrapping-source-registry} and @code{wrapping-output-translations}.
5062If you have magic systems as above, like SBCL used to do,
5063then we explicitly ask you to @emph{NOT} distribute
5064@file{asdf.asd} as part of those magic systems.
5065You should still include the file @file{asdf.lisp} in your source distribution
5066and precompile it in your binary distribution,
5067but @file{asdf.asd} if included at all,
5068should be secluded from the magic systems,
5069in a separate file hierarchy.
5070Alternatively, you may provide the system
5071after renaming it and its @file{.asd} file to e.g.
5072@code{asdf-ecl} and @file{asdf-ecl.asd}, or
5073@code{sb-asdf} and @file{sb-asdf.asd}.
5074Indeed, if you made @file{asdf.asd} a magic system,
5075then users would no longer be able to upgrade ASDF using ASDF itself
5076to some version of their preference that
5077they maintain independently from your Lisp distribution.
5080If you do not have any such magic systems, or have other non-magic systems
5081that you want to bundle with your implementation,
5082then you may add them to the @code{wrapping-source-registry},
5083and you are welcome to include @file{asdf.asd} amongst them.
5084Non-magic systems should be at the back of the @code{wrapping-source-registry}
5085while magic systems are at the front.
5086If they are precompiled,
5087they should also be in the @code{wrapping-output-translations}.
5090Since ASDF 3, the library UIOP comes transcluded in ASDF.
5091But if you want to be nice to users who care for UIOP but not for ASDF,
5092you may package UIOP separately,
5093so that one may @code{(require "uiop")} and not load ASDF,
5094or one may @code{(require "asdf")}
5095which would implicitly require and load the former.
5098Please send us upstream any patches you make to ASDF itself,
5099so we can merge them back in for the benefit of your users
5100when they upgrade to the upstream version.
5102@end itemize
5106@node Issues with configuring ASDF, Issues with using and extending ASDF to define systems, Issues with installing the proper version of ASDF, FAQ
5107@section Issues with configuring ASDF
5110* How can I customize where fasl files are stored?::
5111* How can I wholly disable the compiler output cache?::
5112@end menu
5114@node How can I customize where fasl files are stored?, How can I wholly disable the compiler output cache?, Issues with configuring ASDF, Issues with configuring ASDF
5115@subsection ``How can I customize where fasl files are stored?''
5117@xref{Controlling where ASDF saves compiled files}.
5119Note that in the past there was an add-on to ASDF called
5120@code{ASDF-binary-locations}, developed by Gary King.
5121That add-on has been merged into ASDF proper,
5122then superseded by the @code{asdf-output-translations} facility.
5124Note that use of @code{asdf-output-translations}
5125can interfere with one aspect of your systems
5126--- if your system uses @code{*load-truename*} to find files
5127(e.g., if you have some data files stored with your program),
5128then the relocation that this ASDF customization performs
5129is likely to interfere.
5130Use @code{asdf:system-relative-pathname} to locate a file
5131in the source directory of some system, and
5132use @code{asdf:apply-output-translations} to locate a file
5133whose pathname has been translated by the facility.
5135@node How can I wholly disable the compiler output cache?,  , How can I customize where fasl files are stored?, Issues with configuring ASDF
5136@subsection ``How can I wholly disable the compiler output cache?''
5138To permanently disable the compiler output cache
5139for all future runs of ASDF, you can:
5142mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
5143echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
5144@end example
5146This assumes that you didn't otherwise configure the ASDF files
5147(if you did, edit them again),
5148and don't somehow override the configuration at runtime
5149with a shell variable (see below) or some other runtime command
5150(e.g. some call to @code{asdf:initialize-output-translations}).
5152To disable the compiler output cache in Lisp processes
5153run by your current shell, try (assuming @code{bash} or @code{zsh})
5154(on Unix and cygwin only):
5158@end example
5160To disable the compiler output cache just in the current Lisp process,
5161use (after loading ASDF but before using it):
5165@end example
5167Note that this does @emph{NOT} belong in a @file{.asd} file.
5168Please do not tamper with ASDF configuration from a @file{.asd} file,
5169and only do this from your personal configuration or build scripts.
5171@node Issues with using and extending ASDF to define systems, ASDF development FAQs, Issues with configuring ASDF, FAQ
5172@section Issues with using and extending ASDF to define systems
5175* How can I cater for unit-testing in my system?::
5176* How can I cater for documentation generation in my system?::
5177* How can I maintain non-Lisp (e.g. C) source files?::
5178* I want to put my module's files at the top level.  How do I do this?::
5179* How do I create a system definition where all the source files have a .cl extension?::
5180* How do I mark a source file to be loaded only and not compiled?::
5181* How do I work with readtables?::
5182@end menu
5184@node How can I cater for unit-testing in my system?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems, Issues with using and extending ASDF to define systems
5185@subsection ``How can I cater for unit-testing in my system?''
5187ASDF provides a predefined test operation, @code{test-op}.
5188@xref{Predefined operations of ASDF, test-op}.
5189The test operation, however, is largely left to the system definer to specify.
5190@code{test-op} has been
5191a topic of considerable discussion on the
5192@url{,asdf-devel mailing list},
5193and on the
5194@url{,launchpad bug-tracker}.
5195We provide some guidelines in the discussion of @code{test-op}.
5197@c cut the following because it's discussed in the discussion of test-op.
5198@c Here are some guidelines:
5200@c @itemize
5201@c @item
5202@c For a given system, @var{foo}, you will want to define a corresponding
5203@c test system, such as @var{foo-test}.  The reason that you will want this
5204@c separate system is that ASDF does not out of the box supply components
5205@c that are conditionally loaded.  So if you want to have source files
5206@c (with the test definitions) that will not be loaded except when testing,
5207@c they should be put elsewhere.
5209@c @item
5210@c The @var{foo-test} system can be defined in an asd file of its own or
5211@c together with @var{foo}.  An aesthetic preference against cluttering up
5212@c the filesystem with extra asd files should be balanced against the
5213@c question of whether one might want to directly load @var{foo-test}.
5214@c Typically one would not want to do this except in early stages of
5215@c debugging.
5217@c @item
5218@c Record that testing is implemented by @var{foo-test}.  For example:
5219@c @example
5220@c (defsystem @var{foo}
5221@c    :in-order-to ((test-op (test-op @var{foo-test})))
5222@c    ....)
5224@c (defsystem @var{foo-test}
5225@c    :depends-on (@var{foo} @var{my-test-library} ...)
5226@c    ....)
5227@c @end example
5228@c @end itemize
5230@c This procedure will allow you to support users who do not wish to
5231@c install your test framework.
5233@c One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
5234@c does not return a value.  So in current versions of ASDF there is no
5235@c reliable programmatic means of determining whether or not a set of tests
5236@c has passed, or which tests have failed.  The user must simply read the
5237@c console output.  This limitation has been the subject of much
5238@c discussion.
5240@node How can I cater for documentation generation in my system?, How can I maintain non-Lisp (e.g. C) source files?, How can I cater for unit-testing in my system?, Issues with using and extending ASDF to define systems
5241@subsection ``How can I cater for documentation generation in my system?''
5243Various ASDF extensions provide some kind of @code{doc-op} operation.
5244See also @url{}.
5247@node How can I maintain non-Lisp (e.g. C) source files?, I want to put my module's files at the top level.  How do I do this?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems
5248@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
5250See @code{cffi}'s @code{cffi-grovel}.
5255@node I want to put my module's files at the top level.  How do I do this?, How do I create a system definition where all the source files have a .cl extension?, How can I maintain non-Lisp (e.g. C) source files?, Issues with using and extending ASDF to define systems
5256@subsection ``I want to put my module's files at the top level.  How do I do this?''
5258By default, the files contained in an asdf module go
5259in a subdirectory with the same name as the module.
5260However, this can be overridden by adding a @code{:pathname ""} argument
5261to the module description.
5262For example, here is how it could be done
5263in the spatial-trees ASDF system definition for ASDF 2:
5266(asdf:defsystem :spatial-trees
5267  :components
5268  ((:module base
5269            :pathname ""
5270            :components
5271            ((:file "package")
5272             (:file "basedefs" :depends-on ("package"))
5273             (:file "rectangles" :depends-on ("package"))))
5274   (:module tree-impls
5275            :depends-on (base)
5276            :pathname ""
5277            :components
5278            ((:file "r-trees")
5279             (:file "greene-trees" :depends-on ("r-trees"))
5280             (:file "rstar-trees" :depends-on ("r-trees"))
5281             (:file "rplus-trees" :depends-on ("r-trees"))
5282             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
5283   (:module viz
5284            :depends-on (base)
5285            :pathname ""
5286            :components
5287            ((:static-file "spatial-tree-viz.lisp")))
5288   (:module tests
5289            :depends-on (base)
5290            :pathname ""
5291            :components
5292            ((:static-file "spatial-tree-test.lisp")))
5293   (:static-file "LICENCE")
5294   (:static-file "TODO")))
5295@end example
5297All of the files in the @code{tree-impls} module are at the top level,
5298instead of in a @file{tree-impls/} subdirectory.
5300Note that the argument to @code{:pathname} can be either a pathname object or a string.
5301A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
5302but this is discouraged because the results of parsing a namestring are not portable.
5303A pathname can only be portably constructed with such syntax as
5304@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
5305and similarly the current directory can only be portably specified as
5306@code{#.(make-pathname :directory '(:relative))}.
5307However, as of ASDF 2, you can portably use a string to denote a pathname.
5308The string will be parsed as a @code{/}-separated path from the current directory,
5309such that the empty string @code{""} denotes the current directory, and
5310@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
5311portably denotes the same subdirectory as above.
5312When files are specified, the last @code{/}-separated component is interpreted
5313either as the name component of a pathname
5314(if the component class specifies a pathname type),
5315or as a name component plus optional dot-separated type component
5316(if the component class doesn't specifies a pathname type).
5318@node How do I create a system definition where all the source files have a .cl extension?, How do I mark a source file to be loaded only and not compiled?, I want to put my module's files at the top level.  How do I do this?, Issues with using and extending ASDF to define systems
5319@subsection How do I create a system definition where all the source files have a .cl extension?
5321Starting with ASDF 2.014.14, you may just pass
5322the builtin class @code{} as
5323the @code{:default-component-class} argument to @code{defsystem}:
5326(defsystem my-cl-system
5327  :default-component-class
5328  ...)
5329@end lisp
5331Another builtin class @code{cl-source-file.lsp} is offered
5332for files ending in @file{.lsp}.
5334If you want to use a different extension
5335for which ASDF doesn't provide builtin support,
5336or want to support versions of ASDF
5337earlier than 2.014.14 (but later than 2.000),
5338you can define a class as follows:
5341;; Prologue: make sure we're using a sane package.
5342(defpackage :my-asdf-extension
5343   (:use :asdf :common-lisp)
5344   (:export #:cl-source-file.lis))
5345(in-package :my-asdf-extension)
5347(defclass cl-source-file.lis (cl-source-file)
5348  ((type :initform "lis")))
5349@end lisp
5351Then you can use it as follows:
5353(defsystem my-cl-system
5354  :default-component-class my-asdf-extension:cl-source-file.lis
5355  ...)
5356@end lisp
5358Of course, if you're in the same package, e.g. in the same file,
5359you won't need to use the package qualifier before @code{cl-source-file.lis}.
5360Actually, if all you're doing is defining this class
5361and using it in the same file without other fancy definitions,
5362you might skip package complications:
5365(in-package :asdf)
5366(defclass cl-source-file.lis (cl-source-file)
5367   ((type :initform "lis")))
5368(defsystem my-cl-system
5369  :default-component-class cl-source-file.lis
5370  ...)
5371@end lisp
5373It is possible to achieve the same effect
5374in a way that supports both ASDF 1 and ASDF 2,
5375but really, friends don't let friends use ASDF 1.
5376Please upgrade to ASDF 3.
5377In short, though: do same as above, but
5378@emph{before} you use the class in a @code{defsystem},
5379you also define the following method:
5382(defmethod source-file-type ((f cl-source-file.lis) (s system))
5383  (declare (ignorable f s))
5384  "lis")
5385@end lisp
5387@node How do I mark a source file to be loaded only and not compiled?, How do I work with readtables?, How do I create a system definition where all the source files have a .cl extension?, Issues with using and extending ASDF to define systems
5388@subsection How do I mark a source file to be loaded only and not compiled?
5390There is no provision in ASDF for ensuring that
5391some components are always loaded as source, while others are always
5393There is @code{load-source-op} (@pxref{Predefined operations of
5394ASDF,load-source-op}), but that is an operation to be applied to a
5395system as a whole, not to one or another specific source files.
5396While this idea often comes up in discussions,
5397it doesn't play well with either the linking model of ECL
5398or with various bundle operations.
5399In addition, the dependency model of ASDF would have to be modified incompatibly
5400to allow for such a trick.
5401@c If your code doesn't compile cleanly, fix it.
5402@c If compilation makes it slow, use @code{declaim} or @code{eval-when}
5403@c to adjust your compiler settings,
5404@c or eschew compilation by @code{eval}uating a quoted source form at load-time.
5406@node How do I work with readtables?,  , How do I mark a source file to be loaded only and not compiled?, Issues with using and extending ASDF to define systems
5407@subsection How do I work with readtables?
5409@cindex readtables
5411It is possible to configure the lisp syntax by modifying the currently-active readtable.
5412However, this same readtable is shared globally by all software being compiled by ASDF,
5413especially since @code{load} and @code{compile-file} both bind @var{*readtable*},
5414so that its value is the same across the build at the start of every file
5415(unless overridden by some @code{perform :around} method),
5416even if a file locally binds it to a different readtable during the build.
5418Therefore, the following hygiene restrictions apply. If you don't abide by these restrictions,
5419there will be situations where your output files will be corrupted during an incremental build.
5420We are not trying to prescribe new restrictions for the sake of good style:
5421these restrictions have always applied implicitly, and
5422we are simply describing what they have always been.
5425@item It is forbidden to modifying any standard character or standard macro dispatch defined in the CLHS.
5426@item No two dependencies may assign different meanings to the same non-standard character.
5427@item Using any non-standard character while expecting the implementation to treat some way
5428    counts as such an assignment of meaning.
5429@item libraries need to document these assignments of meaning to non-standard characters.
5430@item free software libraries will register these changes on:
5431        @url{}
5432@end itemize
5434If you want to use readtable modifications that cannot abide by those restrictions,
5435you @emph{must} create a different readtable object and set @var{*readtable*}
5436to temporarily bind it to your new readtable (which will be undone after processing the file).
5438For that, we recommend you use system @code{named-readtables}
5439to define or combine such readtables using @code{named-readtables:defreadtable}
5440and use them using @code{named-readtables:in-readtable}.
5441Equivalently, you can use system @code{cl-syntax},
5442that itself uses @code{named-readtables},
5443but may someday do more with, e.g. @var{*print-pprint-dispatch*}.
5445For even more advanced syntax modification beyond what a readtable can express,
5446you may consider either:
5448@item a @code{perform} method that compiles a constant file that contains a single form
5449  @code{#.*code-read-with-alternate-reader*} in an environment where this special variable
5450  was bound to the code read by your alternate reader, or
5451@item using the system @code{reader-interception}.
5452@end itemize
5454Beware that @c unless and until the @code{syntax-control} branch is merged,
5455it is unsafe to use ASDF from the REPL to compile or load systems
5456while the readtable isn't the shared readtable previously used to build software.
5457You @emph{must} manually undo any binding of @var{*readtable*} at the REPL
5458and restore its initial value whenever you call @code{operate}
5459(via e.g. @code{load-system}, @code{test-system} or @code{require})
5460from a REPL that is using a different readtable.
5462@subsubsection How should my system use a readtable exported by another system?
5464Use from the @code{named-readtables} system the macro @code{named-readtables:in-readtable}.
5466If the other system fails to use @code{named-readtables}, fix it and send a patch upstream.
5467In the day and age of Quicklisp and clbuild, there is little reason
5468to eschew using such an important library anymore.
5470@subsubsection How should my library make a readtable available to other systems?
5472Use from the @code{named-readtables} system the macro @code{named-readtables:defreadtable}.
5474@node ASDF development FAQs,  , Issues with using and extending ASDF to define systems, FAQ
5475@section ASDF development FAQs
5478* How do run the tests interactively in a REPL?::
5479@end menu
5481@node How do run the tests interactively in a REPL?,  , ASDF development FAQs, ASDF development FAQs
5482@subsection How do run the tests interactively in a REPL?
5484This not-so-frequently asked question is primarily for ASDF developers,
5485but those who encounter an unexpected error in some test may be
5486interested, too.
5488Here's the procedure for experimenting with tests in a REPL:
5490;; BEWARE! Some tests expect you to be in the .../asdf/test directory
5491;; If your REPL is not there yet, change your current directory:
5492;; under SLIME, you may: ,change-directory ~/common-lisp/asdf/test/
5493;; otherwise you may evaluate something like:
5494(require "asdf") (asdf:upgrade-asdf) ;load UIOP & update asdf.lisp
5495(uiop:chdir (asdf:system-relative-pathname :asdf "test/"))
5496(setf *default-pathname-defaults* (uiop:getcwd))
5498;; Load the test script support.
5499(load "script-support.lisp")
5501;; Initialize the script support.
5502;; This will also change your *package* to asdf-test.
5503;; NB: this function is also available from package cl-user,
5504;; and also available with the shorter name da in both packages.
5507;; In case you modified ASDF since you last tested it,
5508;; you need to update asdf.lisp itself by evaluating 'make' in a shell,
5509;; or (require "asdf") (asdf:load-system :asdf) in another CL REPL,
5510;; if not done in this REPL above.
5511;; *Then*, in this REPL, you need to evaluate:
5514;; Now, you may experiment with test code from a .script file.
5515;; See the instructions given at the end of your failing test
5516;; to identify which form is needed, e.g.
5518(asdf::with-asdf-cache () (load "test-utilities.script"))
5519@end example
5522@comment FIXME: Add a FAQ about how to use a new system class...
5524@comment  node-name,  next,  previous,  up
5525@node  Ongoing Work, Bibliography, FAQ, Top
5526@unnumbered Ongoing Work
5527For an active list of things to be done,
5528see the @file{TODO} file in the source repository.
5530Also, bugs are now tracked on launchpad:
5533@node Bibliography, Concept Index, Ongoing Work, Top
5534@unnumbered Bibliography
5537@item Francois-Rene Rideau:
5538  ``ASDF 3, or Why Lisp is Now an Acceptable Scripting Language'', 2014.
5539  This article describes the innovations in ASDF 3 and 3.1,
5540  as well as historical information on previous versions.
5541  @url{}
5542@item Alastair Bridgewater:
5543  ``Quick-build'' (private communication), 2012.
5544  @code{quick-build} is a simple and robust one file, one package build system,
5545  similar to @code{faslpath}, in 182 lines of code
5546  (117 of which are not blank, not comments, not docstrings).
5547  Unhappily, it remains unpublished and its IP status is unclear as of April 2014.
5548  @code{asdf/package-system} is mostly compatible with it,
5549  modulo a different setup for toplevel hierarchies.
5550@item Zach Beane:
5551  ``Quicklisp'', 2011.
5552  The Quicklisp blog and Xach's livejournal contain information on Quicklisp.
5553  @url{}
5554  @url{}
5555@item Francois-Rene Rideau and Robert Goldman:
5556  ``Evolving ASDF: More Cooperation, Less Coordination'', 2010.
5557  This article describes the main issues solved by ASDF 2.
5558  @url{}
5559  @url{}
5560@item Francois-Rene Rideau and Spencer Brody:
5561  ``XCVB: an eXtensible Component Verifier and Builder for Common Lisp'', 2009.
5562  This article describes XCVB, a proposed competitor for ASDF,
5563  many ideas of which have been incorporated into ASDF 2 and 3,
5564  though many other of which still haven't.
5565  @url{}
5566@item Peter von Etter:
5567  ``faslpath'', 2009.
5568  @code{faslpath} is similar to the latter @code{quick-build}
5569  and our letter @code{asdf/package-system} extension,
5570  except that it uses the dot @code{.} rather than the slash @code{/} as a separator.
5571  @url{}
5572@item Drew McDermott:
5573  ``A Framework for Maintaining the Coherence of a Running Lisp,''
5574  International Lisp Conference, 2005, available in pre-print form at
5575  @url{}
5576@item Dan Barlow: ``ASDF Manual'', 2004.
5577  Older versions of this document from the days of ASDF 1;
5578  they include ideas laid down by Dan Barlow,
5579  and comparisons with older defsystems (@code{mk-defsystem})
5580  and defsystem (@code{defsystem-4}, kmp's Memo 801).
5581@item Marco Antoniotti and Peter Van Eynde:
5582 ``@code{DEFSYSTEM}: A @code{make} for Common Lisp, A Thoughtful Re-Implementation of an Old Idea'', 2002.
5583  The @file{defsystem-4} proposal available in the CLOCC repository.
5584@item Mark Kantrovitz: ``Defsystem: A Portable Make Facility for Common Lisp'', 1990.
5585  The classic @file{mk-defsystem}, later variants of which
5586  are available in the CLOCC repository as @code{defsystem-3.x}.
5587@item Richard Elliot Robbins:
5588  ``BUILD: A Tool for Maintaining Consistency in Modular Systems'', MIT AI TR 874, 1985.
5589  @url{}
5590@item Kent M. Pitman (kmp): ``The Description of Large Systems'', MIT AI Memo 801, 1984.
5591  Available in updated-for-CL form on the web at
5592  @url{}
5593@item Dan Weinreb and David Moon:
5594  ``Lisp Machine Manual'', MIT, 1981.
5595  The famous CHINE NUAL describes one of the earliest variants of DEFSYSTEM.
5596  @url{}
5597@end itemize
5600@node Concept Index, Function and Class Index, Bibliography, Top
5601@unnumbered Concept Index
5603@printindex cp
5605@node Function and Class Index, Variable Index, Concept Index, Top
5606@unnumbered Function and Class Index
5608@printindex fn
5610@node Variable Index,  , Function and Class Index, Top
5611@unnumbered Variable Index
5613@printindex vr
Note: See TracBrowser for help on using the repository browser.