source: trunk/abcl/doc/asdf/asdf.texinfo @ 14661

Last change on this file since 14661 was 14661, checked in by Mark Evenson, 10 years ago

Update to ASDF

Seems to fix loading of Ironclad, and other Quicklisp failures.

File size: 201.3 KB
1\input texinfo          @c -*- texinfo -*-
2@c %**start of header
4@settitle ASDF Manual
5@c %**end of header
7@c We use @&key, etc to escape & from TeX in lambda lists --
8@c so we need to define them for info as well.
9@macro AallowOtherKeys
11@end macro
12@macro Aoptional
14@end macro
15@macro Arest
17@end macro
18@macro Akey
20@end macro
21@macro Abody
23@end macro
25@c for install-info
26@dircategory Software development
28* asdf: (asdf).           Another System Definition Facility (for Common Lisp)
29@end direntry
32This manual describes ASDF, a system definition facility
33for Common Lisp programs and libraries.
35You can find the latest version of this manual at
38ASDF Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
40This manual Copyright @copyright{} 2001-2014 Daniel Barlow and contributors.
42This manual revised @copyright{} 2009-2014 Robert P. Goldman and Francois-Rene Rideau.
44Permission is hereby granted, free of charge, to any person obtaining
45a copy of this software and associated documentation files (the
46``Software''), to deal in the Software without restriction, including
47without limitation the rights to use, copy, modify, merge, publish,
48distribute, sublicense, and/or sell copies of the Software, and to
49permit persons to whom the Software is furnished to do so, subject to
50the following conditions:
52The above copyright notice and this permission notice shall be
53included in all copies or substantial portions of the Software.
63@end copying
68@title ASDF: Another System Definition Facility
70@c The following two commands start the copyright page.
72@vskip 0pt plus 1filll
74@end titlepage
76@c Output the table of contents at the beginning.
79@c -------------------
83@node Top, Introduction, (dir), (dir)
84@top ASDF: Another System Definition Facility
89* Introduction::
90* Quick start summary::
91* Loading ASDF::
92* Configuring ASDF::
93* Using ASDF::
94* Defining systems with defsystem::
95* The object model of ASDF::
96* Controlling where ASDF searches for systems::
97* Controlling where ASDF saves compiled files::
98* Error handling::
99* Miscellaneous additional functionality::
100* Getting the latest version::
101* FAQ::
102* Ongoing Work::
103* Bibliography::
104* Concept Index::
105* Function and Class Index::
106* Variable Index::              @c @detailmenu
110 --- The Detailed Node Listing ---
112Loading ASDF
114* Loading a pre-installed ASDF::
115* Checking whether ASDF is loaded::
116* Upgrading ASDF::
117* Loading an otherwise installed ASDF::
119Configuring ASDF
121* Configuring ASDF to find your systems::
122* Configuring where ASDF stores object files::
124Defining systems with defsystem
126* The defsystem form::
127* A more involved example::
128* The defsystem grammar::
129* Other code in .asd files::
130* The package-system extension::
132The object model of ASDF
134* Operations::
135* Components::
136* Functions::
140* Predefined operations of ASDF::
141* Creating new operations::
145* Common attributes of components::
146* Pre-defined subclasses of component::
147* Creating new component types::
151* Pre-defined subclasses of component::
152* Creating new component types::
156* Where do I report a bug?::
157* What has changed between ASDF 1 and ASDF 2?::
158* Issues with installing the proper version of ASDF::
159* Issues with configuring ASDF::
160* Issues with using and extending ASDF to define systems::
161* ASDF development FAQs::
163``What has changed between ASDF 1 and ASDF 2?''
165* What are ASDF 1 2 3?::
166* ASDF can portably name files in subdirectories::
167* Output translations::
168* Source Registry Configuration::
169* Usual operations are made easier to the user::
170* Many bugs have been fixed::
171* ASDF itself is versioned::
172* ASDF can be upgraded::
173* Decoupled release cycle::
174* Pitfalls of the transition to ASDF 2::
176Issues with installing the proper version of ASDF
178* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
179* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
181Issues with configuring ASDF
183* How can I customize where fasl files are stored?::
184* How can I wholly disable the compiler output cache?::
186Issues with using and extending ASDF to define systems
188* How can I cater for unit-testing in my system?::
189* How can I cater for documentation generation in my system?::
190* How can I maintain non-Lisp (e.g. C) source files?::
191* I want to put my module's files at the top level.  How do I do this?::
192* How do I create a system definition where all the source files have a .cl extension?::
194ASDF development FAQs
196* How do run the tests interactively in a REPL?::
198@end detailmenu
199@end menu
201@end ifnottex
203@c -------------------
205@node Introduction, Quick start summary, Top, Top
206@comment  node-name,  next,  previous,  up
207@chapter Introduction
208@cindex ASDF-related features
209@vindex *features*
210@cindex Testing for ASDF
211@cindex ASDF versions
212@cindex :asdf
213@cindex :asdf2
214@cindex :asdf3
216ASDF is Another System Definition Facility:
217a tool for specifying how systems of Common Lisp software
218are comprised of components (sub-systems and files),
219and how to operate on these components in the right order
220so that they can be compiled, loaded, tested, etc.
222ASDF presents three faces:
223one for users of Common Lisp software who want to reuse other people's code,
224one for writers of Common Lisp software who want to specify how to build their systems,
225one for implementers of Common Lisp extensions who want to extend the build system.
226@xref{Using ASDF,,Loading a system},
227to learn how to use ASDF to load a system.
228@xref{Defining systems with defsystem},
229to learn how to define a system of your own.
230@xref{The object model of ASDF}, for a description of
231the ASDF internals and how to extend ASDF.
233@emph{Nota Bene}:
234We have released ASDF 2.000 on May 31st 2010,
235and ASDF 3.0.0 on May 15th 2013.
236Releases of ASDF 2 and later have since then been included
237in all actively maintained CL implementations that used to bundle ASDF 1,
238plus some implementations that didn't use to,
239and has been made to work with all actively used CL implementations and a few more.
240@xref{FAQ,,``What has changed between ASDF 1 and ASDF 2?''}.
241Furthermore, it is possible to upgrade from ASDF 1 to ASDF 2 or ASDF 3 on the fly
242(though we recommend instead upgrading your implementation or its ASDF module).
243For this reason, we have stopped supporting ASDF 1 and ASDF 2.
244If you are using ASDF 1 or ASDF 2 and are experiencing any kind of issues or limitations,
245we recommend you upgrade to ASDF 3
246--- and we explain how to do that. @xref{Loading ASDF}.
247(In the context of compatibility requirements,
248ASDF 2.27, released on Feb 1st 2013, and further 2.x releases up to 2.33,
249count as pre-releases of ASDF 3, and define the @code{:asdf3} feature;
250still, please use the latest release).
251Release ASDF 3.1.1 and later also define the @code{:asdf3.1} feature.
253Also note that ASDF is not to be confused with ASDF-Install.
254ASDF-Install is not part of ASDF, but a separate piece of software.
255ASDF-Install is also unmaintained and obsolete.
256We recommend you use Quicklisp instead,
257which works great and is being actively maintained.
258If you want to download software from version control instead of tarballs,
259so you may more easily modify it, we recommend clbuild.
260We recommend @file{~/common-lisp/}
261as a place into which to install Common Lisp software;
262starting with ASDF 3.1.1, it is included in the default source-registry configuration.
264@node  Quick start summary, Loading ASDF, Introduction, Top
265@chapter Quick start summary
269@item To load an ASDF system:
273Load ASDF itself into your Lisp image, either through
274@code{(require "asdf")} (if it's supplied by your lisp implementation)
275or else through
276@code{(load "/path/to/asdf.lisp")}.  For more details, @ref{Loading ASDF}.
279Make sure ASDF can find system definitions
280through proper source-registry configuration.
281For more details, @xref{Configuring ASDF to find your systems}.
282The simplest way is simply to put all your lisp code in subdirectories of
283@file{~/common-lisp/} (starting with ASDF 3.1.1),
284or @file{~/.local/share/common-lisp/source/}
285(for ASDF 2 and later, or if you want to keep source in a hidden directory).
286Such code will automatically be found.
289Load a system with @code{(asdf:load-system :system)}. @xref{Using ASDF}.
291@end itemize
293@item To make your own ASDF system:
297As above, load and configure ASDF.
300Make a new directory for your system, @code{my-system/} in a location
301where ASDF can find it (@pxref{Configuring ASDF to find your systems}).
304Create an ASDF system definition listing the dependencies of
305your system, its components, and their interdependencies,
306and put it in @file{my-system.asd}.
307This file must have the same name as your system.
308@xref{Defining systems with defsystem}.
311Use @code{(asdf:load-system :my-system)}
312to make sure it's all working properly. @xref{Using ASDF}.
314@end itemize
315@end itemize
317@c FIXME: (1) add a sample project that the user can cut and paste to
318@c get started.  (2) discuss the option of starting with Quicklisp.
324@node Loading ASDF, Configuring ASDF, Quick start summary, Top
325@comment  node-name,  next,  previous,  up
326@chapter Loading ASDF
329* Loading a pre-installed ASDF::
330* Checking whether ASDF is loaded::
331* Upgrading ASDF::
332* Loading an otherwise installed ASDF::
333@end menu
335@node  Loading a pre-installed ASDF, Checking whether ASDF is loaded, Loading ASDF, Loading ASDF
336@section Loading a pre-installed ASDF
338Most recent Lisp implementations include a copy of ASDF 3,
339or at least ASDF 2.
340You can usually load this copy using Common Lisp's @code{require} function.@footnote{
341NB: all implementations except GNU CLISP also accept
342@code{(require "ASDF")}, @code{(require 'asdf)} and @code{(require :asdf)}.
343For portability's sake, you should use @code{(require "asdf")}.
347(require "asdf")
348@end lisp
350As of the writing of this manual,
351the following implementations provide ASDF 3 this way:
352ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, mkcl, SBCL.
353The following implementations only provide ASDF 2:
354LispWorks, mocl, XCL.
355The following implementation doesn't provide ASDF yet but will in an upcoming release:
357The following implementations are obsolete, not actively maintained,
358and most probably will never bundle ASDF:
359Corman CL, GCL, Genera, MCL.
361If the implementation you are using doesn't provide ASDF 2 or ASDF 3,
362see @pxref{Loading ASDF,,Loading an otherwise installed ASDF} below.
363If that implementation is still actively maintained,
364you may also send a bug report to your Lisp vendor and complain
365about their failing to provide ASDF.
367@node Checking whether ASDF is loaded, Upgrading ASDF, Loading a pre-installed ASDF, Loading ASDF
368@section Checking whether ASDF is loaded
370To check whether ASDF is properly loaded in your current Lisp image,
371you can run this form:
375@end lisp
377If it returns a string,
378that is the version of ASDF that is currently installed.
380If it raises an error,
381then either ASDF is not loaded, or
382you are using an old version of ASDF.
384You can check whether an old version is loaded
385by checking if the ASDF package is present.
386The form below will allow you to programmatically determine
387whether a recent version is loaded, an old version is loaded,
388or none at all:
391(when (find-package :asdf)
392  (let ((ver (symbol-value (or (find-symbol (string :*asdf-version*) :asdf)
393                               (find-symbol (string :*asdf-revision*) :asdf)))))
394    (etypecase ver
395      (string ver)
396      (cons (with-output-to-string (s)
397              (loop for (n . m) on ver do (princ n s) (when m (princ "." s)))))
398      (null "1.0"))))
399@end lisp
401If it returns @code{nil} then ASDF is not installed.
402Otherwise it should return a string.
403If it returns @code{"1.0"}, then it can actually be
404any version before 1.77 or so, or some buggy variant of 1.x.
406If you are experiencing problems with ASDF,
407please try upgrading to the latest released version,
408using the method below,
409before you contact us and raise an issue.
411@node  Upgrading ASDF, Loading an otherwise installed ASDF, Checking whether ASDF is loaded, Loading ASDF
412@section Upgrading ASDF
413@c FIXME: tighten this up a bit -- there's a lot of stuff here that
414@c doesn't matter to almost anyone.  Move discussion of updating antique
415@c versions of ASDF down, or encapsulate it.
417If you want to upgrade to a more recent ASDF version,
418you need to install and configure your ASDF just like any other system
419(@pxref{Configuring ASDF to find your systems}).
421If your implementation provides ASDF 3 or later,
422you only need to @code{(require "asdf")}:
423ASDF will automatically look whether an updated version of itself is available
424amongst the regularly configured systems, before it compiles anything else.
426@subsection Notes about ASDF 1 and ASDF 2
428Most implementations provide ASDF 3 in their latest release,
429and we recommend upgrading your implementation rather than using ASDF 1 or 2.
430A few implementations only provide ASDF 2,
431that can be overridden by installing a fasl (see below).
432A few implementations don't provide ASDF,
433but a fasl can be installed to provide it.
434GCL so far is still lacking a usable @code{require} interface.
436If your implementation has no suitable upgrade available (yet),
437or somehow upgrading is not an option,
438we assume you're an expert deliberately using a legacy implementation,
439and are proficient enough to install this fasl.
440Still, the ASDF source repository contains a script
441@file{bin/install-asdf-as-module} that can help you do that.
442It relies on @file{cl-launch} 4 for command-line invocation,
443which may depend on ASDF being checked out in @file{~/cl/asdf/}
444if your implementation doesn't even have an ASDF 2;
445but if you don't have @file{cl-launch},
446you can instead @code{(load "bin/install-asdf-as-module")}
447from your implementation's REPL.
449Finally, if your implementation only provides ASDF 2,
450and you can't or won't upgrade it or override its ASDF module,
451you may simply configure ASDF to find a proper upgrade;
452however, to avoid issues with a self-upgrade in mid-build,
453you @emph{must} make sure to upgrade ASDF immediately
454after requiring the builtin ASDF 2:
457(require "asdf")
458;; <--- insert programmatic configuration here if needed
459(asdf:load-system :asdf)
460@end lisp
462@subsection Issues with upgrading ASDF
464Note that there are some limitations to upgrading ASDF:
467Previously loaded ASDF extensions become invalid, and will need to be reloaded.
468Examples include CFFI-Grovel, hacks used by ironclad, etc.
469Since it isn't possible to automatically detect what extensions
470need to be invalidated,
471ASDF will invalidate @emph{all} previously loaded systems
472when it is loaded on top of a forward-incompatible ASDF version.
474@vindex *oldest-forward-compatible-asdf-version*
475Forward incompatibility can be determined using the variable
477which is 2.33 at the time of this writing.}
479Starting with ASDF 3 (2.27 or later),
480this self-upgrade will be automatically attempted as the first step
481to any system operation, to avoid any possibility of
482a catastrophic attempt to self-upgrade in mid-build.
484@c FIXME: Fix grammar below.
486For this and many other reasons,
487you should load, configure and upgrade ASDF
488as one of the very first things done by your build and startup scripts.
489It is safer if you upgrade ASDF and its extensions as a special step
490at the very beginning of whatever script you are running,
491before you start using ASDF to load anything else.
494Until all implementations provide ASDF 3 or later,
495it is unsafe to upgrade ASDF as part of loading a system
496that depends on a more recent version of ASDF,
497since the new one might shadow the old one while the old one is running,
498and the running old one will be confused
499when extensions are loaded into the new one.
500In the meantime, we recommend that your systems should @emph{not} specify
501@code{:depends-on (:asdf)}, or @code{:depends-on ((:version :asdf "3.0.1"))},
502but instead that they check that a recent enough ASDF is installed,
503with such code as:
505(unless (or #+asdf2 (asdf:version-satisfies
506                     (asdf:asdf-version) *required-asdf-version*))
507  (error "FOO requires ASDF ~A or later." *required-asdf-version*))
508@end example
510Until all implementations provide ASDF 3 or later,
511it is unsafe for a system to transitively depend on ASDF
512and not directly depend on ASDF;
513if any of the system you use either depends-on asdf,
514system-depends-on asdf, or transitively does,
515you should also do as well.
516@end itemize
518@node Loading an otherwise installed ASDF,  , Upgrading ASDF, Loading ASDF
519@section Loading an otherwise installed ASDF
521If your implementation doesn't include ASDF,
522if for some reason the upgrade somehow fails,
523does not or cannot apply to your case,
524you will have to install the file @file{asdf.lisp}
525somewhere and load it with:
528(load "/path/to/your/installed/asdf.lisp")
529@end lisp
531The single file @file{asdf.lisp} is all you normally need to use ASDF.
533You can extract this file from latest release tarball on the
534@url{,ASDF website}.
535If you are daring and willing to report bugs, you can get
536the latest and greatest version of ASDF from its git repository.
537@xref{Getting the latest version}.
539For maximum convenience you might want to have ASDF loaded
540whenever you start your Lisp implementation,
541for example by loading it from the startup script or dumping a custom core
542--- check your Lisp implementation's manual for details.
545@node Configuring ASDF, Using ASDF, Loading ASDF, Top
546@comment  node-name,  next,  previous,  up
547@chapter Configuring ASDF
549For standard use cases, ASDF should work pretty much out of the box.
550We recommend you skim the sections on configuring ASDF to find your systems
551and choose the method of installing Lisp software that works best for you.
552Then skip directly to @xref{Using ASDF}. That will probably be enough.
553You are unlikely to have to worry about the way ASDF stores object files,
554and resetting the ASDF configuration is usually only needed in corner cases.
558* Configuring ASDF to find your systems::
559* Configuring ASDF to find your systems --- old style::
560* Configuring where ASDF stores object files::
561* Resetting the ASDF configuration::
562@end menu
564@node Configuring ASDF to find your systems, Configuring ASDF to find your systems --- old style, Configuring ASDF, Configuring ASDF
565@section Configuring ASDF to find your systems
567In order to compile and load your systems, ASDF must be configured to find
568the @file{.asd} files that contain system definitions.
570There are a number of different techniques for setting yourself up with
571ASDF, starting from easiest to the most complex:
573@itemize @bullet
576Put all of your systems in subdirectories of
577@file{~/common-lisp/} or
579If you install software there, you don't need further configuration.
580(NB: @file{~/common-lisp/} is only included in the default configuration
581starting with ASDF 3.1.1 or later)
584If you're using some tool to install software (e.g. Quicklisp),
585the authors of that tool should already have configured ASDF.
588If you have more specific desires about how to lay out your software on
589disk, the preferred way to configure where ASDF finds your systems is
590the @code{source-registry} facility,
591fully described in its own chapter of this manual.
592@xref{Controlling where ASDF searches for systems}.  Here is a quick
593recipe for getting started:
595The simplest way to add a path to your search path,
596say @file{/home/luser/.asd-link-farm/}
597is to create the directory
599and there create a file with any name of your choice,
600and with the type @file{conf}@footnote{By requiring the @file{.conf}
601extension, and ignoring other files, ASDF allows you to have disabled files,
602editor backups, etc. in the same directory with your active
603configuration files.
605ASDF will also ignore files whose names start with a @file{.} character.
607It is customary to start the filename with two digits, to control the
608sorting of the @code{conf} files in the source registry directory, and
609thus the order in which the directories will be scanned.},
610for instance @file{42-asd-link-farm.conf},
611containing the line:
613@kbd{(:directory "/home/luser/.asd-link-farm/")}
615If you want all the subdirectories under @file{/home/luser/lisp/}
616to be recursively scanned for @file{.asd} files, instead use:
618@kbd{(:tree "/home/luser/lisp/")}
620ASDF will automatically read your configuration
621the first time you try to find a system.
622If necessary, you can reset the source-registry configuration with:
626@end lisp
629In earlier versions of ASDF, the system source registry was configured
630using a global variable, @code{asdf:*central-registry*}.
631For more details about this, see the following section,
632@ref{Configuring ASDF to find your systems --- old style}.
633Unless you need to understand this,
634skip directly to @ref{Configuring where ASDF stores object files}.
636@end itemize
638Note that your Operating System distribution or your system administrator
639may already have configured system-managed libraries for you.
643@node Configuring ASDF to find your systems --- old style, Configuring where ASDF stores object files, Configuring ASDF to find your systems, Configuring ASDF
645@c FIXME: this section should be moved elsewhere.  The novice user
646@c should not be burdened with it. [2014/02/27:rpg]
647@section Configuring ASDF to find your systems --- old style
649The old way to configure ASDF to find your systems is by
650@code{push}ing directory pathnames onto the variable
653You must configure this variable between the time you load ASDF
654and the time you first try to use it.
655Loading and configuring ASDF presumably happen
656as part of some initialization script that builds or starts
657your Common Lisp software system.
658(For instance, some SBCL users used to put it in their @file{~/.sbclrc}.)
660The @code{asdf:*central-registry*} is empty by default in ASDF 2 or ASDF 3,
661but is still supported for compatibility with ASDF 1.
662When used, it takes precedence over the above source-registry.@footnote{
663It is possible to further customize
664the system definition file search.
665That's considered advanced use, and covered later:
666search forward for
668@xref{Defining systems with defsystem}.}
670For example, let's say you want ASDF to find the @file{.asd} file
672In your lisp initialization file, you could have the following:
675(require "asdf")
676(push "/home/me/src/foo/" asdf:*central-registry*)
677@end lisp
679Note the trailing slash: when searching for a system,
680ASDF will evaluate each entry of the central registry
681and coerce the result to a pathname.@footnote{
682ASDF will indeed call @code{eval} on each entry.
683It will skip entries that evaluate to @code{nil}.
685Strings and pathname objects are self-evaluating,
686in which case the @code{eval} step does nothing;
687but you may push arbitrary s-expressions onto the central registry.
688These s-expressions may be evaluated to compute context-dependent
689entries, e.g. things that depend
690on the value of shell variables or the identity of the user.
692The variable @code{asdf:*central-registry*} is thus a list of
693``system directory designators''.
694A @dfn{system directory designator} is a form
695which will be evaluated whenever a system is to be found,
696and must evaluate to a directory to look in (or @code{NIL}).
697By ``directory'', we mean
698``designator for a pathname with a non-empty DIRECTORY component''.
700The trailing directory name separator
701is necessary to tell Lisp that you're discussing a directory
702rather than a file.  If you leave it out, ASDF is likely to look in
703@code{/home/me/src/} instead of @code{/home/me/src/foo/} as you
704intended, and fail to find your system definition.
706Typically there are a lot of @file{.asd} files, and
707a common idiom was to put
708@emph{symbolic links} to all of one's @file{.asd} files
709in a common directory
710and push @emph{that} directory (the ``link farm'')
713instead of pushing each individual system directory.
715ASDF knows to follow @emph{symlinks}
716to the actual location of the systems.@footnote{
717On Windows, you can use Windows shortcuts instead of POSIX symlinks.
718if you try aliases under MacOS, we are curious to hear about your experience.}
720For example, if @code{#p"/home/me/cl/systems/"}
721is an element of @code{*central-registry*}, you could set up the
722system @var{foo} as follows:
725$ cd /home/me/cl/systems/
726$ ln -s ~/src/foo/foo.asd .
727@end example
729This old style for configuring ASDF is not recommended for new users,
730but it is supported for old users, and for users who want to programmatically
731control what directories are added to the ASDF search path.
734@node Configuring where ASDF stores object files, Resetting the ASDF configuration, Configuring ASDF to find your systems --- old style, Configuring ASDF
735@section Configuring where ASDF stores object files
736@findex clear-output-translations
738ASDF lets you configure where object files will be stored.
739Sensible defaults are provided and
740you shouldn't normally have to worry about it.
742This allows the same source code repository to be shared
743between several versions of several Common Lisp implementations,
744between several users using different compilation options,
745with users who lack write privileges on shared source directories, etc.
746This also keeps source directories from being cluttered
747with object/fasl files.
749Starting with ASDF 2, the @code{asdf-output-translations} facility
750was added to ASDF itself.  This facility controls where object files will be stored.
751This facility is fully described in a chapter of this manual,
752@ref{Controlling where ASDF saves compiled files}.
754@c FIXME: possibly this should be moved elsewhere.  It's redundant here,
755@c and makes this section of the manual too long and daunting for the
756@c new user. [2014/02/27:rpg]
757@c The simplest way to add a translation to your search path,
758@c say from @file{/foo/bar/baz/quux/}
759@c to @file{/where/i/want/my/fasls/}
760@c is to create the directory
761@c @file{~/.config/common-lisp/asdf-output-translations.conf.d/}
762@c and there create a file with any name of your choice and the type @file{conf},
763@c for instance @file{42-bazquux.conf}
764@c containing the line:
766@c @kbd{("/foo/bar/baz/quux/" "/where/i/want/my/fasls/")}
768@c To disable output translations for source under a given directory,
769@c say @file{/toto/tata/}
770@c you can create a file @file{40-disable-toto.conf}
771@c with the line:
773@c @kbd{("/toto/tata/")}
775@c To wholly disable output translations for all directories,
776@c you can create a file @file{00-disable.conf}
777@c with the line:
779@c @kbd{(t t)}
781@c Note that your Operating System distribution or your system administrator
782@c may already have configured translations for you.
783@c In absence of any configuration, the default is to redirect everything
784@c under an implementation-dependent subdirectory of @file{~/.cache/common-lisp/}.
785@c @xref{Controlling where ASDF searches for systems}, for full details.
787@c The required @file{.conf} extension allows you to have disabled files
788@c or editor backups (ending in @file{~}), and works portably
789@c (for instance, it is a pain to allow both empty and non-empty extension on CLISP).
790@c Excluded are files the name of which start with a @file{.} character.
791@c It is customary to start the filename with two digits
792@c that specify the order in which the directories will be scanned.
794@c ASDF will automatically read your configuration
795@c the first time you try to find a system.
796@c You can reset the source-registry configuration with:
798@c @lisp
799@c (asdf:clear-output-translations)
800@c @end lisp
802@c And you probably should do so before you dump your Lisp image,
803@c if the configuration may change
804@c between the machine where you save it at the time you save it
805@c and the machine you resume it at the time you resume it.
806@c (Once again, you should use @code{(asdf:clear-configuration)}
807@c before you dump your Lisp image, which includes the above.)
809Note that before ASDF 2,
810other ASDF add-ons offered the same functionality,
811each in subtly different and incompatible ways:
812ASDF-Binary-Locations, cl-launch, common-lisp-controller.
813ASDF-Binary-Locations is now not needed anymore and should not be used.
814cl-launch 3.000 and common-lisp-controller 7.2 have been updated
815to delegate object file placement to ASDF.
817@node Resetting the ASDF configuration,  , Configuring where ASDF stores object files, Configuring ASDF
818@c FIXME: this should probably be moved out of the "quickstart" part of
819@c the manual. [2014/02/27:rpg]
820@section Resetting the ASDF configuration
822When you dump and restore an image, or when you tweak your configuration,
823you may want to reset the ASDF configuration.
824For that you may use the following function:
826@defun clear-configuration
827   undoes any ASDF configuration,
828   regarding source-registry or output-translations.
829@end defun
831This function is pushed onto the @code{uiop:*image-dump-hook*} by default,
832which means that if you save an image using @code{uiop:dump-image},
833or via @code{asdf:image-op} and @code{asdf:program-op},
834it will be automatically called to clear your configuration.
835If for some reason you prefer to call your implementation's underlying functionality,
836be sure to call @code{clear-configuration} manually,
837or push it into your implementation's equivalent of @code{uiop:*image-dump-hook*},
838e.g. @code{sb-ext:*save-hooks*} on SBCL, or @code{ext:*before-save-initializations*}
839on CMUCL and SCL, etc.
841@node  Using ASDF, Defining systems with defsystem, Configuring ASDF, Top
842@chapter Using ASDF
844@section Loading a system
846The system @var{foo} is loaded (and compiled, if necessary)
847by evaluating the following Lisp form:
850(asdf:load-system :@var{foo})
851@end example
853On some implementations (namely recent versions of
854ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP,
855LispWorks, MKCL, SBCL and XCL),
856ASDF hooks into the @code{CL:REQUIRE} facility
857and you can just use:
860(require :@var{foo})
861@end example
863In older versions of ASDF, you needed to use
864@code{(asdf:oos 'asdf:load-op :@var{foo})}.
865If your ASDF is too old to provide @code{asdf:load-system} though
866we recommend that you upgrade to ASDF 3.
867@xref{Loading ASDF,,Loading an otherwise installed ASDF}.
869Note the name of a system is specified as a string or a symbol.
870If a symbol (including a keyword), its name is taken and lowercased.
871The name must be a suitable value for the @code{:name} initarg
872to @code{make-pathname} in whatever filesystem the system is to be
875The lower-casing-symbols behaviour is unconventional,
876but was selected after some consideration.
877The type of systems we want to support
878either have lowercase as customary case (Unix, Mac, Windows)
879or silently convert lowercase to uppercase (lpns).
880@c so this makes more sense than attempting to use @code{:case :common},
881@c which is reported not to work on some implementations
884@section Other Operations
886ASDF provides three commands for the most common system operations:
887@code{load-system}, @code{compile-system} or @code{test-system}.
888It also provides @code{require-system}, a version of @code{load-system}
889that skips trying to update systems that are already loaded.
891Because ASDF is an extensible system
892for defining @emph{operations} on @emph{components},
893it also provides a generic function @code{operate}
894(which is usually abbreviated by @code{oos}).
895You'll use @code{oos} whenever you want to do something beyond
896compiling, loading and testing.
898Output from ASDF and ASDF extensions are sent
899to the CL stream @code{*standard-output*},
900so rebinding that stream around calls to @code{asdf:operate}
901should redirect all output from ASDF operations.
903@c Reminder: before ASDF can operate on a system, however,
904@c it must be able to find and load that system's definition.
905@c @xref{Configuring ASDF,,Configuring ASDF to find your systems}.
907@c FIXME: the following is too complicated for here, especially since
908@c :force hasn't been defined yet.  Move it. [2014/02/27:rpg]
909@findex already-loaded-systems
911For advanced users, note that
912@code{require-system} calls @code{load-system}
913with keyword arguments @code{:force-not (already-loaded-systems)}.
914@code{already-loaded-systems} returns a list of the names of loaded systems.
915@code{load-system} applies @code{operate} with the operation from
916@code{*load-system-operation*}, which by default is @code{load-op},
917the system, and any provided keyword arguments.
920@section Moving on
922That's all you need to know to use ASDF to load systems written by others.
923The rest of this manual deals with writing system definitions
924for Common Lisp software you write yourself,
925including how to extend ASDF to define new operation and component types.
928@node Defining systems with defsystem, The object model of ASDF, Using ASDF, Top
929@comment  node-name,  next,  previous,  up
930@chapter Defining systems with defsystem
932This chapter describes how to use asdf to define systems and develop
937* The defsystem form::
938* A more involved example::
939* The defsystem grammar::
940* Other code in .asd files::
941* The package-system extension::
942@end menu
944@node  The defsystem form, A more involved example, Defining systems with defsystem, Defining systems with defsystem
945@comment  node-name,  next,  previous,  up
946@section The defsystem form
948Systems can be constructed programmatically
949by instantiating components using @code{make-instance}.
950Most of the time, however, it is much more practical to use
951a static @code{defsystem} form.
952This section begins with an example of a system definition,
953then gives the full grammar of @code{defsystem}.
955Let's look at a simple system.
956This is a complete file that would
957usually be saved as @file{hello-lisp.asd}:
960(in-package :asdf)
962(defsystem "hello-lisp"
963  :description "hello-lisp: a sample Lisp system."
964  :version "0.2.1"
965  :author "Joe User <>"
966  :licence "Public Domain"
967  :components ((:file "packages")
968               (:file "macros" :depends-on ("packages"))
969               (:file "hello" :depends-on ("macros"))))
970@end lisp
972Some notes about this example:
977The file starts with an @code{in-package} form
978to use package @code{asdf}.
979You could instead start your definition by using
980a qualified name @code{asdf:defsystem}.
983If in addition to simply using @code{defsystem},
984you are going to define functions,
985create ASDF extension, globally bind symbols, etc.,
986it is recommended that to avoid namespace pollution between systems,
987you should create your own package for that purpose,
988for instance replacing the above @code{(in-package :asdf)} with:
991(defpackage :foo-system
992  (:use :cl :asdf))
994(in-package :foo-system)
995@end lisp
998The @code{defsystem} form defines a system named @code{hello-lisp}
999that contains three source files:
1000@file{packages}, @file{macros} and @file{hello}.
1003The file @file{macros} depends on @file{packages}
1004(presumably because the package it's in is defined in @file{packages}),
1005and the file @file{hello} depends on @file{macros}
1006(and hence, transitively on @file{packages}).
1007This means that ASDF will compile and load @file{packages} and @file{macros}
1008before starting the compilation of file @file{hello}.
1011The files are located in the same directory
1012as the file with the system definition.
1013ASDF resolves symbolic links (or Windows shortcuts)
1014before loading the system definition file and
1015stores its location in the resulting system@footnote{
1016It is possible, though almost never necessary, to override this behaviour.}.
1017This is a good thing because the user can move the system sources
1018without having to edit the system definition.
1020@c FIXME: Should have cross-reference to "Version specifiers" in the
1021@c defsystem grammar, but the cross-referencing is so broken by
1022@c insufficient node breakdown that I have not put one in.
1024Make sure you know how the @code{:version} numbers will be parsed!
1025They are parsed as period-separated lists of integers.
1026I.e., in the example, @code{0.2.1} is to be interpreted,
1027roughly speaking, as @code{(0 2 1)}.
1028In particular, version @code{0.2.1}
1029is interpreted the same as @code{0.0002.1} and
1030is strictly version-less-than version @code{0.20.1},
1031even though the two are the same when interpreted as decimal fractions.
1032Instead of a string representing the version,
1033the @code{:version} argument can be an expression that is resolved to
1034such a string using the following trivial domain-specific language:
1035in addition to being a literal string, it can be an expression of the form
1036@code{(:read-file-form <pathname-or-string> :at <access-at-specifier>)},
1037which will be resolved by reading a form in the specified pathname
1038(read as a subpathname of the current system if relative or a unix-namestring).
1039You may use a @code{uiop:access-at} specifier
1040with the (optional) @code{:at} keyword,
1041by default the specifier is @code{0}, meaning the first form is returned.
1043@cindex :version
1045@end itemize
1047@node  A more involved example, The defsystem grammar, The defsystem form, Defining systems with defsystem
1048@comment  node-name,  next,  previous,  up
1049@section A more involved example
1051Let's illustrate some more involved uses of @code{defsystem} via a
1052slightly convoluted example:
1055(defsystem "foo"
1056  :version "1.0.0"
1057  :components ((:module "mod"
1058                            :components ((:file "bar")
1059                                                  (:file"baz")
1060                                                  (:file "quux"))
1061                            :perform (compile-op :after (op c)
1062                                                  (do-something c))
1063                            :explain (compile-op :after (op c)
1064                                            (explain-something c)))
1065                         (:file "blah")))
1066@end lisp
1068The @code{:module} component named @code{"mod"} is a collection of three files,
1069which will be located in a subdirectory of the main code directory named
1070@file{mod} (this location can be overridden; see the discussion of the
1071@code{:pathname} option in @ref{The defsystem grammar}).
1073The method-form tokens provide a shorthand for defining methods on
1074particular components.  This part
1077                :perform (compile-op :after (op c)
1078                          (do-something c))
1079                :explain (compile-op :after (op c)
1080                          (explain-something c))
1081@end lisp
1083has the effect of
1086(defmethod perform :after ((op compile-op) (c (eql ...)))
1087           (do-something c))
1088(defmethod explain :after ((op compile-op) (c (eql ...)))
1089           (explain-something c))
1090@end lisp
1092where @code{...} is the component in question.
1093In this case @code{...} would expand to something like
1096(find-component "foo" "mod")
1097@end lisp
1099For more details on the syntax of such forms, see @ref{The defsystem
1101For more details on what these methods do, @pxref{Operations} in
1102@ref{The object model of ASDF}.
1104@c The following plunge into the weeds is not appropriate in this
1105@c location. [2010/10/03:rpg]
1106@c note that although this also supports @code{:before} methods,
1107@c they may not do what you want them to ---
1108@c a @code{:before} method on perform @code{((op compile-op) (c (eql ...)))}
1109@c will run after all the dependencies and sub-components have been processed,
1110@c but before the component in question has been compiled.
1112@node  The defsystem grammar, Other code in .asd files, A more involved example, Defining systems with defsystem
1113@comment  node-name,  next,  previous,  up
1114@section The defsystem grammar
1116@c FIXME: @var typesetting not consistently used here.  We should either expand
1117@c its use to everywhere, or we should kill it everywhere.
1121system-definition := ( defsystem system-designator @var{system-option}* )
1123system-option := :defsystem-depends-on system-list
1124                 | :weakly-depends-on @var{system-list}
1125                 | :class class-name (see discussion below)
1126                 | module-option
1127                 | option
1129module-option := :components component-list
1130                 | :serial [ t | nil ]
1132option :=
1133        | :pathname pathname-specifier
1134        | :default-component-class class-name
1135        | :perform method-form
1136        | :explain method-form
1137        | :output-files method-form
1138        | :operation-done-p method-form
1139        | :if-feature feature-expression
1140        | :depends-on ( @var{dependency-def}* )
1141        | :in-order-to ( @var{dependency}+ )
1144system-list := ( @var{simple-component-name}* )
1146component-list := ( @var{component-def}* )
1148component-def  := ( component-type simple-component-name @var{option}* )
1150component-type := :module | :file | :static-file | other-component-type
1152other-component-type := symbol-by-name (@pxref{The defsystem grammar,,Component types})
1154# This is used in :depends-on, as opposed to ``dependency,''
1155# which is used in :in-order-to
1156dependency-def := simple-component-name
1157               | ( :feature @var{feature-expression} dependency-def )
1158               | ( :version simple-component-name version-specifier )
1159               | ( :require module-name )
1161# ``dependency'' is used in :in-order-to, as opposed to
1162# ``dependency-def''
1163dependency := (dependent-op @var{requirement}+)
1164requirement := (required-op @var{required-component}+)
1165dependent-op := operation-name
1166required-op := operation-name
1168simple-component-name := string
1169                      |  symbol
1171pathname-specifier := pathname | string | symbol
1173method-form := (operation-name qual lambda-list @Arest body)
1174qual := method qualifier
1176component-dep-fail-option := :fail | :try-next | :ignore
1178feature-expression := keyword | (:and @var{feature-expression}*)
1179                      | (:or @var{feature-expression}*) | (:not @var{feature-expression})
1180@end example
1183@subsection Component names
1185Component names (@code{simple-component-name})
1186may be either strings or symbols.
1188@subsection Component types
1190Component type names, even if expressed as keywords, will be looked up
1191by name in the current package and in the asdf package, if not found in
1192the current package.  So a component type @code{my-component-type}, in
1193the current package @code{my-system-asd} can be specified as
1194@code{:my-component-type}, or @code{my-component-type}.
1196@code{system} and its subclasses are @emph{not}
1197allowed as component types for such children components.
1199@subsection System class names
1201A system class name will be looked up
1202in the same way as a Component type (see above),
1203except that only @code{system} and its subclasses are allowed.
1204Typically, one will not need to specify a system
1205class name, unless using a non-standard system class defined in some
1206ASDF extension, typically loaded through @code{DEFSYSTEM-DEPENDS-ON},
1207see below.  For such class names in the ASDF package, we recommend that
1208the @code{:class} option be specified using a keyword symbol, such as
1212@end example
1214This practice will ensure that package name conflicts are avoided.
1215Otherwise, the symbol @code{MY-NEW-SYSTEM-SUBCLASS} will be read into
1216the current package @emph{before} it has been exported from the ASDF
1217extension loaded by @code{:defsystem-depends-on}, causing a name
1218conflict in the current package.
1220@subsection Defsystem depends on
1221@cindex :defsystem-depends-on
1223The @code{:defsystem-depends-on} option to @code{defsystem} allows the
1224programmer to specify another ASDF-defined system or set of systems that
1225must be loaded @emph{before} the system definition is processed.
1226Typically this is used to load an ASDF extension that is used in the
1227system definition.
1229@subsection Weakly depends on
1230@cindex :weakly-depends-on
1232We do @emph{NOT} recommend you use this feature.
1233If you are tempted to write a system @var{foo}
1234that weakly-depends-on a system @var{bar},
1235we recommend that you should instead
1236write system @var{foo} in a parametric way,
1237and offer some special variable and/or some hook to specialize its behavior;
1238then you should write a system @var{foo+bar}
1239that does the hooking of things together.
1241The (deprecated) @code{:weakly-depends-on} option to @code{defsystem}
1242allows the programmer to specify another ASDF-defined system or set of systems
1243that ASDF should @emph{try} to load,
1244but need not load in order to be successful.
1245Typically this is used if there are a number of systems
1246that, if present, could provide additional functionality,
1247but which are not necessary for basic function.
1249Currently, although it is specified to be an option only to @code{defsystem},
1250this option is accepted at any component, but it probably
1251only makes sense at the @code{defsystem} level.
1252Programmers are cautioned not
1253to use this component option except at the @code{defsystem} level, as
1254this anomalous behavior may be removed without warning.
1256@c Finally, you might look into the @code{asdf-system-connections} extension,
1257@c that will let you define additional code to be loaded
1258@c when two systems are simultaneously loaded.
1259@c It may or may not be considered good style, but at least it can be used
1260@c in a way that has deterministic behavior independent of load order,
1261@c unlike @code{weakly-depends-on}.
1264@subsection Pathname specifiers
1265@cindex pathname specifiers
1267A pathname specifier (@code{pathname-specifier})
1268may be a pathname, a string or a symbol.
1269When no pathname specifier is given for a component,
1270which is the usual case, the component name itself is used.
1272If a string is given, which is the usual case,
1273the string will be interpreted as a Unix-style pathname
1274where @code{/} characters will be interpreted as directory separators.
1275Usually, Unix-style relative pathnames are used
1276(i.e. not starting with @code{/}, as opposed to absolute pathnames);
1277they are relative to the path of the parent component.
1278Finally, depending on the @code{component-type},
1279the pathname may be interpreted as either a file or a directory,
1280and if it's a file,
1281a file type may be added corresponding to the @code{component-type},
1282or else it will be extracted from the string itself (if applicable).
1284For instance, the @code{component-type} @code{:module}
1285wants a directory pathname, and so a string @code{"foo/bar"}
1286will be interpreted as the pathname @file{#p"foo/bar/"}.
1287On the other hand, the @code{component-type} @code{:file}
1288wants a file of type @code{lisp}, and so a string @code{"foo/bar"}
1289will be interpreted as the pathname @file{#p"foo/bar.lisp"},
1290and a string @code{"foo/bar.quux"}
1291will be interpreted as the pathname @file{#p"foo/bar.quux.lisp"}.
1292Finally, the @code{component-type} @code{:static-file}
1293wants a file without specifying a type, and so a string @code{"foo/bar"}
1294will be interpreted as the pathname @file{#p"foo/bar"},
1295and a string @code{"foo/bar.quux"}
1296will be interpreted as the pathname @file{#p"foo/bar.quux"}.
1298ASDF does not interpret the string @code{".."} to designate the parent
1299directory.  This string will be passed through to the underlying
1300operating system for interpretation.  We @emph{believe} that this will
1301work on all platforms where ASDF is deployed, but do not guarantee this
1302behavior.  A pathname object with a relative directory component of
1303@code{:up} or @code{:back} is the only guaranteed way to specify a
1304parent directory.
1306If a symbol is given, it will be translated into a string,
1307and downcased in the process.
1308The downcasing of symbols is unconventional,
1309but was selected after some consideration.
1310Observations suggest that the type of systems we want to support
1311either have lowercase as customary case (Unix, Mac, windows)
1312or silently convert lowercase to uppercase (lpns),
1313so this makes more sense than attempting to use @code{:case :common}
1314as argument to @code{make-pathname},
1315which is reported not to work on some implementations.
1317Pathname objects may be given to override the path for a component.
1318Such objects are typically specified using reader macros such as @code{#p}
1319or @code{#.(make-pathname ...)}.
1320Note however, that @code{#p...} is
1321a shorthand for @code{#.(parse-namestring ...)}
1322and that the behavior of @code{parse-namestring} is completely non-portable,
1323unless you are using Common Lisp @code{logical-pathname}s,
1324which themselves involve other non-portable behavior
1325(@pxref{The defsystem grammar,,Using logical pathnames}, below).
1326Pathnames made with @code{#.(make-pathname ...)}
1327can usually be done more easily with the string syntax above.
1328The only case that you really need a pathname object is to override
1329the component-type default file type for a given component.
1330Therefore, pathname objects should only rarely be used.
1331Unhappily, ASDF 1 didn't properly support
1332parsing component names as strings specifying paths with directories,
1333and the cumbersome @code{#.(make-pathname ...)} syntax had to be used.
1334An alternative to @code{#.} read-time evaluation is to use
1335@code{(eval `(defsystem ... ,pathname ...))}.
1337Note that when specifying pathname objects,
1338ASDF does not do any special interpretation of the pathname
1339influenced by the component type, unlike the procedure for
1340pathname-specifying strings.
1341On the one hand, you have to be careful to provide a pathname that correctly
1342fulfills whatever constraints are required from that component type
1343(e.g. naming a directory or a file with appropriate type);
1344on the other hand, you can circumvent the file type that would otherwise
1345be forced upon you if you were specifying a string.
1347@subsection Version specifiers
1348@cindex version specifiers
1349@cindex :version
1351Version specifiers are strings to be parsed as period-separated lists of integers.
1352I.e., in the example, @code{"0.2.1"} is to be interpreted,
1353roughly speaking, as @code{(0 2 1)}.
1354In particular, version @code{"0.2.1"} is interpreted the same as @code{"0.0002.1"},
1355though the latter is not canonical and may lead to a warning being issued.
1356Also, @code{"1.3"} and @code{"1.4"} are both strictly @code{uiop:version<} to @code{"1.30"},
1357quite unlike what would have happened
1358had the version strings been interpreted as decimal fractions.
1360System definers are encouraged to use version identifiers of the form
1361@var{x}.@var{y}.@var{z} for
1362major version, minor version and patch level,
1363where significant API incompatibilities are signaled by an increased major number.
1365@xref{Common attributes of components}.
1367@subsection Require
1368@cindex :require dependencies
1370Use the implementation's own @code{require} to load the @var{module-name}.
1373@subsection Using logical pathnames
1374@cindex logical pathnames
1376We do not generally recommend the use of logical pathnames,
1377especially not so to newcomers to Common Lisp.
1378However, we do support the use of logical pathnames by old timers,
1379when such is their preference.
1381To use logical pathnames,
1382you will have to provide a pathname object as a @code{:pathname} specifier
1383to components that use it, using such syntax as
1386You only have to specify such logical pathname
1387for your system or some top-level component.
1388Sub-components' relative pathnames,
1389specified using the string syntax for names,
1390will be properly merged with the pathnames of their parents.
1391The specification of a logical pathname host however is @emph{not}
1392otherwise directly supported in the ASDF syntax
1393for pathname specifiers as strings.
1395The @code{asdf-output-translation} layer will
1396avoid trying to resolve and translate logical pathnames.
1397The advantage of this is that
1398you can define yourself what translations you want to use
1399with the logical pathname facility.
1400The disadvantage is that if you do not define such translations,
1401any system that uses logical pathnames will behave differently under
1402asdf-output-translations than other systems you use.
1404If you wish to use logical pathnames you will have to configure the
1405translations yourself before they may be used.
1406ASDF currently provides no specific support
1407for defining logical pathname translations.
1409Note that the reasons we do not recommend logical pathnames are that
1410(1) there is no portable way to set up logical pathnames before they are used,
1411(2) logical pathnames are limited to only portably use
1412a single character case, digits and hyphens.
1413While you can solve the first issue on your own,
1414describing how to do it on each of fifteen implementations supported by ASDF
1415is more than we can document.
1416As for the second issue, mind that the limitation is notably enforced on SBCL,
1417and that you therefore can't portably violate the limitations
1418but must instead define some encoding of your own and add individual mappings
1419to name physical pathnames that do not fit the restrictions.
1420This can notably be a problem when your Lisp files are part of a larger project
1421in which it is common to name files or directories in a way that
1422includes the version numbers of supported protocols,
1423or in which files are shared with software written
1424in different programming languages where conventions include the use of
1425underscores, dots or CamelCase in pathnames.
1428@subsection Serial dependencies
1429@cindex serial dependencies
1431If the @code{:serial t} option is specified for a module,
1432ASDF will add dependencies for each child component,
1433on all the children textually preceding it.
1434This is done as if by @code{:depends-on}.
1437:serial t
1438:components ((:file "a") (:file "b") (:file "c"))
1439@end lisp
1441is equivalent to
1444:components ((:file "a")
1445             (:file "b" :depends-on ("a"))
1446             (:file "c" :depends-on ("a" "b")))
1447@end lisp
1450@subsection Source location
1452The @code{:pathname} option is optional in all cases for systems
1453defined via @code{defsystem},
1454and in the usual case the user is recommended not to supply it.
1456Instead, ASDF follows a hairy set of rules that are designed so that
1460will load a system from disk
1461and have its pathname default to the right place.
1464This pathname information will not be overwritten with
1466(which could be somewhere else altogether)
1467if the user loads up the @file{.asd} file into his editor
1468and interactively re-evaluates that form.
1469@end enumerate
1471If a system is being loaded for the first time,
1472its top-level pathname will be set to:
1476The host/device/directory parts of @code{*load-truename*},
1477if it is bound.
1479@code{*default-pathname-defaults*}, otherwise.
1480@end itemize
1482If a system is being redefined, the top-level pathname will be
1486changed, if explicitly supplied or obtained from @code{*load-truename*}
1487(so that an updated source location is reflected in the system definition)
1490changed if it had previously been set from @code{*default-pathname-defaults*}
1493left as before, if it had previously been set from @code{*load-truename*}
1494and @code{*load-truename*} is currently unbound
1495(so that a developer can evaluate a @code{defsystem} form
1496from within an editor without clobbering its source location)
1497@end itemize
1499@subsection if-feature option
1500@cindex :if-feature component option
1501@anchor{if-feature-option}      @c redo if this ever becomes a node in
1502@c its own right...
1504This option allows you to specify a feature expression to be evaluated
1505as if by @code{#+} to conditionally include a component in your build.
1506If the expression is false, the component is dropped
1507as well as any dependency pointing to it.
1508As compared to using @code{#+} which is expanded at read-time,
1509this allows you to have an object in your component hierarchy
1510that can be used for manipulations beside building your project, and
1511that is accessible to outside code that wishes to reason about system
1514Programmers should be careful to consider @strong{when} the
1515@code{:if-feature} is evaluated.  Recall that ASDF first computes a
1516build plan, and then executes that plan.  ASDF will check to see whether
1517or not a feature is present @strong{at planning time}, not during the
1518build.  It follows that one cannot use @code{:if-feature} to check
1519features that are set during the course of the build.  It can only be
1520used to check the state of features before any build operations have
1521been performed.
1523This option was added in ASDF 3.  For more information,
1524@xref{required-features, Required features}.
1526@subsection if-component-dep-fails option
1527@cindex :if-component-dep-fails component option
1528This option was removed in ASDF 3.
1529Its semantics was limited in purpose and dubious to explain,
1530and its implementation was breaking a hole into the ASDF object model.
1531Please use the @code{if-feature} option instead.
1533@subsection feature requirement
1534This requirement was removed in ASDF 3.1.
1535It used to ensure that a chain of component dependencies will raise an error,
1536which in conjunction with if-component-dep-fails would offer
1537a roundabout way to express conditional compilation.
1540@node Other code in .asd files, The package-system extension, The defsystem grammar, Defining systems with defsystem
1541@section Other code in .asd files
1543Files containing @code{defsystem} forms
1544are regular Lisp files that are executed by @code{load}.
1545Consequently, you can put whatever Lisp code you like into these files.
1546However, it is recommended to keep such forms to a minimal,
1547and to instead define @code{defsystem} extensions
1548that you use with @code{:defsystem-depends-on}.
1550If however, you might insist on including code in the @code{.asd} file itself,
1551e.g., to examine and adjust the compile-time environment,
1552possibly adding appropriate features to @code{*features*}.
1553If so, here are some conventions we recommend you follow,
1554so that users can control certain details of execution
1555of the Lisp in @file{.asd} files:
1559Any informative output
1560(other than warnings and errors,
1561which are the condition system's to dispose of)
1562should be sent to the standard CL stream @code{*standard-output*},
1563so that users can easily control the disposition
1564of output from ASDF operations.
1565@end itemize
1568@node The package-system extension,  , Other code in .asd files, Defining systems with defsystem
1569@section The package-system extension
1571Starting with ASDF 3.0.3,
1572ASDF supports a one-package-per-file style of programming,
1573whereby each file is its own system,
1574and dependencies are deduced from the @code{defpackage} form.
1576In this style, packages referring to a same-named system (downcased);
1577and if a system is defined with @code{:class package-system},
1578then system names that start with that name
1579(using the slash @code{/} separator)
1580refer to files under the filesystem hierarchy where the system is defined.
1581For instance, if system @code{my-lib} is defined in
1582@file{/foo/bar/my-lib/my-lib.asd}, then system @code{my-lib/src/utility}
1583will be found in file @file{/foo/bar/my-lib/src/utility.lisp}.
1585This style was made popular by @code{faslpath} and @code{quick-build} before,
1586and at the cost of a stricter package discipline,
1587seems to make for more maintainable code.
1588It is used by ASDF itself (starting with ASDF 3) and by @code{lisp-interface-library}.
1590To use this style, choose a toplevel system name, e.g. @code{my-lib},
1591and create a file @file{my-lib.asd}
1592with the @code{:class :package-system} option in its @code{defsystem}.
1593For instance:
1595#-asdf (error "my-lib requires ASDF 3")
1596(defsystem my-lib
1597  :class :package-system
1598  :defsystem-depends-on (:asdf-package-system)
1599  :depends-on (:my-lib/src/all)
1600  :in-order-to ((test-op (load-op :my-lib/test/all)))
1601  :perform (test-op (o c) (symbol-call :my-lib/test/all :test-suite)))
1602(register-system-packages :closer-mop
1603 '(:c2mop :closer-common-lisp :c2cl :closer-common-lisp-user :c2cl-user))
1604@end example
1606In the code above, the
1607@code{:defsystem-depends-on (:asdf-package-system)} is
1608for compatibility with older versions of ASDF 3 (ASDF 2 not supported),
1609and requires the @code{asdf-package-system} library to be present
1610(it is implicitly provided by ASDF starting with ASDF 3.0.3).
1612The function @code{register-system-packages} has to be called to register
1613packages used or provided by your system and its components
1614where the name of the system that provides the package
1615is not the downcase of the package name.
1617File @file{my-lib/src/utility.lisp} might start with:
1620(defpackage :my-lib/src/utility
1621  (:use :closer-common-lisp :my-lib/src/macros :my-lib/src/variables)
1622  (:import-from :cl-ppcre #:register-groups-bind)
1623  (:export #:foo #:bar))
1624@end example
1626And from the @code{:use} and @code{:import-from} clauses,
1627ASDF could tell that you depend on systems @code{closer-mop} (registered above),
1628@code{cl-ppcre} (package and system names match), and
1629@code{my-lib/src/macros} and @code{my-lib/src/variables}
1630(package and system names match, and they will be looked up hierarchically).
1632The form @code{uiop:define-package} is supported as well as @code{defpackage},
1633and has many options that prove useful in this context,
1634such as @code{:use-reexport} and @code{:mix-reexport}
1635that allow for ``inheritance'' of symbols being exported.
1637@node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top
1638@comment  node-name,  next,  previous,  up
1639@chapter The object model of ASDF
1641ASDF is designed in an object-oriented way from the ground up.
1642Both a system's structure and the operations that can be performed on systems
1643follow a extensible protocol.
1645This allows the addition of behaviours:
1646for example, @code{cffi} adds support of special FFI description files
1647to interface with C libraries and of wrapper files to embed C code in Lisp;
1648@code{abcl-jar} supports creating Java JAR archives in ABCL;
1649and @code{poiu} supports for compiling code in parallel using background processes.
1651This chapter deals with @code{component}s and @code{operation}s.
1653A @code{component} represents an individual source file or a group of source files,
1654and the things that get transformed into.
1655A @code{system} is a component at the top level of the component hierarchy.
1656A @code{source-file} is a component representing a single source-file
1657and the successive output files into which it is transformed.
1658A @code{module} is an intermediate component itself grouping several other components,
1659themselves source-files or further modules.
1661An @code{Operation} represents a transformation that can be performed on a component,
1662turning them from source files to intermediate results to final outputs.
1664A pair of an @code{operation} and a @code{component} is called an @code{action}.
1665An @code{action} represents a particular build step to be @code{perform}ed,
1666after all its dependencies have been fulfilled.
1667In the ASDF model, actions depend on other actions.
1668The term @emph{action} itself was used by Kent Pitman in his old article,
1669but was only used by ASDF hackers starting with the ASDF 2;
1670but the concept is ubiquitous since the very beginning of ASDF 1,
1671though previously implicit.
1673Then, there are many @emph{functions} available
1674to users, extenders and implementers of ASDF
1675to use, define or implement the activities
1676that are part of building your software.
1677Though they manipulate @code{action}s,
1678most of these functions do not take as an argument
1679a reified pair (a @code{cons} cell) of an operation and a component;
1680instead, they usually take two separate arguments,
1681which allows to take advantage of the power CLOS-style multiple dispatch
1682for fun and profit.
1684There are many @emph{hooks} in which to add functionality,
1685by customizing the behavior of existing @emph{functions}.
1687Last but not least is the notion of @emph{dependency} between two actions.
1688The structure of dependencies between actions is
1689a directed @emph{dependency graph}.
1690ASDF is invoked by being told to @emph{operate}
1691with some @emph{operation} on some toplevel @emph{system};
1692it will then @emph{traverse} the graph and build a @emph{plan}
1693that follows its structure.
1694To be successfully buildable, this graph of actions but be acyclic.
1695If, as a user, extender or implementer of ASDF, you fail
1696to keep the dependency graph without cycles,
1697ASDF will fail loudly as it eventually finds one.
1698To clearly distinguish the direction of dependencies,
1699ASDF 3 uses the words @emph{requiring} and @emph{required}
1700as applied to an action depending on the other:
1701the requiring action @code{depends-on} the completion of all required actions
1702before it may itself be @code{perform}ed.
1704Using the @code{defsystem} syntax, users may easily express
1705direct dependencies along the graph of the object hierarchy:
1706between a component and its parent, its children, and its siblings.
1707By defining custom CLOS methods, you can express more elaborate dependencies as you wish.
1708Most common operations, such as @code{load-op}, @code{compile-op} or @code{load-source-op}
1709are automatically propagate ``downward'' the component hierarchy and are ``covariant'' with it:
1710to act the operation on the parent module, you must first act it on all the children components,
1711with the action on the parent being parent of the action on each child.
1712Other operations, such as @code{prepare-op} and @code{prepare-source-op}
1713(introduced in ASDF 3) are automatically propagated ``upward'' the component hierarchy
1714and are ``contravariant'' with it:
1715to perform the operation of preparing for compilation of a child component,
1716you must perform the operation of preparing for compilation of its parent component, and so on,
1717ensuring that all the parent's dependencies are (compiled and) loaded
1718before the child component may be compiled and loaded.
1719Yet other operations, such as @code{test-op} or @code{load-fasl-op}
1720remain at the system level, and are not propagated along the hierarchy,
1721but instead do something global on the system.
1724* Operations::
1725* Components::
1726* Functions::
1727@end menu
1729@node  Operations, Components, The object model of ASDF, The object model of ASDF
1730@comment  node-name,  next,  previous,  up
1731@section Operations
1732@cindex operation
1734An @dfn{operation} object of the appropriate type is instantiated
1735whenever the user wants to do something with a system like
1738@item compile all its files
1739@item load the files into a running lisp environment
1740@item copy its source files somewhere else
1741@end itemize
1743Operations can be invoked directly, or examined
1744to see what their effects would be without performing them.
1745There are a bunch of methods specialised on operation and component type
1746that actually do the grunt work.
1747Operations are invoked on systems via @code{operate} (@pxref{operate}).
1749ASDF contains a number of pre-defined @t{operation} classes for common,
1750and even fairly uncommon tasks that you might want to do with it.
1751In addition, ASDF contains ``abstract'' @t{operation} classes that
1752programmers can use as building blocks to define ASDF extensions.  We
1753discuss these in turn below.
1755@c The operation object contains whatever state is relevant for this purpose
1756@c (perhaps a list of visited nodes, for example)
1757@c but primarily is a nice thing to specialise operation methods on
1758@c and easier than having them all be @code{EQL} methods.
1761* Predefined operations of ASDF::
1762* Creating new operations::
1763@end menu
1765Operations are invoked on systems via @code{operate}.
1767@deffn {Generic function} @code{operate} @var{operation} @var{system} @Arest @var{initargs} @Akey @code{force} @code{force-not} @code{verbose} @AallowOtherKeys
1768@deffnx {Generic function} @code{oos} @var{operation} @var{system} @Arest @var{initargs} @Akey @AallowOtherKeys
1769@code{operate} invokes @var{operation} on @var{system}.
1770@code{oos} is a synonym for @code{operate}.
1772@var{operation} is a symbol that is passed, along with the supplied
1773@var{initargs}, to @code{make-instance} to create the operation object.
1774@var{system} is a system designator.
1776The @var{initargs} are passed to the @code{make-instance} call
1777when creating the operation object.
1778Note that dependencies may cause the operation
1779to invoke other operations on the system or its components:
1780the new operations will be created
1781with the same @var{initargs} as the original one.
1783If @var{force} is @code{:all}, then all systems
1784are forced to be recompiled even if not modified since last compilation.
1785If @var{force} is @code{t}, then only the system being loaded
1786is forced to be recompiled even if not modified since last compilation,
1787but other systems are not affected.
1788If @var{force} is a list, then it specifies a list of systems that
1789are forced to be recompiled even if not modified since last compilation.
1790If @var{force-not} is @code{:all}, then all systems
1791are forced not to be recompiled even if modified since last compilation.
1792If @var{force-not} is @code{t}, then all systems but the system being loaded
1793are forced not to be recompiled even if modified since last compilation
1794(note: this was changed in ASDF 3.1.1).
1795If @var{force-not} is a list, then it specifies a list of systems that
1796are forced not to be recompiled even if modified since last compilation.
1798Both @var{force} and @var{force-not} apply to systems that are dependencies and were already compiled.
1799@var{force-not} takes precedences over @var{force},
1800as it should, really, but unhappily only since ASDF 3.1.1.
1801Moreover, systems the name of which is member of the set @var{*immutable-systems*}
1802(represented as an equal hash-table) are always considered @var{forced-not}, and
1803even their @file{.asd} is not refreshed from the filesystem.
1805To see what @code{operate} would do, you can use:
1807(asdf:traverse operation-class system-name)
1808@end example
1810@end deffn
1814@node Predefined operations of ASDF, Creating new operations, Operations, Operations
1815@comment  node-name,  next,  previous,  up
1816@subsection Predefined operations of ASDF
1818All the operations described in this section are in the @code{asdf} package.
1819They are invoked via the @code{operate} generic function.
1822(asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@})
1823@end lisp
1825@deffn Operation @code{compile-op}
1827This operation compiles the specified component.
1828A @code{cl-source-file} will be @code{compile-file}'d.
1829All the children and dependencies of a system or module
1830will be recursively compiled by @code{compile-op}.
1832@code{compile-op} depends on @code{prepare-op} which
1833itself depends on a @code{load-op} of all of a component's dependencies,
1834as well as of its parent's dependencies.
1835When @code{operate} is called on @code{compile-op},
1836all these dependencies will be loaded as well as compiled;
1837yet, some parts of the system main remain unloaded,
1838because nothing depends on them.
1839Use @code{load-op} to load a system.
1840@end deffn
1842@deffn Operation @code{load-op}
1844This operation loads the compiled code for a specified component.
1845A @code{cl-source-file} will have its compiled fasl @code{load}ed,
1846which fasl is the output of @code{compile-op} that @code{load-op} depends on.
1847All the children and dependencies of a system or module
1848will be recursively loaded by @code{load-op}.
1850@code{load-op} depends on @code{prepare-op} which
1851itself depends on a @code{load-op} of all of a component's dependencies,
1852as well as of its parent's dependencies.
1853@end deffn
1855@deffn Operation @code{prepare-op}
1857This operation ensures that the dependencies of a component
1858and its recursive parents are loaded (as per @code{load-op}),
1859as a prerequisite before @code{compile-op} and @code{load-op} operations
1860may be performed on a given component.
1861@end deffn
1863@deffn Operation @code{load-source-op}, @code{prepare-source-op}
1865@code{load-source-op} will load the source for the files in a module
1866rather than they compiled fasl output.
1867It has a @code{prepare-source-op} analog to @code{prepare-op},
1868that ensures the dependencies are themselves loaded via @code{load-source-op}.
1870There is no provision in ASDF for ensuring that
1871some components are always loaded as source, while others are always compiled.
1872While this idea often comes up in discussions,
1873it actually doesn't play well with either the linking model of ECL
1874or with various bundle operations (see below), and is eventually not workable;
1875also the dependency model of ASDF would have to be modified incompatibly
1876to allow for such trick.
1877If your code doesn't compile cleanly, fix it.
1878If compilation makes it slow, use @code{declaim} or @code{eval-when}
1879to adjust your compiler settings,
1880or eschew compilation by @code{eval}uating a quoted source form at load-time.
1881@end deffn
1884@deffn Operation @code{test-op}
1886This operation will perform some tests on the module.
1887The default method will do nothing.
1888The default dependency is to require
1889@code{load-op} to be performed on the module first.
1890The default @code{operation-done-p} is that the operation is @emph{never} done
1892we assume that if you invoke the @code{test-op},
1893you want to test the system, even if you have already done so.
1895The results of this operation are not defined by ASDF.
1896It has proven difficult to define how the test operation
1897should signal its results to the user
1898in a way that is compatible with all of the various test libraries
1899and test techniques in use in the community.
1901People typically define @code{test-op} methods like thus:
1903(defmethod perform ((o asdf:test-op)
1904                    (s (eql (asdf:find-system @var{:my-system}))))
1905  (asdf:load-system @var{:my-system-test})
1906  (funcall (read-from-string "my-system-test:test-suite")))
1907@end example
1909Using @code{load-system} in the perform method
1910rather than an @code{:in-order-to} dependency,
1911is sometimes necessary for backward compatibility with ASDF 2 and older,
1912to avoid circular dependencies that could arise
1913because of the way these old versions propagate dependencies.
1915If you don't care for compatibility with ASDF 2,
1916you could use the following options in your @code{defsystem} form:
1918  :in-order-to ((test-op (load-op :my-system-test)))
1919  :perform (test-op (o c) (symbol-call :my-system-test :test-suite))
1920@end example
1921@end deffn
1923@deffn Operation @code{fasl-op}, @code{monolithic-fasl-op}, @code{load-fasl-op}, @code{binary-op}, @code{monolithic-binary-op}, @code{lib-op}, @code{monolithic-lib-op}, @code{dll-op}, @code{monolithic-dll-op}, @code{program-op}
1925These are ``bundle'' operations, that can create a single-file ``bundle''
1926for all the contents of each system in an application,
1927or for the entire application.
1929@code{fasl-op} will create a single fasl file for each of the systems needed,
1930grouping all its many fasls in one,
1931so you can deliver each system as a single fasl.
1932@code{monolithic-fasl-op} will create a single fasl file for target system
1933and all its dependencies,
1934so you can deliver your entire application as a single fasl.
1935@code{load-fasl-op} will load the output of @code{fasl-op}
1936(though if it the output is not up-to-date,
1937it will load the intermediate fasls indeed as part of building it);
1938this matters a lot on ECL, where the dynamic linking involved in loading
1939tens of individual fasls can be noticeably more expensive
1940than loading a single one.
1942Once you have created a fasl with @code{fasl-op},
1943you can use @code{precompiled-system} to deliver it in a way
1944that is compatible with clients having dependencies on your system,
1945whether it is distributed as source or as a single binary;
1946the @file{.asd} file to be delivered with the fasl will look like this:
1948(defsystem :mysystem :class :precompiled-system
1949    :fasl (some expression that will evaluate to a pathname))
1950@end example
1951Or you can use @code{binary-op} to let ASDF create such a system for you
1952as well as the @code{fasl-op} output, or @code{monolithic-binary-op}.
1953This allows you to deliver code for your systems or applications
1954as a single file.
1955Of course, if you want to test the result in the current image,
1956@emph{before} you try to use any newly created @file{.asd} files,
1957you should not forget to @code{(asdf:clear-configuration)}
1958or at least @code{(asdf:clear-source-registry)},
1959so it re-populates the source-registry from the filesystem.
1961The @code{program-op} operation will create an executable program
1962from the specified system and its dependencies.
1963You can use UIOP for its pre-image-dump hooks, its post-image-restore hooks,
1964and its access to command-line arguments.
1965And you can specify an entry point @code{my-app:main}
1966by specifying in your @code{defsystem}
1967the option @code{:entry-point "my-app:main"}.
1968Depending on your implementation,
1969running @code{(asdf:operate 'asdf:program-op :my-app)}
1970may quit the current Lisp image upon completion.
1971See the example in
1972@file{test/hello-world-example.asd} and @file{test/hello.lisp},
1973as built and tested by
1974@file{test/test-program.script} and @file{test/make-hello-world.lisp}.
1976There is also @code{lib-op}
1977for building a linkable @file{.a} file (Windows: @file{.lib})
1978from all linkable object dependencies (FFI files, and on ECL, Lisp files too),
1979and its monolithic equivalent @code{monolithic-lib-op}.
1980And there is also @code{dll-op}
1981(respectively its monolithic equivalent @code{monolithic-lib-op})
1982for building a linkable @file{.so} file
1983(Windows: @file{.dll}, MacOS X: @file{.dynlib})
1984to create a single dynamic library
1985for all the extra FFI code to be linked into each of your systems
1986(respectively your entire application).
1988All these ``bundle'' operations are available since ASDF 3
1989on all actively supported Lisp implementations,
1990but may be unavailable on unmaintained legacy implementations.
1991This functionality was previously available for select implementations,
1992as part of a separate system @code{asdf-bundle},
1993itself descended from the ECL-only @code{asdf-ecl}.
1995The pathname of the output of bundle operations
1996is subject to output-translation as usual,
1997unless the operation is equal to
1998the @code{:build-operation} argument to @code{defsystem}.
1999This behavior is not very satisfactory and may change in the future.
2000Maybe you have suggestions on how to better configure it?
2001@end deffn
2003@deffn Operation @code{concatenate-source-op}, @code{monolithic-concatenate-source-op}, @code{load-concatenated-source-op}, @code{compile-concatenated-source-op}, @code{load-compiled-concatenated-source-op}, @code{monolithic-load-concatenated-source-op}, @code{monolithic-compile-concatenated-source-op}, @code{monolithic-load-compiled-concatenated-source-op}
2005These operation, as their respective names indicate,
2006consist in concatenating all @code{cl-source-file} source files in a system
2007(or in a system and all its dependencies, if monolithic),
2008in the order defined by dependencies,
2009then loading the result, or compiling then loading the result.
2011These operations are useful to deliver a system or application
2012as a single source file,
2013and for testing that said file loads properly, or compiles then loads properly.
2015ASDF itself is notably delivered as a single source file this way
2016using @code{monolithic-concatenate-source-op},
2017transcluding a prelude and the @code{uiop} library
2018before the @code{asdf/defsystem} system itself.
2019@end deffn
2022@node  Creating new operations,  , Predefined operations of ASDF, Operations
2023@comment  node-name,  next,  previous,  up
2024@subsection Creating new operations
2026ASDF was designed to be extensible in an object-oriented fashion.
2027To teach ASDF new tricks, a programmer can implement the behaviour he wants
2028by creating a subclass of @code{operation}.
2030ASDF's pre-defined operations are in no way ``privileged'',
2031but it is requested that developers never use the @code{asdf} package
2032for operations they develop themselves.
2033The rationale for this rule is that we don't want to establish a
2034``global asdf operation name registry'',
2035but also want to avoid name clashes.
2037Your operation @emph{must} usually provide methods
2038for one or more of the following generic functions:
2042@item @code{perform}
2043Unless your operation, like @code{prepare-op},
2044is for dependency propagation only,
2045the most important function for which to define a method
2046is usually @code{perform},
2047which will be called to perform the operation on a specified component,
2048after all dependencies have been performed.
2050The @code{perform} method must call @code{output-files} (see below)
2051to find out where to put its files,
2052because the user is allowed to override the method
2053or tweak the output-translation mechanism.
2054Perform should only use the primary value returned by @code{output-files}.
2055If one and only one output file is expected,
2056it can call @code{output-file} that checks that this is the case
2057and returns the first and only list element.
2059@item @code{output-files}
2060If your perform method has any output,
2061you must define a method for this function.
2062for ASDF to determine where the outputs of performing operation lie.
2064Your method may return two values, a list of pathnames, and a boolean.
2065If the boolean is @code{nil} (or you fail to return multiple values),
2066then enclosing @code{:around} methods may translate these pathnames,
2067e.g. to ensure object files are somehow stored
2068in some implementation-dependent cache.
2069If the boolean is @code{t} then the pathnames are marked
2070not be translated by the enclosing @code{:around} method.
2072@item @code{component-depends-on}
2073If the action of performing the operation on a component has dependencies,
2074you must define a method on @code{component-depends-on}.
2076Your method will take as specialized arguments
2077an operation and a component which together identify an action,
2078and return a list of entries describing actions that this action depends on.
2079The format of entries is described below.
2081It is @emph{strongly} advised that
2082you should always append the results of @code{(call-next-method)}
2083to the results of your method,
2084or ``interesting'' failures will likely occur,
2085unless you're a true specialist of ASDF internals.
2086It is unhappily too late to compatibly use the @code{append} method combination,
2087but conceptually that's the protocol that is being manually implemented.
2089Each entry returned by @code{component-depends-on} is itself a list.
2091The first element of an entry is an operation designator:
2092either an operation object designating itself, or
2093a symbol that names an operation class
2094(that ASDF will instantiate using @code{make-operation}).
2095For instance, @code{load-op}, @code{compile-op} and @code{prepare-op}
2096are common such names, denoting the respective operations.
2098The rest of each entry is a list of component designators:
2099either a component object designating itself,
2100or an identifier to be used with @code{find-component}.
2101@code{find-component} will be called with the current component's parent as parent,
2102and the identifier as second argument.
2103The identifier is typically a string,
2104a symbol (to be downcased as per @code{coerce-name}),
2105or a list of strings or symbols.
2106In particular, the empty list @code{nil} denotes the parent itself.
2108@end itemize
2110An operation @emph{may} provide methods for the following generic functions:
2114@item @code{input-files}
2115A method for this function is often not needed,
2116since ASDF has a pretty clever default @code{input-files} mechanism.
2117You only need create a method if there are multiple ultimate input files,
2118and/or the bottom one doesn't depend
2119on the @code{component-pathname} of the component.
2121@item @code{operation-done-p}
2122You only need to define a method on that function
2123if you can detect conditions that invalidate previous runs of the operation,
2124even though no filesystem timestamp has changed,
2125in which case you return @code{nil} (the default is @code{t}).
2127For instance, the method for @code{test-op} always returns @code{nil},
2128so that tests are always run afresh.
2129Of course, the @code{test-op} for your system could depend
2130on a deterministically repeatable @code{test-report-op},
2131and just read the results from the report files,
2132in which case you could have this method return @code{t}.
2134@end itemize
2136Operations that print output should send that output to the standard
2137CL stream @code{*standard-output*}, as the Lisp compiler and loader do.
2139@node Components, Functions, Operations, The object model of ASDF
2140@comment  node-name,  next,  previous,  up
2141@section Components
2142@cindex component
2143@cindex system
2144@cindex system designator
2145@cindex component designator
2146@vindex *system-definition-search-functions*
2148A @dfn{component} represents a source file or
2149(recursively) a collection of components.
2150A @dfn{system} is (roughly speaking) a top-level component
2151that can be found via @code{find-system}.
2153A @dfn{system designator} is a system itself,
2154or a string or symbol that behaves just like any other component name
2155(including with regard to the case conversion rules for component names).
2157A @dfn{component designator}, relative to a base component,
2158is either a component itself,
2159or a string or symbol,
2160or a list of designators.
2162@defun find-system system-designator @Aoptional (error-p t)
2164Given a system designator, @code{find-system} finds and returns a system.
2165If no system is found, an error of type
2166@code{missing-component} is thrown,
2167or @code{nil} is returned if @code{error-p} is false.
2169To find and update systems, @code{find-system} funcalls each element
2170in the @code{*system-definition-search-functions*} list,
2171expecting a pathname to be returned, or a system object,
2172from which a pathname may be extracted, and that will be registered.
2173The resulting pathname (if any) is loaded
2174if one of the following conditions is true:
2178there is no system of that name in memory
2180the pathname is different from that which was previously loaded
2182the file's @code{last-modified} time exceeds the @code{last-modified} time
2183of the system in memory
2184@end itemize
2186When system definitions are loaded from @file{.asd} files,
2187a new scratch package is created for them to load into,
2188so that different systems do not overwrite each others operations.
2189The user may also wish to (and is recommended to)
2190include @code{defpackage} and @code{in-package} forms
2191in his system definition files, however,
2192so that they can be loaded manually if need be.
2194The default value of @code{*system-definition-search-functions*}
2195is a list of two functions.
2196The first function looks in each of the directories given
2197by evaluating members of @code{*central-registry*}
2198for a file whose name is the name of the system and whose type is @file{asd}.
2199The first such file is returned,
2200whether or not it turns out to actually define the appropriate system.
2201The second function does something similar,
2202for the directories specified in the @code{source-registry}.
2203Hence, it is strongly advised to define a system
2204@var{foo} in the corresponding file @var{foo.asd}.
2205@end defun
2207@defun find-component base path
2209Given a @var{base} component (or designator for such),
2210and a @var{path}, find the component designated by the @var{path}
2211starting from the @var{base}.
2213If @var{path} is a component object, it designates itself,
2214independently from the base.
2216If @var{path} is a string, or symbol denoting a string via @code{coerce-name},
2217then @var{base} is resolved to a component object,
2218which must be a system or module,
2219and the designated component is the child named by the @var{path}.
2221If @var{path} is a @code{cons} cell,
2222@code{find-component} with the base and the @code{car} of the @var{path},
2223and the resulting object is used as the base for a tail call
2224to @code{find-component} with the @code{car} of the @var{path}.
2226If @var{base} is a component object, it designates itself.
2228If @var{base} is null, then @var{path} is used as the base, with @code{nil} as the path.
2230If @var{base} is a string, or symbol denoting a string via @code{coerce-name},
2231it designates a system as per @code{find-system}.
2233If @var{base} is a @code{cons} cell, it designates the component found by
2234@code{find-component} with its @code{car} as base and @code{cdr} as path.
2235@end defun
2239* Common attributes of components::
2240* Pre-defined subclasses of component::
2241* Creating new component types::
2242@end menu
2244@node  Common attributes of components, Pre-defined subclasses of component, Components, Components
2245@comment  node-name,  next,  previous,  up
2246@subsection Common attributes of components
2248All components, regardless of type, have the following attributes.
2249All attributes except @code{name} are optional.
2251@subsubsection Name
2253A component name is a string or a symbol.
2254If a symbol, its name is taken and lowercased.
2256Unless overridden by a @code{:pathname} attribute,
2257the name will be interpreted as a pathname specifier according
2258to a Unix-style syntax.
2259@xref{The defsystem grammar,,Pathname specifiers}.
2261@subsubsection Version identifier
2262@findex version-satisfies
2263@cindex :version
2265This optional attribute specifies a version for the current component.
2266The version should typically be a string of integers separated by dots,
2267for example @samp{1.0.11}.
2268For more information on version specifiers, see @ref{The defsystem grammar}.
2270A version may then be queried by the generic function @code{version-satisfies},
2271to see if @code{:version} dependencies are satisfied,
2272and when specifying dependencies, a constraint of minimal version to satisfy
2273can be specified using e.g. @code{(:version "mydepname" "1.0.11")}.
2275Note that in the wild, we typically see version numbering
2276only on components of type @code{system}.
2277Presumably it is much less useful within a given system,
2278wherein the library author is responsible to keep the various files in synch.
2280@subsubsection Required features
2283Traditionally defsystem users have used @code{#+} reader conditionals
2284to include or exclude specific per-implementation files.
2285For example, CFFI, the portable C foreign function interface contained
2286lines like:
2288     #+sbcl       (:file "cffi-sbcl")
2289@end lisp
2290An unfortunate side effect of this approach is that no single
2291implementation can read the entire system.
2292This causes problems if, for example, one wished to design an @code{archive-op}
2293that would create an archive file containing all the sources, since
2294for example the file @code{cffi-sbcl.lisp} above would be invisible when
2295running the @code{archive-op} on any implementation other than SBCL.
2297Starting with ASDF 3,
2298components may therefore have an @code{:if-feature} option.
2299The value of this option should be
2300a feature expression using the same syntax as @code{#+} does.
2301If that feature expression evaluates to false, any reference to the component will be ignored
2302during compilation, loading and/or linking.
2303Since the expression is read by the normal reader,
2304you must explicitly prefix your symbols with @code{:} so they be read as keywords;
2305this is as contrasted with the @code{#+} syntax
2306that implicitly reads symbols in the keyword package by default.
2308For instance, @code{:if-feature (:and :x86 (:or :sbcl :cmu :scl))} specifies that
2309the given component is only to be compiled and loaded
2310when the implementation is SBCL, CMUCL or Scieneer CL on an x86 machine.
2311You cannot write it as @code{:if-feature (and x86 (or sbcl cmu scl))}
2312since the symbols would not be read as keywords.
2316@subsubsection Dependencies
2318This attribute specifies dependencies of the component on its siblings.
2319It is optional but often necessary.
2321There is an excitingly complicated relationship between the initarg
2322and the method that you use to ask about dependencies
2324Dependencies are between (operation component) pairs.
2325In your initargs for the component, you can say
2328:in-order-to ((compile-op (load-op "a" "b") (compile-op "c"))
2329              (load-op (load-op "foo")))
2330@end lisp
2332This means the following things:
2335before performing compile-op on this component, we must perform
2336load-op on @var{a} and @var{b}, and compile-op on @var{c},
2338before performing @code{load-op}, we have to load @var{foo}
2339@end itemize
2341The syntax is approximately
2344(this-op @{(other-op required-components)@}+)
2346simple-component-name := string
2347                      |  symbol
2349required-components := simple-component-name
2350                     | (required-components required-components)
2352component-name := simple-component-name
2353                | (:version simple-component-name minimum-version-object)
2354@end verbatim
2356Side note:
2358This is on a par with what ACL defsystem does.
2359mk-defsystem is less general: it has an implied dependency
2362  for all source file x, (load x) depends on (compile x)
2363@end verbatim
2365and using a @code{:depends-on} argument to say that @var{b} depends on
2366@var{a} @emph{actually} means that
2369  (compile b) depends on (load a)
2370@end verbatim
2372This is insufficient for e.g. the McCLIM system, which requires that
2373all the files are loaded before any of them can be compiled ]
2375End side note
2377In ASDF, the dependency information for a given component and operation
2378can be queried using @code{(component-depends-on operation component)},
2379which returns a list
2382((load-op "a") (load-op "b") (compile-op "c") ...)
2383@end lisp
2385@code{component-depends-on} can be subclassed for more specific
2386component/operation types: these need to @code{(call-next-method)}
2387and append the answer to their dependency, unless
2388they have a good reason for completely overriding the default dependencies.
2390If it weren't for CLISP, we'd be using @code{LIST} method
2391combination to do this transparently.
2392But, we need to support CLISP.
2393If you have the time for some CLISP hacking,
2394I'm sure they'd welcome your fixes.
2395@c Doesn't CLISP now support LIST method combination?
2397A minimal version can be specified for a component you depend on
2398(typically another system), by specifying @code{(:version "other-system" "1.2.3")}
2399instead of simply @code{"other-system"} as the dependency.
2400See the discussion of the semantics of @code{:version}
2401in the defsystem grammar.
2403@c FIXME: Should have cross-reference to "Version specifiers" in the
2404@c defsystem grammar, but the cross-referencing is so broken by
2405@c insufficient node breakdown that I have not put one in.
2408@subsubsection pathname
2410This attribute is optional and if absent (which is the usual case),
2411the component name will be used.
2413@xref{The defsystem grammar,,Pathname specifiers},
2414for an explanation of how this attribute is interpreted.
2416Note that the @code{defsystem} macro (used to create a ``top-level'' system)
2417does additional processing to set the filesystem location of
2418the top component in that system.
2419This is detailed elsewhere. @xref{Defining systems with defsystem}.
2422@subsubsection properties
2424This attribute is optional.
2426Packaging systems often require information about files or systems
2427in addition to that specified by ASDF's pre-defined component attributes.
2428Programs that create vendor packages out of ASDF systems therefore
2429have to create ``placeholder'' information to satisfy these systems.
2430Sometimes the creator of an ASDF system may know the additional
2431information and wish to provide it directly.
2433@code{(component-property component property-name)} and
2434associated @code{setf} method will allow
2435the programmatic update of this information.
2436Property names are compared as if by @code{EQL},
2437so use symbols or keywords or something.
2440* Pre-defined subclasses of component::
2441* Creating new component types::
2442@end menu
2444@node Pre-defined subclasses of component, Creating new component types, Common attributes of components, Components
2445@comment  node-name,  next,  previous,  up
2446@subsection Pre-defined subclasses of component
2448@deffn Component source-file
2450A source file is any file that the system does not know how to
2451generate from other components of the system.
2453Note that this is not necessarily the same thing as
2454``a file containing data that is typically fed to a compiler''.
2455If a file is generated by some pre-processor stage
2456(e.g. a @file{.h} file from @file{} by autoconf)
2457then it is not, by this definition, a source file.
2458Conversely, we might have a graphic file
2459that cannot be automatically regenerated,
2460or a proprietary shared library that we received as a binary:
2461these do count as source files for our purposes.
2463Subclasses of source-file exist for various languages.
2464@emph{FIXME: describe these.}
2465@end deffn
2467@deffn Component module
2469A module is a collection of sub-components.
2471A module component has the following extra initargs:
2475@code{:components} the components contained in this module
2479All children components which don't specify their class explicitly
2480are inferred to be of this type.
2484This attribute was removed in ASDF 3. Do not use it.
2485Use @code{:if-feature} instead (@pxref{required-features}, and @pxref{if-feature-option}).
2488@code{:serial} When this attribute is set,
2489each subcomponent of this component is assumed to depend on all subcomponents
2490before it in the list given to @code{:components}, i.e.
2491all of them are loaded before a compile or load operation is performed on it.
2493@end itemize
2495The default operation knows how to traverse a module, so
2496most operations will not need to provide methods specialised on modules.
2498@code{module} may be subclassed to represent components such as
2499foreign-language linked libraries or archive files.
2500@end deffn
2502@deffn Component system
2504@code{system} is a subclass of @code{module}.
2506A system is a module with a few extra attributes for documentation
2507purposes; these are given elsewhere.
2508@xref{The defsystem grammar}.
2510Users can create new classes for their systems:
2511the default @code{defsystem} macro takes a @code{:class} keyword argument.
2512@end deffn
2514@node  Creating new component types,  , Pre-defined subclasses of component, Components
2515@comment  node-name,  next,  previous,  up
2516@subsection Creating new component types
2518New component types are defined by subclassing one of the existing
2519component classes and specializing methods on the new component class.
2521@emph{FIXME: this should perhaps be explained more throughly,
2522not only by example ...}
2524As an example, suppose we have some implementation-dependent
2525functionality that we want to isolate
2526in one subdirectory per Lisp implementation our system supports.
2527We create a subclass of
2531(defclass unportable-cl-source-file (cl-source-file)
2532  ())
2533@end lisp
2535Function @code{asdf:implementation-type} (exported since 2.014.14)
2536gives us the name of the subdirectory.
2537All that's left is to define how to calculate the pathname
2538of an @code{unportable-cl-source-file}.
2541(defmethod component-pathname ((component unportable-cl-source-file))
2542  (merge-pathnames*
2543   (parse-unix-namestring (format nil "~(~A~)/" (asdf:implementation-type)))
2544   (call-next-method)))
2545@end lisp
2547The new component type is used in a @code{defsystem} form in this way:
2550(defsystem :foo
2551    :components
2552    ((:file "packages")
2553     ...
2554     (:unportable-cl-source-file "threads"
2555      :depends-on ("packages" ...))
2556     ...
2557    )
2558@end lisp
2560@node Functions,  , Components, The object model of ASDF
2561@comment  node-name,  next,  previous,  up
2562@section Functions
2563@findex version-satisfies
2565@deffn version-satisfies @var{version} @var{version-spec}
2566Does @var{version} satisfy the @var{version-spec}.  A generic function.
2567ASDF provides built-in methods for @var{version} being a @code{component} or @code{string}.
2568@var{version-spec} should be a string.
2569If it's a component, its version is extracted as a string before further processing.
2571A version string satisfies the version-spec if after parsing,
2572the former is no older than the latter.
2573Therefore @code{"1.9.1"}, @code{"1.9.2"} and @code{"1.10"} all satisfy @code{"1.9.1"},
2574but @code{"1.8.4"} or @code{"1.9"} do not.
2575For more information about how @code{version-satisfies} parses and interprets
2576version strings and specifications,
2577@pxref{The defsystem grammar} (version specifiers) and
2578@ref{Common attributes of components}.
2580Note that in versions of ASDF prior to 3.0.1,
2581including the entire ASDF 1 and ASDF 2 series,
2582@code{version-satisfies} would also require that the version and the version-spec
2583have the same major version number (the first integer in the list);
2584if the major version differed, the version would be considered as not matching the spec.
2585But that feature was not documented, therefore presumably not relied upon,
2586whereas it was a nuisance to several users.
2587Starting with ASDF 3.0.1,
2588@code{version-satisfies} does not treat the major version number specially,
2589and returns T simply if the first argument designates a version that isn't older
2590than the one specified as a second argument.
2591If needs be, the @code{(:version ...)} syntax for specifying dependencies
2592could be in the future extended to specify an exclusive upper bound for compatible versions
2593as well as an inclusive lower bound.
2594@end deffn
2596@node Controlling where ASDF searches for systems, Controlling where ASDF saves compiled files, The object model of ASDF, Top
2597@comment  node-name,  next,  previous,  up
2598@chapter Controlling where ASDF searches for systems
2600@section Configurations
2602Configurations specify paths where to find system files.
2607The search registry may use some hardcoded wrapping registry specification.
2608This allows some implementations (notably SBCL) to specify where to find
2609some special implementation-provided systems that
2610need to precisely match the version of the implementation itself.
2613An application may explicitly initialize the source-registry configuration
2614using the configuration API
2615(@pxref{Controlling where ASDF searches for systems,Configuration API,Configuration API}, below)
2616in which case this takes precedence.
2617It may itself compute this configuration from the command-line,
2618from a script, from its own configuration file, etc.
2621The source registry will be configured from
2622the environment variable @code{CL_SOURCE_REGISTRY} if it exists.
2625The source registry will be configured from
2626user configuration file
2628(which defaults to
2630if it exists.
2633The source registry will be configured from
2634user configuration directory
2636(which defaults to
2638if it exists.
2641The source registry will be configured from
2642system configuration file
2644if it exists/
2647The source registry will be configured from
2648system configuration directory
2650if it exists.
2653The source registry will be configured from a default configuration.
2654This configuration may allow for implementation-specific systems
2655to be found, for systems to be found the current directory
2656(at the time that the configuration is initialized) as well as
2657@code{:directory} entries for @file{$XDG_DATA_DIRS/common-lisp/systems/} and
2658@code{:tree} entries for @file{$XDG_DATA_DIRS/common-lisp/source/}.
2659For instance, SBCL will include directories for its contribs
2660when it can find them; it will look for them where SBCL was installed,
2661or at the location specified by the @code{SBCL_HOME} environment variable.
2663@end enumerate
2665Each of these configurations is specified as an s-expression
2666in a trivial domain-specific language (defined below).
2667Additionally, a more shell-friendly syntax is available
2668for the environment variable (defined yet below).
2670Each of these configurations is only used if the previous
2671configuration explicitly or implicitly specifies that it
2672includes its inherited configuration.
2674Additionally, some implementation-specific directories
2675may be automatically prepended to whatever directories are specified
2676in configuration files, no matter if the last one inherits or not.
2679@section Truenames and other dangers
2681One great innovation of the original ASDF was its ability to leverage
2682@code{CL:TRUENAME} to locate where your source code was and where to build it,
2683allowing for symlink farms as a simple but effective configuration mechanism
2684that is easy to control programmatically.
2685ASDF 3 still supports this configuration style, and it is enabled by default;
2686however we recommend you instead use
2687our source-registry configuration mechanism described below,
2688because it is easier to setup in a portable way across users and implementations.
2690Addtionally, some people dislike truename,
2691either because it is very slow on their system, or
2692because they are using content-addressed storage where the truename of a file
2693is related to a digest of its individual contents,
2694and not to other files in the same intended project.
2695For these people, ASDF 3 allows to eschew the @code{TRUENAME} mechanism,
2696by setting the variable @var{asdf:*resolve-symlinks*} to @code{nil}.
2698PS: Yes, if you haven't read Vernor Vinge's short but great classic
2699``True Names... and Other Dangers'' then you're in for a treat.
2702@section XDG base directory
2704Note that we purport to respect the XDG base directory specification
2705as to where configuration files are located,
2706where data files are located,
2707where output file caches are located.
2708Mentions of XDG variables refer to that document.
2712This specification allows the user to specify some environment variables
2713to customize how applications behave to his preferences.
2715On Windows platforms, when not using Cygwin,
2716instead of the XDG base directory specification,
2717we try to use folder configuration from the registry regarding
2718@code{Common AppData} and similar directories.
2719Since support for querying the Windows registry
2720is not possible to do in reasonable amounts of portable Common Lisp code,
2721ASDF 3 relies on the environment variables that Windows usually exports.
2723@section Backward Compatibility
2725For backward compatibility as well as to provide a practical backdoor for hackers,
2726ASDF will first search for @code{.asd} files in the directories specified in
2728before it searches in the source registry above.
2730@xref{Configuring ASDF,,Configuring ASDF to find your systems --- old style}.
2732By default, @code{asdf:*central-registry*} will be empty.
2734This old mechanism will therefore not affect you if you don't use it,
2735but will take precedence over the new mechanism if you do use it.
2737@section Configuration DSL
2738@cindex :inherit-configuration source config directive
2739@cindex inherit-configuration source config directive
2740@cindex :ignore-invalid-entries source config directive
2741@cindex ignore-invalid-entries source config directive
2742@cindex :directory source config directive
2743@cindex directory source config directive
2744@cindex :tree source config directive
2745@cindex tree source config directive
2746@cindex :exclude source config directive
2747@cindex exclude source config directive
2748@cindex :also-exclude source config directive
2749@cindex also-exclude source config directive
2750@cindex :include source config directive
2751@cindex include source config directive
2752@cindex :default-registry source config directive
2753@cindex default-registry source config directive
2755Here is the grammar of the s-expression (SEXP) DSL for source-registry
2758@c FIXME: This is too wide for happy compilation into pdf.
2761;; A configuration is a single SEXP starting with keyword :source-registry
2762;; followed by a list of directives.
2763CONFIGURATION := (:source-registry DIRECTIVE ...)
2765;; A directive is one of the following:
2768    ;; Your configuration expression MUST contain
2769    ;; exactly one of either of these:
2770    :inherit-configuration | ; splices inherited configuration (often specified last)
2771    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
2773    ;; forward compatibility directive (since ASDF 2.011.4), useful when
2774    ;; you want to use new configuration features but have to bootstrap a
2775    ;; the newer required ASDF from an older release that doesn't sport said features:
2776    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
2778    ;; add a single directory to be scanned (no recursion)
2781    ;; add a directory hierarchy, recursing but excluding specified patterns
2784    ;; override the defaults for exclusion patterns
2785    (:exclude EXCLUSION-PATTERN ...) |
2786    ;; augment the defaults for exclusion patterns
2787    (:also-exclude EXCLUSION-PATTERN ...) |
2788    ;; Note that the scope of a an exclude pattern specification is
2789    ;; the rest of the current configuration expression or file.
2791    ;; splice the parsed contents of another config file
2794    ;; This directive specifies that some default must be spliced.
2795    :default-registry
2798DIRECTORY-PATHNAME-DESIGNATOR := PATHNAME-DESIGNATOR ;; interpreted as a directory name
2801    NIL | ;; Special: skip this entry.
2804EXCLUSION-PATTERN := a string without wildcards, that will be matched exactly
2805  against the name of a any subdirectory in the directory component
2806        of a path. e.g. @code{"_darcs"} will match @file{#p"/foo/bar/_darcs/src/bar.asd"}
2807@end example
2809Pathnames are designated using another DSL,
2810shared with the output-translations configuration DSL below.
2811The DSL is resolved by the function @code{asdf::resolve-location},
2812to be documented and exported at some point in the future.
2817    STRING | ;; namestring (better be absolute or bust, directory assumed where applicable).
2818             ;; In output-translations, directory is assumed and **/*.*.* added if it's last.
2819             ;; On MCL, a MacOSX-style POSIX namestring (for MacOS9 style, use #p"...");
2820             ;; Note that none of the above applies to strings used in *central-registry*,
2821             ;; which doesn't use this DSL: they are processed as normal namestrings.
2822             ;; however, you can compute what you put in the *central-registry*
2823             ;; based on the results of say (asdf::resolve-location "/Users/fare/cl/cl-foo/")
2824    PATHNAME | ;; pathname (better be an absolute path, or bust)
2825               ;; In output-translations, unless followed by relative components,
2826               ;; it better have appropriate wildcards, as in **/*.*.*
2827    :HOME | ;; designates the user-homedir-pathname ~/
2828    :USER-CACHE | ;; designates the default location for the user cache
2829    :HERE | ;; designates the location of the configuration file
2830            ;; (or *default-pathname-defaults*, if invoked interactively)
2831    :ROOT ;; magic, for output-translations source only: paths that are relative
2832          ;; to the root of the source host and device
2833    ;; Not valid anymore: :SYSTEM-CACHE (was a security hazard)
2837    STRING | ;; relative directory pathname as interpreted by parse-unix-namestring.
2838             ;; In output translations, if last component, **/*.*.* is added
2839    PATHNAME | ;; pathname; unless last component, directory is assumed.
2840    :IMPLEMENTATION | ;; directory based on implementation, e.g. sbcl-1.0.45-linux-x64
2841    :IMPLEMENTATION-TYPE | ;; a directory based on lisp-implementation-type only, e.g. sbcl
2842    :DEFAULT-DIRECTORY | ;; a relativized version of the default directory
2843    :*/ | ;; any direct subdirectory (since ASDF 2.011.4)
2844    :**/ | ;; any recursively inferior subdirectory (since ASDF 2.011.4)
2845    :*.*.* | ;; any file (since ASDF 2.011.4)
2846    ;; Not supported (anymore): :UID and :USERNAME
2847@end example
2849For instance, as a simple case, my @file{~/.config/common-lisp/source-registry.conf},
2850which is the default place ASDF looks for this configuration, once contained:
2853  (:tree (:home "cl")) ;; will expand to e.g. "/home/joeluser/cl/"
2854  :inherit-configuration)
2855@end example
2857@section Configuration Directories
2859Configuration directories consist in files each containing
2860a list of directives without any enclosing @code{(:source-registry ...)} form.
2861The files will be sorted by namestring as if by @code{string<} and
2862the lists of directives of these files with be concatenated in order.
2863An implicit @code{:inherit-configuration} will be included
2864at the @emph{end} of the list.
2866This allows for packaging software that has file granularity
2867(e.g. Debian's @code{dpkg} or some future version of @code{clbuild})
2868to easily include configuration information about distributed software.
2870The convention is that, for sorting purposes,
2871the names of files in such a directory begin with two digits
2872that determine the order in which these entries will be read.
2873Also, the type of these files is conventionally @code{"conf"}
2874and as a limitation to some implementations (e.g. GNU clisp),
2875the type cannot be @code{nil}.
2877Directories may be included by specifying a directory pathname
2878or namestring in an @code{:include} directive, e.g.:
2881  (:include "/foo/bar/")
2882@end example
2884Hence, to achieve the same effect as
2885my example @file{~/.config/common-lisp/source-registry.conf} above,
2886I could simply create a file
2888alone in its directory with the following contents:
2890(:tree "/home/fare/cl/")
2891@end example
2893@subsection The :here directive
2895The @code{:here} directive is an absolute pathname designator that
2896refers to the directory containing the configuration file currently
2897being processed.
2899The @code{:here} directive is intended to simplify the delivery of
2900complex CL systems, and for easy configuration of projects shared through
2901revision control systems, in accordance with our design principle that
2902each participant should be able to provide all and only the information
2903available to him or her.
2905Consider a person X who has set up the source code repository for a
2906complex project with a master directory @file{dir/}.  Ordinarily, one
2907might simply have the user add a directive that would look something
2908like this:
2910   (:tree  "path/to/dir")
2911@end example
2912But what if X knows that there are very large subtrees
2913under dir that are filled with, e.g., Java source code, image files for
2914icons, etc.?  All of the asdf system definitions are contained in the
2915subdirectories @file{dir/src/lisp/} and @file{dir/extlib/lisp/}, and
2916these are the only directories that should be searched.
2918In this case, X can put into @file{dir/} a file @file{asdf.conf} that
2919contains the following:
2922   (:tree (:here "src/lisp/"))
2923   (:tree (:here "extlib/lisp"))
2924   (:directory (:here "outlier/")))
2925@end example
2927Then when someone else (call her Y) checks out a copy of this
2928repository, she need only add
2930(:include "/path/to/my/checkout/directory/asdf.conf")
2931@end example
2932to one of her previously-existing asdf source location configuration
2933files, or invoke @code{initialize-source-registry} with a configuration
2934form containing that s-expression.  ASDF will find the .conf file that X
2935has provided, and then set up source locations within the working
2936directory according to X's (relative) instructions.
2938@section Shell-friendly syntax for configuration
2940When considering environment variable @code{CL_SOURCE_REGISTRY}
2941ASDF will skip to next configuration if it's an empty string.
2942It will @code{READ} the string as a SEXP in the DSL
2943if it begins with a paren @code{(}
2944and it will be interpreted much like @code{TEXINPUTS}
2945list of paths, where
2947  * paths are separated
2948   by a @code{:} (colon) on Unix platforms (including cygwin),
2949   by a @code{;} (semicolon) on other platforms (mainly, Windows).
2951  * each entry is a directory to add to the search path.
2953  * if the entry ends with a double slash @code{//}
2954    then it instead indicates a tree in the subdirectories
2955    of which to recurse.
2957  * if the entry is the empty string (which may only appear once),
2958    then it indicates that the inherited configuration should be
2959    spliced there.
2962@section Search Algorithm
2963@vindex *default-source-registry-exclusions*
2965In case that isn't clear, the semantics of the configuration is that
2966when searching for a system of a given name,
2967directives are processed in order.
2969When looking in a directory, if the system is found, the search succeeds,
2970otherwise it continues.
2972When looking in a tree, if one system is found, the search succeeds.
2973If multiple systems are found, the consequences are unspecified:
2974the search may succeed with any of the found systems,
2975or an error may be raised.
2976ASDF currently returns the first system found,
2977XCVB currently raised an error.
2978If none is found, the search continues.
2980Exclude statements specify patterns of subdirectories
2981the systems from which to ignore.
2982Typically you don't want to use copies of files kept by such
2983version control systems as Darcs.
2984Exclude statements are not propagated to further included or inherited
2985configuration files or expressions;
2986instead the defaults are reset around every configuration statement
2987to the default defaults from @code{asdf::*default-source-registry-exclusions*}.
2989Include statements cause the search to recurse with the path specifications
2990from the file specified.
2992An inherit-configuration statement cause the search to recurse with the path
2993specifications from the next configuration
2994(@pxref{Controlling where ASDF searches for systems,,Configurations} above).
2997@section Caching Results
2999The implementation is allowed to either eagerly compute the information
3000from the configurations and file system, or to lazily re-compute it
3001every time, or to cache any part of it as it goes.
3002To explicitly flush any information cached by the system, use the API below.
3005@section Configuration API
3007The specified functions are exported from your build system's package.
3008Thus for ASDF the corresponding functions are in package ASDF,
3009and for XCVB the corresponding functions are in package XCVB.
3011@defun initialize-source-registry @Aoptional PARAMETER
3012   will read the configuration and initialize all internal variables.
3013   You may extend or override configuration
3014   from the environment and configuration files
3015   with the given @var{PARAMETER}, which can be
3016   @code{nil} (no configuration override),
3017   or a SEXP (in the SEXP DSL),
3018   a string (as in the string DSL),
3019   a pathname (of a file or directory with configuration),
3020   or a symbol (fbound to function that when called returns one of the above).
3021@end defun
3023@defun clear-source-registry
3024   undoes any source registry configuration
3025   and clears any cache for the search algorithm.
3026   You might want to call this function
3027   (or better, @code{clear-configuration})
3028   before you dump an image that would be resumed
3029   with a different configuration,
3030   and return an empty configuration.
3031   Note that this does not include clearing information about
3032   systems defined in the current image, only about
3033   where to look for systems not yet defined.
3034@end defun
3036@defun ensure-source-registry @Aoptional PARAMETER
3037   checks whether a source registry has been initialized.
3038   If not, initialize it with the given @var{PARAMETER}.
3039@end defun
3041Every time you use ASDF's @code{find-system}, or
3042anything that uses it (such as @code{operate}, @code{load-system}, etc.),
3043@code{ensure-source-registry} is called with parameter @code{nil},
3044which the first time around causes your configuration to be read.
3045If you change a configuration file,
3046you need to explicitly @code{initialize-source-registry} again,
3047or maybe simply to @code{clear-source-registry} (or @code{clear-configuration})
3048which will cause the initialization to happen next time around.
3050@section Introspection
3052@subsection *source-registry-parameter* variable
3053@vindex *source-registry-parameter*
3055We have made available the variable @code{*source-registry-parameter*}
3056that can be used by code that wishes to introspect about the (past)
3057configuration of ASDF's source registry.  @strong{This variable should
3058never be set!}  It will be set as a side-effect of calling
3059@code{initialize-source-registry}; user code should treat it as
3062@subsection Information about system dependencies
3064ASDF makes available three functions to read system interdependencies.
3065These are intended to aid programmers who wish to perform dependency
3068@defun system-defsystem-depends-on system
3069@end defun
3071@defun system-depends-on system
3072@end defun
3074@defun system-weakly-depends-on system
3075Returns a list of names of systems that are weakly depended on by
3076@var{system}.  Weakly depended on systems are optionally loaded only if
3077ASDF can find them; failure to find such systems does @emph{not} cause an
3078error in loading.
3080Note that the return value for @code{system-weakly-depends-on} is simpler
3081than the return values of the other two system dependency introspection
3083@end defun
3085@section Status
3087This mechanism is vastly successful, and we have declared
3088that @code{asdf:*central-registry*} is not recommended anymore,
3089though we will continue to support it.
3090All hooks into implementation-specific search mechanisms
3091have been integrated in the @code{wrapping-source-registry}
3092that everyone uses implicitly.
3095@section Rejected ideas
3097Alternatives I considered and rejected included:
3100@item Keep @code{asdf:*central-registry*} as the master with its current semantics,
3101   and somehow the configuration parser expands the new configuration
3102   language into a expanded series of directories of subdirectories to
3103   lookup, pre-recursing through specified hierarchies. This is kludgy,
3104   and leaves little space of future cleanups and extensions.
3106@item Keep @code{asdf:*central-registry*} remains the master but extend its semantics
3107   in completely new ways, so that new kinds of entries may be implemented
3108   as a recursive search, etc. This seems somewhat backwards.
3110@item Completely remove @code{asdf:*central-registry*}
3111   and break backwards compatibility.
3112   Hopefully this will happen in a few years after everyone migrate to
3113   a better ASDF and/or to XCVB, but it would be very bad to do it now.
3115@item Replace @code{asdf:*central-registry*} by a symbol-macro with appropriate magic
3116   when you dereference it or setf it. Only the new variable with new
3117   semantics is handled by the new search procedure.
3118   Complex and still introduces subtle semantic issues.
3119@end enumerate
3122I've been suggested the below features, but have rejected them,
3123for the sake of keeping ASDF no more complex than strictly necessary.
3127  More syntactic sugar: synonyms for the configuration directives, such as
3128  @code{(:add-directory X)} for @code{(:directory X)}, or @code{(:add-directory-hierarchy X)}
3129  or @code{(:add-directory X :recurse t)} for @code{(:tree X)}.
3132   The possibility to register individual files instead of directories.
3135  Integrate Xach Beane's tilde expander into the parser,
3136  or something similar that is shell-friendly or shell-compatible.
3137  I'd rather keep ASDF minimal. But maybe this precisely keeps it
3138  minimal by removing the need for evaluated entries that ASDF has?
3139  i.e. uses of @code{USER-HOMEDIR-PATHNAME} and @code{$SBCL_HOME}
3140  Hopefully, these are already superseded by the @code{:default-registry}
3143  Using the shell-unfriendly syntax @code{/**} instead of @code{//} to specify recursion
3144  down a filesystem tree in the environment variable.
3145  It isn't that Lisp friendly either.
3146@end itemize
3148@section TODO
3151@item Add examples
3152@end itemize
3155@section Credits for the source-registry
3157Thanks a lot to Stelian Ionescu for the initial idea.
3159Thanks to Rommel Martinez for the initial implementation attempt.
3161All bad design ideas and implementation bugs are mine, not theirs.
3162But so are good design ideas and elegant implementation tricks.
3164 --- Francois-Rene Rideau @email{}, Mon, 22 Feb 2010 00:07:33 -0500
3168@node Controlling where ASDF saves compiled files, Error handling, Controlling where ASDF searches for systems, Top
3169@comment  node-name,  next,  previous,  up
3170@chapter Controlling where ASDF saves compiled files
3171@cindex asdf-output-translations
3174Each Common Lisp implementation has its own format
3175for compiled files (fasls for short, short for ``fast loading'').
3176If you use multiple implementations
3177(or multiple versions of the same implementation),
3178you'll soon find your source directories
3179littered with various @file{fasl}s, @file{dfsl}s, @file{cfsl}s and so on.
3180Worse yet, some implementations use the same file extension
3181while changing formats from version to version (or platform to platform)
3182which means that you'll have to recompile binaries
3183as you switch from one implementation to the next.
3185Since ASDF 2, ASDF includes the @code{asdf-output-translations} facility
3186to mitigate the problem.
3188@section Configurations
3190Configurations specify mappings from input locations to output locations.
3191Once again we rely on the XDG base directory specification for configuration.
3192@xref{Controlling where ASDF searches for systems,,XDG base directory}.
3197Some hardcoded wrapping output translations configuration may be used.
3198This allows special output translations (or usually, invariant directories)
3199to be specified corresponding to the similar special entries in the source registry.
3202An application may explicitly initialize the output-translations
3203configuration using the Configuration API
3204in which case this takes precedence.
3205(@pxref{Controlling where ASDF saves compiled files,,Configuration API}.)
3206It may itself compute this configuration from the command-line,
3207from a script, from its own configuration file, etc.
3210The source registry will be configured from
3211the environment variable @code{ASDF_OUTPUT_TRANSLATIONS} if it exists.
3214The source registry will be configured from
3215user configuration file
3217(which defaults to
3219if it exists.
3222The source registry will be configured from
3223user configuration directory
3225(which defaults to
3227if it exists.
3230The source registry will be configured from
3231system configuration file
3233if it exists.
3236The source registry will be configured from
3237system configuration directory
3239if it exists.
3241@end enumerate
3243Each of these configurations is specified as a SEXP
3244in a trival domain-specific language (defined below).
3245Additionally, a more shell-friendly syntax is available
3246for the environment variable (defined yet below).
3248Each of these configurations is only used if the previous
3249configuration explicitly or implicitly specifies that it
3250includes its inherited configuration.
3252Note that by default, a per-user cache is used for output files.
3253This allows the seamless use of shared installations of software
3254between several users, and takes files out of the way of the developers
3255when they browse source code,
3256at the expense of taking a small toll when developers have to clean up
3257output files and find they need to get familiar with output-translations first.
3260@section Backward Compatibility
3261@cindex ASDF-BINARY-LOCATIONS compatibility
3264We purposefully do NOT provide backward compatibility with earlier versions of
3265@code{ASDF-Binary-Locations} (8 Sept 2009),
3266@code{common-lisp-controller} (7.0) or
3267@code{cl-launch} (2.35),
3268each of which had similar general capabilities.
3269The previous APIs of these programs were not designed
3270for configuration by the end-user
3271in an easy way with configuration files.
3272Recent versions of same packages use
3273the new @code{asdf-output-translations} API as defined below:
3274@code{common-lisp-controller} (7.2) and @code{cl-launch} (3.000).
3275@code{ASDF-Binary-Locations} is fully superseded and not to be used anymore.
3277This incompatibility shouldn't inconvenience many people.
3278Indeed, few people use and customize these packages;
3279these few people are experts who can trivially adapt to the new configuration.
3280Most people are not experts, could not properly configure these features
3281(except inasmuch as the default configuration of
3282@code{common-lisp-controller} and/or @code{cl-launch}
3283might have been doing the right thing for some users),
3284and yet will experience software that ``just works'',
3285as configured by the system distributor, or by default.
3287Nevertheless, if you are a fan of @code{ASDF-Binary-Locations},
3288we provide a limited emulation mode:
3290@defun enable-asdf-binary-locations-compatibility @Akey centralize-lisp-binaries default-toplevel-directory include-per-user-information map-all-source-files source-to-target-mappings
3291This function will initialize the new @code{asdf-output-translations} facility in a way
3292that emulates the behavior of the old @code{ASDF-Binary-Locations} facility.
3293Where you would previously set global variables
3297@var{*map-all-source-files*} or @var{*source-to-target-mappings*}
3298you will now have to pass the same values as keyword arguments to this function.
3299Note however that as an extension the @code{:source-to-target-mappings} keyword argument
3300will accept any valid pathname designator for @code{asdf-output-translations}
3301instead of just strings and pathnames.
3302@end defun
3304If you insist, you can also keep using the old @code{ASDF-Binary-Locations}
3305(the one available as an extension to load of top of ASDF,
3306not the one built into a few old versions of ASDF),
3307but first you must disable @code{asdf-output-translations}
3308with @code{(asdf:disable-output-translations)},
3309or you might experience ``interesting'' issues.
3311Also, note that output translation is enabled by default.
3312To disable it, use @code{(asdf:disable-output-translations)}.
3315@section Configuration DSL
3317Here is the grammar of the SEXP DSL
3318for @code{asdf-output-translations} configuration:
3321;; A configuration is single SEXP starting with keyword :source-registry
3322;; followed by a list of directives.
3323CONFIGURATION := (:output-translations DIRECTIVE ...)
3325;; A directive is one of the following:
3328    ;; Your configuration expression MUST contain
3329    ;; exactly one of either of these:
3330    :inherit-configuration | ; splices inherited configuration (often specified last)
3331    :ignore-inherited-configuration | ; drop inherited configuration (specified anywhere)
3333    ;; forward compatibility directive (since ASDF 2.011.4), useful when
3334    ;; you want to use new configuration features but have to bootstrap a
3335    ;; the newer required ASDF from an older release that doesn't sport said features:
3336    :ignore-invalid-entries | ; drops subsequent invalid entries instead of erroring out
3338    ;; include a configuration file or directory
3339    (:include PATHNAME-DESIGNATOR) |
3341    ;; enable global cache in ~/.common-lisp/cache/sbcl-1.0.45-linux-amd64/ or something.
3342    :enable-user-cache |
3343    ;; Disable global cache. Map / to /
3344    :disable-cache |
3346    ;; add a single directory to be scanned (no recursion)
3349    ;; use a function to return the translation of a directory designator
3353    NIL | ;; As source: skip this entry. As destination: same as source
3354    T | ;; as source matches anything, as destination leaves pathname unmapped.
3355    ABSOLUTE-COMPONENT-DESIGNATOR ;; same as in the source-registry language
3358    SYMBOL | ;; symbol of a function that takes two arguments,
3359             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
3360    LAMBDA   ;; A form which evalutates to a function taking two arguments consisting of
3361             ;; the pathname to be translated and the matching DIRECTORY-DESIGNATOR
3363@end verbatim
3365Relative components better be either relative
3366or subdirectories of the path before them, or bust.
3368The last component, if not a pathname, is notionally completed by @file{/**/*.*}.
3369You can specify more fine-grained patterns
3370by using a pathname object as the last component
3371e.g. @file{#p"some/path/**/foo*/bar-*.fasl"}
3373You may use @code{#+features} to customize the configuration file.
3375The second designator of a mapping may be @code{nil}, indicating that files are not mapped
3376to anything but themselves (same as if the second designator was the same as the first).
3378When the first designator is @code{t},
3379the mapping always matches.
3380When the first designator starts with @code{:root},
3381the mapping matches any host and device.
3382In either of these cases, if the second designator
3383isn't @code{t} and doesn't start with @code{:root},
3384then strings indicating the host and pathname are somehow copied
3385in the beginning of the directory component of the source pathname
3386before it is translated.
3388When the second designator is @code{t}, the mapping is the identity.
3389When the second designator starts with @code{:root},
3390the mapping preserves the host and device of the original pathname.
3391Notably, this allows you to map files
3392to a subdirectory of the whichever directory the file is in.
3393Though the syntax is not quite as easy to use as we'd like,
3394you can have an (source destination) mapping entry such as follows
3395in your configuration file,
3396or you may use @code{enable-asdf-binary-locations-compatibility}
3397with @code{:centralize-lisp-binaries nil}
3398which will do the same thing internally for you:
3400  #.(let ((wild-subdir (make-pathname :directory '(:relative :wild-inferiors)))
3401          (wild-file (make-pathname :name :wild :version :wild :type :wild)))
3402     `((:root ,wild-subdir ,wild-file) ;; Or using the implicit wildcard, just :root
3403       (:root ,wild-subdir :implementation ,wild-file)))
3404@end verbatim
3405Starting with ASDF 2.011.4, you can use the simpler:
3406  @code{`(:root (:root :**/ :implementation :*.*.*))}
3410@code{:include} statements cause the search to recurse with the path specifications
3411from the file specified.
3413If the @code{translate-pathname} mechanism cannot achieve a desired
3414translation, the user may provide a function which provides the
3415required algorithim.  Such a translation function is specified by
3416supplying a list as the second @code{directory-designator}
3417the first element of which is the keyword @code{:function},
3418and the second element of which is
3419either a symbol which designates a function or a lambda expression.
3420The function designated by the second argument must take two arguments,
3421the first being the pathname of the source file,
3422the second being the wildcard that was matched.
3423The result of the function invocation should be the translated pathname.
3425An @code{:inherit-configuration} statement cause the search to recurse with the path
3426specifications from the next configuration.
3427@xref{Controlling where ASDF saves compiled files,,Configurations}, above.
3431@code{:enable-user-cache} is the same as @code{(t :user-cache)}.
3433@code{:disable-cache} is the same as @code{(t t)}.
3435@code{:user-cache} uses the contents of variable @code{asdf::*user-cache*}
3436which by default is the same as using
3437@code{(:home ".cache" "common-lisp" :implementation)}.
3438@end itemize
3441@section Configuration Directories
3443Configuration directories consist in files each contains
3444a list of directives without any enclosing
3445@code{(:output-translations ...)} form.
3446The files will be sorted by namestring as if by @code{string<} and
3447the lists of directives of these files with be concatenated in order.
3448An implicit @code{:inherit-configuration} will be included
3449at the @emph{end} of the list.
3451This allows for packaging software that has file granularity
3452(e.g. Debian's @command{dpkg} or some future version of @command{clbuild})
3453to easily include configuration information about software being distributed.
3455The convention is that, for sorting purposes,
3456the names of files in such a directory begin with two digits
3457that determine the order in which these entries will be read.
3458Also, the type of these files is conventionally @code{"conf"}
3459and as a limitation of some implementations, the type cannot be @code{nil}.
3461Directories may be included by specifying a directory pathname
3462or namestring in an @code{:include} directive, e.g.:
3464  (:include "/foo/bar/")
3465@end verbatim
3467@section Shell-friendly syntax for configuration
3469When considering environment variable @code{ASDF_OUTPUT_TRANSLATIONS}
3470ASDF will skip to next configuration if it's an empty string.
3471It will @code{READ} the string as an SEXP in the DSL
3472if it begins with a paren @code{(}
3473and it will be interpreted as a list of directories.
3474Directories should come by pairs, indicating a mapping directive.
3475Entries are separated
3476by a @code{:} (colon) on Unix platforms (including cygwin),
3477by a @code{;} (semicolon) on other platforms (mainly, Windows).
3479The magic empty entry,
3480if it comes in what would otherwise be the first entry in a pair,
3481indicates the splicing of inherited configuration.
3482If it comes as the second entry in a pair,
3483it indicates that the directory specified first is to be left untranslated
3484(which has the same effect as if the directory had been repeated).
3487@section Semantics of Output Translations
3489From the specified configuration,
3490a list of mappings is extracted in a straightforward way:
3491mappings are collected in order, recursing through
3492included or inherited configuration as specified.
3493To this list is prepended some implementation-specific mappings,
3494and is appended a global default.
3496The list is then compiled to a mapping table as follows:
3497for each entry, in order, resolve the first designated directory
3498into an actual directory pathname for source locations.
3499If no mapping was specified yet for that location,
3500resolve the second designated directory to an output location directory
3501add a mapping to the table mapping the source location to the output location,
3502and add another mapping from the output location to itself
3503(unless a mapping already exists for the output location).
3505Based on the table, a mapping function is defined,
3506mapping source pathnames to output pathnames:
3507given a source pathname, locate the longest matching prefix
3508in the source column of the mapping table.
3509Replace that prefix by the corresponding output column
3510in the same row of the table, and return the result.
3511If no match is found, return the source pathname.
3512(A global default mapping the filesystem root to itself
3513may ensure that there will always be a match,
3514with same fall-through semantics).
3516@section Caching Results
3518The implementation is allowed to either eagerly compute the information
3519from the configurations and file system, or to lazily re-compute it
3520every time, or to cache any part of it as it goes.
3521To explicitly flush any information cached by the system, use the API below.
3524@section Output location API
3526The specified functions are exported from package ASDF.
3528@defun initialize-output-translations @Aoptional PARAMETER
3529   will read the configuration and initialize all internal variables.
3530   You may extend or override configuration
3531   from the environment and configuration files
3532   with the given @var{PARAMETER}, which can be
3533   @code{nil} (no configuration override),
3534   or a SEXP (in the SEXP DSL),
3535   a string (as in the string DSL),
3536   a pathname (of a file or directory with configuration),
3537   or a symbol (fbound to function that when called returns one of the above).
3538@end defun
3540@defun disable-output-translations
3541   will initialize output translations in a way
3542   that maps every pathname to itself,
3543   effectively disabling the output translation facility.
3544@end defun
3546@defun clear-output-translations
3547   undoes any output translation configuration
3548   and clears any cache for the mapping algorithm.
3549   You might want to call this function
3550   (or better, @code{clear-configuration})
3551   before you dump an image that would be resumed
3552   with a different configuration,
3553   and return an empty configuration.
3554   Note that this does not include clearing information about
3555   systems defined in the current image, only about
3556   where to look for systems not yet defined.
3557@end defun
3559@defun ensure-output-translations @Aoptional PARAMETER
3560   checks whether output translations have been initialized.
3561   If not, initialize them with the given @var{PARAMETER}.
3562   This function will be called before any attempt to operate on a system.
3563@end defun
3565@defun apply-output-translations PATHNAME
3566   Applies the configured output location translations to @var{PATHNAME}
3567   (calls @code{ensure-output-translations} for the translations).
3568@end defun
3570Every time you use ASDF's @code{output-files}, or
3571anything that uses it (that may compile, such as @code{operate}, @code{perform}, etc.),
3572@code{ensure-output-translations} is called with parameter @code{nil},
3573which the first time around causes your configuration to be read.
3574If you change a configuration file,
3575you need to explicitly @code{initialize-output-translations} again,
3576or maybe @code{clear-output-translations} (or @code{clear-configuration}),
3577which will cause the initialization to happen next time around.
3580@section Credits for output translations
3582Thanks a lot to Bjorn Lindberg and Gary King for @code{ASDF-Binary-Locations},
3583and to Peter van Eynde for @code{Common Lisp Controller}.
3585All bad design ideas and implementation bugs are to mine, not theirs.
3586But so are good design ideas and elegant implementation tricks.
3588 --- Francois-Rene Rideau @email{}
3590@c @section Default locations
3591@c @findex output-files-for-system-and-operation
3593@c The default binary location for each Lisp implementation
3594@c is a subdirectory of each source directory.
3595@c To account for different Lisps, Operating Systems, Implementation versions,
3596@c and so on, ASDF borrows code from SLIME
3597@c to create reasonable custom directory names.
3598@c Here are some examples:
3600@c @itemize
3601@c @item
3602@c SBCL, version 1.0.45 on Mac OS X for Intel: @code{sbcl-1.0.45-darwin-x86}
3604@c @item
3605@c Franz Allegro, version 8.0, ANSI Common Lisp: @code{allegro-8.0a-macosx-x86}
3607@c @item
3608@c Franz Allegro, version 8.1, Modern (case sensitive) Common Lisp: @code{allegro-8.1m-macosx-x86}
3609@c @end itemize
3611@c By default, all output file pathnames will be relocated
3612@c to some thus-named subdirectory of @file{~/.cache/common-lisp/}.
3614@c See the document @file{README.asdf-output-translations}
3615@c for a full specification on how to configure @code{asdf-output-translations}.
3617@node  Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top
3618@comment  node-name,  next,  previous,  up
3619@chapter Error handling
3623@section ASDF errors
3625If ASDF detects an incorrect system definition, it will signal a generalised instance of
3628Operations may go wrong (for example when source files contain errors).
3629These are signalled using generalised instances of
3632@section Compilation error and warning handling
3633@vindex *compile-file-warnings-behaviour*
3634@vindex *compile-file-errors-behavior*
3636ASDF checks for warnings and errors when a file is compiled.
3637The variables @var{*compile-file-warnings-behaviour*} and
3639control the handling of any such events.
3640The valid values for these variables are
3641@code{:error}, @code{:warn}, and @code{:ignore}.
3643@node  Miscellaneous additional functionality, Getting the latest version, Error handling, Top
3644@comment  node-name,  next,  previous,  up
3645@chapter Miscellaneous additional functionality
3647ASDF includes several additional features that are generally
3648useful for system definition and development.
3650@section Controlling file compilation
3651@cindex :around-compile
3652@cindex around-compile keyword
3653@cindex compile-check keyword
3654@cindex :compile-check
3655@findex compile-file*
3657When declaring a component (system, module, file),
3658you can specify a keyword argument @code{:around-compile function}.
3659If left unspecified (and therefore unbound),
3660the value will be inherited from the parent component if any,
3661or with a default of @code{nil}
3662if no value is specified in any transitive parent.
3664The argument must be a either @code{nil}, a fbound symbol,
3665a lambda-expression (e.g. @code{(lambda (thunk) ...(funcall thunk ...) ...)})
3666a function object (e.g. using @code{#.#'} but that's discouraged
3667because it prevents the introspection done by e.g. asdf-dependency-grovel),
3668or a string that when @code{read} yields a symbol or a lambda-expression.
3669@code{nil} means the normal compile-file function will be called.
3670A non-nil value designates a function of one argument
3671that will be called with a function that will
3672invoke @code{compile-file*} with various arguments;
3673the around-compile hook may supply additional keyword arguments
3674to pass to that call to @code{compile-file*}.
3676One notable argument that is heeded by @code{compile-file*} is
3678a function called when the compilation was otherwise a success,
3679with the same arguments as @code{compile-file};
3680the function shall return true if the compilation
3681and its resulting compiled file respected all system-specific invariants,
3682and false (@code{nil}) if it broke any of those invariants;
3683it may issue warnings or errors before it returns @code{nil}.
3684(NB: The ability to pass such extra flags
3685is only available starting with ASDF 2.22.3.)
3686This feature is notably exercised by asdf-finalizers.
3688By using a string, you may reference
3689a function, symbol and/or package
3690that will only be created later during the build, but
3691isn't yet present at the time the defsystem form is evaluated.
3692However, if your entire system is using such a hook, you may have to
3693explicitly override the hook with @code{nil} for all the modules and files
3694that are compiled before the hook is defined.
3696Using this hook, you may achieve such effects as:
3697locally renaming packages,
3698binding @var{*readtables*} and other syntax-controlling variables,
3699handling warnings and other conditions,
3700proclaiming consistent optimization settings,
3701saving code coverage information,
3702maintaining meta-data about compilation timings,
3703setting gensym counters and PRNG seeds and other sources of non-determinism,
3704overriding the source-location and/or timestamping systems,
3705checking that some compile-time side-effects were properly balanced,
3708Note that there is no around-load hook. This is on purpose.
3709Some implementations such as ECL, GCL or MKCL link object files,
3710which allows for no such hook.
3711Other implementations allow for concatenating FASL files,
3712which doesn't allow for such a hook either.
3713We aim to discourage something that's not portable,
3714and has some dubious impact on performance and semantics
3715even when it is possible.
3716Things you might want to do with an around-load hook
3717are better done around-compile,
3718though it may at times require some creativity
3719(see e.g. the @code{package-renaming} system).
3722@section Controlling source file character encoding
3724Starting with ASDF 2.21, components accept a @code{:encoding} option
3725so authors may specify which character encoding should be used
3726to read and evaluate their source code.
3727When left unspecified, the encoding is inherited
3728from the parent module or system;
3729if no encoding is specified at any point,
3730the default @code{:autodetect} is assumed.
3731By default, only @code{:default}, @code{:utf-8}
3732and @code{:autodetect} are accepted.
3733@code{:autodetect}, the default, calls
3734@code{*encoding-detection-hook*} which by default always returns
3735@code{*default-encoding*} which itself defaults to @code{:default}.
3737In other words, there now are plenty of extension hooks, but
3738by default ASDF follows the backwards compatible behavior
3739of using whichever @code{:default} encoding your implementation uses,
3740which itself may or may not vary based on environment variables
3741and other locale settings.
3742In practice this means that only source code that only uses ASCII
3743is guaranteed to be read the same on all implementations
3744independently from any user setting.
3746Additionally, for backward-compatibility with older versions of ASDF
3747and/or with implementations that do not support unicode and its many encodings,
3748you may want to use
3749the reader conditionals @code{#+asdf-unicode #+asdf-unicode}
3750to protect any @code{:encoding @emph{encoding}} statement
3751as @code{:asdf-unicode} will be present in @code{*features*}
3752only if you're using a recent ASDF
3753on an implementation that supports unicode.
3754We recommend that you avoid using unprotected @code{:encoding} specifications
3755until after ASDF 2.21 or later becomes widespread, hopefully by the end of 2012.
3757While it offers plenty of hooks for extension,
3758and one such extension is being developed (see below),
3759ASDF itself only recognizes one encoding beside @code{:default},
3760and that is @code{:utf-8}, which is the @emph{de facto} standard,
3761already used by the vast majority of libraries that use more than ASCII.
3762On implementations that do not support unicode,
3763the feature @code{:asdf-unicode} is absent, and
3764the @code{:default} external-format is used
3765to read even source files declared as @code{:utf-8}.
3766On these implementations, non-ASCII characters
3767intended to be read as one CL character
3768may thus end up being read as multiple CL characters.
3769In most cases, this shouldn't affect the software's semantics:
3770comments will be skipped just the same, strings with be read and printed
3771with slightly different lengths, symbol names will be accordingly longer,
3772but none of it should matter.
3773But a few systems that actually depend on unicode characters
3774may fail to work properly, or may work in a subtly different way.
3775See for instance @code{lambda-reader}.
3777We invite you to embrace UTF-8
3778as the encoding for non-ASCII characters starting today,
3779even without any explicit specification in your @code{.asd} files.
3780Indeed, on some implementations and configurations,
3781UTF-8 is already the @code{:default},
3782and loading your code may cause errors if it is encoded in anything but UTF-8.
3783Therefore, even with the legacy behavior,
3784non-UTF-8 is guaranteed to break for some users,
3785whereas UTF-8 is pretty much guaranteed not to break anywhere
3786(provided you do @emph{not} use a BOM),
3787although it might be read incorrectly on some implementations.
3788In the future, we intend to make @code{:utf-8}
3789the default value of @code{*default-encoding*},
3790to be enforced everywhere, so at least the code is guaranteed
3791to be read correctly everywhere it can be.
3793If you need non-standard character encodings for your source code,
3794use the extension system @code{asdf-encodings}, by specifying
3795@code{:defsystem-depends-on (:asdf-encodings)} in your @code{defsystem}.
3796This extension system will register support for more encodings using the
3797@code{*encoding-external-format-hook*} facility,
3798so you can explicitly specify @code{:encoding :latin1}
3799in your @code{.asd} file.
3800Using the @code{*encoding-detection-hook*} it will also
3801eventually implement some autodetection of a file's encoding
3802from an emacs-style @code{-*- mode: lisp ; coding: latin1 -*-} declaration,
3803or otherwise based on an analysis of octet patterns in the file.
3804At this point, asdf-encoding only supports the encodings
3805that are supported as part of your implementation.
3806Since the list varies depending on implementations,
3807we once again recommend you use @code{:utf-8} everywhere,
3808which is the most portable (next is @code{:latin1}).
3810If you're not using a version of Quicklisp that has it,
3811you may get the source for @code{asdf-encodings} using git:
3812@kbd{git clone git://}
3814@kbd{git clone ssh://}.
3815You can also browse the repository on
3818In the future, we intend to change the default @code{*default-encoding*}
3819to @code{:utf-8}, which is already the de facto standard
3820for most libraries that use non-ASCII characters:
3821utf-8 works everywhere and was backhandedly enforced by
3822a lot of people using SBCL and utf-8 and sending reports to authors
3823so they make their packages compatible.
3824A survey showed only about a handful few libraries
3825are incompatible with non-UTF-8, and then, only in comments,
3826and we believe that authors will adopt UTF-8 when prompted.
3827See the April 2012 discussion on the asdf-devel mailing-list.
3828For backwards compatibility with users who insist on a non-UTF-8 encoding,
3829but cannot immediately transition to using @code{asdf-encodings}
3830(maybe because it isn't ready), it will still be possible to use
3831the @code{:encoding :default} option in your @code{defsystem} form
3832to restore the behavior of ASDF 2.20 and earlier.
3833This shouldn't be required in libraries,
3834because user pressure as mentioned above will already have pushed
3835library authors towards using UTF-8;
3836but authors of end-user programs might care.
3838When you use @code{asdf-encodings}, any further loaded @code{.asd} file
3839will use the autodetection algorithm to determine its encoding;
3840yet if you depend on this detection happening,
3841you may want to explicitly load @code{asdf-encodings} early in your build,
3842for by the time you can use @code{:defsystem-depends-on},
3843it is already too late to load it.
3844In practice, this means that the @code{*default-encoding*}
3845is usually used for @code{.asd} files.
3846Currently, this defaults to @code{:default} for backwards compatibility,
3847and that means that you shouldn't rely on non-ASCII characters in a .asd file.
3848Since component (path)names are the only real data in these files,
3849and non-ASCII characters are not very portable for file names,
3850this isn't too much of an issue.
3851We still encourage you to use either plain ASCII or UTF-8
3852in @code{.asd} files,
3853as we intend to make @code{:utf-8} the default encoding in the future.
3854This might matter, for instance, in meta-data about author's names.
3857@section Miscellaneous Functions
3859These functions are exported by ASDF for your convenience.
3862@defun system-relative-pathname system name @Akey type
3864It's often handy to locate a file relative to some system.
3865The @code{system-relative-pathname} function meets this need.
3867It takes two mandatory arguments @var{system} and @var{name}
3868and a keyword argument @var{type}:
3869@var{system} is name of a system, whereas @var{name} and optionally @var{type}
3870specify a relative pathname, interpreted like a component pathname specifier
3871by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}.
3873It returns a pathname built from the location of the system's
3874source directory and the relative pathname. For example:
3877> (asdf:system-relative-pathname 'cl-ppcre "")
3879@end lisp
3881@end defun
3883@defun system-source-directory system-designator
3885ASDF does not provide a turnkey solution for locating
3886data (or other miscellaneous) files
3887that are distributed together with the source code of a system.
3888Programmers can use @code{system-source-directory} to find such files.
3889Returns a pathname object.
3890The @var{system-designator} may be a string, symbol, or ASDF system object.
3891@end defun
3893@defun clear-system system-designator
3895It is sometimes useful to force recompilation of a previously loaded system.
3896In these cases, it may be useful to @code{(asdf:clear-system :foo)}
3897to remove the system from the table of currently loaded systems;
3898the next time the system @code{foo} or one that depends on it is re-loaded,
3899@code{foo} will then be loaded again.
3900Alternatively, you could touch @code{foo.asd} or
3901remove the corresponding fasls from the output file cache.
3902(It was once conceived that one should provide
3903a list of systems the recompilation of which to force
3904as the @code{:force} keyword argument to @code{load-system};
3905but this has never worked, and though the feature was fixed in ASDF 2.000,
3906it remains @code{cerror}'ed out as nobody ever used it.)
3908Note that this does not and cannot by itself undo the previous loading
3909of the system. Common Lisp has no provision for such an operation,
3910and its reliance on irreversible side-effects to global datastructures
3911makes such a thing impossible in the general case.
3912If the software being re-loaded is not conceived with hot upgrade in mind,
3913this re-loading may cause many errors, warnings or subtle silent problems,
3914as packages, generic function signatures, structures, types, macros, constants, etc.
3915are being redefined incompatibly.
3916It is up to the user to make sure that reloading is possible and has the desired effect.
3917In some cases, extreme measures such as recursively deleting packages,
3918unregistering symbols, defining methods on @code{update-instance-for-redefined-class}
3919and much more are necessary for reloading to happen smoothly.
3920ASDF itself goes through notable pains to make such a hot upgrade possible
3921with respect to its own code, and what it does is ridiculously complex;
3922look at the beginning of @file{asdf.lisp} to see what it does.
3923@end defun
3925@defun register-preloaded-system name @Arest keys
3926A system with name @var{name},
3927created by @code{make-instance} with extra keys @var{keys}
3928(e.g. @code{:version}),
3929is registered as @emph{preloaded}.
3930That is, its code has already been loaded into the current image,
3931and if at some point some other system @code{:depends-on} it yet no source code is found,
3932it is considered as already provided,
3933and ASDF will not raise a @code{missing-component} error.
3935This function is particularly useful if you distribute your code
3936as fasls with either @code{fasl-op} or @code{monolithic-fasl-op},
3937and want to register systems so that dependencies will work uniformly
3938whether you're using your software from source or from fasl.
3939@end defun
3941@defun run-shell-command control-string @Arest args
3943This function is obsolete and present only for the sake of backwards-compatibility:
3944``If it's not backwards, it's not compatible''. We @emph{strongly} discourage its use.
3945Its current behavior is only well-defined on Unix platforms
3946(which include MacOS X and cygwin). On Windows, anything goes.
3947The following documentation is only for the purpose of your migrating away from it
3948in a way that preserves semantics.
3950Instead we recommend the use @code{run-program}, described in the next section, and
3951available as part of ASDF since ASDF 3.
3953@code{run-shell-command} takes as arguments a format @code{control-string}
3954and arguments to be passed to @code{format} after this control-string
3955to produce a string.
3956This string is a command that will be evaluated with a POSIX shell if possible;
3957yet, on Windows, some implementations will use CMD.EXE,
3958while others (like SBCL) will make an attempt at invoking a POSIX shell
3959(and fail if it is not present).
3960@end defun
3962@section Some Utility Functions
3964The below functions are not exported by ASDF itself, but by UIOP, available since ASDF 3.
3965Some of them have precursors in ASDF 2, but we recommend
3966you rely on ASDF 3 for active developments.
3967UIOP provides many, many more utility functions, and we recommend
3968you read its README and sources for more information.
3971@defun parse-unix-namestring name @Akey type defaults dot-dot ensure-directory @AallowOtherKeys
3972Coerce NAME into a PATHNAME using standard Unix syntax.
3974Unix syntax is used whether or not the underlying system is Unix;
3975on such non-Unix systems it is only usable but for relative pathnames;
3976but especially to manipulate relative pathnames portably, it is of crucial
3977to possess a portable pathname syntax independent of the underlying OS.
3978This is what @code{parse-unix-namestring} provides, and why we use it in ASDF.
3980When given a @code{pathname} object, just return it untouched.
3981When given @code{nil}, just return @code{nil}.
3982When given a non-null @code{symbol}, first downcase its name and treat it as a string.
3983When given a @code{string}, portably decompose it into a pathname as below.
3985@code{#\/} separates directory components.
3987The last @code{#\/}-separated substring is interpreted as follows:
39881- If @var{type} is @code{:directory} or @var{ensure-directory} is true,
3989 the string is made the last directory component, and its @code{name} and @code{type} are @code{nil}.
3990 if the string is empty, it's the empty pathname with all slots @code{nil}.
39912- If @var{type} is @code{nil}, the substring is a file-namestring,
3992 and its @code{name} and @code{type} are separated by @code{split-name-type}.
39933- If @var{type} is a string, it is the given @code{type}, and the whole string is the @code{name}.
3995Directory components with an empty name the name @code{.} are removed.
3996Any directory named @code{..} is read as @var{dot-dot},
3997which must be one of @code{:back} or @code{:up} and defaults to @code{:back}.
3999@code{host}, @code{device} and @code{version} components are taken from @var{defaults},
4000which itself defaults to @code{*nil-pathname*}, also used if @var{defaults} is @code{nil}.
4001No host or device can be specified in the string itself,
4002which makes it unsuitable for absolute pathnames outside Unix.
4004For relative pathnames, these components (and hence the defaults) won't matter
4005if you use @code{merge-pathnames*} but will matter if you use @code{merge-pathnames},
4006which is an important reason to always use @code{merge-pathnames*}.
4008Arbitrary keys are accepted, and the parse result is passed to @code{ensure-pathname}
4009with those keys, removing @var{type}, @var{defaults} and @var{dot-dot}.
4010When you're manipulating pathnames that are supposed to make sense portably
4011even though the OS may not be Unixish, we recommend you use @code{:want-relative t}
4012to throw an error if the pathname is absolute
4013@end defun
4015@defun merge-pathnames* specified @Aoptional defaults
4017This function is a replacement for @code{merge-pathnames} that uses the host and device
4018from the @var{defaults} rather than the @var{specified} pathname when the latter
4019is a relative pathname. This allows ASDF and its users to create and use relative pathnames
4020without having to know beforehand what are the host and device
4021of the absolute pathnames they are relative to.
4023@end defun
4025@defun subpathname pathname subpath @Akey type
4027This function takes a @var{pathname} and a @var{subpath} and a @var{type}.
4028If @var{subpath} is already a @code{pathname} object (not namestring),
4029and is an absolute pathname at that, it is returned unchanged;
4030otherwise, @var{subpath} is turned into a relative pathname with given @var{type}
4031as per @code{parse-unix-namestring} with @code{:want-relative t :type }@var{type},
4032then it is merged with the @code{pathname-directory-pathname} of @var{pathname},
4033as per @code{merge-pathnames*}.
4035We strongly encourage the use of this function
4036for portably resolving relative pathnames in your code base.
4037@end defun
4039@defun subpathname* pathname subpath @Akey type
4041This function returns @code{nil} if the base @var{pathname} is @code{nil},
4042otherwise acts like @code{subpathname}.
4043@end defun
4045@defun run-program command @Akey ignore-error-status force-shell input output error-output
4046  if-input-does-not-exist if-output-exists if-error-output-exists
4047  element-type external-format @AallowOtherKeys
4049@code{run-program} takes a @var{command} argument that is either
4050a list of a program name or path and its arguments,
4051or a string to be executed by a shell.
4052It spawns the command, waits for it to return,
4053verifies that it exited cleanly (unless told not too below),
4054and optionally captures and processes its output.
4055It accepts many keyword arguments to configure its behavior.
4057@code{run-program} returns three values: the first for the output,
4058the second for the error-output, and the third for the return value.
4059(Beware that before ASDF, it didn't handle input or error-output,
4060and returned only one value,
4061the one for the output if any handler was specified, or else the exit code;
4062please upgrade ASDF, or at least UIOP, to rely on the new enhanced behavior.)
4064@var{output} is its most important argument;
4065it specifies how the output is captured and processed.
4066If it is @code{nil}, then the output is redirected to the null device,
4067that will discard it.
4068If it is @code{:interactive}, then it is inherited from the current process
4069(beware: this may be different from your @var{*standard-output*},
4070and under SLIME will be on your @code{*inferior-lisp*} buffer).
4071If it is @code{t}, output goes to your current @var{*standard-output*} stream.
4072Otherwise, @var{output} should be a value that is a suitable first argument to
4073@code{slurp-input-stream} (see below), or
4074a list of such a value and keyword arguments.
4075In this case, @code{run-program} will
4076create a temporary stream for the program output;
4077the program output, in that stream,
4078will be processed by a call to @code{slurp-input-stream},
4079using @var{output} as the first argument
4080(or if it's a list the first element of @var{output} and the rest as keywords).
4081The primary value resulting from that call
4082(or @code{nil} if no call was needed)
4083will be the first value returned by @code{run-program}.
4084E.g., using @code{:output :string}
4085will have it return the entire output stream as a string.
4086And using @code{:output '(:string :stripped t)}
4087will have it return the same string stripped of any ending newline.
4089@var{error-output} is similar to @var{output}, except that
4090the resulting value is returned as the second value of @code{run-program}.
4091@code{t} designates the @var{*error-output*}.
4092Also @code{:output} means redirecting the error output to the output stream,
4093in which case @code{nil} is returned.
4095@var{input} is similar to @var{output}, except that
4096@code{vomit-output-stream} is used, no value is returned,
4097and @code{t} designates the @var{*standard-input*}.
4099@code{element-type} and @code{external-format} are passed on
4100to your Lisp implementation, when applicable, for creation of the output stream.
4102One and only one of the stream slurping or vomiting may or may not happen
4103in parallel in parallel with the subprocess,
4104depending on options and implementation,
4105and with priority being given to output processing.
4106Other streams are completely produced or consumed
4107before or after the subprocess is spawned, using temporary files.
4109@code{force-shell} forces evaluation of the command through a shell,
4110even if it was passed as a list rather than a string.
4111If a shell is used, it is @file{/bin/sh} on Unix or @file{CMD.EXE} on Windows,
4112except on implementations that (erroneously, IMNSHO)
4113insist on consulting @code{$SHELL} like clisp.
4115@code{ignore-error-status} causes @code{run-program}
4116to not raise an error if the spawned program exits in error.
4117Following POSIX convention, an error is anything but
4118a normal exit with status code zero.
4119By default, an error of type @code{subprocess-error} is raised in this case.
4121@code{run-program} works on all platforms supported by ASDF, except Genera.
4122See the source code for more documentation.
4124@end defun
4126@defun slurp-input-stream processor input-stream @Akey
4128It's a generic function of two arguments, a target object and an input stream,
4129and accepting keyword arguments.
4130Predefined methods based on the target object are as follow:
4132If the object is a function, the function is called with the stream as argument.
4134If the object is a cons, its first element is applied to its rest appended by
4135a list of the input stream.
4137If the object is an output stream, the contents of the input stream are copied to it.
4138If the @var{linewise} keyword argument is provided, copying happens line by line,
4139and an optional @var{prefix} is printed before each line.
4140Otherwise, copying happen based on a buffer of size @var{buffer-size},
4141using the specified @var{element-type}.
4143If the object is @code{'string} or @code{:string}, the content is captured into a string.
4144Accepted keywords include the @var{element-type} and a flag @var{stripped},
4145which when true causes any single line ending to be removed as per @code{uiop:stripln}.
4147If the object is @code{:lines}, the content is captured as a list of strings,
4148one per line, without line ending. If the @var{count} keyword argument is provided,
4149it is a maximum count of lines to be read.
4151If the object is @code{:line}, the content is capture as with @code{:lines} above,
4152and then its sub-object is extracted with the @var{at} argument,
4153which defaults to @code{0}, extracting the first line.
4154A number will extract the corresponding line.
4155See the documentation for @code{uiop:access-at}.
4157If the object is @code{:forms}, the content is captured as a list of S-expressions,
4158as read by the Lisp reader.
4159If the @var{count} argument is provided,
4160it is a maximum count of lines to be read.
4161We recommend you control the syntax with such macro as
4164If the object is @code{:form}, the content is capture as with @code{:forms} above,
4165and then its sub-object is extracted with the @var{at} argument,
4166which defaults to @code{0}, extracting the first form.
4167A number will extract the corresponding form.
4168See the documentation for @code{uiop:access-at}.
4169We recommend you control the syntax with such macro as
4172@end defun
4175@node Getting the latest version, FAQ, Miscellaneous additional functionality, Top
4176@comment  node-name,  next,  previous,  up
4177@chapter Getting the latest version
4179Decide which version you want.
4180The @code{master} branch is where development happens;
4181its @code{HEAD} is usually OK, including the latest fixes and portability tweaks,
4182but an occasional regression may happen despite our (limited) test suite.
4184The @code{release} branch is what cautious people should be using;
4185it has usually been tested more, and releases are cut at a point
4186where there isn't any known unresolved issue.
4188You may get the ASDF source repository using git:
4189@kbd{git clone git://}
4191You will find the above referenced tags in this repository.
4192You can also browse the repository on
4195Discussion of ASDF development is conducted on the
4196mailing list
4201@node FAQ, Ongoing Work, Getting the latest version, Top
4202@comment  node-name,  next,  previous,  up
4203@chapter FAQ
4206* Where do I report a bug?::
4207* What has changed between ASDF 1 and ASDF 2?::
4208* Issues with installing the proper version of ASDF::
4209* Issues with configuring ASDF::
4210* Issues with using and extending ASDF to define systems::
4211* ASDF development FAQs::
4212@end menu
4214@node Where do I report a bug?, What has changed between ASDF 1 and ASDF 2?, FAQ, FAQ
4215@section  ``Where do I report a bug?''
4217ASDF bugs are tracked on launchpad: @url{}.
4219If you're unsure about whether something is a bug, or for general discussion,
4220use the @url{,asdf-devel mailing list}
4223@node What has changed between ASDF 1 and ASDF 2?, Issues with installing the proper version of ASDF, Where do I report a bug?, FAQ
4224@section ``What has changed between ASDF 1 and ASDF 2?''
4227* What are ASDF 1 2 3?::
4228* ASDF can portably name files in subdirectories::
4229* Output translations::
4230* Source Registry Configuration::
4231* Usual operations are made easier to the user::
4232* Many bugs have been fixed::
4233* ASDF itself is versioned::
4234* ASDF can be upgraded::
4235* Decoupled release cycle::
4236* Pitfalls of the transition to ASDF 2::
4237@end menu
4239@node What are ASDF 1 2 3?, ASDF can portably name files in subdirectories, What has changed between ASDF 1 and ASDF 2?, What has changed between ASDF 1 and ASDF 2?
4240@subsection What are ASDF 1, ASDF 2 and ASDF 3?
4242ASDF 1 refers to any release earlier than 1.369 or so (from August 2001 to October 2009),
4243and to any development revision earlier than 2.000 (May 2010).
4244If your copy of ASDF doesn't even contain version information, it's an old ASDF 1.
4245Revisions between 1.656 and 1.728 may count as development releases for ASDF 2.
4247ASDF 2 refers to releases from 2.000 (May 31st 2010) to 2.26 (Oct 30 2012),
4248and any development revision newer than ASDF 1 and older than 2.27 (Feb 1 2013).
4250ASDF 3 refers to releases from 2.27 (Feb 1 2013) to 2.33 and 3.0.0 onward (May 15 2013).
42512.27 to 2.33 count as pre-releases to ASDF 3.
4253All releases of ASDF
4254push @code{:asdf} onto @code{*features*}.
4255Releases starting with ASDF 2
4256 push @code{:asdf2} onto @code{*features*}.
4257Releases starting with ASDF 3 (including 2.27 and later pre-releases)
4258push @code{:asdf3} onto @code{*features*}.
4259Furthermore, releases starting with ASDF 3.1 (March 2014),
4260though they count as ASDF 3, includes enough progress that they
4261push @code{:asdf3.1} onto @code{*features*}.
4262You may depend on the presence or absence of these features
4263to write code that takes advantage of recent ASDF functionality
4264but still works on older versions, or at least detects the old version and signals an error.
4266Additionally, all releases starting with ASDF 2
4267define a function @code{(asdf:asdf-version)} you may use to query the version.
4268All releases starting with 2.013 display the version number prominently
4269on the second line of the @file{asdf.lisp} source file.
4271If you are experiencing problems or limitations of any sort with ASDF 1 or ASDF 2,
4272we recommend that you should upgrade to the latest release, be it ASDF 3 or other.
4275@node ASDF can portably name files in subdirectories, Output translations, What are ASDF 1 2 3?, What has changed between ASDF 1 and ASDF 2?
4276@subsection ASDF can portably name files in subdirectories
4278Common Lisp namestrings are not portable,
4279except maybe for logical pathnamestrings,
4280that themselves have various limitations and require a lot of setup
4281that is itself ultimately non-portable.
4283In ASDF 1, the only portable ways to refer to pathnames inside systems and components
4284were very awkward, using @code{#.(make-pathname ...)} and
4285@code{#.(merge-pathnames ...)}.
4286Even the above were themselves were inadequate in the general case
4287due to host and device issues, unless horribly complex patterns were used.
4288Plenty of simple cases that looked portable actually weren't,
4289leading to much confusion and greavance.
4291ASDF 2 implements its own portable syntax for strings as pathname specifiers.
4292Naming files within a system definition becomes easy and portable again.
4293@xref{Miscellaneous additional functionality,system-relative-pathname},
4297On the other hand, there are places where systems used to accept namestrings
4298where you must now use an explicit pathname object:
4299@code{(defsystem ... :pathname "LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4300must now be written with the @code{#p} syntax:
4301@code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)}
4303@xref{The defsystem grammar,,Pathname specifiers}.
4306@node Output translations, Source Registry Configuration, ASDF can portably name files in subdirectories, What has changed between ASDF 1 and ASDF 2?
4307@subsection Output translations
4309A popular feature added to ASDF was output pathname translation:
4310@code{asdf-binary-locations}, @code{common-lisp-controller},
4311@code{cl-launch} and other hacks were all implementing it in ways
4312both mutually incompatible and difficult to configure.
4314Output pathname translation is essential to share
4315source directories of portable systems across multiple implementations
4316or variants thereof,
4317or source directories of shared installations of systems across multiple users,
4318or combinations of the above.
4320In ASDF 2, a standard mechanism is provided for that,
4322with sensible defaults, adequate configuration languages,
4323a coherent set of configuration files and hooks,
4324and support for non-Unix platforms.
4326@xref{Controlling where ASDF saves compiled files}.
4328@node Source Registry Configuration, Usual operations are made easier to the user, Output translations, What has changed between ASDF 1 and ASDF 2?
4329@subsection Source Registry Configuration
4331Configuring ASDF used to require special magic
4332to be applied just at the right moment,
4333between the moment ASDF is loaded and the moment it is used,
4334in a way that is specific to the user,
4335the implementation he is using and the application he is building.
4337This made for awkward configuration files and startup scripts
4338that could not be shared between users, managed by administrators
4339or packaged by distributions.
4341ASDF 2 provides a well-documented way to configure ASDF,
4342with sensible defaults, adequate configuration languages,
4343and a coherent set of configuration files and hooks.
4345We believe it's a vast improvement because it decouples
4346application distribution from library distribution.
4347The application writer can avoid thinking where the libraries are,
4348and the library distributor (dpkg, clbuild, advanced user, etc.)
4349can configure them once and for every application.
4350Yet settings can be easily overridden where needed,
4351so whoever needs control has exactly as much as required.
4353At the same time, ASDF 2 remains compatible
4354with the old magic you may have in your build scripts
4355(using @code{*central-registry*} and
4357to tailor the ASDF configuration to your build automation needs,
4358and also allows for new magic, simpler and more powerful magic.
4360@xref{Controlling where ASDF searches for systems}.
4363@node Usual operations are made easier to the user, Many bugs have been fixed, Source Registry Configuration, What has changed between ASDF 1 and ASDF 2?
4364@subsection Usual operations are made easier to the user
4366In ASDF 1, you had to use the awkward syntax
4367@code{(asdf:oos 'asdf:load-op :foo)}
4368to load a system,
4369and similarly for @code{compile-op}, @code{test-op}.
4371In ASDF 2, you can use shortcuts for the usual operations:
4372@code{(asdf:load-system :foo)}, and
4373similarly for @code{compile-system}, @code{test-system}.
4376@node Many bugs have been fixed, ASDF itself is versioned, Usual operations are made easier to the user, What has changed between ASDF 1 and ASDF 2?
4377@subsection Many bugs have been fixed
4379The following issues and many others have been fixed:
4383The infamous TRAVERSE function has been revamped completely
4384between ASDF 1 and ASDF 2, with many bugs squashed.
4385In particular, dependencies were not correctly propagated
4386across modules but now are.
4387It has been completely rewritten many times over
4388between ASDF 2.000 and ASDF 3,
4389with fundamental issues in the original model being fixed.
4390Timestamps were not propagated at all, and now are.
4391The internal model of how actions depend on each other
4392is now both consistent and complete.
4393The @code{:version} and
4394the @code{:force (system1 .. systemN)} feature have been fixed.
4397Performance has been notably improved for large systems
4398(say with thousands of components) by using
4399hash-tables instead of linear search,
4400and linear-time list accumulation
4401instead of quadratic-time recursive appends.
4404Many features used to not be portable,
4405especially where pathnames were involved.
4406Windows support was notably quirky because of such non-portability.
4409The internal test suite used to massively fail on many implementations.
4410While still incomplete, it now fully passes
4411on all implementations supported by the test suite,
4412except for GCL (due to GCL bugs).
4415Support was lacking for some implementations.
4416ABCL and GCL were notably wholly broken.
4417ECL extensions were not integrated with ASDF release.
4420The documentation was grossly out of date.
4422@end itemize
4425@node ASDF itself is versioned, ASDF can be upgraded, Many bugs have been fixed, What has changed between ASDF 1 and ASDF 2?
4426@subsection ASDF itself is versioned
4428Between new features, old bugs fixed, and new bugs introduced,
4429there were various releases of ASDF in the wild,
4430and no simple way to check which release had which feature set.
4431People using or writing systems had to either make worst-case assumptions
4432as to what features were available and worked,
4433or take great pains to have the correct version of ASDF installed.
4435With ASDF 2, we provide a new stable set of working features
4436that everyone can rely on from now on.
4437Use @code{#+asdf2} to detect presence of ASDF 2,
4438@code{(asdf:version-satisfies (asdf:asdf-version) "2.345.67")}
4439to check the availability of a version no earlier than required.
4442@node ASDF can be upgraded, Decoupled release cycle, ASDF itself is versioned, What has changed between ASDF 1 and ASDF 2?
4443@subsection ASDF can be upgraded
4445When an old version of ASDF was loaded,
4446it was very hard to upgrade ASDF in your current image
4447without breaking everything.
4448Instead you had to exit the Lisp process and
4449somehow arrange to start a new one from a simpler image.
4450Something that can't be done from within Lisp,
4451making automation of it difficult,
4452which compounded with difficulty in configuration,
4453made the task quite hard.
4454Yet as we saw before, the task would have been required
4455to not have to live with the worst case or non-portable
4456subset of ASDF features.
4458With ASDF 2, it is easy to upgrade
4459from ASDF 2 to later versions from within Lisp,
4460and not too hard to upgrade from ASDF 1 to ASDF 2 from within Lisp.
4461We support hot upgrade of ASDF and any breakage is a bug
4462that we will do our best to fix.
4463There are still limitations on upgrade, though,
4464most notably the fact that after you upgrade ASDF,
4465you must also reload or upgrade all ASDF extensions.
4467@node Decoupled release cycle, Pitfalls of the transition to ASDF 2, ASDF can be upgraded, What has changed between ASDF 1 and ASDF 2?
4468@subsection Decoupled release cycle
4470When vendors were releasing their Lisp implementations with ASDF,
4471they had to basically never change version
4472because neither upgrade nor downgrade was possible
4473without breaking something for someone,
4474and no obvious upgrade path was visible and recommendable.
4476With ASDF 2, upgrade is possible, easy and can be recommended.
4477This means that vendors can safely ship a recent version of ASDF,
4478confident that if a user isn't fully satisfied,
4479he can easily upgrade ASDF and deal
4480with a supported recent version of it.
4481This means that release cycles will be causally decoupled,
4482the practical consequence of which will mean faster convergence
4483towards the latest version for everyone.
4486@node Pitfalls of the transition to ASDF 2,  , Decoupled release cycle, What has changed between ASDF 1 and ASDF 2?
4487@subsection Pitfalls of the transition to ASDF 2
4489The main pitfalls in upgrading to ASDF 2 seem to be related
4490to the output translation mechanism.
4495Output translations is enabled by default. This may surprise some users,
4496most of them in pleasant way (we hope), a few of them in an unpleasant way.
4497It is trivial to disable output translations.
4498@xref{FAQ,,``How can I wholly disable the compiler output cache?''}.
4501Some systems in the large have been known
4502not to play well with output translations.
4503They were relatively easy to fix.
4504Once again, it is also easy to disable output translations,
4505or to override its configuration.
4508The new ASDF output translations are incompatible with ASDF-Binary-Locations.
4509They replace A-B-L, and there is compatibility mode to emulate
4510your previous A-B-L configuration.
4511See @code{enable-asdf-binary-locations-compatibility} in
4512@pxref{Controlling where ASDF saves compiled files,,Backward Compatibility}.
4513But thou shalt not load ABL on top of ASDF 2.
4515@end itemize
4517Other issues include the following:
4522ASDF pathname designators are now specified
4523in places where they were unspecified,
4524and a few small adjustments have to be made to some non-portable defsystems.
4525Notably, in the @code{:pathname} argument
4526to a @code{defsystem} and its components,
4527a logical pathname (or implementation-dependent hierarchical pathname)
4528must now be specified with @code{#p} syntax
4529where the namestring might have previously sufficed;
4530moreover when evaluation is desired @code{#.} must be used,
4531where it wasn't necessary in the toplevel @code{:pathname} argument
4532(but necessary in other @code{:pathname} arguments).
4535There is a slight performance bug, notably on SBCL,
4536when initially searching for @file{asd} files,
4537the implicit @code{(directory "/configured/path/**/*.asd")}
4538for every configured path @code{(:tree "/configured/path/")}
4539in your @code{source-registry} configuration can cause a slight pause.
4540Try to @code{(time (asdf:initialize-source-registry))}
4541to see how bad it is or isn't on your system.
4542If you insist on not having this pause,
4543you can avoid the pause by overriding the default source-registry configuration
4544and not use any deep @code{:tree} entry but only @code{:directory} entries
4545or shallow @code{:tree} entries.
4546Or you can fix your implementation to not be quite that slow
4547when recursing through directories.
4548@emph{Update}: This performance bug fixed the hard way in 2.010.
4551On Windows, only LispWorks supports proper default configuration pathnames
4552based on the Windows registry.
4553Other implementations make do with environment variables,
4554that you may have to define yourself
4555if you're using an older version of Windows.
4556Windows support is somewhat less tested than Unix support.
4557Please help report and fix bugs.
4558@emph{Update}: As of ASDF 2.21, all implementations
4559should now use the same proper default configuration pathnames
4560and they should actually work, though they haven't all been tested.
4563The mechanism by which one customizes a system so that Lisp files
4564may use a different extension from the default @file{.lisp} has changed.
4565Previously, the pathname for a component
4566was lazily computed when operating on a system,
4567and you would
4568@code{(defmethod source-file-type ((component cl-source-file) (system (eql (find-system 'foo))))
4569  (declare (ignorable component system)) "lis")}.
4570Now, the pathname for a component is eagerly computed when defining the system,
4571and instead you will @code{(defclass cl-source-file.lis (cl-source-file) ((type :initform "lis")))}
4572and use @code{:default-component-class cl-source-file.lis}
4573as argument to @code{defsystem},
4574as detailed in a @pxref{FAQ,How do I create a system definition where all the source files have a .cl extension?} below.
4576@findex source-file-type
4579@end itemize
4582@node Issues with installing the proper version of ASDF, Issues with configuring ASDF, What has changed between ASDF 1 and ASDF 2?, FAQ
4583@section Issues with installing the proper version of ASDF
4586* My Common Lisp implementation comes with an outdated version of ASDF. What to do?::
4587* I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?::
4588@end menu
4590@node My Common Lisp implementation comes with an outdated version of ASDF. What to do?, I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?, Issues with installing the proper version of ASDF, Issues with installing the proper version of ASDF
4591@subsection ``My Common Lisp implementation comes with an outdated version of ASDF. What to do?''
4593We recommend you upgrade ASDF.
4594@xref{Loading ASDF,,Upgrading ASDF}.
4596If this does not work, it is a bug, and you should report it.
4597@xref{FAQ, report-bugs, Where do I report a bug}.
4598In the meantime, you can load @file{asdf.lisp} directly.
4599@xref{Loading ASDF,Loading an otherwise installed ASDF}.
4602@node I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?,  , My Common Lisp implementation comes with an outdated version of ASDF. What to do?, Issues with installing the proper version of ASDF
4603@subsection ``I'm a Common Lisp implementation vendor. When and how should I upgrade ASDF?''
4605Since ASDF 2,
4606it should always be a good time to upgrade to a recent version of ASDF.
4607You may consult with the maintainer for which specific version they recommend,
4608but the latest @code{release} should be correct.
4609Though we do try to test ASDF releases against all implementations that we can,
4610we may not be testing against all variants of your implementation,
4611and we may not be running enough tests;
4612we trust you to thoroughly test it with your own implementation
4613before you release it.
4614If there are any issues with the current release,
4615it's a bug that you should report upstream and that we will fix ASAP.
4617As to how to include ASDF, we recommend the following:
4621If ASDF isn't loaded yet, then @code{(require "asdf")}
4622should load the version of ASDF that is bundled with your system.
4623If possible so should @code{(require "ASDF")}.
4624You may have it load some other version configured by the user,
4625if you allow such configuration.
4628If your system provides a mechanism to hook into @code{CL:REQUIRE},
4629then it would be nice to add ASDF to this hook the same way that
4631Please send us appropriate code to this end.
4634You may, like SBCL, have ASDF be implicitly used to require systems
4635that are bundled with your Lisp distribution.
4636If you do have a few magic systems that come with your implementation
4637in a precompiled way such that one should only use the binary version
4638that goes with your distribution, like SBCL does,
4639then you should add them in the beginning of @code{wrapping-source-registry}.
4642If you have magic systems as above, like SBCL does,
4643then we explicitly ask you to @emph{NOT} distribute
4644@file{asdf.asd} as part of those magic systems.
4645You should still include the file @file{asdf.lisp} in your source distribution
4646and precompile it in your binary distribution,
4647but @file{asdf.asd} if included at all,
4648should be secluded from the magic systems,
4649in a separate file hierarchy.
4650Alternatively, you may provide the system
4651after renaming it and its @file{.asd} file to e.g.
4652@code{asdf-ecl} and @file{asdf-ecl.asd}, or
4653@code{sb-asdf} and @file{sb-asdf.asd}.
4654Indeed, if you made @file{asdf.asd} a magic system,
4655then users would no longer be able to upgrade ASDF using ASDF itself
4656to some version of their preference that
4657they maintain independently from your Lisp distribution.
4660If you do not have any such magic systems, or have other non-magic systems
4661that you want to bundle with your implementation,
4662then you may add them to the @code{wrapping-source-registry},
4663and you are welcome to include @file{asdf.asd} amongst them.
4664Non-magic systems should be at the back of the @code{wrapping-source-registry}
4665while magic systems are at the front.
4668Since ASDF 3, the library UIOP comes transcluded in ASDF.
4669But if you want to be nice to users who care for UIOP but not for ASDF,
4670you may package UIOP separately,
4671so that one may @code{(require "uiop")} and not load ASDF,
4672or one may @code{(require "asdf")}
4673which would implicitly require and load the former.
4676Please send us upstream any patches you make to ASDF itself,
4677so we can merge them back in for the benefit of your users
4678when they upgrade to the upstream version.
4680@end itemize
4684@node Issues with configuring ASDF, Issues with using and extending ASDF to define systems, Issues with installing the proper version of ASDF, FAQ
4685@section Issues with configuring ASDF
4688* How can I customize where fasl files are stored?::
4689* How can I wholly disable the compiler output cache?::
4690@end menu
4692@node How can I customize where fasl files are stored?, How can I wholly disable the compiler output cache?, Issues with configuring ASDF, Issues with configuring ASDF
4693@subsection ``How can I customize where fasl files are stored?''
4695@xref{Controlling where ASDF saves compiled files}.
4697Note that in the past there was an add-on to ASDF called
4698@code{ASDF-binary-locations}, developed by Gary King.
4699That add-on has been merged into ASDF proper,
4700then superseded by the @code{asdf-output-translations} facility.
4702Note that use of @code{asdf-output-translations}
4703can interfere with one aspect of your systems
4704--- if your system uses @code{*load-truename*} to find files
4705(e.g., if you have some data files stored with your program),
4706then the relocation that this ASDF customization performs
4707is likely to interfere.
4708Use @code{asdf:system-relative-pathname} to locate a file
4709in the source directory of some system, and
4710use @code{asdf:apply-output-translations} to locate a file
4711whose pathname has been translated by the facility.
4713@node How can I wholly disable the compiler output cache?,  , How can I customize where fasl files are stored?, Issues with configuring ASDF
4714@subsection ``How can I wholly disable the compiler output cache?''
4716To permanently disable the compiler output cache
4717for all future runs of ASDF, you can:
4720mkdir -p ~/.config/common-lisp/asdf-output-translations.conf.d/
4721echo ':disable-cache' > ~/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
4722@end example
4724This assumes that you didn't otherwise configure the ASDF files
4725(if you did, edit them again),
4726and don't somehow override the configuration at runtime
4727with a shell variable (see below) or some other runtime command
4728(e.g. some call to @code{asdf:initialize-output-translations}).
4730To disable the compiler output cache in Lisp processes
4731run by your current shell, try (assuming @code{bash} or @code{zsh})
4732(on Unix and cygwin only):
4736@end example
4738To disable the compiler output cache just in the current Lisp process,
4739use (after loading ASDF but before using it):
4743@end example
4745@node Issues with using and extending ASDF to define systems, ASDF development FAQs, Issues with configuring ASDF, FAQ
4746@section Issues with using and extending ASDF to define systems
4749* How can I cater for unit-testing in my system?::
4750* How can I cater for documentation generation in my system?::
4751* How can I maintain non-Lisp (e.g. C) source files?::
4752* I want to put my module's files at the top level.  How do I do this?::
4753* How do I create a system definition where all the source files have a .cl extension?::
4754@end menu
4756@node How can I cater for unit-testing in my system?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems, Issues with using and extending ASDF to define systems
4757@subsection ``How can I cater for unit-testing in my system?''
4759ASDF provides a predefined test operation, @code{test-op}.
4760@xref{Predefined operations of ASDF, test-op}.
4761The test operation, however, is largely left to the system definer to specify.
4762@code{test-op} has been
4763a topic of considerable discussion on the
4764@url{,asdf-devel mailing list},
4765and on the
4766@url{,launchpad bug-tracker}.
4768Here are some guidelines:
4772For a given system, @var{foo}, you will want to define a corresponding
4773test system, such as @var{foo-test}.  The reason that you will want this
4774separate system is that ASDF does not out of the box supply components
4775that are conditionally loaded.  So if you want to have source files
4776(with the test definitions) that will not be loaded except when testing,
4777they should be put elsewhere.
4780The @var{foo-test} system can be defined in an asd file of its own or
4781together with @var{foo}.  An aesthetic preference against cluttering up
4782the filesystem with extra asd files should be balanced against the
4783question of whether one might want to directly load @var{foo-test}.
4784Typically one would not want to do this except in early stages of
4788Record that testing is implemented by @var{foo-test}.  For example:
4790(defsystem @var{foo}
4791   :in-order-to ((test-op (test-op @var{foo-test})))
4792   ....)
4794(defsystem @var{foo-test}
4795   :depends-on (@var{foo} @var{my-test-library} ...)
4796   ....)
4797@end example
4798@end itemize
4800This procedure will allow you to support users who do not wish to
4801install your test framework.
4803One oddity of ASDF is that @code{operate} (@pxref{Operations,operate})
4804does not return a value.  So in current versions of ASDF there is no
4805reliable programmatic means of determining whether or not a set of tests
4806has passed, or which tests have failed.  The user must simply read the
4807console output.  This limitation has been the subject of much
4810@node How can I cater for documentation generation in my system?, How can I maintain non-Lisp (e.g. C) source files?, How can I cater for unit-testing in my system?, Issues with using and extending ASDF to define systems
4811@subsection ``How can I cater for documentation generation in my system?''
4813Various ASDF extensions provide some kind of @code{doc-op} operation.
4814See also @url{}.
4817@node How can I maintain non-Lisp (e.g. C) source files?, I want to put my module's files at the top level.  How do I do this?, How can I cater for documentation generation in my system?, Issues with using and extending ASDF to define systems
4818@subsection ``How can I maintain non-Lisp (e.g. C) source files?''
4820See @code{cffi}'s @code{cffi-grovel}.
4825@node I want to put my module's files at the top level.  How do I do this?, How do I create a system definition where all the source files have a .cl extension?, How can I maintain non-Lisp (e.g. C) source files?, Issues with using and extending ASDF to define systems
4826@subsection ``I want to put my module's files at the top level.  How do I do this?''
4828By default, the files contained in an asdf module go
4829in a subdirectory with the same name as the module.
4830However, this can be overridden by adding a @code{:pathname ""} argument
4831to the module description.
4832For example, here is how it could be done
4833in the spatial-trees ASDF system definition for ASDF 2:
4836(asdf:defsystem :spatial-trees
4837  :components
4838  ((:module base
4839            :pathname ""
4840            :components
4841            ((:file "package")
4842             (:file "basedefs" :depends-on ("package"))
4843             (:file "rectangles" :depends-on ("package"))))
4844   (:module tree-impls
4845            :depends-on (base)
4846            :pathname ""
4847            :components
4848            ((:file "r-trees")
4849             (:file "greene-trees" :depends-on ("r-trees"))
4850             (:file "rstar-trees" :depends-on ("r-trees"))
4851             (:file "rplus-trees" :depends-on ("r-trees"))
4852             (:file "x-trees" :depends-on ("r-trees" "rstar-trees"))))
4853   (:module viz
4854            :depends-on (base)
4855            :pathname ""
4856            :components
4857            ((:static-file "spatial-tree-viz.lisp")))
4858   (:module tests
4859            :depends-on (base)
4860            :pathname ""
4861            :components
4862            ((:static-file "spatial-tree-test.lisp")))
4863   (:static-file "LICENCE")
4864   (:static-file "TODO")))
4865@end example
4867All of the files in the @code{tree-impls} module are at the top level,
4868instead of in a @file{tree-impls/} subdirectory.
4870Note that the argument to @code{:pathname} can be either a pathname object or a string.
4871A pathname object can be constructed with the @file{#p"foo/bar/"} syntax,
4872but this is discouraged because the results of parsing a namestring are not portable.
4873A pathname can only be portably constructed with such syntax as
4874@code{#.(make-pathname :directory '(:relative "foo" "bar"))},
4875and similarly the current directory can only be portably specified as
4876@code{#.(make-pathname :directory '(:relative))}.
4877However, as of ASDF 2, you can portably use a string to denote a pathname.
4878The string will be parsed as a @code{/}-separated path from the current directory,
4879such that the empty string @code{""} denotes the current directory, and
4880@code{"foo/bar"} (no trailing @code{/} required in the case of modules)
4881portably denotes the same subdirectory as above.
4882When files are specified, the last @code{/}-separated component is interpreted
4883either as the name component of a pathname
4884(if the component class specifies a pathname type),
4885or as a name component plus optional dot-separated type component
4886(if the component class doesn't specifies a pathname type).
4888@node How do I create a system definition where all the source files have a .cl extension?,  , I want to put my module's files at the top level.  How do I do this?, Issues with using and extending ASDF to define systems
4889@subsection How do I create a system definition where all the source files have a .cl extension?
4891Starting with ASDF 2.014.14, you may just pass
4892the builtin class @code{} as
4893the @code{:default-component-class} argument to @code{defsystem}:
4896(defsystem my-cl-system
4897  :default-component-class
4898  ...)
4899@end lisp
4901Another builtin class @code{cl-source-file.lsp} is offered
4902for files ending in @file{.lsp}.
4904If you want to use a different extension
4905for which ASDF doesn't provide builtin support,
4906or want to support versions of ASDF
4907earlier than 2.014.14 (but later than 2.000),
4908you can define a class as follows:
4911;; Prologue: make sure we're using a sane package.
4912(defpackage :my-asdf-extension
4913   (:use :asdf :common-lisp)
4914   (:export #:cl-source-file.lis))
4915(in-package :my-asdf-extension)
4917(defclass cl-source-file.lis (cl-source-file)
4918  ((type :initform "lis")))
4919@end lisp
4921Then you can use it as follows:
4923(defsystem my-cl-system
4924  :default-component-class my-asdf-extension:cl-source-file.lis
4925  ...)
4926@end lisp
4928Of course, if you're in the same package, e.g. in the same file,
4929you won't need to use the package qualifier before @code{cl-source-file.lis}.
4930Actually, if all you're doing is defining this class
4931and using it in the same file without other fancy definitions,
4932you might skip package complications:
4935(in-package :asdf)
4936(defclass cl-source-file.lis (cl-source-file)
4937   ((type :initform "lis")))
4938(defsystem my-cl-system
4939  :default-component-class cl-source-file.lis
4940  ...)
4941@end lisp
4943It is possible to achieve the same effect
4944in a way that supports both ASDF 1 and ASDF 2,
4945but really, friends don't let friends use ASDF 1.
4946Please upgrade to ASDF 3.
4947In short, though: do same as above, but
4948@emph{before} you use the class in a @code{defsystem},
4949you also define the following method:
4952(defmethod source-file-type ((f cl-source-file.lis) (s system))
4953  (declare (ignorable f s))
4954  "lis")
4955@end lisp
4957@node ASDF development FAQs,  , Issues with using and extending ASDF to define systems, FAQ
4958@section ASDF development FAQs
4961* How do run the tests interactively in a REPL?::
4962@end menu
4964@node How do run the tests interactively in a REPL?,  , ASDF development FAQs, ASDF development FAQs
4965@subsection How do run the tests interactively in a REPL?
4967This not-so-frequently asked question is primarily for ASDF developers,
4968but those who encounter an unexpected error in some test may be
4969interested, too.
4971Here's the procedure for experimenting with tests in a REPL:
4973(load "script-support.lisp")
4974(in-package :asdf-test)
4975(compile-load-asdf) ; there are a number of other functions to load ASDF
4976;; experiment with test code from a .script file...
4977@end example
4980@comment FIXME: Add a FAQ about how to use a new system class...
4982@comment  node-name,  next,  previous,  up
4983@node  Ongoing Work, Bibliography, FAQ, Top
4984@unnumbered Ongoing Work
4985For an active list of things to be done,
4986see the @file{TODO} file in the source repository.
4988Also, bugs that are now tracked on launchpad:
4991@node Bibliography, Concept Index, Ongoing Work, Top
4992@unnumbered Bibliography
4995@item Zach Beane:
4996  ``Quicklisp'', 2011.
4997  The Quicklisp blog and Xach's livejournal contain information on Quicklisp.
4998  @url{}
4999  @url{}
5000@item Francois-Rene Rideau and Robert Goldman:
5001  ``Evolving ASDF: More Cooperation, Less Coordination'', 2010.
5002  This article describes the main issues solved by ASDF 2.
5003  @url{}
5004  @url{}
5005@item Francois-Rene Rideau and Spencer Brody:
5006  ``XCVB: an eXtensible Component Verifier and Builder for Common Lisp'', 2009.
5007  This article describes XCVB, a proposed competitor for ASDF,
5008  many ideas of which have been incorporated into ASDF 2 and 3,
5009  though many other of which still haven't.
5010  @url{}
5011@item Dan Barlow: ``ASDF Manual'', 2004
5012  Older versions of this document from the days of ASDF 1;
5013  they include ideas laid down by Dan Barlow,
5014  and comparisons with older defsystems (@code{mk-defsystem})
5015  and defsystem (@code{defsystem-4}, kmp's Memo 801).
5016@item Marco Antoniotti and Peter Van Eynde:
5017 ``@code{DEFSYSTEM}: A @code{make} for Common Lisp, A Thoughtful Re-Implementation of an Old Idea'', 2002.
5018  The @file{defsystem-4} proposal available in the CLOCC repository.
5019@item Mark Kantrovitz: ``Defsystem: A Portable Make Facility for Common Lisp'', 1990.
5020  The classic @file{mk-defsystem}, later variants of which
5021  are available in the CLOCC repository as @code{defsystem-3.x}.
5022@item Richard Elliot Robbins:
5023  ``BUILD: A Tool for Maintaining Consistency in Modular Systems'', MIT AI TR 874, 1985.
5024  @url{}
5025@item Kent M. Pitman (kmp): ``The Description of Large Systems'', MIT AI Memo 801, 1984.
5026  Available in updated-for-CL form on the web at
5027  @url{}
5028@item Dan Weinreb and David Moon:
5029  ``Lisp Machine Manual'', MIT, 1981.
5030  The famous CHINE NUAL describes one of the earliest variants of DEFSYSTEM.
5031  @url{}
5032@item Drew McDermott: ``A Framework for Maintaining the Coherence of a
5033  Running Lisp,'' International Lisp Conference, 2005, available in
5034  pre-print form at @url{}.
5035@end itemize
5038@node Concept Index, Function and Class Index, Bibliography, Top
5039@unnumbered Concept Index
5041@printindex cp
5043@node Function and Class Index, Variable Index, Concept Index, Top
5044@unnumbered Function and Class Index
5046@printindex fn
5048@node Variable Index,  , Function and Class Index, Top
5049@unnumbered Variable Index
5051@printindex vr
Note: See TracBrowser for help on using the repository browser.