source: trunk/abcl/doc/manual/abcl.tex @ 13798

Last change on this file since 13798 was 13798, checked in by Mark Evenson, 12 years ago

Merge User manual from what we tagged as 1.0.1

File size: 44.2 KB
Line 
1% -*- mode: latex; -*-
2% http://en.wikibooks.org/wiki/LaTeX/
3\documentclass[10pt]{book}
4\usepackage{abcl}
5
6\begin{document}
7\title{A Manual for Armed Bear Common Lisp}
8\date{January 25, 2012}
9\author{Mark~Evenson, Erik~H\"{u}lsmann, Alessio~Stalla, Ville~Voutilainen}
10
11\maketitle
12
13\tableofcontents
14
15\chapter{Introduction}
16
17Armed Bear is a conforming implementation of the ANSI Common Lisp
18standard (see \ref{chapter:conformance} on page
19\pageref{chapter:conformance} which states the details of the
20conformance level.  This manual documents the Armed Bear Common Lisp
21implementation for users of the system.
22
23\subsection{Version}
24This manual corresponds to abcl-1.1.0-dev, as yet unreleased..
25
26\subsection{License}
27
28The implementation is licensed under the terms of the GPL v2 of June
291991 with the ``classpath-exception'' that makes its deployment in
30commercial settings quite reasonable.  The license is viral in the
31sense that if you change the implementation, and redistribute those
32changes, you are required to provide the source to those changes back
33to be merged with the public trunk.
34
35\subsection{Contributors}
36
37% TODO format this better, optionally link to URI
38
39\begin{itemize}
40\item Philipp Marek
41\texttt{Thanks for the markup}
42\item Douglas Miles
43\texttt{Thanks for the whacky IKVM stuff and keeping the flame alive
44  in the dark years.}
45
46
47\item Alan Ruttenberg
48\texttt{Thanks for JSS.}
49
50\item and of course
51\emph{Peter Graves}
52\end{itemize}
53
54
55\chapter{Running}
56
57\textsc{ABCL} is packaged as a single jar file usually named either
58``abcl.jar'' or possibly``abcl-1.0.1.jar'' if one is using a versioned
59package on the local filesystem from your system vendor.  This byte
60archive can be executed under the control of a suitable JVM \footnote
61{Java Virtual Machine} by using the ``-jar'' option to parse the
62manifest, and select the class named therein
63``\code{org.armedbear.lisp.Main}'' for execution, viz:
64
65\begin{listing-shell}
66  cmd$ java -jar abcl.jar
67\end{listing-shell}
68
69\emph{N.b.} for the proceeding command to work, the ``java'' executable needs
70to be in your path.
71
72To make it easier to facilitate the use of ABCL in tool chains (such
73as SLIME \footnote{SLIME is the Superior Lisp Mode for Interaction
74  under Emacs}) the invocation is wrapped in a Bourne shell script
75under \textsc{UNIX} or a \textsc{DOS} command script under Windows so that ABCL may be
76executed simply as:
77
78\begin{listing-shell}
79  cmd$ abcl
80\end{listing-shell}
81
82\section{Options}
83
84ABCL supports the following command line options:
85
86\begin{verbatim}
87--help
88    Displays this message.
89--noinform
90    Suppresses the printing of startup information and banner.
91--noinit
92    Suppresses the loading of the '~/.abclrc' startup file.
93--nosystem
94    Suppresses loading the 'system.lisp' customization file.
95--eval <FORM>
96    Evaluates the <FORM> before initializing REPL.
97--load <FILE>
98    Loads the file <FILE> before initializing REPL.
99--load-system-file <FILE>
100    Loads the system file <FILE> before initializing REPL.
101--batch
102    The process evaluates forms specified by arguments and possibly by those
103    by those in the intialization file '~/.abcl', and then exits.
104
105The occurance of '--' copies the remaining arguments, unprocessed, into
106the variable EXTENSIONS:*COMMAND-LINE-ARGUMENT-LIST*.
107\end{verbatim}
108
109All of the command line arguments which follow the occurrence of ``----''
110are passed into a list bound to the EXT:*COMMAND-LINE-ARGUMENT-LIST*
111variable.
112
113\section{Initialization}
114
115If the \textsc{ABCL} process is started without the ``--noinit'' flag, it
116attempts to load a file named ``.abclrc'' located in the user's home
117directory and then interpret its contents. 
118
119The user's home directory is determined by the value of the JVM system
120property ``user.home''.  This value may--or may not--correspond to the
121value of the HOME system environment variable at the discretion of the
122JVM implementation that \textsc{ABCL} finds itself hosted upon.
123
124\chapter{Conformance}
125\label{chapter:conformance}
126
127\section{ANSI Common Lisp}
128\textsc{ABCL} is currently a (non)-conforming ANSI Common Lisp
129implementation due to the following known issues:
130
131\begin{itemize}
132  \item The generic function signatures of the DOCUMENTATION symbol do
133    not match the CLHS.
134  \item The TIME form does not return a proper VALUES environment to
135    its caller.
136\end{itemize}
137
138Somewhat confusingly, this statement of non-conformance in the
139accompanying user documentation fulfills the requirements that
140\textsc{ABCL} is a conforming ANSI Common Lisp implementation
141according to the CLHS \footnote{Common Lisp Hyperspec language
142  reference document.}.  Clarifications to this point are solicited.
143
144ABCL aims to be be a fully conforming ANSI Common Lisp implementation.
145Any other behavior should be reported as a bug.
146
147\section{Contemporary Common Lisp}
148In addition to ANSI conformance, \textsc{ABCL} strives to implement features
149expected of a contemporary Common Lisp \footnote{i.e. a Lisp of the
150  post 2005 Renaissance}
151
152\subsection{Deficiencies}
153The following known problems detract from \textsc{ABCL} being a proper
154contemporary Common Lisp.
155\begin{itemize}
156
157  \item An incomplete implementation of interactive debugging
158    mechanisms namely a no-op version of
159    \code{STEP} \footnote{Somewhat surprisingly allowed by
160      \textsc{ANSI}}, the inability to inspect local variables in a
161    given call frame, and the inability to resume a halted computation
162    at an arbitrarily selected call frame.
163
164  \item An incomplete implementation of a properly named metaobject
165    protocol (c.f. the (A)MOP \footnote{The Art of the  Metaobject Protocol} specification)
166
167    % N.b.
168    % TODO go through AMOP with symbols, starting by looking for
169    % matching function signature.
170    % XXX is this really blocking ANSI conformance?  Answer: we have
171    % to start with such a ``census'' to determine what we have.
172
173  \item Incomplete streams abstraction, in that \textsc{ABCL} needs suitable
174    abstraction between ANSI and Gray streams.  The streams could be
175    optimized to the JVM NIO abstractions at great profit for binary
176    byte-level manipulations.
177   
178  \item Incomplete documentation (missing docstrings from exported
179    symbols and the draft status of the User Manual).
180
181\end{itemize}
182
183\chapter{Interaction with Hosting JVM}
184
185%  Plan of Attack
186%
187% describe calling Java from Lisp, and calling Lisp from Java,
188% probably in two separate sections.  Presumably, we can partition our
189% audience into those who are more comfortable with Java, and those
190% that are more comforable with Lisp
191
192The Armed Bear Common Lisp implementation is hosted on a Java Virtual
193Machine.  This chapter describes the mechanisms by which the
194implementation interacts with that hosting mechanism.
195
196\section{Lisp to Java}
197
198\textsc{ABCL} offers a number of mechanisms to interact with Java from its
199Lisp environment. It allows calling both instance and static methods
200of Java objects, manipulation of instance and static fields on Java
201objects, and construction of new Java objects.
202
203When calling Java routines, some values will automatically be
204converted by the FFI \footnote{FFI stands for Foreign Function
205  Interface which is the term of art which describes how a Lisp
206  implementation encapsulates invocation in other languages.}  from
207Lisp values to Java values. These conversions typically apply to
208strings, integers and floats. Other values need to be converted to
209their Java equivalents by the programmer before calling the Java
210object method. Java values returned to Lisp are also generally
211converted back to their Lisp counterparts. Some operators make an
212exception to this rule and do not perform any conversion; those are
213the ``raw'' counterparts of certain FFI functions and are recognizable
214by their name ending with \code{-RAW}.
215
216\subsection{Low-level Java API}
217
218We define a higher level Java API in the topic:Higher level Java JSS
219package developed by Alan Ruttenberg which is available in the
220\code{contrib/} directory, see the  . This package is
221described later in this document, see \ref{section:jss} on page
222\pageref{section:jss}.  This section covers the lower level API
223directly available after evaluating \code{(require 'JAVA)}.
224
225\subsubsection{Calling Java Object Methods}
226
227There are two ways to call a Java object method in the low-level (basic) API:
228
229\begin{itemize}
230\item Call a specific method reference (which was previously acquired)
231\item Dynamic dispatch using the method name and
232  the call-specific arguments provided by finding the
233  \ref{section:Parameter matching for FFI dynamic dispatch}{best match}.
234\end{itemize}
235
236The dynamic dispatch variant is discussed in the next section.
237
238\code{JAVA:JMETHOD} is used to acquire a specific method reference.
239The function takes at two or more arguments. The first is Java class designator
240(a \code{JAVA:JAVA-CLASS} object returned by \code{JAVA:JCLASS} or a string naming
241a Java class). The second is a string naming the method.
242
243Any arguments beyond the first two should be strings naming Java classes with
244one exception as listed in the next paragraph. These
245classes specify the types of the arguments for the method to be returned.
246
247There's additional calling convention to the \code{JAVA:JMETHOD} function:
248When the method is called with three parameters and the last parameter is an
249integer, the first method by that name and matching number of parameters is
250returned.
251
252Once one has a reference to the method, one may invoke it using
253\code{JAVA:JCALL}, which takes the method as the first argument. The
254second argument is the object instance to call the method on, or
255\code{NIL} in case of a static method.  Any remaining parameters are
256used as the remaining arguments for the call.
257
258\subsubsection{Calling Java object methods: dynamic dispatch}
259
260The second way of calling Java object methods is by using dynamic dispatch.
261In this case \code{JAVA:JCALL} is used directly without acquiring a method
262reference first. In this case, the first argument provided to \code{JAVA:JCALL}
263is a string naming the method to be called. The second argument is the instance
264on which the method should be called and any further arguments are used to
265select the best matching method and dispatch the call.
266
267\subsubsection{Dynamic dispatch: Caveats}
268
269Dynamic dispatch is performed by using the Java reflection
270API \footnote{The Java reflection API is found in the
271  \code{java.lang.reflect} package}. Generally the dispatch works
272fine, but there are corner cases where the API does not correctly
273reflect all the details involved in calling a Java method. An example
274is the following Java code:
275
276\begin{listing-java}
277ZipFile jar = new ZipFile("/path/to/some.jar");
278Object els = jar.entries();
279Method method = els.getClass().getMethod("hasMoreElements");
280method.invoke(els);
281\end{listing-java}
282
283even though the method \code{hasMoreElements()} is public in \code{Enumeration},
284the above code fails with
285
286\begin{listing-java}
287java.lang.IllegalAccessException: Class ... can
288not access a member of class java.util.zip.ZipFile\$2 with modifiers
289"public"
290       at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
291       at java.lang.reflect.Method.invoke(Method.java:583)
292       at ...
293\end{listing-java}
294
295because the method has been overridden by a non-public class and the
296reflection API, unlike javac, is not able to handle such a case.
297
298While code like that is uncommon in Java, it is typical of ABCL's FFI
299calls. The code above corresponds to the following Lisp code:
300
301\begin{listing-lisp}
302(let ((jar (jnew "java.util.zip.ZipFile" "/path/to/some.jar")))
303  (let ((els (jcall "entries" jar)))
304    (jcall "hasMoreElements" els)))
305\end{listing-lisp}
306
307except that the dynamic dispatch part is not shown.
308
309To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
310field representing the ``intended class'' of the object. That is the class
311that is used first by \code{JAVA:JCALL} and similar to resolve methods;
312the actual class of the object is only tried if the method is not found
313in the intended class. Of course, the intended class is always a super-class
314of the actual class - in the worst case, they coincide. The intended class
315is deduced by the return type of the method that originally returned
316the Java object; in the case above, the intended class of \code{ELS}
317is \code{java.util.Enumeration} because that's the return type of
318the \code{entries} method.
319
320While this strategy is generally effective, there are cases where the
321intended class becomes too broad to be useful. The typical example
322is the extraction of an element from a collection, since methods in
323the collection API erase all types to \code{Object}. The user can
324always force a more specific intended class by using the \code{JAVA:JCOERCE}
325operator.
326
327% \begin{itemize}
328% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
329% \item The Java FFI presents a Lisp package (JAVA) with many useful
330%   symbols for manipulating the artifacts of expectation on the JVM,
331%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
332%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
333%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
334% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
335%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
336%   methods, and additional convenience functions.
337% \item Java classes and libraries may be dynamically added to the
338%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
339% \end{itemize}
340
341\subsubsection{Calling Java class static methods}
342
343Like with non-static methods, references to static methods can be acquired
344by using the \code{JAVA:JMETHOD} primitive. In order to call this method,
345it's not possible to use the \code{JAVA:JCALL} primitive however: there's a
346separate API to retrieve a reference to static methods. This
347primitive is called \code{JAVA:JSTATIC}.
348
349Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
350passing the name of the method as a string instead of passing a method reference.
351The parameter values should be values to pass in the function call instead of
352a specification of classes for each parameter.
353
354\subsubsection{Parameter matching for FFI dynamic dispatch}
355
356The algorithm used to resolve the best matching method given the name
357and the arguments' types is the same as described in the Java Language
358Specification. Any deviation should be reported as a bug.
359
360% ###TODO reference to correct JLS section
361
362\subsubsection{Instantiating Java objects}
363
364Java objects can be instantiated (created) from Lisp by calling
365a constructor from the class of the object to be created. The same way
366\code{JAVA:JMETHOD} is used to acquire a method reference, the
367\code{JAVA:JCONSTRUCTOR} primitive can be used to acquire a constructor
368reference. It's arguments specify the types of arguments of the constructor
369method the same way as with \code{JAVA:JMETHOD}.
370
371The constructor can't be passed to \code{JAVA:JCALL}, but instead should
372be passed as an argument to \code{JAVA:JNEW}.
373
374\subsubsection{Accessing Java object fields}
375
376Fields in Java objects can be accessed using the getter and setter functions
377\code{JAVA:GETFIELD} and \code{JAVA:PUTFIELD}. This applies to values stored in object
378instances. If you want to access static fields: see the next section.
379
380Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
381an intended class around and values which can be converted to Lisp values will
382be converted.
383
384\subsubsection{Accessing Java static fields}
385
386Static fields in Java objects (class fields) can be accessed using the getter
387and setter functions \code{JAVA:GETSTATIC} and \code{JAVA:PUTSTATIC}. Values
388stored in object instance fields can be accessed as described in the previous
389section.
390
391Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
392an intended class around and values which can be converted to Lisp values will
393be converted.
394
395\section{Lisp from Java}
396
397In order to access the Lisp world from Java, one needs to be aware
398of a few things. The most important ones are listed below.
399
400\begin{itemize}
401\item All Lisp values are descendants of LispObject.java
402\item In order to
403\item Lisp symbols are accessible via either directly referencing the
404  Symbol.java instance or by dynamically introspecting the
405  corresponding Package.java instance.
406\item The Lisp dynamic environment may be saved via
407  \code{LispThread.bindSpecial(Binding)} and restored via
408  \code{LispThread.resetSpecialBindings(Mark)}.
409\item Functions may be executed by invocation of the
410  Function.execute(args [...])
411\end{itemize}
412
413\subsection{Lisp FFI}
414
415FFI stands for "Foreign Function Interface" which is the phase which
416the contemporary Lisp world refers to methods of "calling out" from
417Lisp into "foreign" languages and environments.  This document
418describes the various ways that one interacts with Lisp world of ABCL
419from Java, considering the hosted Lisp as the "Foreign Function" that
420needs to be "Interfaced".
421
422\subsection{Calling Lisp from Java}
423
424Note: As the entire ABCL Lisp system resides in the org.armedbear.lisp
425package the following code snippets do not show the relevant import
426statements in the interest of brevity.  An example of the import
427statement would be
428
429\begin{listing-java}
430  import org.armedbear.lisp.*;
431\end{listing-java}
432
433to potentially import all the JVM symbol from the `org.armedbear.lisp'
434namespace.
435
436Per JVM, there can only ever be a single Lisp interpreter.  This is
437started by calling the static method `Interpreter.createInstance()`.
438
439\begin{listing-java}
440  Interpreter interpreter = Interpreter.createInstance();
441\end{listing-java}
442
443If this method has already been invoked in the lifetime of the current
444Java process it will return null, so if you are writing Java whose
445life-cycle is a bit out of your control (like in a Java servlet), a
446safer invocation pattern might be:
447
448\begin{listing-java}
449  Interpreter interpreter = Interpreter.getInstance();
450  if (interpreter == null) {
451    interpreter = Interpreter.createInstance();
452  }
453\end{listing-java}
454
455
456The Lisp \code{eval} primitive may be simply passed strings for evaluation,
457as follows
458
459\begin{listing-java}
460  String line = "(load \"file.lisp\")";
461  LispObject result = interpreter.eval(line);
462\end{listing-java}
463
464Notice that all possible return values from an arbitrary Lisp
465computation are collapsed into a single return value.  Doing useful
466further computation on the ``LispObject'' depends on knowing what the
467result of the computation might be, usually involves some amount
468of \code{instanceof} introspection, and forms a whole topic to itself
469(c.f. [Introspecting a LispObject])
470
471Using \code{eval} involves the Lisp interpreter.  Lisp functions may
472be directly invoked by Java method calls as follows.  One simply
473locates the package containing the symbol, then obtains a reference to
474the symbol, and then invokes the \code{execute()} method with the
475desired parameters.
476
477\begin{listing-java}
478    interpreter.eval("(defun foo (msg) (format nil \"You told me '~A'~%\" msg))");
479    Package pkg = Packages.findPackage("CL-USER");
480    Symbol foo = pkg.findAccessibleSymbol("FOO");
481    Function fooFunction = (Function)foo.getSymbolFunction();
482    JavaObject parameter = new JavaObject("Lisp is fun!");
483    LispObject result = fooFunction.execute(parameter);
484    // How to get the "naked string value"?
485    System.out.println("The result was " + result.writeToString());
486\end{listing-java}
487
488If one is calling an primitive function in the CL package the syntax
489becomes considerably simpler.  If we can locate the instance of
490definition in the ABCL Java source, we can invoke the symbol directly.
491For instance, to tell if a `LispObject` contains a reference to a symbol.
492
493\begin{listing-java}
494    boolean nullp(LispObject object) {
495      LispObject result = Primitives.NULL.execute(object);
496      if (result == NIL) { // the symbol 'NIL' is explicitly named in the Java
497                           // namespace at ``Symbol.NIL''
498                           // but is always present in the
499                           // local namespace in its unadorned form for
500                           // the convenience of the User.
501        return false;
502      }
503      return true;
504   }
505\end{listing-java}
506
507\subsubsection{Introspecting a LispObject}
508\label{topic:Introspecting a LispObject}
509
510We present various patterns for introspecting an an arbitrary
511`LispObject` which can represent the result of every Lisp evaluation
512into semantics that Java can meaningfully deal with.
513
514\subsubsection{LispObject as \code{boolean}}
515
516If the LispObject a generalized boolean values, one can use
517\code{getBooleanValue()} to convert to Java:
518
519\begin{listing-java}
520     LispObject object = Symbol.NIL;
521     boolean javaValue = object.getBooleanValue();
522\end{listing-java}
523
524Although since in Lisp, any value other than NIL means "true"
525(so-called generalized Boolean), the use of Java equality it quite a
526bit easier to type and more optimal in terms of information it conveys
527to the compiler would be:
528
529\begin{listing-java}
530    boolean javaValue = (object != Symbol.NIL);
531\end{listing-java}
532
533\paragraph{LispObject is a list}
534
535If LispObject is a list, it will have the type `Cons`.  One can then use
536the \code{copyToArray} to make things a bit more suitable for Java
537iteration.
538
539\begin{listing-java}
540    LispObject result = interpreter.eval("'(1 2 4 5)");
541    if (result instanceof Cons) {
542      LispObject array[] = ((Cons)result.copyToArray());
543      ...
544    }
545\end{listing-java}
546
547A more Lispy way to iterated down a list is to use the `cdr()` access
548function just as like one would traverse a list in Lisp:;
549
550\begin{listing-java}
551    LispObject result = interpreter.eval("'(1 2 4 5)");
552    while (result != Symbol.NIL) {
553      doSomething(result.car());
554      result = result.cdr();
555    }
556\end{listing-java}
557
558\subsection{Java Scripting API (JSR-223)}
559
560ABCL can be built with support for JSR-223, which offers a
561language-agnostic API to invoke other languages from Java. The binary
562distribution download-able from ABCL's common-lisp.net home is built
563with JSR-223 support. If you're building ABCL from source on a pre-1.6
564JVM, you need to have a JSR-223 implementation in your CLASSPATH (such
565as Apache Commons BSF 3.x or greater) in order to build ABCL with
566JSR-223 support; otherwise, this feature will not be built.
567
568This section describes the design decisions behind the ABCL JSR-223
569support. It is not a description of what JSR-223 is or a tutorial on
570how to use it. See
571http://trac.common-lisp.net/armedbear/browser/trunk/abcl/examples/jsr-223
572for example usage.
573
574\subsubsection{Conversions}
575
576In general, ABCL's implementation of the JSR-223 API performs implicit
577conversion from Java objects to Lisp objects when invoking Lisp from
578Java, and the opposite when returning values from Java to Lisp. This
579potentially reduces coupling between user code and ABCL. To avoid such
580conversions, wrap the relevant objects in \code{JavaObject} instances.
581
582\subsubsection{Implemented JSR-223 interfaces}
583
584JSR-223 defines three main interfaces, of which two (Invocable and
585Compilable) are optional. ABCL implements all the three interfaces -
586ScriptEngine and the two optional ones - almost completely. While the
587JSR-223 API is not specific to a single scripting language, it was
588designed with languages with a more or less Java-like object model in
589mind: languages such as Javascript, Python, Ruby, which have a concept
590of "class" or "object" with "fields" and "methods". Lisp is a bit
591different, so certain adaptations were made, and in one case a method
592has been left unimplemented since it does not map at all to Lisp.
593
594\subsubsection{The ScriptEngine}
595
596The main interface defined by JSR-223, javax.script.ScriptEngine, is
597implemented by the class
598\code{org.armedbear.lisp.scripting.AbclScriptEngine}. AbclScriptEngine
599is a singleton, reflecting the fact that ABCL is a singleton as
600well. You can obtain an instance of AbclScriptEngine using the
601AbclScriptEngineFactory or by using the service provider mechanism
602through ScriptEngineManager (refer to the javax.script documentation).
603
604\subsubsection{Start-up and configuration file}
605
606At start-up (i.e. when its constructor is invoked, as part of the
607static initialization phase of AbclScriptEngineFactory) the ABCL
608script engine attempts to load an "init file" from the classpath
609(/abcl-script-config.lisp). If present, this file can be used to
610customize the behavior of the engine, by setting a number of
611variables in the ABCL-SCRIPT package. Here is a list of the available
612variables:
613
614\begin{itemize}
615\item *use-throwing-debugger* Controls whether ABCL uses a
616  non-standard debugging hook function to throw a Java exception
617  instead of dropping into the debugger in case of unhandled error
618  conditions.
619  \begin{itemize}
620  \item Default value: T
621  \item Rationale: it is more convenient for Java programmers using
622    Lisp as a scripting language to have it return exceptions to Java
623    instead of handling them in the Lisp world.
624  \item Known Issues: the non-standard debugger hook has been reported
625    to misbehave in certain circumstances, so consider disabling it if
626    it doesn't work for you.
627  \end{itemize}
628\item *launch-swank-at-startup* If true, Swank will be launched at
629  startup. See *swank-dir* and *swank-port*.
630  \begin{itemize}
631  \item Default value: NIL
632  \end{itemize}
633\item *swank-dir* The directory where Swank is installed. Must be set
634  if *launch-swank-at-startup* is true.
635\item *swank-port* The port where Swank will listen for
636  connections. Must be set if *launch-swank-at-startup* is true.
637  \begin{itemize}
638  \item Default value: 4005
639  \end{itemize}
640\end{itemize}
641
642Additionally, at startup the AbclScriptEngine will \code{(require
643  'asdf)} - in fact, it uses asdf to load Swank.
644
645\subsubsection{Evaluation}
646
647Code is read and evaluated in the package ABCL-SCRIPT-USER. This
648packages USEs the COMMON-LISP, JAVA and ABCL-SCRIPT packages. Future
649versions of the script engine might make this default package
650configurable. The \code{CL:LOAD} function is used under the hood for
651evaluating code, and thus the same behavior of LOAD is
652guaranteed. This allows, among other things, \code{IN-PACKAGE} forms
653to change the package in which the loaded code is read.
654
655It is possible to evaluate code in what JSR-223 calls a
656"ScriptContext" (basically a flat environment of name->value
657pairs). This context is used to establish special bindings for all the
658variables defined in it; since variable names are strings from Java's
659point of view, they are first interned using READ-FROM-STRING with, as
660usual, ABCL-SCRIPT-USER as the default package. Variables are declared
661special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
662functions work in a null lexical environment and would ignore
663non-special bindings.
664
665Contrary to what the function \code{LOAD} does, evaluation of a series
666of forms returns the value of the last form instead of T, so the
667evaluation of short scripts does the Right Thing.
668
669\subsubsection{Compilation}
670
671AbclScriptEngine implements the \code{javax.script.Compilable}
672interface. Currently it only supports compilation using temporary
673files. Compiled code, returned as an instance of
674javax.script.CompiledScript, is read, compiled and executed by default
675in the ABCL-SCRIPT-USER package, just like evaluated code. Differently
676from evaluated code, though, due to the way the ABCL compiler works,
677compiled code contains no reference to top-level self-evaluating
678objects (like numbers or strings). Thus, when evaluated, a piece of
679compiled code will return the value of the last non-self-evaluating
680form: for example the code "(do-something) 42" will return 42 when
681interpreted, but will return the result of (do-something) when
682compiled and later evaluated. To ensure consistency of behavior
683between interpreted and compiled code, make sure the last form is
684always a compound form - at least (identity some-literal-object). Note
685that this issue should not matter in real code, where it is unlikely a
686top-level self-evaluating form will appear as the last form in a file
687(in fact, the Common Lisp load function always returns T upon success;
688with JSR-223 this policy has been changed to make evaluation of small
689code snippets work as intended).
690
691\subsubsection{Invocation of functions and methods}
692
693AbclScriptEngine implements the \code{javax.script.Invocable}
694interface, which allows to directly call Lisp functions and methods,
695and to obtain Lisp implementations of Java interfaces. This is only
696partially possible with Lisp since it has functions, but not methods -
697not in the traditional OO sense, at least, since Lisp methods are not
698attached to objects but belong to generic functions. Thus, the method
699\code{invokeMethod()} is not implemented and throws an
700UnsupportedOperationException when called. The \code{invokeFunction()}
701method should be used to call both regular and generic functions.
702
703\subsubsection{Implementation of Java interfaces in Lisp}
704
705ABCL can use the Java reflection-based proxy feature to implement Java
706interfaces in Lisp. It has several built-in ways to implement an
707interface, and supports definition of new ones. The
708\code{JAVA:JMAKE-PROXY} generic function is used to make such
709proxies. It has the following signature:
710
711\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
712
713\code{interface} is a Java interface metaobject (e.g. obtained by
714invoking \code{jclass}) or a string naming a Java
715interface. \code{implementation} is the object used to implement the
716interface - several built-in methods of jmake-proxy exist for various
717types of implementations. \code{lisp-this} is an object passed to the
718closures implementing the Lisp "methods" of the interface, and
719defaults to \code{NIL}.
720
721The returned proxy is an instance of the interface, with methods
722implemented with Lisp functions.
723
724Built-in interface-implementation types include:
725
726\begin{itemize}
727\item a single Lisp function which upon invocation of any method in
728  the interface will be passed the method name, the Lisp-this object,
729  and all the parameters. Useful for interfaces with a single method,
730  or to implement custom interface-implementation strategies.
731\item a hash-map of method-name -> Lisp function mappings. Function
732  signature is \code{(lisp-this \&rest args)}.
733\item a Lisp package. The name of the Java method to invoke is first
734  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
735  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
736  the package. If it exists and is fbound, the corresponding function
737  will be called. Function signature is as the hash-table case.
738\end{itemize}
739
740This functionality is exposed by the AbclScriptEngine with the two
741methods getInterface(Class) and getInterface(Object, Class). The
742former returns an interface implemented with the current Lisp package,
743the latter allows the programmer to pass an interface-implementation
744object which will in turn be passed to the jmake-proxy generic
745function.
746
747\chapter{Implementation Dependent Extensions}
748
749As outlined by the CLHS ANSI conformance guidelines, we document the
750extensions to the Armed Bear Lisp implementation made accessible to
751the user by virtue of being an exported symbol in the JAVA, THREADS,
752or EXTENSIONS packages.
753
754\section{JAVA}
755
756\subsection{Modifying the JVM CLASSPATH}
757
758The JAVA:ADD-TO-CLASSPATH generic functions allows one to add the
759specified pathname or list of pathnames to the current classpath
760used by ABCL, allowing the dynamic loading of JVM objects:
761
762\begin{listing-lisp}
763CL-USER> (add-to-classpath "/path/to/some.jar")
764\end{listing-lisp}
765
766N.b \code{ADD-TO-CLASSPATH} only affects the classloader used by ABCL
767(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
768no effect on Java code outside ABCL.
769
770% include autogen docs for the JAVA package.
771\include{java}
772
773\section{THREADS}
774
775The extensions for handling multithreaded execution are collected in
776the \code{THREADS} package.  Most of the abstractions in Doug Lea's
777excellent \code{java.util.concurrent} packages may be manipulated
778directly via the JSS contrib to great effect.
779
780% include autogen docs for the THREADS package.
781\include{threads}
782
783\section{EXTENSIONS}
784
785The symbols in the EXTENSIONS package (nicknamed ``EXT'') constitutes
786extensions to the ANSI standard that are potentially useful to the
787user.  They include functions for manipulating network sockets,
788running external programs, registering object finalizers, constructing
789reference weakly held by the garbage collector and others.
790
791See \ref{Rhodes2007} for a generic function interface to the native
792JVM contract for \code{java.util.List}.
793
794% include autogen docs for the EXTENSIONS package.
795\include{extensions}
796
797\chapter{Beyond ANSI}
798
799Naturally, in striving to be a useful contemporary Common Lisp
800implementation, ABCL endeavors to include extensions beyond the ANSI
801specification which are either widely adopted or are especially useful
802in working with the hosting JVM.
803
804\section{Implementation Dependent}
805\begin{enumerate}
806  \item Compiler to JVM 5 bytecode
807  \item Pathname extensions
808\end{enumerate}
809
810\section{Pathname}
811
812We implement an extension to the Pathname that allows for the
813description and retrieval of resources named in a URI scheme that the
814JVM ``understands''.  Support is built-in to the ``http'' and
815``https'' implementations but additional protocol handlers may be
816installed at runtime by having JVM symbols present in the
817sun.net.protocol.dynmamic pacakge. See Java2007 \cite{Java2007} for more
818details.
819
820ABCL has created specializations of the ANSI Pathname object to
821enable to use of URIs to address dynamically loaded resources for the
822JVM.  A URL-PATHNAME has a corresponding URL whose canonical
823representation is defined to be the NAMESTRING of the Pathname.
824
825%
826\begin{verbatim}
827
828# RDF description of type hierarchy
829% TODO Render via some LaTeX mode for graphviz?
830
831  <jar-pathname> a <url-pathname>.
832  <url-pathname> a <pathname>.
833  <logical-pathname> a <pathname> .
834\end{verbatim}
835
836Both URL-PATHNAME and JAR-PATHNAME may be used anywhere a PATHNAME is
837accepted with the following caveats:
838
839\begin{itemize}
840
841\item A stream obtained via OPEN on a URL-PATHNAME cannot be the
842  target of write operations.
843
844\item No canonicalization is performed on the underlying URI (i.e. the
845implementation does not attempt to compute the current name of the
846representing resource unless it is requested to be resolved.)  Upon
847resolution, any cannoicalization procedures followed in resolving the
848resource (e.g. following redirects) are discarded. 
849
850\end{itemize}
851
852The implementation of URL-PATHNAME allows the ABCL user to laod dynamically
853code from the network.  For example, for Quicklisp (\cite{Xach2011}):
854
855\begin{listing-lisp}
856  CL-USER> (load "http://beta.quicklisp.org/quicklisp.lisp")
857\end{listing-lisp}
858
859will load and execute the Quicklisp setup code.
860
861See \ref{_:XACH2011} on page \pageref{_:XACH2011}.
862
863\subsubsection{Implementation}
864
865\code{DEVICE} either a string denoting a drive letter under DOS or a cons
866specifying a \code{URL-PATHNAME}.
867         
868\section{Extensible Sequences}
869
870See Rhodes2007 \cite{RHODES2007} for the design.
871
872The SEQUENCE package fully implements Christopher Rhodes' proposal for
873extensible sequences.  These user extensible sequences are used
874directly in \code{java-collections.lisp} provide these CLOS
875abstractions on the standard Java collection classes as defined by the
876\code{java.util.List} contract.
877
878This extension is not automatically loaded by the implementation.   It
879may be loaded via:
880
881\begin{listing-lisp}
882CL-USER> (require 'java-collections)
883\end{listing-lisp}
884
885if both extensible sequences and their application to Java collections
886is required, or
887
888\begin{listing-lisp}
889CL-USER> (require 'extensible-sequences)
890\end{listing-lisp}
891
892if only the extensible sequences API as specified in \ref{RHODES2007} is
893required.
894
895Note that \code{(require 'java-collections)} must be issued before
896\code{java.util.List} or any subclass is used as a specializer in a CLOS
897method definition (see the section below).
898
899\section{Extensions to CLOS}
900
901There is an additional syntax for specializing the parameter of a
902generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
903where \code{CLASS-STRING} is a string naming a Java class in dotted package
904form.
905
906For instance the following specialization would perhaps allow one to
907print more information about the contents of a java.util.Collection
908object
909
910\begin{listing-lisp}
911(defmethod print-object ((coll (java:jclass "java.util.Collection"))
912                         stream)
913  ;;; ...
914)
915\end{listing-lisp}
916
917If the class had been loaded via a classloader other than the original
918the class you wish to specialize on, one needs to specify the
919classloader as an optional third argument.
920
921\begin{listing-lisp}
922
923(defparameter *other-classloader*
924  (jcall "getBaseLoader" cl-user::*classpath-manager*))
925 
926(defmethod print-object ((device-id (java:jclass "dto.nbi.service.hdm.alcatel.com.NBIDeviceID" *other-classloader*))
927                         stream)
928  ;;; ...
929)
930\end{listing-lisp}
931
932\section{Extensions to the Reader}
933
934We implement a special hexadecimal escape sequence for specifying
935characters to the Lisp reader, namely we allow a sequences of the form
936\# \textbackslash Uxxxx to be processed by the reader as character
937whose code is specified by the hexadecimal digits ``xxxx''.  The
938hexadecimal sequence must be exactly four digits long \footnote{This
939  represents a compromise with contemporary in 2011 32bit hosting
940  architecures for which we wish to make text processing efficient.
941  Should the User require more control over UNICODE processing we
942  recommend Edi Weisz' excellent work with FLEXI-STREAMS which we
943  fully support}, padded by leading zeros for values less than 0x1000.
944
945Note that this sequence is never output by the implementation.  Instead,
946the corresponding Unicode character is output for characters whose
947code is greater than 0x00ff.
948
949\subsection{JSS optionally extends the Reader}
950
951The JSS contrib consitutes an additional, optional extension to the
952reader in the definition of the \#\" reader macro.  See
953\ref{section:jss} on page \pageref{section:jss} for more information.
954
955\section{ASDF}
956
957asdf-2.017.22 is packaged as core component of ABCL, but not
958initialized by default, as it relies on the CLOS subsystem which can
959take a bit of time to start \footnote{While this time is ``merely'' on
960  the order of seconds for contemporary 2011 machines, for
961  applications that need to initialize quickly, for example a web
962  server, this time might be unnecessarily long}.  ASDF may be loaded
963by the \textsc{ANSI} \code{REQUIRE} mechanism as follows:
964
965\begin{listing-lisp}
966CL-USER> (require 'asdf)
967\end{listing-lisp}
968
969\chapter{Contrib}
970
971The ABCL contrib is packaged as a separate jar archive usually named
972\code{abcl-contrib.jar} or possibly something like
973\code{abcl-contrib-1.0.0.jar}.  The contrib jar is not loaded by the
974implementation by default, and must be first intialized by the
975\code{REQUIRE} mechanism before using any specific contrib:
976
977\begin{listing-lisp}
978CL-USER> (require 'abcl-contrib)
979\end{listing-lisp}
980
981\section{abcl-asdf}
982
983This contrib enables an additional syntax for \textsc{ASDF} system
984definition which dynamically loads \textsc{JVM} artifacts such as jar
985archives via encapsulation of the Maven build tool.  The Maven Aether
986component can also be directly manipulated by the function associated
987with the \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} symbol.
988
989%ABCL specific contributions to ASDF system definition mainly
990%concerned with finding JVM artifacts such as jar archives to be
991%dynamically loaded.
992
993
994The following \textsc{ASDF} components are added: \code{JAR-FILE},
995\code{JAR-DIRECTORY}, \code{CLASS-FILE-DIRECTORY} and \code{MVN}.
996
997
998\subsection{Referencing Maven Artifacts via ASDF}
999
1000Maven artifacts may be referenced within \textsc{ASDF} system
1001definitions, as the following example references the
1002\code{log4j-1.4.9.jar} JVM artifact which provides a widely-used
1003abstraction for handling logging systems:
1004
1005\begin{listing-lisp}
1006    ;;;; -*- Mode: LISP -*-
1007    (in-package :asdf)
1008
1009    (defsystem :log4j
1010      :components ((:mvn "log4j/log4j"
1011                    :version "1.4.9")))
1012\end{listing-lisp}
1013
1014\subsection{API}
1015
1016We define an API for \textsc{ABCL-ASDF} as consisting of the following
1017ASDF classes:
1018
1019\code{JAR-DIRECTORY}, \code{JAR-FILE}, and
1020\code{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
1021valid pathname representation.
1022
1023Both the MVN and IRI classes descend from ASDF-COMPONENT, but do not
1024directly have a filesystem location.
1025
1026For use outside of ASDF system definitions, we currently define one
1027method, \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} which locates,
1028downloads, caches, and then loads into the currently executing JVM
1029process all recursive dependencies annotated in the Maven pom.xml
1030graph.
1031
1032\subsection{Directly Instructing Maven to Download JVM Artifacts}
1033
1034Bypassing \textsc{ASDF}, one can directly issue requests for the Maven
1035artifacts to be downloaded
1036
1037\begin{listing-lisp}
1038CL-USER> (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user")
1039WARNING: Using LATEST for unspecified version.
1040"/Users/evenson/.m2/repository/com/google/gwt/gwt-user/2.4.0-rc1/gwt-user-2.4.0-rc1.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA-sources.jar"
1041\end{listing-lisp}
1042
1043To actually load the dependency, use the \code{JAVA:ADD-TO-CLASSPATH} generic
1044function:
1045
1046\begin{listing-lisp}
1047CL-USER> (java:add-to-classpath (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user"))
1048\end{listing-lisp}
1049
1050Notice that all recursive dependencies have been located and installed
1051locally from the network as well.
1052
1053\section{asdf-jar}
1054
1055The asdf-jar contrib provides a system for packaging ASDF systems into
1056jar archives for ABCL.  Given a running ABCL image with loadable ASDF
1057systems the code in this package will recursively package all the
1058required source and fasls in a jar archive.
1059
1060See \url{http://svn.common-lisp.net/armedbear/trunk/abcl/contrib/asdf-jar/README.markdown}.
1061
1062
1063\section{jss}
1064\label{section:jss}
1065
1066To one used to the more universal syntax of Lisp pairs for which the
1067definition of read and compile time macros is quite natural, the Java
1068syntax available to the Java programmer may be said to suck.  To
1069alleviate this situation, we introduce the
1070\code{SHARPSIGN-DOUBLE-QUOTE} (``\\#"Q'') macro, the first of perhaps
1071  many exper
1072
1073\subsection{JSS usage}
1074
1075\label{section:jss}
1076
1077Example:
1078
1079\begin{listing-lisp}
1080
1081CL-USER> (require 'jss)
1082
1083CL-USER) (#"getProperties" 'java.lang.System)
1084
1085CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
1086
1087\end{listing-lisp}
1088
1089\url{http://svn.common-lisp.net/armedbear/trunk/abcl/contrib/jss/README.markdown}
1090
1091\section{asdf-install}
1092
1093The asdf-install contrib provides an implementation of ASDF-INSTALL.
1094Superseded by Quicklisp (see Xach2011 \cite{Xach2011}).
1095
1096The \code{require} of the \code{asdf-install} symbol has the side
1097effect of pushing the directory ``~/.asdf-install-dir/systems/'' into
1098the value of the \textsc{ASDF} central registry in
1099\code{asdf:*central-registry*}, providing a convenient mechanism for
1100stashing \textsc{ABCL} specific system definitions for convenient
1101access.
1102
1103\chapter{History}
1104
1105\textsc{ABCL} was originally the extension language for the J editor, which was
1106started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1107code that had previously not been released publically was suddenly
1108committed that enabled ABCL to be plausibly termed an emergent ANSI
1109Common Lisp implementation canidate.
1110
1111From 2006 to 2008, Peter manned the development lists, incorporating
1112patches as made sense.  After a suitable search, Peter nominated Erik
1113H\"{u}lsmann to take over the project.
1114
1115In 2008, the implementation was transferred to the current
1116maintainers, who have strived to improve its usability as a
1117contemporary Common Lisp implementation.
1118
1119On October 22, 2011, with the publication of this Manual explicitly
1120stating the conformance of Armed Bear Common Lisp to \textsc{ANSI}, we
1121released abcl-1.0.0.  We released abcl-1.0.1 as a maintainence release
1122on January 10, 2012.
1123
1124
1125\begin{thebibliography}{9}
1126
1127\label{_:1}
1128\bibitem{Java2000}
1129  ``A New Era for Java Protocol Handlers.''
1130  \url{http://java.sun.com/developer/onlineTraining/protocolhandlers/}
1131
1132\label{_:XACH2011}
1133\bibitem{Xach2011}
1134  Zach Beene
1135  ``Quicklisp:  A system for quickly constructing Common Lisp''
1136  \url{http://www.quicklisp.org/}
1137
1138\label{_:RHODES2007}
1139\bibitem{Rhodes2007}
1140Christopher Rhodes
1141``User-extensible Sequences in Common Lisp''
1142ILC '07 Proceedings of the 2007 International Lisp Conference
1143% An early draft.  XXX where is the real one?
1144\url{http://jcsu.jesus.cam.ac.uk/~csr21/spec.pdf}
1145
1146\label{_:AMOP}
1147\bibitem{AMOP}
1148Gregor Kiczales, Jim de Rivieres, and Daniel G. Bobrow
1149The Art of the Metaobject Protocol
1150% XXX online citation
1151
1152\end{thebibliography}
1153
1154\printindex
1155
1156\end{document}
1157
1158% TODO
1159%   1.  Create mechanism for swigging DocString and Lisp docs into
1160%       sections ('grovel.lisp')
1161
Note: See TracBrowser for help on using the repository browser.