source: trunk/abcl/doc/manual/abcl.tex @ 13933

Last change on this file since 13933 was 13933, checked in by Mark Evenson, 9 years ago

doc: include the package qualification of referenced symbols.

File size: 49.2 KB
[13438]1% -*- mode: latex; -*-
[13853]6\usepackage{hyperref} % Put this one last, it redefines lots of internals
[13853]10\title{Armed Bear Common Lisp User Manual}
11\date{Version 1.1.0-dev\\
[13933]13May 14, 2012}
[13853]14\author{Mark Evenson \and Erik H\"{u}lsmann \and Rudolf Schlatte \and
15  Alessio Stalla \and Ville Voutilainen}
[13853]23Armed Bear Common Lisp (ABCL) is an implementation of Common Lisp that
24runs on the Java Virtual Machine.  It compiles to Java bytecode and
25integrates with Java code and libraries in a seamless way:
27\item Lisp code can create Java objects and call their methods (see
28  Section~\ref{sec:lisp-java}, page~\pageref{sec:lisp-java}).
29\item Java code can call Lisp functions and generic functions, either
30  directly (Section~\ref{sec:calling-lisp-from-java},
31  page~\pageref{sec:calling-lisp-from-java}) or via \texttt{JSR-223}
32  (Section~\ref{sec:java-scripting-api},
33  page~\pageref{sec:java-scripting-api}).
34\item \code{jinterface-implementation} creates Lisp-side implementations
35  of Java interfaces that can be used as listeners for Swing classes and
36  similar.
38ABCL is supported by the Lisp library manager
39QuickLisp\footnote{\url{}} and can run many of the
40programs and libraries provided therein out-of-the-box.
[13853]45\subsection{ANSI Common Lisp}
46\textsc{ABCL} is currently a (non)-conforming ANSI Common Lisp
47implementation due to the following known issues:
50\item The generic function signatures of the \code{DOCUMENTATION} symbol
51  do not match the specification.
52\item The \code{TIME} form does not return a proper \code{VALUES}
53  environment to its caller.
[13853]56Somewhat confusingly, this statement of non-conformance in the
57accompanying user documentation fulfills the requirements that
58\textsc{ABCL} is a conforming ANSI Common Lisp implementation according
59to the Common Lisp HyperSpec~\cite{CLHS}.  Clarifications to this point
60are solicited.
[13853]62ABCL aims to be be a fully conforming ANSI Common Lisp implementation.
63Any other behavior should be reported as a bug.
[13853]65\subsection{Contemporary Common Lisp}
66In addition to ANSI conformance, \textsc{ABCL} strives to implement
67features expected of a contemporary Common Lisp, i.e. a Lisp of the
68post-2005 Renaissance.
70The following known problems detract from \textsc{ABCL} being a proper
71contemporary Common Lisp.
[13853]73\item An incomplete implementation of interactive debugging mechanisms,
74  namely a no-op version of \code{STEP} \footnote{Somewhat surprisingly
75    allowed by \textsc{ANSI}}, the inability to inspect local variables
76  in a given call frame, and the inability to resume a halted
77  computation at an arbitrarily selected call frame.
78\item An incomplete implementation of a proper metaobject protocol
79  (c.f. the (A)MOP specification~\cite{AMOP, AMOPspec})
80\item Incomplete streams abstraction, in that \textsc{ABCL} needs
81  suitable abstraction between ANSI and Gray streams.  The streams could
82  be optimized to the JVM NIO abstractions at great profit for binary
83  byte-level manipulations.
84\item Incomplete documentation (missing docstrings from exported symbols
85  and the draft status of this user manual).
92ABCL is licensed under the terms of the GPL v2 of June 1991 with the
93``classpath-exception'' (see the file \texttt{COPYING} in the source
94distribution for the license, term 13 in the same file for the classpath
95exception).  This license broadly means that you must distribute the
96sources to ABCL, including any changes you make, together with a program
97that includes ABCL, but that you are not required to distribute the
98sources of the whole program.  Submitting your changes upstream to the
99ABCL development team is actively encouraged and very much appreciated,
100of course.
105\item Philipp Marek \texttt{Thanks for the markup}
106\item Douglas Miles \texttt{Thanks for the whacky IKVM stuff and keeping the flame alive
107  in the dark years.}
108\item Alan Ruttenberg \texttt{Thanks for JSS.}
[13749]109\item and of course
110\emph{Peter Graves}
[13853]114\chapter{Running ABCL}
[13628]116\textsc{ABCL} is packaged as a single jar file usually named either
[13854]117\texttt{abcl.jar} or possibly something like \texttt{abcl-1.1.0.jar} if
[13853]118using a versioned package on the local filesystem from your system
119vendor.  This jar file can be executed from the command line to obtain a
120REPL\footnote{Read-Eval Print Loop, a Lisp commandline}, viz:
[13610]123  cmd$ java -jar abcl.jar
[13853]124\end{listing-shell} %$ unconfuse Emacs syntax highlighting
[13853]126\emph{N.b.} for the proceeding command to work, the \texttt{java}
127executable needs to be in your path.
[13853]129To facilitate the use of ABCL in tool chains such as SLIME~\cite{slime}
130(the Superior Lisp Interaction Mode for Emacs), we provide both a Bourne
131shell script and a \textsc{DOS} batch file.  If you or your
132administrator adjusted the path properly, ABCL may be executed simply
[13610]136  cmd$ abcl
[13853]139Probably the easiest way of setting up an editing environment using the
140Emacs editor is to use QuickLisp and follow the instructions at
[13628]145ABCL supports the following command line options:
148\item[\texttt{--help}] displays a help message.
149\item[\texttt{--noinform}] Suppresses the printing of startup information and banner.
[13868]150\item[\texttt{--noinit}] suppresses the loading of the \verb+~/.abclrc+ startup file.
[13853]151\item[\texttt{--nosystem}] suppresses loading the \texttt{system.lisp} customization file.
152\item[\texttt{--eval FORM}] evaluates FORM before initializing the REPL.
153\item[\texttt{--load FILE}] loads the file FILE before initializing the REPL.
154\item[\texttt{--load-system-file FILE}] loads the system file FILE before initializing the REPL.
155\item[\texttt{--batch}] evaluates forms specified by arguments and in
[13868]156  the intialization file \verb+~/.abclrc+, and then exits without
[13853]157  starting a REPL.
[13853]160All of the command line arguments following the occurrence of \verb+--+
161are passed unprocessed into a list of strings accessible via the
162variable \code{EXT:*COMMAND-LINE-ARGUMENT-LIST*} from within ABCL.
[13853]166If the \textsc{ABCL} process is started without the \code{--noinit}
167flag, it attempts to load a file named \code{.abclrc} in the user's home
168directory and then interpret its contents.
170The user's home directory is determined by the value of the JVM system
[13853]171property \texttt{user.home}.  This value may or may not correspond
172to the value of the \texttt{HOME} system environment variable, at the
173discretion of the JVM implementation that \textsc{ABCL} finds itself
174hosted upon.
[13853]176\chapter{Interaction with the Hosting JVM}
[13660]178%  Plan of Attack
[13331]180% describe calling Java from Lisp, and calling Lisp from Java,
181% probably in two separate sections.  Presumably, we can partition our
182% audience into those who are more comfortable with Java, and those
183% that are more comforable with Lisp
[13660]185The Armed Bear Common Lisp implementation is hosted on a Java Virtual
186Machine.  This chapter describes the mechanisms by which the
187implementation interacts with that hosting mechanism.
[13438]189\section{Lisp to Java}
[13628]192\textsc{ABCL} offers a number of mechanisms to interact with Java from its
193Lisp environment. It allows calling both instance and static methods
194of Java objects, manipulation of instance and static fields on Java
195objects, and construction of new Java objects.
[13853]197When calling Java routines, some values will automatically be converted
198by the FFI\footnote{Foreign Function Interface, the term for the part of
199  a Lisp implementation that implements calling code written in other
200  languages.}  from Lisp values to Java values. These conversions
201typically apply to strings, integers and floats. Other values need to be
202converted to their Java equivalents by the programmer before calling the
203Java object method. Java values returned to Lisp are also generally
[13628]204converted back to their Lisp counterparts. Some operators make an
[13853]205exception to this rule and do not perform any conversion; those are the
206``raw'' counterparts of certain FFI functions and are recognizable by
207their name ending with \code{-RAW}.
[13634]209\subsection{Low-level Java API}
[13853]211This subsection covers the low-level API available after evaluating
212\code{(require 'JAVA)}.  A higher level Java API, developed by Alan
213Ruttenberg, is available in the \code{contrib/} directory and described
214later in this document, see Section~\ref{section:jss} on page
[13660]217\subsubsection{Calling Java Object Methods}
[13660]219There are two ways to call a Java object method in the low-level (basic) API:
[13634]222\item Call a specific method reference (which was previously acquired)
[13801]223\item Dynamic dispatch using the method name and the call-specific
224  arguments provided by finding the best match (see
225  Section~\ref{sec:param-matching-for-ffi}).
[13853]228\code{JAVA:JMETHOD} is used to acquire a specific method reference.  The
229function takes two or more arguments. The first is a Java class
230designator (a \code{JAVA:JAVA-CLASS} object returned by
231\code{JAVA:JCLASS} or a string naming a Java class). The second is a
232string naming the method.
[13853]234Any arguments beyond the first two should be strings naming Java
235classes, with one exception as listed in the next paragraph. These
236classes specify the types of the arguments for the method.
[13853]238When \code{JAVA:JMETHOD} is called with three parameters and the last
239parameter is an integer, the first method by that name and matching
240number of parameters is returned.
[13853]242Once a method reference has been acquired, it can be invoked using
[13660]243\code{JAVA:JCALL}, which takes the method as the first argument. The
244second argument is the object instance to call the method on, or
245\code{NIL} in case of a static method.  Any remaining parameters are
246used as the remaining arguments for the call.
248\subsubsection{Calling Java object methods: dynamic dispatch}
250The second way of calling Java object methods is by using dynamic dispatch.
251In this case \code{JAVA:JCALL} is used directly without acquiring a method
252reference first. In this case, the first argument provided to \code{JAVA:JCALL}
253is a string naming the method to be called. The second argument is the instance
254on which the method should be called and any further arguments are used to
255select the best matching method and dispatch the call.
[13660]257\subsubsection{Dynamic dispatch: Caveats}
[13628]259Dynamic dispatch is performed by using the Java reflection
260API \footnote{The Java reflection API is found in the
261  \code{java.lang.reflect} package}. Generally the dispatch works
262fine, but there are corner cases where the API does not correctly
263reflect all the details involved in calling a Java method. An example
264is the following Java code:
267ZipFile jar = new ZipFile("/path/to/some.jar");
268Object els = jar.entries();
269Method method = els.getClass().getMethod("hasMoreElements");
[13853]273Even though the method \code{hasMoreElements()} is public in
274\code{Enumeration}, the above code fails with
277java.lang.IllegalAccessException: Class ... can
[13660]278not access a member of class\$2 with modifiers
280       at sun.reflect.Reflection.ensureMemberAccess(
281       at java.lang.reflect.Method.invoke(
282       at ...
[13853]285This is because the method has been overridden by a non-public class and
286the reflection API, unlike \texttt{javac}, is not able to handle such a case.
288While code like that is uncommon in Java, it is typical of ABCL's FFI
289calls. The code above corresponds to the following Lisp code:
292(let ((jar (jnew "" "/path/to/some.jar")))
293  (let ((els (jcall "entries" jar)))
294    (jcall "hasMoreElements" els)))
297except that the dynamic dispatch part is not shown.
[13628]299To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
[13853]300field representing the ``intended class'' of the object. That class is
301used first by \code{JAVA:JCALL} and similar to resolve methods; the
302actual class of the object is only tried if the method is not found in
303the intended class. Of course, the intended class is always a
304super-class of the actual class -- in the worst case, they coincide. The
305intended class is deduced by the return type of the method that
306originally returned the Java object; in the case above, the intended
307class of \code{ELS} is \code{java.util.Enumeration} because that is the
308return type of the \code{entries} method.
310While this strategy is generally effective, there are cases where the
311intended class becomes too broad to be useful. The typical example
312is the extraction of an element from a collection, since methods in
313the collection API erase all types to \code{Object}. The user can
314always force a more specific intended class by using the \code{JAVA:JCOERCE}
[13623]317% \begin{itemize}
318% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
319% \item The Java FFI presents a Lisp package (JAVA) with many useful
320%   symbols for manipulating the artifacts of expectation on the JVM,
321%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
322%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
323%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
324% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
325%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
326%   methods, and additional convenience functions.
327% \item Java classes and libraries may be dynamically added to the
328%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
329% \end{itemize}
[13625]331\subsubsection{Calling Java class static methods}
[13853]333Like non-static methods, references to static methods can be acquired by
334using the \code{JAVA:JMETHOD} primitive. Static methods are called with
335\code{JAVA:JSTATIC} instead of \code{JAVA:JCALL}.
337Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
338passing the name of the method as a string instead of passing a method reference.
339The parameter values should be values to pass in the function call instead of
340a specification of classes for each parameter.
[13623]342\subsubsection{Parameter matching for FFI dynamic dispatch}
[13624]345The algorithm used to resolve the best matching method given the name
346and the arguments' types is the same as described in the Java Language
[13625]347Specification. Any deviation should be reported as a bug.
[13624]349% ###TODO reference to correct JLS section
[13625]351\subsubsection{Instantiating Java objects}
353Java objects can be instantiated (created) from Lisp by calling
[13853]354a constructor from the class of the object to be created. The
355\code{JAVA:JCONSTRUCTOR} primitive is used to acquire a constructor
[13625]356reference. It's arguments specify the types of arguments of the constructor
357method the same way as with \code{JAVA:JMETHOD}.
[13853]359The obtained constructor is passed as an argument to \code{JAVA:JNEW},
360together with any arguments.  \code{JAVA:JNEW} can also be invoked with
361a string naming the class as its first argument.
[13853]363\subsubsection{Accessing Java object and class fields}
[13853]365Fields in Java objects can be accessed using the getter and setter
366functions \code{JAVA:JFIELD} and \code{(SETF JAVA:JFIELD)}.  Static
367(class) fields are accessed the same way, but with a class object or
368string naming a class as first argument.
370Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
[13853]371an intended class around, and values which can be converted to Lisp values will
[13657]372be converted.
[13853]374\section{Java to Lisp}
[13853]376This section describes the various ways that one interacts with Lisp
377from Java code.  In order to access the Lisp world from Java, one needs
378to be aware of a few things, the most important ones being listed below:
[13853]381\item All Lisp values are descendants of \code{LispObject}.
382\item Lisp symbols are accessible either via static members of the
383  \code{Symbol} class, or by dynamically introspecting a \code{Package}
384  object.
[13338]385\item The Lisp dynamic environment may be saved via
[13437]386  \code{LispThread.bindSpecial(Binding)} and restored via
387  \code{LispThread.resetSpecialBindings(Mark)}.
[13853]388\item Functions can be executed by invoking \code{LispObject.execute(args
389    [...])}
[13438]392\subsection{Calling Lisp from Java}
[13853]395Note: the entire ABCL Lisp system resides in the
396\texttt{org.armedbear.lisp} package, but the following code snippets do
397not show the relevant import statements in the interest of brevity.  An
398example of the import statement would be
[13338]400  import org.armedbear.lisp.*;
[13338]402to potentially import all the JVM symbol from the `org.armedbear.lisp'
[13853]405There can only ever be a single Lisp interpreter per JVM instance.  A
406reference to this interpreter is obtained by calling the static method
[13338]410  Interpreter interpreter = Interpreter.createInstance();
413If this method has already been invoked in the lifetime of the current
[13853]414Java process it will return \texttt{null}, so if you are writing Java
415whose life-cycle is a bit out of your control (like in a Java servlet),
416a safer invocation pattern might be:
[13338]419  Interpreter interpreter = Interpreter.getInstance();
420  if (interpreter == null) {
421    interpreter = Interpreter.createInstance();
422  }
[13853]426The Lisp \code{eval} primitive may simply be passed strings for evaluation:
[13338]429  String line = "(load \"file.lisp\")";
430  LispObject result = interpreter.eval(line);
433Notice that all possible return values from an arbitrary Lisp
434computation are collapsed into a single return value.  Doing useful
[13853]435further computation on the \code{LispObject} depends on knowing what the
436result of the computation might be.  This usually involves some amount
[13338]437of \code{instanceof} introspection, and forms a whole topic to itself
[13853]438(see Section~\ref{topic:Introspecting a LispObject},
439page~\pageref{topic:Introspecting a LispObject}).
[13438]441Using \code{eval} involves the Lisp interpreter.  Lisp functions may
[13853]442also be directly invoked by Java method calls as follows.  One simply
443locates the package containing the symbol, obtains a reference to the
444symbol, and then invokes the \code{execute()} method with the desired
[13853]448  interpreter.eval("(defun foo (msg)" +
449    "(format nil \"You told me '~A'~%\" msg))");
450  Package pkg = Packages.findPackage("CL-USER");
451  Symbol foo = pkg.findAccessibleSymbol("FOO");
452  Function fooFunction = (Function)foo.getSymbolFunction();
453  JavaObject parameter = new JavaObject("Lisp is fun!");
454  LispObject result = fooFunction.execute(parameter);
455  // How to get the "naked string value"?
456  System.out.println("The result was " + result.writeToString());
[13853]459If one is calling a function in the CL package, the syntax can become
460considerably simpler.  If we can locate the instance of definition in
461the ABCL Java source, we can invoke the symbol directly.  For instance,
462to tell if a \code{LispObject} is (Lisp) \texttt{NIL}, we can invoke the
463CL function \code{NULL} in the following way:
[13853]466  boolean nullp(LispObject object) {
467    LispObject result = Primitives.NULL.execute(object);
468    if (result == NIL) { // the symbol 'NIL' is explicitly named in the Java
469                         // namespace at ``Symbol.NIL''
470                         // but is always present in the
471                         // local namespace in its unadorned form for
472                         // the convenience of the User.
473      return false;
474    }
475    return true;
476 }
[13438]479\subsubsection{Introspecting a LispObject}
[13338]480\label{topic:Introspecting a LispObject}
[13853]482We present various patterns for introspecting an arbitrary
483\code{LispObject} which can hold the result of every Lisp evaluation
[13580]484into semantics that Java can meaningfully deal with.
[13853]486\paragraph{LispObject as \code{boolean}}
[13853]488If the \code{LispObject} is to be interpreted as a generalized boolean
489value, one can use \code{getBooleanValue()} to convert to Java:
[13853]492   LispObject object = Symbol.NIL;
493   boolean javaValue = object.getBooleanValue();
[13853]496Since in Lisp any value other than \code{NIL} means "true", Java
497equality can also be used, which is a bit easier to type and better in
498terms of information it conveys to the compiler:
[13335]501    boolean javaValue = (object != Symbol.NIL);
[13853]504\paragraph{LispObject as a list}
[13853]506If \code{LispObject} is a list, it will have the type \code{Cons}.  One
507can then use the \code{copyToArray} method to make things a bit more
508suitable for Java iteration.
[13853]511  LispObject result = interpreter.eval("'(1 2 4 5)");
512  if (result instanceof Cons) {
513    LispObject array[] = ((Cons)result.copyToArray());
514    ...
515  }
[13853]518A more Lispy way to iterate down a list is to use the `cdr()` access
[13335]519function just as like one would traverse a list in Lisp:;
[13853]522  LispObject result = interpreter.eval("'(1 2 4 5)");
523  while (result != Symbol.NIL) {
524    doSomething(;
525    result = result.cdr();
526  }
[13853]529\section{Java Scripting API (JSR-223)}
[13853]532ABCL can be built with support for JSR-223~\cite{jsr-223}, which offers
533a language-agnostic API to invoke other languages from Java. The binary
534distribution download-able from ABCL's homepage is built with JSR-223
535support. If you're building ABCL from source on a pre-1.6 JVM, you need
536to have a JSR-223 implementation in your classpath (such as Apache
537Commons BSF 3.x or greater) in order to build ABCL with JSR-223 support;
538otherwise, this feature will not be built.
[13634]540This section describes the design decisions behind the ABCL JSR-223
541support. It is not a description of what JSR-223 is or a tutorial on
542how to use it. See
[13634]544for example usage.
[13634]548In general, ABCL's implementation of the JSR-223 API performs implicit
549conversion from Java objects to Lisp objects when invoking Lisp from
550Java, and the opposite when returning values from Java to Lisp. This
551potentially reduces coupling between user code and ABCL. To avoid such
552conversions, wrap the relevant objects in \code{JavaObject} instances.
[13853]554\subsection{Implemented JSR-223 interfaces}
[13853]556JSR-223 defines three main interfaces, of which two (\code{Invocable}
557and \code{Compilable}) are optional. ABCL implements all the three
558interfaces - \code{ScriptEngine} and the two optional ones - almost
559completely. While the JSR-223 API is not specific to a single scripting
560language, it was designed with languages with a more or less Java-like
561object model in mind: languages such as Javascript, Python, Ruby, which
562have a concept of "class" or "object" with "fields" and "methods". Lisp
563is a bit different, so certain adaptations were made, and in one case a
564method has been left unimplemented since it does not map at all to Lisp.
566\subsubsection{The ScriptEngine}
[13853]568The main interface defined by JSR-223, \code{javax.script.ScriptEngine},
569is implemented by the class
570\code{org.armedbear.lisp.scripting.AbclScriptEngine}. \code{AbclScriptEngine}
[13634]571is a singleton, reflecting the fact that ABCL is a singleton as
[13853]572well. You can obtain an instance of \code{AbclScriptEngine} using the
573\code{AbclScriptEngineFactory} or by using the service provider
574mechanism through \code{ScriptEngineManager} (refer to the
575\texttt{javax.script} documentation).
[13853]577\subsection{Start-up and configuration file}
[13798]579At start-up (i.e. when its constructor is invoked, as part of the
[13853]580static initialization phase of \code{AbclScriptEngineFactory}) the ABCL
[13634]581script engine attempts to load an "init file" from the classpath
[13853]582(\texttt{/abcl-script-config.lisp}). If present, this file can be used to
[13798]583customize the behavior of the engine, by setting a number of
[13853]584variables in the \code{ABCL-SCRIPT} package. Here is a list of the available
588\item[\texttt{*use-throwing-debugger*}] controls whether ABCL uses a
[13634]589  non-standard debugging hook function to throw a Java exception
590  instead of dropping into the debugger in case of unhandled error
591  conditions.
[13626]592  \begin{itemize}
[13853]593  \item Default value: \texttt{T}
[13634]594  \item Rationale: it is more convenient for Java programmers using
595    Lisp as a scripting language to have it return exceptions to Java
596    instead of handling them in the Lisp world.
597  \item Known Issues: the non-standard debugger hook has been reported
598    to misbehave in certain circumstances, so consider disabling it if
599    it doesn't work for you.
[13626]600  \end{itemize}
[13853]601\item[\texttt{*launch-swank-at-startup*}] If true, Swank will be launched at
602  startup. See \texttt{*swank-dir*} and \texttt{*swank-port*}.
[13626]603  \begin{itemize}
[13853]604  \item Default value: \texttt{NIL}
[13626]605  \end{itemize}
[13853]606\item[\texttt{*swank-dir*}] The directory where Swank is installed. Must be set
607  if \texttt{*launch-swank-at-startup*} is true.
608\item[\texttt{*swank-port*}] The port where Swank will listen for
609  connections. Must be set if \texttt{*launch-swank-at-startup*} is
610  true.
[13626]611  \begin{itemize}
612  \item Default value: 4005
613  \end{itemize}
[13634]616Additionally, at startup the AbclScriptEngine will \code{(require
617  'asdf)} - in fact, it uses asdf to load Swank.
[13853]621Code is read and evaluated in the package \code{ABCL-SCRIPT-USER}. This
622packages \texttt{USE}s the \code{COMMON-LISP}, \code{JAVA} and
623\code{ABCL-SCRIPT} packages. Future versions of the script engine might
624make this default package configurable. The \code{CL:LOAD} function is
625used under the hood for evaluating code, and thus the behavior of
626\code{LOAD} is guaranteed. This allows, among other things,
627\code{IN-PACKAGE} forms to change the package in which the loaded code
628is read.
[13634]630It is possible to evaluate code in what JSR-223 calls a
[13853]631``ScriptContext'' (basically a flat environment of name$\rightarrow$value
[13634]632pairs). This context is used to establish special bindings for all the
633variables defined in it; since variable names are strings from Java's
[13853]634point of view, they are first interned using \code{READ-FROM-STRING} with, as
635usual, \code{ABCL-SCRIPT-USER} as the default package. Variables are declared
[13634]636special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
637functions work in a null lexical environment and would ignore
638non-special bindings.
[13634]640Contrary to what the function \code{LOAD} does, evaluation of a series
641of forms returns the value of the last form instead of T, so the
642evaluation of short scripts does the Right Thing.
[13660]646AbclScriptEngine implements the \code{javax.script.Compilable}
[13634]647interface. Currently it only supports compilation using temporary
648files. Compiled code, returned as an instance of
[13853]649\texttt{javax.script.CompiledScript}, is read, compiled and executed by
650default in the \texttt{ABCL-SCRIPT-USER} package, just like evaluated
651code.  In contrast to evaluated code, though, due to the way the ABCL
652compiler works, compiled code contains no reference to top-level
653self-evaluating objects (like numbers or strings). Thus, when evaluated,
654a piece of compiled code will return the value of the last
655non-self-evaluating form: for example the code ``\code{(do-something)
656  42}'' will return 42 when interpreted, but will return the result of
657(do-something) when compiled and later evaluated. To ensure consistency
658of behavior between interpreted and compiled code, make sure the last
659form is always a compound form - at least \code{(identity
660some-literal-object)}. Note that this issue should not matter in real
661code, where it is unlikely a top-level self-evaluating form will appear
662as the last form in a file (in fact, the Common Lisp load function
663always returns \texttt{T} upon success; with JSR-223 this policy has been changed
664to make evaluation of small code snippets work as intended).
[13853]666\subsection{Invocation of functions and methods}
[13634]668AbclScriptEngine implements the \code{javax.script.Invocable}
669interface, which allows to directly call Lisp functions and methods,
670and to obtain Lisp implementations of Java interfaces. This is only
671partially possible with Lisp since it has functions, but not methods -
672not in the traditional OO sense, at least, since Lisp methods are not
673attached to objects but belong to generic functions. Thus, the method
674\code{invokeMethod()} is not implemented and throws an
[13853]675\texttt{UnsupportedOperationException} when called. The \code{invokeFunction()}
[13634]676method should be used to call both regular and generic functions.
[13853]678\subsection{Implementation of Java interfaces in Lisp}
[13634]680ABCL can use the Java reflection-based proxy feature to implement Java
681interfaces in Lisp. It has several built-in ways to implement an
682interface, and supports definition of new ones. The
683\code{JAVA:JMAKE-PROXY} generic function is used to make such
684proxies. It has the following signature:
686\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
[13634]688\code{interface} is a Java interface metaobject (e.g. obtained by
689invoking \code{jclass}) or a string naming a Java
690interface. \code{implementation} is the object used to implement the
691interface - several built-in methods of jmake-proxy exist for various
692types of implementations. \code{lisp-this} is an object passed to the
693closures implementing the Lisp "methods" of the interface, and
694defaults to \code{NIL}.
[13634]696The returned proxy is an instance of the interface, with methods
697implemented with Lisp functions.
699Built-in interface-implementation types include:
[13634]702\item a single Lisp function which upon invocation of any method in
703  the interface will be passed the method name, the Lisp-this object,
704  and all the parameters. Useful for interfaces with a single method,
705  or to implement custom interface-implementation strategies.
[13853]706\item a hash-map of method-name $\rightarrow$ Lisp function mappings. Function
[13634]707  signature is \code{(lisp-this \&rest args)}.
708\item a Lisp package. The name of the Java method to invoke is first
709  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
710  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
711  the package. If it exists and is fbound, the corresponding function
712  will be called. Function signature is as the hash-table case.
[13853]715This functionality is exposed by the class \code{AbclScriptEngine} via
716the two methods \code{getInterface(Class)} and
717\code{getInterface(Object, Class)}. The former returns an interface
718implemented with the current Lisp package, the latter allows the
719programmer to pass an interface-implementation object which will in turn
720be passed to the \code{jmake-proxy} generic function.
[13618]722\chapter{Implementation Dependent Extensions}
[13618]724As outlined by the CLHS ANSI conformance guidelines, we document the
725extensions to the Armed Bear Lisp implementation made accessible to
726the user by virtue of being an exported symbol in the JAVA, THREADS,
727or EXTENSIONS packages.
731\subsection{Modifying the JVM CLASSPATH}
733The JAVA:ADD-TO-CLASSPATH generic functions allows one to add the
[13626]734specified pathname or list of pathnames to the current classpath
735used by ABCL, allowing the dynamic loading of JVM objects:
738CL-USER> (add-to-classpath "/path/to/some.jar")
[13690]741N.b \code{ADD-TO-CLASSPATH} only affects the classloader used by ABCL
[13626]742(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
743no effect on Java code outside ABCL.
[13331]745% include autogen docs for the JAVA package.
[13660]750The extensions for handling multithreaded execution are collected in
751the \code{THREADS} package.  Most of the abstractions in Doug Lea's
752excellent \code{java.util.concurrent} packages may be manipulated
753directly via the JSS contrib to great effect.
[13619]755% include autogen docs for the THREADS package.
[13618]760The symbols in the EXTENSIONS package (nicknamed ``EXT'') constitutes
[13854]761extensions to the \textsc{ANSI} standard that are potentially useful to the
[13618]762user.  They include functions for manipulating network sockets,
763running external programs, registering object finalizers, constructing
764reference weakly held by the garbage collector and others.
[13801]766See \cite{RHODES2007} for a generic function interface to the native
[13854]767\textsc{JVM} contract for \code{java.util.List}.
[13619]769% include autogen docs for the EXTENSIONS package.
[13438]772\chapter{Beyond ANSI}
774Naturally, in striving to be a useful contemporary Common Lisp
775implementation, ABCL endeavors to include extensions beyond the ANSI
776specification which are either widely adopted or are especially useful
[13854]777in working with the hosting \textsc{JVM}.
[13854]779\section{Compiler to Java 5 Bytecode}
[13854]781The \code{CL:COMPILE-FILE} interface emits a packed fasl format whose
782Pathname has the type  ``abcl''.  These fasls are operating system neutral
783byte archives packaged by the zip compression format which contain
784artifacts whose loading \code{CL:LOAD} understands.
[13660]788We implement an extension to the Pathname that allows for the
[13854]789description and retrieval of resources named in a
790\textsc{URI} \footnote{A \textsc{URI} is essentially a superset of
791  what is commonly understood as a \textsc{URL} We sometimesuse the
792  term URL as shorthand in describing the URL Pathnames, even though
793  the corresponding encoding is more akin to a URI as described in
794  RFC3986 \cite{rfc3986}.}  scheme that the \textsc{JVM}
795``understands''.  Support is built-in to comprehend the ``http'' and
[13612]796``https'' implementations but additional protocol handlers may be
[13854]797installed at runtime by having \textsc{JVM} symbols present in the
[13801] pacakge. See \cite{maso2000} for more
[13854]801\textsc{ABCL} has created specializations of the ANSI Pathname object to
802enable to use of \textsc{URI}s to address dynamically loaded resources for the
803JVM.  A \code{URL-PATHNAME} has a corresponding \textsc{URI} whose canonical
804representation is defined to be the \code{NAMESTRING} of the Pathname.
809# RDF description of type hierarchy
810% TODO Render via some LaTeX mode for graphviz?
812  <jar-pathname> a <url-pathname>.
[13854]813  <url-pathname> a <cl:pathname>.
814  <cl:logical-pathname> a <cl:pathname> .
[13933]823Both \code{EXT:URL-PATHNAME} and \code{EXT:JAR-PATHNAME} may be used anywhere
[13854]824a \code{CL:PATHNAME} is accepted with the following caveats:
[13933]828\item A stream obtained via \code{CL:OPEN} on a \code{CL:URL-PATHNAME}
829  cannot be the target of write operations.
832\item No canonicalization is performed on the underlying \textsc{URI}
833  (i.e. the implementation does not attempt to compute the current
834  name of the representing resource unless it is requested to be
835  resolved.)  Upon resolution, any cannoicalization procedures
836  followed in resolving the resource (e.g. following redirects) are
837  discarded.  Users may programatically initiate a new, local
838  computation by applying the \code{CL:TRUENAME} function to a
839  \code{EXT:URL-PATHNAME} object.  Depending on the reliablity and
840  properties of your local \textsc{REST} infrastructure, these results
841  may not necessarily be idempotent over time\footnote {See
842  \cite{evenson2011} for the draft of the publication of the technical
843  details}.
[13933]849The implementation of \code{EXT:URL-PATHNAME} allows the \textsc{ABCL}
[13854]850user to dynamically load code from the network.  For example,
851Quicklisp (\cite{quicklisp}) may be completely installed from the REPL
852as the single form:
[13618]855  CL-USER> (load "")
[13618]858will load and execute the Quicklisp setup code.
[13801]860%See \ref{_:quicklisp} on page \pageref{_:quicklisp}.
[13854]864The implementation of these extensions stores all the additional
865information in the PATHNAME object itself in ways that while strictly
866speaking are conformant, nonetheless may trip up libraries that don't
867expect the following:
870\item \code{DEVICE} can be either a string denoting a drive letter
871  under DOS or a list of exactly one or two elements.  If
872  \code{DEVICE} is a list, it denotes a \code{JAR-PATHNAME}, with the entries
873  containing \code{PATHNAME} objects which describe the outer and (possibley)
874  locations of the jar archive.
876\item A \code{URL-PATHNAME} always has a \code{HOST} component that is a
877  property list.  The values of the \code{HOST} property list are
878  always character strings.  The allowed keys have the following meanings:
879  \begin{description}
880  \item[:SCHEME] Scheme of URI ("http", "ftp", "bundle", etc.)
881  \item[:AUTHORITY] Valid authority according to the URI scheme.  For
882    "http" this could be "".
883  \item[:QUERY] The query of the \textsc{URI} 
884  \item[:FRAGMENT] The fragment portion of the \textsc{URI}
885  \end{description}
888\item In order to encapsulate the implementation decisions for these
889  meanings, the following functions provide a setf-able API for
890  reading and writing such values: \code{URL-PATHNAME-QUERY},
892  \code{URL-PATHNAME-SCHEME}.  The specific subtype of a Pathname may
893  be determined with the predicates \code{PATHNAME-URL-P} and
894  \code{PATHNAME-JAR-P}.
[13579]914\section{Extensible Sequences}
[13684]916See Rhodes2007 \cite{RHODES2007} for the design.
[13619]918The SEQUENCE package fully implements Christopher Rhodes' proposal for
[13580]919extensible sequences.  These user extensible sequences are used
[13579]920directly in \code{java-collections.lisp} provide these CLOS
921abstractions on the standard Java collection classes as defined by the
[13580]922\code{java.util.List} contract.
[13622]924This extension is not automatically loaded by the implementation.   It
925may be loaded via:
[13626]928CL-USER> (require 'java-collections)
931if both extensible sequences and their application to Java collections
932is required, or
[13622]935CL-USER> (require 'extensible-sequences)
[13801]938if only the extensible sequences API as specified in \cite{RHODES2007} is
941Note that \code{(require 'java-collections)} must be issued before
942\code{java.util.List} or any subclass is used as a specializer in a CLOS
943method definition (see the section below).
[13438]945\section{Extensions to CLOS}
947There is an additional syntax for specializing the parameter of a
[13437]948generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
949where \code{CLASS-STRING} is a string naming a Java class in dotted package
952For instance the following specialization would perhaps allow one to
953print more information about the contents of a java.util.Collection
957(defmethod print-object ((coll (java:jclass "java.util.Collection"))
958                         stream)
959  ;;; ...
963If the class had been loaded via a classloader other than the original
964the class you wish to specialize on, one needs to specify the
965classloader as an optional third argument.
[13438]969(defparameter *other-classloader*
970  (jcall "getBaseLoader" cl-user::*classpath-manager*))
[13855]972(defmethod print-object ((device-id (java:jclass
973                                     ""
974                                     *other-classloader*))
[13438]975                         stream)
976  ;;; ...
[13438]980\section{Extensions to the Reader}
[13415]982We implement a special hexadecimal escape sequence for specifying
983characters to the Lisp reader, namely we allow a sequences of the form
[13660]984\# \textbackslash Uxxxx to be processed by the reader as character
985whose code is specified by the hexadecimal digits ``xxxx''.  The
986hexadecimal sequence must be exactly four digits long \footnote{This
987  represents a compromise with contemporary in 2011 32bit hosting
988  architecures for which we wish to make text processing efficient.
989  Should the User require more control over UNICODE processing we
990  recommend Edi Weisz' excellent work with FLEXI-STREAMS which we
991  fully support}, padded by leading zeros for values less than 0x1000.
993Note that this sequence is never output by the implementation.  Instead,
994the corresponding Unicode character is output for characters whose
995code is greater than 0x00ff.
[13855]997\section{Overloading of the CL:REQUIRE Mechanism}
[13855]999The \code{CL:REQUIRE} mechanism is overloaded by attaching the following
1000semantic to the execution of \code{REQUIRE} on the following symbols:
[13855]1004  \item{\code{ASDF}} Loads the \textsc{ASDF} implementation shipped
1005    with the implementation.  After \textsc{ASDF} has been loaded in
1006    this manner, symbols passed to \code{CL:REQUIRE} which are
1007    otherwise unresolved, are passed to ASDF for a chance for
1008    resolution.  This means, for instance if \code{CL-PPCRE} can be
1009    located as a loadable \textsc{ASDF} system \code{(require
1010      'cl-ppcre)} is equivalent to \code{(asdf:load-system
1011      'cl-ppcre)}.
[13855]1013  \item{\code{ABCL-CONTRIB}} Locates and pushes the toplevel contents of
1014    ``abcl-contrib.jar'' into the \textsc{ASDF} central registry. 
[13882]1016    \begin{enumerate}
1017      \item \code{abcl-asdf}
1018        Functions for loading JVM artifacts dynamically, hooking into ASDF 2 objects where possible.
1019      \item \code{asdf-jar} Package addressable JVM artifacts via
1020        \code{abcl-asdf} descriptions as a single binary artifact
1021        including recursive dependencies.
1022      \item \code{mvn}
1023        These systems name common JVM artifacts from the distributed pom.xml graph of Maven Aether:
1024        \begin{enumerate}
1025          \item \code{jna} Dynamically load 'jna.jar' version 3.0.9 from the network.
[13886]1026        \end{enumerate}
1027      \item \code{quicklisp-abcl} (Not working) boot a local Quicklisp
1028        installation via the ASDF:IRI type introduced bia ABCL-ASDF.
[13905]1030    \end{enumerate}
[13855]1034The user may extend the \code{CL:REQUIRE} mechanism by pushing
1035function hooks into \code{SYSTEM:*MODULE-PROVIDER-FUNCTIONS*}.  Each
1036such hook function takes a single argument containing the symbol
1037passed to \code{CL:REQUIRE} and returns a non-\code{NIL} value if it
1038can successful resolve the symbol.
[13855]1040\section{JSS optionally extends the Reader}
1042The JSS contrib consitutes an additional, optional extension to the
[13854]1043reader in the definition of the \#\" reader macro.  See section
[13683]1044\ref{section:jss} on page \pageref{section:jss} for more information.
[13854]1048asdf-2.019 (see \cite{asdf}) is packaged as core component of ABCL,
1049but not initialized by default, as it relies on the CLOS subsystem
1050which can take a bit of time to start \footnote{While this time is
1051  ``merely'' on the order of seconds for contemporary 2011 machines,
1052  for applications that need to initialize quickly, for example a web
1053  server, this time might be unnecessarily long}.  The packaged ASDF
1054may be loaded by the \textsc{ANSI} \code{REQUIRE} mechanism as
1058CL-USER> (require 'asdf)
[13683]1063The ABCL contrib is packaged as a separate jar archive usually named
1064\code{abcl-contrib.jar} or possibly something like
[13854]1065\code{abcl-contrib-1.1.0.jar}.  The contrib jar is not loaded by the
[13683]1066implementation by default, and must be first intialized by the
1067\code{REQUIRE} mechanism before using any specific contrib:
1070CL-USER> (require 'abcl-contrib)
[13683]1075This contrib enables an additional syntax for \textsc{ASDF} system
1076definition which dynamically loads \textsc{JVM} artifacts such as jar
1077archives via encapsulation of the Maven build tool.  The Maven Aether
1078component can also be directly manipulated by the function associated
1079with the \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} symbol.
[13683]1081%ABCL specific contributions to ASDF system definition mainly
1082%concerned with finding JVM artifacts such as jar archives to be
1083%dynamically loaded.
[13683]1086The following \textsc{ASDF} components are added: \code{JAR-FILE},
1087\code{JAR-DIRECTORY}, \code{CLASS-FILE-DIRECTORY} and \code{MVN}.
[13683]1090\subsection{Referencing Maven Artifacts via ASDF}
[13683]1092Maven artifacts may be referenced within \textsc{ASDF} system
1093definitions, as the following example references the
1094\code{log4j-1.4.9.jar} JVM artifact which provides a widely-used
1095abstraction for handling logging systems:
[13815]1098;;;; -*- Mode: LISP -*-
1099(in-package :asdf)
[13815]1101(defsystem :log4j
1102  :components ((:mvn "log4j/log4j" :version "1.4.9")))
[13683]1107We define an API for \textsc{ABCL-ASDF} as consisting of the following
1108ASDF classes:
[13685]1110\code{JAR-DIRECTORY}, \code{JAR-FILE}, and
1111\code{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
[13683]1112valid pathname representation.
[13683]1114Both the MVN and IRI classes descend from ASDF-COMPONENT, but do not
[13636]1115directly have a filesystem location.
[13683]1117For use outside of ASDF system definitions, we currently define one
[13685]1118method, \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} which locates,
[13683]1119downloads, caches, and then loads into the currently executing JVM
1120process all recursive dependencies annotated in the Maven pom.xml
[13683]1123\subsection{Directly Instructing Maven to Download JVM Artifacts}
[13683]1125Bypassing \textsc{ASDF}, one can directly issue requests for the Maven
[13636]1126artifacts to be downloaded
[13815]1129CL-USER> (abcl-asdf:resolve-dependencies ""
1130                                         "gwt-user")
[13683]1131WARNING: Using LATEST for unspecified version.
[13683]1139To actually load the dependency, use the \code{JAVA:ADD-TO-CLASSPATH} generic
[13815]1143CL-USER> (java:add-to-classpath
1144          (abcl-asdf:resolve-dependencies ""
1145                                          "gwt-user"))
[13636]1148Notice that all recursive dependencies have been located and installed
[13660]1149locally from the network as well.
[13683]1153The asdf-jar contrib provides a system for packaging ASDF systems into
1154jar archives for ABCL.  Given a running ABCL image with loadable ASDF
[13660]1155systems the code in this package will recursively package all the
1156required source and fasls in a jar archive.
[13815]1158The documentation for this contrib can be found at
[13886]1165To one used to the more universal syntax of Lisp pairs upon which the
1166definition of read and compile time macros is quite
1167natural \footnote{See Graham's ``On Lisp''}, the Java syntax available to
1168the Java programmer may be said to suck.  To alleviate this situation,
1169the JSS contrib introduces the \code{SHARPSIGN-DOUBLE-QUOTE}
1170(\code{\#"}) reader macro, which allows the the specification of the
1171name of invoking function as the first element of the relevant s-expr
1172which tends to be more congruent to how Lisp programmers seem to be
1173wired to think.
[13854]1175While quite useful, we don't expect that the JSS contrib will be the
1176last experiment in wrangling Java from Common Lisp.
[13634]1178\subsection{JSS usage}
[13815]1183CL-USER> (require 'abcl-contrib)
1184==> ("ABCL-CONTRIB")
[13683]1185CL-USER> (require 'jss)
[13815]1186==> ("JSS")
[13683]1187CL-USER) (#"getProperties" 'java.lang.System)
[13815]1188==> #<java.util.Properties { {2FA21ACF}>
[13683]1189CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
[13815]1190==> #<java.util.Hashtable$Enumerator java.util.Has.... {36B4361A}>
1191\end{listing-lisp} %$ <-- un-confuse Emacs font-lock
[13815]1193Some more information on jss can be found in its documentation at
[13683]1198The asdf-install contrib provides an implementation of ASDF-INSTALL.
[13801]1199Superseded by Quicklisp (see Xach2011 \cite{quicklisp}).
[13699]1201The \code{require} of the \code{asdf-install} symbol has the side
[13868]1202effect of pushing the directory \verb+~/.asdf-install-dir/systems/+ into
[13699]1203the value of the \textsc{ASDF} central registry in
1204\code{asdf:*central-registry*}, providing a convenient mechanism for
1205stashing \textsc{ABCL} specific system definitions for convenient
[13683]1210\textsc{ABCL} was originally the extension language for the J editor, which was
[13660]1211started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1212code that had previously not been released publically was suddenly
1213committed that enabled ABCL to be plausibly termed an emergent ANSI
[13854]1214Common Lisp implementation candidate.
[13660]1216From 2006 to 2008, Peter manned the development lists, incorporating
1217patches as made sense.  After a suitable search, Peter nominated Erik
[13684]1218H\"{u}lsmann to take over the project.
[13657]1220In 2008, the implementation was transferred to the current
[13415]1221maintainers, who have strived to improve its usability as a
1222contemporary Common Lisp implementation.
[13660]1224On October 22, 2011, with the publication of this Manual explicitly
[13749]1225stating the conformance of Armed Bear Common Lisp to \textsc{ANSI}, we
[13798]1226released abcl-1.0.0.  We released abcl-1.0.1 as a maintainence release
1227on January 10, 2012.
Note: See TracBrowser for help on using the repository browser.