source: trunk/abcl/doc/manual/abcl.tex @ 15265

Last change on this file since 15265 was 15265, checked in by Mark Evenson, 2 years ago

abcl-1.6.1: further corrections to metadata and documentation

File size: 68.4 KB
Line 
1% -*- mode: latex; -*-
2% http://en.wikibooks.org/wiki/LaTeX/
3\documentclass[10pt]{book}
4% also need to have cm-super installed for high quality rendering
5\usepackage[T1]{fontenc}
6\usepackage{abcl}
7
8\usepackage{hyperref} % Put this one last, it redefines lots of internals
9
10\begin{document}
11\title{Armed Bear Common Lisp User Manual}
12\date{Version 1.6.1\\
13\smallskip
14April 2020}
15\author{Mark Evenson \and Erik H\"{u}lsmann \and Rudolf Schlatte \and
16  Alessio Stalla \and Ville Voutilainen}
17
18\maketitle
19
20\tableofcontents
21%%Preface to the Seventh edition, abcl-1.6.0
22\subsection{Preface to the Seventh Edition}
23
24Long overdue, we turn our Java to 11.
25
26Reflecting the management's best estimates as to implementation most
27easily available to the potential \textsc{ABCL} 1.6 User, the Seventh
28release implementation works best with Java 8 or Java 11 runtimes.
29Since freely available implementations of jdk6 and jdk7 exist, we
30still strive to maintain compatibility with the Java 6 and Java 7
31runtime environments but those environments are less tested.  The User
32may need to use the facilities of the ABCL-BUILD contrib to recompile
33the implementation locally in more exotic runtimes (see
34Section~\ref{sec:abcl-build} page~\pageref{sec:abcl-build}).
35
36%%Preface to the Sixth edition, abcl-1.5.0
37\subsection{Preface to the Sixth Edition}
38
39With the sixth major release of the implementation, we make the
40following explicit revision of our compatibility to the underlying
41JVM.  Since we are an open source implementation, we insist on
42possible open access to the sources from with an JDK may both be built
43and run upon.  This requirement is no longer met by Java 5, so
44henceforth with the release of \textsc{ABCL} 1.5, we will support
45Java 6, Java 7 and Java 8 runtimes.
46
47%%Preface to the Fifth edition, abcl-1.4.0
48\subsection{Preface to the Fifth Edition}
49
50\textsc{ABCL} 1.4 consolidates eighteen months of production bug-fixes,
51and substantially improves the support for invoking external
52processes via \code{SYS:RUN-PROGRAM}.
53
54%%Preface to the Fourth edition, abcl-1.3.
55\subsection{Preface to the Fourth Edition}
56
57\textsc{ABCL} 1.3 now implements an optimized implementation of the
58\code{org.armedbear.lisp.LispStack} abstraction thanks to Dmitry
59Nadezhin which runs on ORCL JVMs from 1.[5-8] conformantly.
60
61%%Preface to the Third edition, abcl-1.2.
62\subsection{Preface to the Third Edition}
63The implementation now contains a performant and conformant
64implementation of (A)MOP to the point of inclusion in CLOSER-MOP's
65test suite.
66
67%%Preface to the second edition, abcl-1.1.
68
69\subsection{Preface to the Second Edition}
70
71\textsc{ABCL} 1.1 now contains \textsc{(A)MOP}.  We hope you enjoy!  --The Mgmt.
72
73\chapter{Introduction}
74
75Armed Bear Common Lisp (\textsc{ABCL}) is an implementation of
76\textsc{Common Lisp} that runs on the Java Virtual Machine
77(\textsc{JVM}).  \textsc{ABCL} compiles \textsc{Common Lisp} to
78\textsc{Java} byte-code\footnote{The class files produced by the
79compiler have a bytecode version of ``49.0''.}, with an efficiency
80that varies upon the hosting JVM implementation.  \textsc{ABCL}
81supports running on the Java 6, Java 7, Java 8, Java 11, Java 13, and
82Java 14 openjdk \textsc{JVM} implementations\footnote{As of April
832020, the AdoptOpenJDK.net community \url{https://adoptopenjdk.net/}
84provides perhaps the easiest installation of unencumbered openjdk
85implementations}.
86
87Armed Bear provides the following integration methods for interfacing
88with Java code and libraries:
89\begin{itemize}
90\item Lisp code can create Java objects and call their methods (see
91  Section~\ref{sec:lisp-java}, page~\pageref{sec:lisp-java}).
92\item Java code can call Lisp functions and generic functions, either
93  directly (Section~\ref{sec:calling-lisp-from-java},
94  page~\pageref{sec:calling-lisp-from-java}) or via \texttt{JSR-223}
95  (Section~\ref{sec:java-scripting-api},
96  page~\pageref{sec:java-scripting-api}).
97\item \code{jinterface-implementation} creates Lisp-side implementations
98  of Java interfaces that can be used as listeners for Swing classes and
99  similar.
100\item \code{java:jnew-runtime-class} can inject fully synthetic Java
101  classes--and their objects-- into the current JVM process whose
102  behavior is specified via closures expressed in \textsc{Common
103    Lisp}. \footnote{Parts of the current implementation are not fully
104    finished, so the status of some interfaces here should be treated
105    with skepticism if you run into problems.}
106
107\end{itemize}
108\textsc{ABCL} is supported by the Lisp library manager
109\textsc{QuickLisp}\footnote{\url{http://quicklisp.org/}} and can run many of the
110programs and libraries provided therein out-of-the-box.
111
112\section{Conformance}
113\label{section:conformance}
114
115\subsection{ANSI Common Lisp}
116\textsc{ABCL} is currently a (non)-conforming \textsc{ANSI} Common Lisp
117implementation due to the following known issues:
118
119\begin{itemize}
120\item The generic function signatures of the \code{CL:DOCUMENTATION} symbol
121  do not match the specification.
122\item The \code{CL:TIME} form does not return a proper \code{CL:VALUES}
123  environment to its caller.
124\item When merging pathnames and the defaults point to a \code{EXT:JAR-PATHNAME},
125  we set the \code{DEVICE} of the result to \code{:UNSPECIFIC} if the pathname to be
126  be merged does not contain a specified \code{DEVICE}, does not contain a
127  specified \code{HOST}, does contain a relative \code{DIRECTORY}, and we are
128  not running on a \textsc{MSFT} Windows platform.\footnote{The intent of this
129    rather arcane sounding deviation from conformance is so that the
130    result of a merge won't fill in a \code{DEVICE} with the wrong "default
131    device for the host" in the sense of the fourth paragraph in the
132    \textsc{CLHS} description of MERGE-PATHNAMES (see in \cite{CLHS} the paragraph beginning
133    "If the PATHNAME explicitly specifies a host and not a device
").
134    A future version of the implementation may return to conformance
135    by using the \code{HOST} value to reflect the type explicitly.
136  }
137
138\end{itemize}
139
140Somewhat confusingly, this statement of non-conformance in the
141accompanying user documentation fulfills the requirements that
142\textsc{ABCL} is a conforming ANSI Common Lisp implementation according
143to the Common Lisp HyperSpec~\cite{CLHS}.  Clarifications to this point
144are solicited.
145
146\textsc{ABCL} aims to be be a fully conforming \textsc{ANSI} Common Lisp implementation.
147Any other behavior should be reported as a bug.
148
149\subsection{Contemporary Common Lisp}
150In addition to \textsc{ANSI} conformance, \textsc{ABCL} strives to implement
151features expected of a contemporary \textsc{Common Lisp}, i.e. a Lisp of the
152post-2005 Renaissance.
153
154The following known problems detract from \textsc{ABCL} being a proper
155contemporary Common Lisp.
156\begin{itemize}
157\item An incomplete implementation of interactive debugging
158  mechanisms, namely a no-op version of \code{STEP}\footnote{Somewhat
159    surprisingly allowed by \textsc{ANSI}}, the inability to inspect
160  local variables in a given call frame, and the inability to resume a
161  halted computation at an arbitrarily selected call frame.
162\item Incomplete streams abstraction, in that \textsc{ABCL} needs a
163  suitable abstraction between \textsc{ANSI} and \textsc{Gray streams}
164  with a runtime switch for the beyond conforming
165  behavior\footnote{The streams could be optimized to the
166    \textsc{JVM} NIO \cite{nio} abstractions at great profit for
167    binary byte-level manipulations.}.
168\item Incomplete documentation: source code is missing docstrings from
169  all exported symbols from the \code{EXTENSIONS}, \code{SYSTEM},
170  \code{JAVA}, \code{MOP}, and \code{THREADS} packages.  This user
171  manual is currently in draft status.
172\end{itemize}
173
174
175
176\section{License}
177
178\textsc{ABCL} is licensed under the terms of the \textsc{GPL} v2 of
179June 1991 with an added ``classpath-exception'' clause (see the file
180\texttt{COPYING} in the source distribution\footnote{See
181  \url{http://abcl.org/svn/trunk/tags/1.6.1/COPYING}} for the license,
182term 13 in the same file for the classpath exception).  This license
183broadly means that you must distribute the sources to \textsc{ABCL},
184including any changes you make, together with a program that includes
185\textsc{ABCL}, but that you are not required to distribute the sources
186of the whole program.  Submitting your changes upstream to the \textsc{ABCL}
187development team is actively encouraged and very much appreciated, of
188course.
189
190\section{Contributors}
191
192\begin{itemize}
193\item Philipp Marek \texttt{Thanks for the markup, and review of the Manual}
194\item Douglas Miles \texttt{Thanks for the whacky IKVM stuff and keeping the flame alive
195  in the dark years.}
196\item Alan Ruttenberg \texttt{Thanks for JSS.}
197\item Olof-Joachim Frahm 
198\item piso
199\item and of course \emph{Peter Graves}
200\end{itemize}
201
202
203\chapter{Running ABCL}
204
205
206\textsc{ABCL} is packaged as a single jar file usually named either
207\texttt{abcl.jar} or possibly something like \texttt{abcl-1.6.1.jar} if
208using a versioned package on the local filesystem from your system
209vendor.  This jar file can be executed from the command line to obtain a
210\textsc{REPL}\footnote{Read-Eval Print Loop, a Lisp command-line}, viz:
211
212\index{REPL}
213
214\begin{listing-shell}
215  cmd$ java -jar abcl.jar
216\end{listing-shell} %$ unconfuse Emacs syntax highlighting
217
218\emph{N.b.} for the proceeding command to work, the \texttt{java}
219executable needs to be in your path.
220
221To facilitate the use of ABCL in tool chains such as SLIME~\cite{slime}
222(the Superior Lisp Interaction Mode for Emacs), we provide both a Bourne
223shell script and a \textsc{DOS} batch file.  If you or your
224administrator adjusted the path properly, ABCL may be executed simply
225as:
226
227\begin{listing-shell}
228  cmd$ abcl
229\end{listing-shell}%$
230
231Probably the easiest way of setting up an editing environment using the
232\textsc{Emacs} editor is to use \textsc{Quicklisp} and follow the instructions at
233\url{http://www.quicklisp.org/beta/#slime}.
234
235\section{Options}
236
237ABCL supports the following command line options:
238
239\index{Command Line Options}
240
241\begin{description}
242  % FIXME move this fuggliness to our macros.  sigh.
243\item \code{\textendash\textendash help} displays a help message.
244\item \code{\textendash\textendash noinform} Suppresses the printing of startup information and banner.
245\item \code{\textendash\textendash noinit} suppresses the loading of the \verb+~/.abclrc+ startup file.
246\item \code{\textendash\textendash nosystem} suppresses loading the \texttt{system.lisp} customization file.
247\item \code{\textendash\textendash eval FORM} evaluates FORM before initializing the REPL.
248\item \code{\textendash\textendash load FILE} loads the file FILE before initializing the REPL.
249\item \code{\textendash\textendash load-system-file FILE} loads the system file FILE before initializing the REPL.
250\item \code{\textendash\textendash batch} evaluates forms specified by arguments and in
251  the initialization file \verb+~/.abclrc+, and then exits without
252  starting a \textsc{REPL}.
253\end{description}
254
255All of the command line arguments following the occurrence of \verb+--+
256are passed unprocessed into a list of strings accessible via the
257variable \code{EXT:*COMMAND-LINE-ARGUMENT-LIST*} from within ABCL.
258
259\section{Initialization}
260
261If the \textsc{ABCL} process is started without the
262\code{\textendash\textendash noinit} flag, it attempts to load a file
263named \code{.abclrc} in the user's home directory and then interpret
264its contents.
265
266The user's home directory is determined by the value of the JVM system
267property \texttt{user.home}.  This value may or may not correspond
268to the value of the \texttt{HOME} system environment variable, at the
269discretion of the JVM implementation that \textsc{ABCL} finds itself
270hosted upon.
271
272\chapter{Interaction with the Hosting JVM}
273
274%  Plan of Attack
275%
276% describe calling Java from Lisp, and calling Lisp from Java,
277% probably in two separate sections.  Presumably, we can partition our
278% audience into those who are more comfortable with Java, and those
279% that are more comforable with Lisp
280
281The Armed Bear Common Lisp implementation is hosted on a Java Virtual
282Machine.  This chapter describes the mechanisms by which the
283implementation interacts with that hosting mechanism.
284
285\section{Lisp to Java}
286\label{sec:lisp-java}
287
288\textsc{ABCL} offers a number of mechanisms to interact with Java from its
289Lisp environment. It allows calling both instance and static methods
290of Java objects, manipulation of instance and static fields on Java
291objects, and construction of new Java objects.
292
293When calling Java routines, some values will automatically be
294converted by the FFI\footnote{Foreign Function Interface is the term
295  of art for the part of a \textsc{Lisp} implementation which implements
296  calling code written in other languages, typically normalized to the
297  local C compiler calling conventions.}  from \textsc{Lisp} values to Java
298values. These conversions typically apply to strings, integers and
299floats. Other values need to be converted to their \textsc{Java} equivalents by
300the programmer before calling the Java object method. Java values
301returned to \textsc{Lisp} are also generally converted back to their Lisp
302counterparts. Some operators make an exception to this rule and do not
303perform any conversion; those are the ``raw'' counterparts of certain
304FFI functions and are recognizable by their name ending with
305\code{-RAW}.
306
307\subsection{Low-level Java API}
308
309This subsection covers the low-level API available after evaluating
310\code{(require :java)}.  A higher level \textsc{Java} API, developed by Alan
311Ruttenberg, is available in the \code{contrib/jss} directory and described
312later in this document, see Section~\ref{section:jss} on page
313\pageref{section:jss}.
314
315\subsubsection{Calling Java Object Methods}
316
317There are two ways to call a Java object method in the low-level (basic) API:
318
319\begin{itemize}
320\item Call a specific method reference (which was previously acquired)
321\item Dynamic dispatch using the method name and the call-specific
322  arguments provided by finding the best match (see
323  Section~\ref{sec:param-matching-for-ffi}).
324\end{itemize}
325
326\code{JAVA:JMETHOD} is used to acquire a specific method reference.  The
327function takes two or more arguments. The first is a Java class
328designator (a \code{JAVA:JAVA-CLASS} object returned by
329\code{JAVA:JCLASS} or a string naming a Java class). The second is a
330string naming the method.
331
332Any arguments beyond the first two should be strings naming Java
333classes, with one exception as listed in the next paragraph. These
334classes specify the types of the arguments for the method.
335
336When \code{JAVA:JMETHOD} is called with three parameters and the last
337parameter is an integer, the first method by that name and matching
338number of parameters is returned.
339
340Once a method reference has been acquired, it can be invoked using
341\code{JAVA:JCALL}, which takes the method as the first argument. The
342second argument is the object instance to call the method on, or
343\code{NIL} in case of a static method.  Any remaining parameters are
344used as the remaining arguments for the call.
345
346\subsubsection{Calling Java object methods: dynamic dispatch}
347
348The second way of calling Java object methods is by using dynamic dispatch.
349In this case \code{JAVA:JCALL} is used directly without acquiring a method
350reference first. In this case, the first argument provided to \code{JAVA:JCALL}
351is a string naming the method to be called. The second argument is the instance
352on which the method should be called and any further arguments are used to
353select the best matching method and dispatch the call.
354
355\subsubsection{Dynamic dispatch: Caveats}
356
357Dynamic dispatch is performed by using the Java reflection
358API\footnote{The Java reflection API is found in the
359  \code{java.lang.reflect} package}. Generally the dispatch works
360fine, but there are corner cases where the API does not correctly
361reflect all the details involved in calling a Java method. An example
362is the following Java code:
363
364\begin{listing-java}
365ZipFile jar = new ZipFile("/path/to/some.jar");
366Object els = jar.entries();
367Method method = els.getClass().getMethod("hasMoreElements");
368method.invoke(els);
369\end{listing-java}
370
371Even though the method \code{hasMoreElements()} is public in
372\code{Enumeration}, the above code fails with
373
374\begin{listing-java}
375java.lang.IllegalAccessException: Class ... can
376not access a member of class java.util.zip.ZipFile\$2 with modifiers
377"public"
378       at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
379       at java.lang.reflect.Method.invoke(Method.java:583)
380       at ...
381\end{listing-java}
382
383This is because the method has been overridden by a non-public class and
384the reflection API, unlike \texttt{javac}, is not able to handle such a case.
385
386While code like that is uncommon in Java, it is typical of ABCL's FFI
387calls. The code above corresponds to the following Lisp code:
388
389\begin{listing-lisp}
390(let ((jar (jnew "java.util.zip.ZipFile" "/path/to/some.jar")))
391  (let ((els (jcall "entries" jar)))
392    (jcall "hasMoreElements" els)))
393\end{listing-lisp}
394
395except that the dynamic dispatch part is not shown.
396
397To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
398field representing the ``intended class'' of the object. That class is
399used first by \code{JAVA:JCALL} and similar to resolve methods; the
400actual class of the object is only tried if the method is not found in
401the intended class. Of course, the intended class is always a
402super-class of the actual class -- in the worst case, they coincide. The
403intended class is deduced by the return type of the method that
404originally returned the Java object; in the case above, the intended
405class of \code{ELS} is \code{java.util.Enumeration} because that is the
406return type of the \code{entries} method.
407
408While this strategy is generally effective, there are cases where the
409intended class becomes too broad to be useful. The typical example
410is the extraction of an element from a collection, since methods in
411the collection API erase all types to \code{Object}. The user can
412always force a more specific intended class by using the \code{JAVA:JCOERCE}
413operator.
414
415% \begin{itemize}
416% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
417% \item The Java FFI presents a Lisp package (JAVA) with many useful
418%   symbols for manipulating the artifacts of expectation on the JVM,
419%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
420%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
421%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
422% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
423%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
424%   methods, and additional convenience functions.
425% \item Java classes and libraries may be dynamically added to the
426%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
427% \end{itemize}
428
429\subsubsection{Calling Java class static methods}
430
431Like non-static methods, references to static methods can be acquired by
432using the \code{JAVA:JMETHOD} primitive. Static methods are called with
433\code{JAVA:JSTATIC} instead of \code{JAVA:JCALL}.
434
435Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
436passing the name of the method as a string instead of passing a method reference.
437The parameters should be values to pass in the function call instead of
438a specification of classes for each parameter.
439
440\subsubsection{Parameter matching for FFI dynamic dispatch}
441\label{sec:param-matching-for-ffi}
442
443The algorithm used to resolve the best matching method given the name
444and the arguments' types is the same as described in the Java Language
445Specification. Any deviation should be reported as a bug.
446
447% ###TODO reference to correct JLS section
448
449\subsubsection{Instantiating Java objects}
450
451\textsc{Java} objects can be instantiated (created) from \textsc{Lisp} by calling
452a constructor from the class of the object to be created. The
453\code{JAVA:JCONSTRUCTOR} primitive is used to acquire a constructor
454reference. Its arguments specify the types of arguments of the constructor
455method the same way as with \code{JAVA:JMETHOD}.
456
457The obtained constructor is passed as an argument to \code{JAVA:JNEW},
458together with any arguments.  \code{JAVA:JNEW} can also be invoked with
459a string naming the class as its first argument.
460
461\subsubsection{Accessing Java object and class fields}
462
463Fields in Java objects can be accessed using the getter and setter
464functions \code{JAVA:JFIELD} and \code{(SETF JAVA:JFIELD)}.  Static
465(class) fields are accessed the same way, but with a class object or
466string naming a class as first argument.
467
468Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
469an intended class around, and values which can be converted to Lisp values will
470be converted.
471
472\section{Java to Lisp}
473
474This section describes the various ways that one interacts with \textsc{Lisp}
475from \textsc{Java} code.  In order to access the \textsc{Lisp} world from \textsc{Java}, one needs
476to be aware of a few things, the most important ones being listed below:
477
478\begin{itemize}
479\item All Lisp values are descendants of \code{LispObject}.
480\item Lisp symbols are accessible either via static members of the
481  \code{Symbol} class, or by dynamically introspecting a \code{Package}
482  object.
483\item The Lisp dynamic environment may be saved via
484  \code{LispThread.bindSpecial(Binding)} and restored via
485  \code{LispThread.resetSpecialBindings(Mark)}.
486\item Functions can be executed by invoking \code{LispObject.execute(args
487    [...])}
488\end{itemize}
489
490\subsection{Calling Lisp from Java}
491\label{sec:calling-lisp-from-java}
492
493Note: the entire \textsc{ABCL} \textsc{Lisp} system implementation in
494\textsc{Java} is resident in the \texttt{org.armedbear.lisp} package,
495but the following code snippets do not show the relevant import
496statements in the interest of brevity.  An example of the import
497statement would be
498\begin{listing-java}
499  import org.armedbear.lisp.*;
500\end{listing-java}
501to potentially import all the JVM symbol from the \code{org.armedbear.lisp}
502namespace.
503
504There can only ever be a single Lisp interpreter per \textsc{JVM} instance.  A
505reference to this interpreter is obtained by calling the static method
506\code{Interpreter.createInstance()}.
507
508\begin{listing-java}
509  Interpreter interpreter = Interpreter.createInstance();
510\end{listing-java}
511
512If this method has already been invoked in the lifetime of the current
513Java process it will return \texttt{null}, so if you are writing \textsc{Java}
514whose life-cycle is a bit out of your control (like in a \textsc{Java} servlet),
515a safer invocation pattern might be:
516
517\begin{listing-java}
518  Interpreter interpreter = Interpreter.getInstance();
519  if (interpreter == null) {
520    interpreter = Interpreter.createInstance();
521  }
522\end{listing-java}
523
524
525The Lisp \code{eval} primitive may simply be passed strings for evaluation:
526
527\begin{listing-java}
528  String line = "(load \"file.lisp\")";
529  LispObject result = interpreter.eval(line);
530\end{listing-java}
531
532Notice that all possible return values from an arbitrary Lisp
533computation are collapsed into a single return value.  Doing useful
534further computation on the \code{LispObject} depends on knowing what the
535result of the computation might be.  This usually involves some amount
536of \code{instanceof} introspection, and forms a whole topic to itself
537(see Section~\ref{topic:Introspecting a LispObject},
538page~\pageref{topic:Introspecting a LispObject}).
539
540Using \code{eval} involves the Lisp interpreter.  Lisp functions may
541also be directly invoked by Java method calls as follows.  One simply
542locates the package containing the symbol, obtains a reference to the
543symbol, and then invokes the \code{execute()} method with the desired
544parameters.
545
546\begin{listing-java}
547  interpreter.eval("(defun foo (msg)" +
548    "(format nil \"You told me '~A'~%\" msg))");
549  Package pkg = Packages.findPackage("CL-USER");
550  Symbol foo = pkg.findAccessibleSymbol("FOO");
551  Function fooFunction = (Function)foo.getSymbolFunction();
552  JavaObject parameter = new JavaObject("Lisp is fun!");
553  LispObject result = fooFunction.execute(parameter);
554  // How to get the "naked string value"?
555  System.out.println("The result was " + result.writeToString());
556\end{listing-java}
557
558If one is calling a function in the CL package, the syntax can become
559considerably simpler.  If we can locate the instance of definition in
560the ABCL Java source, we can invoke the symbol directly.  For instance,
561to tell if a \code{LispObject} is (Lisp) \texttt{NIL}, we can invoke the
562CL function \code{NULL} in the following way:
563
564
565\begin{listing-java}
566  boolean nullp(LispObject object) {
567    LispObject result = Primitives.NULL.execute(object);
568    if (result == NIL) { 
569      return false;
570    }
571    return true;
572 }
573\end{listing-java}
574
575Note, that the symbol \code{nil} is explicitly named in the
576\textsc{Java} namespace as \code{Symbol.NIL} but is always present in
577the local namespace in its unadorned form for the convenience of the
578User.
579
580\subsubsection{Multiple Values}
581
582After a call to a function that returns Lisp multiple values, the
583values are associated with the executing \code{LispThread} until the
584next call into Lisp.  One may access the values object as a list of
585\code{LispObject} instances via a call to \code{getValues()} on that
586thread reference
587as evidenced by the following code:
588
589\begin{listing-java}
590
591  org.armedbear.lisp.Package cl = Packages.findPackage("CL");
592  Symbol valuesSymbol = cl.findAccessibleSymbol("VALUES");
593  LispObject[] valuesArgs = {
594    LispInteger.getInstance(1), LispInteger.getInstance(2)
595  };
596  // equivalent to ``(values 1 2)''
597  LispObject result = valuesSymbol.execute(valuesArgs);
598  LispObject[] values = LispThread.currentThread().getValues();
599  for (LispObject value: values) {
600    System.out.println("value ==> " + value.printObject());
601  }
602\end{listing-java}
603
604\subsubsection{Introspecting a LispObject}
605\label{topic:Introspecting a LispObject}
606
607We present various patterns for introspecting an arbitrary
608\code{LispObject} which can hold the result of every Lisp evaluation
609into semantics that Java can meaningfully deal with.
610
611\paragraph{LispObject as \code{boolean}}
612
613If the \code{LispObject} is to be interpreted as a generalized boolean
614value, one can use \code{getBooleanValue()} to convert to Java:
615
616\begin{listing-java}
617   LispObject object = Symbol.NIL;
618   boolean javaValue = object.getBooleanValue();
619\end{listing-java}
620
621Since in Lisp any value other than \code{NIL} means "true", Java
622equality can also be used, which is a bit easier to type and better in
623terms of information it conveys to the compiler:
624
625\begin{listing-java}
626    boolean javaValue = (object != Symbol.NIL);
627\end{listing-java}
628
629\paragraph{LispObject as a list}
630
631If \code{LispObject} is a list, it will have the type \code{Cons}.  One
632can then use the \code{copyToArray} method to make things a bit more
633suitable for Java iteration.
634
635\begin{listing-java}
636  LispObject result = interpreter.eval("'(1 2 4 5)");
637  if (result instanceof Cons) {
638    LispObject array[] = ((Cons)result.copyToArray());
639    ...
640  }
641\end{listing-java}
642
643A more Lispy way to iterate down a list is to use the `cdr()` access
644function just as like one would traverse a list in Lisp:;
645
646\begin{listing-java}
647  LispObject result = interpreter.eval("'(1 2 4 5)");
648  while (result != Symbol.NIL) {
649    doSomething(result.car());
650    result = result.cdr();
651  }
652\end{listing-java}
653
654\section{Java Scripting API (JSR-223)}
655\label{sec:java-scripting-api}
656
657ABCL can be built with support for JSR-223~\cite{jsr-223}, which offers
658a language-agnostic API to invoke other languages from Java. The binary
659distribution download-able from ABCL's homepage is built with JSR-223
660support. If you're building ABCL from source on a pre-1.6 JVM, you need
661to have a JSR-223 implementation in your classpath (such as Apache
662Commons BSF 3.x or greater) in order to build ABCL with JSR-223 support;
663otherwise, this feature will not be built.
664
665This section describes the design decisions behind the ABCL JSR-223
666support. It is not a description of what JSR-223 is or a tutorial on
667how to use it. See
668\url{http://abcl.org/trac/browser/trunk/abcl/examples/jsr-223}
669for example usage.
670
671\subsection{Conversions}
672
673In general, ABCL's implementation of the JSR-223 API performs implicit
674conversion from Java objects to Lisp objects when invoking Lisp from
675Java, and the opposite when returning values from Java to Lisp. This
676potentially reduces coupling between user code and ABCL. To avoid such
677conversions, wrap the relevant objects in \code{JavaObject} instances.
678
679\subsection{Implemented JSR-223 interfaces}
680
681JSR-223 defines three main interfaces, of which two (\code{Invocable}
682and \code{Compilable}) are optional. ABCL implements all the three
683interfaces - \code{ScriptEngine} and the two optional ones - almost
684completely. While the JSR-223 API is not specific to a single scripting
685language, it was designed with languages with a more or less Java-like
686object model in mind: languages such as Javascript, Python, Ruby, which
687have a concept of "class" or "object" with "fields" and "methods". Lisp
688is a bit different, so certain adaptations were made, and in one case a
689method has been left unimplemented since it does not map at all to Lisp.
690
691\subsubsection{The ScriptEngine}
692
693The main interface defined by JSR-223, \code{javax.script.ScriptEngine},
694is implemented by the class
695\code{org.armedbear.lisp.scripting.AbclScriptEngine}. \code{AbclScriptEngine}
696is a singleton, reflecting the fact that ABCL is a singleton as
697well. You can obtain an instance of \code{AbclScriptEngine} using the
698\code{AbclScriptEngineFactory} or by using the service provider
699mechanism through \code{ScriptEngineManager} (refer to the
700\texttt{javax.script} documentation).
701
702\subsection{Start-up and configuration file}
703
704At start-up (i.e. when its constructor is invoked, as part of the
705static initialization phase of \code{AbclScriptEngineFactory}) the ABCL
706script engine attempts to load an "init file" from the classpath
707(\texttt{/abcl-script-config.lisp}). If present, this file can be used to
708customize the behavior of the engine, by setting a number of
709variables in the \code{ABCL-SCRIPT} package. Here is a list of the available
710variables:
711
712\begin{description}
713\item[\texttt{*use-throwing-debugger*}] controls whether ABCL uses a
714  non-standard debugging hook function to throw a Java exception
715  instead of dropping into the debugger in case of unhandled error
716  conditions.
717  \begin{itemize}
718  \item Default value: \texttt{T}
719  \item Rationale: it is more convenient for Java programmers using
720    Lisp as a scripting language to have it return exceptions to Java
721    instead of handling them in the Lisp world.
722  \item Known Issues: the non-standard debugger hook has been reported
723    to misbehave in certain circumstances, so consider disabling it if
724    it doesn't work for you.
725  \end{itemize}
726\item[\texttt{*launch-swank-at-startup*}] If true, Swank will be launched at
727  startup. See \texttt{*swank-dir*} and \texttt{*swank-port*}.
728  \begin{itemize}
729  \item Default value: \texttt{NIL}
730  \end{itemize}
731\item[\texttt{*swank-dir*}] The directory where Swank is installed. Must be set
732  if \texttt{*launch-swank-at-startup*} is true.
733\item[\texttt{*swank-port*}] The port where Swank will listen for
734  connections. Must be set if \texttt{*launch-swank-at-startup*} is
735  true.
736  \begin{itemize}
737  \item Default value: 4005
738  \end{itemize}
739\end{description}
740
741Additionally, at startup the AbclScriptEngine will \code{(require
742  'asdf)} - in fact, it uses asdf to load Swank.
743
744\subsection{Evaluation}
745
746Code is read and evaluated in the package \code{ABCL-SCRIPT-USER}. This
747packages \texttt{USE}s the \code{COMMON-LISP}, \code{JAVA} and
748\code{ABCL-SCRIPT} packages. Future versions of the script engine might
749make this default package configurable. The \code{CL:LOAD} function is
750used under the hood for evaluating code, and thus the behavior of
751\code{LOAD} is guaranteed. This allows, among other things,
752\code{IN-PACKAGE} forms to change the package in which the loaded code
753is read.
754
755It is possible to evaluate code in what JSR-223 calls a
756``ScriptContext'' (basically a flat environment of name$\rightarrow$value
757pairs). This context is used to establish special bindings for all the
758variables defined in it; since variable names are strings from Java's
759point of view, they are first interned using \code{READ-FROM-STRING} with, as
760usual, \code{ABCL-SCRIPT-USER} as the default package. Variables are declared
761special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
762functions work in a null lexical environment and would ignore
763non-special bindings.
764
765Contrary to what the function \code{LOAD} does, evaluation of a series
766of forms returns the value of the last form instead of T, so the
767evaluation of short scripts does the Right Thing.
768
769\subsection{Compilation}
770
771\code{AbclScriptEngine} implements the \code{javax.script.Compilable}
772interface. Currently it only supports compilation using temporary
773files. Compiled code, returned as an instance of
774\texttt{javax.script.CompiledScript}, is read, compiled and executed
775by default in the \code{abcl-script-user} package, just like evaluated
776code.  In contrast to evaluated code, though, due to the way the
777\textsc{ABCL} compiler works, compiled code contains no reference to
778top-level self-evaluating objects (like numbers or strings). Thus,
779when evaluated, a piece of compiled code will return the value of the
780last non-self-evaluating form: for example the code
781``\code{(do-something) 42}'' will return 42 when interpreted, but will
782return the result of (do-something) when compiled and later
783evaluated. To ensure consistency of behavior between interpreted and
784compiled code, make sure the last form is always a compound form - at
785least \code{(identity some-literal-object)}. Note that this issue
786should not matter in real code, where it is unlikely that a top-level
787self-evaluating form will appear as the last form in a file (in fact,
788the Common Lisp load function always returns \code{t} upon success;
789with \textsc{JSR-223} this policy has been changed to make evaluation
790of small code snippets work as intended).
791
792\subsection{Invocation of functions and methods}
793
794AbclScriptEngine implements the \code{javax.script.Invocable}
795interface, which allows to directly call Lisp functions and methods,
796and to obtain Lisp implementations of Java interfaces. This is only
797partially possible with Lisp since it has functions, but not methods -
798not in the traditional OO sense, at least, since Lisp methods are not
799attached to objects but belong to generic functions. Thus, the method
800\code{invokeMethod()} is not implemented and throws an
801\texttt{UnsupportedOperationException} when called. The \code{invokeFunction()}
802method should be used to call both regular and generic functions.
803
804\subsection{Implementation of Java interfaces in Lisp}
805
806ABCL can use the Java reflection-based proxy feature to implement Java
807interfaces in Lisp. It has several built-in ways to implement an
808interface, and supports definition of new ones. The
809\code{JAVA:JMAKE-PROXY} generic function is used to make such
810proxies. It has the following signature:
811
812\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
813
814\code{interface} is a Java interface metaobject (e.g. obtained by
815invoking \code{jclass}) or a string naming a Java
816interface. \code{implementation} is the object used to implement the
817interface - several built-in methods of jmake-proxy exist for various
818types of implementations. \code{lisp-this} is an object passed to the
819closures implementing the Lisp "methods" of the interface, and
820defaults to \code{NIL}.
821
822The returned proxy is an instance of the interface, with methods
823implemented with Lisp functions.
824
825Built-in interface-implementation types include:
826
827\begin{itemize}
828\item a single Lisp function which, upon invocation of any method in
829  the interface, will be passed the method name, the Lisp-this object,
830  and all the parameters. Useful for interfaces with a single method,
831  or to implement custom interface-implementation strategies.
832\item a hash-map of method-name $\rightarrow$ Lisp function mappings. Function
833  signature is \code{(lisp-this \&rest args)}.
834\item a Lisp package. The name of the Java method to invoke is first
835  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
836  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
837  the package. If it exists and is fbound, the corresponding function
838  will be called. Function signature is as the hash-table case.
839\end{itemize}
840
841This functionality is exposed by the class \code{AbclScriptEngine} via
842the two methods \code{getInterface(Class)} and
843\code{getInterface(Object, Class)}. The former returns an interface
844implemented with the current Lisp package, the latter allows the
845programmer to pass an interface-implementation object which will in turn
846be passed to the \code{jmake-proxy} generic function.
847
848\subsection{Implementation of Java classes in Lisp}
849
850See \code{JAVA:JNEW-RUNTIME-CLASS} in \ref{JAVA:JNEW-RUNTIME-CLASS}.
851
852
853\chapter{Implementation Dependent Extensions}
854
855As outlined by the \textsc{CLHS} \textsc{ANSI} conformance guidelines,
856we document the extensions to the Armed Bear Common Lisp implementation made
857accessible to the user by virtue of being an exported symbol in the
858\code{java}, \code{threads}, or \code{extensions} packages.
859
860\section{JAVA}
861
862\subsection{Modifying the JVM CLASSPATH}
863
864The \code{JAVA:ADD-TO-CLASSPATH} generic functions allows one to add the
865specified pathname or list of pathnames to the current classpath
866used by \textsc{ABCL}, allowing the dynamic loading of \textsc{JVM} objects:
867
868\begin{listing-lisp}
869CL-USER> (add-to-classpath "/path/to/some.jar")
870\end{listing-lisp}
871
872N.b \code{ADD-TO-CLASSPATH} only affects the classloader used by \textsc{ABCL}
873(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
874no effect on \textsc{Java} code outside \textsc{ABCL}.
875
876\subsection{Creating a synthetic Java Class at Runtime}
877
878See \code{JAVA:JNEW-RUNTIME-CLASS} on \ref{JAVA:JNEW-RUNTIME-CLASS}.
879
880% include autogen docs for the JAVA package.
881\include{java}
882
883\section{THREADS}
884
885The extensions for handling multithreaded execution are collected in
886the \code{THREADS} package.  Most of the abstractions in Doug Lea's
887excellent \code{java.util.concurrent} packages may be manipulated
888directly via the JSS contrib to great effect \cite{lea-1998}
889
890% include autogen docs for the THREADS package.
891\include{threads}
892
893\section{EXTENSIONS}
894
895The symbols in the \code{extensions} package (often referenced by its
896shorter nickname \code{ext}) constitutes extensions to the
897\textsc{ANSI} standard that are potentially useful to the user.  They
898include functions for manipulating network sockets, running external
899programs, registering object finalizers, constructing reference weakly
900held by the garbage collector and others.
901
902See \cite{RHODES2007} for a generic function interface to the native
903\textsc{JVM} contract for \code{java.util.List}.
904
905% include autogen docs for the EXTENSIONS package.
906\include{extensions}
907
908\chapter{Beyond ANSI}
909
910Naturally, in striving to be a useful contemporary \textsc{Common Lisp}
911implementation, \textsc{ABCL} endeavors to include extensions beyond the ANSI
912specification which are either widely adopted or are especially useful
913in working with the hosting \textsc{JVM}.
914
915\section{Compiler to Java Virtual Machine Bytecode}
916
917The \code{CL:COMPILE-FILE} interface emits a packed fasl\footnote{The
918term ``fasl'' is short for ``fast loader'', which in \textsc{Common
919  Lisp} implementations refers} format whose \code{CL:PATHNAME} has
920the \code{TYPE} ``abcl''.  Structurally, \textsc{ABCL}'s fasls are an
921operating system neutral byte archives packaged in the zip compression
922format which contain artifacts whose loading \code{CL:LOAD}
923understands.  Internally, our fasls contain a piece of Lisp that
924\code{CL:LOAD} interprets as instructions to load the Java classes
925emitted by the \textsc{ABCL} Lisp compiler.  These ``synthetic''
926classes have a JVM class file version of ``49.0''.
927
928% TODO check on what the compiler is currently emitting
929
930\subsection{Compiler Diagnostics}
931
932By default, the interface to the compiler does not signal warnings
933that result in its invocation, outputing diagnostics to the standard
934reporting stream.  The generalized boolean
935\code{JVM:*RESIGNAL-COMPILER-WARNINGS*} provides the interface to
936enabling the compiler to signal all warnings.
937
938
939\section{Pathname}
940
941We implement an extension to the \code{CL:PATHNAME} that allows for
942the description and retrieval of resources named in a
943\textsc{URI} \footnote{A \textsc{URI} is essentially a superset of
944  what is commonly understood as a \textsc{URL} We sometimes use the
945  term URL as shorthand in describing the URL Pathnames, even though
946  the corresponding encoding is more akin to a URI as described in
947  RFC3986 \cite{rfc3986}.}  scheme that the \textsc{JVM}
948``understands''.  By definition, support is built-in into the JVM to
949access the ``http'' and ``https'' schemes but additional protocol
950handlers may be installed at runtime by having \textsc{JVM} symbols
951present in the \code{sun.net.protocol.dynamic} package. See
952\cite{maso2000} for more details.
953
954\textsc{ABCL} has created specializations of the ANSI
955\code{CL:PATHNAME} object to enable to use of \textsc{URI}s to address
956dynamically loaded resources for the JVM.  The \code{EXT:URL-PATHNAME}
957specialization has a corresponding \textsc{URI} whose canonical
958representation is defined to be the \code{NAMESTRING} of the
959\code{CL:PATHNAME}. The \code{EXT:JAR-PATHNAME} extension further
960specializes the the \code{EXT:URL-PATHNAME} to provide access to
961components of zip archives. 
962
963% RDF description of type hierarchy
964% TODO Render via some LaTeX mode for graphviz?
965\begin{verbatim}
966  @prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .
967  @prefix ext:   <http://abcl.org/cl/package/extensions/> .
968  @prefix cl:    <http://abcl.org/cl/package/common-lisp/> .
969 
970  <ext:jar-pathname>    rdfs:subClassOf <ext:url-pathname> .
971  <ext:url-pathname>    rdfs:subClassOf <cl:pathname> .
972  <cl:logical-pathname> rdfs:subClassOf <cl:pathname> .
973\end{verbatim}
974
975\label{EXTENSIONS:URL-PATHNAME}
976\index{URL-PATHNAME}
977
978\label{EXTENSIONS:JAR-PATHNAME}
979\index{JAR-PATHNAME}
980
981Both the \code{EXT:URL-PATHNAME} and \code{EXT:JAR-PATHNAME} objects
982may be used anywhere a \code{CL:PATHNAME} is accepted with the
983following caveats:
984
985\begin{itemize}
986
987\item A stream obtained via \code{CL:OPEN} on a \code{CL:URL-PATHNAME}
988  cannot be the target of write operations.
989
990\index{URI}
991\item Any results of canonicalization procedures performed on the
992  \textsc{URI} via local or network resolutions are discarded between
993  attempts (i.e. the implementation does not attempt to cache the
994  results of current name resolution of the URI for underlying
995  resource unless it is requested to be resolved.)  Upon resolution,
996  any canonicalization procedures followed in resolving the resource
997  (e.g. following redirects) are discarded.  Users may programatically
998  initiate a new, local computation of the resolution of the resource
999  by applying the \code{CL:TRUENAME} function to a
1000  \code{EXT:URL-PATHNAME} object.  Depending on the reliability and
1001  properties of your local \textsc{REST} infrastructure, these results
1002  may not necessarily be idempotent over time\footnote {See
1003    \cite{uri-pathname} for the design and implementation notes for
1004    the technical details}.  A future implementation may attempt to
1005  expose an API to observer/customize the content negotiation
1006  initiated during retrieval of the representation of a given
1007  resource, which is currently handled at the application level.
1008
1009\end{itemize}
1010
1011The implementation of \code{EXT:URL-PATHNAME} allows the \textsc{ABCL}
1012user to dynamically load code from the network.  For example,
1013\textsc{Quicklisp} (\cite{quicklisp}) may be completely installed from
1014the \textsc{REPL} as the single form:
1015
1016\begin{listing-lisp}
1017  CL-USER> (load "http://beta.quicklisp.org/quicklisp.lisp")
1018\end{listing-lisp}
1019
1020will load and execute the Quicklisp setup code.
1021
1022The implementation currently breaks \textsc{ANSI} conformance by allowing the
1023types able to be \code{CL:READ} for the \code{DEVICE} to return a possible \code{CONS} of
1024\code{CL:PATHNAME} objects.  %% citation from CLHS needed.
1025
1026In order to ``smooth over'' the bit about types being \code{CL:READ}
1027from \code{CL:PATHNAME} components, we extend the semantics for the
1028usual PATHNAME merge semantics when \code{*DEFAULT-PATHNAME-DEFAULTS*}
1029contains a \code{EXT:JAR-PATHNAME} with the ``do what I mean''
1030algorithm described in \ref{section:conformance} on page
1031\pageref{section:conformance}.
1032
1033%See \ref{_:quicklisp} on page \pageref{_:quicklisp}.
1034
1035\subsubsection{Implementation}
1036
1037The implementation of these extensions stores all the additional
1038information in the \code{CL:PATHNAME} object itself in ways that while strictly
1039speaking are conformant, nonetheless may trip up libraries that don't
1040expect the following:
1041
1042\begin{itemize}
1043\item \code{DEVICE} can be either a string denoting a drive letter
1044  under \textsc{DOS} or a list of exactly one or two elements.  If
1045  \code{DEVICE} is a list, it denotes a \code{EXT:JAR-PATHNAME}, with
1046  the entries containing \code{CL:PATHNAME} objects which describe the
1047  outer and (possibly inner) locations of the jar
1048  archive \footnote{The case of inner and outer
1049    \code{EXT:JAR-PATHNAME} \ref{EXT:JAR-PATHNAME} arises when zip
1050    archives themselves contain zip archives which is the case when
1051    the ABCL fasl is included in the abcl.jar zip archive.}.
1052
1053\item A \code{EXT:URL-PATHNAME} always has a \code{HOST} component that is a
1054  property list.  The values of the \code{HOST} property list are
1055  always character strings.  The allowed keys have the following meanings:
1056  \begin{description}
1057  \item[:SCHEME] Scheme of URI ("http", "ftp", "bundle", etc.)
1058  \item[:AUTHORITY] Valid authority according to the URI scheme.  For
1059    "http" this could be "example.org:8080".
1060  \item[:QUERY] The query of the \textsc{URI} 
1061  \item[:FRAGMENT] The fragment portion of the \textsc{URI}
1062  \end{description}
1063
1064\item In order to encapsulate the implementation decisions for these
1065  meanings, the following functions provide a SETF-able API for
1066  reading and writing such values: \code{URL-PATHNAME-QUERY},
1067  \code{URL-PATHNAME-FRAGMENT}, \code{URL-PATHNAME-AUTHORITY}, and
1068  \code{URL-PATHNAME-SCHEME}.  The specific subtype of a Pathname may
1069  be determined with the predicates \code{PATHNAME-URL-P} and
1070  \code{PATHNAME-JAR-P}.
1071
1072\label{EXTENSIONS:URL-PATHNAME-SCHEME}
1073\index{URL-PATHNAME-SCHEME}
1074
1075\label{EXTENSIONS:URL-PATHNAME-FRAGMENT}
1076\index{URL-PATHNAME-FRAGMENT}
1077
1078\label{EXTENSIONS:URL-PATHNAME-AUTHORITY}
1079\index{URL-PATHNAME-AUTHORITY}
1080
1081\label{EXTENSIONS:PATHNAME-URL-P}
1082\index{PATHNAME-URL-P}
1083
1084\label{EXTENSIONS:URL-PATHNAME-QUERY}
1085\index{URL-PATHNAME-QUERY}
1086
1087\end{itemize}
1088
1089\section{Package-Local Nicknames}
1090\label{sec:pack-local-nickn}
1091
1092ABCL allows giving packages local nicknames: they allow short and
1093easy-to-use names to be used without fear of name conflict associated
1094with normal nicknames.\footnote{Package-local nicknames were originally
1095developed in SBCL.}
1096
1097A local nickname is valid only when inside the package for which it
1098has been specified. Different packages can use same local nickname for
1099different global names, or different local nickname for same global
1100name.
1101
1102Symbol \code{:package-local-nicknames} in \code{*features*} denotes the
1103support for this feature.
1104
1105\index{DEFPACKAGE}
1106The options to \code{defpackage} are extended with a new option
1107\code{:local-nicknames (local-nickname actual-package-name)*}.
1108
1109The new package has the specified local nicknames for the corresponding
1110actual packages.
1111
1112Example:
1113\begin{listing-lisp}
1114(defpackage :bar (:intern "X"))
1115(defpackage :foo (:intern "X"))
1116(defpackage :quux (:use :cl)
1117  (:local-nicknames (:bar :foo) (:foo :bar)))
1118(find-symbol "X" :foo) ; => FOO::X
1119(find-symbol "X" :bar) ; => BAR::X
1120(let ((*package* (find-package :quux)))
1121  (find-symbol "X" :foo))               ; => BAR::X
1122(let ((*package* (find-package :quux)))
1123  (find-symbol "X" :bar))               ; => FOO::X
1124\end{listing-lisp}
1125
1126\index{PACKAGE-LOCAL-NICKNAMES}
1127--- Function: \textbf{package-local-nicknames} [\textbf{ext}] \textit{package-designator}
1128
1129\begin{adjustwidth}{5em}{5em}
1130  Returns an ALIST of \code{(local-nickname . actual-package)}
1131  describing the nicknames local to the designated package.
1132
1133  When in the designated package, calls to \code{find-package} with any
1134  of the local-nicknames will return the corresponding actual-package
1135  instead. This also affects all implied calls to \code{find-package},
1136  including those performed by the reader.
1137
1138  When printing a package prefix for a symbol with a package local
1139  nickname, the local nickname is used instead of the real name in order
1140  to preserve print-read consistency.
1141\end{adjustwidth}
1142
1143\index{PACKAGE-LOCALLY-NICKNAMED-BY-LIST}
1144--- Function: \textbf{package-locally-nicknamed-by-list} [\textbf{ext}] \textit{package-designator}
1145
1146\begin{adjustwidth}{5em}{5em}
1147Returns a list of packages which have a local nickname for the
1148designated package.
1149\end{adjustwidth}
1150
1151\index{ADD-PACKAGE-LOCAL-NICKNAME}
1152--- Function: \textbf{add-package-local-nickname} [\textbf{ext}] \textit{local-nickname actual-package \&optional package-designator}
1153
1154\begin{adjustwidth}{5em}{5em}
1155  Adds \code{local-nickname} for \code{actual-package} in the designated
1156  package, defaulting to current package. \code{local-nickname} must be
1157  a string designator, and \code{actual-package} must be a package
1158  designator.
1159
1160  Returns the designated package.
1161
1162  Signals an error if \code{local-nickname} is already a package local
1163  nickname for a different package, or if \code{local-nickname} is one
1164  of "CL", "COMMON-LISP", or, "KEYWORD", or if \code{local-nickname} is
1165  a global name or nickname for the package to which the nickname would
1166  be added.
1167
1168  When in the designated package, calls to \code{find-package} with the
1169  \code{local-nickname} will return the package the designated
1170  \code{actual-package} instead. This also affects all implied calls to
1171  \code{find-package}, including those performed by the reader.
1172
1173  When printing a package prefix for a symbol with a package local
1174  nickname, local nickname is used instead of the real name in order to
1175  preserve print-read consistency.
1176\end{adjustwidth}
1177
1178\index{REMOVE-PACKAGE-LOCAL-NICKNAME}
1179--- Function: \textbf{remove-package-local-nickname} [\textbf{ext}] \textit{old-nickname \&optional package-designator}
1180
1181\begin{adjustwidth}{5em}{5em}
1182  If the designated package had \code{old-nickname} as a local nickname
1183  for another package, it is removed. Returns true if the nickname
1184  existed and was removed, and \code{nil} otherwise.
1185\end{adjustwidth}
1186
1187
1188         
1189\section{Extensible Sequences}
1190
1191The SEQUENCE package fully implements Christopher Rhodes' proposal for
1192extensible sequences.  These user extensible sequences are used
1193directly in \code{java-collections.lisp} provide these CLOS
1194abstractions on the standard Java collection classes as defined by the
1195\code{java.util.List} contract.
1196
1197
1198%% an Example of using java.util.Lisp in Lisp would be nice
1199
1200This extension is not automatically loaded by the implementation.   It
1201may be loaded via:
1202
1203\begin{listing-lisp}
1204CL-USER> (require :java-collections)
1205\end{listing-lisp}
1206
1207if both extensible sequences and their application to Java collections
1208is required, or
1209
1210\begin{listing-lisp}
1211CL-USER> (require :extensible-sequences)
1212\end{listing-lisp}
1213
1214if only the extensible sequences API as specified in \cite{RHODES2007} is
1215required.
1216
1217Note that \code{(require :java-collections)} must be issued before
1218\code{java.util.List} or any subclass is used as a specializer in a \textsc{CLOS}
1219method definition (see the section below).
1220
1221See Rhodes2007 \cite{RHODES2007} for the an overview of the
1222abstractions of the \code{java.util.collection} package afforded by
1223\code{JAVA-COLLECTIONS}.
1224
1225\section{Extensions to CLOS}
1226
1227\subsection{Metaobject Protocol}
1228
1229\textsc{ABCL} implements the metaobject protocol for \textsc{CLOS} as
1230specified in \textsc{(A)MOP}.  The symbols are exported from the
1231package \code{MOP}.
1232
1233Contrary to the AMOP specification and following \textsc{SBCL}'s lead,
1234the metaclass \code{funcallable-standard-object} has
1235\code{funcallable-standard-class} as metaclass instead of
1236\code{standard-class}.
1237
1238\textsc{ABCL}'s fidelity to the AMOP specification is codified as part
1239of Pascal Costanza's \code{closer-mop} \ref{closer-mop} \cite{closer-mop}.
1240
1241\subsection{Specializing on Java classes}
1242
1243There is an additional syntax for specializing the parameter of a
1244generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
1245where \code{CLASS-STRING} is a string naming a Java class in dotted package
1246form.
1247
1248For instance the following specialization would perhaps allow one to
1249print more information about the contents of a \code{java.util.Collection}
1250object
1251
1252\begin{listing-lisp}
1253(defmethod print-object ((coll (java:jclass "java.util.Collection"))
1254                         stream)
1255  ;;; ...
1256)
1257\end{listing-lisp}
1258
1259If the class had been loaded via a classloader other than the original
1260the class you wish to specialize on, one needs to specify the
1261classloader as an optional third argument.
1262
1263\begin{listing-lisp}
1264
1265(defparameter *other-classloader*
1266  (jcall "getBaseLoader" cl-user::*classpath-manager*))
1267 
1268(defmethod print-object
1269   ((device-id (java:jclass "dto.nbi.service.hdm.alcatel.com.NBIDeviceID"
1270                            *other-classloader*))
1271    stream)
1272  ;;; ...
1273)
1274\end{listing-lisp}
1275
1276\section{Extensions to the Reader}
1277
1278We implement a special hexadecimal escape sequence for specifying 32
1279bit characters to the Lisp reader\footnote{This represents a
1280  compromise with contemporary in 2011 32bit hosting architectures for
1281  which we wish to make text processing efficient.  Should the User
1282  require more control over \textsc{UNICODE} processing we recommend Edi Weisz'
1283  excellent work with \textsc|{FLEXI-STREAMS}  which we fully support}, namely we
1284allow a sequences of the form \verb~#\U~\emph{\texttt{xxxx}} to be processed
1285by the reader as character whose code is specified by the hexadecimal
1286digits \emph{\texttt{xxxx}}.  The hexadecimal sequence may be one to four digits
1287long.
1288
1289% Why doesn't ALEXANDRIA work?  Good question: Alexandria from
1290% Quicklisp 2010-10-07 fails a number of tests:
1291%% Form: (ALEXANDRIA.0.DEV:TYPE= 'LIST '(OR NULL CONS))
1292%% Expected values: T
1293%%                  T
1294%% Actual values: NIL
1295%%                T.
1296%% Test ALEXANDRIA-TESTS::TYPE=.3 failed
1297%% Form: (ALEXANDRIA.0.DEV:TYPE= 'NULL '(AND SYMBOL LIST))
1298%% Expected values: T
1299%%                  T
1300%% Actual values: NIL
1301%%                NIL.
1302
1303
1304Note that this sequence is never output by the implementation.  Instead,
1305the corresponding Unicode character is output for characters whose
1306code is greater than 0x00ff.
1307
1308\section{Overloading of the CL:REQUIRE Mechanism}
1309
1310The \code{CL:REQUIRE} mechanism is overloaded by attaching these
1311semantics to the execution of \code{REQUIRE} on the following symbols:
1312
1313\begin{description}
1314
1315  \item{\code{ASDF}} 
1316    Loads the \textsc{ASDF} implementation shipped
1317    with the implementation.  After \textsc{ASDF} has been loaded in
1318    this manner, symbols passed to \code{CL:REQUIRE} which are
1319    otherwise unresolved, are passed to ASDF for a chance for
1320    resolution.  This means, for instance if \code{CL-PPCRE} can be
1321    located as a loadable \textsc{ASDF} system \code{(require
1322      :cl-ppcre)} is equivalent to \code{(asdf:load-system
1323      :cl-ppcre)}.
1324
1325  \item{\code{ABCL-CONTRIB}} 
1326    Locates and pushes the top-level contents of
1327    ``abcl-contrib.jar'' into the \textsc{ASDF} central registry. 
1328
1329    \begin{enumerate}
1330      \item \code{abcl-asdf} 
1331        Functions for loading JVM artifacts
1332        dynamically, hooking into ASDF 3 objects where possible.
1333      \item \code{asdf-jar} 
1334        Package addressable JVM artifacts via
1335        \code{abcl-asdf} descriptions as a single binary artifact
1336        including recursive dependencies.
1337      \item \code{mvn} 
1338        These systems name common JVM artifacts from
1339        the distributed pom.xml graph of Maven Aether:
1340        \begin{enumerate}
1341          \item \code{jna} 
1342            Dynamically load 'jna.jar' version 5.5.0
1343            from the network \footnote{This loading can be inhibited
1344              if, at runtime, the Java class corresponding
1345              ``:classname'' clause of the system definition is present.}
1346        \end{enumerate}
1347      \item \code{quicklisp-abcl} Boot a local Quicklisp installation
1348        via the ASDF:IRI type introduced via ABCL-ASDF.
1349
1350      \item \code{jfli} A descendant of Rich Hickey's pre-Clojure work
1351        on the JVM.
1352      \item \code{jss} Introduces dynamic inspection of present
1353        symbols via the \code{SHARPSIGN-QUOTATION\_MARK} macros as Java Syntax
1354        Sucks
1355      \item \code{abcl-introspect} Provides a framework for
1356        introspecting runtime Java and Lisp object values.
1357      \item \code{abcl-build} Provides a toolkit for building ABCL
1358        from source, as well as installing the necessary tools for
1359        such builds.
1360       
1361\end{enumerate}
1362
1363\end{description}
1364
1365The user may extend the \code{CL:REQUIRE} mechanism by pushing
1366function hooks into \code{SYSTEM:*MODULE-PROVIDER-FUNCTIONS*}.  Each
1367such hook function takes a single argument containing the symbol
1368passed to \code{CL:REQUIRE} and returns a non-\code{NIL} value if it
1369can successful resolve the symbol.
1370
1371\section{JSS extension of the Reader by SHARPSIGN-DOUBLE-QUOTE}
1372
1373The JSS contrib constitutes an additional, optional extension to the
1374reader in the definition of the \code{SHARPSIGN-DOUBLE-QUOTE}
1375(``\#\"'') reader macro.  See section \ref{section:jss} on page
1376\pageref{section:jss} for more information.
1377
1378\section{ASDF}
1379
1380asdf-3.3.3 (see \cite{asdf}) is packaged as core component of \textsc{ABCL},
1381but not initialized by default, as it relies on the \textsc{CLOS} subsystem
1382which can take a bit of time to start \footnote{While this time is
1383  ``merely'' on the order of seconds for contemporary 2011 machines,
1384  for applications that need to initialize quickly, for example a web
1385  server, this time might be unnecessarily long}.  The packaged \textsc{ASDF}
1386may be loaded by the \textsc{ANSI} \code{REQUIRE} mechanism as
1387follows:
1388
1389\begin{listing-lisp}
1390CL-USER> (require :asdf)
1391\end{listing-lisp}
1392
1393\chapter{Contrib}
1394
1395The \textsc{ABCL} contrib is packaged as a separate jar archive usually named
1396\code{abcl-contrib.jar} or possibly something like
1397\code{abcl-contrib-1.6.1.jar}.  The contrib jar is not loaded by the
1398implementation by default, and must be first initialized by the
1399\code{REQUIRE} mechanism before using any specific contrib:
1400
1401\begin{listing-lisp}
1402CL-USER> (require :abcl-contrib)
1403\end{listing-lisp}
1404
1405\section{abcl-asdf}
1406
1407This contrib enables an additional syntax for \textsc{ASDF} system
1408definition which dynamically loads \textsc{JVM} artifacts such as jar
1409archives via encapsulation of the Maven build tool.  The Maven Aether
1410component can also be directly manipulated by the function associated
1411with the \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} symbol.
1412
1413%ABCL specific contributions to ASDF system definition mainly
1414%concerned with finding JVM artifacts such as jar archives to be
1415%dynamically loaded.
1416
1417
1418When loaded, abcl-asdf adds the following objects to \textsc{ASDF}:
1419\code{JAR-FILE}, \code{JAR-DIRECTORY}, \code{CLASS-FILE-DIRECTORY} and
1420\code{MVN}, exporting them (and others) as public symbols.
1421
1422\subsection{Referencing Maven Artifacts via ASDF}
1423
1424Maven artifacts may be referenced within \textsc{ASDF} system
1425definitions, as the following example references the
1426\code{log4j-1.4.9.jar} JVM artifact which provides a widely-used
1427abstraction for handling logging systems:
1428
1429\begin{listing-lisp}
1430  ;;;; -*- Mode: LISP -*-
1431  (require :asdf)
1432  (in-package :cl-user)
1433
1434  (asdf:defsystem :log4j
1435     :defsystem-depends-on (abcl-asdf)
1436     :components ((:mvn "log4j/log4j" :version "1.4.9")))
1437\end{listing-lisp}
1438
1439\subsection{API}
1440
1441We define an API for \textsc{ABCL-ASDF} as consisting of the following
1442ASDF classes:
1443
1444\code{JAR-DIRECTORY}, \code{JAR-FILE}, and
1445\code{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
1446valid pathname representation.
1447
1448Both the \code{MVN} and \code{IRI} classes descend from
1449\code{ASDF-COMPONENT}, but do not directly have a filesystem location.
1450
1451For use outside of ASDF system definitions, we currently define one
1452method, \code{ABCL-ASDF:RESOLVE-DEPENDENCIES} which locates,
1453downloads, caches, and then loads into the currently executing JVM
1454process all recursive dependencies annotated in the Maven pom.xml
1455graph.
1456
1457\subsection{Directly Instructing Maven to Download JVM Artifacts}
1458
1459Bypassing \textsc{ASDF}, one can directly issue requests for the Maven
1460artifacts to be downloaded
1461
1462\begin{listing-lisp}
1463CL-USER> (abcl-asdf:resolve-dependencies "com.google.gwt"
1464                                         "gwt-user")
1465WARNING: Using LATEST for unspecified version.
1466"/Users/evenson/.m2/repository/com/google/gwt/gwt-user/2.4.0-rc1
1467/gwt-user-2.4.0-rc1.jar:/Users/evenson/.m2/repository/javax/vali
1468dation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/User
1469s/evenson/.m2/repository/javax/validation/validation-api/1.0.0.G
1470A/validation-api-1.0.0.GA-sources.jar"
1471\end{listing-lisp}
1472
1473To actually load the dependency, use the \code{JAVA:ADD-TO-CLASSPATH} generic
1474function:
1475
1476\begin{listing-lisp}
1477CL-USER> (java:add-to-classpath
1478          (abcl-asdf:resolve-dependencies "com.google.gwt"
1479                                          "gwt-user"))
1480\end{listing-lisp}
1481
1482Notice that all recursive dependencies have been located and installed
1483locally from the network as well.
1484
1485More extensive documentations and examples can be found at
1486\url{http://abcl.org/svn/tags/1.6.1/contrib/abcl-asdf/README.markdown}.
1487
1488\section{asdf-jar}
1489
1490The asdf-jar contrib provides a system for packaging \textsc{ASDF}
1491systems into jar archives for \textsc{ABCL}.  Given a running
1492\textsc{ABCL} image with loadable \textsc{ASDF} systems the code in
1493this package will recursively package all the required source and
1494fasls in a jar archive.
1495
1496The documentation for this contrib can be found at
1497\url{http://abcl.org/svn/tags/1.6.1/contrib/asdf-jar/README.markdown}.
1498
1499\section{jss}
1500\label{section:jss}
1501
1502To one used to the more universal syntax of Lisp pairs upon which the
1503definition of read and compile time macros is quite
1504natural \footnote{See Graham's ``On Lisp''
1505  http://lib.store.yahoo.net/lib/paulgraham/onlisp.pdf.}, the Java
1506syntax available to the Java programmer may be said to suck.  To
1507alleviate this situation, the JSS contrib introduces the
1508\code{SHARPSIGN-DOUBLE-QUOTE} (\code{\#"}) reader macro, which allows
1509the the specification of the name of invoking function as the first
1510element of the relevant s-expr which tends to be more congruent to how
1511Lisp programmers seem to be wired to think.
1512
1513While quite useful, we don't expect that the JSS contrib will be the
1514last experiment in wrangling Java from Common Lisp.
1515
1516\subsection{JSS usage}
1517
1518Example:
1519
1520\begin{listing-lisp}
1521CL-USER> (require :abcl-contrib)
1522==> ("ABCL-CONTRIB")
1523CL-USER> (require :jss)
1524==> ("JSS")
1525CL-USER) (#"getProperties" 'java.lang.System)
1526==> #<java.util.Properties {java.runtime.name=Java.... {2FA21ACF}>
1527CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
1528==> #<java.util.Hashtable$Enumerator java.util.Has.... {36B4361A}>
1529\end{listing-lisp} %$ <-- un-confuse Emacs font-lock
1530
1531Some more information on jss can be found in its documentation at
1532\url{http://abcl.org/svn/tags/1.6.1/contrib/jss/README.markdown}
1533
1534\section{jfli}
1535\label{section:jfli}
1536
1537The contrib contains a pure-Java version of JFLI.
1538
1539\url{http://abcl.org/svn/tags/1.6.1/contrib/jfli/README}.
1540
1541\section{abcl-introspect}
1542
1543\textsc{ABCL-INTROSPECT} offers more extensive systems for inspecting the state
1544of the implementation, most notably in integration with SLIME, where
1545the backtrace mechanism is augmented to the point that local
1546variables are inspectable.
1547
1548A compiled function is an instance of a class - This class has
1549multiple instances if it represents a closure, or a single instance if
1550it represents a non-closed-over function.
1551
1552The \textsc{ABCL} compiler stores constants that are used in function
1553execution as private java fields. This includes symbols used to invoke
1554function, locally-defined functions (such as via \code{LABEL} or
1555\code{flet}) and string and other literal objects.
1556\textsc{ABCL-INTROSPECT} implements a ``do what I mean'' API for
1557introspecting these constants.
1558
1559ABCL-INTROSPECT provides access to those internal values, and
1560uses them in at least two ways. First, to annotate locally defined
1561functions with the top-level function they are defined within, and
1562second to search for callers of a give function \footnote{ Since java
1563  functions are strings, local fields also have these strings. In the
1564  context of looking for callers of a function you can also give a
1565  string that names a java method. Same caveat re: false positives.}
1566. This may yield some false positives, such as when a symbol that
1567names a function is also used for some other purpose. It can also have
1568false negatives, as when a function is inlined. Still, it's pretty
1569useful. The second use to to find source locations for frames in the
1570debugger. If the source location for a local function is asked for the
1571location of its 'owner' is instead returns.
1572
1573In order to record information about local functions, ABCL defines a
1574function-plist, which is for the most part unused, but is used here
1575with set of keys indicating where the local function was defined and
1576in what manner, i.e. as normal local function, as a method function,
1577or as an initarg function. There may be other places functions are
1578stashed away (defstructs come to mind) and this file should be added
1579to to take them into account as they are discovered.
1580
1581ABCL-INTROSPECT does not depend on jss, but provides a bit of
1582jss-specific functionality if jss *is* loaded.
1583
1584\url{http://abcl.org/svn/tags/1.6.1/contrib/abcl-introspect/}.
1585
1586\section{abcl-build}
1587\label{sec:abcl-build}
1588
1589\textsc{ABCL-BUILD} constitutes a new implementation for the original
1590Lisp-hosted ABCL build system API in the package \code{ABCL-BUILD}
1591that uses the same build artifacts as all of the other current builds.
1592
1593\subsection{Utilities}
1594
1595\textsc{ABCL-BUILD} consolidates various utilities that are useful
1596for system construction, namely
1597\begin{itemize}
1598
1599\item The ability to introspect the invocation of given executable in
1600  the current implementation process PATH.
1601
1602\item Downloading and unpackaging selected JVM artifacts, namely the
1603  Ant and Maven build tools.  The \code{ABCL-BUILD:WITH-ANT} and
1604  \code{ABCL-BUILD:WITH-MVN} macros abstracts this installation
1605  procedure conveniently away from the User.
1606
1607\item The beginnings of a generic framework to download arbitrary
1608    archives from the network.
1609\end{itemize}
1610
1611\url{http://abcl.org/svn/tags/1.6.1/contrib/abcl-build/}.
1612
1613\section{named-readtables}
1614
1615\code{NAMED-READTABLES} is a library that provides a namespace for
1616readtables akin to the already-existing namespace of packages.
1617
1618\url{https://github.com/melisgl/named-readtables/}.
1619\url{http://abcl.org/svn/tags/1.6.1/contrib/named-readtables/}.
1620
1621\chapter{History}
1622\index{History}
1623
1624\textsc{ABCL} was originally the extension language for the J editor, which was
1625started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1626code that had previously not been released publicly was suddenly
1627committed that enabled ABCL to be plausibly termed an emergent ANSI
1628Common Lisp implementation candidate.
1629
1630From 2006 to 2008, Peter manned the development lists, incorporating
1631patches as made sense.  After a suitable search, Peter nominated Erik
1632H\"{u}lsmann to take over the project.
1633
1634In 2008, the implementation was transferred to the current
1635maintainers, who have strived to improve its usability as a
1636contemporary Common Lisp implementation.
1637
1638On October 22, 2011, with the publication of this Manual explicitly
1639stating the conformance of Armed Bear Common Lisp to \textsc{ANSI}, we
1640released abcl-1.0.0.  We released abcl-1.0.1 as a maintenance release
1641on January 10, 2012.
1642
1643In December 2012, we revised the implementation by adding
1644\textsc{(A)MOP} with the release of abcl-1.1.0.  We released
1645abcl-1.1.1 as a maintenance release on February 14, 2013.
1646
1647At the beginning of June 2013, we enhanced the stability of the
1648implementation with the release of abcl-1.2.1.
1649
1650In March 2014, we introduced the Fourth Edition of the implementation
1651with abcl-1.3.0.  At the end of April 2014, we released abcl-1.3.1 as
1652a maintenance release.
1653
1654In October 2016 we blessed the current \textsc{svn} trunk
1655\url{http://abcl.org/svn/trunk/} as 1.4.0, which includes the
1656community contributions from Vihbu, Olof, Pipping, and Cyrus.  We
1657gingerly stepped into current century by establishing \textsc{git}
1658bridges to the source repositories avaiable via the URIs
1659\url{https://github.com/armedbear/abcl/} and
1660\url{https://gitlab.common-lisp.net/abcl/abcl/} so that pull requests
1661for enhancements to the implementation many be more easily
1662facilitated.
1663
1664In June 2017, we released ABCL 1.5.0 which dropped support for running
1665upon Java 5.
1666
1667Against the falling canvas of 2019 we released ABCL 1.6.0 which
1668provided compatibility with Java 11 while skipping Java 9 and 10.  In
1669April 2020, we offered abcl-1.6.1 as a maintenance release for usage
1670around ELS2020.
1671
1672\appendix 
1673
1674\chapter{The MOP Dictionary}
1675
1676\include{mop}
1677
1678\chapter{The SYSTEM Dictionary}
1679
1680The public interfaces in this package are subject to change with
1681\textsc{ABCL} 1.7.
1682
1683\include{system}
1684
1685\chapter{The JSS Dictionary}
1686
1687These public interfaces are provided by the JSS contrib.
1688
1689\include{jss}
1690
1691\bibliography{abcl}
1692\bibliographystyle{alpha}
1693
1694\printindex
1695
1696\end{document}
Note: See TracBrowser for help on using the repository browser.