source: branches/1.0.x/abcl/doc/manual/abcl.tex @ 13664

Last change on this file since 13664 was 13664, checked in by Mark Evenson, 10 years ago

Ville: add table of contents to Manual.

File size: 40.9 KB
Line 
1% -*- mode: latex; -*-
2% http://en.wikibooks.org/wiki/LaTeX/
3\documentclass[10pt]{book}
4\usepackage{abcl}
5
6\begin{document}
7\title{A Manual for Armed Bear Common Lisp}
8\date{October 21, 2011}
9\author{Mark~Evenson, Erik~Huelsmann, Alessio~Stalla, Ville~Voutilainen}
10
11\maketitle
12
13\tableofcontents
14
15\chapter{Introduction}
16
17Armed Bear is a (mostly) conforming implementation of the ANSI Common
18Lisp standard.  This manual documents the Armed Bear Common Lisp
19implementation for users of the system.
20
21\subsection{Version}
22This manual corresponds to abcl-1.0.0, released on October 22, 2011.
23
24\subsection{License}
25
26The implementation is licensed under the terms of the GPL v2 of June
271991 with the ``classpath-exception'' that makes its deployment in
28commercial settings quite reasonable.  The license is viral in the
29sense that if you change the implementation, and redistribute those
30changes, you are required to provide the source to those changes back
31to be merged with the public trunk.
32
33\subsection{Contributors}
34
35% TODO format this better, optionally link to URI
36
37% Thanks for the markup
38Philipp Marek
39
40% Thanks for the whacky IKVM stuff and keeping the flame alive
41Douglas Miles
42
43% Thanks for JSS
44Alan Ruttenberg
45
46and of course
47
48Peter Graves
49
50\chapter{Running}
51
52\textsc{ABCL} is packaged as a single jar file usually named either
53``abcl.jar'' or possibly``abcl-1.0.0.jar'' if one is using a versioned
54package from your system vendor.  This byte archive can be executed
55under the control of a suitable JVM by using the ``-jar'' option to
56parse the manifest, and select the named class
57(\code{org.armedbear.lisp.Main}) for execution, viz:
58
59\begin{listing-shell}
60  cmd$ java -jar abcl.jar
61\end{listing-shell}
62
63N.b. for the proceeding command to work, the ``java'' executable needs
64to be in your path.
65
66To make it easier to facilitate the use of ABCL in tool chains (such
67as SLIME \footnote{SLIME is the Superior Lisp Mode for Interaction
68  under Emacs}) the invocation is wrapped in a Bourne shell script
69under UNIX or a DOS command script under Windows so that ABCL may be
70executed simply as:
71
72\begin{listing-shell}
73  cmd$ abcl
74\end{listing-shell}
75
76\section{Options}
77
78ABCL supports the following command line options:
79
80\begin{verbatim}
81--help
82    Displays this message.
83--noinform
84    Suppresses the printing of startup information and banner.
85--noinit
86    Suppresses the loading of the '~/.abclrc' startup file.
87--nosystem
88    Suppresses loading the 'system.lisp' customization file.
89--eval <FORM>
90    Evaluates the <FORM> before initializing REPL.
91--load <FILE>
92    Loads the file <FILE> before initializing REPL.
93--load-system-file <FILE>
94    Loads the system file <FILE> before initializing REPL.
95--batch
96    The process evaluates forms specified by arguments and possibly by those
97    by those in the intialization file '~/.abcl', and then exits.
98
99The occurance of '--' copies the remaining arguments, unprocessed, into
100the variable EXTENSIONS:*COMMAND-LINE-ARGUMENT-LIST*.
101\end{verbatim}
102
103All of the command line arguments which follow the occurrence of ``----''
104are passed into a list bound to the EXT:*COMMAND-LINE-ARGUMENT-LIST*
105variable.
106
107\section{Initialization}
108
109If the ABCL process is started without the ``--noinit'' flag, it
110attempts to load a file named ``.abclrc'' located in the user's home
111directory and then interpret its contents. 
112
113The user's home directory is determined by the value of the JVM system
114property ``user.home''.  This value may--or may not--correspond to the
115value of the HOME system environment variable at the discretion of the
116JVM implementation that \textsc{ABCL} finds itself hosted upon.
117
118\chapter{Conformance}
119
120\section{ANSI Common Lisp}
121\textsc{ABCL} is currently a (non)-conforming ANSI Common Lisp
122implementation due to the following known issues:
123
124\begin{itemize}
125  \item The generic function signatures of the DOCUMENTATION symbol do
126    not match the CLHS.
127  \item The TIME form does not return a proper VALUES environment to
128    its caller.
129\end{itemize}
130
131Somewhat confusingly, this statement of non-conformance in the
132accompanying user documentation fullfills the requirements that
133\textsc{ABCL} is a conforming ANSI Common Lisp implementation
134according to the CLHS \footnote{Common Lisp Hyperspec language
135  reference document.}.  Clarifications to this point are solicited.
136
137ABCL aims to be be a fully conforming ANSI Common Lisp implementation.
138Any other behavior should be reported as a bug.
139
140\section{Contemporary Common Lisp}
141In addition to ANSI conformance, \textsc{ABCL} strives to implement features
142expected of a contemporary Common Lisp \footnote{i.e. a Lisp of the
143  post 2005 Renaissance}
144
145\subsection{Deficiencies}
146The following known problems detract from \textsc{ABCL} being a proper
147contemporary Comon Lisp.
148\begin{itemize}
149  \item An incomplete implementation of a properly named metaobject
150    protocol (viz. (A)MOP \footnote{Another Metaobject Protocol} )
151
152    % N.b.
153    % TODO go through AMOP with symbols, starting by looking for
154    % matching function signature.
155    % XXX is this really blocking ANSI conformance?  Answer: we have
156    % to start with such a ``census'' to determine what we have.
157
158  \item Incomplete streams abstraction, in that \textsc{ABCL} needs suitable
159    abstraction between ANSI and Gray streams.  The streams could be
160    optimized to the JVM NIO abstractions at great profit for binary
161    byte-level manipulations.
162   
163  \item Incomplete documentation (missing docstrings from exported
164      symbols.
165
166\end{itemize}
167
168\chapter{Interaction with Hosting JVM}
169
170%  Plan of Attack
171%
172% describe calling Java from Lisp, and calling Lisp from Java,
173% probably in two separate sections.  Presumably, we can partition our
174% audience into those who are more comfortable with Java, and those
175% that are more comforable with Lisp
176
177The Armed Bear Common Lisp implementation is hosted on a Java Virtual
178Machine.  This chapter describes the mechanisms by which the
179implementation interacts with that hosting mechanism.
180
181\section{Lisp to Java}
182
183\textsc{ABCL} offers a number of mechanisms to interact with Java from its
184Lisp environment. It allows calling both instance and static methods
185of Java objects, manipulation of instance and static fields on Java
186objects, and construction of new Java objects.
187
188When calling Java routines, some values will automatically be
189converted by the FFI \footnote{FFI stands for Foreign Function
190  Interface which is the term of art which describes how a Lisp
191  implementation encapsulates invocation in other languages.}  from
192Lisp values to Java values. These conversions typically apply to
193strings, integers and floats. Other values need to be converted to
194their Java equivalents by the programmer before calling the Java
195object method. Java values returned to Lisp are also generally
196converted back to their Lisp counterparts. Some operators make an
197exception to this rule and do not perform any conversion; those are
198the ``raw'' counterparts of certain FFI functions and are recognizable
199by their name ending with \code{-RAW}.
200
201\subsection{Low-level Java API}
202
203We define a higher level Java API in the \ref{topic:Higher level Java
204  API: JSS}(JSS package) which is available in the \code{contrib/} \ref{topic:contrib}
205directory. This package is described later in this document.  This
206section covers the lower level API directly available after evaluating
207\code{(require 'JAVA)}.
208
209\subsubsection{Calling Java Object Methods}
210
211There are two ways to call a Java object method in the low-level (basic) API:
212
213\begin{itemize}
214\item Call a specific method reference (which was previously acquired)
215\item Dynamic dispatch using the method name and
216  the call-specific arguments provided by finding the
217  \ref{section:Parameter matching for FFI dynamic dispatch}{best match}.
218\end{itemize}
219
220The dynamic dispatch variant is discussed in the next section.
221
222\code{JAVA:JMETHOD} is used to acquire a specific method reference.
223The function takes at two or more arguments. The first is Java class designator
224(a \code{JAVA:JAVA-CLASS} object returned by \code{JAVA:JCLASS} or a string naming
225a Java class). The second is a string naming the method.
226
227Any arguments beyond the first two should be strings naming Java classes with
228one exception as listed in the next paragraph. These
229classes specify the types of the arguments for the method to be returned.
230
231There's additional calling convention to the \code{JAVA:JMETHOD} function:
232When the method is called with three parameters and the last parameter is an
233integer, the first method by that name and matching number of parameters is
234returned.
235
236Once one has a reference to the method, one may invoke it using
237\code{JAVA:JCALL}, which takes the method as the first argument. The
238second argument is the object instance to call the method on, or
239\code{NIL} in case of a static method.  Any remaining parameters are
240used as the remaining arguments for the call.
241
242\subsubsection{Calling Java object methods: dynamic dispatch}
243
244The second way of calling Java object methods is by using dynamic dispatch.
245In this case \code{JAVA:JCALL} is used directly without acquiring a method
246reference first. In this case, the first argument provided to \code{JAVA:JCALL}
247is a string naming the method to be called. The second argument is the instance
248on which the method should be called and any further arguments are used to
249select the best matching method and dispatch the call.
250
251\subsubsection{Dynamic dispatch: Caveats}
252
253Dynamic dispatch is performed by using the Java reflection
254API \footnote{The Java reflection API is found in the
255  \code{java.lang.reflect} package}. Generally the dispatch works
256fine, but there are corner cases where the API does not correctly
257reflect all the details involved in calling a Java method. An example
258is the following Java code:
259
260\begin{listing-java}
261ZipFile jar = new ZipFile("/path/to/some.jar");
262Object els = jar.entries();
263Method method = els.getClass().getMethod("hasMoreElements");
264method.invoke(els);
265\end{listing-java}
266
267even though the method \code{hasMoreElements()} is public in \code{Enumeration},
268the above code fails with
269
270\begin{listing-java}
271java.lang.IllegalAccessException: Class ... can
272not access a member of class java.util.zip.ZipFile\$2 with modifiers
273"public"
274       at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
275       at java.lang.reflect.Method.invoke(Method.java:583)
276       at ...
277\end{listing-java}
278
279because the method has been overridden by a non-public class and the
280reflection API, unlike javac, is not able to handle such a case.
281
282While code like that is uncommon in Java, it is typical of ABCL's FFI
283calls. The code above corresponds to the following Lisp code:
284
285\begin{listing-lisp}
286(let ((jar (jnew "java.util.zip.ZipFile" "/path/to/some.jar")))
287  (let ((els (jcall "entries" jar)))
288    (jcall "hasMoreElements" els)))
289\end{listing-lisp}
290
291except that the dynamic dispatch part is not shown.
292
293To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
294field representing the ``intended class'' of the object. That is the class
295that is used first by \code{JAVA:JCALL} and similar to resolve methods;
296the actual class of the object is only tried if the method is not found
297in the intended class. Of course, the intended class is always a super-class
298of the actual class - in the worst case, they coincide. The intended class
299is deduced by the return type of the method that originally returned
300the Java object; in the case above, the intended class of \code{ELS}
301is \code{java.util.Enumeration} because that's the return type of
302the \code{entries} method.
303
304While this strategy is generally effective, there are cases where the
305intended class becomes too broad to be useful. The typical example
306is the extraction of an element from a collection, since methods in
307the collection API erase all types to \code{Object}. The user can
308always force a more specific intended class by using the \code{JAVA:JCOERCE}
309operator.
310
311% \begin{itemize}
312% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
313% \item The Java FFI presents a Lisp package (JAVA) with many useful
314%   symbols for manipulating the artifacts of expectation on the JVM,
315%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
316%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
317%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
318% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
319%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
320%   methods, and additional convenience functions.
321% \item Java classes and libraries may be dynamically added to the
322%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
323% \end{itemize}
324
325\subsubsection{Calling Java class static methods}
326
327Like with non-static methods, references to static methods can be acquired
328by using the \code{JAVA:JMETHOD} primitive. In order to call this method,
329it's not possible to use the \code{JAVA:JCALL} primitive however: there's a
330separate API to retrieve a reference to static methods. This
331primitive is called \code{JAVA:JSTATIC}.
332
333Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
334passing the name of the method as a string instead of passing a method reference.
335The parameter values should be values to pass in the function call instead of
336a specification of classes for each parameter.
337
338\subsubsection{Parameter matching for FFI dynamic dispatch}
339
340The algorithm used to resolve the best matching method given the name
341and the arguments' types is the same as described in the Java Language
342Specification. Any deviation should be reported as a bug.
343
344% ###TODO reference to correct JLS section
345
346\subsubsection{Instantiating Java objects}
347
348Java objects can be instantiated (created) from Lisp by calling
349a constructor from the class of the object to be created. The same way
350\code{JAVA:JMETHOD} is used to acquire a method reference, the
351\code{JAVA:JCONSTRUCTOR} primitive can be used to acquire a constructor
352reference. It's arguments specify the types of arguments of the constructor
353method the same way as with \code{JAVA:JMETHOD}.
354
355The constructor can't be passed to \code{JAVA:JCALL}, but instead should
356be passed as an argument to \code{JAVA:JNEW}.
357
358\subsubsection{Accessing Java object fields}
359
360Fields in Java objects can be accessed using the getter and setter functions
361\code{JAVA:GETFIELD} and \code{JAVA:PUTFIELD}. This applies to values stored in object
362instances. If you want to access static fields: see the next section.
363
364Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
365an intended class around and values which can be converted to Lisp values will
366be converted.
367
368\subsubsection{Accessing Java static fields}
369
370Static fields in Java objects (class fields) can be accessed using the getter
371and setter functions \code{JAVA:GETSTATIC} and \code{JAVA:PUTSTATIC}. Values
372stored in object instance fields can be accessed as described in the previous
373section.
374
375Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
376an intended class around and values which can be converted to Lisp values will
377be converted.
378
379\section{Lisp from Java}
380
381In order to access the Lisp world from Java, one needs to be aware
382of a few things. The most important ones are listed below.
383
384\begin{itemize}
385\item All Lisp values are descendants of LispObject.java
386\item In order to
387\item Lisp symbols are accessible via either directly referencing the
388  Symbol.java instance or by dynamically introspecting the
389  corresponding Package.java instance.
390\item The Lisp dynamic environment may be saved via
391  \code{LispThread.bindSpecial(Binding)} and restored via
392  \code{LispThread.resetSpecialBindings(Mark)}.
393\item Functions may be executed by invocation of the
394  Function.execute(args [...])
395\end{itemize}
396
397\subsection{Lisp FFI}
398
399FFI stands for "Foreign Function Interface" which is the phase which
400the contemporary Lisp world refers to methods of "calling out" from
401Lisp into "foreign" languages and environments.  This document
402describes the various ways that one interacts with Lisp world of ABCL
403from Java, considering the hosted Lisp as the "Foreign Function" that
404needs to be "Interfaced".
405
406\subsection{Calling Lisp from Java}
407
408Note: As the entire ABCL Lisp system resides in the org.armedbear.lisp
409package the following code snippets do not show the relevant import
410statements in the interest of brevity.  An example of the import
411statement would be
412
413\begin{listing-java}
414  import org.armedbear.lisp.*;
415\end{listing-java}
416
417to potentially import all the JVM symbol from the `org.armedbear.lisp'
418namespace.
419
420Per JVM, there can only ever be a single Lisp interpreter.  This is
421started by calling the static method `Interpreter.createInstance()`.
422
423\begin{listing-java}
424  Interpreter interpreter = Interpreter.createInstance();
425\end{listing-java}
426
427If this method has already been invoked in the lifetime of the current
428Java process it will return null, so if you are writing Java whose
429life-cycle is a bit out of your control (like in a Java servlet), a
430safer invocation pattern might be:
431
432\begin{listing-java}
433  Interpreter interpreter = Interpreter.getInstance();
434  if (interpreter == null) {
435    interpreter = Interpreter.createInstance();
436  }
437\end{listing-java}
438
439
440The Lisp \code{eval} primitive may be simply passed strings for evaluation,
441as follows
442
443\begin{listing-java}
444  String line = "(load \"file.lisp\")";
445  LispObject result = interpreter.eval(line);
446\end{listing-java}
447
448Notice that all possible return values from an arbitrary Lisp
449computation are collapsed into a single return value.  Doing useful
450further computation on the ``LispObject'' depends on knowing what the
451result of the computation might be, usually involves some amount
452of \code{instanceof} introspection, and forms a whole topic to itself
453(c.f. [Introspecting a LispObject])
454
455Using \code{eval} involves the Lisp interpreter.  Lisp functions may
456be directly invoked by Java method calls as follows.  One simply
457locates the package containing the symbol, then obtains a reference to
458the symbol, and then invokes the \code{execute()} method with the
459desired parameters.
460
461\begin{listing-java}
462    interpreter.eval("(defun foo (msg) (format nil \"You told me '~A'~%\" msg))");
463    Package pkg = Packages.findPackage("CL-USER");
464    Symbol foo = pkg.findAccessibleSymbol("FOO");
465    Function fooFunction = (Function)foo.getSymbolFunction();
466    JavaObject parameter = new JavaObject("Lisp is fun!");
467    LispObject result = fooFunction.execute(parameter);
468    // How to get the "naked string value"?
469    System.out.println("The result was " + result.writeToString());
470\end{listing-java}
471
472If one is calling an primitive function in the CL package the syntax
473becomes considerably simpler.  If we can locate the instance of
474definition in the ABCL Java source, we can invoke the symbol directly.
475For instance, to tell if a `LispObject` contains a reference to a symbol.
476
477\begin{listing-java}
478    boolean nullp(LispObject object) {
479      LispObject result = Primitives.NULL.execute(object);
480      if (result == NIL) { // the symbol 'NIL' is explicitly named in the Java
481                           // namespace at ``Symbol.NIL''
482                           // but is always present in the
483                           // local namespace in its unadorned form for
484                           // the convenience of the User.
485        return false;
486      }
487      return true;
488   }
489\end{listing-java}
490
491\subsubsection{Introspecting a LispObject}
492\label{topic:Introspecting a LispObject}
493
494We present various patterns for introspecting an an arbitrary
495`LispObject` which can represent the result of every Lisp evaluation
496into semantics that Java can meaningfully deal with.
497
498\subsubsection{LispObject as \code{boolean}}
499
500If the LispObject a generalized boolean values, one can use
501\code{getBooleanValue()} to convert to Java:
502
503\begin{listing-java}
504     LispObject object = Symbol.NIL;
505     boolean javaValue = object.getBooleanValue();
506\end{listing-java}
507
508Although since in Lisp, any value other than NIL means "true"
509(so-called generalized Boolean), the use of Java equality it quite a
510bit easier to type and more optimal in terms of information it conveys
511to the compiler would be:
512
513\begin{listing-java}
514    boolean javaValue = (object != Symbol.NIL);
515\end{listing-java}
516
517\paragraph{LispObject is a list}
518
519If LispObject is a list, it will have the type `Cons`.  One can then use
520the \code{copyToArray} to make things a bit more suitable for Java
521iteration.
522
523\begin{listing-java}
524    LispObject result = interpreter.eval("'(1 2 4 5)");
525    if (result instanceof Cons) {
526      LispObject array[] = ((Cons)result.copyToArray());
527      ...
528    }
529\end{listing-java}
530
531A more Lispy way to iterated down a list is to use the `cdr()` access
532function just as like one would traverse a list in Lisp:;
533
534\begin{listing-java}
535    LispObject result = interpreter.eval("'(1 2 4 5)");
536    while (result != Symbol.NIL) {
537      doSomething(result.car());
538      result = result.cdr();
539    }
540\end{listing-java}
541
542\subsection{Java Scripting API (JSR-223)}
543
544ABCL can be built with support for JSR-223, which offers a
545language-agnostic API to invoke other languages from Java. The binary
546distribution downloadable from ABCL's common-lisp.net home is built
547with JSR-223 support. If you're building ABCL from source on a pre-1.6
548JVM, you need to have a JSR-223 implementation in your CLASSPATH (such
549as Apache Commons BSF 3.x or greater) in order to build ABCL with
550JSR-223 support; otherwise, this feature will not be built.
551
552This section describes the design decisions behind the ABCL JSR-223
553support. It is not a description of what JSR-223 is or a tutorial on
554how to use it. See
555http://trac.common-lisp.net/armedbear/browser/trunk/abcl/examples/jsr-223
556for example usage.
557
558\subsubsection{Conversions}
559
560In general, ABCL's implementation of the JSR-223 API performs implicit
561conversion from Java objects to Lisp objects when invoking Lisp from
562Java, and the opposite when returning values from Java to Lisp. This
563potentially reduces coupling between user code and ABCL. To avoid such
564conversions, wrap the relevant objects in \code{JavaObject} instances.
565
566\subsubsection{Implemented JSR-223 interfaces}
567
568JSR-223 defines three main interfaces, of which two (Invocable and
569Compilable) are optional. ABCL implements all the three interfaces -
570ScriptEngine and the two optional ones - almost completely. While the
571JSR-223 API is not specific to a single scripting language, it was
572designed with languages with a more or less Java-like object model in
573mind: languages such as Javascript, Python, Ruby, which have a concept
574of "class" or "object" with "fields" and "methods". Lisp is a bit
575different, so certain adaptations were made, and in one case a method
576has been left unimplemented since it does not map at all to Lisp.
577
578\subsubsection{The ScriptEngine}
579
580The main interface defined by JSR-223, javax.script.ScriptEngine, is
581implemented by the class
582\code{org.armedbear.lisp.scripting.AbclScriptEngine}. AbclScriptEngine
583is a singleton, reflecting the fact that ABCL is a singleton as
584well. You can obtain an instance of AbclScriptEngine using the
585AbclScriptEngineFactory or by using the service provider mechanism
586through ScriptEngineManager (refer to the javax.script documentation).
587
588\subsubsection{Startup and configuration file}
589
590At startup (i.e. when its constructor is invoked, as part of the
591static initialization phase of AbclScriptEngineFactory) the ABCL
592script engine attempts to load an "init file" from the classpath
593(/abcl-script-config.lisp). If present, this file can be used to
594customize the behaviour of the engine, by setting a number of
595variables in the ABCL-SCRIPT package. Here is a list of the available
596variables:
597
598\begin{itemize}
599\item *use-throwing-debugger* Controls whether ABCL uses a
600  non-standard debugging hook function to throw a Java exception
601  instead of dropping into the debugger in case of unhandled error
602  conditions.
603  \begin{itemize}
604  \item Default value: T
605  \item Rationale: it is more convenient for Java programmers using
606    Lisp as a scripting language to have it return exceptions to Java
607    instead of handling them in the Lisp world.
608  \item Known Issues: the non-standard debugger hook has been reported
609    to misbehave in certain circumstances, so consider disabling it if
610    it doesn't work for you.
611  \end{itemize}
612\item *launch-swank-at-startup* If true, Swank will be launched at
613  startup. See *swank-dir* and *swank-port*.
614  \begin{itemize}
615  \item Default value: NIL
616  \end{itemize}
617\item *swank-dir* The directory where Swank is installed. Must be set
618  if *launch-swank-at-startup* is true.
619\item *swank-port* The port where Swank will listen for
620  connections. Must be set if *launch-swank-at-startup* is true.
621  \begin{itemize}
622  \item Default value: 4005
623  \end{itemize}
624\end{itemize}
625
626Additionally, at startup the AbclScriptEngine will \code{(require
627  'asdf)} - in fact, it uses asdf to load Swank.
628
629\subsubsection{Evaluation}
630
631Code is read and evaluated in the package ABCL-SCRIPT-USER. This
632packages USEs the COMMON-LISP, JAVA and ABCL-SCRIPT packages. Future
633versions of the script engine might make this default package
634configurable. The \code{CL:LOAD} function is used under the hood for
635evaluating code, and thus the same behavior of LOAD is
636guaranteed. This allows, among other things, \code{IN-PACKAGE} forms
637to change the package in which the loaded code is read.
638
639It is possible to evaluate code in what JSR-223 calls a
640"ScriptContext" (basically a flat environment of name->value
641pairs). This context is used to establish special bindings for all the
642variables defined in it; since variable names are strings from Java's
643point of view, they are first interned using READ-FROM-STRING with, as
644usual, ABCL-SCRIPT-USER as the default package. Variables are declared
645special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
646functions work in a null lexical environment and would ignore
647non-special bindings.
648
649Contrary to what the function \code{LOAD} does, evaluation of a series
650of forms returns the value of the last form instead of T, so the
651evaluation of short scripts does the Right Thing.
652
653\subsubsection{Compilation}
654
655AbclScriptEngine implements the \code{javax.script.Compilable}
656interface. Currently it only supports compilation using temporary
657files. Compiled code, returned as an instance of
658javax.script.CompiledScript, is read, compiled and executed by default
659in the ABCL-SCRIPT-USER package, just like evaluated code. Differently
660from evaluated code, though, due to the way the ABCL compiler works,
661compiled code contains no reference to top-level self-evaluating
662objects (like numbers or strings). Thus, when evaluated, a piece of
663compiled code will return the value of the last non-self-evaluating
664form: for example the code "(do-something) 42" will return 42 when
665interpreted, but will return the result of (do-something) when
666compiled and later evaluated. To ensure consistency of behavior
667between interpreted and compiled code, make sure the last form is
668always a compound form - at least (identity some-literal-object). Note
669that this issue should not matter in real code, where it is unlikely a
670top-level self-evaluating form will appear as the last form in a file
671(in fact, the Common Lisp load function always returns T upon success;
672with JSR-223 this policy has been changed to make evaluation of small
673code snippets work as intended).
674
675\subsubsection{Invocation of functions and methods}
676
677AbclScriptEngine implements the \code{javax.script.Invocable}
678interface, which allows to directly call Lisp functions and methods,
679and to obtain Lisp implementations of Java interfaces. This is only
680partially possible with Lisp since it has functions, but not methods -
681not in the traditional OO sense, at least, since Lisp methods are not
682attached to objects but belong to generic functions. Thus, the method
683\code{invokeMethod()} is not implemented and throws an
684UnsupportedOperationException when called. The \code{invokeFunction()}
685method should be used to call both regular and generic functions.
686
687\subsubsection{Implementation of Java interfaces in Lisp}
688
689ABCL can use the Java reflection-based proxy feature to implement Java
690interfaces in Lisp. It has several built-in ways to implement an
691interface, and supports definition of new ones. The
692\code{JAVA:JMAKE-PROXY} generic function is used to make such
693proxies. It has the following signature:
694
695\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
696
697\code{interface} is a Java interface metaobject (e.g. obtained by
698invoking \code{jclass}) or a string naming a Java
699interface. \code{implementation} is the object used to implement the
700interface - several built-in methods of jmake-proxy exist for various
701types of implementations. \code{lisp-this} is an object passed to the
702closures implementing the Lisp "methods" of the interface, and
703defaults to \code{NIL}.
704
705The returned proxy is an instance of the interface, with methods
706implemented with Lisp functions.
707
708Built-in interface-implementation types include:
709
710\begin{itemize}
711\item a single Lisp function which upon invocation of any method in
712  the interface will be passed the method name, the Lisp-this object,
713  and all the parameters. Useful for interfaces with a single method,
714  or to implement custom interface-implementation strategies.
715\item a hash-map of method-name -> Lisp function mappings. Function
716  signature is \code{(lisp-this \&rest args)}.
717\item a Lisp package. The name of the Java method to invoke is first
718  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
719  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
720  the package. If it exists and is fbound, the corresponding function
721  will be called. Function signature is as the hash-table case.
722\end{itemize}
723
724This functionality is exposed by the AbclScriptEngine with the two
725methods getInterface(Class) and getInterface(Object, Class). The
726former returns an interface implemented with the current Lisp package,
727the latter allows the programmer to pass an interface-implementation
728object which will in turn be passed to the jmake-proxy generic
729function.
730
731\chapter{Implementation Dependent Extensions}
732
733As outlined by the CLHS ANSI conformance guidelines, we document the
734extensions to the Armed Bear Lisp implementation made accessible to
735the user by virtue of being an exported symbol in the JAVA, THREADS,
736or EXTENSIONS packages.
737
738\section{JAVA}
739
740\subsection{Modifying the JVM CLASSPATH}
741
742The JAVA:ADD-TO-CLASSPATH generic functions allows one to add the
743specified pathname or list of pathnames to the current classpath
744used by ABCL, allowing the dynamic loading of JVM objects:
745
746\begin{listing-lisp}
747CL-USER> (add-to-classpath "/path/to/some.jar")
748\end{listing-lisp}
749
750N.b \code{add-to-classpath} only affects the classloader used by ABCL
751(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
752no effect on Java code outside ABCL.
753
754\subsection{API}
755
756% include autogen docs for the JAVA package.
757\include{java}
758
759\section{THREADS}
760
761The extensions for handling multithreaded execution are collected in
762the \code{THREADS} package.  Most of the abstractions in Doug Lea's
763excellent \code{java.util.concurrent} packages may be manipulated
764directly via the JSS contrib to great effect.
765
766\subsection{API}
767
768% include autogen docs for the THREADS package.
769\include{threads}
770
771\section{EXTENSIONS}
772
773The symbols in the EXTENSIONS package (nicknamed ``EXT'') constitutes
774extensions to the ANSI standard that are potentially useful to the
775user.  They include functions for manipulating network sockets,
776running external programs, registering object finalizers, constructing
777reference weakly held by the garbage collector and others.
778
779See \ref{Extensible Sequences} for a generic function interface to
780the native JVM contract for \code{java.util.List}.
781
782\subsection{API}
783
784% include autogen docs for the EXTENSIONS package.
785\include{extensions}
786
787\chapter{Beyond ANSI}
788
789Naturally, in striving to be a useful contemporary Common Lisp
790implementation, ABCL endeavors to include extensions beyond the ANSI
791specification which are either widely adopted or are especially useful
792in working with the hosting JVM.
793
794\section{Implementation Dependent}
795\begin{enumerate}
796  \item Compiler to JVM 5 bytecode
797  \item Pathname extensions
798\end{enumerate}
799
800\section{Pathname}
801
802We implement an extension to the Pathname that allows for the
803description and retrieval of resources named in a URI scheme that the
804JVM ``understands''.  Support is built-in to the ``http'' and
805``https'' implementations but additional protocol handlers may be
806installed at runtime by having JVM symbols present in the
807sun.net.protocol.dynmamic pacakge. See [JAVA2006] for more details.
808
809ABCL has created specializations of the ANSI Pathname object to
810enable to use of URIs to address dynamically loaded resources for the
811JVM.  A URL-PATHNAME has a corresponding URL whose cannoical
812representation is defined to be the NAMESTRING of the Pathname.
813
814\begin{verbatim}
815  JAR-PATHNAME isa URL-PATHNAME isa PATHNAME
816\end{verbatim}
817
818Both URL-PATHNAME and JAR-PATHNAME may be used anywhere a PATHNAME is
819accepted with the following caveats:
820
821\begin{itemize}
822
823\item A stream obtained via OPEN on a URL-PATHNAME cannot be the
824  target of write operations.
825
826\item No canonicalization is performed on the underlying URI (i.e. the
827implementation does not attempt to compute the current name of the
828representing resource unless it is requested to be resolved.)  Upon
829resolution, any cannoicalization procedures followed in resolving the
830resource (e.g. following redirects) are discarded. 
831
832\end{itemize}
833
834The implementation of URL-PATHNAME allows the ABCL user to laod dynamically
835code from the network.  For example, for Quicklisp.
836
837\begin{listing-lisp}
838  CL-USER> (load "http://beta.quicklisp.org/quicklisp.lisp")
839\end{listing-lisp}
840
841will load and execute the Quicklisp setup code.
842
843\ref{XACH2011}
844
845\subsubsection{Implementation}
846
847\textsc{DEVICE} either a string denoting a drive letter under DOS or a cons
848specifying a \textsc{URL-PATHNAME}.
849         
850\section{Extensible Sequences}
851
852See \ref{RHODES2007} RHODES2007 for the design.
853
854The SEQUENCE package fully implements Christopher Rhodes' proposal for
855extensible sequences.  These user extensible sequences are used
856directly in \code{java-collections.lisp} provide these CLOS
857abstractions on the standard Java collection classes as defined by the
858\code{java.util.List} contract.
859
860This extension is not automatically loaded by the implementation.   It
861may be loaded via:
862
863\begin{listing-lisp}
864CL-USER> (require 'java-collections)
865\end{listing-lisp}
866
867if both extensible sequences and their application to Java collections
868is required, or
869
870\begin{listing-lisp}
871CL-USER> (require 'extensible-sequences)
872\end{listing-lisp}
873
874if only the extensible sequences API as specified in \ref{RHODES2007} is
875required.
876
877Note that \code{(require 'java-collections)} must be issued before
878\code{java.util.List} or any subclass is used as a specializer in a CLOS
879method definition (see the section below).
880
881\section{Extensions to CLOS}
882
883There is an additional syntax for specializing the parameter of a
884generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
885where \code{CLASS-STRING} is a string naming a Java class in dotted package
886form.
887
888For instance the following specialization would perhaps allow one to
889print more information about the contents of a java.util.Collection
890object
891
892\begin{listing-lisp}
893(defmethod print-object ((coll (java:jclass "java.util.Collection"))
894                         stream)
895  ;;; ...
896)
897\end{listing-lisp}
898
899If the class had been loaded via a classloader other than the original
900the class you wish to specialize on, one needs to specify the
901classloader as an optional third argument.
902
903\begin{listing-lisp}
904
905(defparameter *other-classloader*
906  (jcall "getBaseLoader" cl-user::*classpath-manager*))
907 
908(defmethod print-object ((device-id (java:jclass "dto.nbi.service.hdm.alcatel.com.NBIDeviceID" *other-classloader*))
909                         stream)
910  ;;; ...
911)
912\end{listing-lisp}
913
914\section{Extensions to the Reader}
915
916We implement a special hexadecimal escape sequence for specifying
917characters to the Lisp reader, namely we allow a sequences of the form
918\# \textbackslash Uxxxx to be processed by the reader as character
919whose code is specified by the hexadecimal digits ``xxxx''.  The
920hexadecimal sequence must be exactly four digits long \footnote{This
921  represents a compromise with contemporary in 2011 32bit hosting
922  architecures for which we wish to make text processing efficient.
923  Should the User require more control over UNICODE processing we
924  recommend Edi Weisz' excellent work with FLEXI-STREAMS which we
925  fully support}, padded by leading zeros for values less than 0x1000.
926
927Note that this sequence is never output by the implementation.  Instead,
928the corresponding Unicode character is output for characters whose
929code is greater than 0x00ff.
930
931\subsection{JSS optionally extends the Reader}
932
933The JSS contrib consitutes an additional, optional extension to the
934reader in the definition of the \#\" reader macro.
935
936\section{ASDF}
937
938asdf-2.017.22 is packaged as core component of ABCL, but not
939intialized by default, as it relies on the CLOS subsystem which can
940take a bit of time to initialize.  It may be initialized by the ANSI
941\textsc{REQUIRE} mechanism as follows:
942
943\begin{listing-lisp}
944CL-USER> (require 'asdf)
945\end{listing-lisp}
946
947\chapter{Contrib}
948
949\section{abcl-asdf}
950
951This contrib to ABCL enables an additional syntax for ASDF system
952definition which dynamically loads JVM artifacts such as jar archives
953via a Maven encapsulation.  The Maven Aether can also be directly
954manipulated by the function associated with the RESOLVE-DEPENDENCIES symbol.
955
956%ABCL specific contributions to ASDF system definition mainly concerned
957%with finding JVM artifacts such as jar archives to be dynamically loaded.
958
959
960The following ASDF components are added: \textsc{JAR-FILE}, \textsc{JAR-DIRECTORY},
961\textsc{CLASS-FILE-DIRECTORY} and \textsc{MVN}.
962
963
964
965\subsection{ABCL-ASDF Examples}
966
967\begin{listing-lisp}
968    ;;;; -*- Mode: LISP -*-
969    (in-package :asdf)
970
971    (defsystem :log4j
972      :components ((:mvn "log4j/log4j"
973                    :version "1.4.9")))
974\end{listing-lisp}
975
976\subsection{abcl-asdf API}
977
978We define an API as consisting of the following ASDF classes:
979
980\textsc{JAR-DIRECTORY}, \textsc{JAR-FILE}, and
981\textsc{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
982valid pathname representation
983
984And the MVN and IRI classes descend from ASDF-COMPONENT, but do not
985directly have a filesystem location.
986
987For use outside of ASDF, we currently define one method,
988\textsc{RESOLVE-DEPENDENCIES} which locates, downloads, caches, and then loads
989into the currently executing JVM process all recursive dependencies
990annotated in the Maven pom.xml graph.
991
992\subsection{ABCL-ASDF Example 2}
993
994Bypassing ASDF, one can directly issue requests for the Maven
995artifacts to be downloaded
996
997\begin{listing-lisp}
998    CL-USER> (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user")
999    WARNING: Using LATEST for unspecified version.
1000    "/Users/evenson/.m2/repository/com/google/gwt/gwt-user/2.4.0-rc1/gwt-user-2.4.0-rc1.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA-sources.jar"
1001\end{listing-lisp}
1002
1003To actually load the dependency, use the JAVA:ADD-TO-CLASSPATH generic
1004function:
1005
1006\begin{listing-lisp}
1007    CL-USER> (java:add-to-classpath (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user"))
1008\end{listing-lisp}
1009
1010Notice that all recursive dependencies have been located and installed
1011locally from the network as well.
1012
1013
1014\section{asdf-jar}
1015
1016ASDF-JAR provides a system for packaging ASDF systems into jar
1017archives for ABCL.  Given a running ABCL image with loadable ASDF
1018systems the code in this package will recursively package all the
1019required source and fasls in a jar archive.
1020
1021\section{jss}
1022
1023To one used to a syntax that can construct macros, the Java syntax
1024sucks, so we introduce the \#" macro.
1025
1026\subsection{JSS usage}
1027
1028Example:
1029
1030\begin{listing-lisp}
1031  CL-USER> (require 'jss)
1032
1033  CL-USER) (#"getProperties" 'java.lang.System)
1034
1035  CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
1036\end{listing-lisp}
1037
1038\section{asdf-install}
1039
1040An implementation of ASDF-INSTALL.  Superseded by Quicklisp (qv.)
1041
1042\chapter{History}
1043
1044ABCL was originally the extension language for the J editor, which was
1045started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1046code that had previously not been released publically was suddenly
1047committed that enabled ABCL to be plausibly termed an emergent ANSI
1048Common Lisp implementation canidate.
1049
1050From 2006 to 2008, Peter manned the development lists, incorporating
1051patches as made sense.  After a suitable search, Peter nominated Erik
1052Huelsmann to take over the project.
1053
1054In 2008, the implementation was transferred to the current
1055maintainers, who have strived to improve its usability as a
1056contemporary Common Lisp implementation.
1057
1058On October 22, 2011, with the publication of this Manual explicitly
1059stating the conformance of Armed Bear Common Lisp to ANSI, we released
1060abcl-1.0.0.
1061
1062
1063
1064
1065\section{References}
1066
1067[Java2000]:  A New Era for Java Protocol Handlers.
1068\url{http://java.sun.com/developer/onlineTraining/protocolhandlers/}
1069
1070[Xach2011]:  Quicklisp:  A system for quickly constructing Common Lisp
1071libraries.  \url{http://www.quicklisp.org/}
1072
1073[RHODES2007]:  Christopher Rhodes
1074
1075
1076\end{document}
1077
1078% TODO
1079%   1.  Create mechanism for swigging DocString and Lisp docs into
1080%       sections ('grovel.lisp')
1081
Note: See TracBrowser for help on using the repository browser.