source: trunk/abcl/doc/manual/abcl.tex @ 13679

Last change on this file since 13679 was 13679, checked in by Mark Evenson, 10 years ago

Minor manual revisions (in progress commit).

File size: 41.0 KB
Line 
1% -*- mode: latex; -*-
2% http://en.wikibooks.org/wiki/LaTeX/
3\documentclass[10pt]{book}
4\usepackage{abcl}
5
6\begin{document}
7\title{A Manual for Armed Bear Common Lisp}
8\date{October 27, 2011}
9\author{Mark~Evenson, Erik~Huelsmann, Alessio~Stalla, Ville~Voutilainen}
10
11\maketitle
12
13\tableofcontents
14
15\chapter{Introduction}
16
17Armed Bear is a (mostly) conforming implementation of the ANSI Common
18Lisp standard.  This manual documents the Armed Bear Common Lisp
19implementation for users of the system.
20
21\subsection{Version}
22This manual corresponds to abcl-1.0.0,  released on October 22, 2011.
23
24\subsection{License}
25
26The implementation is licensed under the terms of the GPL v2 of June
271991 with the ``classpath-exception'' that makes its deployment in
28commercial settings quite reasonable.  The license is viral in the
29sense that if you change the implementation, and redistribute those
30changes, you are required to provide the source to those changes back
31to be merged with the public trunk.
32
33\subsection{Contributors}
34
35% TODO format this better, optionally link to URI
36
37% Thanks for the markup
38Philipp Marek
39
40% Thanks for the whacky IKVM stuff and keeping the flame alive
41Douglas Miles
42
43% Thanks for JSS
44Alan Ruttenberg
45
46and of course
47
48Peter Graves
49
50\chapter{Running}
51
52\textsc{ABCL} is packaged as a single jar file usually named either
53``abcl.jar'' or possibly``abcl-1.0.0.jar'' if one is using a versioned
54package from your system vendor.  This byte archive can be executed
55under the control of a suitable JVM \footnote {Java Virtual Machine}
56by using the ``-jar'' option to parse the manifest, and select the
57class named therein ``\code{org.armedbear.lisp.Main}'' for execution,
58viz:
59
60\begin{listing-shell}
61  cmd$ java -jar abcl.jar
62\end{listing-shell}
63
64\emph{N.b.} for the proceeding command to work, the ``java'' executable needs
65to be in your path.
66
67To make it easier to facilitate the use of ABCL in tool chains (such
68as SLIME \footnote{SLIME is the Superior Lisp Mode for Interaction
69  under Emacs}) the invocation is wrapped in a Bourne shell script
70under \textsc{UNIX} or a \textsc{DOS} command script under Windows so that ABCL may be
71executed simply as:
72
73\begin{listing-shell}
74  cmd$ abcl
75\end{listing-shell}
76
77\section{Options}
78
79ABCL supports the following command line options:
80
81\begin{verbatim}
82--help
83    Displays this message.
84--noinform
85    Suppresses the printing of startup information and banner.
86--noinit
87    Suppresses the loading of the '~/.abclrc' startup file.
88--nosystem
89    Suppresses loading the 'system.lisp' customization file.
90--eval <FORM>
91    Evaluates the <FORM> before initializing REPL.
92--load <FILE>
93    Loads the file <FILE> before initializing REPL.
94--load-system-file <FILE>
95    Loads the system file <FILE> before initializing REPL.
96--batch
97    The process evaluates forms specified by arguments and possibly by those
98    by those in the intialization file '~/.abcl', and then exits.
99
100The occurance of '--' copies the remaining arguments, unprocessed, into
101the variable EXTENSIONS:*COMMAND-LINE-ARGUMENT-LIST*.
102\end{verbatim}
103
104All of the command line arguments which follow the occurrence of ``----''
105are passed into a list bound to the EXT:*COMMAND-LINE-ARGUMENT-LIST*
106variable.
107
108\section{Initialization}
109
110If the \textsc{ABCL} process is started without the ``--noinit'' flag, it
111attempts to load a file named ``.abclrc'' located in the user's home
112directory and then interpret its contents. 
113
114The user's home directory is determined by the value of the JVM system
115property ``user.home''.  This value may--or may not--correspond to the
116value of the HOME system environment variable at the discretion of the
117JVM implementation that \textsc{ABCL} finds itself hosted upon.
118
119\chapter{Conformance}
120
121\section{ANSI Common Lisp}
122\textsc{ABCL} is currently a (non)-conforming ANSI Common Lisp
123implementation due to the following known issues:
124
125\begin{itemize}
126  \item The generic function signatures of the DOCUMENTATION symbol do
127    not match the CLHS.
128  \item The TIME form does not return a proper VALUES environment to
129    its caller.
130\end{itemize}
131
132Somewhat confusingly, this statement of non-conformance in the
133accompanying user documentation fullfills the requirements that
134\textsc{ABCL} is a conforming ANSI Common Lisp implementation
135according to the CLHS \footnote{Common Lisp Hyperspec language
136  reference document.}.  Clarifications to this point are solicited.
137
138ABCL aims to be be a fully conforming ANSI Common Lisp implementation.
139Any other behavior should be reported as a bug.
140
141\section{Contemporary Common Lisp}
142In addition to ANSI conformance, \textsc{ABCL} strives to implement features
143expected of a contemporary Common Lisp \footnote{i.e. a Lisp of the
144  post 2005 Renaissance}
145
146\subsection{Deficiencies}
147The following known problems detract from \textsc{ABCL} being a proper
148contemporary Comon Lisp.
149\begin{itemize}
150  \item An incomplete implementation of a properly named metaobject
151    protocol (viz. (A)MOP \footnote{Another Metaobject Protocol} )
152
153    % N.b.
154    % TODO go through AMOP with symbols, starting by looking for
155    % matching function signature.
156    % XXX is this really blocking ANSI conformance?  Answer: we have
157    % to start with such a ``census'' to determine what we have.
158
159  \item Incomplete streams abstraction, in that \textsc{ABCL} needs suitable
160    abstraction between ANSI and Gray streams.  The streams could be
161    optimized to the JVM NIO abstractions at great profit for binary
162    byte-level manipulations.
163   
164  \item Incomplete documentation (missing docstrings from exported
165      symbols.
166
167\end{itemize}
168
169\chapter{Interaction with Hosting JVM}
170
171%  Plan of Attack
172%
173% describe calling Java from Lisp, and calling Lisp from Java,
174% probably in two separate sections.  Presumably, we can partition our
175% audience into those who are more comfortable with Java, and those
176% that are more comforable with Lisp
177
178The Armed Bear Common Lisp implementation is hosted on a Java Virtual
179Machine.  This chapter describes the mechanisms by which the
180implementation interacts with that hosting mechanism.
181
182\section{Lisp to Java}
183
184\textsc{ABCL} offers a number of mechanisms to interact with Java from its
185Lisp environment. It allows calling both instance and static methods
186of Java objects, manipulation of instance and static fields on Java
187objects, and construction of new Java objects.
188
189When calling Java routines, some values will automatically be
190converted by the FFI \footnote{FFI stands for Foreign Function
191  Interface which is the term of art which describes how a Lisp
192  implementation encapsulates invocation in other languages.}  from
193Lisp values to Java values. These conversions typically apply to
194strings, integers and floats. Other values need to be converted to
195their Java equivalents by the programmer before calling the Java
196object method. Java values returned to Lisp are also generally
197converted back to their Lisp counterparts. Some operators make an
198exception to this rule and do not perform any conversion; those are
199the ``raw'' counterparts of certain FFI functions and are recognizable
200by their name ending with \code{-RAW}.
201
202\subsection{Low-level Java API}
203
204We define a higher level Java API in the \ref{topic:Higher level Java
205  API: JSS}(JSS package) which is available in the \code{contrib/} \ref{topic:contrib}
206directory. This package is described later in this document.  This
207section covers the lower level API directly available after evaluating
208\code{(require 'JAVA)}.
209
210\subsubsection{Calling Java Object Methods}
211
212There are two ways to call a Java object method in the low-level (basic) API:
213
214\begin{itemize}
215\item Call a specific method reference (which was previously acquired)
216\item Dynamic dispatch using the method name and
217  the call-specific arguments provided by finding the
218  \ref{section:Parameter matching for FFI dynamic dispatch}{best match}.
219\end{itemize}
220
221The dynamic dispatch variant is discussed in the next section.
222
223\code{JAVA:JMETHOD} is used to acquire a specific method reference.
224The function takes at two or more arguments. The first is Java class designator
225(a \code{JAVA:JAVA-CLASS} object returned by \code{JAVA:JCLASS} or a string naming
226a Java class). The second is a string naming the method.
227
228Any arguments beyond the first two should be strings naming Java classes with
229one exception as listed in the next paragraph. These
230classes specify the types of the arguments for the method to be returned.
231
232There's additional calling convention to the \code{JAVA:JMETHOD} function:
233When the method is called with three parameters and the last parameter is an
234integer, the first method by that name and matching number of parameters is
235returned.
236
237Once one has a reference to the method, one may invoke it using
238\code{JAVA:JCALL}, which takes the method as the first argument. The
239second argument is the object instance to call the method on, or
240\code{NIL} in case of a static method.  Any remaining parameters are
241used as the remaining arguments for the call.
242
243\subsubsection{Calling Java object methods: dynamic dispatch}
244
245The second way of calling Java object methods is by using dynamic dispatch.
246In this case \code{JAVA:JCALL} is used directly without acquiring a method
247reference first. In this case, the first argument provided to \code{JAVA:JCALL}
248is a string naming the method to be called. The second argument is the instance
249on which the method should be called and any further arguments are used to
250select the best matching method and dispatch the call.
251
252\subsubsection{Dynamic dispatch: Caveats}
253
254Dynamic dispatch is performed by using the Java reflection
255API \footnote{The Java reflection API is found in the
256  \code{java.lang.reflect} package}. Generally the dispatch works
257fine, but there are corner cases where the API does not correctly
258reflect all the details involved in calling a Java method. An example
259is the following Java code:
260
261\begin{listing-java}
262ZipFile jar = new ZipFile("/path/to/some.jar");
263Object els = jar.entries();
264Method method = els.getClass().getMethod("hasMoreElements");
265method.invoke(els);
266\end{listing-java}
267
268even though the method \code{hasMoreElements()} is public in \code{Enumeration},
269the above code fails with
270
271\begin{listing-java}
272java.lang.IllegalAccessException: Class ... can
273not access a member of class java.util.zip.ZipFile\$2 with modifiers
274"public"
275       at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
276       at java.lang.reflect.Method.invoke(Method.java:583)
277       at ...
278\end{listing-java}
279
280because the method has been overridden by a non-public class and the
281reflection API, unlike javac, is not able to handle such a case.
282
283While code like that is uncommon in Java, it is typical of ABCL's FFI
284calls. The code above corresponds to the following Lisp code:
285
286\begin{listing-lisp}
287(let ((jar (jnew "java.util.zip.ZipFile" "/path/to/some.jar")))
288  (let ((els (jcall "entries" jar)))
289    (jcall "hasMoreElements" els)))
290\end{listing-lisp}
291
292except that the dynamic dispatch part is not shown.
293
294To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
295field representing the ``intended class'' of the object. That is the class
296that is used first by \code{JAVA:JCALL} and similar to resolve methods;
297the actual class of the object is only tried if the method is not found
298in the intended class. Of course, the intended class is always a super-class
299of the actual class - in the worst case, they coincide. The intended class
300is deduced by the return type of the method that originally returned
301the Java object; in the case above, the intended class of \code{ELS}
302is \code{java.util.Enumeration} because that's the return type of
303the \code{entries} method.
304
305While this strategy is generally effective, there are cases where the
306intended class becomes too broad to be useful. The typical example
307is the extraction of an element from a collection, since methods in
308the collection API erase all types to \code{Object}. The user can
309always force a more specific intended class by using the \code{JAVA:JCOERCE}
310operator.
311
312% \begin{itemize}
313% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
314% \item The Java FFI presents a Lisp package (JAVA) with many useful
315%   symbols for manipulating the artifacts of expectation on the JVM,
316%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
317%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
318%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
319% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
320%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
321%   methods, and additional convenience functions.
322% \item Java classes and libraries may be dynamically added to the
323%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
324% \end{itemize}
325
326\subsubsection{Calling Java class static methods}
327
328Like with non-static methods, references to static methods can be acquired
329by using the \code{JAVA:JMETHOD} primitive. In order to call this method,
330it's not possible to use the \code{JAVA:JCALL} primitive however: there's a
331separate API to retrieve a reference to static methods. This
332primitive is called \code{JAVA:JSTATIC}.
333
334Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
335passing the name of the method as a string instead of passing a method reference.
336The parameter values should be values to pass in the function call instead of
337a specification of classes for each parameter.
338
339\subsubsection{Parameter matching for FFI dynamic dispatch}
340
341The algorithm used to resolve the best matching method given the name
342and the arguments' types is the same as described in the Java Language
343Specification. Any deviation should be reported as a bug.
344
345% ###TODO reference to correct JLS section
346
347\subsubsection{Instantiating Java objects}
348
349Java objects can be instantiated (created) from Lisp by calling
350a constructor from the class of the object to be created. The same way
351\code{JAVA:JMETHOD} is used to acquire a method reference, the
352\code{JAVA:JCONSTRUCTOR} primitive can be used to acquire a constructor
353reference. It's arguments specify the types of arguments of the constructor
354method the same way as with \code{JAVA:JMETHOD}.
355
356The constructor can't be passed to \code{JAVA:JCALL}, but instead should
357be passed as an argument to \code{JAVA:JNEW}.
358
359\subsubsection{Accessing Java object fields}
360
361Fields in Java objects can be accessed using the getter and setter functions
362\code{JAVA:GETFIELD} and \code{JAVA:PUTFIELD}. This applies to values stored in object
363instances. If you want to access static fields: see the next section.
364
365Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
366an intended class around and values which can be converted to Lisp values will
367be converted.
368
369\subsubsection{Accessing Java static fields}
370
371Static fields in Java objects (class fields) can be accessed using the getter
372and setter functions \code{JAVA:GETSTATIC} and \code{JAVA:PUTSTATIC}. Values
373stored in object instance fields can be accessed as described in the previous
374section.
375
376Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
377an intended class around and values which can be converted to Lisp values will
378be converted.
379
380\section{Lisp from Java}
381
382In order to access the Lisp world from Java, one needs to be aware
383of a few things. The most important ones are listed below.
384
385\begin{itemize}
386\item All Lisp values are descendants of LispObject.java
387\item In order to
388\item Lisp symbols are accessible via either directly referencing the
389  Symbol.java instance or by dynamically introspecting the
390  corresponding Package.java instance.
391\item The Lisp dynamic environment may be saved via
392  \code{LispThread.bindSpecial(Binding)} and restored via
393  \code{LispThread.resetSpecialBindings(Mark)}.
394\item Functions may be executed by invocation of the
395  Function.execute(args [...])
396\end{itemize}
397
398\subsection{Lisp FFI}
399
400FFI stands for "Foreign Function Interface" which is the phase which
401the contemporary Lisp world refers to methods of "calling out" from
402Lisp into "foreign" languages and environments.  This document
403describes the various ways that one interacts with Lisp world of ABCL
404from Java, considering the hosted Lisp as the "Foreign Function" that
405needs to be "Interfaced".
406
407\subsection{Calling Lisp from Java}
408
409Note: As the entire ABCL Lisp system resides in the org.armedbear.lisp
410package the following code snippets do not show the relevant import
411statements in the interest of brevity.  An example of the import
412statement would be
413
414\begin{listing-java}
415  import org.armedbear.lisp.*;
416\end{listing-java}
417
418to potentially import all the JVM symbol from the `org.armedbear.lisp'
419namespace.
420
421Per JVM, there can only ever be a single Lisp interpreter.  This is
422started by calling the static method `Interpreter.createInstance()`.
423
424\begin{listing-java}
425  Interpreter interpreter = Interpreter.createInstance();
426\end{listing-java}
427
428If this method has already been invoked in the lifetime of the current
429Java process it will return null, so if you are writing Java whose
430life-cycle is a bit out of your control (like in a Java servlet), a
431safer invocation pattern might be:
432
433\begin{listing-java}
434  Interpreter interpreter = Interpreter.getInstance();
435  if (interpreter == null) {
436    interpreter = Interpreter.createInstance();
437  }
438\end{listing-java}
439
440
441The Lisp \code{eval} primitive may be simply passed strings for evaluation,
442as follows
443
444\begin{listing-java}
445  String line = "(load \"file.lisp\")";
446  LispObject result = interpreter.eval(line);
447\end{listing-java}
448
449Notice that all possible return values from an arbitrary Lisp
450computation are collapsed into a single return value.  Doing useful
451further computation on the ``LispObject'' depends on knowing what the
452result of the computation might be, usually involves some amount
453of \code{instanceof} introspection, and forms a whole topic to itself
454(c.f. [Introspecting a LispObject])
455
456Using \code{eval} involves the Lisp interpreter.  Lisp functions may
457be directly invoked by Java method calls as follows.  One simply
458locates the package containing the symbol, then obtains a reference to
459the symbol, and then invokes the \code{execute()} method with the
460desired parameters.
461
462\begin{listing-java}
463    interpreter.eval("(defun foo (msg) (format nil \"You told me '~A'~%\" msg))");
464    Package pkg = Packages.findPackage("CL-USER");
465    Symbol foo = pkg.findAccessibleSymbol("FOO");
466    Function fooFunction = (Function)foo.getSymbolFunction();
467    JavaObject parameter = new JavaObject("Lisp is fun!");
468    LispObject result = fooFunction.execute(parameter);
469    // How to get the "naked string value"?
470    System.out.println("The result was " + result.writeToString());
471\end{listing-java}
472
473If one is calling an primitive function in the CL package the syntax
474becomes considerably simpler.  If we can locate the instance of
475definition in the ABCL Java source, we can invoke the symbol directly.
476For instance, to tell if a `LispObject` contains a reference to a symbol.
477
478\begin{listing-java}
479    boolean nullp(LispObject object) {
480      LispObject result = Primitives.NULL.execute(object);
481      if (result == NIL) { // the symbol 'NIL' is explicitly named in the Java
482                           // namespace at ``Symbol.NIL''
483                           // but is always present in the
484                           // local namespace in its unadorned form for
485                           // the convenience of the User.
486        return false;
487      }
488      return true;
489   }
490\end{listing-java}
491
492\subsubsection{Introspecting a LispObject}
493\label{topic:Introspecting a LispObject}
494
495We present various patterns for introspecting an an arbitrary
496`LispObject` which can represent the result of every Lisp evaluation
497into semantics that Java can meaningfully deal with.
498
499\subsubsection{LispObject as \code{boolean}}
500
501If the LispObject a generalized boolean values, one can use
502\code{getBooleanValue()} to convert to Java:
503
504\begin{listing-java}
505     LispObject object = Symbol.NIL;
506     boolean javaValue = object.getBooleanValue();
507\end{listing-java}
508
509Although since in Lisp, any value other than NIL means "true"
510(so-called generalized Boolean), the use of Java equality it quite a
511bit easier to type and more optimal in terms of information it conveys
512to the compiler would be:
513
514\begin{listing-java}
515    boolean javaValue = (object != Symbol.NIL);
516\end{listing-java}
517
518\paragraph{LispObject is a list}
519
520If LispObject is a list, it will have the type `Cons`.  One can then use
521the \code{copyToArray} to make things a bit more suitable for Java
522iteration.
523
524\begin{listing-java}
525    LispObject result = interpreter.eval("'(1 2 4 5)");
526    if (result instanceof Cons) {
527      LispObject array[] = ((Cons)result.copyToArray());
528      ...
529    }
530\end{listing-java}
531
532A more Lispy way to iterated down a list is to use the `cdr()` access
533function just as like one would traverse a list in Lisp:;
534
535\begin{listing-java}
536    LispObject result = interpreter.eval("'(1 2 4 5)");
537    while (result != Symbol.NIL) {
538      doSomething(result.car());
539      result = result.cdr();
540    }
541\end{listing-java}
542
543\subsection{Java Scripting API (JSR-223)}
544
545ABCL can be built with support for JSR-223, which offers a
546language-agnostic API to invoke other languages from Java. The binary
547distribution downloadable from ABCL's common-lisp.net home is built
548with JSR-223 support. If you're building ABCL from source on a pre-1.6
549JVM, you need to have a JSR-223 implementation in your CLASSPATH (such
550as Apache Commons BSF 3.x or greater) in order to build ABCL with
551JSR-223 support; otherwise, this feature will not be built.
552
553This section describes the design decisions behind the ABCL JSR-223
554support. It is not a description of what JSR-223 is or a tutorial on
555how to use it. See
556http://trac.common-lisp.net/armedbear/browser/trunk/abcl/examples/jsr-223
557for example usage.
558
559\subsubsection{Conversions}
560
561In general, ABCL's implementation of the JSR-223 API performs implicit
562conversion from Java objects to Lisp objects when invoking Lisp from
563Java, and the opposite when returning values from Java to Lisp. This
564potentially reduces coupling between user code and ABCL. To avoid such
565conversions, wrap the relevant objects in \code{JavaObject} instances.
566
567\subsubsection{Implemented JSR-223 interfaces}
568
569JSR-223 defines three main interfaces, of which two (Invocable and
570Compilable) are optional. ABCL implements all the three interfaces -
571ScriptEngine and the two optional ones - almost completely. While the
572JSR-223 API is not specific to a single scripting language, it was
573designed with languages with a more or less Java-like object model in
574mind: languages such as Javascript, Python, Ruby, which have a concept
575of "class" or "object" with "fields" and "methods". Lisp is a bit
576different, so certain adaptations were made, and in one case a method
577has been left unimplemented since it does not map at all to Lisp.
578
579\subsubsection{The ScriptEngine}
580
581The main interface defined by JSR-223, javax.script.ScriptEngine, is
582implemented by the class
583\code{org.armedbear.lisp.scripting.AbclScriptEngine}. AbclScriptEngine
584is a singleton, reflecting the fact that ABCL is a singleton as
585well. You can obtain an instance of AbclScriptEngine using the
586AbclScriptEngineFactory or by using the service provider mechanism
587through ScriptEngineManager (refer to the javax.script documentation).
588
589\subsubsection{Startup and configuration file}
590
591At startup (i.e. when its constructor is invoked, as part of the
592static initialization phase of AbclScriptEngineFactory) the ABCL
593script engine attempts to load an "init file" from the classpath
594(/abcl-script-config.lisp). If present, this file can be used to
595customize the behaviour of the engine, by setting a number of
596variables in the ABCL-SCRIPT package. Here is a list of the available
597variables:
598
599\begin{itemize}
600\item *use-throwing-debugger* Controls whether ABCL uses a
601  non-standard debugging hook function to throw a Java exception
602  instead of dropping into the debugger in case of unhandled error
603  conditions.
604  \begin{itemize}
605  \item Default value: T
606  \item Rationale: it is more convenient for Java programmers using
607    Lisp as a scripting language to have it return exceptions to Java
608    instead of handling them in the Lisp world.
609  \item Known Issues: the non-standard debugger hook has been reported
610    to misbehave in certain circumstances, so consider disabling it if
611    it doesn't work for you.
612  \end{itemize}
613\item *launch-swank-at-startup* If true, Swank will be launched at
614  startup. See *swank-dir* and *swank-port*.
615  \begin{itemize}
616  \item Default value: NIL
617  \end{itemize}
618\item *swank-dir* The directory where Swank is installed. Must be set
619  if *launch-swank-at-startup* is true.
620\item *swank-port* The port where Swank will listen for
621  connections. Must be set if *launch-swank-at-startup* is true.
622  \begin{itemize}
623  \item Default value: 4005
624  \end{itemize}
625\end{itemize}
626
627Additionally, at startup the AbclScriptEngine will \code{(require
628  'asdf)} - in fact, it uses asdf to load Swank.
629
630\subsubsection{Evaluation}
631
632Code is read and evaluated in the package ABCL-SCRIPT-USER. This
633packages USEs the COMMON-LISP, JAVA and ABCL-SCRIPT packages. Future
634versions of the script engine might make this default package
635configurable. The \code{CL:LOAD} function is used under the hood for
636evaluating code, and thus the same behavior of LOAD is
637guaranteed. This allows, among other things, \code{IN-PACKAGE} forms
638to change the package in which the loaded code is read.
639
640It is possible to evaluate code in what JSR-223 calls a
641"ScriptContext" (basically a flat environment of name->value
642pairs). This context is used to establish special bindings for all the
643variables defined in it; since variable names are strings from Java's
644point of view, they are first interned using READ-FROM-STRING with, as
645usual, ABCL-SCRIPT-USER as the default package. Variables are declared
646special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
647functions work in a null lexical environment and would ignore
648non-special bindings.
649
650Contrary to what the function \code{LOAD} does, evaluation of a series
651of forms returns the value of the last form instead of T, so the
652evaluation of short scripts does the Right Thing.
653
654\subsubsection{Compilation}
655
656AbclScriptEngine implements the \code{javax.script.Compilable}
657interface. Currently it only supports compilation using temporary
658files. Compiled code, returned as an instance of
659javax.script.CompiledScript, is read, compiled and executed by default
660in the ABCL-SCRIPT-USER package, just like evaluated code. Differently
661from evaluated code, though, due to the way the ABCL compiler works,
662compiled code contains no reference to top-level self-evaluating
663objects (like numbers or strings). Thus, when evaluated, a piece of
664compiled code will return the value of the last non-self-evaluating
665form: for example the code "(do-something) 42" will return 42 when
666interpreted, but will return the result of (do-something) when
667compiled and later evaluated. To ensure consistency of behavior
668between interpreted and compiled code, make sure the last form is
669always a compound form - at least (identity some-literal-object). Note
670that this issue should not matter in real code, where it is unlikely a
671top-level self-evaluating form will appear as the last form in a file
672(in fact, the Common Lisp load function always returns T upon success;
673with JSR-223 this policy has been changed to make evaluation of small
674code snippets work as intended).
675
676\subsubsection{Invocation of functions and methods}
677
678AbclScriptEngine implements the \code{javax.script.Invocable}
679interface, which allows to directly call Lisp functions and methods,
680and to obtain Lisp implementations of Java interfaces. This is only
681partially possible with Lisp since it has functions, but not methods -
682not in the traditional OO sense, at least, since Lisp methods are not
683attached to objects but belong to generic functions. Thus, the method
684\code{invokeMethod()} is not implemented and throws an
685UnsupportedOperationException when called. The \code{invokeFunction()}
686method should be used to call both regular and generic functions.
687
688\subsubsection{Implementation of Java interfaces in Lisp}
689
690ABCL can use the Java reflection-based proxy feature to implement Java
691interfaces in Lisp. It has several built-in ways to implement an
692interface, and supports definition of new ones. The
693\code{JAVA:JMAKE-PROXY} generic function is used to make such
694proxies. It has the following signature:
695
696\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
697
698\code{interface} is a Java interface metaobject (e.g. obtained by
699invoking \code{jclass}) or a string naming a Java
700interface. \code{implementation} is the object used to implement the
701interface - several built-in methods of jmake-proxy exist for various
702types of implementations. \code{lisp-this} is an object passed to the
703closures implementing the Lisp "methods" of the interface, and
704defaults to \code{NIL}.
705
706The returned proxy is an instance of the interface, with methods
707implemented with Lisp functions.
708
709Built-in interface-implementation types include:
710
711\begin{itemize}
712\item a single Lisp function which upon invocation of any method in
713  the interface will be passed the method name, the Lisp-this object,
714  and all the parameters. Useful for interfaces with a single method,
715  or to implement custom interface-implementation strategies.
716\item a hash-map of method-name -> Lisp function mappings. Function
717  signature is \code{(lisp-this \&rest args)}.
718\item a Lisp package. The name of the Java method to invoke is first
719  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
720  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
721  the package. If it exists and is fbound, the corresponding function
722  will be called. Function signature is as the hash-table case.
723\end{itemize}
724
725This functionality is exposed by the AbclScriptEngine with the two
726methods getInterface(Class) and getInterface(Object, Class). The
727former returns an interface implemented with the current Lisp package,
728the latter allows the programmer to pass an interface-implementation
729object which will in turn be passed to the jmake-proxy generic
730function.
731
732\chapter{Implementation Dependent Extensions}
733
734As outlined by the CLHS ANSI conformance guidelines, we document the
735extensions to the Armed Bear Lisp implementation made accessible to
736the user by virtue of being an exported symbol in the JAVA, THREADS,
737or EXTENSIONS packages.
738
739\section{JAVA}
740
741\subsection{Modifying the JVM CLASSPATH}
742
743The JAVA:ADD-TO-CLASSPATH generic functions allows one to add the
744specified pathname or list of pathnames to the current classpath
745used by ABCL, allowing the dynamic loading of JVM objects:
746
747\begin{listing-lisp}
748CL-USER> (add-to-classpath "/path/to/some.jar")
749\end{listing-lisp}
750
751N.b \code{add-to-classpath} only affects the classloader used by ABCL
752(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
753no effect on Java code outside ABCL.
754
755\subsection{API}
756
757% include autogen docs for the JAVA package.
758\include{java}
759
760\section{THREADS}
761
762The extensions for handling multithreaded execution are collected in
763the \code{THREADS} package.  Most of the abstractions in Doug Lea's
764excellent \code{java.util.concurrent} packages may be manipulated
765directly via the JSS contrib to great effect.
766
767\subsection{API}
768
769% include autogen docs for the THREADS package.
770\include{threads}
771
772\section{EXTENSIONS}
773
774The symbols in the EXTENSIONS package (nicknamed ``EXT'') constitutes
775extensions to the ANSI standard that are potentially useful to the
776user.  They include functions for manipulating network sockets,
777running external programs, registering object finalizers, constructing
778reference weakly held by the garbage collector and others.
779
780See \ref{Extensible Sequences} for a generic function interface to
781the native JVM contract for \code{java.util.List}.
782
783\subsection{API}
784
785% include autogen docs for the EXTENSIONS package.
786\include{extensions}
787
788\chapter{Beyond ANSI}
789
790Naturally, in striving to be a useful contemporary Common Lisp
791implementation, ABCL endeavors to include extensions beyond the ANSI
792specification which are either widely adopted or are especially useful
793in working with the hosting JVM.
794
795\section{Implementation Dependent}
796\begin{enumerate}
797  \item Compiler to JVM 5 bytecode
798  \item Pathname extensions
799\end{enumerate}
800
801\section{Pathname}
802
803We implement an extension to the Pathname that allows for the
804description and retrieval of resources named in a URI scheme that the
805JVM ``understands''.  Support is built-in to the ``http'' and
806``https'' implementations but additional protocol handlers may be
807installed at runtime by having JVM symbols present in the
808sun.net.protocol.dynmamic pacakge. See [JAVA2006] for more details.
809
810ABCL has created specializations of the ANSI Pathname object to
811enable to use of URIs to address dynamically loaded resources for the
812JVM.  A URL-PATHNAME has a corresponding URL whose cannoical
813representation is defined to be the NAMESTRING of the Pathname.
814
815\begin{verbatim}
816  JAR-PATHNAME isa URL-PATHNAME isa PATHNAME
817\end{verbatim}
818
819Both URL-PATHNAME and JAR-PATHNAME may be used anywhere a PATHNAME is
820accepted with the following caveats:
821
822\begin{itemize}
823
824\item A stream obtained via OPEN on a URL-PATHNAME cannot be the
825  target of write operations.
826
827\item No canonicalization is performed on the underlying URI (i.e. the
828implementation does not attempt to compute the current name of the
829representing resource unless it is requested to be resolved.)  Upon
830resolution, any cannoicalization procedures followed in resolving the
831resource (e.g. following redirects) are discarded. 
832
833\end{itemize}
834
835The implementation of URL-PATHNAME allows the ABCL user to laod dynamically
836code from the network.  For example, for Quicklisp.
837
838\begin{listing-lisp}
839  CL-USER> (load "http://beta.quicklisp.org/quicklisp.lisp")
840\end{listing-lisp}
841
842will load and execute the Quicklisp setup code.
843
844\ref{XACH2011}
845
846\subsubsection{Implementation}
847
848\textsc{DEVICE} either a string denoting a drive letter under DOS or a cons
849specifying a \textsc{URL-PATHNAME}.
850         
851\section{Extensible Sequences}
852
853See \ref{RHODES2007} RHODES2007 for the design.
854
855The SEQUENCE package fully implements Christopher Rhodes' proposal for
856extensible sequences.  These user extensible sequences are used
857directly in \code{java-collections.lisp} provide these CLOS
858abstractions on the standard Java collection classes as defined by the
859\code{java.util.List} contract.
860
861This extension is not automatically loaded by the implementation.   It
862may be loaded via:
863
864\begin{listing-lisp}
865CL-USER> (require 'java-collections)
866\end{listing-lisp}
867
868if both extensible sequences and their application to Java collections
869is required, or
870
871\begin{listing-lisp}
872CL-USER> (require 'extensible-sequences)
873\end{listing-lisp}
874
875if only the extensible sequences API as specified in \ref{RHODES2007} is
876required.
877
878Note that \code{(require 'java-collections)} must be issued before
879\code{java.util.List} or any subclass is used as a specializer in a CLOS
880method definition (see the section below).
881
882\section{Extensions to CLOS}
883
884There is an additional syntax for specializing the parameter of a
885generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
886where \code{CLASS-STRING} is a string naming a Java class in dotted package
887form.
888
889For instance the following specialization would perhaps allow one to
890print more information about the contents of a java.util.Collection
891object
892
893\begin{listing-lisp}
894(defmethod print-object ((coll (java:jclass "java.util.Collection"))
895                         stream)
896  ;;; ...
897)
898\end{listing-lisp}
899
900If the class had been loaded via a classloader other than the original
901the class you wish to specialize on, one needs to specify the
902classloader as an optional third argument.
903
904\begin{listing-lisp}
905
906(defparameter *other-classloader*
907  (jcall "getBaseLoader" cl-user::*classpath-manager*))
908 
909(defmethod print-object ((device-id (java:jclass "dto.nbi.service.hdm.alcatel.com.NBIDeviceID" *other-classloader*))
910                         stream)
911  ;;; ...
912)
913\end{listing-lisp}
914
915\section{Extensions to the Reader}
916
917We implement a special hexadecimal escape sequence for specifying
918characters to the Lisp reader, namely we allow a sequences of the form
919\# \textbackslash Uxxxx to be processed by the reader as character
920whose code is specified by the hexadecimal digits ``xxxx''.  The
921hexadecimal sequence must be exactly four digits long \footnote{This
922  represents a compromise with contemporary in 2011 32bit hosting
923  architecures for which we wish to make text processing efficient.
924  Should the User require more control over UNICODE processing we
925  recommend Edi Weisz' excellent work with FLEXI-STREAMS which we
926  fully support}, padded by leading zeros for values less than 0x1000.
927
928Note that this sequence is never output by the implementation.  Instead,
929the corresponding Unicode character is output for characters whose
930code is greater than 0x00ff.
931
932\subsection{JSS optionally extends the Reader}
933
934The JSS contrib consitutes an additional, optional extension to the
935reader in the definition of the \#\" reader macro.
936
937\section{ASDF}
938
939asdf-2.017.22 is packaged as core component of ABCL, but not
940intialized by default, as it relies on the CLOS subsystem which can
941take a bit of time to initialize.  It may be initialized by the ANSI
942\textsc{REQUIRE} mechanism as follows:
943
944\begin{listing-lisp}
945CL-USER> (require 'asdf)
946\end{listing-lisp}
947
948\chapter{Contrib}
949
950\section{abcl-asdf}
951
952This contrib to ABCL enables an additional syntax for ASDF system
953definition which dynamically loads JVM artifacts such as jar archives
954via a Maven encapsulation.  The Maven Aether can also be directly
955manipulated by the function associated with the RESOLVE-DEPENDENCIES symbol.
956
957%ABCL specific contributions to ASDF system definition mainly concerned
958%with finding JVM artifacts such as jar archives to be dynamically loaded.
959
960
961The following ASDF components are added: \textsc{JAR-FILE}, \textsc{JAR-DIRECTORY},
962\textsc{CLASS-FILE-DIRECTORY} and \textsc{MVN}.
963
964
965
966\subsection{ABCL-ASDF Examples}
967
968\begin{listing-lisp}
969    ;;;; -*- Mode: LISP -*-
970    (in-package :asdf)
971
972    (defsystem :log4j
973      :components ((:mvn "log4j/log4j"
974                    :version "1.4.9")))
975\end{listing-lisp}
976
977\subsection{abcl-asdf API}
978
979We define an API as consisting of the following ASDF classes:
980
981\textsc{JAR-DIRECTORY}, \textsc{JAR-FILE}, and
982\textsc{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
983valid pathname representation
984
985And the MVN and IRI classes descend from ASDF-COMPONENT, but do not
986directly have a filesystem location.
987
988For use outside of ASDF, we currently define one method,
989\textsc{RESOLVE-DEPENDENCIES} which locates, downloads, caches, and then loads
990into the currently executing JVM process all recursive dependencies
991annotated in the Maven pom.xml graph.
992
993\subsection{ABCL-ASDF Example 2}
994
995Bypassing ASDF, one can directly issue requests for the Maven
996artifacts to be downloaded
997
998\begin{listing-lisp}
999    CL-USER> (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user")
1000    WARNING: Using LATEST for unspecified version.
1001    "/Users/evenson/.m2/repository/com/google/gwt/gwt-user/2.4.0-rc1/gwt-user-2.4.0-rc1.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA-sources.jar"
1002\end{listing-lisp}
1003
1004To actually load the dependency, use the JAVA:ADD-TO-CLASSPATH generic
1005function:
1006
1007\begin{listing-lisp}
1008    CL-USER> (java:add-to-classpath (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user"))
1009\end{listing-lisp}
1010
1011Notice that all recursive dependencies have been located and installed
1012locally from the network as well.
1013
1014
1015\section{asdf-jar}
1016
1017ASDF-JAR provides a system for packaging ASDF systems into jar
1018archives for ABCL.  Given a running ABCL image with loadable ASDF
1019systems the code in this package will recursively package all the
1020required source and fasls in a jar archive.
1021
1022\section{jss}
1023
1024To one used to a syntax that can construct macros, the Java syntax
1025sucks, so we introduce the \#" macro.
1026
1027\subsection{JSS usage}
1028
1029Example:
1030
1031\begin{listing-lisp}
1032  CL-USER> (require 'jss)
1033
1034  CL-USER) (#"getProperties" 'java.lang.System)
1035
1036  CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
1037\end{listing-lisp}
1038
1039\section{asdf-install}
1040
1041An implementation of ASDF-INSTALL.  Superseded by Quicklisp (qv.)
1042
1043\chapter{History}
1044
1045ABCL was originally the extension language for the J editor, which was
1046started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1047code that had previously not been released publically was suddenly
1048committed that enabled ABCL to be plausibly termed an emergent ANSI
1049Common Lisp implementation canidate.
1050
1051From 2006 to 2008, Peter manned the development lists, incorporating
1052patches as made sense.  After a suitable search, Peter nominated Erik
1053Huelsmann to take over the project.
1054
1055In 2008, the implementation was transferred to the current
1056maintainers, who have strived to improve its usability as a
1057contemporary Common Lisp implementation.
1058
1059On October 22, 2011, with the publication of this Manual explicitly
1060stating the conformance of Armed Bear Common Lisp to ANSI, we released
1061abcl-1.0.0.
1062
1063
1064
1065
1066\section{References}
1067
1068[Java2000]:  A New Era for Java Protocol Handlers.
1069\url{http://java.sun.com/developer/onlineTraining/protocolhandlers/}
1070
1071[Xach2011]:  Quicklisp:  A system for quickly constructing Common Lisp
1072libraries.  \url{http://www.quicklisp.org/}
1073
1074[RHODES2007]:  Christopher Rhodes
1075
1076
1077\end{document}
1078
1079% TODO
1080%   1.  Create mechanism for swigging DocString and Lisp docs into
1081%       sections ('grovel.lisp')
1082
Note: See TracBrowser for help on using the repository browser.