source: branches/1.0.x/abcl/doc/manual/abcl.tex @ 13661

Last change on this file since 13661 was 13661, checked in by Mark Evenson, 10 years ago

Backport Erik's contribution to accessing Java fields.

File size: 40.9 KB
Line 
1% -*- mode: latex; -*-
2% http://en.wikibooks.org/wiki/LaTeX/
3\documentclass[10pt]{book}
4\usepackage{abcl}
5
6\begin{document}
7\title{A Manual for Armed Bear Common Lisp}
8\date{October 21, 2011}
9\author{Mark~Evenson, Erik~Huelsmann, Alessio~Stalla, Ville~Voutilainen}
10
11\maketitle
12
13\chapter{Introduction}
14
15Armed Bear is a (mostly) conforming implementation of the ANSI Common
16Lisp standard.  This manual documents the Armed Bear Common Lisp
17implementation for users of the system.
18
19\subsection{Version}
20This manual corresponds to abcl-1.0.0, released on October 22, 2011.
21
22\subsection{License}
23
24The implementation is licensed under the terms of the GPL v2 of June
251991 with the ``classpath-exception'' that makes its deployment in
26commercial settings quite reasonable.  The license is viral in the
27sense that if you change the implementation, and redistribute those
28changes, you are required to provide the source to those changes back
29to be merged with the public trunk.
30
31\subsection{Contributors}
32
33% TODO format this better, optionally link to URI
34
35% Thanks for the markup
36Philipp Marek
37
38% Thanks for the whacky IKVM stuff and keeping the flame alive
39Douglas Miles
40
41% Thanks for JSS
42Alan Ruttenberg
43
44and of course
45
46Peter Graves
47
48\chapter{Running}
49
50\textsc{ABCL} is packaged as a single jar file usually named either
51``abcl.jar'' or possibly``abcl-1.0.0.jar'' if one is using a versioned
52package from your system vendor.  This byte archive can be executed
53under the control of a suitable JVM by using the ``-jar'' option to
54parse the manifest, and select the named class
55(\code{org.armedbear.lisp.Main}) for execution, viz:
56
57\begin{listing-shell}
58  cmd$ java -jar abcl.jar
59\end{listing-shell}
60
61N.b. for the proceeding command to work, the ``java'' executable needs
62to be in your path.
63
64To make it easier to facilitate the use of ABCL in tool chains (such
65as SLIME \footnote{SLIME is the Superior Lisp Mode for Interaction
66  under Emacs}) the invocation is wrapped in a Bourne shell script
67under UNIX or a DOS command script under Windows so that ABCL may be
68executed simply as:
69
70\begin{listing-shell}
71  cmd$ abcl
72\end{listing-shell}
73
74\section{Options}
75
76ABCL supports the following command line options:
77
78\begin{verbatim}
79--help
80    Displays this message.
81--noinform
82    Suppresses the printing of startup information and banner.
83--noinit
84    Suppresses the loading of the '~/.abclrc' startup file.
85--nosystem
86    Suppresses loading the 'system.lisp' customization file.
87--eval <FORM>
88    Evaluates the <FORM> before initializing REPL.
89--load <FILE>
90    Loads the file <FILE> before initializing REPL.
91--load-system-file <FILE>
92    Loads the system file <FILE> before initializing REPL.
93--batch
94    The process evaluates forms specified by arguments and possibly by those
95    by those in the intialization file '~/.abcl', and then exits.
96
97The occurance of '--' copies the remaining arguments, unprocessed, into
98the variable EXTENSIONS:*COMMAND-LINE-ARGUMENT-LIST*.
99\end{verbatim}
100
101All of the command line arguments which follow the occurrence of ``----''
102are passed into a list bound to the EXT:*COMMAND-LINE-ARGUMENT-LIST*
103variable.
104
105\section{Initialization}
106
107If the ABCL process is started without the ``--noinit'' flag, it
108attempts to load a file named ``.abclrc'' located in the user's home
109directory and then interpret its contents. 
110
111The user's home directory is determined by the value of the JVM system
112property ``user.home''.  This value may--or may not--correspond to the
113value of the HOME system environment variable at the discretion of the
114JVM implementation that \textsc{ABCL} finds itself hosted upon.
115
116\chapter{Conformance}
117
118\section{ANSI Common Lisp}
119\textsc{ABCL} is currently a (non)-conforming ANSI Common Lisp
120implementation due to the following known issues:
121
122\begin{itemize}
123  \item The generic function signatures of the DOCUMENTATION symbol do
124    not match the CLHS.
125  \item The TIME form does not return a proper VALUES environment to
126    its caller.
127\end{itemize}
128
129Somewhat confusingly, this statement of non-conformance in the
130accompanying user documentation fullfills the requirements that
131\textsc{ABCL} is a conforming ANSI Common Lisp implementation
132according to the CLHS \footnote{Common Lisp Hyperspec language
133  reference document.}.  Clarifications to this point are solicited.
134
135ABCL aims to be be a fully conforming ANSI Common Lisp implementation.
136Any other behavior should be reported as a bug.
137
138\section{Contemporary Common Lisp}
139In addition to ANSI conformance, \textsc{ABCL} strives to implement features
140expected of a contemporary Common Lisp \footnote{i.e. a Lisp of the
141  post 2005 Renaissance}
142
143\subsection{Deficiencies}
144The following known problems detract from \textsc{ABCL} being a proper
145contemporary Comon Lisp.
146\begin{itemize}
147  \item An incomplete implementation of a properly named metaobject
148    protocol (viz. (A)MOP \footnote{Another Metaobject Protocol} )
149
150    % N.b.
151    % TODO go through AMOP with symbols, starting by looking for
152    % matching function signature.
153    % XXX is this really blocking ANSI conformance?  Answer: we have
154    % to start with such a ``census'' to determine what we have.
155
156  \item Incomplete streams abstraction, in that \textsc{ABCL} needs suitable
157    abstraction between ANSI and Gray streams.  The streams could be
158    optimized to the JVM NIO abstractions at great profit for binary
159    byte-level manipulations.
160   
161  \item Incomplete documentation (missing docstrings from exported
162      symbols.
163
164\end{itemize}
165
166\chapter{Interaction with Hosting JVM}
167
168%  Plan of Attack
169%
170% describe calling Java from Lisp, and calling Lisp from Java,
171% probably in two separate sections.  Presumably, we can partition our
172% audience into those who are more comfortable with Java, and those
173% that are more comforable with Lisp
174
175The Armed Bear Common Lisp implementation is hosted on a Java Virtual
176Machine.  This chapter describes the mechanisms by which the
177implementation interacts with that hosting mechanism.
178
179\section{Lisp to Java}
180
181\textsc{ABCL} offers a number of mechanisms to interact with Java from its
182Lisp environment. It allows calling both instance and static methods
183of Java objects, manipulation of instance and static fields on Java
184objects, and construction of new Java objects.
185
186When calling Java routines, some values will automatically be
187converted by the FFI \footnote{FFI stands for Foreign Function
188  Interface which is the term of art which describes how a Lisp
189  implementation encapsulates invocation in other languages.}  from
190Lisp values to Java values. These conversions typically apply to
191strings, integers and floats. Other values need to be converted to
192their Java equivalents by the programmer before calling the Java
193object method. Java values returned to Lisp are also generally
194converted back to their Lisp counterparts. Some operators make an
195exception to this rule and do not perform any conversion; those are
196the ``raw'' counterparts of certain FFI functions and are recognizable
197by their name ending with \code{-RAW}.
198
199\subsection{Low-level Java API}
200
201We define a higher level Java API in the \ref{topic:Higher level Java
202  API: JSS}(JSS package) which is available in the \code{contrib/} \ref{topic:contrib}
203directory. This package is described later in this document.  This
204section covers the lower level API directly available after evaluating
205\code{(require 'JAVA)}.
206
207\subsubsection{Calling Java Object Methods}
208
209There are two ways to call a Java object method in the low-level (basic) API:
210
211\begin{itemize}
212\item Call a specific method reference (which was previously acquired)
213\item Dynamic dispatch using the method name and
214  the call-specific arguments provided by finding the
215  \ref{section:Parameter matching for FFI dynamic dispatch}{best match}.
216\end{itemize}
217
218The dynamic dispatch variant is discussed in the next section.
219
220\code{JAVA:JMETHOD} is used to acquire a specific method reference.
221The function takes at two or more arguments. The first is Java class designator
222(a \code{JAVA:JAVA-CLASS} object returned by \code{JAVA:JCLASS} or a string naming
223a Java class). The second is a string naming the method.
224
225Any arguments beyond the first two should be strings naming Java classes with
226one exception as listed in the next paragraph. These
227classes specify the types of the arguments for the method to be returned.
228
229There's additional calling convention to the \code{JAVA:JMETHOD} function:
230When the method is called with three parameters and the last parameter is an
231integer, the first method by that name and matching number of parameters is
232returned.
233
234Once one has a reference to the method, one may invoke it using
235\code{JAVA:JCALL}, which takes the method as the first argument. The
236second argument is the object instance to call the method on, or
237\code{NIL} in case of a static method.  Any remaining parameters are
238used as the remaining arguments for the call.
239
240\subsubsection{Calling Java object methods: dynamic dispatch}
241
242The second way of calling Java object methods is by using dynamic dispatch.
243In this case \code{JAVA:JCALL} is used directly without acquiring a method
244reference first. In this case, the first argument provided to \code{JAVA:JCALL}
245is a string naming the method to be called. The second argument is the instance
246on which the method should be called and any further arguments are used to
247select the best matching method and dispatch the call.
248
249\subsubsection{Dynamic dispatch: Caveats}
250
251Dynamic dispatch is performed by using the Java reflection
252API \footnote{The Java reflection API is found in the
253  \code{java.lang.reflect} package}. Generally the dispatch works
254fine, but there are corner cases where the API does not correctly
255reflect all the details involved in calling a Java method. An example
256is the following Java code:
257
258\begin{listing-java}
259ZipFile jar = new ZipFile("/path/to/some.jar");
260Object els = jar.entries();
261Method method = els.getClass().getMethod("hasMoreElements");
262method.invoke(els);
263\end{listing-java}
264
265even though the method \code{hasMoreElements()} is public in \code{Enumeration},
266the above code fails with
267
268\begin{listing-java}
269java.lang.IllegalAccessException: Class ... can
270not access a member of class java.util.zip.ZipFile\$2 with modifiers
271"public"
272       at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
273       at java.lang.reflect.Method.invoke(Method.java:583)
274       at ...
275\end{listing-java}
276
277because the method has been overridden by a non-public class and the
278reflection API, unlike javac, is not able to handle such a case.
279
280While code like that is uncommon in Java, it is typical of ABCL's FFI
281calls. The code above corresponds to the following Lisp code:
282
283\begin{listing-lisp}
284(let ((jar (jnew "java.util.zip.ZipFile" "/path/to/some.jar")))
285  (let ((els (jcall "entries" jar)))
286    (jcall "hasMoreElements" els)))
287\end{listing-lisp}
288
289except that the dynamic dispatch part is not shown.
290
291To avoid such pitfalls, all Java objects in \textsc{ABCL} carry an extra
292field representing the ``intended class'' of the object. That is the class
293that is used first by \code{JAVA:JCALL} and similar to resolve methods;
294the actual class of the object is only tried if the method is not found
295in the intended class. Of course, the intended class is always a super-class
296of the actual class - in the worst case, they coincide. The intended class
297is deduced by the return type of the method that originally returned
298the Java object; in the case above, the intended class of \code{ELS}
299is \code{java.util.Enumeration} because that's the return type of
300the \code{entries} method.
301
302While this strategy is generally effective, there are cases where the
303intended class becomes too broad to be useful. The typical example
304is the extraction of an element from a collection, since methods in
305the collection API erase all types to \code{Object}. The user can
306always force a more specific intended class by using the \code{JAVA:JCOERCE}
307operator.
308
309% \begin{itemize}
310% \item Java values are accessible as objects of type JAVA:JAVA-OBJECT.
311% \item The Java FFI presents a Lisp package (JAVA) with many useful
312%   symbols for manipulating the artifacts of expectation on the JVM,
313%   including creation of new objects \ref{JAVA:JNEW}, \ref{JAVA:JMETHOD}), the
314%   introspection of values \ref{JAVA:JFIELD}, the execution of methods
315%   (\ref{JAVA:JCALL}, \ref{JAVA:JCALL-RAW}, \ref{JAVA:JSTATIC})
316% \item The JSS package (\ref{JSS}) in contrib introduces a convenient macro
317%   syntax \ref{JSS:SHARPSIGN_DOUBLEQUOTE_MACRO} for accessing Java
318%   methods, and additional convenience functions.
319% \item Java classes and libraries may be dynamically added to the
320%   classpath at runtime (JAVA:ADD-TO-CLASSPATH).
321% \end{itemize}
322
323\subsubsection{Calling Java class static methods}
324
325Like with non-static methods, references to static methods can be acquired
326by using the \code{JAVA:JMETHOD} primitive. In order to call this method,
327it's not possible to use the \code{JAVA:JCALL} primitive however: there's a
328separate API to retrieve a reference to static methods. This
329primitive is called \code{JAVA:JSTATIC}.
330
331Like \code{JAVA:JCALL}, \code{JAVA:JSTATIC} supports dynamic dispatch by
332passing the name of the method as a string instead of passing a method reference.
333The parameter values should be values to pass in the function call instead of
334a specification of classes for each parameter.
335
336\subsubsection{Parameter matching for FFI dynamic dispatch}
337
338The algorithm used to resolve the best matching method given the name
339and the arguments' types is the same as described in the Java Language
340Specification. Any deviation should be reported as a bug.
341
342% ###TODO reference to correct JLS section
343
344\subsubsection{Instantiating Java objects}
345
346Java objects can be instantiated (created) from Lisp by calling
347a constructor from the class of the object to be created. The same way
348\code{JAVA:JMETHOD} is used to acquire a method reference, the
349\code{JAVA:JCONSTRUCTOR} primitive can be used to acquire a constructor
350reference. It's arguments specify the types of arguments of the constructor
351method the same way as with \code{JAVA:JMETHOD}.
352
353The constructor can't be passed to \code{JAVA:JCALL}, but instead should
354be passed as an argument to \code{JAVA:JNEW}.
355
356\subsubsection{Accessing Java object fields}
357
358Fields in Java objects can be accessed using the getter and setter functions
359\code{JAVA:GETFIELD} and \code{JAVA:PUTFIELD}. This applies to values stored in object
360instances. If you want to access static fields: see the next section.
361
362Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
363an intended class around and values which can be converted to Lisp values will
364be converted.
365
366\subsubsection{Accessing Java static fields}
367
368Static fields in Java objects (class fields) can be accessed using the getter
369and setter functions \code{JAVA:GETSTATIC} and \code{JAVA:PUTSTATIC}. Values
370stored in object instance fields can be accessed as described in the previous
371section.
372
373Like \code{JAVA:JCALL} and friends, values returned from these accessors carry
374an intended class around and values which can be converted to Lisp values will
375be converted.
376
377\section{Lisp from Java}
378
379In order to access the Lisp world from Java, one needs to be aware
380of a few things. The most important ones are listed below.
381
382\begin{itemize}
383\item All Lisp values are descendants of LispObject.java
384\item In order to
385\item Lisp symbols are accessible via either directly referencing the
386  Symbol.java instance or by dynamically introspecting the
387  corresponding Package.java instance.
388\item The Lisp dynamic environment may be saved via
389  \code{LispThread.bindSpecial(Binding)} and restored via
390  \code{LispThread.resetSpecialBindings(Mark)}.
391\item Functions may be executed by invocation of the
392  Function.execute(args [...])
393\end{itemize}
394
395\subsection{Lisp FFI}
396
397FFI stands for "Foreign Function Interface" which is the phase which
398the contemporary Lisp world refers to methods of "calling out" from
399Lisp into "foreign" languages and environments.  This document
400describes the various ways that one interacts with Lisp world of ABCL
401from Java, considering the hosted Lisp as the "Foreign Function" that
402needs to be "Interfaced".
403
404\subsection{Calling Lisp from Java}
405
406Note: As the entire ABCL Lisp system resides in the org.armedbear.lisp
407package the following code snippets do not show the relevant import
408statements in the interest of brevity.  An example of the import
409statement would be
410
411\begin{listing-java}
412  import org.armedbear.lisp.*;
413\end{listing-java}
414
415to potentially import all the JVM symbol from the `org.armedbear.lisp'
416namespace.
417
418Per JVM, there can only ever be a single Lisp interpreter.  This is
419started by calling the static method `Interpreter.createInstance()`.
420
421\begin{listing-java}
422  Interpreter interpreter = Interpreter.createInstance();
423\end{listing-java}
424
425If this method has already been invoked in the lifetime of the current
426Java process it will return null, so if you are writing Java whose
427life-cycle is a bit out of your control (like in a Java servlet), a
428safer invocation pattern might be:
429
430\begin{listing-java}
431  Interpreter interpreter = Interpreter.getInstance();
432  if (interpreter == null) {
433    interpreter = Interpreter.createInstance();
434  }
435\end{listing-java}
436
437
438The Lisp \code{eval} primitive may be simply passed strings for evaluation,
439as follows
440
441\begin{listing-java}
442  String line = "(load \"file.lisp\")";
443  LispObject result = interpreter.eval(line);
444\end{listing-java}
445
446Notice that all possible return values from an arbitrary Lisp
447computation are collapsed into a single return value.  Doing useful
448further computation on the ``LispObject'' depends on knowing what the
449result of the computation might be, usually involves some amount
450of \code{instanceof} introspection, and forms a whole topic to itself
451(c.f. [Introspecting a LispObject])
452
453Using \code{eval} involves the Lisp interpreter.  Lisp functions may
454be directly invoked by Java method calls as follows.  One simply
455locates the package containing the symbol, then obtains a reference to
456the symbol, and then invokes the \code{execute()} method with the
457desired parameters.
458
459\begin{listing-java}
460    interpreter.eval("(defun foo (msg) (format nil \"You told me '~A'~%\" msg))");
461    Package pkg = Packages.findPackage("CL-USER");
462    Symbol foo = pkg.findAccessibleSymbol("FOO");
463    Function fooFunction = (Function)foo.getSymbolFunction();
464    JavaObject parameter = new JavaObject("Lisp is fun!");
465    LispObject result = fooFunction.execute(parameter);
466    // How to get the "naked string value"?
467    System.out.println("The result was " + result.writeToString());
468\end{listing-java}
469
470If one is calling an primitive function in the CL package the syntax
471becomes considerably simpler.  If we can locate the instance of
472definition in the ABCL Java source, we can invoke the symbol directly.
473For instance, to tell if a `LispObject` contains a reference to a symbol.
474
475\begin{listing-java}
476    boolean nullp(LispObject object) {
477      LispObject result = Primitives.NULL.execute(object);
478      if (result == NIL) { // the symbol 'NIL' is explicitly named in the Java
479                           // namespace at ``Symbol.NIL''
480                           // but is always present in the
481                           // local namespace in its unadorned form for
482                           // the convenience of the User.
483        return false;
484      }
485      return true;
486   }
487\end{listing-java}
488
489\subsubsection{Introspecting a LispObject}
490\label{topic:Introspecting a LispObject}
491
492We present various patterns for introspecting an an arbitrary
493`LispObject` which can represent the result of every Lisp evaluation
494into semantics that Java can meaningfully deal with.
495
496\subsubsection{LispObject as \code{boolean}}
497
498If the LispObject a generalized boolean values, one can use
499\code{getBooleanValue()} to convert to Java:
500
501\begin{listing-java}
502     LispObject object = Symbol.NIL;
503     boolean javaValue = object.getBooleanValue();
504\end{listing-java}
505
506Although since in Lisp, any value other than NIL means "true"
507(so-called generalized Boolean), the use of Java equality it quite a
508bit easier to type and more optimal in terms of information it conveys
509to the compiler would be:
510
511\begin{listing-java}
512    boolean javaValue = (object != Symbol.NIL);
513\end{listing-java}
514
515\paragraph{LispObject is a list}
516
517If LispObject is a list, it will have the type `Cons`.  One can then use
518the \code{copyToArray} to make things a bit more suitable for Java
519iteration.
520
521\begin{listing-java}
522    LispObject result = interpreter.eval("'(1 2 4 5)");
523    if (result instanceof Cons) {
524      LispObject array[] = ((Cons)result.copyToArray());
525      ...
526    }
527\end{listing-java}
528
529A more Lispy way to iterated down a list is to use the `cdr()` access
530function just as like one would traverse a list in Lisp:;
531
532\begin{listing-java}
533    LispObject result = interpreter.eval("'(1 2 4 5)");
534    while (result != Symbol.NIL) {
535      doSomething(result.car());
536      result = result.cdr();
537    }
538\end{listing-java}
539
540\subsection{Java Scripting API (JSR-223)}
541
542ABCL can be built with support for JSR-223, which offers a
543language-agnostic API to invoke other languages from Java. The binary
544distribution downloadable from ABCL's common-lisp.net home is built
545with JSR-223 support. If you're building ABCL from source on a pre-1.6
546JVM, you need to have a JSR-223 implementation in your CLASSPATH (such
547as Apache Commons BSF 3.x or greater) in order to build ABCL with
548JSR-223 support; otherwise, this feature will not be built.
549
550This section describes the design decisions behind the ABCL JSR-223
551support. It is not a description of what JSR-223 is or a tutorial on
552how to use it. See
553http://trac.common-lisp.net/armedbear/browser/trunk/abcl/examples/jsr-223
554for example usage.
555
556\subsubsection{Conversions}
557
558In general, ABCL's implementation of the JSR-223 API performs implicit
559conversion from Java objects to Lisp objects when invoking Lisp from
560Java, and the opposite when returning values from Java to Lisp. This
561potentially reduces coupling between user code and ABCL. To avoid such
562conversions, wrap the relevant objects in \code{JavaObject} instances.
563
564\subsubsection{Implemented JSR-223 interfaces}
565
566JSR-223 defines three main interfaces, of which two (Invocable and
567Compilable) are optional. ABCL implements all the three interfaces -
568ScriptEngine and the two optional ones - almost completely. While the
569JSR-223 API is not specific to a single scripting language, it was
570designed with languages with a more or less Java-like object model in
571mind: languages such as Javascript, Python, Ruby, which have a concept
572of "class" or "object" with "fields" and "methods". Lisp is a bit
573different, so certain adaptations were made, and in one case a method
574has been left unimplemented since it does not map at all to Lisp.
575
576\subsubsection{The ScriptEngine}
577
578The main interface defined by JSR-223, javax.script.ScriptEngine, is
579implemented by the class
580\code{org.armedbear.lisp.scripting.AbclScriptEngine}. AbclScriptEngine
581is a singleton, reflecting the fact that ABCL is a singleton as
582well. You can obtain an instance of AbclScriptEngine using the
583AbclScriptEngineFactory or by using the service provider mechanism
584through ScriptEngineManager (refer to the javax.script documentation).
585
586\subsubsection{Startup and configuration file}
587
588At startup (i.e. when its constructor is invoked, as part of the
589static initialization phase of AbclScriptEngineFactory) the ABCL
590script engine attempts to load an "init file" from the classpath
591(/abcl-script-config.lisp). If present, this file can be used to
592customize the behaviour of the engine, by setting a number of
593variables in the ABCL-SCRIPT package. Here is a list of the available
594variables:
595
596\begin{itemize}
597\item *use-throwing-debugger* Controls whether ABCL uses a
598  non-standard debugging hook function to throw a Java exception
599  instead of dropping into the debugger in case of unhandled error
600  conditions.
601  \begin{itemize}
602  \item Default value: T
603  \item Rationale: it is more convenient for Java programmers using
604    Lisp as a scripting language to have it return exceptions to Java
605    instead of handling them in the Lisp world.
606  \item Known Issues: the non-standard debugger hook has been reported
607    to misbehave in certain circumstances, so consider disabling it if
608    it doesn't work for you.
609  \end{itemize}
610\item *launch-swank-at-startup* If true, Swank will be launched at
611  startup. See *swank-dir* and *swank-port*.
612  \begin{itemize}
613  \item Default value: NIL
614  \end{itemize}
615\item *swank-dir* The directory where Swank is installed. Must be set
616  if *launch-swank-at-startup* is true.
617\item *swank-port* The port where Swank will listen for
618  connections. Must be set if *launch-swank-at-startup* is true.
619  \begin{itemize}
620  \item Default value: 4005
621  \end{itemize}
622\end{itemize}
623
624Additionally, at startup the AbclScriptEngine will \code{(require
625  'asdf)} - in fact, it uses asdf to load Swank.
626
627\subsubsection{Evaluation}
628
629Code is read and evaluated in the package ABCL-SCRIPT-USER. This
630packages USEs the COMMON-LISP, JAVA and ABCL-SCRIPT packages. Future
631versions of the script engine might make this default package
632configurable. The \code{CL:LOAD} function is used under the hood for
633evaluating code, and thus the same behavior of LOAD is
634guaranteed. This allows, among other things, \code{IN-PACKAGE} forms
635to change the package in which the loaded code is read.
636
637It is possible to evaluate code in what JSR-223 calls a
638"ScriptContext" (basically a flat environment of name->value
639pairs). This context is used to establish special bindings for all the
640variables defined in it; since variable names are strings from Java's
641point of view, they are first interned using READ-FROM-STRING with, as
642usual, ABCL-SCRIPT-USER as the default package. Variables are declared
643special because CL's \code{LOAD}, \code{EVAL} and \code{COMPILE}
644functions work in a null lexical environment and would ignore
645non-special bindings.
646
647Contrary to what the function \code{LOAD} does, evaluation of a series
648of forms returns the value of the last form instead of T, so the
649evaluation of short scripts does the Right Thing.
650
651\subsubsection{Compilation}
652
653AbclScriptEngine implements the \code{javax.script.Compilable}
654interface. Currently it only supports compilation using temporary
655files. Compiled code, returned as an instance of
656javax.script.CompiledScript, is read, compiled and executed by default
657in the ABCL-SCRIPT-USER package, just like evaluated code. Differently
658from evaluated code, though, due to the way the ABCL compiler works,
659compiled code contains no reference to top-level self-evaluating
660objects (like numbers or strings). Thus, when evaluated, a piece of
661compiled code will return the value of the last non-self-evaluating
662form: for example the code "(do-something) 42" will return 42 when
663interpreted, but will return the result of (do-something) when
664compiled and later evaluated. To ensure consistency of behavior
665between interpreted and compiled code, make sure the last form is
666always a compound form - at least (identity some-literal-object). Note
667that this issue should not matter in real code, where it is unlikely a
668top-level self-evaluating form will appear as the last form in a file
669(in fact, the Common Lisp load function always returns T upon success;
670with JSR-223 this policy has been changed to make evaluation of small
671code snippets work as intended).
672
673\subsubsection{Invocation of functions and methods}
674
675AbclScriptEngine implements the \code{javax.script.Invocable}
676interface, which allows to directly call Lisp functions and methods,
677and to obtain Lisp implementations of Java interfaces. This is only
678partially possible with Lisp since it has functions, but not methods -
679not in the traditional OO sense, at least, since Lisp methods are not
680attached to objects but belong to generic functions. Thus, the method
681\code{invokeMethod()} is not implemented and throws an
682UnsupportedOperationException when called. The \code{invokeFunction()}
683method should be used to call both regular and generic functions.
684
685\subsubsection{Implementation of Java interfaces in Lisp}
686
687ABCL can use the Java reflection-based proxy feature to implement Java
688interfaces in Lisp. It has several built-in ways to implement an
689interface, and supports definition of new ones. The
690\code{JAVA:JMAKE-PROXY} generic function is used to make such
691proxies. It has the following signature:
692
693\code{jmake-proxy interface implementation \&optional lisp-this ==> proxy}
694
695\code{interface} is a Java interface metaobject (e.g. obtained by
696invoking \code{jclass}) or a string naming a Java
697interface. \code{implementation} is the object used to implement the
698interface - several built-in methods of jmake-proxy exist for various
699types of implementations. \code{lisp-this} is an object passed to the
700closures implementing the Lisp "methods" of the interface, and
701defaults to \code{NIL}.
702
703The returned proxy is an instance of the interface, with methods
704implemented with Lisp functions.
705
706Built-in interface-implementation types include:
707
708\begin{itemize}
709\item a single Lisp function which upon invocation of any method in
710  the interface will be passed the method name, the Lisp-this object,
711  and all the parameters. Useful for interfaces with a single method,
712  or to implement custom interface-implementation strategies.
713\item a hash-map of method-name -> Lisp function mappings. Function
714  signature is \code{(lisp-this \&rest args)}.
715\item a Lisp package. The name of the Java method to invoke is first
716  transformed in an idiomatic Lisp name (\code{javaMethodName} becomes
717  \code{JAVA-METHOD-NAME}) and a symbol with that name is searched in
718  the package. If it exists and is fbound, the corresponding function
719  will be called. Function signature is as the hash-table case.
720\end{itemize}
721
722This functionality is exposed by the AbclScriptEngine with the two
723methods getInterface(Class) and getInterface(Object, Class). The
724former returns an interface implemented with the current Lisp package,
725the latter allows the programmer to pass an interface-implementation
726object which will in turn be passed to the jmake-proxy generic
727function.
728
729\chapter{Implementation Dependent Extensions}
730
731As outlined by the CLHS ANSI conformance guidelines, we document the
732extensions to the Armed Bear Lisp implementation made accessible to
733the user by virtue of being an exported symbol in the JAVA, THREADS,
734or EXTENSIONS packages.
735
736\section{JAVA}
737
738\subsection{Modifying the JVM CLASSPATH}
739
740The JAVA:ADD-TO-CLASSPATH generic functions allows one to add the
741specified pathname or list of pathnames to the current classpath
742used by ABCL, allowing the dynamic loading of JVM objects:
743
744\begin{listing-lisp}
745CL-USER> (add-to-classpath "/path/to/some.jar")
746\end{listing-lisp}
747
748N.b \code{add-to-classpath} only affects the classloader used by ABCL
749(the value of the special variable \code{JAVA:*CLASSLOADER*}. It has
750no effect on Java code outside ABCL.
751
752\subsection{API}
753
754% include autogen docs for the JAVA package.
755\include{java}
756
757\section{THREADS}
758
759The extensions for handling multithreaded execution are collected in
760the \code{THREADS} package.  Most of the abstractions in Doug Lea's
761excellent \code{java.util.concurrent} packages may be manipulated
762directly via the JSS contrib to great effect.
763
764\subsection{API}
765
766% include autogen docs for the THREADS package.
767\include{threads}
768
769\section{EXTENSIONS}
770
771The symbols in the EXTENSIONS package (nicknamed ``EXT'') constitutes
772extensions to the ANSI standard that are potentially useful to the
773user.  They include functions for manipulating network sockets,
774running external programs, registering object finalizers, constructing
775reference weakly held by the garbage collector and others.
776
777See \ref{Extensible Sequences} for a generic function interface to
778the native JVM contract for \code{java.util.List}.
779
780\subsection{API}
781
782% include autogen docs for the EXTENSIONS package.
783\include{extensions}
784
785\chapter{Beyond ANSI}
786
787Naturally, in striving to be a useful contemporary Common Lisp
788implementation, ABCL endeavors to include extensions beyond the ANSI
789specification which are either widely adopted or are especially useful
790in working with the hosting JVM.
791
792\section{Implementation Dependent}
793\begin{enumerate}
794  \item Compiler to JVM 5 bytecode
795  \item Pathname extensions
796\end{enumerate}
797
798\section{Pathname}
799
800We implement an extension to the Pathname that allows for the
801description and retrieval of resources named in a URI scheme that the
802JVM ``understands''.  Support is built-in to the ``http'' and
803``https'' implementations but additional protocol handlers may be
804installed at runtime by having JVM symbols present in the
805sun.net.protocol.dynmamic pacakge. See [JAVA2006] for more details.
806
807ABCL has created specializations of the ANSI Pathname object to
808enable to use of URIs to address dynamically loaded resources for the
809JVM.  A URL-PATHNAME has a corresponding URL whose cannoical
810representation is defined to be the NAMESTRING of the Pathname.
811
812\begin{verbatim}
813  JAR-PATHNAME isa URL-PATHNAME isa PATHNAME
814\end{verbatim}
815
816Both URL-PATHNAME and JAR-PATHNAME may be used anywhere a PATHNAME is
817accepted with the following caveats:
818
819\begin{itemize}
820
821\item A stream obtained via OPEN on a URL-PATHNAME cannot be the
822  target of write operations.
823
824\item No canonicalization is performed on the underlying URI (i.e. the
825implementation does not attempt to compute the current name of the
826representing resource unless it is requested to be resolved.)  Upon
827resolution, any cannoicalization procedures followed in resolving the
828resource (e.g. following redirects) are discarded. 
829
830\end{itemize}
831
832The implementation of URL-PATHNAME allows the ABCL user to laod dynamically
833code from the network.  For example, for Quicklisp.
834
835\begin{listing-lisp}
836  CL-USER> (load "http://beta.quicklisp.org/quicklisp.lisp")
837\end{listing-lisp}
838
839will load and execute the Quicklisp setup code.
840
841\ref{XACH2011}
842
843\subsubsection{Implementation}
844
845\textsc{DEVICE} either a string denoting a drive letter under DOS or a cons
846specifying a \textsc{URL-PATHNAME}.
847         
848\section{Extensible Sequences}
849
850See \ref{RHODES2007} RHODES2007 for the design.
851
852The SEQUENCE package fully implements Christopher Rhodes' proposal for
853extensible sequences.  These user extensible sequences are used
854directly in \code{java-collections.lisp} provide these CLOS
855abstractions on the standard Java collection classes as defined by the
856\code{java.util.List} contract.
857
858This extension is not automatically loaded by the implementation.   It
859may be loaded via:
860
861\begin{listing-lisp}
862CL-USER> (require 'java-collections)
863\end{listing-lisp}
864
865if both extensible sequences and their application to Java collections
866is required, or
867
868\begin{listing-lisp}
869CL-USER> (require 'extensible-sequences)
870\end{listing-lisp}
871
872if only the extensible sequences API as specified in \ref{RHODES2007} is
873required.
874
875Note that \code{(require 'java-collections)} must be issued before
876\code{java.util.List} or any subclass is used as a specializer in a CLOS
877method definition (see the section below).
878
879\section{Extensions to CLOS}
880
881There is an additional syntax for specializing the parameter of a
882generic function on a java class, viz. \code{(java:jclass CLASS-STRING)}
883where \code{CLASS-STRING} is a string naming a Java class in dotted package
884form.
885
886For instance the following specialization would perhaps allow one to
887print more information about the contents of a java.util.Collection
888object
889
890\begin{listing-lisp}
891(defmethod print-object ((coll (java:jclass "java.util.Collection"))
892                         stream)
893  ;;; ...
894)
895\end{listing-lisp}
896
897If the class had been loaded via a classloader other than the original
898the class you wish to specialize on, one needs to specify the
899classloader as an optional third argument.
900
901\begin{listing-lisp}
902
903(defparameter *other-classloader*
904  (jcall "getBaseLoader" cl-user::*classpath-manager*))
905 
906(defmethod print-object ((device-id (java:jclass "dto.nbi.service.hdm.alcatel.com.NBIDeviceID" *other-classloader*))
907                         stream)
908  ;;; ...
909)
910\end{listing-lisp}
911
912\section{Extensions to the Reader}
913
914We implement a special hexadecimal escape sequence for specifying
915characters to the Lisp reader, namely we allow a sequences of the form
916\# \textbackslash Uxxxx to be processed by the reader as character
917whose code is specified by the hexadecimal digits ``xxxx''.  The
918hexadecimal sequence must be exactly four digits long \footnote{This
919  represents a compromise with contemporary in 2011 32bit hosting
920  architecures for which we wish to make text processing efficient.
921  Should the User require more control over UNICODE processing we
922  recommend Edi Weisz' excellent work with FLEXI-STREAMS which we
923  fully support}, padded by leading zeros for values less than 0x1000.
924
925Note that this sequence is never output by the implementation.  Instead,
926the corresponding Unicode character is output for characters whose
927code is greater than 0x00ff.
928
929\subsection{JSS optionally extends the Reader}
930
931The JSS contrib consitutes an additional, optional extension to the
932reader in the definition of the \#\" reader macro.
933
934\section{ASDF}
935
936asdf-2.017.22 is packaged as core component of ABCL, but not
937intialized by default, as it relies on the CLOS subsystem which can
938take a bit of time to initialize.  It may be initialized by the ANSI
939\textsc{REQUIRE} mechanism as follows:
940
941\begin{listing-lisp}
942CL-USER> (require 'asdf)
943\end{listing-lisp}
944
945\chapter{Contrib}
946
947\section{abcl-asdf}
948
949This contrib to ABCL enables an additional syntax for ASDF system
950definition which dynamically loads JVM artifacts such as jar archives
951via a Maven encapsulation.  The Maven Aether can also be directly
952manipulated by the function associated with the RESOLVE-DEPENDENCIES symbol.
953
954%ABCL specific contributions to ASDF system definition mainly concerned
955%with finding JVM artifacts such as jar archives to be dynamically loaded.
956
957
958The following ASDF components are added: \textsc{JAR-FILE}, \textsc{JAR-DIRECTORY},
959\textsc{CLASS-FILE-DIRECTORY} and \textsc{MVN}.
960
961
962
963\subsection{ABCL-ASDF Examples}
964
965\begin{listing-lisp}
966    ;;;; -*- Mode: LISP -*-
967    (in-package :asdf)
968
969    (defsystem :log4j
970      :components ((:mvn "log4j/log4j"
971                    :version "1.4.9")))
972\end{listing-lisp}
973
974\subsection{abcl-asdf API}
975
976We define an API as consisting of the following ASDF classes:
977
978\textsc{JAR-DIRECTORY}, \textsc{JAR-FILE}, and
979\textsc{CLASS-FILE-DIRECTORY} for JVM artifacts that have a currently
980valid pathname representation
981
982And the MVN and IRI classes descend from ASDF-COMPONENT, but do not
983directly have a filesystem location.
984
985For use outside of ASDF, we currently define one method,
986\textsc{RESOLVE-DEPENDENCIES} which locates, downloads, caches, and then loads
987into the currently executing JVM process all recursive dependencies
988annotated in the Maven pom.xml graph.
989
990\subsection{ABCL-ASDF Example 2}
991
992Bypassing ASDF, one can directly issue requests for the Maven
993artifacts to be downloaded
994
995\begin{listing-lisp}
996    CL-USER> (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user")
997    WARNING: Using LATEST for unspecified version.
998    "/Users/evenson/.m2/repository/com/google/gwt/gwt-user/2.4.0-rc1/gwt-user-2.4.0-rc1.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/Users/evenson/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA-sources.jar"
999\end{listing-lisp}
1000
1001To actually load the dependency, use the JAVA:ADD-TO-CLASSPATH generic
1002function:
1003
1004\begin{listing-lisp}
1005    CL-USER> (java:add-to-classpath (abcl-asdf:resolve-dependencies "com.google.gwt" "gwt-user"))
1006\end{listing-lisp}
1007
1008Notice that all recursive dependencies have been located and installed
1009locally from the network as well.
1010
1011
1012\section{asdf-jar}
1013
1014ASDF-JAR provides a system for packaging ASDF systems into jar
1015archives for ABCL.  Given a running ABCL image with loadable ASDF
1016systems the code in this package will recursively package all the
1017required source and fasls in a jar archive.
1018
1019\section{jss}
1020
1021To one used to a syntax that can construct macros, the Java syntax
1022sucks, so we introduce the \#" macro.
1023
1024\subsection{JSS usage}
1025
1026Example:
1027
1028\begin{listing-lisp}
1029  CL-USER> (require 'jss)
1030
1031  CL-USER) (#"getProperties" 'java.lang.System)
1032
1033  CL-USER) (#"propertyNames" (#"getProperties" 'java.lang.System))
1034\end{listing-lisp}
1035
1036\section{asdf-install}
1037
1038An implementation of ASDF-INSTALL.  Superseded by Quicklisp (qv.)
1039
1040\chapter{History}
1041
1042ABCL was originally the extension language for the J editor, which was
1043started in 1998 by Peter Graves.  Sometime in 2003, a whole lot of
1044code that had previously not been released publically was suddenly
1045committed that enabled ABCL to be plausibly termed an emergent ANSI
1046Common Lisp implementation canidate.
1047
1048From 2006 to 2008, Peter manned the development lists, incorporating
1049patches as made sense.  After a suitable search, Peter nominated Erik
1050Huelsmann to take over the project.
1051
1052In 2008, the implementation was transferred to the current
1053maintainers, who have strived to improve its usability as a
1054contemporary Common Lisp implementation.
1055
1056On October 22, 2011, with the publication of this Manual explicitly
1057stating the conformance of Armed Bear Common Lisp to ANSI, we released
1058abcl-1.0.0.
1059
1060
1061
1062
1063\section{References}
1064
1065[Java2000]:  A New Era for Java Protocol Handlers.
1066\url{http://java.sun.com/developer/onlineTraining/protocolhandlers/}
1067
1068[Xach2011]:  Quicklisp:  A system for quickly constructing Common Lisp
1069libraries.  \url{http://www.quicklisp.org/}
1070
1071[RHODES2007]:  Christopher Rhodes
1072
1073
1074\end{document}
1075
1076% TODO
1077%   1.  Create mechanism for swigging DocString and Lisp docs into
1078%       sections ('grovel.lisp')
1079
Note: See TracBrowser for help on using the repository browser.