Defining Java classes in Lisp (aka runtime-class)

Rationale

Contrary to most other Lisp implementation, ABCL is deeply intertwined with its hosting platform, the JVM. It is an important design goal for ABCL not only to coexist with Java and other languages, but to actually interoperate with them at a deep level. This includes providing components to be consumed by foreign libraries, such as callbacks, event listeners, configuration objects, et cetera.

Generally, when possible, it's advisable to use interfaces rather than concrete classes for that purpose (see Implementing Java interfaces in Lisp). However, not all foreign libraries are properly designed; and sometimes a concrete class is needed for valid reasons (e.g. to inherit important functionality). The "runtime-class" functionality presented here addresses those use cases.

History

ABCL used to have a form of runtime-class functionality in its early days. It was implemented using the popular ASM library. At one point, though, it was disabled in order to eliminate the need for external dependencies. It stayed disabled for quite a long time - enough for getting affected by bit rot with a high probability.

Between late 2011 and early 2012, part of the functionality was resurrected by Alessio Stalla, this time completely in Lisp with no external dependencies. This was made possible by ABCL's class file writer, an internal library coded some time earlier by Erik Huelsmann to better separate concerns in the compiler. The class file writer was slightly extended in order to make it generate Java 1.5 annotation metadata.

The API has been inspired by the original, but has been changed in some places.

Terminology and definitions

Before digging into the API, a few terms must be defined for clarity:

• access flag: a keyword understood by the Java class file writer as a modifier for a class, field, or method. Examples: :public, :private, :static.
• Java type designator: either a string naming a Java class, or a keyword corresponding to a primitive type such as :int, :double, :boolean etc., or (only when indicated) the keyword :void.

Main API entry points

The define-java-class macro

This functionality is not yet implemented. The java:define-java-class macro is the high-level entry point to the runtime-class functionality. It takes a specification of a Java class and expands into code for generating such a class at runtime. If the macro is expanded as part of file compilation, it (optionally?) generates the class at compile-time, dumps it to a file, and expands into code for loading the class from that file.

The jnew-runtime-class function

The java:jnew-runtime-class is the lower-level entry point to the runtime-class functionality. It takes a specification of a Java class and generates the bytecode for it, optionally saving it to a file (not yet implemented) and (not yet optionally) loading it into the current JVM.

Dissecting a Java class definition

In the context of runtime-class, a class definition is a series of arguments that can (or must) be passed to either define-java-class or jnew-runtime-class. They are detailed below.

• class-name - mandatory argument. A string that must be a valid Java class name according to the Java Language Specification, specifying the name of the defined class.
• superclass - keyword argument; defaults to "java.lang.Object". A string that must be a valid Java class name according to the Java Language Specification, specifying the superclass of the class being defined.
• interfaces - keyword argument. A (possibly empty) list of strings naming Java interfaces to be implemented by the defined class.
• methods - keyword argument. A (possibly empty) list of method definitions (see the appropriate section below).
• fields - keyword argument. A (possibly empty) list of field definitions (see the appropriate section below).
• access-flags - keyword argument; defaults to (:public). A (possibly empty) list of access flags as accepted by the Java class file writer, which will be applied to the defined class.
• annotations - keyword argument. A (possibly empty) list of annotation definitions (see the appropriate section below) to be applied to the defined class.

Method definitions

Arguably, the most important feature of runtime-class is to define Java methods in Lisp. Each method is implemented as a call to a pre-existing Lisp function, with the necessary conversions for arguments and return values. Contrarily to the old implementation of runtime-class, no API is specified to replace method implementations at runtime; instead, you can leverage the usual mechanism of passing a symbol as a function designator and later replacing its symbol-function.

A method definition is a list in the following form:

(method-name return-type argument-types function &key modifiers annotations override)

• method-name is self-explicative.
• return-type - the Java type designator (including :void) for the return type of the method. Note: currently many primitive types are not supported.
• argument-types - a list of Java type designators for the argument types of the method. Note: currently many primitive types are not supported.
• function - a Lisp function designator. The function must have (1+ (length argument-types)) parameters: the first is the receiving object (what would be the this reference in Java), the others are the method arguments, opportunely converted to Lisp values. In the body of the function, you can call other methods on the same object with the usual jcall operator. Note: the calling convention for functions with > 8 arguments is currently not supported.
• modifiers - a list of modifier keywords for the method. The default is (:public).
• annotations - a (possibly empty) list of annotation definitions (see the appropriate section below) to be placed on the method.
• override - defines the override policy (see below). By default it's nil.

Overriding superclass methods

There's no special action to be performed to make a method override another; if your class contains a method with a signature compatible to some other method defined in one of its superclasses, your method will override the other. However, you may want to call the overridden method (e.g. to provide a fallback case). How that happens is controlled by the override keyword argument.

• if override is nil (the default), you cannot call the superclass method.
• if override is non-nil, a private method called super$<the name of your method> will be defined, with the following behaviour:
  • if it is T, then the super$... method will call a method with the exact same signature as yours and defined in the direct superclass. Note: a smarter implementation could be provided that, when possible, would search in all the superclasses for the best match, according to the rules in the JLS, but it is not implemented at the moment.
  • (not yet implemented) else, it is assumed to be a list in the form (class-name argument-types). A call to a method with the same name, with the provided signature and defined in the specified superclass, will be generated.

Note: what happens with the super$name convention if you're overriding two overloads of the same method? A naming strategy must be defined, or the user must be given the possibility to specify another name instead of super$name.

Field definitions

Not yet implemented.

Constructors

(not yet implemented) The policy is to define a matching constructor for each constructor in the superclass. A constructor cannot contain direct calls to Lisp code because of the restrictions on the bytecode that can be present in the body of a constructor. However, you can optionally (how?) specify an "init method" to be invoked by each constructor after calling the corresponding superclass constructor.

The feature in the old runtime-class implementation to choose how to pass parameters to parent constructors has been deemed not sufficiently useful to warrant reimplementation.

Annotation definitions

An annotation definition is either:

• a list (annotation-type &rest args), or
• a string annotation-type, which is equivalent to the list (annotation-type).

Each argument is in the form (arg-name &key value enum). arg-name can be omitted, in that case the name of the argument is "value".

value can be:

• an immediate value (of type boolean, fixnum, float, or string)
• a list of values, for arguments of an array type
• if enum is not nil, value is a string naming one of the enum constants. enum must then name a Java enumeration class.