Loading compiled functions

ABCL compiles functions into Java classes, implementing the appropriate overloads of the execute() method.

Compiled functions are loaded with the Lisp function LOAD-COMPILED-FUNCTION, which relies on the Java method org.armedbear.lisp.Lisp.loadCompiledFunction which has several overloads. These methods can read the bytecode representing the compiled function from various sources, and ultimately turn it into a live Java (and Lisp) object.

A custom classloader is used to convert the bytecode (in the form of a byte array) into a Java class.

Performance considerations

To be used, a Java class must be instantiated, and compiled Lisp functions are no different.
Currently, ABCL uses java.lang.Class.newInstance() to create a singleton object from the compiled function class.
This is heavy performance-wise since it uses reflection; Erik Huelsmann profiled ABCL to discover that 40% of its startup time is spent in the native methods that look for the appropriate constructor to be invoked for each compiled function class.
Several alternatives were discussed, but none proved satisfactory. I'm reporting them here for further reference.

• Using a static method instead of a constructor. This is equivalent to using a constructor, since a constructor is just a special kind of method and the same performance penalty from reflection applies in both cases.
• Avoiding reflection altogether. From a test of mine (Alessio Stalla) using X.class.newInstance() is around 100 times slower than new X() - if the class X is only used once; since constructors are cached, calling newInstance() many times incurs in a much lower performance hit. However, since for ABCL X is the class being loaded as a compiled function, no bytecode can reference X at load time except the one describing X itself. We thought about using static initialization to create an instance of X and storing somewhere - something along the lines of

  //Hypothetical Java source of the class generated by the compiler
  public class X extends Function {
      static {
        Lisp.remember(X.class.getName(), new X());
      }
      ...
  }
  

  then, after loading the class, ABCL could fetch the function instance with

  Lisp.recall(loadedClass.getName());
  

  Unfortunately, there's a catch: the static initialization block (implemented in the method <clinit> at the bytecode level) is only executed when the class is initialized, and that happens only just before an instance of the class is created, or one of its static members is accessed (​http://java.sun.com/docs/books/jvms/second_edition/html/Concepts.doc.html#19075) - but to do one of those things on an unknown class you need reflection. So it appears reflection can not be avoided.
• Class.forName(className) should initialize the class too, and thus execute its static initialization code. From another test of mine, forName() is around 10 times faster than newInstance(). However, Erik profiled it on ABCL and it seems this technique is as slow as using newInstance(), presumably because class initialization time for complex classes as ABCL's is the dominant performance bottleneck.