SYSTEM:RUN-PROGRAM does not work on Java 6

[Mark Evenson]

In chasing down the errors with CFFI on CL-TEST-GRID <​https://mailman.common-lisp.net/pipermail/armedbear-devel/2016-October/003719.html>, I have found that the [java.lang.ProcessBuilder?$Redirect][] interface used by Elias and Olof to extend SYS:RUN-PROGRAM for different types of I/O abstractions was introduced with Java 7, and will hence not work on earlier Java implementations.

[java.lang.ProcessBuilder?$Redirect]: ​https://docs.oracle.com/javase/8/docs/api/java/lang/ProcessBuilder.Redirect.html

Invoking ABCL-ASDF:ENSURE-MVN-VERSION, the following form causes the error

(JFIELD "java.lang.ProcessBuilder$Redirect" "INHERIT")  

TODO: investigate the Java 6 APIs to see if there is a way to do I/O redirection with backwards compatibility. I currently suspect that there is no way to support Java 5/6 for this usage, which is why we never implemented I/O redirection previously.

There is undoubtedly a way re-write the SYS:RUN-PROGRAM interface so that we may invoke a process to read its output as a string in Java 5/Java 6, as it worked before. But we will have to figure out a way to advertise the different features of SYS:RUN-PROGRAM depending on the hosting VM.

Longer term, we may want to deprecate Java 5/6, but I would really have the compiler emit Java 7-compatible bytecode (mainly by passing the Java 6 verification process) before we begin that.

[Mark Evenson]

Current thinking on resolution of issue:

1. Un-deprecate SYS:RUN-SHELL-COMMAND. Use the pre Java 7 APIs as best we can to support invoking programs.

2. Re-code the SYS:RUN-PROGRAM Java callsite linkages so that ABCL may be compiled on pre-Java7 JDKs. Produce an intelligible error if it is invoked on a pre-Java7 JVM.

3. Create an API to determine runtime JVM version. Wrap UIOP/RUN-PROGRAM invocation in some sort of handler that will "fall back" to using SYS:RUN-SHELL-COMMAND if it has compatible behavior via specified args. This will potentially be fairly ugly code ASDF-side so we might provide a shim that UIOP/RUN-PROGRAM invokes that contains the logic outside the ASDF code base.

[Mark Evenson]

The attached patch restores compilation under JDK6.

[Mark Evenson]

After further analysis, the new plan of attack:

1. Continue to deprecate SYS:RUN-SHELL-COMMAND, as the JVM-[56] actually seem to provide all the behavior we need.

2. Fully support the current signature of the abcl-1.5.0-dev SYS:RUN-PROGRAM as extended by Pipping and Olof. The environment clearing/setting will work without alterations. "All that is needed" is to create an org.abcl.util.AsyncIOManager using the non-blocking file descriptor framework in java.jnio (introduced in JVM5) to drain/fill input/output org.armedbear.lisp.Stream instances. Pipping has collected a thorough test suite for the SYS:RUN-PROGRAM behavior, which will be of great use in implementation.

The use of SYS:RUN-PROGRAM as invoked by UIOP/RUN-PROGRAM as the implementation interface should mean that ASDF should "just work" when the new code passes Pipping's test suite.

TODO: transcribe details of SYS:RUN-PROGAM test suite invocation; implement ASyncIOManager.

Such work is intended for an abcl-1.4.1 release by the end of November 2016.

[Mark Evenson]

When using development versions of ABCL, [ASDF needs to be patched][1].

[1]: ​https://gitlab.common-lisp.net/asdf/asdf/merge_requests/57

[Mark Evenson]

Plan of attack:

1) Use Java interfaces in org.abcl.java[5678] packages to mark JVM-side availablity until we get to the Java9 module system.

2) Keep SYS:RUN-SHELL-COMMAND deprecated Lisp-side; mody SYS:RUN-PROGRAM to fall back to the appropriate org.abcl.java5 interface capabilities signaling an exception if unable to emulate requested behavior

3) Test!

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Replying to mevenson:

Plan of attack:

1) Use Java interfaces in org.abcl.java[5678] packages to mark JVM-side availablity until we get to the Java9 module system.

2) Keep SYS:RUN-SHELL-COMMAND deprecated Lisp-side; mody SYS:RUN-PROGRAM to fall back to the appropriate org.abcl.java5 interface capabilities signaling an exception if unable to emulate requested behavior

3) Test!

Provide the following interfaces

org.abcl.runtime.Java5
org.abcl.runtime.Java6
org.abcl.runtime.Java7
org.abcl.runtime.Java8

for which

org.abcl.runtime.Available.getInstance()

returns something meaningful.

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Milestone renamed

[Mark Evenson]

Ticket retargeted after milestone closed

[Mark Evenson]

Support for openjdk6 no longer in scope. Patches would be accepted.