Opened 9 months ago

Last modified 2 months ago

#422 accepted defect

SYSTEM:RUN-PROGRAM does not work on Java 6

Reported by: mevenson Owned by: mevenson
Priority: blocker Milestone: 1.6.0
Component: interpreter Version: 1.4.0
Keywords: cffi sys:run-program java-5 java-6 Cc: epipping
Parent Tickets:

Description

In chasing down the errors with CFFI on CL-TEST-GRID <https://mailman.common-lisp.net/pipermail/armedbear-devel/2016-October/003719.html>, I have found that the [java.lang.ProcessBuilder?$Redirect][] interface used by Elias and Olof to extend SYS:RUN-PROGRAM for different types of I/O abstractions was introduced with Java 7, and will hence not work on earlier Java implementations.

[java.lang.ProcessBuilder?$Redirect]: https://docs.oracle.com/javase/8/docs/api/java/lang/ProcessBuilder.Redirect.html

Invoking ABCL-ASDF:ENSURE-MVN-VERSION, the following form causes the error

(JFIELD "java.lang.ProcessBuilder$Redirect" "INHERIT")  

TODO: investigate the Java 6 APIs to see if there is a way to do I/O redirection with backwards compatibility. I currently suspect that there is no way to support Java 5/6 for this usage, which is why we never implemented I/O redirection previously.

There is undoubtedly a way re-write the SYS:RUN-PROGRAM interface so that we may invoke a process to read its output as a string in Java 5/Java 6, as it worked before. But we will have to figure out a way to advertise the different features of SYS:RUN-PROGRAM depending on the hosting VM.

Longer term, we may want to deprecate Java 5/6, but I would really have the compiler emit Java 7-compatible bytecode (mainly by passing the Java 6 verification process) before we begin that.

Subtickets (add)

Attachments (2)

run-program-20161120a.diff (1.3 KB) - added by mevenson 9 months ago.
abcl-run-program-20161129a.diff (5.0 KB) - added by mevenson 9 months ago.

Download all attachments as: .zip

Change History (13)

comment:1 Changed 9 months ago by mevenson

Current thinking on resolution of issue:

  1. Un-deprecate SYS:RUN-SHELL-COMMAND. Use the pre Java 7 APIs as best we can to support invoking programs.
  1. Re-code the SYS:RUN-PROGRAM Java callsite linkages so that ABCL may be compiled on pre-Java7 JDKs. Produce an intelligible error if it is invoked on a pre-Java7 JVM.
  1. Create an API to determine runtime JVM version. Wrap UIOP/RUN-PROGRAM invocation in some sort of handler that will "fall back" to using SYS:RUN-SHELL-COMMAND if it has compatible behavior via specified args. This will potentially be fairly ugly code ASDF-side so we might provide a shim that UIOP/RUN-PROGRAM invokes that contains the logic outside the ASDF code base.

Changed 9 months ago by mevenson

comment:2 Changed 9 months ago by mevenson

The attached patch restores compilation under JDK6.

comment:3 Changed 9 months ago by mevenson

After further analysis, the new plan of attack:

  1. Continue to deprecate SYS:RUN-SHELL-COMMAND, as the JVM-[56] actually seem to provide all the behavior we need.
  1. Fully support the current signature of the abcl-1.5.0-dev SYS:RUN-PROGRAM as extended by Pipping and Olof. The environment clearing/setting will work without alterations. "All that is needed" is to create an org.abcl.util.AsyncIOManager using the non-blocking file descriptor framework in java.jnio (introduced in JVM5) to drain/fill input/output org.armedbear.lisp.Stream instances. Pipping has collected a thorough test suite for the SYS:RUN-PROGRAM behavior, which will be of great use in implementation.

The use of SYS:RUN-PROGRAM as invoked by UIOP/RUN-PROGRAM as the implementation interface should mean that ASDF should "just work" when the new code passes Pipping's test suite.

TODO: transcribe details of SYS:RUN-PROGAM test suite invocation; implement ASyncIOManager.

Such work is intended for an abcl-1.4.1 release by the end of November 2016.

Last edited 9 months ago by mevenson (previous) (diff)

comment:4 Changed 9 months ago by mevenson

  • Milestone changed from 1.5.0 to 1.4.1
  • Owner set to mevenson
  • Priority changed from major to blocker
  • Status changed from new to assigned

comment:5 Changed 9 months ago by epipping

  • Cc epipping added
  • Keywords sys:run-program added; sys:run-progrom removed

comment:6 Changed 9 months ago by mevenson

When using development versions of ABCL, [ASDF needs to be patched][1].

[1]: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/57

Changed 9 months ago by mevenson

comment:7 follow-up: Changed 5 months ago by mevenson

  • Status changed from assigned to accepted

Plan of attack:

1) Use Java interfaces in org.abcl.java[5678] packages to mark JVM-side availablity until we get to the Java9 module system.

2) Keep SYS:RUN-SHELL-COMMAND deprecated Lisp-side; mody SYS:RUN-PROGRAM to fall back to the appropriate org.abcl.java5 interface capabilities signaling an exception if unable to emulate requested behavior

3) Test!

comment:8 Changed 5 months ago by mevenson

  • Milestone changed from 1.4.1 to 1.5.0

Ticket retargeted after milestone closed

comment:9 in reply to: ↑ 7 Changed 5 months ago by mevenson

Replying to mevenson:

Plan of attack:

1) Use Java interfaces in org.abcl.java[5678] packages to mark JVM-side availablity until we get to the Java9 module system.

2) Keep SYS:RUN-SHELL-COMMAND deprecated Lisp-side; mody SYS:RUN-PROGRAM to fall back to the appropriate org.abcl.java5 interface capabilities signaling an exception if unable to emulate requested behavior

3) Test!

Provide the following interfaces

org.abcl.runtime.Java5
org.abcl.runtime.Java6
org.abcl.runtime.Java7
org.abcl.runtime.Java8

for which

org.abcl.runtime.Available.getInstance()

returns something meaningful.

comment:10 Changed 2 months ago by mevenson

  • Milestone changed from 1.5.0 to 1.6.0

Ticket retargeted after milestone closed

comment:11 Changed 2 months ago by mevenson

  • Summary changed from SYSTEM:RUN-PROGRAM does not work on Java 5/6 to SYSTEM:RUN-PROGRAM does not work on Java 6
Note: See TracTickets for help on using tickets.