ABCL problems with cl-test-grid

[Avodonosov]

I encountered crash of ABCL 1.0.1 when running tests of many libraries.

(Some work may be done to isolate the problem by experimenting with the set of libraries used, but I didn't find time for this during the last 2 month, so decided to just submit an exact way to reproduce the bug).

I supposed it may be a memory issue, and increased memory by JVM options (as shown below), but it doesn't help.

To ensure the problem is reproducible I've created a branch of my project and create a script which load particular version of quicklisp
(quicklisp is bootstrapped in local directory to avoid interference with the quickips installation user may have on his computer).

The steps to reproduce:

git clone ​git://github.com/cl-test-grid/cl-test-grid.git
cd cl-test-grid
git checkout abcl-crash-reproduce
java -Xmx2048M -XX:ThreadStackSize=12048 -XX:MaxPermSize=256m -jar C:\Users\anton\unpacked\abcl\abcl-bin-1.0.1\abcl.jar --noinit --nosystem --batch --load crash-abcl.lisp > crash-abcl.log 2>&1

[ ... lot of output, and after ~20 mins ABCL crashed ]

crash-abcl.log is attached. The stacktrace of interest is at the bottom.

[Avodonosov]

log of the ABCL crash (the relevant stacktras is at the bottom)

[Avodonosov]

My platform: Windows 7, 64 bit.

java -version

java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot?(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

But other users also reported this crash, so I assume the platform is not important.

[Avodonosov]

BTW, this crash prevented me to collect ABCL test results for cl-test-grid last month, but I am now developing a version which will run test suites in separate lisp (ABCL) processes, so this issue is not blocker for me anymore.

[ehuelsmann]

The trace shows 2 crashes. One in the compilation of Alexandria-TESTS. This one has been explicitly fixed.

The other one seems to be related to the test suite(s) belonging to cl-containers. I've compiled and run the test suite of cl-containers, but I haven't seen an issue:

CL-USER(6): (cl-containers-test::run-tests)
Start: TEST-DLIST-CONTAINER
#<Results for TEST-DLIST-CONTAINER [1 Successful test]>

mevenson: can you reproduce? If not, can we close as fixed?

[Avodonosov]

Yes, the first crash is a separate ticket and is unrelated.

I have tried separate cl-containers before submitting the ticket,
and the same as you haven't reproduced it.

The crash happened only during testing, which involved loading of all the libraries and their test suites into a single ABCL process. Use the steps in the ticket description, I had 100% reproduction rate with them.

Note, this issue is NOT a blocker for cl-test-grid anymore, as now every
test suite is run in a separate process and we collect test results for ABCL too.

[ehuelsmann]

Not a blocker for cl-test-grid, so it shouldn't be a blocker for 1.1.0.

Could very well be that the JVM has run out of space to store classes, since so many systems were loaded, thousands (possibly 100s of thousands) of classes got loaded. It would be interesting to get back to the old behaviour and see if changing some of the default JVM settings to much larger values would shift the crash location.

[Avodonosov]

I experimented with increasing memory for heap, thread stack, and perm gen (where classes are stored). The values shown in the steps to reproduce didn't help to prevent crash.

[Avodonosov]

Yes, and the exception in the output stream confirms it is something different than class storage - it is usually signaled with "java.lang.OutOfMemoryError?: PermGen? space", while we have some NullPointer?.

[Mark Evenson]

Need to tune local JVM properties to increase the maximum used for "permanent" objects (like JVM class definitions).

When building from source copying abcl.properties.in

    cmd$ cp abcl.properties.in abcl.properties 

sets the following switches: -d64 -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=1g .