Openj9: Exception testing Apache Lucene with OpenJ9

[r30shah]

While testing Apache Lucene on Linux on IBM Z with OpenJDK9 both with hot spot and openJ9 I ran in to following exception with OpenJ9(There are total 475 test suites with total 3831 - All had following exception) .

[junit4] Suite: org.apache.lucene.index.TestIndexInput
   [junit4] ERROR   0.00s J2 | TestIndexInput.initializationError <<<
   [junit4]    > Throwable #1: java.lang.NoClassDefFoundError: com.carrotsearch.randomizedtesting.RandomizedRunner (initialization failure)
   [junit4]    >    at java.lang.J9VMInternals.initializationAlreadyFailed(java.base@9-internal/J9VMInternals.java:141)
   [junit4]    >    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(java.base@9-internal/Native Method)
   [junit4]    >    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(java.base@9-internal/NativeConstructorAccessorImpl.java:62)
   [junit4]    >    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(java.base@9-internal/DelegatingConstructorAccessorImpl.java:45)
   [junit4]    >    at java.lang.reflect.Constructor.newInstance(java.base@9-internal/Constructor.java:488)
   [junit4]    > Caused by: java.security.AccessControlException: Access denied ("java.util.logging.LoggingPermission" "control")
   [junit4]    >    at java.security.AccessControlContext.checkPermissionWithCache(java.base@9-internal/AccessControlContext.java:645)
   [junit4]    >    at java.security.AccessController.checkPermissionHelper(java.base@9-internal/AccessController.java:241)
   [junit4]    >    at java.security.AccessController.checkPermission(java.base@9-internal/AccessController.java:383)
   [junit4]    >    at java.lang.SecurityManager.checkPermission(java.base@9-internal/SecurityManager.java:558)
   [junit4]    >    at sun.util.logging.internal.LoggingProviderImpl.setLogManagerAccess(java.logging@9-internal/LoggingProviderImpl.java:460)
   [junit4]    >    at java.util.logging.LogManager$LoggingProviderAccess.run(java.logging@9-internal/LogManager.java:2691)
   [junit4]    >    at java.util.logging.LogManager$LoggingProviderAccess.run(java.logging@9-internal/LogManager.java:2641)
   [junit4]    >    at java.security.AccessController.doPrivileged(java.base@9-internal/AccessController.java:833)
   [junit4]    >    at java.util.logging.LogManager.<clinit>(java.logging@9-internal/LogManager.java:2699)
   [junit4]    >    at java.util.logging.Logger.demandLogger(java.logging@9-internal/Logger.java:648)
   [junit4]    >    at java.util.logging.Logger.getLogger(java.logging@9-internal/Logger.java:717)
   [junit4]    >    at java.util.logging.Logger.getLogger(java.logging@9-internal/Logger.java:701)
   [junit4]    >    ... 13 more

Which I did not hit with OpenJDK9 with HotSpot and yield to following output at the end of test,

[junit4] Tests summary: 475 suites (1 ignored), 3831 tests, 2 failures, 87 ignored (79 assumptions)

Steps to build and test lucene,

$ git clone https://github.com/apache/lucene-solr.git
$ cd lucene-solr/lucene
$ export LUCENE_HOME=`pwd`
$ cd -

Make sure you have ant installed, If you want to manually install ant, you can do it by following these steps

$ wget http://httpd-mirror.sergal.org/apache//ant/binaries/apache-ant-1.9.9-bin.zip
$ unzip apache-ant-1.9.9-bin.zip
$ cd apache-ant-1.9.9
$ export ANT_HOME=`pwd`
$ export PATH=$ANT_HOME/bin:$PATH

Make sure you have java installed.

To build lucene you need to first set ivy bootstrap path,

$ cd $LUCENE_HOME
$ ant ivy-bootstrap
$ echo "rpm_mode=false" >> ~/.ant/ant.conf
$ ant compile

To test lucene

$ cd $LUCENE_HOME
$ ant test

To pass any option to JVM in ant use ANT_OPTS, e.g export ANT_OPTS="-Xms5G -Xmx5G"

[pshipton]

Thanks for reporting the problem. This is most likely a duplicate of https://github.com/eclipse/openj9/pull/488. Once this fix is finalized and merged we can see if this one is also resolved.

[pshipton]

@r30shah #488 is delivered. Can you please check if the problem is resolved. You can build a new OpenJ9 runtime or download a nightly build from https://adoptopenjdk.net/nightly.html?variant=openjdk9-openj9 from 17 November 2017.

[r30shah]

@pshipton I tried latest build (20th November) from the https://adoptopenjdk.net/nightly.html?variant=openjdk9-openj9 .
It still fails , but this time with different error,

[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexInput -Dtests.seed=ADF87034DAFC05F4 -Dtests.slow=true -Dtests.asserts=false -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestIndexInput (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Assertions mismatch: -ea was not specified but -Dtests.asserts=true
   [junit4]    >    at __randomizedtesting.SeedInfo.seed([ADF87034DAFC05F4]:0)
   [junit4]    >    at java.lang.Thread.run(java.base@9-internal/Thread.java:835)
   [junit4] Completed [69/475 (65!)] on J0 in 0.01s, 0 tests, 1 error <<< FAILURES!

[pshipton]

Based on the error message, I looked at the code which enables the -ea option, and found a bug in ClassLoader.initializeClassLoaderAssertStatus() related to the following line. This test doesn't work for Java 9. Opened #633 to fix it.

boolean bootLoader = bootstrapClassLoader == null;

[pshipton]

633 was merged a week ago, @r30shah can you please retry.

[r30shah]

@pshipton : I tried driver from 28th November and it seems like issue has been resolved. I got all the test running. There were 3 failures which is one more than what I got with OpenJDK9 with hot spot. Going to check if it is new issue in both Java or something with OpenJ9. Will open up separate issue for that. For now this one seems to be resolved now.

Java version

rahil30@torlinxj:/sandbox1/rahil/lucene> sdk/bin/java -version
openjdk version "9-internal"
OpenJDK Runtime Environment (build 9-internal+0-adhoc.jenkins.openjdk)
Eclipse OpenJ9 VM (build 2.9, JRE 9 Linux s390x-64 Compressed References 20171128_59 (JIT enabled, AOT enabled)
OpenJ9   - ea57db3
OMR      - 92e9349a
OpenJDK  - e180b84d8 based on )

Test Output

/sandbox1/rahil/lucene/lucene-solr/lucene/common-build.xml:1038: There were test failures: 475 suites (1 ignored), 3831 tests, 3 failures, 87 ignored (79 assumptions) [seed: 56545650D4D60353]

[pshipton]

Awesome. You should be able to close this Issue if the problems are resolved.