openj9 🚀 - JEP 393: Foreign-Memory Access API (Third Incubator)

@tajila

pshipton on 28 Oct 2020

This has been assigned to @babsingh

tajila on 28 Oct 2020

👍1

https://github.com/eclipse/openj9/pull/11182 introduces the following two native stubs:

Java_jdk_internal_misc_ScopedMemoryAccess_registerNatives: stays empty since our tooling handles the registration of Java native methods.
Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0: currently, throws an assertion; needs an implementation.

New tests and failures are seen for JEP393 (Foreign Memory Access API's Third Incubator):

[SUMMARY] Test results: passed: 20; failed: 4; error: 1

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java

More details about the failures can be found in the following log: jep393_tests_19Nov20.log.

Need to implement Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0 in order to resolve some of the above JEP393 failures.

babsingh on 20 Nov 2020

More JEP393 failures: https://github.com/eclipse/openj9/issues/11135#issuecomment-724814903.

babsingh on 20 Nov 2020

Note: following tests are excluded via https://github.com/AdoptOpenJDK/openjdk-tests/pull/2085

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java

JasonFengJ9 on 4 Dec 2020

More excludes for java/foreign tests via https://github.com/AdoptOpenJDK/openjdk-tests/pull/2088 before the implementation is ready.

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestAdaptVarHandles.java 
java/foreign/TestAddressHandle.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestMemoryAccess.java 
java/foreign/TestMemoryAccessStatics.java 
java/foreign/TestMemoryAlignment.java 
java/foreign/TestMemoryCopy.java 
java/foreign/TestMemoryHandleAsUnsigned.java 
java/foreign/TestMismatch.java 
java/foreign/TestNativeScope.java 
java/foreign/TestNulls.java 
java/foreign/TestRebase.java 
java/foreign/TestSegments.java 
java/foreign/TestSlices.java 
java/foreign/TestTypeAccess.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java 
java/foreign/TestVarHandleCombinators.java

fyi @tajila @babsingh @pshipton

JasonFengJ9 on 4 Dec 2020

More excludes for java/foreign tests via AdoptOpenJDK/openjdk-tests#2088

Do all the tests, which are being excluded, fail? As per the log file in https://github.com/eclipse/openj9/issues/11027#issuecomment-730849265 (generated 2 weeks ago), few of these tests pass.

babsingh on 4 Dec 2020

More excludes for java/foreign tests

Some of these tests are related to JEP389 rather than JEP393

tajila on 4 Dec 2020

Some of these tests are related to JEP389 rather than JEP393

JEP389 tests also run as part of the jdk_foreign test suite. I do not think it's possible to run JEP389 and JEP393 tests separately. So, I have identified the JEP389 tests below:

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java

babsingh on 4 Dec 2020

👍1

Do all the tests, which are being excluded, fail? As per the log file in #11027 (comment) (generated 2 weeks ago), few of these tests pass.

Yes, all excluded tests still fail in internal tests.

JasonFengJ9 on 4 Dec 2020

👍1

Some of these tests are related to JEP389 rather than JEP393

Will check and refer JEP 389 instead.

JasonFengJ9 on 4 Dec 2020

We need to track them differently as there is a chance that JEP393 will be ready for 0.25 and JEP389 will not

tajila on 4 Dec 2020

👍1

From an internal build Test_openjdknext_j9_sanity.openjdk_x86-64_linux_Nightly/363:

22:36:40  --------------------------------------------------
22:36:43  TEST: java/foreign/TestAdaptVarHandles.java

22:36:43  Caused by: java.lang.ClassCastException: java.lang.invoke.DirectHandle incompatible with java.lang.invoke.BoundMethodHandle
22:36:43    at java.base/java.lang.invoke.VarHandles.noCheckedExceptions(VarHandles.java:636)
22:36:43    at java.base/java.lang.invoke.VarHandles.filterCoordinates(VarHandles.java:476)
22:36:43    at java.base/java.lang.invoke.MethodHandleImpl$1.filterCoordinates(MethodHandleImpl.java:1790)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryHandles.filterCoordinates(MemoryHandles.java:374)
22:36:43    at jdk.incubator.foreign/jdk.internal.foreign.Utils.fixUpVarHandle(Utils.java:92)
22:36:43    at jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.dereferenceHandle(LayoutPath.java:159)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.lambda$varHandle$3(MemoryLayout.java:411)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout$$Lambda$55/0x0000000000000000.apply(Unknown Source)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.computePathOp(MemoryLayout.java:457)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.varHandle(MemoryLayout.java:411)
22:36:43    at TestAdaptVarHandles.<clinit>(TestAdaptVarHandles.java:90)
22:36:43    ... 30 more
22:36:43  

22:36:43  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestAdaptVarHandles: null

Similar exception as above
22:36:43  ===============================================
22:36:43  java/foreign/TestAddressHandle.java
22:36:43  Total tests run: 6, Failures: 4, Skips: 2
22:36:43  ===============================================

Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccess.java
22:37:09  Total tests run: 256, Failures: 252, Skips: 0
22:37:09  ===============================================

Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccessStatics.java
22:37:09  Total tests run: 50, Failures: 50, Skips: 0
22:37:09  ===============================================

Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryAlignment.java
22:37:13  Total tests run: 117, Failures: 63, Skips: 0
22:37:13  ===============================================

Similar exception as above
22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMemoryCopy.java
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMemoryCopy: null

Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryHandleAsUnsigned.java
22:37:13  Total tests run: 2360, Failures: 2360, Skips: 0
22:37:13  ===============================================



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:16  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNulls.java
Similar exception as above
22:37:16  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestNulls: null



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:16  --------------------------------------------------
22:37:20  TEST: java/foreign/TestRebase.java
Similar exception as above
22:37:20  TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 1



md5-22e22aa222b1944fcc8a62c793f8adf2



Similar exception as above
22:37:20  ===============================================
22:37:20  java/foreign/TestSegments.java
22:37:20  Total tests run: 120, Failures: 14, Skips: 1
22:37:20  ===============================================



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:20  --------------------------------------------------
22:37:23  TEST: java/foreign/TestSlices.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestSlices: null



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:23  --------------------------------------------------
22:37:23  TEST: java/foreign/TestTypeAccess.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestTypeAccess: null



md5-22e22aa222b1944fcc8a62c793f8adf2



Similar exception as above
22:37:26  ===============================================
22:37:26  java/foreign/TestVarHandleCombinators.java
22:37:26  Total tests run: 13, Failures: 6, Skips: 0
22:37:26  ===============================================



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMismatch.java
Similar exception as above
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMismatch: null



md5-22e22aa222b1944fcc8a62c793f8adf2



22:37:13  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNativeScope.java

Similar exception as above and ASSERTION failure:

22:37:16  STDERR:
22:37:16  WARNING: Using incubator modules: jdk.incubator.foreign
22:37:16  03:37:13.015 0x1c6800   j9jcl.347    *   ** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))
22:37:16  JVMDUMP039I Processing dump event "traceassert", detail "" at 2020/12/02 19:37:13 - please wait.

JasonFengJ9 on 4 Dec 2020

👍1

Re: https://github.com/eclipse/openj9/issues/11027#issuecomment-738965440 and https://github.com/eclipse/openj9/issues/11027#issuecomment-738929992

Locally, some of the jdk/foreign tests pass for me because I have the following fix applied to my JDK16 extensions repo: https://github.com/ibmruntimes/openj9-openjdk-jdk/pull/248. This fix adds support for OpenJ9 VarHandles but at the same time, it breaks support for OpenJDK VarHandles. This fix will not be merged since we plan to use OpenJDK VarHandles in JDK16. Related: https://github.com/eclipse/openj9/issues/11135#issuecomment-724814903. For the nightly openjdk sanity builds, @JasonFengJ9's failures are correctly documented.

babsingh on 4 Dec 2020

👍1

Below is the updated list of JEP393 test failures after https://github.com/ibmruntimes/openj9-openjdk-jdk16/pull/9 is merged for the 0.25 release:

java/foreign/TestByteBuffer.java
java/foreign/TestSharedAccess.java
java/foreign/TestSpliterator.java
java/foreign/TestHandshake.java
java/foreign/TestCleaner.java
java/foreign/TestNativeScope.java

The above tests fail because of

** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))

The above assertion failure is being tracked via this issue https://github.com/eclipse/openj9/issues/11027#issuecomment-730849265 and https://github.com/eclipse/openj9/issues/11360 (duplicate).

babsingh on 21 Jan 2021

Openj9: JEP 393: Foreign-Memory Access API (Third Incubator)

All 15 comments

Related issues