Openj9: Test-extended.functional-JDK8-linux cmdLineTester_jep178_staticLinking_SE80_0 crash libdl.so

Created on 14 Nov 2018 · 36Comments · Source: eclipse/openj9

Note the test has just been enabled via https://github.com/eclipse/openj9/pull/3626. Gathered the following, but it is likely a pre-existing issue.

Crashing build

02:10:09 Eclipse OpenJ9 VM (build master-7aa1014, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20181114_645 (JIT enabled, AOT enabled)
02:10:09 OpenJ9   - 7aa1014
02:10:09 OMR      - fdc5695
02:10:09 JCL      - 4fb7da1 based on jdk8u192-b12)

Previous build

03:31:45 Eclipse OpenJ9 VM (build master-1a63acd, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20181113_637 (JIT enabled, AOT enabled)
03:31:45 OpenJ9   - 1a63acd
03:31:45 OMR      - e706e5c
03:31:45 JCL      - 4fb7da1 based on jdk8u192-b12)

1a63acd...7aa1014
https://github.com/eclipse/omr/compare/e706e5c...fdc5695

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/149

Testing: jep178 testing dynamic linking
Test start time: 2018/11/14 02:08:13 Eastern Standard Time
Running command: /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_dynamic -jvmpath:/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/s390x/j9vm -classpath:/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xcompressedrefs  com/ibm/j9/tests/jeptests/StaticLinking
Time spent starting: 3 milliseconds
Time spent executing: 743 milliseconds
Test result: FAILED
Output from test:
 [OUT] [MSG] Starting up JEP 178 test.
 [OUT] [MSG] Opening jvm from /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/s390x/j9vm/libjvm.so
 [ERR] Unhandled exception
 [ERR] Type=Segmentation error vmState=0x00000000
 [ERR] J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=007dbbd0 Signal_Code=00000001
 [ERR] Handler1=000003FF8BEAEC70 Handler2=000003FF8BAAAD30 InaccessibleAddress=0000000000000000
 [ERR] gpr0=0000000000000000 gpr1=000003FF8C181CD8 gpr2=0000000000000000 gpr3=000003FF38000020
 [ERR] gpr4=000003FF8C1815FE gpr5=00000000FFFFFFFF gpr6=000003FF38004660 gpr7=00000000A7F4BFA8
 [ERR] gpr8=000003FF8C160FC8 gpr9=000003FF8B47ABA7 gpr10=000003FF3800467F gpr11=000003FF8C181CE6
 [ERR] gpr12=0000000000000000 gpr13=000003FF8C184118 gpr14=000003FF8C1815FE gpr15=000003FF8B47A568
 [ERR] psw=000003FF8C181614 mask=0705100180000000 fpc=0008000000000000 bea=000003FF8C2583B2
 [ERR] fpr0 000003ffca0ff9ed (f: 3390044672.000000, d: 2.172477e-311)
 [ERR] fpr1 000003ff8b47aba7 (f: 2336730112.000000, d: 2.171956e-311)
 [ERR] fpr2 00000000a7e9f2e0 (f: 2817127168.000000, d: 1.391846e-314)
 [ERR] fpr3 000003ff8c181078 (f: 2350387200.000000, d: 2.171963e-311)
 [ERR] fpr4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] fpr5 000003ff8b47a248 (f: 2336727552.000000, d: 2.171956e-311)
 [ERR] fpr6 ffffffff90000002 (f: 2415919104.000000, d: -nan)
 [ERR] fpr7 3c5d000000000000 (f: 0.000000, d: 6.288373e-18)
 [ERR] fpr8 0000000000040000 (f: 262144.000000, d: 1.295163e-318)
 [ERR] fpr9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] fpr10 000003ff8b440000 (f: 2336489472.000000, d: 2.171956e-311)
 [ERR] fpr11 000003fff8aff45b (f: 4172280832.000000, d: 2.172863e-311)
 [ERR] fpr12 000003ffc897c20c (f: 3365388800.000000, d: 2.172464e-311)
 [ERR] fpr13 000003ffe337cc08 (f: 3812084736.000000, d: 2.172685e-311)
 [ERR] fpr14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] fpr15 000003ffcfe7f1b8 (f: 3488084480.000000, d: 2.172525e-311)
 [ERR] Module=/lib/s390x-linux-gnu/libdl.so.2
 [ERR] Module_base_address=000003FF8C180000 Symbol=dlerror
 [ERR] Symbol_address=000003FF8C1814D8
 [ERR] Target=2_90_20181114_878 (Linux 4.4.0-128-generic)
 [ERR] CPU=s390x (4 logical CPUs) (0x1e9a1c000 RAM)
 [ERR] ----------- Stack Backtrace -----------
 [ERR] dlerror+0x13c (0x000003FF8C181614 [libdl.so.2+0x1614])
 [ERR] (0x000003FF8BAAE84A [libj9prt29.so+0x2e84a])
 [ERR] (0x000003FF8BAAEDB6 [libj9prt29.so+0x2edb6])
 [ERR] (0x000003FF8BEE95E0 [libj9vm29.so+0xe95e0])
 [ERR] (0x000003FF8BEE9FEC [libj9vm29.so+0xe9fec])
 [ERR] (0x000003FF8BEEA266 [libj9vm29.so+0xea266])
 [ERR] (0x000003FF8BE20358 [libj9vm29.so+0x20358])
 [ERR] (0x000003FF8BEF2EE4 [libj9vm29.so+0xf2ee4])
 [ERR] ---------------------------------------
 [ERR] JVMDUMP039I Processing dump event "gpf", detail "" at 2018/11/14 02:08:13 - please wait.
 [ERR] JVMDUMP032I JVM requested System dump using '/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.020813.31154.0001.dmp' in response to an event
 [ERR] JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.31174.
 [ERR] 
 [ERR] JVMDUMP010I System dump written to /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.020813.31154.0001.dmp
 [ERR] JVMDUMP032I JVM requested Java dump using '/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.020813.31154.0002.txt' in response to an event
 [ERR] JVMDUMP010I Java dump written to /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.020813.31154.0002.txt
 [ERR] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.020813.31154.0003.trc' in response to an event
 [ERR] JVMDUMP010I Snap dump written to /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.020813.31154.0003.trc
 [ERR] JVMDUMP007I JVM Requesting JIT dump using '/home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.020813.31154.0004.dmp'
 [ERR] JVMDUMP010I JIT dump written to /home/jenkins/jenkins-agent/workspace/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421774804484/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.020813.31154.0004.dmp
 [ERR] JVMDUMP013I Processed dump event "gpf", detail "".
>> Required condition was not found: [Output match: [MSG] Reached OnLoad (testlibA): JNI_OnLoad [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnLoad (testlibB): JNI_OnLoad [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached native fooImpl() [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached native barImpl() [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload (testlibA): JNI_OnUnload [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload (testlibB): JNI_OnUnload [dynamically]]
>> Failure condition was not found: [Output match: [ERR]]
>> Failure condition was not found: [Output match: [statically]]
>> Success condition was not found: [Output match: [MSG] Test jep178 passed with error code: 0]

test failure

Source

pshipton

All 36 comments

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/134

Testing: jep178 native (jvmti) agent testing static linking during live (OnAttach)
Test start time: 2018/11/14 07:57:32 Coordinated Universal Time
Running command: /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_static -jvmpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm -Xbootclasspath/a:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../lib/tools.jar -classpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xcompressedrefs  com/ibm/j9/tests/jeptests/StaticAgents --attach
Time spent starting: 2 milliseconds
Time spent executing: 993 milliseconds
Test result: FAILED
Output from test:
 [OUT] [MSG] Starting up JEP 178 test.
 [OUT] [MSG] Opening jvm from /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm/libjvm.so
 [ERR] Unhandled exception
 [ERR] Type=Segmentation error vmState=0x00000000
 [ERR] J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
 [ERR] Handler1=00007FAF271B9B90 Handler2=00007FAF265A7A20 InaccessibleAddress=0000000000000000
 [ERR] RDI=00007FAF108991C8 RSI=00007FAF27D489CB RAX=00007FAF27D489C3 RBX=00007FAF27F4A0C0
 [ERR] RCX=00007FAF27D489C1 RDX=0000000000000000 R8=0000000000000000 R9=00007FAF280D7F94
 [ERR] R10=0000000000000001 R11=0000000000000001 R12=00007FAF27B2D420 R13=00007FAF10899270
 [ERR] R14=00007FAF27B2D420 R15=00007FAF22FA1418
 [ERR] RIP=00007FAF27D483A9 GS=0000 FS=0000 RSP=00007FAF108991C0
 [ERR] EFlags=0000000000010246 CS=0033 RBP=00007FAF280D7F94 ERR=0000000000000004
 [ERR] TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000000
 [ERR] xmm0 000000000000ff00 (f: 65280.000000, d: 3.225261e-319)
 [ERR] xmm1 00000000000000ff (f: 255.000000, d: 1.259867e-321)
 [ERR] xmm2 ffffffff0000006f (f: 111.000000, d: -nan)
 [ERR] xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm5 ffffff0000000000 (f: 0.000000, d: -nan)
 [ERR] xmm6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm7 612f62696c2f2e2e (f: 1815031296.000000, d: 1.378865e+160)
 [ERR] xmm8 652d747365542f65 (f: 1700016000.000000, d: 2.387178e+179)
 [ERR] xmm9 6e6f6974636e7566 (f: 1668183424.000000, d: 9.083672e+223)
 [ERR] xmm10 38785f78756e696c (f: 1970170240.000000, d: 1.146010e-36)
 [ERR] xmm11 2f73727470737372 (f: 1886614400.000000, d: 4.100301e-80)
 [ERR] xmm12 326a2f7972616e69 (f: 1918987904.000000, d: 7.770178e-66)
 [ERR] xmm13 69622f65726a2f65 (f: 1919561600.000000, d: 4.349946e+199)
 [ERR] xmm14 632f3436646d612f (f: 1684889856.000000, d: 5.888128e+169)
 [ERR] xmm15 696c2f7366657264 (f: 1717924480.000000, d: 6.742029e+199)
 [ERR] Module=/lib/x86_64-linux-gnu/libdl.so.2
 [ERR] Module_base_address=00007FAF27D47000 Symbol=dlerror
 [ERR] Symbol_address=00007FAF27D482B0
 [ERR] Target=2_90_20181114_645 (Linux 4.4.0-134-generic)
 [ERR] CPU=amd64 (4 logical CPUs) (0x1f2ae3000 RAM)
 [ERR] ----------- Stack Backtrace -----------
 [ERR] dlerror+0xf9 (0x00007FAF27D483A9 [libdl.so.2+0x13a9])
 [ERR] (0x00007FAF265AAE55 [libj9prt29.so+0x22e55])
 [ERR] (0x00007FAF265AB21C [libj9prt29.so+0x2321c])
 [ERR] (0x00007FAF271EB85A [libj9vm29.so+0xcd85a])
 [ERR] (0x00007FAF271EC59E [libj9vm29.so+0xce59e])
 [ERR] (0x00007FAF271EC771 [libj9vm29.so+0xce771])
 [ERR] (0x00007FAF27143886 [libj9vm29.so+0x25886])
 [ERR] (0x00007FAF2713714B [libj9vm29.so+0x1914b])
 [ERR] (0x00007FAF271F4252 [libj9vm29.so+0xd6252])
 [ERR] ---------------------------------------
 [ERR] JVMDUMP039I Processing dump event "gpf", detail "" at 2018/11/14 07:57:32 - please wait.
 [ERR] JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075732.9521.0001.dmp' in response to an event
 [ERR] JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.9542.
 [ERR] 
 [ERR] JVMDUMP010I System dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075732.9521.0001.dmp
 [ERR] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075732.9521.0002.txt' in response to an event
 [ERR] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075732.9521.0002.txt
 [ERR] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075732.9521.0003.trc' in response to an event
 [ERR] JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075732.9521.0003.trc
 [ERR] JVMDUMP007I JVM Requesting JIT dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075732.9521.0004.dmp'
 [ERR] JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/test_output_15421794109160/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075732.9521.0004.dmp
 [ERR] JVMDUMP013I Processed dump event "gpf", detail "".
>> Required condition was not found: [Output match: [MSG] Attaching native agent testjvmtiA]
>> Required condition was not found: [Output match: [MSG] Reached OnAttach: Agent_OnAttach_testjvmtiA [statically]]
>> Required condition was not found: [Output match: [MSG] Attaching native agent testjvmtiB]
>> Required condition was not found: [Output match: [MSG] Reached OnAttach: Agent_OnAttach_testjvmtiB [statically]]
>> Required condition was not found: [Output match: [MSG] Testing jep178 for native agents during Live phase (OnAttach)]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload: Agent_OnUnload_testjvmtiA [statically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload: Agent_OnUnload_testjvmtiB [statically]]
>> Failure condition was not found: [Output match: [ERR]]
>> Failure condition was not found: [Output match: [dynamically]]
>> Success condition was not found: [Output match: [MSG] Test jep178 passed with error code: 0]

Testing: jep178 native (jvmti) agent testing dynamic linking at startup (OnLoad)
Test start time: 2018/11/14 07:57:33 Coordinated Universal Time
Running command: /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_dynamic -agentlib:testjvmtiA -agentlib:testjvmtiB -jvmpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm -Xbootclasspath/a:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../lib/tools.jar -classpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xcompressedrefs  com/ibm/j9/tests/jeptests/StaticAgents
Time spent starting: 2 milliseconds
Time spent executing: 2475 milliseconds
Test result: FAILED
Output from test:
 [OUT] [MSG] Starting up JEP 178 test.
 [OUT] [MSG] Opening jvm from /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm/libjvm.so
 [OUT] [MSG] Reached OnLoad (testjvmtiA): Agent_OnLoad [dynamically]
 [OUT] [MSG] Reached OnLoad (testjvmtiB): Agent_OnLoad [dynamically]
 [OUT] [MSG] Testing jep178 for native agents at JVM startup (OnLoad)
 [ERR] *** Error in `/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_dynamic': double free or corruption (!prev): 0x00007fef7400db10 ***
 [ERR] ======= Backtrace: =========
 [ERR] /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fefc59f97e5]
 [ERR] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fefc5a0237a]
 [ERR] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fefc5a0653c]
 [ERR] /lib/x86_64-linux-gnu/libdl.so.2(dlerror+0x228)[0x7fefc577f4d8]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9prt29.so(+0x22e55)[0x7fefc3fe1e55]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9prt29.so(+0x2321c)[0x7fefc3fe221c]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0xcd85a)[0x7fefc4c2285a]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0xce5dd)[0x7fefc4c235dd]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0xce771)[0x7fefc4c23771]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0x25886)[0x7fefc4b7a886]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0x1914b)[0x7fefc4b6e14b]
 [ERR] /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/compressedrefs/libj9vm29.so(+0xd6252)[0x7fefc4c2b252]
 [ERR] ======= Memory map: ========
 [ERR] 00400000-00402000 r-xp 00000000 ca:21 5898491                            /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/native-test-libs/testjep178_dynamic
 [ERR] 00602000-00603000 r--p 00002000 ca:21 5898491                            /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/native-test-libs/testjep178_dynamic
 [ERR] 00603000-00604000 rw-p 00003000 ca:21 5898491                            /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/native-test-libs/testjep178_dynamic
 [ERR] 00b8f000-00bd2000 rw-p 00000000 00:00 0                                  [heap]
 [ERR] 00bd2000-03dd2000 rw-p 00000000 00:00 0                                  [heap]
 [ERR] 03dd2000-0d3d2000 ---p 00000000 00:00 0 
 [ERR] dfff1000-e0000000 ---p 00000000 00:00 0 
 [ERR] e0000000-e0600000 rw-p 00000000 00:00 0 
 [ERR] e0600000-ffe00000 ---p 00000000 00:00 0 
 [ERR] ffe00000-100000000 rw-p 00000000 00:00 0 
 [ERR] 7fef68000000-7fef68021000 rw-p 00000000 00:00 0 
 [ERR] 7fef68021000-7fef6c000000 ---p 00000000 00:00 0 
 [ERR] 7fef70000000-7fef70022000 rw-p 00000000 00:00 0 
 [ERR] 7fef70022000-7fef74000000 ---p 00000000 00:00 0 
 [ERR] 7fef74000000-7fef74021000 rw-p 00000000 00:00 0 
 [ERR] 7fef74021000-7fef78000000 ---p 00000000 00:00 0 
 [ERR] 7fef78000000-7fef78021000 rw-p 00000000 00:00 0 
 [ERR] 7fef78021000-7fef7c000000 ---p 00000000 00:00 0 
 [ERR] 7fef7c000000-7fef7c021000 rw-p 00000000 00:00 0 
 [ERR] 7fef7c021000-7fef80000000 ---p 00000000 00:00 0 
 [ERR] 7fef80000000-7fef80021000 rw-p 00000000 00:00 0 
 [ERR] 7fef80021000-7fef84000000 ---p 00000000 00:00 0 
 [ERR] 7fef84000000-7fef84021000 rw-p 00000000 00:00 0 
 [ERR] 7fef84021000-7fef88000000 ---p 00000000 00:00 0 
 [ERR] 7fef88000000-7fef88021000 rw-p 00000000 00:00 0 
 [ERR] 7fef88021000-7fef8c000000 ---p 00000000 00:00 0 
 [ERR] 7fef8c000000-7fef8c021000 rw-p 00000000 00:00 0 
 [ERR] 7fef8c021000-7fef90000000 ---p 00000000 00:00 0 
 [ERR] 7fef90000000-7fef90024000 rw-p 00000000 00:00 0 
 [ERR] 7fef90024000-7fef94000000 ---p 00000000 00:00 0 
 [ERR] 7fef94000000-7fef94021000 rw-p 00000000 00:00 0 
 [ERR] 7fef94021000-7fef98000000 ---p 00000000 00:00 0 
 [ERR] 7fef98000000-7fef98021000 rw-p 00000000 00:00 0 
 [ERR] 7fef98021000-7fef9c000000 ---p 00000000 00:00 0 
 [ERR] 7fef9c000000-7fef9c021000 rw-p 00000000 00:00 0 
 [ERR] 7fef9c021000-7fefa0000000 ---p 00000000 00:00 0 
 [ERR] 7fefa0000000-7fefa0021000 rw-p 00000000 00:00 0 
 [ERR] 7fefa0021000-7fefa4000000 ---p 00000000 00:00 0 
 [ERR] 7fefa4000000-7fefa4021000 rw-p 00000000 00:00 0 
 [ERR] 7fefa4021000-7fefa8000000 ---p 00000000 00:00 0 
 [ERR] 7fefa8000000-7fefa8021000 rw-p 00000000 00:00 0 
 [ERR] 7fefa8021000-7fefac000000 ---p 00000000 00:00 0 
 [ERR] 7fefada52000-7fefada69000 r-xp 00000000 ca:02 7853                       /lib/x86_64-linux-gnu/libgcc_s.so.1
 [ERR] 7fefada69000-7fefadc68000 ---p 00017000 ca:02 7853                       /lib/x86_64-linux-gnu/libgcc_s.so.1
 [ERR] 7fefadc68000-7fefadc69000 r--p 00016000 ca:02 7853                       /lib/x86_64-linux-gnu/libgcc_s.so.1
 [ERR] 7fefadc69000-7fefadc6a000 rw-p 00017000 ca:02 7853                       /lib/x86_64-linux-gnu/libgcc_s.so.1
 [ERR] 7fefadc6a000-7fefadc8b000 rw-p 00000000 00:00 0 
 [ERR] 7fefadc8b000-7fefadc8e000 r-xp 00000000 ca:02 24714                      /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
 [ERR] 7fefadc8e000-7fefade8d000 ---p 00003000 ca:02 24714                      /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
 [ERR] 7fefade8d000-7fefade8e000 r--p 00002000 ca:02 24714                      /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
 [ERR] 7fefade8e000-7fefade8f000 rw-p 00003000 ca:02 24714                      /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
 [ERR] 7fefade8f000-7fefade90000 ---p 00000000 00:00 0 
 [ERR] 7fefade90000-7fefaded0000 rw-p 00000000 00:00 0 
 [ERR] 7fefaded0000-7fefaded1000 ---p 00000000 00:00 0 
 [ERR] 7fefaded1000-7fefadf11000 rw-p 00000000 00:00 0 
 [ERR] 7fefadf11000-7fefadf12000 ---p 00000000 00:00 0 
 [ERR] 7fefadf12000-7fefadf52000 rw-p 00000000 00:00 0 
 [ERR] 7fefadf52000-7fefadf53000 ---p 00000000 00:00 0 
 [ERR] 7fefadf53000-7fefadf93000 rw-p 00000000 00:00 0 
 [ERR] 7fefadf93000-7fefadf94000 ---p 00000000 00:00 0 
 [ERR] 7fefadf94000-7fefae0d4000 rw-p 00000000 00:00 0 
 [ERR] 7fefae0d4000-7fefae0ef000 r-xp 00000000 ca:21 5899006                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libzip.so
 [ERR] 7fefae0ef000-7fefae2ee000 ---p 0001b000 ca:21 5899006                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libzip.so
 [ERR] 7fefae2ee000-7fefae2ef000 r--p 0001a000 ca:21 5899006                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libzip.so
 [ERR] 7fefae2ef000-7fefae2f0000 rw-p 0001b000 ca:21 5899006                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libzip.so
 [ERR] 7fefae2f0000-7fefae5f1000 rw-p 00000000 00:00 0 
 [ERR] 7fefae5f1000-7fefae5fe000 r-xp 00000000 ca:21 5898875                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libverify.so
 [ERR] 7fefae5fe000-7fefae7fd000 ---p 0000d000 ca:21 5898875                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libverify.so
 [ERR] 7fefae7fd000-7fefae7ff000 r--p 0000c000 ca:21 5898875                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libverify.so
 [ERR] 7fefae7ff000-7fefae800000 rw-p 0000e000 ca:21 5898875                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libverify.so
 [ERR] 7fefae800000-7fefbe800000 rwxp 00000000 00:00 0 
 [ERR] 7fefbe80e000-7fefbe82f000 rw-p 00000000 00:00 0 
 [ERR] 7fefbe82f000-7fefbe830000 ---p 00000000 00:00 0 
 [ERR] 7fefbe830000-7fefbe9f4000 rw-p 00000000 00:00 0 
 [ERR] 7fefbe9f4000-7fefbea19000 r-xp 00000000 ca:21 5898978                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libjava.so
 [ERR] 7fefbea19000-7fefbec18000 ---p 00025000 ca:21 5898978                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libjava.so
 [ERR] 7fefbec18000-7fefbec19000 r--p 00024000 ca:21 5898978                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libjava.so
 [ERR] 7fefbec19000-7fefbec1a000 rw-p 00025000 ca:21 5898978                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/libjava.so
 [ERR] 7fefbec1a000-7fefbec1b000 rw-p 00000000 00:00 0 
 [ERR] 7fefbec1b000-7fefbec22000 r--s 00000000 ca:02 24695                      /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
 [ERR] 7fefbec22000-7fefbec23000 ---p 00000000 00:00 0 
 [ERR] 7fefbec23000-7fefbec63000 rw-p 00000000 00:00 0 
 [ERR] 7fefbec63000-7fefbec64000 r-xp 00000000 ca:21 5898954                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiB.so
 [ERR] 7fefbec64000-7fefbee63000 ---p 00001000 ca:21 5898954                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiB.so
 [ERR] 7fefbee63000-7fefbee64000 r--p 00000000 ca:21 5898954                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiB.so
 [ERR] 7fefbee64000-7fefbee65000 rw-p 00001000 ca:21 5898954                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiB.so
 [ERR] 7fefbee65000-7fefbee66000 r-xp 00000000 ca:21 5898909                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiA.so
 [ERR] 7fefbee66000-7fefbf065000 ---p 00001000 ca:21 5898909                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiA.so
 [ERR] 7fefbf065000-7fefbf066000 r--p 00000000 ca:21 5898909                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiA.so
 [ERR] 7fefbf066000-7fefbf067000 rw-p 00001000 ca:21 5898909                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libtestjvmtiA.so
 [ERR] 7fefbf067000-7fefbf068000 ---p 00000000 00:00 0 
 [ERR] 7fefbf068000-7fefbf168000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf168000-7fefbf169000 ---p 00000000 00:00 0 
 [ERR] 7fefbf169000-7fefbf269000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf269000-7fefbf26a000 ---p 00000000 00:00 0 
 [ERR] 7fefbf26a000-7fefbf36a000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf36a000-7fefbf36b000 ---p 00000000 00:00 0 
 [ERR] 7fefbf36b000-7fefbf46b000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf46b000-7fefbf46c000 ---p 00000000 00:00 0 
 [ERR] 7fefbf46c000-7fefbf56c000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf56c000-7fefbf56d000 ---p 00000000 00:00 0 
 [ERR] 7fefbf56d000-7fefbf66d000 rw-p 00000000 00:00 0 
 [ERR] 7fefbf66d000-7fefbf66e000 ---p 00000000 00:00 0 
 [ERR] 7fefbf66e000-7fefc076e000 rw-p 00000000 00:00 0 
 [ERR] 7fefc076e000-7fefc076f000 ---p 00000000 00:00 0 
 [ERR] 7fefc076f000-7fefc0b70000 rw-p 00000000 00:00 0 
 [ERR] 7fefc0b70000-7fefc0b73000 rw-p 00000000 00:00 0 
 [ERR] 7fefc0b73000-7fefc0c6f000 ---p 00000000 00:00 0 
 [ERR] 7fefc0c6f000-7fefc0c70000 rw-p 00000000 00:00 0 
 [ERR] 7fefc0c70000-7fefc0c71000 ---p 00000000 00:00 0 
 [ERR] 7fefc0c71000-7fefc0d7a000 rw-p 00000000 00:00 0 
 [ERR] 7fefc0d7a000-7fefc0d92000 rw-p 00000000 00:00 0 
 [ERR] 7fefc0d92000-7fefc1572000 ---p 00000000 00:00 0 
 [ERR] 7fefc1572000-7fefc157a000 rw-p 00000000 00:00 0 
 [ERR] 7fefc157a000-7fefc157b000 ---p 00000000 00:00 0 
 [ERR] 7fefc157b000-7fefc15e5000 r-xp 00000000 ca:21 5898948                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjclse7b_29.so
 [ERR] 7fefc15e5000-7fefc17e5000 ---p 0006a000 ca:21 5898948                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjclse7b_29.so
 [ERR] 7fefc17e5000-7fefc17e6000 r--p 0006a000 ca:21 5898948                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjclse7b_29.so
 [ERR] 7fefc17e6000-7fefc17e9000 rw-p 0006b000 ca:21 5898948                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjclse7b_29.so
 [ERR] 7fefc17e9000-7fefc183d000 r-xp 00000000 ca:21 5898955                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jvmti29.so
 [ERR] 7fefc183d000-7fefc1a3d000 ---p 00054000 ca:21 5898955                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jvmti29.so
 [ERR] 7fefc1a3d000-7fefc1a3f000 r--p 00054000 ca:21 5898955                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jvmti29.so
 [ERR] 7fefc1a3f000-7fefc1a42000 rw-p 00056000 ca:21 5898955                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jvmti29.so
 [ERR] 7fefc1a42000-7fefc1ac1000 r-xp 00000000 ca:21 5898934                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9shr29.so
 [ERR] 7fefc1ac1000-7fefc1cc1000 ---p 0007f000 ca:21 5898934                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9shr29.so
 [ERR] 7fefc1cc1000-7fefc1cc3000 r--p 0007f000 ca:21 5898934                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9shr29.so
 [ERR] 7fefc1cc3000-7fefc1cc9000 rw-p 00081000 ca:21 5898934                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9shr29.so
 [ERR] 7fefc1cc9000-7fefc1d67000 r-xp 00000000 ca:21 5898938                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vrb29.so
 [ERR] 7fefc1d67000-7fefc1f66000 ---p 0009e000 ca:21 5898938                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vrb29.so
 [ERR] 7fefc1f66000-7fefc1f6b000 r--p 0009d000 ca:21 5898938                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vrb29.so
 [ERR] 7fefc1f6b000-7fefc1f6f000 rw-p 000a2000 ca:21 5898938                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vrb29.so
 [ERR] 7fefc1f6f000-7fefc220a000 r-xp 00000000 ca:21 5898973                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9gc29.so
 [ERR] 7fefc220a000-7fefc240a000 ---p 0029b000 ca:21 5898973                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9gc29.so
 [ERR] 7fefc240a000-7fefc2417000 r--p 0029b000 ca:21 5898973                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9gc29.so
 [ERR] 7fefc2417000-7fefc241b000 rw-p 002a8000 ca:21 5898973                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9gc29.so
 [ERR] 7fefc241b000-7fefc2d82000 r-xp 00000000 ca:21 5898921                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jit29.so
 [ERR] 7fefc2d82000-7fefc2f82000 ---p 00967000 ca:21 5898921                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jit29.so
 [ERR] 7fefc2f82000-7fefc2fb6000 r--p 00967000 ca:21 5898921                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jit29.so
 [ERR] 7fefc2fb6000-7fefc2fde000 rw-p 0099b000 ca:21 5898921                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9jit29.so
 [ERR] 7fefc2fde000-7fefc300c000 rw-p 00000000 00:00 0 
 [ERR] 7fefc300c000-7fefc301f000 r-xp 00000000 ca:21 5898970                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9zlib29.so
 [ERR] 7fefc301f000-7fefc321e000 ---p 00013000 ca:21 5898970                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9zlib29.so
 [ERR] 7fefc321e000-7fefc321f000 r--p 00012000 ca:21 5898970                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9zlib29.so
 [ERR] 7fefc321f000-7fefc3220000 rw-p 00013000 ca:21 5898970                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9zlib29.so
 [ERR] 7fefc3220000-7fefc322b000 r-xp 00000000 ca:02 1991                       /lib/x86_64-linux-gnu/libnss_files-2.23.so
 [ERR] 7fefc322b000-7fefc342a000 ---p 0000b000 ca:02 1991                       /lib/x86_64-linux-gnu/libnss_files-2.23.so
 [ERR] 7fefc342a000-7fefc342b000 r--p 0000a000 ca:02 1991                       /lib/x86_64-linux-gnu/libnss_files-2.23.so
 [ERR] 7fefc342b000-7fefc342c000 rw-p 0000b000 ca:02 1991                       /lib/x86_64-linux-gnu/libnss_files-2.23.so
 [ERR] 7fefc342c000-7fefc3432000 rw-p 00000000 00:00 0 
 [ERR] 7fefc3432000-7fefc343d000 r-xp 00000000 ca:02 1995                       /lib/x86_64-linux-gnu/libnss_nis-2.23.so
 [ERR] 7fefc343d000-7fefc363c000 ---p 0000b000 ca:02 1995                       /lib/x86_64-linux-gnu/libnss_nis-2.23.so
 [ERR] 7fefc363c000-7fefc363d000 r--p 0000a000 ca:02 1995                       /lib/x86_64-linux-gnu/libnss_nis-2.23.so
 [ERR] 7fefc363d000-7fefc363e000 rw-p 0000b000 ca:02 1995                       /lib/x86_64-linux-gnu/libnss_nis-2.23.so
 [ERR] 7fefc363e000-7fefc3654000 r-xp 00000000 ca:02 1968                       /lib/x86_64-linux-gnu/libnsl-2.23.so
 [ERR] 7fefc3654000-7fefc3853000 ---p 00016000 ca:02 1968                       /lib/x86_64-linux-gnu/libnsl-2.23.so
 [ERR] 7fefc3853000-7fefc3854000 r--p 00015000 ca:02 1968                       /lib/x86_64-linux-gnu/libnsl-2.23.so
 [ERR] 7fefc3854000-7fefc3855000 rw-p 00016000 ca:02 1968                       /lib/x86_64-linux-gnu/libnsl-2.23.so
 [ERR] 7fefc3855000-7fefc3857000 rw-p 00000000 00:00 0 
 [ERR] 7fefc3857000-7fefc385f000 r-xp 00000000 ca:02 1983                       /lib/x86_64-linux-gnu/libnss_compat-2.23.so
 [ERR] 7fefc385f000-7fefc3a5e000 ---p 00008000 ca:02 1983                       /lib/x86_64-linux-gnu/libnss_compat-2.23.so
 [ERR] 7fefc3a5e000-7fefc3a5f000 r--p 00007000 ca:02 1983                       /lib/x86_64-linux-gnu/libnss_compat-2.23.so
 [ERR] 7fefc3a5f000-7fefc3a60000 rw-p 00008000 ca:02 1983                       /lib/x86_64-linux-gnu/libnss_compat-2.23.so
 [ERR] 7fefc3a60000-7fefc3a8e000 r-xp 00000000 ca:21 5898904                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9trc29.so
 [ERR] 7fefc3a8e000-7fefc3c8e000 ---p 0002e000 ca:21 5898904                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9trc29.so
 [ERR] 7fefc3c8e000-7fefc3c8f000 r--p 0002e000 ca:21 5898904                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9trc29.so
 [ERR] 7fefc3c8f000-7fefc3c91000 rw-p 0002f000 ca:21 5898904                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9trc29.so
 [ERR] 7fefc3c91000-7fefc3ccf000 r-xp 00000000 ca:21 5898975                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9dmp29.so
 [ERR] 7fefc3ccf000-7fefc3ecf000 ---p 0003e000 ca:21 5898975                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9dmp29.so
 [ERR] 7fefc3ecf000-7fefc3ed0000 r--p 0003e000 ca:21 5898975                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9dmp29.so
 [ERR] 7fefc3ed0000-7fefc3ed1000 rw-p 0003f000 ca:21 5898975                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9dmp29.so
 [ERR] 7fefc3ed1000-7fefc3f7e000 rw-p 00000000 00:00 0 
 [ERR] 7fefc3f7e000-7fefc3f7f000 ---p 00000000 00:00 0 
 [ERR] 7fefc3f7f000-7fefc3fbf000 rw-p 00000000 00:00 0 
 [ERR] 7fefc3fbf000-7fefc4017000 r-xp 00000000 ca:21 5898906                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9prt29.so
 [ERR] 7fefc4017000-7fefc4216000 ---p 00058000 ca:21 5898906                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9prt29.so
 [ERR] 7fefc4216000-7fefc4217000 r--p 00057000 ca:21 5898906                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9prt29.so
 [ERR] 7fefc4217000-7fefc4221000 rw-p 00058000 ca:21 5898906                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9prt29.so
 [ERR] 7fefc4221000-7fefc4223000 rw-p 00000000 00:00 0 
 [ERR] 7fefc4223000-7fefc422a000 r-xp 00000000 ca:02 1996                       /lib/x86_64-linux-gnu/librt-2.23.so
 [ERR] 7fefc422a000-7fefc4429000 ---p 00007000 ca:02 1996                       /lib/x86_64-linux-gnu/librt-2.23.so
 [ERR] 7fefc4429000-7fefc442a000 r--p 00006000 ca:02 1996                       /lib/x86_64-linux-gnu/librt-2.23.so
 [ERR] 7fefc442a000-7fefc442b000 rw-p 00007000 ca:02 1996                       /lib/x86_64-linux-gnu/librt-2.23.so
 [ERR] 7fefc442b000-7fefc4533000 r-xp 00000000 ca:02 1967                       /lib/x86_64-linux-gnu/libm-2.23.so
 [ERR] 7fefc4533000-7fefc4732000 ---p 00108000 ca:02 1967                       /lib/x86_64-linux-gnu/libm-2.23.so
 [ERR] 7fefc4732000-7fefc4733000 r--p 00107000 ca:02 1967                       /lib/x86_64-linux-gnu/libm-2.23.so
 [ERR] 7fefc4733000-7fefc4734000 rw-p 00108000 ca:02 1967                       /lib/x86_64-linux-gnu/libm-2.23.so
 [ERR] 7fefc4734000-7fefc4738000 r-xp 00000000 ca:21 5898966                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9hookable29.so
 [ERR] 7fefc4738000-7fefc4938000 ---p 00004000 ca:21 5898966                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9hookable29.so
 [ERR] 7fefc4938000-7fefc4939000 r--p 00004000 ca:21 5898966                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9hookable29.so
 [ERR] 7fefc4939000-7fefc493a000 rw-p 00005000 ca:21 5898966                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9hookable29.so
 [ERR] 7fefc493a000-7fefc4952000 r-xp 00000000 ca:21 5898931                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9thr29.so
 [ERR] 7fefc4952000-7fefc4b52000 ---p 00018000 ca:21 5898931                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9thr29.so
 [ERR] 7fefc4b52000-7fefc4b53000 r--p 00018000 ca:21 5898931                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9thr29.so
 [ERR] 7fefc4b53000-7fefc4b54000 rw-p 00019000 ca:21 5898931                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9thr29.so
 [ERR] 7fefc4b54000-7fefc4b55000 rw-p 00000000 00:00 0 
 [ERR] 7fefc4b55000-7fefc4cde000 r-xp 00000000 ca:21 5898956                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vm29.so
 [ERR] 7fefc4cde000-7fefc4ede000 ---p 00189000 ca:21 5898956                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vm29.so
 [ERR] 7fefc4ede000-7fefc4ee4000 r--p 00189000 ca:21 5898956                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vm29.so
 [ERR] 7fefc4ee4000-7fefc4eea000 rw-p 0018f000 ca:21 5898956                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libj9vm29.so
 [ERR] 7fefc4eea000-7fefc4eec000 rw-p 00000000 00:00 0 
 [ERR] 7fefc4eec000-7fefc4eee000 r-xp 00000000 ca:21 5898963                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libomrsig.so
 [ERR] 7fefc4eee000-7fefc50ed000 ---p 00002000 ca:21 5898963                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libomrsig.so
 [ERR] 7fefc50ed000-7fefc50ee000 r--p 00001000 ca:21 5898963                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libomrsig.so
 [ERR] 7fefc50ee000-7fefc50ef000 rw-p 00002000 ca:21 5898963                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libomrsig.so
 [ERR] 7fefc50ef000-7fefc50f3000 rw-p 00000000 00:00 0 
 [ERR] 7fefc50f3000-7fefc510b000 r-xp 00000000 ca:02 1970                       /lib/x86_64-linux-gnu/libpthread-2.23.so
 [ERR] 7fefc510b000-7fefc530a000 ---p 00018000 ca:02 1970                       /lib/x86_64-linux-gnu/libpthread-2.23.so
 [ERR] 7fefc530a000-7fefc530b000 r--p 00017000 ca:02 1970                       /lib/x86_64-linux-gnu/libpthread-2.23.so
 [ERR] 7fefc530b000-7fefc530c000 rw-p 00018000 ca:02 1970                       /lib/x86_64-linux-gnu/libpthread-2.23.so
 [ERR] 7fefc530c000-7fefc5310000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5310000-7fefc535f000 r-xp 00000000 ca:21 5898914                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjvm.so
 [ERR] 7fefc535f000-7fefc555f000 ---p 0004f000 ca:21 5898914                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjvm.so
 [ERR] 7fefc555f000-7fefc5562000 r--p 0004f000 ca:21 5898914                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjvm.so
 [ERR] 7fefc5562000-7fefc5565000 rw-p 00052000 ca:21 5898914                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/compressedrefs/libjvm.so
 [ERR] 7fefc5565000-7fefc5568000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5568000-7fefc557c000 r-xp 00000000 ca:21 5898893                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/j9vm/libjvm.so
 [ERR] 7fefc557c000-7fefc577b000 ---p 00014000 ca:21 5898893                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/j9vm/libjvm.so
 [ERR] 7fefc577b000-7fefc577c000 r--p 00013000 ca:21 5898893                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/j9vm/libjvm.so
 [ERR] 7fefc577c000-7fefc577d000 rw-p 00014000 ca:21 5898893                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/amd64/j9vm/libjvm.so
 [ERR] 7fefc577d000-7fefc577e000 rw-p 00000000 00:00 0 
 [ERR] 7fefc577e000-7fefc5781000 r-xp 00000000 ca:02 1973                       /lib/x86_64-linux-gnu/libdl-2.23.so
 [ERR] 7fefc5781000-7fefc5980000 ---p 00003000 ca:02 1973                       /lib/x86_64-linux-gnu/libdl-2.23.so
 [ERR] 7fefc5980000-7fefc5981000 r--p 00002000 ca:02 1973                       /lib/x86_64-linux-gnu/libdl-2.23.so
 [ERR] 7fefc5981000-7fefc5982000 rw-p 00003000 ca:02 1973                       /lib/x86_64-linux-gnu/libdl-2.23.so
 [ERR] 7fefc5982000-7fefc5b42000 r-xp 00000000 ca:02 1971                       /lib/x86_64-linux-gnu/libc-2.23.so
 [ERR] 7fefc5b42000-7fefc5d42000 ---p 001c0000 ca:02 1971                       /lib/x86_64-linux-gnu/libc-2.23.so
 [ERR] 7fefc5d42000-7fefc5d46000 r--p 001c0000 ca:02 1971                       /lib/x86_64-linux-gnu/libc-2.23.so
 [ERR] 7fefc5d46000-7fefc5d48000 rw-p 001c4000 ca:02 1971                       /lib/x86_64-linux-gnu/libc-2.23.so
 [ERR] 7fefc5d48000-7fefc5d4c000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5d4c000-7fefc5d72000 r-xp 00000000 ca:02 1969                       /lib/x86_64-linux-gnu/ld-2.23.so
 [ERR] 7fefc5d73000-7fefc5d78000 r--s 0003b000 ca:21 5899078                    /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/ext/dtfjview.jar
 [ERR] 7fefc5d78000-7fefc5d79000 ---p 00000000 00:00 0 
 [ERR] 7fefc5d79000-7fefc5d99000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5d99000-7fefc5f31000 r--p 00000000 ca:02 28956                      /usr/lib/locale/locale-archive
 [ERR] 7fefc5f31000-7fefc5f66000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5f66000-7fefc5f67000 ---p 00000000 00:00 0 
 [ERR] 7fefc5f67000-7fefc5f6f000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5f6f000-7fefc5f70000 ---p 00000000 00:00 0 
 [ERR] 7fefc5f70000-7fefc5f71000 rw-p 00000000 00:00 0 
 [ERR] 7fefc5f71000-7fefc5f72000 r--p 00025000 ca:02 1969                       /lib/x86_64-linux-gnu/ld-2.23.so
 [ERR] 7fefc5f72000-7fefc5f73000 rw-p 00026000 ca:02 1969                       /lib/x86_64-linux-gnu/ld-2.23.so
 [ERR] 7fefc5f73000-7fefc5f74000 rw-p 00000000 00:00 0 
 [ERR] 7ffd5417e000-7ffd541a0000 rw-p 00000000 00:00 0                          [stack]
 [ERR] 7ffd541d9000-7ffd541dc000 r--p 00000000 00:00 0                          [vvar]
 [ERR] 7ffd541dc000-7ffd541de000 r-xp 00000000 00:00 0                          [vdso]
 [ERR] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
 [ERR] JVMDUMP039I Processing dump event "abort", detail "" at 2018/11/14 07:57:33 - please wait.

pshipton on 14 Nov 2018

@DanHeidinga

pshipton on 14 Nov 2018

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64/138

===============================================
Running test cmdLineTester_jep178_staticLinking_SE80_0 ...
===============================================
cmdLineTester_jep178_staticLinking_SE80_0 Start Time: Wed Nov 14 07:54:33 2018 Epoch Time (ms): 1542182073249
variation: NoOptions
JVM_OPTIONS: -Xnocompressedrefs 
*** Starting test suite: JEP178 static linking tests ***
Testing: jep178 testing static linking
Test start time: 2018/11/14 07:54:33 Coordinated Universal Time
Running command: /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_static -jvmpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm -classpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xnocompressedrefs  com/ibm/j9/tests/jeptests/StaticLinking
Time spent starting: 31 milliseconds
Time spent executing: 1027 milliseconds
Test result: FAILED
Output from test:
 [OUT] [MSG] Starting up JEP 178 test.
 [OUT] [MSG] Opening jvm from /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm/libjvm.so
 [ERR] Unhandled exception
 [ERR] Type=Segmentation error vmState=0x00000000
 [ERR] J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
 [ERR] Handler1=00007F179FB19CD0 Handler2=00007F179EF09A20 InaccessibleAddress=0000000000000000
 [ERR] RDI=00007F17694B6208 RSI=00007F17A06A79CB RAX=00007F17A06A79C3 RBX=00007F17A08A90C0
 [ERR] RCX=00007F17A06A79C1 RDX=0000000000000000 R8=0000000000000000 R9=00007F17A0A36F94
 [ERR] R10=0000000000000001 R11=0000000000000001 R12=00007F17A048C420 R13=00007F17694B62B0
 [ERR] R14=00007F17A048C420 R15=0000000001119DB8
 [ERR] RIP=00007F17A06A73A9 GS=0000 FS=0000 RSP=00007F17694B6200
 [ERR] EFlags=0000000000010246 CS=0033 RBP=00007F17A0A36F94 ERR=0000000000000004
 [ERR] TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000000
 [ERR] xmm0 000000000000ff00 (f: 65280.000000, d: 3.225261e-319)
 [ERR] xmm1 00000000000000ff (f: 255.000000, d: 1.259867e-321)
 [ERR] xmm2 ffffffff0000006f (f: 111.000000, d: -nan)
 [ERR] xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm8 616e69626b646a6e (f: 1801742976.000000, d: 2.137804e+161)
 [ERR] xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm10 3d72628e03bf8574 (f: 62883188.000000, d: 1.045065e-12)
 [ERR] xmm11 402ab8ee105da5a4 (f: 274572704.000000, d: 1.336119e+01)
 [ERR] xmm12 3cc1000000000000 (f: 0.000000, d: 4.718448e-16)
 [ERR] xmm13 bfc87fa06520c900 (f: 1696647424.000000, d: -1.913949e-01)
 [ERR] xmm14 402a56ef8ec92280 (f: 2395546112.000000, d: 1.316980e+01)
 [ERR] xmm15 bc6bf7fdbfa08d9a (f: 3214970368.000000, d: -1.212950e-17)
 [ERR] Module=/lib/x86_64-linux-gnu/libdl.so.2
 [ERR] Module_base_address=00007F17A06A6000 Symbol=dlerror
 [ERR] Symbol_address=00007F17A06A72B0
 [ERR] Target=2_90_20181114_594 (Linux 4.4.0-134-generic)
 [ERR] CPU=amd64 (4 logical CPUs) (0x1f2ae3000 RAM)
 [ERR] ----------- Stack Backtrace -----------
 [ERR] dlerror+0xf9 (0x00007F17A06A73A9 [libdl.so.2+0x13a9])
 [ERR] (0x00007F179EF0CE55 [libj9prt29.so+0x22e55])
 [ERR] (0x00007F179EF0D21C [libj9prt29.so+0x2321c])
 [ERR] (0x00007F179FB4B33A [libj9vm29.so+0xcb33a])
 [ERR] (0x00007F179FB4C07E [libj9vm29.so+0xcc07e])
 [ERR] (0x00007F179FB4C251 [libj9vm29.so+0xcc251])
 [ERR] (0x00007F179FAA404B [libj9vm29.so+0x2404b])
 [ERR] (0x00007F179FA9913F [libj9vm29.so+0x1913f])
 [ERR] (0x00007F179FB53CE2 [libj9vm29.so+0xd3ce2])
 [ERR] ---------------------------------------
 [ERR] JVMDUMP039I Processing dump event "gpf", detail "" at 2018/11/14 07:54:33 - please wait.
 [ERR] JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075433.21315.0001.dmp' in response to an event
 [ERR] JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.21337.
 [ERR] 
 [ERR] JVMDUMP010I System dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075433.21315.0001.dmp
 [ERR] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075433.21315.0002.txt' in response to an event
 [ERR] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075433.21315.0002.txt
 [ERR] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075433.21315.0003.trc' in response to an event
 [ERR] JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075433.21315.0003.trc
 [ERR] JVMDUMP007I JVM Requesting JIT dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075433.21315.0004.dmp'
 [ERR] JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075433.21315.0004.dmp
 [ERR] JVMDUMP013I Processed dump event "gpf", detail "".
>> Required condition was not found: [Output match: [MSG] Reached OnLoad: JNI_OnLoad_testlibA [statically]]
>> Required condition was not found: [Output match: [MSG] Reached OnLoad: JNI_OnLoad_testlibB [statically]]
>> Required condition was not found: [Output match: [MSG] Reached native fooImpl() [statically]]
>> Required condition was not found: [Output match: [MSG] Reached native barImpl() [statically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload: JNI_OnUnload_testlibA [statically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload: JNI_OnUnload_testlibB [statically]]
>> Failure condition was not found: [Output match: [ERR]]
>> Failure condition was not found: [Output match: [dynamically]]
>> Success condition was not found: [Output match: [MSG] Test jep178 passed with error code: 0]

Testing: jep178 testing dynamic linking
Test start time: 2018/11/14 07:54:34 Coordinated Universal Time
Running command: /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../../../native-test-libs//testjep178_dynamic -jvmpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm -classpath:/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xnocompressedrefs  com/ibm/j9/tests/jeptests/StaticLinking
Time spent starting: 2 milliseconds
Time spent executing: 1408 milliseconds
Test result: FAILED
Output from test:
 [OUT] [MSG] Starting up JEP 178 test.
 [OUT] [MSG] Opening jvm from /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdkbinary/j2sdk-image/jre/bin/../lib/amd64/j9vm/libjvm.so
 [ERR] Unhandled exception
 [ERR] Type=Segmentation error vmState=0x00000000
 [ERR] J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
 [ERR] Handler1=00007FBB4919ECD0 Handler2=00007FBB4858EA20 InaccessibleAddress=0000000000000000
 [ERR] RDI=00007FBB49D2C9B3 RSI=0000000000000000 RAX=0000000000000000 RBX=00007FBB49F2E0C0
 [ERR] RCX=000000000000000E RDX=00007FBAD0000020 R8=0000000000000000 R9=0000000000000000
 [ERR] R10=0000000000000019 R11=00007FBB4A0BBF94 R12=00007FBB49B11420 R13=00007FBB0EB3A2B0
 [ERR] R14=00007FBB49B11420 R15=0000000001704DB8
 [ERR] RIP=00007FBB49D2C3D3 GS=0000 FS=0000 RSP=00007FBB0EB3A200
 [ERR] EFlags=0000000000010213 CS=0033 RBP=00007FBB4A0BBF94 ERR=0000000000000004
 [ERR] TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000000
 [ERR] xmm0 00000000000000ff (f: 255.000000, d: 1.259867e-321)
 [ERR] xmm1 2525252525252525 (f: 623191360.000000, d: 9.532824e-130)
 [ERR] xmm2 ffffffff0000006f (f: 111.000000, d: -nan)
 [ERR] xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm4 00000000000000ff (f: 255.000000, d: 1.259867e-321)
 [ERR] xmm5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm8 616e69626b646a6e (f: 1801742976.000000, d: 2.137804e+161)
 [ERR] xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
 [ERR] xmm10 3d75450bc6f1da85 (f: 3337739008.000000, d: 1.209043e-12)
 [ERR] xmm11 402f2b53d74e37d9 (f: 3612227584.000000, d: 1.558462e+01)
 [ERR] xmm12 bcc2000000000000 (f: 0.000000, d: -4.996004e-16)
 [ERR] xmm13 bfd576f716c55b20 (f: 382032672.000000, d: -3.353861e-01)
 [ERR] xmm14 402e7f9c1e980d00 (f: 513281280.000000, d: 1.524924e+01)
 [ERR] xmm15 3c752658c393cc74 (f: 3281243136.000000, d: 1.834452e-17)
 [ERR] Module=/lib/x86_64-linux-gnu/libdl.so.2
 [ERR] Module_base_address=00007FBB49D2B000 Symbol=dlerror
 [ERR] Symbol_address=00007FBB49D2C2B0
 [ERR] Target=2_90_20181114_594 (Linux 4.4.0-134-generic)
 [ERR] CPU=amd64 (4 logical CPUs) (0x1f2ae3000 RAM)
 [ERR] ----------- Stack Backtrace -----------
 [ERR] dlerror+0x123 (0x00007FBB49D2C3D3 [libdl.so.2+0x13d3])
 [ERR] (0x00007FBB48591E55 [libj9prt29.so+0x22e55])
 [ERR] (0x00007FBB4859221C [libj9prt29.so+0x2321c])
 [ERR] (0x00007FBB491D033A [libj9vm29.so+0xcb33a])
 [ERR] (0x00007FBB491D107E [libj9vm29.so+0xcc07e])
 [ERR] (0x00007FBB491D1251 [libj9vm29.so+0xcc251])
 [ERR] (0x00007FBB4912904B [libj9vm29.so+0x2404b])
 [ERR] (0x00007FBB4911E13F [libj9vm29.so+0x1913f])
 [ERR] (0x00007FBB491D8CE2 [libj9vm29.so+0xd3ce2])
 [ERR] ---------------------------------------
 [ERR] JVMDUMP039I Processing dump event "gpf", detail "" at 2018/11/14 07:54:34 - please wait.
 [ERR] JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075434.21340.0001.dmp' in response to an event
 [ERR] JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.21361.
 [ERR] 
 [ERR] JVMDUMP010I System dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/core.20181114.075434.21340.0001.dmp
 [ERR] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075434.21340.0002.txt' in response to an event
 [ERR] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/javacore.20181114.075434.21340.0002.txt
 [ERR] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075434.21340.0003.trc' in response to an event
 [ERR] JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/Snap.20181114.075434.21340.0003.trc
 [ERR] JVMDUMP007I JVM Requesting JIT dump using '/home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075434.21340.0004.dmp'
 [ERR] JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test-extended.functional-JDK8-linux_x86-64/openjdk-tests/TestConfig/test_output_15421812987582/cmdLineTester_jep178_staticLinking_SE80_0/jitdump.20181114.075434.21340.0004.dmp
 [ERR] JVMDUMP013I Processed dump event "gpf", detail "".
>> Required condition was not found: [Output match: [MSG] Reached OnLoad (testlibA): JNI_OnLoad [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnLoad (testlibB): JNI_OnLoad [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached native fooImpl() [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached native barImpl() [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload (testlibA): JNI_OnUnload [dynamically]]
>> Required condition was not found: [Output match: [MSG] Reached OnUnload (testlibB): JNI_OnUnload [dynamically]]
>> Failure condition was not found: [Output match: [ERR]]
>> Failure condition was not found: [Output match: [statically]]
>> Success condition was not found: [Output match: [MSG] Test jep178 passed with error code: 0]

pshipton on 14 Nov 2018

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_390-64_cmprssptrs/150
https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64/139

pshipton on 15 Nov 2018

@pdbain-ibm Can you take a look at this?

DanHeidinga on 15 Nov 2018

Will do.

pdbain-ibm on 15 Nov 2018

The test fails when via the command line tester, but not when I run it directly from bash:

/home/pdbain/vm/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/jre/bin//../../../native-test-libs//testjep178_static -agentlib:testjvmtiA -agentlib:testjvmtiB -jvmpath:/home/pdbain/vm/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/jre/bin//../lib/amd64/j9vm -Xbootclasspath/a:/home/pdbain/vm/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/jre/bin//../../lib/tools.jar -classpath:/home/pdbain/test/openj9/openjdk-tests/TestConfig/scripts/testKitGen/../../../../jvmtest/functional/cmdLineTests/jep178staticLinkingTest/jep178staticLinkingTest.jar -Xcompressedrefs com/ibm/j9/tests/jeptests/StaticAgents
[MSG] Starting up JEP 178 test.
[MSG] Opening jvm from /home/pdbain/vm/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/jre/bin//../lib/amd64/j9vm/libjvm.so
[MSG] Reached OnLoad: Agent_OnLoad_testjvmtiA [statically]
[MSG] Reached OnLoad: Agent_OnLoad_testjvmtiB [statically]
[MSG] Testing jep178 for native agents at JVM startup (OnLoad)
[MSG] Reached OnUnload: Agent_OnUnload_testjvmtiA [statically]
[MSG] Reached OnUnload: Agent_OnUnload_testjvmtiB [statically]
[MSG] Test jep178 passed with error code: 0

pdbain-ibm on 15 Nov 2018

Note the problems are intermittent.

pshipton on 15 Nov 2018

Failure is intermittent. I am seeing native-test-libs//testjep178_static': double free or corruption (top), native-test-libs//testjep178_dynamic': corrupted size vs. prev_size: 0x00007f05d000c180 on the console output. Finally managed to reproduce it in the console.

pdbain-ibm on 16 Nov 2018

The test also occurs if we are loading a single agent at launch time.
The segfault is in dlerror().

pdbain-ibm on 16 Nov 2018

... in fact, it fails even if we load no agents, so it's not related to JVMTI. The crash occurs (intermittently) while the port library is trying to load libnio.so.

pdbain-ibm on 20 Nov 2018

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64_cmprssptrs/139
https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64/143

pshipton on 20 Nov 2018

Reproduced the problem with glaunch:
glaunch /path/to/jre/lib/amd64/compressedrefs HelloWorld
also intermittent with about 1% failure rate.

pdbain-ibm on 20 Nov 2018

https://ci.eclipse.org/openj9/job/Test-extended.functional-JDK8-linux_x86-64/144

pshipton on 21 Nov 2018

The failures are in the dynamic library functions, mostly with the dl_action_result, which appears to be getting corrupted.

pdbain-ibm on 21 Nov 2018

I instrumented the dlfcn functions (dlopen, dlsym, dlerror, et al.). They use a dl_action_result buffer which _should_ be different for each thread, but don't appear to be. One thread is calling dlopen and anther thread dlsym, and using the same result buffer. Working on why this is so: either __libc_getspecific() isn't working or __libc_key_create() is failing.

pdbain-ibm on 22 Nov 2018

I can check this in gdb:

gdb --args /path/to/vm/glaunch /path/to/jre/lib/amd64/j9vm/libjvm.so HelloWorld
(gdb) b _dlerror_run
Breakpoint 1 at 0x7ffff7bd4500: file dlerror.c, line 129.
(gdb) r
...Breakpoint 1, _dlerror_run (operate=operate@entry=0x7ffff7bd3eb0 <dlopen_doit>, args=args@entry=0x7fffffffe930) at dlerror.c:129
129 dlerror.c: No such file or directory.
(gdb) n
125 in dlerror.c
...
154 in dlerror.c
(gdb) p static_buf
$1 = (struct dl_action_result *) 0x7ffff7dd60c0 <last_result>
(gdb) p result
$2 = (struct dl_action_result *) 0x7ffff7dd60c0 <last_result>

This indicates the __libc_key_create() is failing.

Trying with gdb --args /path/to/jre/bin/java:

150 in dlerror.c
(gdb) 
154 in dlerror.c
(gdb) p static_buf
$1 = (struct dl_action_result *) 0x0
(gdb) p result
$2 = (struct dl_action_result *) 0x6020d0

This is a different execution path through _dlerror_run. static_buf is null, indicating the __libc_key_create() succeeded.

Testing with a simple test case:

#include <dlfcn.h>
#include <stdio.h>
int main(int argc, char *argv[]) {
    dlopen("bar", RTLD_NOW);
    printf("hello, world\n");
}

$ gcc foo.c -o foo -ldl

shows that the call is failing.
Must be something in the compile options...

pdbain-ibm on 23 Nov 2018

Adding the following (unused) code to genlauncher.c causes it to run reliably:

static pthread_key_t key;
static pthread_once_t key_once = PTHREAD_ONCE_INIT;
static void unused_function()
{
    pthread_key_create(&key, NULL);
}

pdbain-ibm on 23 Nov 2018

Tried forcing the symbol into the object file via -u. No joy.

dlerror.c:init() calls __libc_key_create.
glibc-2.23/sysdeps/nptl/libc-lockP.h:

/* Create thread-specific key.  */
#define __libc_key_create(KEY, DESTRUCTOR) \
  __libc_ptf_call (__pthread_key_create, (KEY, DESTRUCTOR), 1)

# define __libc_ptf_call(FUNC, ARGS, ELSE) \
  (__libc_pthread_functions_init ? PTHFCT_CALL (ptr_##FUNC, ARGS) : ELSE)

sysdeps/nptl/pthread-functions.h:

#ifdef PTR_DEMANGLE
# define PTHFCT_CALL(fct, params) \
  ({ __typeof (__libc_pthread_functions.fct) __p;                 \
     __p = __libc_pthread_functions.fct;                      \
     PTR_DEMANGLE (__p);                              \
     __p params; })
#else
# define PTHFCT_CALL(fct, params) \
  __libc_pthread_functions.fct params
#endif

_dlerror_run() calls __libc_once (once, init); to set up the error buffer.
If the main file, e.g. foo.c, doesn't include a reference to pthread_key_create(),
the function table __libc_pthread_functions is not initialized.

pdbain-ibm on 26 Nov 2018

__pthread_initialize_minimal_internal() initializes the table. It isn't called if the application makes no reference to the pthread functions. It seems we need to force pthreads to initialize.

pdbain-ibm on 26 Nov 2018

Known issue in glibc 2.23: https://sourceware.org/bugzilla/show_bug.cgi?id=16628 is Fixed in GLIBC 2.25. Fixed in 2.25. Many thanks to @babsingh for finding the bug report.
@AlenBadel FYI.

pdbain-ibm on 26 Nov 2018

👍1

From the glibc bug:

If the main executable is linked without pthread dlopen()s a shared library linked with pthread, it causes a memory corruption in the dynamic loader.

@pdbain-ibm Is fixing this just a matter of adding -lpthread to our exe link lines?

DanHeidinga on 26 Nov 2018

@DanHeidinga in reply to https://github.com/eclipse/openj9/issues/3672#issuecomment-441701856, the makefiles already do that: -lm -lpthread -lc -lrt -ldl -lutil. I tried a few other linker options, like adding -u <some pthread symbol>, but to no avail. Note that the regular Java launcher obviously doesn't have this problem; I assume it makes some pthread calls.

Options are:

Upgrade to glibc 2.25. I will test that on my Ubuntu 16 machine. @jdekonin or @AdamBrousseau is it possible to upgrade the various build machines? We will need to do that eventually, no?
Add a dummy call to the jep178test source to force pthread initialization until 1) is done.
Exclude the test or ignore the failures until 1) is done.

pdbain-ibm on 27 Nov 2018

Looks like option 1) is impractical. My Ubuntu machine is Ubuntu 16.04.5 LTS, for which glibc 2.23 is the latest release. I propose option 2), kludgy though it is.

pdbain-ibm on 27 Nov 2018

@pdbain-ibm Can you supply the full link line for the testjep178_dynamic executable? And also the ldd output?

I'm trying to figure out why the behaviour we're seeing doesn't match the bug description.

To be clear, (1) is not an option as changing the glibc level changes the supported platforms. We're not going to drop support for some OS levels for an issue that has so far only appeared in our own test executables.

DanHeidinga on 27 Nov 2018

@DanHeidinga Note the failures are occurring on the Ubuntu 16 machines which OpenJ9 uses for nightly testing. These machines are already not using the minimum glibc level that OpenJ9 supports. The machines could be upgraded to Ubuntu 18, although I'm not sure what other issues this may cause.

pshipton on 27 Nov 2018

why the behaviour we're seeing doesn't match the bug description.

Bug description has one of the use cases where the issue is seen. Comment 5 in the bug report describes another use case where the issue is seen. There can be other use cases where this error is observed. The issue is related to libdl functions and lazy initialization of TLS. Refer to comment 6 and comment 7 in the bug report. Comment 6 has the fix where libdl functions that interact with TLS are moved in libc.so for direct usage of TLS. Comment 7 suggests that issues related to TLS are resolved in glibc 2.25 but there is still substantial work before dlopen-then-pthread_create works reliably. This issue was reported in glibc 2.17.

babsingh on 27 Nov 2018

Thanks for the links @babsingh. My point that we need to validate we're correctly linking libpthread in our test executables still stands. Let's rule out the easy / obvious things first.

DanHeidinga on 27 Nov 2018

In reply to
https://github.com/eclipse/openj9/issues/3672#issuecomment-442082731:

$ export VERSION_MAJOR=8 OPENJDK_VERSION_NUMBER_FOUR_POSITIONS=8.0.1920.0
$ make -C build/linux-x86_64-normal-server-slowdebug/vm/tests/redirector/jep178/ -B
/usr/bin/gcc -O3 -fno-strict-aliasing -fgnu89-inline -g -MMD -DLINUX -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -DIPv6_FUNCTION_SUPPORT -DJ9HAMMER -m64 -I. -I../../../include -I../../../oti -I../../../nls -I../../../omr/include_core    -DUT_DIRECT_TRACE_REGISTRATION -DTR_HOST_X86 -c -o copyright.o ../../../makelib/copyright.c
/usr/bin/gcc -O3 -fno-strict-aliasing -fgnu89-inline -g -MMD -DLINUX -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -DIPv6_FUNCTION_SUPPORT -DJ9HAMMER -m64 -I. -I../../../include -I../../../oti -I../../../nls -I../../../omr/include_core    -DUT_DIRECT_TRACE_REGISTRATION -DTR_HOST_X86 -c -o testjep178.o testjep178.c
/usr/bin/gcc  -L. -L../../../ -L../../../lib/  \
    copyright.o testjep178.o \
    -Xlinker --start-group \
      \
    -Xlinker --end-group \
      \
    -o ../../../testjep178_dynamic -lm -lpthread -lc -lrt -ldl -lutil -Wl,-z,origin,-rpath,\$ORIGIN,--disable-new-dtags,-rpath-link,../../../
$ ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23

pdbain-ibm on 27 Nov 2018

We can reproduce the problem using the code from
https://github.com/eclipse/openj9/issues/3672#issuecomment-441292519:

$ cat foo.c
#include <dlfcn.h>
#include <stdio.h>
int main(int argc, char *argv[]) {
    dlopen("bar", RTLD_NOW);
    printf("hello, world\n");
}
$ gcc foo.c -o foo -ldl -lpthread
$ gdb --args ./foo
(gdb) b _dlerror_run
<run until ~line 163>
(gdb) p static_buf
$1 = (struct dl_action_result *) 0x7ffff7dd60c0 <last_result>  <-- would be NULL if pthread initialized.

pdbain-ibm on 27 Nov 2018

Thanks for getting this info @pdbain-ibm
For the ldd command, I was thinking $ ldd testjep178_dynamic

Is https://github.com/eclipse/openj9/issues/3672#issuecomment-442174613 the case we're hitting? ie: trying to open a non-existent library?

I'm on board with adding a workaround to the test provided we understand what we're doing to provoke the failure so can help any user that hits similar issues.

DanHeidinga on 27 Nov 2018

Per @DanHeidinga 's request:

$ ldd  build/linux-x86_64-normal-server-slowdebug/vm/testjep178_static
    linux-vdso.so.1 =>  (0x00007fff36a55000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9f5d6e7000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9f5d4e3000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9f5dab1000)

Same thing for my test program:

$ ldd foo
    linux-vdso.so.1 =>  (0x00007ffc2c122000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4e44da7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4e449dd000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f4e44fab000)

Adding a dummy pthread call to foo.c and recompiling it, we get a reference to libpthread:

$ cat foo.c
#include <dlfcn.h>
#include <stdio.h>
#include <pthread.h>

static pthread_key_t key;
static pthread_once_t key_once = PTHREAD_ONCE_INIT;
static void unused_function()
{
    pthread_key_create(&key, NULL);
}
int main(int argc, char *argv[]) {
    dlopen("bar", RTLD_NOW);
    printf("hello, world\n");
}
$ gcc foo.c -o foo -ldl -lpthread
$ ldd foo
    linux-vdso.so.1 =>  (0x00007fffaed4b000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8ff6713000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8ff64f6000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ff612c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8ff6917000)

trying to open a non-existent library?

Testing with strace, the executable doesn't even try to open pthread . The modified foo.c does open the library when run:

$ strace ./foo 2>&1 | grep thread
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3

pdbain-ibm on 27 Nov 2018

@pdbain-ibm I'm not sure it makes a difference, but the ldd output is from testjep178_static while the rest of the discussion has been about testjep178_dynamic

trying to open a non-existent library?

I'm not familiar with the details of testjep178_dynamic so my (poorly worded) question was whether the test attempted to dlopen a library that didn't exist and if that was the case of the crash as per Comment 5 in the bug report

DanHeidinga on 27 Nov 2018

The question of dynamic vs static is irrelevant: we crash even on genlauncher during VM startup due to a race condition amongst initialization threads. The crash is not related to https://sourceware.org/bugzilla/show_bug.cgi?id=16628#c5: it is due to multiple conflicting accesses to a shared result buffer. I am in the process of tweaking the tests per option 2) (ugh!) and retesting.

pdbain-ibm on 27 Nov 2018

Tested a candidate "fix" manually with both the static and dynamic versions: 600 runs without a failure. Testing in a personal build on other platforms now...

pdbain-ibm on 28 Nov 2018

👍1

@DanHeidinga personal build ran: no failures related to this change. Pull request opened: https://github.com/eclipse/openj9/pull/3874
I took a look at the openj9 and test repositories: there are a few other places which call dlopen() without referring to pthread. I am not aware of failures in those tests, so I suggest leaving them for now.

pdbain-ibm on 29 Nov 2018

👍1

Was this page helpful?

0 / 5 - 0 ratings