Openjdk-infrastructure: openjdk_openj9_jdk11 jtreg tests are failing throughout the platforms - causes infrastructure isues

[archanaNogriya]

Here is AdoptOpenJDK openjdk_openj9_jdk11 builds for all platform: https://ci.adoptopenjdk.net/view/Test_openjdk/

openjdk_ppc64le_linux

Test Name | Duration | Age
-- | -- | --
java/util/logging/CheckZombieLockTest.java.CheckZombieLockTest | 7.2 sec | 1

openjdk_s390x_linux

java/net/httpclient/ConnectTimeoutNoProxyAsync.java.ConnectTimeoutNoProxyAsync | 10 sec | 22
-- | -- | --
java/net/httpclient/ConnectTimeoutNoProxyAsync.java.ConnectTimeoutNoProxyAsync | 12 sec | 22
java/net/httpclient/ConnectTimeoutNoProxyAsync.java.ConnectTimeoutNoProxyAsync |   |  

openjdk_x86-64_linux

Test Name | Duration | Age
-- | -- | --
java/util/stream/test/org/openjdk/tests/java/util/SplittableRandomTest.java.SplittableRandomTest | 56 sec | 4

openjdk_x86-64_mac_xl

Test Name | Duration | Age
-- | -- | --
java/util/UUID/UUIDTest.java.UUIDTest | 16 min | 1

openjdk_x86-64_windows

Test Name | Duration | Age
-- | -- | --
java/net/DatagramSocket/ReuseAddressTest.java.ReuseAddressTest | 3 sec | 2

.openjdk_x86-64_windows_xl

java/net/DatagramSocket/ReuseAddressTest.java.ReuseAddressTest | 2.7 sec | 2
-- | -- | --

I have also tested these above tests in my local machine Ubuntu 16.4 having same configuration and build as Adopt and all test passed.

I used this document to create the same setup as Adopt https://github.com/eclipse/openj9/wiki/Reproducing-Test-Failures-Locally

[karianna]

Not sure if that's _bad_ infra or bad tests...

[sxa]

Tried java/util/stream/test/org/openjdk/tests/java/util/SplittableRandomTest.java on test-scaleway-ubuntu1604-x64-1 and it passed on this JVM:

jenkins@test-scaleway-ubuntu1604-x64-1:/dev/shm/sxa/jdk-11.0.4+11/bin$ ./java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11-201908121810)
Eclipse OpenJ9 VM AdoptOpenJDK (build master-81195db9c, JRE 11 Linux amd64-64-Bit Compressed References 20190812_306 (JIT enabled, AOT enabled)
OpenJ9   - 81195db9c
OMR      - 6e99760b
JCL      - 965d0c0df3 based on jdk-11.0.4+11)

Up to 16 repeats without errors

[archanaNogriya]

So does that means individual tests are working fine but not as full sanity?

There is one common test which is failing all over java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java.JarHandlerPkgPrefix, This also failing in my local machine too, but passes in hotspot. I am debugging that bit more but what I am seeing is openj9 issue. I start wondering if this test is causing all failures. @smlambert any view in that?

[archanaNogriya]

Let me run Sanity in my local. It should fail with same test in that case.

[archanaNogriya]

OK I see Openj9 is already working on this test failure https://github.com/eclipse/openj9/issues/6466

Test java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java.JarHandlerPkgPrefix, they have done the fix but not yet delivered the changes until they run all tests to find any new regression.

Will update here once that is closed, my assumption is this could be the cause of all other test failures. I am running the sanity test in my local that should explain me well. Since individual tests is working fine in (our internal jenkins), in my Local and now as Adopt confirms it passed there too.

Except java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java which is failing in any case.

[karianna]

Should this whole issue be moved to the test repo?

[archanaNogriya]

I ran the full openJDk sanity tests in my local machine with the same build as Adopt nighly.
Build details
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11-201908061830)
Eclipse OpenJ9 VM AdoptOpenJDK (build master-85b0693e8, JRE 11 Linux amd64-64-Bit Compressed References 20190806_300 (JIT enabled, AOT enabled)
OpenJ9 - 85b0693e8
OMR - 6e99760b
JCL - 965d0c0df3 based on jdk-11.0.4+11)

All Test has been passed except java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java. I strongly believe that tests are good tests.

Note: All these tests are passing in Adopt from 18th July 2019. after that ever nightly build started failing with the same errors. So taking any nightly build won't change errors.

My Local Sanity test logs to see more details if you need.
test_output_15657742234865.zip

If they are bad tests I was hoping that it should fail in my local.

[archanaNogriya]

I ran same nightly build in IBM Internal Jenkin and got the same results as my Local machine, just one test failure java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java

[smlambert]

Hi @archanaNogriya, we have this guidelines to help triagers decide where to raise issues:
https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/doc/Triage.md

Normally we would not raise an issue in a repo until it was clearly indicating a certain category of problem (infra, test or product). Reading through your investigation so far, I am trying to see what had you categorize this as an infra issue?

Right now, I do not see anything here that the infra team can use to investigate an issue. Can you articulate why you created this issue in the infra repo?

Also, as a reminder, we typically do not share internal links as they are only accessible to a subset of people working in the open projects.

[sxa]

@archanaNogriya Can you address Shelley's comment above please?

[sxa]

@archanaNogriya I intend to close this in a few days unless you can confirm it's still an issue and that Shelley's queries as to why this should be in the infra repo are not addressed.

[archanaNogriya]

Hi @sxa555, Sorry for not coming to this. Later found that java/util/stream/test/org/openjdk/tests/java/util/SplittableRandomTest.java was excluded as it was JDK13 testcase which explained why it failed for 11. and rest of test works absolutely fine. so I dont see any point to keep this open.