So me and @M-Davies have been tracking any intermittent failing tests from https://ci.adoptopenjdk.net/ for a little while now, but this should be displayed publicly so that it can be seen and contributed by anyone else here.
Once a test fails more than a couple times, we can then investigate by looking which platforms/versions etc it failed on.
java/lang/ProcessBuilder/Basic.java#id0 jdk13-j9-ppc64le-linux 02-Dec-2019
Program '/home/jenkins/workspace/Test_openjdk13_j9_sanity.openjdk_ppc64le_linux/openjdkbinary/j2sdk-image/bin/java' timed out (timeout set to 2400000ms, elapsed time including timeout handling was 2402121ms).
java/lang/invoke/MethodHandlesSpreadArgumentsTest.java
jdk11-j9-x64-windows 03-Dec-2019
jdk11-j9-x64-windows 16-Dec-2019
jdk11-j9-x64-windows 19-Dec-2019
RASIED: https://github.com/eclipse/openj9/issues/8208
junit -XX:+IgnoreUnrecognizedVMOptions -XX:-VerifyDependencies -esa test.java.lang.invoke.MethodHandlesSpreadArgumentsTest
Unhandled exception
Type=Segmentation error vmState=0x00000000
Windows_ExceptionCode=c0000005 J9Generic_Signal=00000004 ExceptionAddress=00007FFD2D4EB95D ContextFlags=0010005f
...
TEST RESULT: Failed. Unexpected exit from test [exit code: -1]
java/net/httpclient/RestrictedHeadersTest.java jdk13-j9-x64-mac 03-Dec-2019
Error: Port Library failed to initialize: -1
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
TEST RESULT: Failed. Unexpected exit from test [exit code: 1]
java/util/zip/ZipFile/TestCleaner.java
jdk11-j9-x64-mac 04-Dec-2019
jdk11-j9-s390x-linux 04-Dec-2019
jdk11-j9-s390x-linux 12-Dec-2019
jdk11-j9-x64-macXL 19-Dec-2019
Agent error: java.lang.Exception: Agent 0 timed out with a timeout of 960 seconds; check console log for any additional details
Running jstack on process 45583
2019-12-04T18:09:21.047972
Error getting data from 45583: Exception connecting to 45583
java/io/RandomAccessFile/OpenSync.java jdk11-j9-s390x-linux 12-Dec-2019
Mode "d": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "d" must be one of "r", "rw", "rws", or "rwd"
Mode "s": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "s" must be one of "r", "rw", "rws", or "rwd"
Mode "rd": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "rd" must be one of "r", "rw", "rws", or "rwd"
Mode "rs": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "rs" must be one of "r", "rw", "rws", or "rwd"
Mode "rwx": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "rwx" must be one of "r", "rw", "rws", or "rwd"
Mode "foo": Thrown as expected: java.lang.IllegalArgumentException
Illegal mode "foo" must be one of "r", "rw", "rws", or "rwd"
java.lang.NullPointerException
at java.base/java.io.DataOutputStream.writeUTF(DataOutputStream.java:353)
at java.base/java.io.RandomAccessFile.writeUTF(RandomAccessFile.java:1177)
at OpenSync.main(OpenSync.java:60)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
at java.base/java.lang.Thread.run(Thread.java:834)
java/net/httpclient/ShortResponseBody.java jdk11-j9-ppc64le-linuxXL 12-Dec-2019
Test Failure unclear. 1/379 failed
java/util/regex/RegExTest.java jdk13-j9-x64-linux 17-Dec-2019
07:12:03 Invalid compile flags : Passed
07:12:03 java.lang.NullPointerException
07:12:03 at java.base/java.util.regex.Pattern.peek(Pattern.java:1892)
07:12:03 at java.base/java.util.regex.Pattern.sequence(Pattern.java:2118)
07:12:03 at java.base/java.util.regex.Pattern.expr(Pattern.java:2068)
07:12:03 at java.base/java.util.regex.Pattern.compile(Pattern.java:1782)
07:12:03 at java.base/java.util.regex.Pattern.<init>(Pattern.java:1429)
07:12:03 at java.base/java.util.regex.Pattern.compile(Pattern.java:1069)
07:12:03 at RegExTest.embeddedFlags(RegExTest.java:4763)
07:12:03 at RegExTest.main(RegExTest.java:183)
07:12:03 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
07:12:03 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
07:12:03 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
07:12:03 at java.base/java.lang.reflect.Method.invoke(Method.java:567)
07:12:03 at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
07:12:03 at java.base/java.lang.Thread.run(Thread.java:847)
07:12:26 [2019-12-17 07:12:24,044] Agent[0]: stderr: JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2019/12/17 07:12:24 - please wait.
openjdk11_hs_sanity.openjdk_aarch64_linux, VarHandles tests:
java/lang/invoke/VarHandles/VarHandleTestAccessBoolean.java.VarHandleTestAccessBoolean
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessLong.java.VarHandleTestMethodHandleAccessLong
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessShort.java.VarHandleTestMethodHandleAccessShort
sun/net/www/AuthHeaderTest.java jdk8-j9-s390x-linux 18-Dec-2019
Server: listening on port: 35395
client opening connection to: http://localhost:35395/d1/foo.html
SERVER: new connection. chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56124]]
SERVER: connection readable. chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56124]]
Basic dXNlcjpwYXNzd29yZE5vdENoZWNrZWRBbnl3YXk=
SERVER: orderlyCloseChannel chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56124]]
SERVER: connection readable. chan[java.nio.channels.SocketChannel[connected oshut local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56124]]
Server: comsume
SERVER: new connection. chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56126]]
SERVER: connection readable. chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56126]]
Basic dXNlcjpwYXNzd29yZE5vdENoZWNrZWRBbnl3YXk=
SERVER: orderlyCloseChannel chan[java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:35395 remote=/127.0.0.1:56126]]
reading
Hello .
finished reading
java.lang.RuntimeException: Authenticator was called 0 times. Should be 1
at AuthHeaderTest.except(AuthHeaderTest.java:117)
at AuthHeaderTest.main(AuthHeaderTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
at java.lang.Thread.run(Thread.java:821)
I am going to assume the plan is to:
Pretty much yeah, it's so that we can determine between actual intermittent failures and tests that just fail on a whim. Once we've decided it's an actual issue, we can investigate as you stated above
sun/nio/cs/FindDecoderBugs.java jdk11-j9-s390x-linux 19-Dec-2019
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ed"[0/5] => null ""[0/2]
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ed"[0/5] => null ""[0/2] (direct)
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ee"[0/5] => null ""[0/19]
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ee"[0/5] => null ""[0/19] (direct)
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ef"[0/5] => null ""[0/19]
01:45:05 Unexpected exception: x-JISAutoDetect NullPointerException "1b 28 42 a4 ef"[0/5] => null ""[0/19] (direct)
01:45:05 Too many failures for x-JISAutoDetect
https://github.com/AdoptOpenJDK/openjdk-tests/issues/1511#issuecomment-566062160
java/io/RandomAccessFile/OpenSync.java jdk11-j9-s390x-linux 12-Dec-2019
java/io/RandomAccessFile/OpenSync.java jdk13-j9_x86-64-linux 6-Jan-2020 (Count 2)
we have several intermittent test failure issues open in the repo, can/should they be collapsed them into one?
https://github.com/AdoptOpenJDK/openjdk-tests/issues/821
https://github.com/AdoptOpenJDK/openjdk-tests/issues/1259
https://github.com/AdoptOpenJDK/openjdk-tests/issues/1008
@smlambert for #821 and #1259 I would agree as they are also collections of failing tests, I can already see some duplicates. The others are more specific issues about a single test so shouldn't we leave those alone? In this issue we're just collecting more information about the test and then creating its own separate issue when we can determine what the problem seems to be (eg. openj9, infra etc).
java/lang/ProcessHandle/TreeTest.java jdk13-j9-x64-windows-xl 8-Jan-2020
test TreeTest.test5(): failure
20:20:27 java.lang.AssertionError: expected at least 14, actual: 11 expected [true] but found [false]
20:20:27 at org.testng.Assert.fail(Assert.java:94)
20:20:27 at org.testng.Assert.failNotEquals(Assert.java:496)
20:20:27 at org.testng.Assert.assertTrue(Assert.java:42)
20:20:27 at TreeTest.test5(TreeTest.java:451)
20:20:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
20:20:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
20:20:27 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
20:20:27 at java.base/java.lang.reflect.Method.invoke(Method.java:567)
20:20:27 at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
20:20:27 at org.testng.internal.Invoker.invokeMethod(Invoker.java:639)
20:20:27 at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:821)
20:20:27 at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1131)
20:20:27 at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
20:20:27 at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
20:20:27 at org.testng.TestRunner.privateRun(TestRunner.java:773)
20:20:27 at org.testng.TestRunner.run(TestRunner.java:623)
20:20:27 at org.testng.SuiteRunner.runTest(SuiteRunner.java:357)
20:20:27 at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352)
20:20:27 at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310)
20:20:27 at org.testng.SuiteRunner.run(SuiteRunner.java:259)
20:20:27 at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
20:20:27 at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
20:20:27 at org.testng.TestNG.runSuitesSequentially(TestNG.java:1185)
20:20:27 at org.testng.TestNG.runSuitesLocally(TestNG.java:1110)
20:20:27 at org.testng.TestNG.run(TestNG.java:1018)
20:20:27 at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:94)
20:20:27 at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:54)
20:20:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
20:20:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
20:20:27 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
20:20:27 at java.base/java.lang.reflect.Method.invoke(Method.java:567)
20:20:27 at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
20:20:27 at java.base/java.lang.Thread.run(Thread.java:847)
java/security/KeyRep/Serial jdk8-j9-s390x-linuxXL 10-Jan-2020
java.lang.SecurityException: PBE test failed
at Serial.main(Serial.java:168)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.lang.Thread.run(Thread.java:821)
java/net/httpclient/DependentPromiseActionsTest.java
jdk13-j9-x64-linux 09-Jan-2020
jdk11-j9-x64-linux 12-Jan-2020
testng -Djdk.internal.httpclient.debug=true DependentPromiseActionsTest
20:37:40 test DependentPromiseActionsTest.testAsStringAsync("http://localhost:35296/http2/chunk/y", true, SemaphoreStaller): failure
20:37:40 java.lang.AssertionError
20:37:40 at DependentPromiseActionsTest.testDependent(DependentPromiseActionsTest.java:366)
20:37:40 at DependentPromiseActionsTest.testDependent(DependentPromiseActionsTest.java:336)
20:37:40 at DependentPromiseActionsTest.testAsStringAsync(DependentPromiseActionsTest.java:295)
java/lang/invoke/RevealDirectTest.java jdk8-j9-x64-windows 12-Jan-2020
RAISED: https://github.com/eclipse/openj9/issues/8268
08:12:32 *** FAIL: public final native java.lang.invoke.MethodHandle.invoke(Object[])Object => java.lang.AssertionError: expected:<4369> but was:<401>
08:12:32 java.lang.AssertionError: expected:<4369> but was:<401>
08:12:32 at org.junit.Assert.fail(Assert.java:93)
08:12:32 at org.junit.Assert.failNotEquals(Assert.java:647)
08:12:32 at org.junit.Assert.assertEquals(Assert.java:128)
08:12:32 at org.junit.Assert.assertEquals(Assert.java:472)
08:12:32 at org.junit.Assert.assertEquals(Assert.java:456)
08:12:32 at test.java.lang.invoke.RevealDirectTest.consistent(RevealDirectTest.java:639)
08:12:32 at test.java.lang.invoke.RevealDirectTest.testWithMember(RevealDirectTest.java:718)
08:12:32 at test.java.lang.invoke.RevealDirectTest.testOnMembersImpl(RevealDirectTest.java:384)
08:12:32 at test.java.lang.invoke.RevealDirectTest.testOnMembers(RevealDirectTest.java:357)
08:12:32 at test.java.lang.invoke.RevealDirectTest.testMethodHandleInvokes(RevealDirectTest.java:184)
java/util/concurrent/ScheduledThreadPoolExecutor/BasicCancelTest.java
jdk11-j9-x64-macXL 12-Jan-2020
22:19:14 Timeout information:
22:19:14 Running jstack on process 50361
22:19:14 2020-01-12T22:19:08.670622
22:19:14 Error getting data from 50361: Exception connecting to 50361
22:19:14 Passed = 13, failed = 0
java/util/Locale/LocaleProvidersRun.java jdk11-j9-x64-mac 13-Jan-2020
23:40:27 Unhandled exception
23:40:27 Type=Segmentation error vmState=0x0005ffff
23:40:27 J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000000
23:40:27 STDERR:
23:40:27 java.lang.RuntimeException: Unexpected exit code: 255
23:40:27 at LocaleProvidersRun.testRun(LocaleProvidersRun.java:172)
23:40:27 at LocaleProvidersRun.main(LocaleProvidersRun.java:93)
23:40:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
23:40:27 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
23:40:27 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
23:40:27 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
23:40:27 at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
23:40:27 at java.base/java.lang.Thread.run(Thread.java:834)
java/nio/file/WatchService/LotsOfCancels.java.LotsOfCancels
12:15:28 LotsOfCancels
12:15:28 STDERR:
12:15:28 java.lang.RuntimeException: Thread pool did not terminate
12:15:28 at LotsOfCancels.main(LotsOfCancels.java:67)
12:15:28 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
12:15:28 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
12:15:28 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
12:15:28 at java.lang.reflect.Method.invoke(Method.java:498)
12:15:28 at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
12:15:28 at java.lang.Thread.run(Thread.java:821)
12:15:28
12:15:28 JavaTest Message: Test threw exception: java.lang.RuntimeException
12:15:28 JavaTest Message: shutting down test
12:15:28
Platform
s390x_linux
Detail
sanity.openjdk
java/rmi/activation/rmidViaInheritedChannel/InheritedChannelNotServerSocket.java.InheritedChannelNotServerSocket
java.rmi.server.ExportException: Port already in use: 60004; nested exception is:
13:28:12 java.net.BindException: Address already in use (Bind failed)
13:28:12 at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
13:28:12 at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
13:28:12 at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412)
13:28:12 at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
13:28:12 at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:237)
13:28:12 at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
13:28:12 at sun.rmi.registry.RegistryImpl.
13:28:12 at java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:203)
13:28:12 at InheritedChannelNotServerSocket.main(InheritedChannelNotServerSocket.java:91)
13:28:12 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java/util/TimeZone/TimeZoneTest.java Test_openjdk8_hs_sanity.openjdk_x86-64_linux 4-2-2020
Looks like a jenkins issue, test passes locally #1356
TimeZoneTest {
TestShortZoneIDs {
Fail: Expected Zone["BET", GMT-3:00, Standard]; got Zone["BET", GMT-3:00, Daylight]
Uncaught exception thrown in test method TestShortZoneIDs
java.lang.RuntimeException: Uncaught exception thrown in test method TestShortZoneIDs
at IntlTest.errImpl(IntlTest.java:190)
at IntlTest.errln(IntlTest.java:177)
at IntlTest.run(IntlTest.java:123)
at TimeZoneTest.main(TimeZoneTest.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
at java.lang.Thread.run(Thread.java:748)
New list for Feb - https://github.com/AdoptOpenJDK/openjdk-tests/issues/1596 can you copy that over @M-Davies ?