Openj9: Test_openjdk8_j9_sanity.system_ppc64le_linux MauveMultiThreadLoadTest_OpenJ9_0 Date.range (number 5)

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/70/
MauveMultiThreadLoadTest_OpenJ9_0

LT  FAIL: gnu.testlet.java.util.Date.range (number 5)

01:42:01  To rebuild the failed test in a jenkins job, copy the following link and fill out the <Jenkins URL> and <FAILED test target>:
01:42:01  <Jenkins URL>/parambuild/?JDK_VERSION=8&JDK_IMPL=openj9&BUILD_LIST=system/mauveLoadTest&Jenkinsfile=openjdk_ppc64le_linux&TARGET=<FAILED test target>

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/267
MauveMultiThreadLoadTest_special_26
variation: Mode107-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xgcpolicy:optthruput -Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,onthrow=no.pkg.foo,launch=echo -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation

LT  01:17:17.938 - Test failed
LT    Failure num.  = 1
LT    Test number   = 3241
LT    Test details  = 'Mauve[gnu.testlet.java.util.Date.range]'
LT    Suite number  = 0
LT    Thread number = 13
LT  >>> Captured test output >>>
LT  PASS: gnu.testlet.java.util.Date.range (number 0)
LT  FAIL: gnu.testlet.java.util.Date.range (number 1)

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/294
MauveMultiThreadLoadTest_special_20
00:51:38 variation: Mode557
00:51:38 JVM_OPTIONS: -Xcompressedrefs -XX:+UseCompressedOops -Xgcpolicy:balanced -Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,onthrow=no.pkg.foo,launch=echo -Xjit:count=0

00:52:09  LT  00:52:08.428 - Test failed
00:52:09  LT    Failure num.  = 1
00:52:09  LT    Test number   = 3241
00:52:09  LT    Test details  = 'Mauve[gnu.testlet.java.util.Date.range]'
00:52:09  LT    Suite number  = 0
00:52:09  LT    Thread number = 3
00:52:09  LT  >>> Captured test output >>>
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 0)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 1)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 2)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 3)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 4)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 5)
00:52:09  LT  PASS: gnu.testlet.java.util.Date.range (number 6)
00:52:09  LT  FAIL: gnu.testlet.java.util.Date.range (number 7)

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/303
MauveMultiThreadLoadTest_special_20

LT  00:54:30.418 - Test failed
LT    Failure num.  = 1
LT    Test number   = 3241
LT    Test details  = 'Mauve[gnu.testlet.java.util.Date.range]'
LT    Suite number  = 0
LT    Thread number = 6

[pshipton]

@gita-omr fyi, this seems to happen fairly often recently, it would be good to take a look.

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/312
MauveMultiThreadLoadTest_special_20
variation: Mode612-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xgcpolicy:gencon -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation

LT  00:38:33.714 - Test failed
LT    Failure num.  = 1
LT    Test number   = 3241
LT    Test details  = 'Mauve[gnu.testlet.java.util.Date.range]'
LT    Suite number  = 0
LT    Thread number = 4

[pshipton]

fyi it did occur on zLinux as well https://github.com/eclipse/openj9/issues/6803, but happens on pLinux much more often.

[gita-omr]

@aviansie-ben if you make a comment here I will be able to assign this issue to you.

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/366
ub16p8j93
MauveMultiThreadLoadTest_special_26
variation: Mode612-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xgcpolicy:gencon -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation

[aviansie-ben]

I've been trying to look into this on and off for the last week or so, but I unfortunately haven't had any luck so far. Since this is a test failure rather than a crash, there's no corefile for me to examine. The failing test also doesn't seem to be logging the actual value it's getting, so I can't get any leads from that. I've tried reproducing the problem in a controlled environment, but I haven't been able to do so thus far.

Without any of the normal information we'd need to debug this sort of failure, it's not going to be possible to determine what's going on. Beyond continuing to attempt to reproduce the problem in an environment where I can hopefully examine and debug it, I'm not sure there's much I can do here.

[pshipton]

Trying a grinder at openj9
https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/513/ - ub16-ppcle-1 5x

[pshipton]

https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/517/ - ub16-ppcle-1 10x
https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/518/ - ub16p8j93 5x
all passed

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/392/
ub16p8j93
MauveMultiThreadLoadTest_special_23
variation: Mode107-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xgcpolicy:optthruput -Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,onthrow=no.pkg.foo,launch=echo -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/404
ub16p8j93
MauveMultiThreadLoadTest_special_19
variation: Mode610
JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xjit -Xgcpolicy:gencon

[pshipton]

The failure always seems to occur on the same machine.

[Mesbah-Alam]

Similar failure in MauveMultiThreadLoadTest_special_19 (JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xjit -Xgcpolicy:gencon)

On same machine: ub16p8j93

FAIL: gnu.testlet.java.util.Date.range (number 7)

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/404/tapResults/

[pshipton]

@Mesbah-Alam I had already reported this failure https://github.com/eclipse/openj9/issues/6236#issuecomment-534080197
I am monitoring all the failures and reporting the issues.

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/416/

ub16p8j92 - different machine this time

MauveMultiThreadLoadTest_special_14
variation: Mode553
JVM_OPTIONS: -Xcompressedrefs -XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:count=0

MauveMultiThreadLoadTest_special_25
variation: Mode610-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation -Xgcpolicy:gencon

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/434
MauveMultiThreadLoadTest_special_26
variation: Mode612-OSRG
JVM_OPTIONS: -Xcompressedrefs -Xcompressedrefs -Xgcpolicy:gencon -Xjit:enableOSR,enableOSROnGuardFailure,count=1,disableAsyncCompilation
ub16p8j93

[pshipton]

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_special.system_ppc64le_linux_Nightly/458
MauveMultiThreadLoadTest_special_14
variation: Mode553
JVM_OPTIONS: -Xcompressedrefs -XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:count=0
ub16p8j95

[pshipton]

@JasonFengJ9 can you pls take a look at the test code for this one. Maybe it needs to be excluded similarly to Calendar.getInstance().

[JasonFengJ9]

Looked at the test code in question:

    check(70, 0, 1, 0, 0, 0, 0);
    check(104, 9, 12, 0, 0, 0, 1097539200000L);
    check(104, 9, 12, 0, 0, 0, 1097539200000L);
    check(104, 9, 12, 12, 34, 0, 1097584440000L);
    check(104, 9, 12, 12, 34, 56, 1097584496000L);
    check(104, -1, 0, 0, 0, 0, 1070150400000L);
    check(104, 99, 99, 99, 99, 99, 1342068039000L);
    check(104, 999, 999, 999, 999, 999, 3789878139000L);
    check(104, -1, -1, -1, -1, -1, 1070060339000L);
    check(104, -999, -999, -999, -999, -999, -1644306939000L);

  private void check(int year, int month, int day, int hours, int mins, int secs, long l)
  {
    try
      {
        Date d = new Date(year, month, day, hours, mins, secs);
        harness.check(d.getTime() - d.getTimezoneOffset() * 60 * 1000 == l);  <-- failed here
      }
    catch (Throwable t)
      {
        harness.debug(t);
        harness.check(false);
      }
  }

java.util.Date.getTimezoneOffset() code snippets:

    public int getTimezoneOffset() {
        int zoneOffset;
        if (cdate == null) {
            TimeZone tz = TimeZone.getDefaultRef();  <-- this could be affected by another thread TimeZone.setDefault(TimeZone zone)
            if (tz instanceof ZoneInfo) {
                zoneOffset = ((ZoneInfo)tz).getOffsets(fastTime, null);
            } else {
                zoneOffset = tz.getOffset(fastTime);
            }
        } else {
            normalize();
            zoneOffset = cdate.getZoneOffset();
        }
        return -zoneOffset/60000;  // convert to minutes
    }

Similarly with https://github.com/eclipse/openj9/issues/7364#issuecomment-538803439, this is a test issue as well.

[pshipton]

Thanks Jason.

@Mesbah-Alam can you please exclude this test in addition to #7364

[pshipton]

Should be resolved by https://github.com/AdoptOpenJDK/openjdk-systemtest/pull/299