Openjdk-tests: jdk8 openj9 special.openjdk fail, "JDK not found"

[andrew-m-leonard]

Describe the bug
Seems a recent change in location of the JDK8 image is causing the error below, not clear why as location exists I think...? The JAVA_HOME has the build level in it now, which it didn't used to have... this job is failing since that change...?
j2sdk-image/jdk8u252-b07/bin/../

To Reproduce
https://ci.adoptopenjdk.net/view/Test_openjdk_special/job/Test_openjdk8_j9_special.openjdk_s390x_linux/71/console
Run a Test_openjdk8_j9_special.openjdk_s390x_linux, any platform fails (except Mac)

Expected behavior
JDK is found

Screenshots

19:50:32  cd "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../TKG/test_output_15852522321835/jdk_math_jre_0"; \
19:50:32  "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdkbinary/j2sdk-image/jdk8u252-b07/bin/..//bin/java" -Xmx512m -jar "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../../jvmtest/openjdk/jtreg/lib/jtreg.jar" \
19:50:32  -agentvm -a -ea -esa -v:fail,error,time,nopass -retain:fail,error,*.dmp,javacore.*,heapdump.*,*.trc -ignore:quiet -timeoutFactor:8 -xml:verify -concurrency:2 -vmoptions:"-Xmx512m -Xcompressedrefs " \
19:50:32  -w ""/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../TKG/test_output_15852522321835/jdk_math_jre_0"/work" \
19:50:32  -r "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../../jvmtest/openjdk/report" \
19:50:32  -compilejdk:"/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdkbinary/j2sdk-image/jdk8u252-b07/bin/../" \
19:50:32  -jdk:"/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdkbinary/j2re-image/jdk8u252-b07/bin/../" \
19:50:32  -exclude:"/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../openjdk/openjdk-jdk/jdk/test/ProblemList.txt" \
19:50:32  -exclude:"/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../../jvmtest/openjdk/ProblemList_openjdk8-openj9.txt" \
19:50:32  "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../openjdk/openjdk-jdk/jdk/test:jdk_math"; \
19:50:32  if [ $? -eq 0 ] ; then echo ""; echo "jdk_math_jre_0""_PASSED"; echo ""; cd /home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/..;  else echo ""; echo "jdk_math_jre_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdk-tests/TKG/../TKG/test_output_15852522321835/TestTargetResult";
19:50:32  Error: JDK not found: /home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_s390x_linux/openjdkbinary/j2re-image/jdk8u252-b07/bin/..

[andrew-m-leonard]

@smlambert hoping you may know where this change is coming from?
but it looks as though the extra get.sh openjdk zip file downloads maybe causing the issue?
When the job was working we had just:

20:41:18  get jdk binary...
20:41:18  --sdkdir is set to upstream. Therefore, skip download jdk binary
20:41:18  unzip file: OpenJDK8U-jdk_x64_linux_openj9_2020-03-24-17-36.tar.gz ...
20:41:23  unzip file: OpenJDK8U-jre_x64_linux_openj9_2020-03-24-17-36.tar.gz ...

we now have:

18:00:43  get jdk binary...
18:00:43  --sdkdir is set to upstream. Therefore, skip download jdk binary
18:00:43  unzip file: OpenJDK8U-debugimage_x64_linux_openj9_2020-03-26-17-08.tar.gz ...
18:00:43  unzip file: OpenJDK8U-jdk_x64_linux_openj9_2020-03-26-17-08.tar.gz ...
18:00:49  unzip file: OpenJDK8U-jre_x64_linux_openj9_2020-03-26-17-08.tar.gz ...
18:00:51  unzip file: OpenJDK8U-testimage_x64_linux_openj9_2020-03-26-17-08.tar.gz ...
18:00:51  /home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_x86-64_linux/openjdkbinary/j2sdk-image/bin/java does not exist! Searching under TEST_JDK_HOME: /home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_x86-64_linux/openjdkbinary/j2sdk-image...
18:00:51  javac_path: /home/jenkins/workspace/Test_openjdk8_j9_special.openjdk_x86-64_linux/openjdkbinary/j2sdk-image/jdk8u252-b07/bin/javac

testimage & debugimage added...?

[sophia-guo]

Yes, I think that is the issue. OpenJ9 build with extra testimage and debugimages now. And this should be fixed by https://github.com/AdoptOpenJDK/openjdk-tests/pull/1692 ? @llxia

[andrew-m-leonard]

@sophia-guo ah I see yes, the debugimage output is an "empty" tar.gz but gets extracted to the "j2sdk-image" directory, subsequent zip for the actual jdk tries to do the "mv $jar_dir_name ../j2dsk-image" will fail as the directory already exists...?

You may want to run a test of your new code against Test_openjdk8_j9_special.openjdk_x86-64_linux as that does fail as it stands.
thanks

[andrew-m-leonard]

@llxia ^

[sophia-guo]

@andrew-m-leonard see test channel. This should be fixed by https://github.com/AdoptOpenJDK/openjdk-api-v3/pull/134 for now and I think we could close it.

[M-Davies]

@sophia-guo Sorry I still seem to be having this issue. Please see TRSS https://trss.adoptopenjdk.net/resultSummary?parentId=5e82202a214b905e3b6c0a9f for the jdk8-m2 builds.

Ref https://github.com/AdoptOpenJDK/openjdk-tests/issues/1703

[llxia]

AdoptOpenJDK/openjdk-api-v3#134 should exclude debugimage, but I still see debugimage is produced and downloaded in the test build
https://ci.adoptopenjdk.net/job/Test_openjdk8_j9_special.openjdk_s390x_linux/76/

13:12:36  unzip file: OpenJDK8U-debugimage_s390x_linux_openj9_8u252b08_openj9-0.20.0-m2.tar.gz ...
13:12:37  unzip file: OpenJDK8U-jdk_s390x_linux_openj9_8u252b08_openj9-0.20.0-m2.tar.gz ...
13:12:38  unzip file: OpenJDK8U-jre_s390x_linux_openj9_8u252b08_openj9-0.20.0-m2.tar.gz ...
13:12:40  unzip file: OpenJDK8U-testimage_s390x_linux_openj9_8u252b08_openj9-0.20.0-m2.tar.gz ...

@johnoliver does AdoptOpenJDK/openjdk-api-v3#134 only exclude debugimage in API calls?

[sophia-guo]

Those builds are triggered by upstream. Hence directly grab all build jar files ( CopyArtifacts )from the upstream builds, which do not download the jdks or others by openjdk-api.

[sophia-guo]

In AdoptOpenJDK test builds triggered by upstream is different from the tests build triggered by Eclipse Openj9. Eclipse Openj9 tests builds are triggered by upstream and use CUSTOMIZED_SDK_URL.

So https://github.com/AdoptOpenJDK/openjdk-tests/pull/1692 may not fix this issue, correct? @llxia?

[llxia]

Due to the inconsistent naming in Openj9 and Adopt. PR #1692 only addresses debug-image, not debugimage. I can do a quick fix now and remove it once we have the name fixed.

[llxia]

PR #1706 is merged and the next build is green https://ci.adoptopenjdk.net/view/Test_openjdk_special/job/Test_openjdk8_j9_special.openjdk_s390x_linux/78/

Close this issue.

[M-Davies]

Thanks @llxia :)