Openj9: AdoptOpenJDK + OpenJ9 11 to 15 java process hanging on mac os catalina

Created on 14 Oct 2020 · 21Comments · Source: eclipse/openj9

Java -version output

openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.2+12)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 14 Mac OS X amd64-64-Bit Compressed References 20200715_143 (JIT enabled, AOT enabled)
OpenJ9 - 34cf4c075
OMR - 113e54219
JCL - 1d231bd6a2 based on jdk-14.0.2+12)

Summary of problem

I am running a spring boot application and when doing control + C or stopping the application from within IntelliJ a java process keeps hanging and I need to kill it manually. I have tried version 11, 14 and 15 and the same happens.

I have attached screenshots for reference from the terminal.

If I run OpenJDK without OpenJ9 everything works fine.

Also, it could be something similar to this one https://github.com/eclipse/openj9/issues/3468

Please, can someone take a look at it?

Thank you,
Vitor

Diagnostic files

Screen Shot 2020-10-14 at 14 08 30

Screen Shot 2020-10-14 at 14 10 21

vm userRaised

Source

vitorpavanelli

Most helpful comment

I was able to replicate this issue on Mac and zOS

dusanboskovic on 20 Oct 2020

👍2

All 21 comments

@tajila @babsingh fyi

pshipton on 14 Oct 2020

@vitorpavanelli Is the issue reproducible with any spring boot application? If not, can you provide your spring boot application for reproducing the issue?

babsingh on 19 Oct 2020

hi @babsingh! yes, you can go ahead and create a simple example from spring initialiser. You will notice when trying doing ctrl + c the java process won't quit. It seems whatever is waiting for the kill signal is not receiving or something got stuck. I even had this issue when running within IntelliJ IDE where I had to click twice on the stop button to actually kill the application.

vitorpavanelli on 19 Oct 2020

👍1

Thanks, @vitorpavanelli. We will look into this issue. fyi @dusanboskovic ^^^.

babsingh on 19 Oct 2020

👍1

I was able to replicate this issue on Mac and zOS

dusanboskovic on 20 Oct 2020

👍2

I was able to replicate the issue on Linux and zOS using a test springboot application. Narrowed down the problem between these 2 IBM Java builds
SR6_FP11 - built June 1st 2020
SR6_FP15 - built July 24th 2020
On the open side, matched the dates from these builds with the nightly builds within the same time frame: https://adoptopenjdk.net/nightly.html?variant=openjdk8&jvmVariant=openj9
Narrowed it down to these builds:

_Works as expected (Nightly build June 15):_

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006151723-b06)
Eclipse OpenJ9 VM (build master-cec5b1c53, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200615_679 (JIT enabled, AOT enabled)
OpenJ9   - cec5b1c53
OMR      - 1db16ee21
JCL      - a6d49b5699 based on jdk8u262-b06)

_Sometimes works (Nightly build June 16)_

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006161721-b06)
Eclipse OpenJ9 VM (build master-77c3af970, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200616_680 (JIT enabled, AOT enabled)
OpenJ9   - 77c3af970
OMR      - fa22a8385
JCL      - a6d49b5699 based on jdk8u262-b06)

_Never works (nightly build June 18)_

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006171720-b07)
Eclipse OpenJ9 VM (build master-50ba90c72, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200617_681 (JIT enabled, AOT enabled)
OpenJ9   - 50ba90c72
OMR      - b884d8266
JCL      - 8d231b1446 based on jdk8u262-b07)

These are the git compare SHAs:
'https://github.com/eclipse/openj9/compare/cec5b1c53...50ba90c72'
'https://github.com/eclipse/openj9-omr/compare/1db16ee21...b884d8266' <-- probably nothing here
'https://github.com/ibmruntimes/openj9-openjdk-jdk8/compare/a6d49b5699...8d231b1446' <-- probably nothing here as well

Reverted @gacholio change: https://github.com/eclipse/openj9/commit/ac6141ad13bd9bfa990c1c34a149bd64f33c1673
on the customer’s version that doesn’t work (SR6_FP15), and now the SIGTERM is properly handled
FYI: @babsingh @tajila @gacholio

Steps to reproduce

Very simple, you can use my springboot jar and run with any Java version above the IBM Java level SR6_FP15 or the Eclipse OpenJ9 release after the 17th of June.
Use the following command to execute the jar file: <path_to_java>/bin/java -jar spring-boot-0.0.1-SNAPSHOT.jar
On another terminal of the same machine, run ps -ef | grep spring and send a kill -15 <PID> to the springboot process. It should hang on anything above the releases mentioned above, but it should exit properly on anything before the releases above
spring-boot-0.0.1-SNAPSHOT.jar.zip

dusanboskovic on 21 Oct 2020

👍1

Summary: Reverting https://github.com/eclipse/openj9/pull/9885 resolves the hang.

babsingh on 21 Oct 2020

That code has already been reverted. Current builds should work.

gacholio on 21 Oct 2020

👍1

Ran the test scenario with a nightly build from the 20th of October and SIGTERM was properly handled. Seems like the fix will be officially released in one of the next versions of Eclipse OpenJ9

dusanboskovic on 21 Oct 2020

Changes in #9885 were reverted in https://github.com/eclipse/openj9/pull/10328 (4 Aug 2020).

babsingh on 21 Oct 2020

Two PRs fixed this - https://github.com/eclipse/openj9/pull/10273 and https://github.com/eclipse/openj9/pull/10328

gacholio on 21 Oct 2020

JDK14 is no longer supported since it only had a 6 month support life cycle. The latest release of JDK15 replaces JDK14 and was released on 23 Sept 20. It should have the above fixes.

The latest release of macOS x64 JDK 15 can be downloaded from Adopt: https://adoptopenjdk.net/archive.html?variant=openjdk15&jvmVariant=openj9.

@vitorpavanelli Can you please close this issue after verifying that the latest release of JDK15 fixes the hang?

babsingh on 21 Oct 2020

Hi @babsingh,

15 doesn't work. I actually installed it first and then tried older versions to make sure it wasn't an issue with 15. I understand 14 it isn't supported but if there will be no fixes for 11, 12, 13 and 14 then it should be removed from download page as well. At least 11 must be supported because it LTS.

Thank you for your hard work on this.

vitorpavanelli on 21 Oct 2020

@vitorpavanelli Can you please try the latest nightly build? The fix has not yet made it into a public release.

gacholio on 21 Oct 2020

Nightly builds: https://adoptopenjdk.net/nightly.html?variant=openjdk15&jvmVariant=openj9. Latest one was generated on 21 Oct 20.

At least 11 must be supported because it LTS.

LTS versions (8 & 11) will be supported along with JDK-latest. I didn't realize that the fix was not delivered to a release branch yet.

babsingh on 21 Oct 2020

it should be removed from download page

The old unsupported JDKs (9, 12, 13 and 14) are not publicized at Adopt's home page: https://adoptopenjdk.net/index.html?variant=openjdk8&jvmVariant=openj9. Only LTS (8 & 11) and JDK-latest versions are publicized.

I don't think the old unsupported JDKs will be removed. They will be retained for future reference. But, they won't be publicized at Adopt.

babsingh on 21 Oct 2020

it should be removed from download page

The old unsupported JDKs (9, 12, 13 and 14) are not publicized at Adopt's home page: https://adoptopenjdk.net/index.html?variant=openjdk8&jvmVariant=openj9. Only LTS (8 & 11) and JDK-latest versions are publicized.

I was able to access them in another manner, I don't recall where but anyway.

Your fix did work.

Thank you

vitorpavanelli on 21 Oct 2020

👍1

when can I download the release version, please?

vitorpavanelli on 21 Oct 2020

@pshipton When do you expect 0.23 to be available?

gacholio on 21 Oct 2020

I hope it will be available on Friday if nothing goes wrong, otherwise early next week.

pshipton on 21 Oct 2020

The 0.23 release occurred on Oct 23 with a jdk15 respin for Mac on Oct 26.