Openj9: Migrate to CMake buildsystem

[dnakamura]

Long running issue to track progress on migrating openj9 from UMA to CMake/

• Platform Support:

• [x] xLinux
• [x] pLinux

Timeline

~Nov 23~

• [x] Integrate VMCPTool and NLSTool
  
  
  
  • ~#3675~
  
  

Dec 17

• [x] JDK 11 Extension changes
  
  
  
  • Remaining changes are ready, just need to open PRs. All fairly trivial, should be merged quickly
  
  
  • ~class library natives~
  
  
  • redirector
  
  
  • #3659
  
  
  • ~#3675~
  
  
  • ~#4041~
  
  
  • ~#4042~
  
  
  • ~#4043~
  
  

Dec 21

• [x] osx
  
  
  
  • ~#3293~
  
  
  • ~eclipse/omr#3124~
  
  

March 1

• [ ] DDR support for non-windows platforms
  
  
  
  • ~eclipse/omr#3209~
  
  
  • ~eclipse/omr#3308~
  
  

March 1

• [ ] stop staging the source files
  
  
  
  • ~eclipse/omr#3189~
  
  
  • ~eclipse/omr#3213~
  
  
  • ~#3725~
  
  
  • ~#4928~
  
  
  • ~#3726~
  
  
  • ~#5341~
  
  
  • ibmruntimes/openj9-openjdk-jdk11#68
  
  

Milestone: First pass of linux x86-64 cmake build modulo any bugs

March 20

• [ ] Windows support (un-optimized bytecode interpreter)
  
  
  
  • ~eclipse/omr#3677~
  
  
  • ~eclipse/omr#3669~
  
  
  • ~eclipse/omr#3670~
  
  
  • ~#5227~
  
  
  • ~#5228~
  
  
  • ~#5240~
  
  
  • ~#5267~
  
  
  • #5268
  
  
  • ~#5269~
  
  
  • ~#5270~
  
  
  • ~#5271~
  
  
  • ~#5280~
  
  
  • eclipse/omr#3705
  
  
  • #5308
  
  

March 22

• [ ] Optimized bytecode interpreter on windows

Milestone: First pass of win x86-64 cmake build modulo any bugs

April 12

• [ ] AIX Support
• [ ] All supported platforms work on cmake
• [ ] Turn off UMA

Validation

• TODO: how to validate the cmake build is equivalent to the autotools build?
• Test equivilent build flags between uma and cmake builds
• Testing
• Perf

Misc TODOS

• [ ] Copyright info
• [ ] Documentation describing how the cmake build is structured

[DanHeidinga]

Thanks @dnakamura. Can you outline the plan (including the great details in PR #78) for how we'll transition to cmake?

[dnakamura]

Yeah working on fleshing out some details now

[jbduncan]

I'd completely understand if the decision to use CMake is set in stone now, but I wondered if Bazel was at all considered as an option? By my understanding, it is made specifically for building and testing large code bases quickly, which may be beneficial for OpenJ9.

[dnakamura]

@jbduncan I don't recall all the details but I do remember looking into Bazel somewhat. IIRC it didn't really get us anything that CMake can't do. Also it has some drawbacks compared with CMake, namely lack of platform support (I don't think it can handle AIX or z/OS)

[jbduncan]

@dnakamura Okay, that's totally fair! Many thanks for explaining a bit of the history behind the decision to use CMake for me. :)

[DanHeidinga]

This won't complete in 0.12.0. Moving to 0.14.0. @dnakamura Can you update the timeline for when this will finish?

[dnakamura]

DDR has proven to be a bigger issue than anticipated. Have WIP pr open at omr. Plan to get finishing touches done today and move on (although I suspect the PR will spend considerable time in review)

[dnakamura]

DDR PRs pending review, as is one or two changes regarding staging the source files.
Only one other changre required in the jdk extensions repo, but leaving that for now as I am on vacation this week, and the chances of a build breakage is high

[dnakamura]

Things running behind due to some old PRs that fell on the back burner and got forgotten about, as well as having to deal with some unexpected flooding at my house. WIll work through the weekend to see if we can get things back on track

[DanHeidinga]

some unexpected flooding at my house

Good luck with the cleanup! And hopefully the rising temps don't lead to more.

[fjeremic]

FYI a bunch of z/OS CMake issues are fixed in [1]. I think we'll need all of them for OpenJ9 as well.

[1] https://github.com/eclipse/omr/pull/3693

[dnakamura]

Status update:
Just polishing off a large PR which significantly refactors the way cmake handles ddr, will open the PR before end of day. (largely a matter of isolating the changes from my current dev tree).

My current monolthic working tree can now successfully run a full make all on the openjdk extensions as well as test matterial /wconfigure --enable-ddr

Looking at making sanity tests pass in order to run cmake builds/testing as part of nightly builds. Out of sanity test set, there are currently 20 failing tests. Will look at resolving once DDR related prs are open

FAILED test targets:
        cmdLineTester_jvmtitests_5
        cmdLineTester_jvmtitests_7
        cmdLineTester_jvmtitests_8
        cmdLineTester_jvmtitests_hcr_7
        cmdLineTester_jvmtitests_hcr_8
        cmdLineTester_jvmtitests_hcr_10
        cmdLineTester_jvmtitests_debug_3
        cmdLineTester_jvmtitests_debug_5
        cmdLineTester_jvmtitests_debug_7
        cmdLineTester_jvmtitests_debug_8
        cmdLineTester_jvmtitests_debug_9
        cmdLineTester_jvmtitests_debug_11
        cmdLineTester_decompilationTests_1
        cmdLineTester_CryptoTest_0
        cmdLineTester_callsitedbgddrext_openj9_0
        J9vmTest_2
        J9vmTest_4
        J9vmTest_5
        SharedCPEntryInvokerTests_0
        SharedCPEntryInvokerTests_1

[fjeremic]

@dnakamura FYI I just completed full migration of OMR configure/build/test to CMake in https://github.com/eclipse/omr/pull/3693. I have DDR disabled at the moment in the pipeline file (search of -DOMR_DDR=0). I hope my changes play nice with yours!

[DanHeidinga]

Discussed with @dnakamura:

• We are a week to a week+half away from a completed xlinux jdk11 cmake build.
• Timeline: 6 weeks after xlinux for all of JDK 11 to be complete