Openj9: Understand JCK implications on the OpenJ9 release process

[sxa]

Looking at the OpenJ9 release process at https://github.com/eclipse/openj9/blob/master/doc/processes/release_process.md one thing that isn't immediately obvious is that "Create the github release corresponding to the tagged level" will result in a rebuild of the code level (in order to remove the -rcX tag)

Once this is done my understanding is that it will be necessary to re-run any JCK testing that has been performed, as it is the binaries we are certifying, not the source level. We can run the automated tests on the rc builds, but I do not anticipate running the manual JCK tests until a proper final binary has been produced.

This should be added to the release process, as the way it is worded at the moment implies that no additional testing will be required at this point.

Enumerating alternative options for clarity (thanks to @lumpfish) - Option 2 seems the most sensible one

1. Run all the testing against rc2 and repeat the JCK testing against the retagged build.
2. Run all the testing except the JCK interactives against rc2 and run all the JCK testing against the retagged build.
3. Run all the testing against rc2 and then document in some way that the rc2 JCK results apply to the retagged build since none of the underlying code was changed.
4. Retag the build before running the JCK so that the results will relate to the release

[DanHeidinga]

Re: JCK statements - this in relation to AdoptOpenJDK running the JCKs on OpenJ9, correct? The Eclipse OpenJ9 project is not a JCK licensee.

Based on my experience in IBM supporting J9 before it was open source, I believe that option 3 is sufficient. If the the -rcX build passes the tests, the release build will as well. If there is every a question about whether the build passes the tests, the questioned test could be re-run at that point in time.

[sxa]

Re: JCK statements - this in relation to AdoptOpenJDK running the JCKs on OpenJ9, correct? The Eclipse OpenJ9 project is not a JCK licensee.

Correct

[lumpfish]

While option 3 may be sufficient it creates a level of obscurity vs. matching the java -version output of a particular java build and the output logged in the JCK test results.
Something along these lines might work better given the compliance test run requirement:

1. Create a tag with the name of the final release
2. Test the release
3. If the testing is successful, archive the test results, declare the build as a release
4. If the testing is unsuccessful, rename the tag to rc1 (or rc2 etc. as required), fix the issue(s), go to 1.

[DanHeidinga]

The point of using -rcX tags is to validate the build is good before it's tagged with the release version. Otherwise users may pull down what looks like the final tag and get a build with issues.

We can't apply the final tag until the testing has passed.

[lumpfish]

How do users know what a final tag 'looks like' before a release has been published?

[pshipton]

The tag and branch formats are documented in the OpenJ9 release process https://github.com/eclipse/openj9/blob/master/doc/processes/release_process.md

[DanHeidinga]

@sxa555 @lumpfish Have your concerns been addressed? If so, I'll close this out.

[lumpfish]

I don't think we can close this issue until the matter of aligning release builds at AdoptOpenJDK with JCK test results has been discussed and agreed with the AdoptOpenJDK project.

[DanHeidinga]

@lumpfish Given the 0.8.0 release is complete, do you agree this can be closed?

[sxa]

Can't speak for @lumpfish and his latest comment but I'm personally happy with it. I'll need to do some minor updates to the build jobs to better handle building from a branch next time but otherwise it's good. My only closing comment is that it would be preferable for the tag in the extensions repository to full match the version e.g. jdk8u162-b12_openj9_0.8.0

[lumpfish]

Yes, this can now be closed. See https://github.com/AdoptOpenJDK/JCK8/issues/9 for details of the JCK test results.

[DanHeidinga]

Thanks everyone for all the work that went into making 0.8.0 compliant!