Openj9: Testing for out-of-process JIT compilation (aka JITaaS)

[harryyu1994]

While it’s true that JITaaS can run the same set of tests that OpenJ9 runs, we are going to need two “java” instances running instead of one. (One act as the server and one act as the client)

For example:
We need to first launch the server like this:
./java -XX:JITaaSServer &

Then runs the OpenJ9 tests with this extra option.
make _sanity EXTRA_OPTIONS=” -XX:JITaaSClient ”

Could this setup be done?

@llxia @smlambert @vsebe @mpirvu

[llxia]

I think we can have a variable, say PRE_TEST_CMD, set in https://github.com/AdoptOpenJDK/openjdk-tests/blob/f887cbaeaecfecc2dcf8f8b644c41b58cff9328a/maketest.sh#L19

By default, PRE_TEST_CMD is empty. If JITaaS build, Jenkins build/local run can pass in the cmd it needs run before running any test.

fyi @renfeiw

[harryyu1994]

Also I think we need to somehow kill/clean up the server after the test run is over.

[vsebe]

Depends on https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/493

[harryyu1994]

Hi, I'm wondering roughly when this will be completed. Thanks!

[llxia]

For test part, if this is urgent, please feel free to modify maketest.sh as needed.

[llxia]

I saw the PR in https://github.com/AdoptOpenJDK/openjdk-tests/pull/534. Do we plan to run tests both in docker and directly on machines?

[harryyu1994]

Running tests directly on machines are more important to us. Testing in docker is a nice-to-have.

[harryyu1994]

@llxia Hi Lan, jitaas is open-sourcing soon, there is going to be a jitaas branch in OpenJ9 and a jitaas branch in OpenJ9-OMR. Once it's open-sourced, we are expecting it to have PR testing. For example: a committer should be able to request a build+test via comments like this Jenkins test sanity xlinux jdk8 depends eclipse/openj9-omr#jitaas. As I explained earlier, jitaas needs an extra JIT server java instance that needs to be launched before testing and an extra option in the java instance that runs the tests.

Can this be done soon?

Thanks in advance

[harryyu1994]

When we do Jenkins test sanity xlinux jdk8 depends eclipse/openj9-omr#jitaas, does it pick up the openj9/test directory from the branch that the PR is opened against? If so, we could possibly add in the necessary changes in jitaas branch's openj9/test directory and nothing needs to be changed for OpenJ9.

[llxia]

• Test build uses scripts that are in AdoptOpenJDK, not in openj9/test. Only functional test material is openj9/test
• We can help to add parameters/script for running jitaas tests. @renfeiw Could you help on this? Thanks
• For jitaas PR build, have you talked with @AdamBrousseau and @jdekonin ?

[harryyu1994]

I've talked with Adam, he said we likely need conversations with the test folks.

[AdamBrousseau]

The PR builds will build the OpenJ9 test material from your PR. Ie. The merge of your PR branch with the target (jitaas) branch. This will not change when we alter the PR builds in #2728 .

[llxia]

We do need to upstream build to pass in extra parameters for jitaas test builds. Either extra cmds or a flag, so test build knows to start server before running the tests.

[harryyu1994]

So could the jitaas PR testing be worked on first, that's the more important/urgent thing right now as we are planing to open source today. And please let me know if there's anything I can help with. Thanks!

[AdamBrousseau]

You should already be able to launch PR builds from any PR (no matter the source/target branch). The hard part will be determining how to coordinate with the test jobs to indicate you want extra commands run before testing and also make sure it will be cleaned up after the tests are finished. I will talk to @llxia and see if we can figure something out.

[llxia]

On the second thought, we can have maketest_jitaas.sh similar as maketest.sh, but with jit specific cmds added. The test build can run maketest_jitaas.sh or maketest.sh based on the flag.

[renfeiw]

We have added setup and teardown in the framework (#3059) If environment variable TEST_FLAG is JITAAS, the framework will startup and kill jitaasserver before and after test execution.

[harryyu1994]

So when we invoke the PR testing like this Jenkins test sanity xlinux jdk11 depends eclipse/openj9-omr#jitaas, how are we passing in the TEST_FLAG? will it just be an extra argument at the end?

[renfeiw]

For the PR build story, you will need help from @AdamBrousseau.

[AdamBrousseau]

To have this option via PR build #2728 needs to be resolved.

[smlambert]

BTW, what is the expectation for a user that wants to use jitaas. They have instructions to launch a server? Is there initial doc for how a user would set it up?

[AdamBrousseau]

I _think_, based on #3059, if the user exports TEST_FLAG=JITAAS, they will get the setup and teardown for "free".

[smlambert]

Yes, I know what we have done from the test side in order to set up for testing. What I am asking is how we expect a user to use this in the field.

[harryyu1994]

We have instructions on how to launch a server and use jitaas here https://github.com/eclipse/openj9/blob/jitaas/doc/compiler/jitaas/Usage.md,
Also have instructions on how to run testing locally and in docker
https://github.com/eclipse/openj9/blob/jitaas/doc/compiler/jitaas/Testing.md,
^they are just rough draft

So our expectation was for users to manually start their own server and do testings locally/in docker.

The purpose of this PR was just for us to have PR testing and personal testing.

Are you suggesting that we should make use of this setup so that jitaas users can launch testing without manually kicking off their own sever + extra option to JVM? @smlambert Sorry for the late reply, didn't notice the question.

[smlambert]

I am not suggesting that users should make use of PR testing setup. I wanted to understand how users invoke it, to keep our testing story as close to the user story as possible.

Thanks for the doc links!

[dsouzai]

Has this been addressed via the JITServer PR builds? Can this be closed?