Openjdk-tests: Use of ci.role.perf label

[sxa]

Describe the bug
Performance tests should run on machines with the ci.role.perf label

To Reproduce
Run any sanity.perf test

Expected behavior
Machines will only schedule on machines with ci.role.perf labels in jenkins.

Additional context
I thought we had been using the ci.role.test&&ci.role.perf label for performance test suites but from what I can tell we only have one machine with that label now and I can't see any indication in the logs that we're targetting any of the perf jobs to machines with that label. I would propose setting up a suitable set of machines with a ci.role.perf label which would go on machines that can produce consistent results where possible, and ensure that the perf tests only go onto those. I was about to set up some labels before realising they're not currently used.

I think I've mentioned this before but I would also propose using this label INSTEAD of ci.role.test and not require ci.role.test&&ci.role.perf to be present, to cover any case where we might be using a completely separate machien that we don't want to be used for other general testing.

[smlambert]

We HAD configured certain perf test jobs to use ci.role.perf instead of ci.role.test back before when we had dedicated perf machines. Since that time, we have:

• lost those dedicated machines and only recently added 2 mac minis back in that role
• regenerated the perf jobs overwriting the special label they were configured with
• made changes to now auto-generate test jobs from build pipelines in cases where test jobs do not exist

I had intended to create an issue similar to this one, but as you might be able to guess, am overwhelmed and am quite raw when taking 'suggestions/criticisms' about things on the AQA roadmap, that we have yet to do, that then suddenly someone takes an interest in. Be gentle.

Indeed we need to add support back in, and should update our test pipelines for perf testing to check first if there is an option to send jobs to ci.role.perf&&hw.arch.xxx&&sw.os.yyyy machines, and if no label matches then select ci.role.test&&hw.arch.xxx&&sw.os.yyyy machines instead.

[sxa]

I'm quite raw when taking 'suggestions/criticisms' about things on the AQA roadmap

I know exactly what you mean - I feel the same way ref infra :-(

Thanks for the info. Was mostly trying to get up to speed on where we are at the moment since I didn't match what I thought I'd seen before and figure out what we will need in terms of infra requirements to make things happen before I disapper down an invalid rabbit hole.

[smlambert]

To address this issue, we could update https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/buildenv/jenkins/openjdk_tests to do a check, if BUILD_LIST.contains("perf") and Jenkins nodes exist that match ci.role.perf&&hw.arch.xxx&&sw.os.yyy for the platform to run on, then use that LABEL, otherwise use ci.role.test&&hw.arch.xxx&&sw.os.yyy.

In this way, if we have bare metal perf machines online we will send perf jobs to them, if not, then we send them to the regular test nodes.

[sxa]

• 2x linux&&x64 (Quad Atom C2550 @ 2.4GHz, 8GB)
• 2x linux&&ppc64le (Quad POWER8 3.4GHz, ~8GB RAM)
• 4x linux&&s390x (Quad s390x, 8GB RAM)
• 1x linux&&aarch64 (16xCPU, 32GB) (temporary: we will need to look at running on the Ampere systems though!)
• 2x win&&x64 (8xE2683v3 2GHz, 8GB)
• 2x aix&&ppc64 (3*SMT8 POWER8, 32GB)
• 2x linux&&arm32 (4 core, 8Gb - pi4)
• 2x mac&&x64 (4 core, 16GB) (Switch to nine once stable)

[sxa]

Given the above, I think we're covered on all platforms so we can make the change you described to the jenkinsfile described above :-)