Openjdk-tests: JDK16: jdk_foreign_native_# reports no tests selected (Hotspot only)
Describe the bug
The test starts, reports "Test results: no tests selected", and immediately fails, but only on ARM32, ppcle64le_linux, and ppc64_aix.
https://trss.adoptopenjdk.net/output/test?id=60252b4a5730424dbc7115d5
Looking at the deep history, I'm not seeing any evidence that this has ever worked.
Note: We only run this test on hotspot, as it was excluded from J9 testing by https://github.com/AdoptOpenJDK/openjdk-tests/issues/2218
Expected behavior
I expect the test to either run something, or not to be executed in the first place.
Additional context
This test has been excluded on OpenJ9 for a seemingly unrelated UnsatisfiedLinkError. I think this isn't related because the OpenJ9 issue occurs far later in the process of test execution.
adamfarley
All 19 comments
Table summarizing the platform-specific results for the jdk_foreign_native_# targets\:
Platform | jdk_foreign_native_0 | jdk_foreign_native_1 | jdk_foreign_native_2 | Example
---------|-------------------------|------------------------|-------------------------|-------------------
ppc64_aix | FAILED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Grinder/6805/console
arm32 | SKIPPED | SKIPPED | FAILED | https://ci.adoptopenjdk.net/job/Grinder/6803/console
aarch64 | PASSED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Grinder/6804/console
s390x_linux | FAILED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Grinder/6806/console
x86-64_linux | PASSED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Grinder/6807/console
x86-64_windows | PASSED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Test_openjdk16_hs_sanity.openjdk_x86-64_windows/25/console
x86-64_mac | PASSED | SKIPPED | SKIPPED | https://ci.adoptopenjdk.net/job/Grinder/6808/console
Where all instances of "FAILED" are this issue, ie: "Test results: no tests selected"
adamfarley
on 15 Feb 2021
https://github.com/AdoptOpenJDK/openjdk-tests/issues/2218
This test was disabled for jdk16+, my expectation is that the target should have been SKIPPED... this may be either an issue with how it was disabled, or an edge case to fix in TKG.
FYI @renfeiw
smlambert
on 15 Feb 2021
@smlambert - I knew it was excluded for J9. These failures are on Hotspot.
Will update the title and summary to mention that.
adamfarley
on 15 Feb 2021
I compared the test output between the passing x86 64bit linux run and the failing aix run.
No significant differences after the time stamps are removed.
Investigating whether the native test images aren't being generated/retrieved on the affected platforms.
adamfarley
on 15 Feb 2021
This may explain it. From TestNative.java:
@requires ((os.arch == "amd64" | os.arch == "x86_64") & sun.arch.data.model == "64") | os.arch == "aarch64"
So anything running on s390, ppc, or arm32 is failing because the test was never intended to run on these platforms.
Step 1: Exclude this test on these platforms.
Step 2: Investigate the reason these tests were created, to see why the platform restriction is in place.
Step 3: Either:
A) Close this issue, concluding that it is correct for these tests to be permanently excluded.
or
B) Improve the tests to cover ppc and s390 platforms.
adamfarley
on 15 Feb 2021
auto exclude test jdk_foreign_native plat=.ppc.
adamfarley
on 15 Feb 2021
auto exclude test jdk_foreign_native plat=.arm.
adamfarley
on 15 Feb 2021
A few additional notes:
Step 3 a) from https://github.com/AdoptOpenJDK/openjdk-tests/issues/2265#issuecomment-779373593
- Closing an item that some test is going to be excluded under is not quite correct... especially when we introduce automation to re-include tests (related: https://github.com/AdoptOpenJDK/openjdk-tests/issues/1853), but we can address it once we get to that point. For other permanent excludes, we keep a single issue open (so that automation will consider it exempt and reference the original issue so there is a trail of understanding to follow).
Since the auto exclude mechanism takes a regex for plat, the 2 PRs created above could be done in one. Not a big deal if temporary exclude... if permanent, then likely can collapse into a single entry in the playlist.
jtreg takes care of skipping these platforms for us (by the
@requiresannotation at the top of the test), and when we ran this test case as part of the jdk_foreign target along with other test cases, we do not see this failure of not finding any tests to run (because other test cases run). Because we moved this test case into its own target (jdk_foreign_native) so that we can continue to run all of the jdk_foreign non-native tests in environments when we do not have access to native test images, we now see this issue.
smlambert
on 15 Feb 2021
auto exclude test jdk_foreign_native plat=.(ppc|arm).
adamfarley
on 16 Feb 2021
Doing the single regex now to avoid having to do a manual edit later.
The auto-enable could handle this for us when it's ready, but given that there's a separate openj9 disable that we want to keep, this way we avoid potentially removing that accidentally, or corrupting it, etc etc.
adamfarley
on 16 Feb 2021
Might help if I paid more attention. Adding s390.
adamfarley
on 16 Feb 2021
auto exclude test jdk_foreign_native plat=.(ppc|arm|s390).
adamfarley
on 16 Feb 2021
auto exclude test jdk_foreign_native plat=.(ppc|arm|390).
adamfarley
on 16 Feb 2021
390 platforms don't have the "s" in the list. Apologies for the mess.
adamfarley
on 16 Feb 2021
Was it determined whether this is a permanent or temporary exclude?
Discussing with @renfeiw - if permanent, we can just use the <platformCapabilities> tag for removal. <disabled> tags are best for temporary excludes as the intention is to develop automatic re-inclusion based on queries of the issues excluded under.
smlambert
on 17 Feb 2021
It's determined that this is long-term temporary.
The problem is that this test won't run on platforms that don't have a CLinker implementation, as detailed in JDK-8256862, which includes the platforms we've excluded this test from running on as part of this issue.
The bug seems to imply that the work to port CLinker to more platforms will be completed in a separate branch, "And then we can push support to mainline in the next incubator round."
As this doesn't sound like "soon", we're probably better off leaving the test disabled for now on the affected platforms. Feel free to modify the \ Will find out when the next incubator round is. I'll c+p the date here and add a reminder to my calendar so I remember to revisit this.
adamfarley
on 17 Feb 2021
As this is not a test failure I would prefer to use <platformCapabilities>, which makes more clear. Same way as jdk_lang_native_win (CheckHandles.java, @requires ((os.family == "windows")) is excluded https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/openjdk/playlist.xml#L346.
If the test is available for other platforms later either way we need to update the playlist.
sophia-guo
on 17 Feb 2021
The upstream community has no intention of porting CLinker (a dependency for this test) to new platforms any time soon.
Here's the PR to exclude the test permanently.
https://github.com/AdoptOpenJDK/openjdk-tests/pull/2334
Reviews requested.
adamfarley
on 3 Mar 2021
Resolution merged. Closing.
adamfarley
on 4 Mar 2021
Related issues
smlambert
路
6Comments
piyush286
路
5Comments
smlambert
路
3Comments
smlambert
路
5Comments
smlambert
路
5Comments
Most helpful comment
This may explain it. From TestNative.java:
So anything running on s390, ppc, or arm32 is failing because the test was never intended to run on these platforms.
Step 1: Exclude this test on these platforms.
Step 2: Investigate the reason these tests were created, to see why the platform restriction is in place.
Step 3: Either:
A) Close this issue, concluding that it is correct for these tests to be permanently excluded.
or
B) Improve the tests to cover ppc and s390 platforms.