Graal: Request to revert color profiles fix or backport awt library support to 20.3.1

Created on 17 Dec 2020 · 11Comments · Source: oracle/graal

Feature request
Since GraalVM CE 20.3 was released in Nov 17 2020 and is an LTS release we would like to upgrade Quarkus to support this new release.

However despite efforts on resolving Quarkus issues with 20.3:

there is still an outstanding issue that we were not able to work around on the Quarkus side (see #13644).

Is your feature request related to a problem? Please describe.
In GraalVM 20.2 there was an issue with color profiles that was (partially) fixed in 20.3 through https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3 by requiring ColorSpace and ICC_Profile classes to be reinitialized at runtime and by forbidding any instances of the said classes in the native image heap.

Unfortunately though to get an instance of ICC_Profile at runtime one typically invokes getInstance which ends up trying to load libawt at runtime
(see https://github.com/quarkusio/quarkus/pull/13644#issuecomment-738892352) something that is not supported in 20.3.

The whole issue seems to be fixed in master through https://github.com/oracle/graal/commit/5d2997a9ac8cafdfc6b5c316532165839ad7d1af, which is expected to land in GraalVM 21.0 on January.

Describe the solution you'd like.
Testing has shown that reverting https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3 or backporting https://github.com/oracle/graal/commit/5d2997a9ac8cafdfc6b5c316532165839ad7d1af to GraalVM CE 20.3.1 solves the issue.

Describe who do you think will benefit the most.
Developers of Quarkus, Quarkus users, and GraalVM users (see https://github.com/oracle/graal/issues/2842)

Describe alternatives you've considered.
Working around the issue in Quarkus has been considered but it looks like there is no way to tackle this in a generic way. So even if we manage to fix the issue for some tests like we did in https://github.com/quarkusio/quarkus/pull/13718 we cannot catch all possible cases.

Express whether you'd like to help contributing this feature
Yes

feature native-image

Source

zakkak

👍1

Most helpful comment

I'm glad to hear that! :)
As this has been solved, I'll close this issue. If you hit any more problems, let us know and we'll investigate.

gradinac on 22 Dec 2020

👍2

All 11 comments

\cc @gradinac @vjovanov

zakkak on 17 Dec 2020

Hey @zakkak!
The issue with the color profiles was a sneaky one - due to the way the JDK handles color profile deferring, you might end up with a standard profile that has not actually been loaded. https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3 introduced a fix that would prevent that from ever happening, however, it did lead to the issues you're facing now.

I don't think we can backport the entire AWT support to 20.3.1 as it wasn't in 20.3 and we're way past the feature freeze anyway, but we can work around the issue in the fix itself.

Could you test if the following fix would work for you:

Remove this substitution: https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3#diff-7fef70ac05d49832c18e504903baefa3c2e61a986bf6527774f3494891fcb6acR52-R59
Remove code that triggers the class init rerun: https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3#diff-99245eb275ce7bc2ec68c2f390d3e29e55a87bc09ae7af2ddea6c9d59db9dde9R58-R60

If the above fix works for you, I'll start integrating it to 20.3.1. If it doesn't, we can revert the original fix.

gradinac on 18 Dec 2020

Hi @gradinac thanks for your help (again :) )

Removing the substitution in https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3#diff-7fef70ac05d49832c18e504903baefa3c2e61a986bf6527774f3494891fcb6acR52-R59 doesn't do the trick
Removing the code that triggers the class init rerun: https://github.com/oracle/graal/commit/39d7ff72e9b02d24a333d176c9dc3166afabd6a3#diff-99245eb275ce7bc2ec68c2f390d3e29e55a87bc09ae7af2ddea6c9d59db9dde9R58-R60 does work, but that essentially leaves users with possibly wrong color profile, right? If we follow that route we could add a warning message to state that having instances of those classes in the image heap may result in issues, and that the safe approach is to avoid such instances and to reinitialize the classes at runtime.

zakkak on 18 Dec 2020

No problem at all! :)
We should remove both the substitution and the class init rerun for this to work. If we were to leave the substitution, then we would definitely hit issues as the deferred profiles would never be activated. Looking back at this now, disabling the lazy initialization of the caches was probably an overkill as with the profile resources registered even that should work out of the box (with valid profiles). In that case, we shouldn't have an issue with the invalid profiles. Sorry for that!

I'll push a commit to 20.3.1 that patches this and I'll let you know when it gets merged

gradinac on 18 Dec 2020

👍1

Hey @zakkak!
The fix for the color profile issue has been merged to 20.3.1 - you should no longer hit the above problems :)

I'm not sure if we mirror the 20.3.1 branch somewhere outside - here's the diff of the applied fix:

diff --git a/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/jdk/JavaAWTSubstitutions.java b/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/jdk/JavaAWTSubstitutions.java
index afb4fe19f65..4d35f82ecc2 100644
--- a/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/jdk/JavaAWTSubstitutions.java
+++ b/substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/jdk/JavaAWTSubstitutions.java
@@ -26,8 +26,6 @@ package com.oracle.svm.core.jdk;

 import java.awt.Toolkit;

-import com.oracle.svm.core.annotate.Alias;
-import com.oracle.svm.core.annotate.RecomputeFieldValue;
 import com.oracle.svm.core.annotate.Substitute;
 import com.oracle.svm.core.annotate.TargetClass;

@@ -43,17 +41,3 @@ final class Target_java_awt_Toolkit {
 /** Dummy class to have a class with the file's name. */
 public final class JavaAWTSubstitutions {
 }
-
-/**
- * The purpose of this substitution is to prevent standard color profile loading deferral. The only
- * reason this substitution exists is to ensure that our fix for loading standard color profiles
- * works. It should be removed once standard color profile loading is tested.
- */
-@TargetClass(className = "sun.java2d.cmm.ProfileDeferralMgr", onlyWith = JDK11OrLater.class)
-final class Target_sun_java2d_cmm_ProfileDeferralMgr {
-
-    @Alias//
-    @RecomputeFieldValue(kind = RecomputeFieldValue.Kind.FromAlias)//
-    public static boolean deferring = false;
-
-}
diff --git a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/jdk/JDKRegistrations.java b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/jdk/JDKRegistrations.java
index 0e128b5124a..68694cdb52b 100644
--- a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/jdk/JDKRegistrations.java
+++ b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/jdk/JDKRegistrations.java
@@ -55,10 +55,6 @@ class JDKRegistrations extends JNIRegistrationUtil implements GraalFeature {
     public void beforeAnalysis(BeforeAnalysisAccess access) {
         if (JavaVersionUtil.JAVA_SPEC > 8) {
             access.registerReachabilityHandler(this::registerColorProfileResources, clazz(access, "java.awt.color.ICC_Profile"));
-
-            /* These classes contain standard color profile caches which may not be loaded yet */
-            rerunClassInit(access, "java.awt.color.ColorSpace");
-            rerunClassInit(access, "java.awt.color.ICC_Profile");
         }
     }

You should be able to verify it by applying it to 20.3 and testing if the previously failing apps that used ICC_ColorProfiles now work properly.
If you hit any more issues or a problem with the above fix, please let me know and I'll investigate.

gradinac on 21 Dec 2020

Thanks @gradinac!

I take it the push was to some internal tree? Doesn't look like it's showing up on the public graal-vm/20.3 tree (yet):
https://github.com/oracle/graal/commits/release/graal-vm/20.3