Graal: SVM: Reflection failure at build-time

[FroMage]

As noted in #603, reflection calls at build-time often fail at random places without this patch:

diff --git a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageCodeCache.java b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageCodeCache.java
index ef5e012779..45befb0107 100644
--- a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageCodeCache.java
+++ b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageCodeCache.java
@@ -366,7 +366,8 @@ public class NativeImageCodeCache {
             throw VMError.shouldNotReachHere("Non-instantiated type referenced by a compiled method: " + obj.getClass().getName());
         }

-        imageHeap.addObject(debug, obj, false, false, constantReasons.get(constant));
+        Object replaced = imageHeap.getAnalysisUniverse().replaceObject(obj);
+        imageHeap.addObject(debug, replaced, false, false, constantReasons.get(constant));
     }

     /**
diff --git a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageHeap.java b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageHeap.java
index 3088f93747..b8b9abdd96 100644
--- a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageHeap.java
+++ b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeImageHeap.java
@@ -936,6 +936,10 @@ public final class NativeImageHeap {
         }
     }

+    protected AnalysisUniverse getAnalysisUniverse() {
+        return aUniverse;
+    }
+
     protected HostedUniverse getUniverse() {
         return universe;
     }

One such random error is:

error: Static field or an object referenced from a static field changed during native image generation?
  object:[Ljava.lang.reflect.Parameter;@6b5c5d60  of class: java.lang.reflect.Parameter[]
  reachable through:
    object: public sun.net.www.protocol.http.spnego.NegotiatorImpl(sun.net.www.protocol.http.HttpCallerInfo) throws java.io.IOException  of class: java.lang.reflect.Constructor
    root: sun.net.www.protocol.http.Negotiator.getNegotiator(HttpCallerInfo)

Error: Image building with exit status 1

[ilopmar]

I've reproduced pretty much the same error in a Micronaut application. The stacktrace is:

...
...
[mn-security:31516]     (inline):  24,028.15 ms
[mn-security:31516]    (compile): 175,691.97 ms
[mn-security:31516]      compile: 229,211.55 ms
[mn-security:31516]        image:  15,413.45 ms
Error: Static field or an object referenced from a static field changed during native image generation?
  object:[Ljava.lang.reflect.Parameter;@60d0b3b0  of class: java.lang.reflect.Parameter[]
  reachable through:
    object: public sun.net.www.protocol.http.spnego.NegotiatorImpl(sun.net.www.protocol.http.HttpCallerInfo) throws java.io.IOException  of class: java.lang.reflect.Constructor
    root: null

com.oracle.svm.core.util.UserError$UserException: Static field or an object referenced from a static field changed during native image generation?
  object:[Ljava.lang.reflect.Parameter;@60d0b3b0  of class: java.lang.reflect.Parameter[]
  reachable through:
    object: public sun.net.www.protocol.http.spnego.NegotiatorImpl(sun.net.www.protocol.http.HttpCallerInfo) throws java.io.IOException  of class: java.lang.reflect.Constructor
    root: null

    at com.oracle.svm.core.util.UserError.abort(UserError.java:67)
    at com.oracle.svm.hosted.image.NativeImageHeap.verifyTargetDidNotChange(NativeImageHeap.java:558)
    at com.oracle.svm.hosted.image.NativeImageHeap.writeReference(NativeImageHeap.java:595)
    at com.oracle.svm.hosted.image.NativeImageHeap.write(NativeImageHeap.java:584)
    at com.oracle.svm.hosted.image.NativeImageHeap.writeField(NativeImageHeap.java:578)
    at com.oracle.svm.hosted.image.NativeImageHeap.writeObject(NativeImageHeap.java:841)
    at com.oracle.svm.hosted.image.NativeImageHeap.writeHeap(NativeImageHeap.java:290)
    at com.oracle.svm.hosted.image.NativeBootImage.build(NativeBootImage.java:419)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:593)
    at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:410)
    at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Image building with exit status 1

Please let me know if you want me to upload a sample project that reproduces the issue.

[graemerocher]

Can this patch be applied?

[graemerocher]

Relates to https://github.com/micronaut-projects/micronaut-core/issues/684 and https://github.com/oracle/graal/issues/717

[cstancu]

FYI I managed to identify the underlying problem and will have a fix soon. The above patch is an workaround that doesn't really fix the root problem, and it can actually lead to other issues.

[cstancu]

Fixed by https://github.com/oracle/graal/commit/f68406219a9edb55922605fc4782f135bb2739b3.

The problem was that the reflection metadata objects intrinsified during compilation were not properly initialized. Objects like java.lang.reflect.Method have some lazily initialized fields. We eagerly trigger the initialization during analysis to make sure that the reflection metadata objects written in the heap are fully initialized. However, the ReflectionPlugins were getting the Method objects again via Class.getMethod() during compilation. Class.getMethod() gives you back an un-initialized copy. The initialization of the copy was triggered sometimes between compilation and native image heap write, hence the problem. Now we cache the reflection metadata objects intrinsified during analysis and reuse them during compilation. See this comment for more details.

[graemerocher]

Great news!