Graal: [native-image] Netty with native EPoll support

[Jotschi]

I'm trying to build a native image for netty / Vert.x which makes use of the native EPoll library (netty-transport-native-epoll).

The image generation is however failing with error:

Error: Must not have a started Thread in the image heap.

Full Log:
https://gist.github.com/Jotschi/c9ba9e54fed1a7bdb6d1acf4c1f1aaa4

I think this is similar to the error in this issue:
https://github.com/oracle/graal/issues/365

A very basic reproducer can be found in the netty-native-epoll branch:
https://github.com/Jotschi/vertx-graalvm-native-image-test/tree/netty-native-epoll

Native Libs are for linux-amd64 only.

I'm not sure how the JNIConfigurationFiles should actually be used. I added the native classes and all native methods. Let me know whether this is actually needed.

[kontrafiktion]

Would it help to call the static block in ThreadDeathWatcher from the main function?

    static {
        String poolName = "threadDeathWatcher";
        String serviceThreadPrefix = SystemPropertyUtil.get("io.netty.serviceThreadPrefix");
        if (!StringUtil.isNullOrEmpty(serviceThreadPrefix)) {
            poolName = serviceThreadPrefix + poolName;
        }
        // because the ThreadDeathWatcher is a singleton, tasks submitted to it can come from arbitrary threads and
        // this can trigger the creation of a thread from arbitrary thread groups; for this reason, the thread factory
        // must not be sticky about its thread group
        threadFactory = new DefaultThreadFactory(poolName, true, Thread.MIN_PRIORITY, null);
    }

see https://github.com/oracle/graal/issues/424

[Jotschi]

@kontrafiktion Thanks. I patched a few more classes and handled the static constructor calls in the main. That solved the issue. I'm now down to two errors.

fatal error: java.lang.NoClassDefFoundError: Could not initialize class io.netty.channel.unix.FileDescriptor
        at sun.misc.Unsafe.ensureClassInitialized(Native Method)
        at jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.initialize(HotSpotResolvedObjectTypeImpl.java:381)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.createType(AnalysisUniverse.java:211)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:191)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:168)
        at com.oracle.graal.pointsto.meta.AnalysisField.<init>(AnalysisField.java:102)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.createField(AnalysisUniverse.java:355)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:343)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:310)
        at com.oracle.graal.pointsto.meta.AnalysisType.convertFields(AnalysisType.java:845)
        at com.oracle.graal.pointsto.meta.AnalysisType.convertInstanceFields(AnalysisType.java:838)
        at com.oracle.graal.pointsto.meta.AnalysisType.getInstanceFields(AnalysisType.java:829)
        at com.oracle.graal.pointsto.flow.context.object.AnalysisObject.getInstanceFieldTypeStore(AnalysisObject.java:207)
        at com.oracle.graal.pointsto.flow.context.object.AnalysisObject.getInstanceFieldFlow(AnalysisObject.java:197)
        at com.oracle.graal.pointsto.flow.StoreFieldTypeFlow$StoreInstanceFieldTypeFlow.onObservedUpdate(StoreFieldTypeFlow.java:158)
        at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:345)
        at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:387)
        at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:498)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:172)
        at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

[kontrafiktion]

Is that the full stack trace? No “caused by ...”?

[Jotschi]

@kontrafiktion Yes. That is the full stacktrace.

[kontrafiktion]

Your article (https://vertx.io/blog/eclipse-vert-x-goes-native/) does not mention how you handle JNI

https://github.com/oracle/graal/blob/master/substratevm/JNI.md

Especially I do not see

-H:+JNI

in your command line.

(But I am admittedly very far out of my depth)

[Jotschi]

@kontrafiktion I'm actually using the netty-native-epoll branch to test this and the build.sh script uses the mentioned flag. I'm a bit clueless why the FileDescriptor class can't be initialized. It has some native methods but those should afaik be provided by the netty-transport-native-epoll native lib.

[diroussel]

+1 would be great to get this working

[perezd]

My most recent attempt at this resulted in:

com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
 io.netty.util.AbstractReferenceCounted the class was requested to be initialized at build time (from the command line). io.netty.util.AbstractReferenceCounted has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of io.netty.util.AbstractReferenceCounted

However that is the only error I received so far.
Netty 4.1.52
GraalVM/native-image: 19.3.1

Here's a repro:
https://github.com/perezd/netty-epoll-native-repro

[cstancu]

This should be fixed in latest release, you may need --initialize-at-run-time=io.netty.channel.epoll.

[perezd]

@cstancu do you mean https://github.com/graalvm/graalvm-ce-builds/releases/tag/vm-20.2.0 ?

If so, that is not working.

EDIT: Actually, I'll double check and get back to you!