Graal: [native-image] SubstrateSegfault at Netty App runtime

[sxend]

Is there a solution?
(And is this the right way to ask questions? please teach right way to issues report if I wrong......)

% ls -l
total 3736
-rw-r--r-- 1 sxend sxend    1027 Apr 26 23:30 App.java
-rw-r--r-- 1 sxend sxend 3819730 Apr 26 23:22 netty-all-4.1.24.Final.jar

% cat App.java
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;

public class App {
    public static void main(String... args) {
        new ServerBootstrap()
                .group(new NioEventLoopGroup())
                .channelFactory(new ChannelFactory<ServerChannel>() {
                    @Override
                    public ServerChannel newChannel() {
                        return new NioServerSocketChannel();
                    }
                })
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel ch) throws Exception {
                        ch.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(ChannelFutureListener.CLOSE);
                    }
                })
                .bind(9090);
    }
}

% javac -cp ./netty-all-4.1.24.Final.jar:./ App.java

% native-image -cp ./netty-all-4.1.24.Final.jar:./ App     
Warning: Native image server limit exceeded. Use options --server{-list,-shutdown[-all]} to fix the problem.
   classlist:   1,004.13 ms
       (cap):   1,300.50 ms
       setup:   2,183.56 ms
RecomputeFieldValue.ArrayIndexScale automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.arrayIndexScale(Class) was detected in the static initializer of io.netty.util.internal.PlatformDependent0. Add a RecomputeFieldValue.ArrayIndexScale manual substitution for io.netty.util.internal.PlatformDependent0. 
RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of io.netty.util.internal.PlatformDependent0. Add a RecomputeFieldValue.FieldOffset manual substitution for io.netty.util.internal.PlatformDependent0. 
RecomputeFieldValue.ArrayIndexScale automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.arrayIndexScale(Class) was detected in the static initializer of io.netty.util.internal.shaded.org.jctools.util.UnsafeRefArrayAccess. Add a RecomputeFieldValue.ArrayIndexScale manual substitution for io.netty.util.internal.shaded.org.jctools.util.UnsafeRefArrayAccess. 
RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of io.netty.util.internal.CleanerJava6. Add a RecomputeFieldValue.FieldOffset manual substitution for io.netty.util.internal.CleanerJava6. 
  (typeflow):   5,868.49 ms
   (objects):   3,233.09 ms
  (features):      60.19 ms
    analysis:   9,406.92 ms
    universe:     520.33 ms
     (parse):   1,763.38 ms
    (inline):   1,657.49 ms
   (compile):   6,557.39 ms
     compile:  11,031.05 ms
       image:   1,769.05 ms
       write:     253.72 ms
     [total]:  26,237.58 ms

% ./app
[ [ SubstrateSegfaultHandler caught signal 11 ] ]

General Purpose Register Set Values: 

  RAX b1201a88b12018f0
  RBX 0000000000000002
  RCX 0000000000000002
  RDX 00007f4db2801a00
  RBP 0000000000000000
  RSI b1201a88b12018f0
  RDI fffffffffffffff8
  RSP 00007f4db1dfdc80
  R8  0000000000000014
  R9  b1201a88b12018f0
  R10 0000000000000009
  R11 0000000000000002
  R12 0000000000000014
  R13 000000000000000b
  R14 0000000000000000
  R15 00007f4da40008c0
  EFL 0000000000010286
  RIP 00000000004b3250


JavaFrameAnchor dump:

  No anchors

DeoptStubPointer address: 0000000000000000

TopFrame info:

  Lookup TotalFrameSize in CodeInfoTable:
  SourceTotalFrameSize 48

VMThreads info:

  VMThread 00007f4da40008c0  STATUS_IN_JAVA
  VMThread 00007f4dac0008c0  STATUS_IN_NATIVE
  VMThread 0000000002b2a010  STATUS_IN_NATIVE

VM Thread State for current thread 00007f4da40008c0:

  0 (8 bytes): com.oracle.svm.core.genscavenge.PinnedAllocatorImpl.openPinnedAllocator = null
  8 (8 bytes): com.oracle.svm.core.heap.NoAllocationVerifier.openVerifiers = null
  16 (8 bytes): com.oracle.svm.core.jdk.IdentityHashCodeSupport.hashCodeGeneratorTL = java.util.SplittableRandom  00007f4db1201728
  24 (8 bytes): com.oracle.svm.core.snippets.SnippetRuntime.currentException = null
  32 (8 bytes): com.oracle.svm.core.thread.JavaThreads.currentThread = io.netty.util.concurrent.FastThreadLocalThread  00007f4db268fbb8
  40 (8 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.activeTimer = null
  48 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.pinnedTLAB = 
    00007f4da40008f0: 0000000000000000 0000000000000000
    00007f4da4000900: 0000000000000000 0000000000000000


  80 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = 
    00007f4da4000910: 00007f4db1200000 00007f4db1300000
    00007f4da4000920: 00007f4db1201ab8 0000000000000000


  112 (8 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.freeList = 0  0000000000000000
  120 (8 bytes): com.oracle.svm.core.posix.thread.PosixVMThreads.IsolateTL = 94598687557484880  0150150150150150
  128 (8 bytes): com.oracle.svm.core.stack.JavaFrameAnchors.lastAnchor = 0  0000000000000000
  136 (8 bytes): com.oracle.svm.core.thread.VMThreads.nextTL = 139971574892736  00007f4dac0008c0
  144 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequested = -3810  fffff11e
  148 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequestedValueBeforeSafepoint = 0  00000000
  152 (4 bytes): com.oracle.svm.core.thread.VMOperationControl.isLockOwner = 0  00000000
  156 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.statusTL = 1  00000001

VMOperation dump:

  No VMOperation in progress


RuntimeCodeCache dump:

  == [Recent RuntimeCodeCache operations: ]

  == [RuntimeCodeCache: 0 methods]

Deoptimizer dump:

  == [Recent Deoptimizer Events: 
  ]

Dump Counters:


Raw Stacktrace:

  00007f4db1dfdc80: 0000000000000000 00000000004ec345
  00007f4db1dfdc90: 00007f4db268fa30 b1201a88b12018f0
  00007f4db1dfdca0: 00007f4db28019b8 0000000000495b25
  00007f4db1dfdcb0: 0000000000000000 0000000000045078
  00007f4db1dfdcc0: 00007f4db2801800 00000000004b35ea
  00007f4db1dfdcd0: 00007f4db1dfdd28 00007f4db2af8876
  00007f4db1dfdce0: 00007f4db2805cf8 0000000000000000
  00007f4db1dfdcf0: 0000000000000000 00000000004ec345
  00007f4db1dfdd00: 00000000008dd400 00000009db9d3e74
  00007f4db1dfdd10: fffefee242ac7c42 0000000000495fab
  00007f4db1dfdd20: 00007f4db268fa70 00000000000003a1
  00007f4db1dfdd30: 00007f4db2801800 0000000000497718
  00007f4db1dfdd40: c0100700b12012a8 0000000000b9dbf8
  00007f4db1dfdd50: 00000000000003a1 0001012798f0bb5f
  00007f4db1dfdd60: 0000000000000032 0000003200427d47
  00007f4db1dfdd70: 8000000000000000 0000000000000032
  00007f4db1dfdd80: 0001012798f0b7be 0000000100045078
  00007f4db1dfdd90: 00007f4db2801800 00000000004ae991
  00007f4db1dfdda0: 0000000000893f00 000000000000000f
  00007f4db1dfddb0: 00007f4db1201438 00007f4db1201418
  00007f4db1dfddc0: 00000000006d8368 00007f4db2fc742f
  00007f4db1dfddd0: 00007f4db1201080 00000000004e68c6
  00007f4db1dfdde0: 000000000089bb78 000000000040950e
  00007f4db1dfddf0: 00007f4db1dfe700 000000000089bb78
  00007f4db1dfde00: 00007f4db268fbb8 00000000008affb8
  00007f4db1dfde10: 00007f4da40008c0 00007f4db268fa90
  00007f4db1dfde20: 00007f4db1dfe700 00000000004ab23f
  00007f4db1dfde30: 00007f4db12010c8 00007f4db1201080
  00007f4db1dfde40: 00007f4db268faa8 00000000004ed74e
  00007f4db1dfde50: 000000000089b978 00007f4db268fbb8
  00007f4db1dfde60: 00007f4db268fbb8 0000000000444344
  00007f4db1dfde70: 0000000000b9dc20 000000000044d4c8
  00007f4db1dfde80: 0000000000000000 000000000089b978
  00007f4db1dfde90: 0000000100000000 00007f4db268fbb8
  00007f4db1dfdea0: 00000000008ae4f8 0000000000000002
  00007f4db1dfdeb0: 0000000002cba300 00007f4db1dfde70
  00007f4db1dfdec0: 0000000000000000 000000000040a5f4
  00007f4db1dfded0: 0000000000000000 0000000000000000
  00007f4db1dfdee0: 0000000000000000 0000000000000000
  00007f4db1dfdef0: 0000000002cba300 0000000000000000
  00007f4db1dfdf00: 0000000002b2a010 00007f4db1dfe9c0
  00007f4db1dfdf10: 00007ffee0d1c54f 0000000000000000
  00007f4db1dfdf20: 0000000000000000 0000000000000000
  00007f4db1dfdf30: 0000000000000000 00007ffee0d1c54f
  00007f4db1dfdf40: 00007f4db1dfe9c0 0000000002b2a010
  00007f4db1dfdf50: 0000000000000000 00007f4db2fbc6ba
  00007f4db1dfdf60: 0000000000000000 00007f4db1dfe700
  00007f4db1dfdf70: 00007f4db1dfe700 eb400bbca51d9a28
  00007f4db1dfdf80: 0000000000000000 00007ffee0d1c54f
  00007f4db1dfdf90: 00007f4db1dfe9c0 0000000002b2a010
  00007f4db1dfdfa0: 15db68031bdd9a28 15db6e4b29e79a28
  00007f4db1dfdfb0: 0000000000000000 0000000000000000
  00007f4db1dfdfc0: 0000000000000000 0000000000000000
  00007f4db1dfdfd0: 0000000000000000 0000000000000000
  00007f4db1dfdfe0: 0000000000000000 0000000000000000
  00007f4db1dfdff0: 00007f4db1dfe700 00007f4db2aea41d
  00007f4db1dfe000: 0000000000000000 0000000000000000
  00007f4db1dfe010: 0000000000000000 0000000000000000
  00007f4db1dfe020: 0000000000000000 0000000000000000
  00007f4db1dfe030: 0000000000000000 0000000000000000
  00007f4db1dfe040: 0000000000000000 0000000000000000
  00007f4db1dfe050: 0000000000000000 0000000000000000
  00007f4db1dfe060: 0000000000000000 0000000000000000
  00007f4db1dfe070: 0000000000000000 0000000000000000

Stacktrace Stage0:

  RSP 00007f4db1dfdc80 RIP 00000000004b3250 FrameSize 48
  RSP 00007f4db1dfdcb0 RIP 0000000000495b25 FrameSize 32
  RSP 00007f4db1dfdcd0 RIP 00000000004b35ea FrameSize 112
  RSP 00007f4db1dfdd40 RIP 0000000000497718 FrameSize 96
  RSP 00007f4db1dfdda0 RIP 00000000004ae991 FrameSize 144
  RSP 00007f4db1dfde30 RIP 00000000004ab23f FrameSize 32
  RSP 00007f4db1dfde50 RIP 00000000004ed74e FrameSize 32
  RSP 00007f4db1dfde70 RIP 0000000000444344 FrameSize 96
  RSP 00007f4db1dfded0 RIP 000000000040a5f4 FrameSize 144

Stacktrace Stage1:

  RSP 00007f4db1dfdc80 RIP 00000000004b3250  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfdcb0 RIP 0000000000495b25  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfdcd0 RIP 00000000004b35ea  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfdd40 RIP 0000000000497718  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfdda0 RIP 00000000004ae991  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfde30 RIP 00000000004ab23f  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfde50 RIP 00000000004ed74e  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfde70 RIP 0000000000444344  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code
  RSP 00007f4db1dfded0 RIP 000000000040a5f4  com.oracle.svm.core.code.ImageCodeInfo@0x8942c0 name = image code

Full Stacktrace:


  RSP 00007f4db1dfdcb0 RIP 0000000000495b25  [image code] io.netty.util.concurrent.SingleThreadEventExecutor.pollTask(SingleThreadEventExecutor.java:204)
  RSP 00007f4db1dfdcb0 RIP 0000000000495b25  [image code] io.netty.channel.nio.NioEventLoop.pollTask(NioEventLoop.java:523)
  RSP 00007f4db1dfdcd0 RIP 00000000004b35ea  [image code] io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:417)
  RSP 00007f4db1dfdd40 RIP 0000000000497718  [image code] io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
  RSP 00007f4db1dfdda0 RIP 00000000004ae991  [image code] io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
  RSP 00007f4db1dfde30 RIP 00000000004ab23f  [image code] io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  RSP 00007f4db1dfde50 RIP 00000000004ed74e  [image code] java.lang.Thread.run(Thread.java:748)
  RSP 00007f4db1dfde70 RIP 0000000000444344  [image code] com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:222)
  RSP 00007f4db1dfded0 RIP 000000000040a5f4  [image code] Lcom/oracle/svm/core/code/CEntryPointCallStubs;.com_002eoracle_002esvm_002ecore_002eposix_002ethread_002ePosixJavaThreads_002epthreadStartRoutine_0028com_002eoracle_002esvm_002ecore_002eposix_002ethread_002ePosixJavaThreads_0024ThreadStartData_0029(generated:0)

Use runtime option -R:-InstallSegfaultHandler if you don't want to use SubstrateSegfaultHandler.

Bye bye ...

[cstancu]

Hi @sxend, thank you for your report. The segmentation fault is caused by missing some field recomputations; SubstrateVM uses a different object layout than JVM and while most of the fields can be recomputed automatically there are some exceptions that require the manual use of @RecomputeFieldValue as suggested by the warning message. We will merge a solution for this soon.

[skjolber]

Here is a minimal working Netty example: https://github.com/skjolber/substratevm-netty-hello-world. Basically adding -Dio.netty.noUnsafe=true helped a lot. Would think some of the unsafe-stuff would be possible to support natively, as it is more or less about doing native things in Java?

[cstancu]

I am closing this issue as a solution was discussed here: https://medium.com/graalvm/instant-netty-startup-using-graalvm-native-image-generation-ed6f14ff7692.