Graal: [native-image] Binary execution causes segfault on Alpine linux.

[sureshg]

Created a simple native image on Ubuntu and trying to run it on the latest alpine causes segfault error.

Using this sample project by changing the running image to alpine:3.7 - https://github.com/JurrianFahner/play-with-graalvm/blob/master/Dockerfile

Since alpine uses musl libc, I had to make the following workaround to make it executable.

/app # ldd server
    /lib64/ld-linux-x86-64.so.2 (0x7f301782a000)
    libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f301782a000)
    libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f301782a000)
    libz.so.1 => /lib/libz.so.1 (0x7f3016488000)
    librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f301782a000)
    libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f301782a000)


/app # mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2

After that the native-images throws Segmation Fault error

/app # ./server
Segmentation fault

So is this (running native-images on alpine) a supported scenario?

[olpaw]

So is this (running native-images on alpine) a supported scenario?

So far, we haven't tested native-images on systems that use musl libc. I'll have a look.

[olpaw]

Hmmm, I cannot reproduce the problem:

# cd /app
# pwd
/app
# ./server
[Thread-1] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - == Spark has ignited ...
[Thread-1] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4567
[Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[Thread-1] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
[Thread-1] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
[Thread-1] INFO org.eclipse.jetty.server.session - Scavenging every 600000ms
[Thread-1] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@6fa86f41{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}
[Thread-1] INFO org.eclipse.jetty.server.Server - Started @-1ms

Seems to work just fine with the following change in Dockerfile

diff --git a/Dockerfile b/Dockerfile
index 36cd2d2..c223b25 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -8,7 +8,7 @@ RUN cd /workbench && \
                  -H:EnableURLProtocols=http \
                  -jar server.jar

-FROM scratch
+FROM alpine:3.7
 COPY --from=builder /workbench/target/server /app/server
 COPY --from=builder /lib64 /lib64
 COPY --from=builder /lib /lib

[olpaw]

Oh my bad. Of course I need to remove the copy commands otherwise it's no alpine linux anymore.

[olpaw]

For now the following can be used as a workaround

diff --git a/Dockerfile b/Dockerfile
index 36cd2d2..fdb538c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -8,10 +8,13 @@ RUN cd /workbench && \
                  -H:EnableURLProtocols=http \
                  -jar server.jar

-FROM scratch
-COPY --from=builder /workbench/target/server /app/server
-COPY --from=builder /lib64 /lib64
-COPY --from=builder /lib /lib
-COPY --from=builder /bin/sh /bin/sh
+FROM alpine:3.7
+RUN apk --no-cache add ca-certificates wget && \
+    wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub && \
+    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-2.27-r0.apk && \
+    apk add glibc-2.27-r0.apk zlib
+
+COPY --from=builder /workbench/target/server /usr/local/bin/server
 EXPOSE 4567
-CMD /app/server
+ENV LD_LIBRARY_PATH /lib
+CMD /usr/local/bin/server

Note that this will provide a glibc within the alpine image which is suboptimal. Currently we do not support linking native images against musl libc.

[olpaw]

I experimented a bit more and it turns out that when I force image building to statically link the binary with the following hack

diff --git a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeBootImageViaCC.java b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeBootImageViaCC.java
index 0224b5ace9f..76c570f57a9 100644
--- a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeBootImageViaCC.java
+++ b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/image/NativeBootImageViaCC.java
@@ -67,6 +67,7 @@ public abstract class NativeBootImageViaCC extends NativeBootImage {
         protected void setOutputKind(List<String> cmd) {
             switch (kind) {
                 case EXECUTABLE:
+                    cmd.add("-static");
                     break;
                 case SHARED_LIBRARY:
                     cmd.add("-shared");

I'm able to produce a binary that doesn't need a libc at all:

[master *>] ~/Downloads/play-with-graalvm/target> ~/OLabs/git/svm-master/graal/substratevm/native-image -H:+ReportUnsupportedElementsAtRuntime -H:EnableURLProtocols=http -jar hello-world-1.0-SNAPSHOT-jar-with-dependencies.jar -H:Name=server
Build on Server(pid: 22774, port: 46055)
   classlist:     309.14 ms
       (cap):     777.83 ms
       setup:   1,327.71 ms
[ForkJoinPool-4-worker-4] INFO org.eclipse.jetty.util.log - Logging initialized @21912ms to org.eclipse.jetty.util.log.Slf4jLog
  (typeflow):   5,241.97 ms
   (objects):   2,670.41 ms
  (features):      64.84 ms
    analysis:   8,203.53 ms
    universe:     422.47 ms
     (parse):   1,036.74 ms
    (inline):   2,312.87 ms
   (compile):   4,838.51 ms
     compile:   8,779.27 ms
       image:   1,011.44 ms
       write:     403.36 ms
     [total]:  20,528.37 ms
[master *>] ~/Downloads/play-with-graalvm/target> ldd ./server
    not a dynamic executable
[master *>] ~/Downloads/play-with-graalvm/target> ./server 
[Thread-2] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - == Spark has ignited ...
[Thread-2] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4567
[Thread-2] INFO org.eclipse.jetty.server.Server - jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[Thread-2] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
[Thread-2] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
[Thread-2] INFO org.eclipse.jetty.server.session - Scavenging every 660000ms
[Thread-2] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@ff3c0015{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}
[Thread-2] INFO org.eclipse.jetty.server.Server - Started @-1ms

:-)

[olpaw]

Then Dockerfile for that image is trivial

FROM scratch
COPY server server
EXPOSE 4567
ENTRYPOINT ["/server"]

and the size of the docker image now equals the size of the native-image

[master *>] ~/Downloads/play-with-graalvm/target> sudo docker build . -t static_server
Sending build context to Docker daemon  17.08MB
Step 1/4 : FROM scratch
 ---> 
Step 2/4 : COPY server server
 ---> Using cache
 ---> f1fef44e3003
Step 3/4 : EXPOSE 4567
 ---> Using cache
 ---> 66fd584d69a7
Step 4/4 : ENTRYPOINT ["/server"]
 ---> Running in b1ebeb3ae3c6
Removing intermediate container b1ebeb3ae3c6
 ---> ad1d01284898
Successfully built ad1d01284898
Successfully tagged static_server:latest
[master *>] ~/Downloads/play-with-graalvm/target> sudo docker run -it -p 8084:4567 static_server
[Thread-2] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - == Spark has ignited ...
[Thread-2] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4567
[Thread-2] INFO org.eclipse.jetty.server.Server - jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[Thread-2] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
[Thread-2] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
[Thread-2] INFO org.eclipse.jetty.server.session - Scavenging every 600000ms
[Thread-2] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@666cb029{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}
[Thread-2] INFO org.eclipse.jetty.server.Server - Started @-1ms

[master *>] ~/Downloads/play-with-graalvm/target> ls -lh server 
-rwxrwxr-x 1 pwoegere pwoegere 14M May  2 15:09 server
[master *>] ~/Downloads/play-with-graalvm/target> sudo docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
static_server            latest              ad1d01284898        5 minutes ago       14.4MB

[cstancu]

@olpaw do you think we should make this "hack" into an option?

[olpaw]

See https://github.com/oracle/graal/issues/394#issuecomment-386106570

[olpaw]

see https://github.com/oracle/graal/issues/394#issuecomment-387355567