Graal: Nearly Static Native Images

[nebhale]

Building a fully-static native image (--static) to work against scratch, while possible, has some downsides when it comes to statically linking the musl libc implementation. An alternative popularized by Google and Go applications is nearly-static applications that statically link in everything but libc which stays dynamically linked. This enables a very lightweight image called distroless that includes only glibc and runs natively compiled applications.

GraalVM native image support gets very close to being able to run on distroless, primarily being thwarted by the requirement for libz which isn't included. The community has asked for that to be added but justifiably, Google pushed back. While a request could be made for native-image to change to statically link in libz, libstdc++, and libgcc_s there are good reasons to have them dynamically linked in cases that aren't distroless. So the next step is to try to control the linker directly with -H:NativeLinkerOption.

A command line with "-H:NativeLinkerOption=-static-libstdc++ -H:NativeLinkerOption=-static-libgcc -Wl,-Bstatic -lz" _should_ statically link in those to libraries given full control over the gcc command line, but unfortunately, they're always appended to the generated arguments. This means that the prior -lstdc++ and -lz will always cause those libraries to be linked dynamically before our arguments can request that they be linked statically.

As an end goal, we're aiming to make native images run on distroless with a nearly-static compilation but without extensive changes to either distroless or the default native image compilation. At a technical level this should be solvable with more control over the linker command line and ordering, but any technical solution that enables this would be acceptable.

I'd be happy to test snapshots as they become available to ensure that the chosen solution meets the goal of running on distroless, but I don't think I'm the right person to work on the internals of native-image itself.

[eginez]

To make sure we are in the same page this is the generic distroless you are talking about correct: https://github.com/GoogleContainerTools/distroless

[nebhale]

That's the correct repo and I'm aiming for gcr.io/distroless/static which includes libc, libssl, and openssl on top of the base distroless image.

[eginez]

great, thanks for the info, do you have any sample app you have been working on to test this with spring by any chance? If so could you share the repo url?

[nebhale]

spring-graalvm-native's spring-petclinic-jdbc (something I'm guessing you're quite familiar with 😃). There is no script with my various command line arguments as I'm still just playing around with commands on the command line rather than any automation yet.

[eginez]

Hi @nebhale I have committed experimental support for nearly static images with https://github.com/oracle/graal/commit/3e06acc6b106212611853d4171ad89a098178abd

There is new experimental flag that can be used like so:

 native-image \
-H:+StaticExecutableWithDynamicLibC \
... 

Please let me know how this works for you

[nebhale]

Taking at look this morning. I'll let you know how it goes.

[nebhale]

@eginez We're close. I just tested against 20.2.0-dev-20200703_0213 and it appears that most of the libraries are static/dynamically linked as expected. The one exception is that libgcc_s is still dynamically linked and should be statically linked (gcc flag -static-libstdc++). I believe that this particular library isn't always required; some feature of an application can trigger a code path that requires it, sometimes.

[eginez]

gotcha, yes I think I missed that flag. This https://github.com/oracle/graal/commit/b0c00ace24586ab2ddbc24e69db00995b5521263 should have fixed that.
Let me know if this works

[nebhale]

Thanks for the quick turnaround. I'll get back to you after the next snapshot.

[nebhale]

SHIP IT! Confirmed that this builds and runs properly. Thanks again for all the hard (and fast) work.

[eginez]

closed with https://github.com/oracle/graal/commit/b0c00ace24586ab2ddbc24e69db00995b5521263

[sdeleuze]

Thanks a lot @eginez