Graal: Separate Compilation and Caching for SVM

[olegchir]

Hi Team,

Is it possible to implement separate compilation or any other type of compilation artifacts reuse for SubstrateVM?

SVM is very good for very tiny microservices, but in real production use, especially on desktops, we'll meet with scalability problems.

Imagine Google Chromium that's written in SVM Java.

A small note about motivation. It's a good idea to write things like Chromium in Java, because most of the code is about menus and internal services, and isn't about fast hardware-accelerated rendering in C++. Writing this code can be trusted to ordinary and sane people, not just to a few divinely crazy gurus. And Chromium is a very startup speed and time-to-performance dependent thing, so it's not an option to run it on vanilla OpenJDK with C2.

So here's the first problem in this way. Building Chromium-C++ the first time can take from 20 minutes to 12 hours, depending on the scope of dependencies, hardware, and linking type. But consecutive rebuilding is much faster due to nature of C++ separate compilation and tricks in the Ninja build system. It's a case from real life of any Chromium developer.

Imagine that you're building native-image with SVM for 12 hours. And 12 hours again. And again, and again. In the end, you'll surely go mad as Sheogorath. No pre-running JVM processes for quick warm-up won't be able to fix this problem.

So, the question is, is it possible to implement some kind of partial compilation or mid-run caching for SVM and native-image? In theory? In practice? Do you have any plans on this topic?

-O.

[cstancu]

@olegchir we currently don't have incremental compilation for SubstrateVM. We thought about it before and we think it is theoretically possible. We never went further than a thought experiment, but with enough effort it can probably be implemented. One way to do it is to cache the static analysis results and only recompute them for code that changes. The static analysis takes about half the time of image building, and it doesn't scale linearly with increase in analyzed code size. The granularity of the code units could be package, module, etc. We currently don't have any concrete plans to implement it, but it is a possibility for the future.

[olpaw]

Hi @olegchir ,
for such scenarios I suggest running on JVM during development/debugging as much as possible and only build images when you need to deploy/test it in production (e.g. on a nightly basis). That way you get the convenience of full debugging support of modern Java IDEs plus the instant-startup characteristics of SVM when you need it.

[olegchir]

@olpaw is there any way to check if a code is native-image ready? Static checking for LIMITATIONS.md and things like that. Writing something all day to understand that it doesn't work at all on SVM after a couple of days is a very special type of perversion :)

[olpaw]

@olpaw is there any way to check if a code is native-image ready

Other than performing native-image generation not really. For our own use (making sure the whole truffle stack + languages can be AOT-compiled with native-image) over time people got used to the do's and don't's. Also we have gate checks to ensure every PR that goes to master is checked for SVM compatibility.

There is a flag to make native-image generation stop right after the static analysis (-H:+ExitAfterAnalysis) that you could use if you are resource constraint and cannot afford image building on each pull-request.

Also if you use native-images server mode you can make image building twice as fast compared to --no-server. See Server options help in native-image --help-extra output. By using --server-session=<custom-session-name> you can make sure you are reusing the same image-build server again and again (and therefore benefiting from a warmed-up image build server).

[olpaw]

And you can combine both of the above things:

pwoegere@olpaw:~/OLabs/graalvm-ce-1.0.0-rc2/test> ../bin/native-image HelloJava -H:+ExitAfterAnalysis
Build on Server(pid: 14340, port: 40107)*
   classlist:     715.10 ms
       (cap):   1,261.10 ms
       setup:   2,359.48 ms
  (typeflow):   4,155.74 ms
   (objects):   1,964.11 ms
  (features):      37.73 ms
    analysis:   6,254.06 ms
pwoegere@olpaw:~/OLabs/graalvm-ce-1.0.0-rc2/test> ../bin/native-image HelloJava -H:+ExitAfterAnalysis
Build on Server(pid: 14340, port: 40107)
   classlist:     134.02 ms
       (cap):     624.96 ms
       setup:     907.34 ms
  (typeflow):   2,530.15 ms
   (objects):   1,033.38 ms
  (features):      26.60 ms
    analysis:   3,671.33 ms
pwoegere@olpaw:~/OLabs/graalvm-ce-1.0.0-rc2/test> ../bin/native-image HelloJava -H:+ExitAfterAnalysis
Build on Server(pid: 14340, port: 40107)
   classlist:     110.71 ms
       (cap):     693.70 ms
       setup:     957.17 ms
  (typeflow):   2,316.71 ms
   (objects):     943.20 ms
  (features):      28.21 ms
    analysis:   3,366.00 ms

As you can see above in subsequent requests time per request drops by half.

[knyttl]

This feature would be super useful for us.

1. We want to debug on native-image build, as the potential issues because of reflection/introspection must be discovered in development, not in production.
2. Our microservices can be super-tiny, but needs a lot of dependencies. changing just one of our classes leads to total recompilation of all the dependencies and overall it takes us 10 minutes.

I am not aware of how it is implemented, but some kind of class.hashcode -> compiled file cache would solve the issue for us completely.