Graal: noop garbage collector for native-images

[JAremko]

It can be really handy to have something like Epsilon GC for native-images.

From the Epsilon docs(motivation section):

A short-lived job might rely on exiting quickly to free the resources (e.g. heap memory). In this case, accepting the GC cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway. Note that the GC cycle might take a while, because it would depend on the amount of live data in the heap, which can be a lot.

---

For the reference https://github.com/oracle/graal/issues/651#issuecomment-445536141

[Peter-B-Kessler]

Among the ways you can sort of get an Epsilon GC for SubstrateVM is by building your image with -H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' (note the single-quotes to protect the '$' in there from shell expansion). That gets you a collection policy that never calls the collector. The policy gets called when we have allocated a young-generation-ful of objects to see if there should be a collection, but the answer is always "not now". Running a sample (relatively-long-lived) application, I see collection requests every 256MB, but they take a few hundred nanoseconds each. (And the heap "only" grew to 14GB.) And see below for changing the "256MB".

Another way to avoid collections is to set your -Xmn to a really large size. (For my sample application -Xmn16g.) Then even the policy to ask if now is the time for a collection will not get called. People do that for cutting corners on benchmarks, even on the HotSpot Java virtual machine.

Obviously, either of those options can use a lot of memory, unless your "short-lived CLI" application really is short-lived.

Neither of those options achieves what Epsilon GC does in terms of eliding all the overheads for being prepared to collect the heap. Write-barriers (code and space), code for the collector itself, etc. One could write a heap implementation for SubstrateVM that removed all those overhead, but it would have to be motivated by some use case.

[JAremko]

@Peter-B-Kessler Thanks for the in depth explanation.

Neither of those options achieves what Epsilon GC does in terms of eliding all the overheads for being prepared to collect the heap. Write-barriers (code and space), code for the collector itself, etc. One could write a heap implementation for SubstrateVM that removed all those overhead, but it would have to be motivated by some use case.

Well, I have a Clojure(it is known to generate a lot of garbage) based tool(compiled into native image) that I currently use in CI. But I also want to use it from my IDE(Emacs) as a back-end for text editing functionality.
I could've go with JVM based server and TCP protocol but it adds complexity, takes long to start, adds constant resource tax and requires JVM.
Using native-image and calling the tool from IDE(Emacs) via CLI seems like a great alternative (especially since I already use it in CI).
Native-image of the tool starts to do actual work instantly (perceptually) and gets the job done fast enough. But when we are talking about something that interacts with a human all the speedups I can get from it are welcome :)

---

Also if I can get it to run "extremely" fast I'll call(from IDE) the tool synchronously(blocking IDE's UI) instead of asynchronously(starting it and eventually notifying that it's done). This is a big deal because in the second case the IDE's user can edit processed text while the tool is running and it may cause conflicts.
So a bunch of small optimizations cumulatively can lead to reduced complexity :thinking:

[Peter-B-Kessler]

You have moved from the "extremely short-lived jobs" motivation for EpsilonGC (avoid unexpected collection when the application is about to exit) to the "last-drop throughput" motivation, but without being sensitive to allocations.

How fast does your tool run with a young generation large enough to avoid any collection before exiting? How much time is spent allocating and initializing all that memory, compared to the time it takes to collect the garbage? (Unreachable objects are not examined by the collector.) Maybe allocating less garbage is a bigger win, in your case, than not having a collector.

I do not deny that EpsilonGC could have its uses. I just wonder about the motivation in this case.

[JAremko]

You have moved from the "extremely short-lived jobs" motivation for EpsilonGC (avoid unexpected collection when the application is about to exit) to the "last-drop throughput" motivation, but without being sensitive to allocations.

I think it is both in my case. For CI I want to release worker ASAP and for IDE integration I want to cut the latency(with the caveat that the tool will be restarted each time it is used). :thinking:

Maybe allocating less garbage is a bigger win, in your case, than not having a collector.

Yeah collector isn't dealbreaker for me. Just a potential shortcut.

I do not deny that EpsilonGC could have its uses. I just wonder about the motivation in this case.

I think the point that I'm trying to make here is that having something like EpsilonGC in GraalVM may allow end-users(like me) to cut corners by flipping a flag and observing the results. And if it improved things - great! But if it didn't - also fine since it doesn't take a lot of effort to flip a flag and it always can be un-flipped. At the same time, I don't know if it is a worthwhile investment of Graal devs time or not simply because I have no idea how time consuming implementing a "GC stub" is (I'm pretty sure I heavily underestimating it :smile: ) and how much of an improvement can it be. I'll prolly benchmark my tool with EpsilonGC when I'll be able to use Java 11. Although it should be a bigger deal for native-images.

[SergejIsbrecht]

-H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect'

@Peter-B-Kessler, I am getting an error while using given command for native image generation.

Executing [
/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/bin/java \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-XX:-UseJVMCICompiler \
-Dtruffle.TrustAllTruffleRuntimeProviders=true \
-d64 \
-noverify \
-XX:-UseJVMCIClassLoader \
-Xss10m \
-Xms1g \
-Xmx12931425888 \
-Duser.country=US \
-Duser.language=en \
-Dgraalvm.version=1.0.0-rc11 \
-Dorg.graalvm.version=1.0.0-rc11 \
-Dcom.oracle.graalvm.isaot=true \
-Djvmci.class.path.append=/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/graal.jar \
-Xbootclasspath/a:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/boot/graal-sdk.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/boot/graaljs-scriptengine.jar \
-cp \
/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/objectfile.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/pointsto.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/svm.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/jvmci-hotspot.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/graal.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/graal-management.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/jvmci-api.jar \
com.oracle.svm.hosted.NativeImageGeneratorRunner \
-watchpid \
14816 \
-H:Name=hello-world \
-H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' \
-H:+UseStackBasePointer \
-H:CLibraryPath=/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/clibraries/linux-amd64 \
-H:Path=/home/sergej/Development/GitHub/GraalVMTest/oracleIssue/build/graal \
-H:Class=TopTen \
-imagecp \
/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/boot/graal-sdk.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/boot/graaljs-scriptengine.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/objectfile.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/pointsto.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/builder/svm.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/jvmci-hotspot.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/graal.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/graal-management.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/jvmci/jvmci-api.jar:/home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/library-support.jar:/home/sergej/.gradle/caches/modules-2/files-2.1/org.graalvm.compiler/compiler/1.0.0-rc10/348252f37a155aba52e3ee9da9fbe483589d5d6e/compiler-1.0.0-rc10.jar:/home/sergej/.gradle/caches/modules-2/files-2.1/com.oracle.substratevm/svm/1.0.0-rc10/58e31195b7330226169b4b9ede13d1314171a4f0/svm-1.0.0-rc10.jar:/home/sergej/.gradle/caches/modules-2/files-2.1/org.graalvm.sdk/graal-sdk/1.0.0-rc10/12900d233d14f6382dd15c6481763777c56bf183/graal-sdk-1.0.0-rc10.jar:/home/sergej/Development/GitHub/GraalVMTest/oracleIssue/build/libs/oracleIssue.jar
]
[hello-world:14817] classlist: 1,112.19 ms
[hello-world:14817] setup: 14.14 ms
error: policy 'com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' does not exist. It must be a fully qualified class name.
Error: Image building with exit status 1

[Peter-B-Kessler]

The class com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect is there in

$ jar tf graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/svm.jar | grep NeverCollect
com/oracle/svm/core/genscavenge/CollectionPolicy$NeverCollect.class

which you have on your -cp.

I can build a native image using a test program of mine that just allocates a lot:

$ graalvm-ce-1.0.0-rc11/Contents/Home/bin/native-image --verbose --no-server -cp ~/Play/Java -H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' AllocateALot
Executing [
/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/bin/java \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-XX:-UseJVMCICompiler \
-Dtruffle.TrustAllTruffleRuntimeProviders=true \
-d64 \
-noverify \
-XX:-UseJVMCIClassLoader \
-Xss10m \
-Xms1g \
-Xmx13743895344 \
-Duser.country=US \
-Duser.language=en \
-Dgraalvm.version=1.0.0-rc11 \
-Dorg.graalvm.version=1.0.0-rc11 \
-Dcom.oracle.graalvm.isaot=true \
-Djvmci.class.path.append=/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/graal.jar \
-Xbootclasspath/a:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/boot/graal-sdk.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/boot/graaljs-scriptengine.jar \
-cp \
/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/objectfile.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/pointsto.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/svm.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/graal-management.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/graal.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/jvmci-api.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/jvmci-hotspot.jar \
com.oracle.svm.hosted.NativeImageGeneratorRunner \
-H:Path=/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM \
-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect \
-H:CLibraryPath=/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/clibraries/darwin-amd64 \
-H:Class=AllocateALot \
-H:Name=allocatealot \
-imagecp \
/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/boot/graal-sdk.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/boot/graaljs-scriptengine.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/objectfile.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/pointsto.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/builder/svm.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/graal-management.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/graal.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/jvmci-api.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/jvmci/jvmci-hotspot.jar:/Users/pbkessle/Oracle/Labs/SubstrateVM/Clones/GraalVM/graalvm-ce-1.0.0-rc11/Contents/Home/jre/lib/svm/library-support.jar:/Users/pbkessle/Play/Java
]
[allocatealot:199]    classlist:   1,490.65 ms
[allocatealot:199]        (cap):   1,712.85 ms
[allocatealot:199]        setup:   3,361.43 ms
[allocatealot:199]   (typeflow):   2,366.18 ms
[allocatealot:199]    (objects):     999.84 ms
[allocatealot:199]   (features):     166.97 ms
[allocatealot:199]     analysis:   3,613.85 ms
[allocatealot:199]     universe:     221.60 ms
[allocatealot:199]      (parse):     540.67 ms
[allocatealot:199]     (inline):   1,100.20 ms
[allocatealot:199]    (compile):   4,248.01 ms
[allocatealot:199]      compile:   6,149.55 ms
[allocatealot:199]        image:     424.14 ms
[allocatealot:199]        write:     157.84 ms
[allocatealot:199]      [total]:  15,496.93 ms

When I run the generated image

$ ./allocatealot -XX:+PrintGC
....
[Incremental GC (CollectOnAllocation.Sometimes) 320321K->320321K, 0.0000070 secs]
....

the collection did not collect anything, but it did not take long. (And turning on more tracing shows that the image is, in fact, using the NeverCollect collector.)

I will investigate further.

[Peter-B-Kessler]

Why does your native image build say

-H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' 

where my native image build says

-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect

? That is, your build has single-quotes around the collection policy class name, in the logging from the native image builder.

The error message comes from https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/HeapPolicy.java#L78, which does not put single-quotes around the class name, though in your log there are single-quotes in the class name in the error message.

What does your image build command line look like? My command line (shown above) runs in bash on MacOSX. What command line shell are you using to execute the image build? Does it need the single-quotes to protect the dollar-sign in the class name?

[SergejIsbrecht]

@Peter-B-Kessler, my bad. I was using an gradle plugin to generate the native image. I just pasted your option with ' into the plugin configuration. But as you said the ' is only needed for bash, in order to escape the $. After removing the ' or just typing it into the bash (Linux) as you did it works just fine.

Linux sergej-P50 4.18.0-13-generic #14-Ubuntu SMP Wed Dec 5 09:04:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

This does the Trick:

sergej@sergej-P50:~/Development/GitHub/GraalVMTest/oracleIssue/build/classes/java/main$ /home/sergej/.gradle/caches/com.palantir.graal/1.0.0-rc11/graalvm-ce-1.0.0-rc11/jre/lib/svm/bin/native-image --verbose --no-server -H:InitialCollectionPolicy='com.oracle.svm.core.genscavenge.CollectionPolicy$NeverCollect' TopTen

Thank you for your support.