Graal: JMH Blackhole and Graal interaction

[shipilev]

@thomaswue said that there is a special support for Blackhole in Graal. Yet, our early JMH testing shows it breaks at least in one case. Seen this with JMH validation tests:
https://twitter.com/shipilev/status/992332163055017986

Try this benchmark:

import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

import java.util.concurrent.TimeUnit;

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Fork(1)
public class GraalBHBreakage {
    @Param("1")
    private int steps;

    private Double[] objects;

    @Setup
    public void prepare() {
        objects = new Double[steps];
        for (int c = 0; c < steps; c++) {
            objects[c] = (double) c;
        }
    }

    @Benchmark
    public void test(Blackhole bh) {
        for (int c = 0; c < steps; c++) {
            bh.consume((Double) Math.log(objects[c]));
        }
    }
}

On 8u171, C2 the test yields:

Benchmark             (steps)  Mode  Cnt   Score   Error  Units
GraalBHBreakage.test        1  avgt    5  38.442 ± 0.108  ns/op

With GraalVM 1.0.0-rc1 and now-current JMH 1.20:

Benchmark             (steps)  Mode  Cnt  Score   Error  Units
GraalBHBreakage.test        1  avgt    5  2.327 ± 0.008  ns/op

So, it breaks: it seems the Blackhole method body is present/inlined, but the boxing allocation is stripped out. This is partially due to JMH not supporting compiler hints with Graal (see CODETOOLS-79012164), and after that is fixed, we get back the behavior we want:

With GraalVM 1.0.0-rc1 and JMH 1.21-rc:

$ java -jar target/benchmarks.jar -f 1 -wi 5 -i 5 -r 1s -w 1s
Benchmark             (steps)  Mode  Cnt   Score   Error  Units
GraalBHBreakage.test        1  avgt    5  36.760 ± 0.189  ns/op

But, there is a special testing mode in JMH that forces the inlining of Blackholes to test if Blackhole still works should inline hints fail: -Djmh.blackhole.forceInline=true. It still fails, see with GraalVM 1.0.0-rc1 and JMH 1.21-rc:

$ java -Djmh.blackhole.forceInline=true -jar target/benchmarks.jar -f 1 -wi 5 -i 5 -r 1s -w 1s
...
Benchmark             (steps)  Mode  Cnt  Score   Error  Units
GraalBHBreakage.test        1  avgt    5  2.323 ± 0.020  ns/op

Is this still a bug? I think Blackhole should work even with forced inlining, like it does for C2.

[dougxc]

Strangely, this seems to be an issue only with GraalVM CE, not GraalVM EE. I am investigating.

[dougxc]

So it seems as though Graal's partial escape analysis causes problems for JMH's attempt to force the boxing allocation:

    public final void consume(Object obj) {
        int tlrMask = this.tlrMask; // volatile read
        int tlr = (this.tlr = (this.tlr * 1664525 + 1013904223));
        if ((tlr & tlrMask) == 0) {
            // SHOULD ALMOST NEVER HAPPEN IN MEASUREMENT
            this.obj1 = new WeakReference<>(obj);
            this.tlrMask = (tlrMask << 1) + 1;
        }
    }

The boxing allocation is moved inside the SHOULD ALMOST NEVER HAPPEN IN MEASUREMENT branch.

Now I'm just trying to work out why it doesn't happen in GraalVM EE.

[shipilev]

Yes, we thought that would eventually happen, and this is why inline hints are providing another layer of defense. Yet, Thomas implied there is BlackholeNode that this code should have been parsed into, and then Graal would not optimize this, even if inline hints are broken through?

[thomaswue]

Yes, it is a bug that this is not turned into a BlackholeNode. https://github.com/oracle/graal/blob/9b06d880d796296665bd420c45cd57a7a8e81cfe/compiler/src/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/debug/BlackholeNode.java

[dougxc]

The problem is that Graal doesn't trust classes on the app class path in terms of intrinsification. I'm not sure what the right solution is here. Maybe something like -Dgraal.TrustedIntrinsifiableClasses=<value>?

[dougxc]

Or try make JMH defend against Graal's PEA.

[rschatz]

Or introduce a standard blackhole intrinsic in the JDK, and use that. ;)

[dougxc]

@thomaswue had a good idea: we insert the BlackholeNode for these calls but then also call/inline the method.

[shipilev]

FYI, JMH 1.21 is released with first part of the fix mentioned in the first comment: compiler hints are no longer broken. It would still be nice to have the defense-in-depth to make Graal more resistant to inline hints failures.

[dougxc]

This should now be fixed by 40e02e0f3d13bd7db19f8bb98e7a4cae7f6fba36. Please re-open if you still see the problem in the next GraalVM CE release.
Note you should also be able to test on JMH < 1.21 by building Graal and then:

compiler> mx makegraaljdk graaljdk
compiler> echo "name=GraalVM 1.0.0-rc1" >graaljdk/jre/lib/server/vm.properties
compiler> graaljdk/bin/java -jar /path/to/benchmarks.jar ...