Graal: Performance regression of creation of List values in Scala 2.13.0

Created on 24 Jul 2019 · 6Comments · Source: oracle/graal

About 7x times slowdown comparing to OpenJDK was detected by a benchmark which creates lists in Scala 2.13.0.

Initially, the issue was opened in the Scala repo, but then was factored out on 2 separated problems. This issue is only for 2nd one part that is reproducible only on GraalVM CE/EE.

Steps to reproduce:

Install sbt tool (or ensure that it is already installed)
Clone the benchmark repo:

git clone https://github.com/plokhotnyuk/linkedlist-bench
cd linkedlist-bench

Run benchmarks with OpenJDK:

sbt -java-home /usr/lib/jvm/openjdk-11 -no-colors clean 'bench/jmh:run LinkedListBenchmark.scalaListAddOne'

Run benchmarks with GraalVM CE:

sbt -java-home /usr/lib/jvm/graalvm-ce-19 -no-colors clean 'bench/jmh:run LinkedListBenchmark.scalaListAddOne'

Run benchmarks with GraalVM EE:

sbt -java-home /usr/lib/jvm/graalvm-ee-19 -no-colors clean 'bench/jmh:run LinkedListBenchmark.scalaListAddOne'

Compare throughput scores of benchmark results

Bellow is a flame graph that was built with the help of async-profiler extra for the sbt-jmh plugin by the following command:

sbt -java-home /usr/lib/jvm/graalvm-ce-19 -no-colors 'bench/jmh:run -jvmArgsAppend "-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints" -prof "jmh.extras.Async:event=cpu;dir=/tmp/profile-async;asyncProfilerDir=/home/andriy/Projects/com/github/jvm-profiling-tools/async-profiler;flameGraphDir=/home/andriy/Projects/com/github/brendangregg/FlameGraph;flameGraphOpts=--color,java;verbose=true" -wi 5 -i 25 -f 1 -p size=128 LinkedListBenchmark.scalaListAddOne'

compiler performance

Source

plokhotnyuk

Most helpful comment

For the record, Scala library's indirection in the Scala through method handles to absrtract over Unsafe and VarHandle is done to be cross-compatible between Java 8 and later versions that might hide Unsafe. Perhaps multi-release JARs would have been a better answer but I didn't want to complicate our build with them.

At the time, I only tested inlining with C2.

I'm glad to hear this is easy to fix in Graal -- thanks for the swift response.

retronym on 24 Jul 2019

👍2

All 6 comments

Thanks for reporting this! The 7x slowdown is against what?
Also I get the following error when running the sbt command:
[error] java.lang.NoSuchMethodError: sbt.package$.stringToOrganization(Ljava/lang/String;)Lsbt/librarymanagement/DependencyBuilders$Organization;
Is sbt.version=1.3.0-RC2 mandatory?

mur47x111 on 24 Jul 2019

@mur47x111 possible the latest stable version of sbt should be installed

The slowdown is comparing to OpenJDK. The README of the benchmark repo has results for OpenJDK 11.0.1, GraalVM CE 19.1.0, and GraalVM EE 19.1.0.

plokhotnyuk on 24 Jul 2019

The issue is that we did not inline a lambda to native intrinsic method (in this case Unsafe.storeFence). A fix is on the way.

Before the fix:

Benchmark                            (size)   Mode  Cnt          Score          Error  Units
LinkedListBenchmark.javaList              1  thrpt    5  142646871.127 ± 16532396.440  ops/s
LinkedListBenchmark.javaList             10  thrpt    5   29313071.400 ±  1208985.108  ops/s
LinkedListBenchmark.javaList            100  thrpt    5    3320663.346 ±   191464.668  ops/s
LinkedListBenchmark.scalaListAddOne       1  thrpt    5   36030717.547 ±  2881247.579  ops/s
LinkedListBenchmark.scalaListAddOne      10  thrpt    5    5994007.286 ±   273589.270  ops/s
LinkedListBenchmark.scalaListAddOne     100  thrpt    5     668134.399 ±    27039.656  ops/s
LinkedListBenchmark.scalaListPlusEq       1  thrpt    5   36231592.230 ±  1787748.862  ops/s
LinkedListBenchmark.scalaListPlusEq      10  thrpt    5    5968853.279 ±   382349.728  ops/s
LinkedListBenchmark.scalaListPlusEq     100  thrpt    5     670726.207 ±    30612.494  ops/s

After the fix:

Benchmark                            (size)   Mode  Cnt          Score          Error  Units
LinkedListBenchmark.javaList              1  thrpt    5  142080993.426 ± 10097082.204  ops/s
LinkedListBenchmark.javaList             10  thrpt    5   29254494.533 ±  1627330.071  ops/s
LinkedListBenchmark.javaList            100  thrpt    5    3318488.630 ±   185085.064  ops/s
LinkedListBenchmark.scalaListAddOne       1  thrpt    5  188728525.384 ±  3971013.011  ops/s
LinkedListBenchmark.scalaListAddOne      10  thrpt    5   32926116.644 ±  2105792.099  ops/s
LinkedListBenchmark.scalaListAddOne     100  thrpt    5    3401675.403 ±   124602.515  ops/s
LinkedListBenchmark.scalaListPlusEq       1  thrpt    5  188138519.033 ± 10662199.583  ops/s
LinkedListBenchmark.scalaListPlusEq      10  thrpt    5   33038825.221 ±  2018025.951  ops/s
LinkedListBenchmark.scalaListPlusEq     100  thrpt    5    3402109.303 ±   216543.259  ops/s

mur47x111 on 24 Jul 2019

❤1 🎉1

@mur47x111 thanks!

I'm closing the issue and looking forward to the new version of GraalVM with impatience!

plokhotnyuk on 24 Jul 2019

👍1

At the time, I only tested inlining with C2.

I'm glad to hear this is easy to fix in Graal -- thanks for the swift response.

retronym on 24 Jul 2019

👍2

FYI, https://github.com/oracle/graal/commit/539964ba3aef2cd4346eb2662b84f3cef5dfea35

mur47x111 on 25 Jul 2019