Bazel: Java 14 toolchain support

[uri-canva]

Description of the problem / feature request:

Support for java 14 as --java_toolchain and --host_java_toolchain.

Feature requests: what underlying problem are you trying to solve with this feature?

Target java 14 and using java 14 source level when building java code

Have you found anything relevant by searching the web?

According to https://github.com/bazelbuild/bazel/blob/da370c1da61555b9f9d71fb99f52dfc16055ab44/src/BUILD#L560 only 11 and 12 are supported, https://github.com/bazelbuild/java_tools only has tags / branches for 9, 10 and 11.

[davido]

Workaround: absolute_javabase in combination with toolchain_vanilla.

Have you checked Gerrit Code Review project? It is documented how to use JDK 13 with vanilla toochain (or JDK 14 or newer):

  $ bazel build \
    --define=ABSOLUTE_JAVABASE=<path-to-java-13> \
    --javabase=@bazel_tools//tools/jdk:absolute_javabase \
    --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \
    --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
    --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
    :release

Say, JDK 14 is installed in directory: /home/davido/pgm/jdk-14 then the command is:

  $ bazel build --define=ABSOLUTE_JAVABASE=/home/davido/pgm/jdk-14 \
  --javabase=@bazel_tools//tools/jdk:absolute_javabase \
  --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \
  --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
  --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
   java/com/google/gerrit/common:server

And the produced Java major byte version is 58, as expected:

  $ javap -v -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.Version | grep major 
  major version: 58

[razfriman]

Thank you for the workaround - Is there any reason why Bazel would not offer support for this out of the box? Seems like a useful feature that could be easily prioritised.

Seems like prior art exists for adding a new version as found in this PR: https://github.com/bazelbuild/bazel/pull/8534

I created a related issue in the java_tools repo to track the progress of adding java_tools javac14-v1.0: https://github.com/bazelbuild/java_tools/issues/27

[aiuto]

The reason is simple. There are so many important things to do and only so many engineers.

[davido]

Bazel team has to upload the right binaries to their OpenJDK mirror site: [1]. E.g. this one for Linux: [2] to start to experiment with JDK14 (and newer releases).

@aiuto @meteorcloudy Who is the right person on Bazel team to ask to upload JDK 14 binaries to Bazel Mirror? Or, can you grant me the ACL to do that?

[1] https://mirror.bazel.build/openjdk/index.html
[2] https://cdn.azul.com/zulu/bin/zulu14.28.21-ca-jdk14.0.1-linux_x64.tar.gz

[aiuto]

We only mirror what we need for the Bazel build or for the current rules_java, so adding more to the mirror is not strictly needed. It might even be confusing since we are in no rush to update our internal JDK bundle to JDK14.

But that does not stop anyone from trying to add JDK14 support to rules_java. During development the download can come from anywhere and does not have to be in the mirror. If someone sends a workable PR against rules_java to add a JDK14 toolchain, then we'll have a reason to add it to the mirror.

[uri-canva]

Just to make sure I got it correctly: the order of changes to introduce a new version of the JDK is:

1. Update https://github.com/bazelbuild/rules_java to include the new version of java
2. Use that new version of java to build https://github.com/bazelbuild/bazel/tree/master/src/java_tools
3. Publish new version of java_tools to https://github.com/bazelbuild/java_tools
4. Update https://github.com/bazelbuild/bazel to use the newly published version of java_tools

Is https://github.com/bazelbuild/bazel/tree/master/src/java_tools supposed to be compatible at HEAD with all currently supported java versions? So any changes to it need to be backwards compatible with jdk11 and jdk12?

[davido]

Some bullet points are missing in your list:

• Update to java_tools_langtools_javac, see: [1]. Or may be patching of JDK compiler is not needed for Java 14?
• Update outdated Error Prone version, to the version that supports JDK14, see: [2]. See, e.g., this PR, where Error Prone was bumped to 2.3.5-SNAPSHOT: [3]. However, EP still does not support JDK14/JDK15, so this is basically a show stopper for now.

Update https://github.com/bazelbuild/bazel to use the newly published version of java_tools

You could skip this step, as documented in the README file of java_tools project.

I have built and released custom java_tools from this PR: [3] and published it to my release page here: [4]. So, now I was able to switch in Gerrit the java_tools to this custom release in this CL: [5]. Relevant pieces for the switch are added lines in the WORKSPACE file and these two lines in .bazelrc:

# WARNING: This is not portable and only added to this CL to activate
# new Error Prone release 2.3.5-SNPASHOT on the CI and trigger
# additional EP checks
build --java_toolchain=@remote_java_tools_linux//:toolchain
build --host_java_toolchain=@remote_java_tools_linux//:toolchain

Also note, that changing java_tools per default to a new releases in Bazel may be problematic because javac or Error Prone may introduce some breaking changes. To avoid that I added this new option in this PR: [6]: --incompatible_use_java_tools_beta_release option.

[1] https://github.com/bazelbuild/bazel/blob/cb5dc080dae009c43b30c2981ee7590e8f9c2ea4/src/BUILD#L565-L580
[2] https://github.com/bazelbuild/bazel/issues/11262#issuecomment-629754123
[3] https://github.com/bazelbuild/bazel/pull/11426
[4] https://github.com/davido/java_tools/releases/tag/javac11-v12.0
[5] https://gerrit-review.googlesource.com/c/gerrit/+/265493
[6] https://github.com/bazelbuild/bazel/pull/11447

[davido]

I uploaded a PR for toolchain 14 support.

I have built and conducted java_tools release here (currently only Linux version).

Example project that demonstrates that Java 14 syntax class can be built out of the box is here:

  $ bazel version
Starting local Bazel server and connecting to it...
Build label: 3.2.0

  $ bazel run java
   bazel run java:Javac14Example 
INFO: Analyzed target //java:Javac14Example (18 packages loaded, 707 targets configured).
INFO: Found 1 target...
INFO: Deleting stale sandbox base /home/davido/.cache/bazel/_bazel_davido/1790aabdce25864b2f68511bd57514be/sandbox
Target //java:Javac14Example up-to-date:
  bazel-bin/java/Javac14Example.jar
  bazel-bin/java/Javac14Example
INFO: Elapsed time: 9.035s, Critical Path: 0.73s
INFO: 0 processes.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
0

The real world example building with JDK 14 toolchain, Gerrit Code Review built on real CI with installed Bazel 3.1.0 only) is here.

Update:

CI logs emitting these warnings, that must be fixed in the toolchain_jdk_14 definition:

OpenJDK 64-Bit Server VM warning: Option UseParallelOldGC was deprecated in version 14.0 and will likely be removed in a future release.
INFO: From Compiling Java headers java/com/google/gerrit/server/libconstants-hjar.jar (1 source file) [for host]:
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.

I would look into new and shiny Z garbage collector: [1] that has seen substantial
improvements in latest JDK releases and is scheduled to be production
ready in upcoming JDK 15 release: [2]. It supports up to 16 TB heap
size, that should be sufficient for medium to big Bazel projects ;-)

But even today I would be interested to hear feedback of running Bazel on JDK14 with ZGC enabled and run existing Bazel load tests.

[1] https://wiki.openjdk.java.net/display/zgc/Main
[2] http://openjdk.java.net/jeps/377

[davido]

@uri-canva

Now, that java_tools_javac14 were published on GitHub (see: https://github.com/bazelbuild/java_tools/releases/tag/javac14_v1.0) can you confirm that it works for you?

I also updated this Java 14 example repository: [1] and this gerrit change: [2] correspondingly.

[1] https://github.com/davido/bazel_java_toolchain14/blob/master/WORKSPACE#L4
[2] https://gerrit-review.googlesource.com/c/gerrit/+/269382

[MikhailTymchukFT]

Does it work for MacOS as well?

[davido]

Does it work for MacOS as well?

Sure. All you need is to add these lines to your WORKSPACE file:

# Java tools javac14 for Darwin
http_archive(
    name = "remote_java_tools_darwin",
    sha256 = "e20f002ceb3f3353d64c022e1f3400d8539ee56ffcfd4a6680d73d6a2cac938b",
    urls = [
        "https://mirror.bazel.build/bazel_java_tools/releases/javac14/v1.0/java_tools_javac14_darwin-v1.0.zip",
        "https://github.com/bazelbuild/java_tools/releases/download/javac14-v1.0/java_tools_javac14_darwin-v1.0.zip",
    ],
)

# Zulu OpenJDK for Darwin
http_archive(
    name = "openjdk14_darwin_archive",
    build_file_content = "java_runtime(name = 'runtime', srcs =  glob(['**']), visibility = ['//visibility:public'])",
    sha256 = "088bd4d0890acc9f032b738283bf0f26b2a55c50b02d1c8a12c451d8ddf080dd",
    strip_prefix = "zulu14.28.21-ca-jdk14.0.1-macosx_x64",
    urls = ["https://mirror.bazel.build/cdn.azul.com/zulu/bin/zulu14.28.21-ca-jdk14.0.1-macosx_x64.tar.gz"],
)

And then invoke bazel build command with these options:

  $ bazel build \
     --java_toolchain=@remote_java_tools_darwin//:toolchain_jdk_14 \
     --host_java_toolchain=@remote_java_tools_darwin//:toolchain_jdk_14 \
     --javabase=@openjdk14_darwin_archive//:runtime \
     --host_javabase=@openjdk14_darwin_archive//:runtime \
   :foo

[uri-canva]

@davido Can confirm it's building our codebase correctly on both macOS and Linux 🙇 .

[Ubehebe]

As an FYI, it's possible to extend this approach to get Java 14 preview features (records, text blocks) to work with Bazel. (Note: Java preview features probably shouldn't be used in production.) This involves passing --enable-preview to both javac and the JVM. You can get most of the way there by defining a java_toolchain similar to @remote_java_tools_linux//:toolchain_jdk_14 that contains --enable-preview in both its javacopts and jvm_opts. Your .bazelrc should also set --jvmopt --enable-preview for bazel running java_binary targets that contain preview-feature-specific bytecode.

If you do this, you may be interested in a few issues I ran into along the way: https://github.com/bazelbuild/bazel/issues/8585, https://github.com/bazelbuild/bazel/issues/11893, https://github.com/protocolbuffers/protobuf/commit/6bbd56dfd9600adc9b084028d99a0aed880d7622.