Bazel: java_plugin header compilation failure

[jfancher]

Description of the problem / feature request:

Two parts to this issue:

First, I ran into an exception during Java header compilation. Turbine does an unsafe cast here, which caused an annotation processor to fail during header compilation. This seems like a straightforward bug -- I'm not sure if there's a better place to report it since the turbine repo does not allow issues.

Second, and more bazel-specific, I only ran into this issue upon upgrading to JDK 15. I followed the process outlined here to use the JDK 15 release of the remote tools. For some reason, this failure only happens with that toolchain, not with Java 8 or 11. Is there something different about the turbine jar it uses or how it invokes it? I could not figure out the difference.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Full reproducer here: https://github.com/jfancher/bazel-issues/tree/turbine-elements

The basic setup is:

• Have an annotation processor (and java_plugin) P that calls the offending method (note: with generates_api flag)
• Have a lib A that uses P
• Have a lib B that references A
• Build B

What operating system are you running Bazel on?

macOS

What's the output of bazel info release?

release 3.7.0

Have you found anything relevant by searching the web?

Nope.

Any other information, logs, or outputs that you want to share?

Full build log:

$ bazel build --config=jdk15 //:C2
INFO: Build options --host_java_toolchain, --host_javabase, --java_toolchain, and 1 more have changed, discarding analysis cache.
INFO: Analyzed target //:C2 (1 packages loaded, 714 targets configured).
INFO: Found 1 target...
INFO: From Compiling Java headers libAnno-hjar.jar (1 source file):
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
INFO: From Compiling Java headers libAnno-hjar.jar (1 source file):
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
INFO: From Building libC1.jar (1 source file) and running annotation processors (Proc):
warning: deprecated

ERROR: /Users/jfancher/github/jfancher/bazel-issues/BUILD.bazel:14:13: Compiling Java headers libC1-hjar.jar (1 source file) and running annotation processors (Proc) failed (Exit 1): java failed: error executing command external/jdk15_macos/bin/java -Xverify:none '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' ... (remaining 12 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox java failed: error executing command external/jdk15_macos/bin/java -Xverify:none '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' ... (remaining 12 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
<>: error: java.lang.ClassCastException: class com.google.turbine.processing.TurbineElement$TurbineExecutableElement cannot be cast to class com.google.turbine.processing.TurbineElement$TurbineTypeElement (com.google.turbine.processing.TurbineElement$TurbineExecutableElement and com.google.turbine.processing.TurbineElement$TurbineTypeElement are in unnamed module of loader 'app')
    at com.google.turbine.processing.TurbineElements.isDeprecated(TurbineElements.java:133)
    at ex.Proc.process(Proc.java:22)
    at com.google.turbine.binder.Processing.process(Processing.java:181)
    at com.google.turbine.binder.Binder.bind(Binder.java:102)
    at com.google.turbine.main.Main.bind(Main.java:246)
    at com.google.turbine.main.Main.fallback(Main.java:221)
    at com.google.turbine.main.Main.compile(Main.java:165)
    at com.google.turbine.main.Main.compile(Main.java:127)
    at com.google.turbine.main.Main.main(Main.java:84)

[comius]

cc @cushon

[cushon]

Thanks for the report and the nice repro. I agree the cast seems like a straightforward bug.

(The turbine repo now accepts issues also, I think that was just an oversight.)

The mac/linux difference here is because it's using different turbine implementations, see turbine_direct_binary_deploy.jar vs. turbine_deploy.jar. @comius do you know why that's happening here? I'd expect that behaviour to be the same for
a particular java_tools release, regardless of whether it was the mac or linux version. There was a recent-ish switch to using the 'direct' version of turbine everywhere (https://github.com/bazelbuild/bazel/commit/10ffddb13ac070f2c6b9d64cb927ab3b659c0b72); I'm not sure why we're only seeing it for --cpu=darwin here.

$ bazel aquery --cpu=darwin --config=jdk15 //:C1
...
action 'Compiling Java headers libC1-hjar.jar (1 source file) and running annotation processors (Proc)'
  Mnemonic: JavacTurbine
  Target: //:C1
  Configuration: darwin-fastbuild
...
  Command Line: (exec external/jdk15_linux/bin/java \
...
    -jar \
    external/jdk15_remote_java_tools_darwin/java_tools/turbine_direct_binary_deploy.jar \

$ bazel aquery --config=jdk15 //:C1
...
action 'Compiling Java headers libC1-hjar.jar (1 source file) and running annotation processors (Proc)'
  Mnemonic: JavacTurbine
  Target: //:C1
  Configuration: k8-fastbuild
...
  Command Line: (exec external/jdk15_linux/bin/java \
...
    external/jdk15_remote_java_tools_linux/java_tools/turbine_deploy.jar \

[cushon]

I submitted a fix: https://github.com/google/turbine/commit/b2a0c703a7c870f3f895fce1a00231085c1a9ae9

We'll need to update the version of turbine in java_tools for the fix to be visible in Bazel.

[comius]

Linux in the first output of $ bazel aquery --cpu=darwin --config=jdk15 //:C1 is explained by not using --host_cpu = darwin, and in Bazel java runtime for JavaBuilder or Turbine is still determined by host transition.

The second query $ bazel aquery --config=jdk15 //:C1 is "explained", by there is a different version of java_toolchain_default.bzl packed into java_tools_linux 10.1 and java_tools_darwin 10.1. For rc1 (before 10ffddb) tests failed and I reuploaded rc1 (when test were disabled and after 10ffddb). This somehow mixed versions, having content java_tools_linux older and content of java_tools_darwin newer (ironically the failing test was on windows).

I checked version 10.4 is fine (in this respect, not turbine fix). https://github.com/bazelbuild/java_tools/releases/tag/javac11_v10.4

java_tools v11.0 will only work on bazel@head

[comius]

@jfancher would you mind adding a license to your reproduction repo? I would like to use the code for the regression test.

The failure happens on all JDK versions (not just 15).

[comius]

For blaze versions <= 4.0.0 the issues is fixed by java_tools release 10.5 https://github.com/bazelbuild/java_tools/releases/tag/javac11_v10.5

[jfancher]

When you say it happens on all versions, do you mean at Bazel head? Or with that tools release? To clarify my original report -- on macOS, Bazel 3.7, and Java 11, my reproducer build passes (for me). I updated the .bazelrc to specify the exact Java 11 runtime/toolchain I tested with; I forgot to push that previously.

Anyway, my confusion aside, it sounds like you figured out what's going on -- thanks! I've added a license as requested; let me know if there's anything else I can do.

[comius]

When you say it happens on all versions, do you mean at Bazel head? Or with that tools release?

It worked for you because of double failure. java_tools release 10.1 incorrectly uses Turbine instead of TurbineDirect on some OS, JDK combinations.

With newer java_tools TurbineDirect is correctly used and fails for all JDK versions.

[philwo]

This is currently marked as an open release blocker for Bazel 4.0 - is this still the case?

[comius]

This is currently marked as an open release blocker for Bazel 4.0 - is this still the case?

Not blocker, it was cherrypicked for 4.0. Still need to release java_tools at @head before I can close it.