Bazel: Tests which use java stub template with large classpaths fails because of local_jdk implicit dependency

[ittaiz]

Description of the problem / feature request:

Running a java_test (we're actually using scala_junit_test but that uses the java_stub_template.txt) with a very long classpath fails since the template assumes existence of local_jdk/bin/jar in runfiles.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I'll try to work on a full repro but given a target with a very long classpath using the java_stub_template (java_test/ scala_junit_test) try to test it with bazel 0.19.0 (maybe reproduces earlier) and it will fail with .../foo.runfiles/local_jdk/bin/jar: No such file or directory

I've worked around it for now by adding the following data dependency explicitly: "@bazel_tools//tools/jdk:jar",.
This generates a warning about using a deprecated dependency.

What operating system are you running Bazel on?

OS X

What's the output of bazel info release?

release 0.19.0

Have you found anything relevant by searching the web?

java_stub_template relevant line: https://github.com/bazelbuild/bazel/blob/1afef648af1851121c54f9e138d0e94cdd27ea00/src/main/java/com/google/devtools/build/lib/bazel/rules/java/java_stub_template.txt#L354
I think this is related to @cushon's work on #5594

[ittaiz]

cc @meisterT

[meisterT]

cc @iirina

[cushon]

@laszlocsomor @meteorcloudy

I thought we were only using the classpath jar hack on windows, but that changed in 102ce6ddc04063d26165c6a2167059f2348026bf because not all JDKs support @argument files. Note that OpenJDK 9 and newer do support parameter files, so there's a more principled fix available in that case: #6354

[ittaiz]

For now we still use JDK 8... I think we'll move in a few months but now we
have a problem.
Also my workaround won't work on RBE, right? Is there a workaround I can do
to use current java runtime? Note I'm willing to mangle the
java_stub_template in my custom test rule for the time being until bazel
releases a fix (I guess that will probably land only at February)

On Tue, Dec 18, 2018 at 7:21 PM Liam Miller-Cushon [e-mail]
wrote:

@laszlocsomor https://github.com/laszlocsomor @meteorcloudy
https://github.com/meteorcloudy

I thought we were only using the classpath jar hack on windows, but that
changed in 102ce6d
https://github.com/bazelbuild/bazel/commit/102ce6ddc04063d26165c6a2167059f2348026bf
because not all JDKs support @argument files. Note that OpenJDK 9 and
newer do support parameter files, so there's a more principled fix
available in that case: #6354
https://github.com/bazelbuild/bazel/issues/6354

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/6955#issuecomment-448299900,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFyFbEYiBWZ8ZMQ7_QhhJNZHBF2Dqks5u6SQFgaJpZM4ZXsIw
.

[ittaiz]

any thoughts? also #6354 is still open

[laszlocsomor]

I'm not a domain expert, but I think Bazel could set JARBIN akin to JAVABIN as it does here:

https://github.com/bazelbuild/bazel/blob/f55ae7a5b90da9b2b954c90a84e6e0d38e85a73e/src/main/java/com/google/devtools/build/lib/bazel/rules/java/java_stub_template.txt#L231-L232

https://github.com/bazelbuild/bazel/blob/f55ae7a5b90da9b2b954c90a84e6e0d38e85a73e/src/main/java/com/google/devtools/build/lib/bazel/rules/java/BazelJavaSemantics.java#L298-L300

[beala-stripe]

I have also run into this issue in a java_binary rule while setting the --javabase and --host_javabase to a custom target (an Azul JDK8). Thankfully @ittaiz's workaround worked (adding a data dep to the target: data = ["@local_jdk//:bin/jar"])

[ittaiz]

Any news about this?

[beala-stripe]

I'm also eager for a fix for this. Although adding the data dep gets the build working, I worry that depending on the local jdk negatively affects caching.

[ittaiz]

@cushon @laszlocsomor's suggestion sounds really reasonable and small. WDYT?
An even simpler (but hackier) solution can be to get the parent directory of JAVABIN and then append jar to it.

[ittaiz]

@cushon @lberki I've implemented the above in rules_scala (copied the stub template file).
WDYT? We need this in many more places and don't want to use the deprecated local_jdk.

[ittaiz]

@cushon @lberki wdyt?

[ittaiz]

Any news?

[joshrosen-stripe]

Maybe we should submit a PR to more clearly show the effective diff? As it stands now, this discussion links to PRs against bazelbuild/rules_scala and in that PR it's a little tricky to see the actual net change because it duplicates the entire java stub template. If we submitted a PR directly against bazelbuild/bazel then it might be easier to reason about exactly what's being proposed.

[iirina]

@ittaiz is this still happening?

[TheImplementer]

We've bumped into this issue as well, it looks like it's still happening. (bazel v2.2.0)