Nixpkgs: bazel: building java targets fails because bazel downloads ijar binary

Created on 13 Jun 2019  路  13Comments  路  Source: NixOS/nixpkgs

Issue description

Bazel fetches a binary at runtime and tries to execute it, which of course fails on NixOS:

ERROR: /home/philip/.cache/bazel/_bazel_philip/cccb2d2da8416067607b8c5cff088df0/external/org_apache_spark_spark_core_2_10/BUILD:7:1: Extracting interface @org_apache_spark_spark_core_2_10//:org_apache_spark_spark_core_2_10 failed (Exit 1) ijar failed: error executing command external/remote_java_tools_linux/java_tools/ijar/ijar external/org_apache_spark_spark_core_2_10/spark-core_2.10-1.6.0.jar ... (remaining 3 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
src/main/tools/linux-sandbox-pid1.cc:427: "execvp(external/remote_java_tools_linux/java_tools/ijar/ijar, 0x62e860)": No such file or directory

Related to https://github.com/bazelbuild/bazel/issues/8575

Steps to reproduce

Build any Java rule with bazel 0.26.1 in nixpkgs on NixOS.

cc @groodt @kalbasit

bug
Source
picture Profpatsch

All 13 comments

I'm having the same issue. They have a new target for the java_runtime, called @bazel_tools//tools/jdk:absolute_javabase: https://github.com/bazelbuild/bazel/blob/cfc6800c64bc97f433e602d0e9d0753b3036502e/tools/jdk/BUILD#L67-L75. I hardcoded the ABSOLUTE_PATH, but it did not work.

diff --git a/src/.bazelrc b/src/.bazelrc
index d67eb3c81..5a2bcc9ce 100644
--- a/src/.bazelrc
+++ b/src/.bazelrc
@@ -48,20 +48,11 @@ test --features=race
 # https://docs.bazel.build/versions/master/be/java.html#java_toolchain
 # https://github.com/bazelbuild/bazel/blob/17794d58cc493d918a98c3250302ca44b1d53e3f/tools/jdk/default_java_toolchain.bzl#L84-L93

-build --host_java_toolchain='@bazel_tools//tools/jdk:toolchain_hostjdk8'
-build --host_javabase='@local_jdk//:jdk'
-build --java_toolchain='@bazel_tools//tools/jdk:toolchain_hostjdk8'
-build --javabase='@local_jdk//:jdk'
-build --proto_toolchain_for_java='@bazel_tools//tools/jdk:toolchain_hostjdk8'
+build --host_javabase='@bazel_tools//tools/jdk:absolute_javabase'
+build --define=ABSOLUTE_JAVABASE=/nix/store/543yiclmkvmpy8b9fg00y0bw4xwh3865-openjdk-8u212-ga/lib/openjdk
kalbasit picture kalbasit on 13 Jun 2019

I suspect it's coming from here which is called from here. The absolute_javabase java_runtime defines java_home but not the ijar target.

kalbasit picture kalbasit on 13 Jun 2019

@Profpatsch have you made any progress on this?

kalbasit picture kalbasit on 14 Jun 2019

tbh I was hoping someone else would fix it. :P

I don鈥檛 have much time allocated to bazel work for next week, so it might be ~10 days until I can take an earnest stab.

Profpatsch picture Profpatsch on 15 Jun 2019

I was honestly expecting the following patch to work, but :man_shrugging:! I'm not really sure why it's not working.


Force Bazel to build ijar locally

diff --git a/tools/jdk/BUILD b/tools/jdk/BUILD
index d99fb7adb9..b582f7ba67 100644
--- a/tools/jdk/BUILD
+++ b/tools/jdk/BUILD
@@ -327,24 +327,12 @@ default_java_toolchain(

 alias(
     name = "toolchain",
-    actual = select({
-        "//src/conditions:darwin": "@remote_java_tools_darwin//:toolchain",
-        "//src/conditions:darwin_x86_64": "@remote_java_tools_darwin//:toolchain",
-        "//src/conditions:windows": "@remote_java_tools_windows//:toolchain",
-        "//src/conditions:linux_x86_64": "@remote_java_tools_linux//:toolchain",
-        "//conditions:default": "@bazel_tools//tools/jdk:legacy_toolchain",
-    }),
+    actual = "@bazel_tools//tools/jdk:legacy_toolchain",
 )

 alias(
     name = "remote_toolchain",
-    actual = select({
-        "//src/conditions:darwin": "@remote_java_tools_darwin//:toolchain",
-        "//src/conditions:darwin_x86_64": "@remote_java_tools_darwin//:toolchain",
-        "//src/conditions:windows": "@remote_java_tools_windows//:toolchain",
-        "//src/conditions:linux_x86_64": "@remote_java_tools_linux//:toolchain",
-        "//conditions:default": "@bazel_tools//tools/jdk:legacy_toolchain",
-    }),
+    actual = "@bazel_tools//tools/jdk:legacy_toolchain",
 )

 # The 'vanilla' toolchain is an unsupported alternative to the default.
diff --git a/tools/jdk/BUILD.java_tools b/tools/jdk/BUILD.java_tools
index 7d4ee45498..30d26ebd5c 100644
--- a/tools/jdk/BUILD.java_tools
+++ b/tools/jdk/BUILD.java_tools
@@ -18,7 +18,7 @@ java_toolchain(
     genclass = [":Genclass"],
     header_compiler = [":Turbine"],
     header_compiler_direct = [":TurbineDirect"],
-    ijar = [":ijar"],
+    ijar = [":ijar_cc_binary"],
     javabuilder = [":JavaBuilder"],
     javac = [":javac_jar"],
     javac_supports_workers = 1,
@@ -52,7 +52,7 @@ java_toolchain(
         "-g",
         "-parameters",
     ],
-    singlejar = [":singlejar"],
+    singlejar = [":singlejar_cc_bin"],
     source_version = "8",
     target_version = "8",
     tools = [
diff --git a/tools/jdk/BUILD.java_tools.old b/tools/jdk/BUILD.java_tools.old
index ff438af752..6b6e2aea3f 100644
--- a/tools/jdk/BUILD.java_tools.old
+++ b/tools/jdk/BUILD.java_tools.old
@@ -10,7 +10,7 @@ java_toolchain(
     genclass = [":Genclass"],
     header_compiler = [":turbine"],
     header_compiler_direct = [":turbine_direct"],
-    ijar = [":ijar"],
+    ijar = [":ijar_cc_binary"],
     javabuilder = [":javabuilder"],
     javac = [":javac_jar"],
     tools = [
@@ -52,7 +52,7 @@ java_toolchain(
         # Restrict protos to Java 7 so that they are compatible with Android.
        "proto": ["-source", "7", "-target", "7"],
     },
-    singlejar = [":singlejar"],
+    singlejar = [":singlejar_cc_bin"],
     bootclasspath = ["@bazel_tools//tools/jdk:platformclasspath"],
 )

kalbasit picture kalbasit on 15 Jun 2019

This issue is typical with bazel on nixos: every "prebuilt" binary fetched by any bazel rule will fail with the same issue.

We can fix each issue manually as they arise, but can we do better in a potential more generic way?

From what I know, the only place bazel may generate binaries from the outerworld are in repository rules, and especially the following rules:

What are your thoughts about applying a recursive patchelf to patch at least the dynamic loader on the result of these rules?

Another option may be to just patch https://docs.bazel.build/versions/master/skylark/lib/repository_ctx.html#execute to rewrite the call to /path/to/know/ld/loader binary_path instead.

guibou picture guibou on 18 Jun 2019

This issue is typical with bazel on nixos: every "prebuilt" binary fetched by any bazel rule will fail with the same issue.

We can fix each issue manually as they arise, but can we do better in a potential more generic way?

From what I know, the only place bazel may generate binaries from the outerworld are in repository rules, and especially the following rules:

What are your thoughts about applying a recursive patchelf to patch at least the dynamic loader on the result of these rules?

It would be wonderful if that would work, as we don't have to keep specifying Nix provided toolchains. Making this work well might not be a trivial task, as you would most likely have to replicate autoPatchelfHook.

Another option may be to just patch https://docs.bazel.build/versions/master/skylark/lib/repository_ctx.html#execute to rewrite the call to /path/to/know/ld/loader binary_path instead.

How would this allow it to find, say go or javac?

kalbasit picture kalbasit on 18 Jun 2019

It would be wonderful if that would work, as we don't have to keep specifying Nix provided toolchains. Making this work well might not be a trivial task, as you would most likely have to replicate autoPatchelfHook.

Yes. That's the most difficult solution.

How would this allow it to find, say go or javac?

I don't understand. The only things I'm thinking about is that, instead of using execute(arguments=foo) we rewrite it as execute(arguments=["path/to/a/known/ld/linker", call_to_which(foo[0])] + foo[1:]).

guibou picture guibou on 18 Jun 2019

It would be wonderful if that would work, as we don't have to keep specifying Nix provided toolchains. Making this work well might not be a trivial task, as you would most likely have to replicate autoPatchelfHook.

Yes. That's the most difficult solution.

How would this allow it to find, say go or javac?

I don't understand. The only things I'm thinking about is that, instead of using execute(arguments=foo) we rewrite it as execute(arguments=["path/to/a/known/ld/linker", call_to_which(foo[0])] + foo[1:]).

I see what you mean now. I think the autoPatchelfHook is more elegant. Given the amount of work that might require, it might be prudent to just fix the immediate issue right now. @guibou would you be able to take a stab at it?

kalbasit picture kalbasit on 18 Jun 2019

@kalbasit I'm on it.

guibou picture guibou on 18 Jun 2019
🚀1

Progress: I've built a local package with fixed ijar and I'm using --override_repository to point to that package. This setting is done inside a system-wide bazelrc file.

Hopefully I'll be able to open a PR today.

guibou picture guibou on 19 Jun 2019
🎉1

That's great news @guibou! Thank you! Please include a Java test (similar to current python test) with your PR.

kalbasit picture kalbasit on 19 Jun 2019

That's great news @guibou! Thank you! Please include a Java test (similar to current python test) with your PR.

AFAIK there is a java test in the installCheck phase that I also repaired.

guibou picture guibou on 20 Jun 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

chris-martin picture chris-martin  路  3Comments
teto picture teto  路  3Comments
retrry picture retrry  路  3Comments
rzetterberg picture rzetterberg  路  3Comments
copumpkin picture copumpkin  路  3Comments