Bazel: Bazel build is failing due to ln error - ln: failed to create symbolic link '/tmp/bazel_C1FCGEax/embedded_tools/tools/jdk/BUILD.pkg': File exists when run second time

[srikanthjoshi]

ATTENTION! Please read and follow:

• if this is a _question_ about how to build / test / query / deploy using Bazel, ask it on StackOverflow instead: https://stackoverflow.com/questions/tagged/bazel
• if this is a _discussion starter_, send it to [email protected]
• if this is a _bug_ or _feature request_, fill the form below as best as you can.

Description of the problem / feature request:

bazel build is failing with --host_javabase not set. But after failure, if I run compile again, it is saying ln failed with some file exists in /tmp. But that file doesn't exist anymore.

FYI:
[root@brazossrik01 bazel]# ./compile.sh --host_javabase /usr/bin/java
🍃 Building Bazel from scratch......
🍃 Building Bazel with Bazel.
.WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
ERROR: /tmp/bazel_e9MSVlMl/out/external/bazel_tools/tools/jdk/BUILD:443:14: Configurable attribute "actual" doesn't match this configuration: Could not find a JDK for host execution environment, please explicitly provide one using --host_javabase.
ERROR: Analysis of target '//src:bazel_nojdk' failed; build aborted:

/tmp/bazel_e9MSVlMl/out/external/bazel_tools/tools/jdk/BUILD:443:14: Configurable attribute "actual" doesn't match this configuration: Could not find a JDK for host execution environment, please explicitly provide one using --host_javabase.
INFO: Elapsed time: 5.915s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (118 packages loaded, 945 targets
configured)
currently loading: src/main/java/com/google/devtools/build/lib/profiler/me
mory ... (6 packages)

ERROR: Could not build Bazel
[root@brazossrik01 bazel]# ./compile.sh --host_javabase /usr/bin/java
🍃 Building Bazel from scratch...ln: failed to create symbolic link '/tmp/bazel_C1FCGEax/embedded_tools/tools/jdk/BUILD.pkg': File exists
[root@brazossrik01 bazel]# env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
🍃 Building Bazel from scratch...ln: failed to create symbolic link '/tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg': File exists <<<<
[root@brazossrik01 bazel]# ls -l /tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg
ls: cannot access '/tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg': No such file or directory <<<<
[root@brazossrik01 bazel]#

Feature requests: what underlying problem are you trying to solve with this feature?

bazel build is failing with ln error. If we remove bazel and do wget, then bazel build will fail with --host_javabase error. I am working on that error. This issue is for why link problem is seen when trying to build.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. wget https://github.com/bazelbuild/bazel/releases/download/0.24.0/bazel-0.24.0-dist.zip
2. unzip bazel-0.24.0-dist.zip
3. ./compile.sh --host_javabase /usr/bin/java
4. ./compile.sh --host_javabase /usr/bin/java - Run again and it fails

What operating system are you running Bazel on?

RHEL 8.0 and IBM PPC64 LE

What's the output of bazel info release?

Can't run as build is failing.

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Can't run as build is failing.

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/20773773/467ebd80-4fef-11e9-9c9b-b8bce3551db9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190404%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190404T062157Z&X-Amz-Expires=300&X-Amz-Signature=f7a801f6253bfdbf5c8d29c2a14585881a18e7ed0fa918697287bcd68a724943&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dbazel-0.24.0-dist.zip&response-content-type=application%2Foctet-stream [following]
--2019-04-04 01:21:57-- https://github-production-release-asset-2e65be.s3.amazonaws.com/20773773/467ebd80-4fef-11e9-9c9b-b8bce3551db9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190404%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190404T062157Z&X-Amz-Expires=300&X-Amz-Signature=f7a801f6253bfdbf5c8d29c2a14585881a18e7ed0fa918697287bcd68a724943&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dbazel-0.24.0-dist.zip&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.169.227
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.169.227|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 174691612 (167M) [application/octet-stream]

Have you found anything relevant by searching the web?

Replace these lines with your answer.

Places to look:

• StackOverflow: http://stackoverflow.com/questions/tagged/bazel
• GitHub issues: https://github.com/bazelbuild/bazel/issues
• email threads on https://groups.google.com/forum/#!forum/bazel-discuss

Nothing usual found for this ln failure problem

Any other information, logs, or outputs that you want to share?

[root@brazossrik01 bazel]# ./compile.sh --host_javabase /usr/bin/java
🍃 Building Bazel from scratch......
🍃 Building Bazel with Bazel.
.WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
ERROR: /tmp/bazel_e9MSVlMl/out/external/bazel_tools/tools/jdk/BUILD:443:14: Configurable attribute "actual" doesn't match this configuration: Could not find a JDK for host execution environment, please explicitly provide one using --host_javabase.
ERROR: Analysis of target '//src:bazel_nojdk' failed; build aborted:

/tmp/bazel_e9MSVlMl/out/external/bazel_tools/tools/jdk/BUILD:443:14: Configurable attribute "actual" doesn't match this configuration: Could not find a JDK for host execution environment, please explicitly provide one using --host_javabase.
INFO: Elapsed time: 5.915s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (118 packages loaded, 945 targets
configured)
currently loading: src/main/java/com/google/devtools/build/lib/profiler/me
mory ... (6 packages)

ERROR: Could not build Bazel
[root@brazossrik01 bazel]#

[root@brazossrik01 bazel]# ./compile.sh --host_javabase /usr/bin/java
🍃 Building Bazel from scratch...ln: failed to create symbolic link '/tmp/bazel_C1FCGEax/embedded_tools/tools/jdk/BUILD.pkg': File exists
[root@brazossrik01 bazel]#

[root@brazossrik01 bazel]# env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
🍃 Building Bazel from scratch...ln: failed to create symbolic link '/tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg': File exists
[root@brazossrik01 bazel]#

[root@brazossrik01 bazel]# ls -l /tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg
ls: cannot access '/tmp/bazel_WPn8tXvn/embedded_tools/tools/jdk/BUILD.pkg': No such file or directory
[root@brazossrik01 bazel]#

[zjchong]

I have the same problem...

[bamert]

I have the same problem as well on CentOS

[hoonkai]

Same here on Ubuntu 18.04.

[smparkes]

FWIW, this happens in Alpine and occurs in 0.24.{0,1} but doesn't occur in 0.23.2.

[jin]

cc @lberki bumping to P1 due to the number of reports.

[sethbruder]

In case it's helpful and can save you the several hours it cost me:
Check whether you are building with JDK 8.
If so, try building with JDK 10.

I had a very similar error message: it seems that despite the docs (https://docs.bazel.build/versions/master/install-compile-source.html) -- which, as of today say only JDK 8 is supported -- JDK 8 is no longer well supported (see https://blog.bazel.build/2018/09/14/bazel-0.17.html for some hints about this, but I did not find those workarounds worked for later releases).
You do still need the "--host_javabase=@local_jdk//:jdk" magic.

If you look at the referenced line of code, you will see that the error occurs when applying an alias, remote_jdk10 -- which is a hint. Of course it would be nice if one were instead notified that his or her request for the local JDK was being ignored, and why.

I'm sure Bazel is a great piece of software -- and it has gotten better -- but, as someone outside Google (and therefore perhaps not privy to some internal lore), building Bazel, and working around it's many interwoven and often hard-coded or deeply embedded expectations, continues to be the hardest part of building Tensorflow.

[samikama]

I am building with JDK 11 which is current default on ubuntu18.04 and it is still failing.

[sethbruder]

@samikama I am not a Bazel expert, but I would suggest taking a look at the release notes for your version of Bazel and the several prior versions. Also take a look at the context of the specific line of code referenced in the error message to get a hint about which JDK bazel is trying to find. The release notes seem to suggest that some versions need a special command-line option or EXTRA_BAZEL_ARGS setting to get JDK 11 to work. Make sure you also set host_javabase. I also explicitly set JAVA_HOME (and PATH of course), although I don't know whether Bazel uses this.

[a-gn]

Same problem on Ubuntu 16.04:

env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
🍃  Building Bazel from scratch...ln: failed to create symbolic link '/tmp/bazel_7LpWZa6W/embedded_tools/tools/jdk/BUILD.pkg': File exists

I got the same result with OpenJDK 8, 10 and 11.

[dev-guy]

I get this when trying to build openssl per https://github.com/openssl/openssl/issues/3840 on Ubuntu .. works on OSX though

[Cypher1]

Still seeing this. Does anyone have a work around?

For me I only see it when trying to set up caching for the bazel directories using travis's cache configuration with bazel output directories.

[ZouYunzhe]

I use bazel version 0.24.1 and openjdk 8. This error occurs if bazel fails to build at the first time. I spent hours solving this problem and here's my solution. Actually it's quite simple.
1) remove the whole bazel folder
2) unzip the dist.zip file again to get a new one
3) env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
I hope it also works for you and saves your time.

[jin]

Now documented here: https://docs.bazel.build/versions/master/install-compile-source.html#bootstrap-unix-bootstrap