Bazel: Illegal reflective access by com.google.protobuf.UnsafeUtil

[buchgr]

After having upgraded Bazel's protobuf to 3.6.0 and Bazel's JDK to 9, we are seeing these warnings wen building protobufs. It's an annoying cosmetic issue.

INFO: From Compiling Java headers src/main/java/com/google/devtools/build/lib/windows/libwindows-hjar.jar (3 files):
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/usr/local/google/home/buchgr/.cache/bazel/_bazel_buchgr/install/e676aae4e6c63bf8d7cfde16175ea874/_embedded_binaries/embedded_tools/tools/jdk/turbine_deploy.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: From Compiling Java headers src/main/java/com/google/devtools/build/lib/libunix-hjar.jar (10 files) [for host]:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/usr/local/google/home/buchgr/.cache/bazel/_bazel_buchgr/install/e676aae4e6c63bf8d7cfde16175ea874/_embedded_binaries/embedded_tools/tools/jdk/turbine_deploy.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
...

Any thoughts @cushon @rongjiecomputer? Can we possibly prevent them from being printed? The corresponding protobuf issue: https://github.com/google/protobuf/issues/3781

[rongjiecomputer]

My local workaround is adding build --jvmopt=--add-opens=java.base/java.nio=ALL-UNNAMED to .bazelrc.

[cushon]

@buchgr the toolchain includes JVM flags that are supposed to quiet those warnings, but I guess they aren't working:

https://github.com/bazelbuild/bazel/blob/6c8166e2ff915f8a903365c9ec223068ae453751/tools/jdk/default_java_toolchain.bzl#L42

[cushon]

Ah, we also want --add-opens=java.base/java.lang=ALL-UNNAMED.

[jowagner]

This may be more than just cosmetic. I get this error when trying to build bazel with compile.sh. A few lines after the above error, it says:

ERROR: /build/src/main/java/com/google/devtools/build/lib/BUILD:1417:1: Building src/main/java/com/google/devtools/build/lib/bazel/BazelServer.jar () failed: Worker process returned an unparseable WorkResponse!

Did you try to print something to stdout? Workers aren't allowed to do this, as it breaks the protocol between Bazel and the worker process.

See also issue https://github.com/protocolbuffers/protobuf/issues/5008

[cushon]

I'd expect those warnings to be printed to stderr, not stdout, so they shouldn't affect workers.

[jowagner]

I applied the patch https://github.com/bazelbuild/bazel/commit/543918dcfd1744422273962eede43dede297e0f5 to bazel-0.16.1-dist.zip and still get the "Illegal reflective access" warning followed by the error in my above comment (this time in "src/java_tools/junitrunner/java/com/google/testing/junit/runner/BUILD:52:1") when running bash ./compile.sh. While the error may be unrelated to issue #5599, shouldn't the warning go away with the patch?

[jowagner]

Found issue #6009 that discusses the broken workers. Both warning and error go away when using JDK 8 instead of 11. One should read https://docs.bazel.build/versions/master/install-compile-source.html each time when compiling on a new system.

[connesso]

Tried to install tensorflow with Bazel 0.17.2

[7,813 / 12,405] 32 actions running
    Compiling tensorflow/stream_executor/stream.cc [for host]; 50s local
    Compiling tensorflow/core/util/batch_util.cc [for host]; 50s local
    Compiling tensorflow/core/distributed_runtime/master_session.cc; 34s local
    Compiling tensorflow/core/common_runtime/direct_session.cc; 30s local
    Compiling tensorflow/core/kernels/remote_fused_graph_execute_utils.cc; 29s local
    Compiling tensorflow/core/kernels/argmax_op.cc; 29s local
    Compiling tensorflow/core/kernels/argmax_op_gpu.cu.cc; 29s local
    Compiling tensorflow/core/kernels/l2loss_op_gpu.cu.cc; 28s local ...

Server terminated abruptly (error code: 14, error message: '', log file: '/home/xxx/.cache/bazel/_bazel_xxx/1ac92dbf7fec9d8d5e9cfda761e288d7/server/jvm.out')

xxx@yyy:~/tensorflow$ cat /home/xxx/.cache/bazel/_bazel_xxx/1ac92dbf7fec9d8d5e9cfda761e288d7/server/jvm.out
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/home/xxx/.cache/bazel/_bazel_xxx/install/792a28b07894763eaa2bd870f8776b23/_embedded_binaries/A-server.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

So it doesn't seem just cosmetic...

[andreamlin]

I had run into these same errors, along with the Server terminated abruptly (error code: 14, error message: '', log file:.../jvm.out` message, but it turns out that I was just getting an out-of-memory error while running the bazel target in a CircleCI container with limited memory. After constraining bazel to run with a smaller memory footprint (tips from SO), the target built as expected.

[NelsonPython]

Ubuntu 18.04.1 Server, JDK 8
Attempting to load bazel using:
./bazel-0.18.0-installer-linux-x86_64.sh --user
worked great last week. now get

Build informations

• [Commit] (https://github.com/bazelbuild/bazel/commit/d13df47)
  Uncompressing...WARNING: An illegal reflective access operation has occurred
  WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil

Can't load bazel - any ideas?

[jlturriff]

I'm getting it also while trying to build tensorflow in my OpenSuSE Leap 15.1 system:

/usr/local/tensorflow-master
█ ./configure
WARNING: detected http_proxy set in env, setting no_proxy for localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/root/.cache/bazel/_bazel_root/install/73e92e4d21505d8fa81af652d5c23b82/_embedded_binaries/A-server.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=159
INFO: Reading rc options for 'version' from /usr/local/tensorflow-master/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
ERROR: Unrecognized option: --experimental_repo_remote_exec
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
INFO: Invocation ID: 3fdad2d2-f19c-41b0-bbcd-dec1ce9c73eb
Traceback (most recent call last):
  File "./configure.py", line 1548, in <module>
    main()
  File "./configure.py", line 1368, in main
    _TF_MAX_BAZEL_VERSION)
  File "./configure.py", line 483, in check_bazel_version
    ['bazel', '--batch', '--bazelrc=/dev/null', 'version'])
  File "./configure.py", line 159, in run_shell
    output = subprocess.check_output(cmd, stderr=stderr)
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['bazel', '--batch', '--bazelrc=/dev/null', 'version']' returned non-zero exit status 2

[jowagner]

@jlturriff Did you try using JDK 8 (available in the openSUSE default repo) instead of 11? Also note "Illegal reflective access" is only a warning. Your output shows a proper error "Unrecognized option: --experimental_repo_remote_exec" that may have a different cause than what is discussed here. Searching for this string may help you find a solution. My guess is that your bazel is too old.

[jlturriff]

On 2020-02-27 09:01:31 Joachim Wagner wrote:

@jlturriff Did you try using JDK 8 (available in the openSUSE default repo)
instead of 11? Also note "Illegal reflective access" is only a warning.
Your output shows a proper error "Unrecognized option:
--experimental_repo_remote_exec" that may have a different cause than what
is discussed here. Searching for this string may help you find a solution.
My guess is that your bazel is too old.

As you see here, when I downloaded bazel via zypper it pulled in JDK 8 as

well, and the errors came up without my making any modifications to the
downloaded packages. If one or both of the packages are too old, it must be
due to the conservatism of OpenSuSE, which seldom provides near bleeding-edge
packages.

@06:12:31,root@pinto
/usr/local/tensorflow-master
█ zypper in bazel
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 5 NEW packages are going to be installed:
bazel java-1_8_0-openjdk java-1_8_0-openjdk-devel
java-1_8_0-openjdk-headless lksctp-tools

5 new packages to install.
Overall download size: 176.4 MiB. Already cached: 0 B. After the operation,
additional 219.9 MiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Retrieving package lksctp-tools-1.0.16-lp151.2.3.x86_64
(1/5), 325.5 KiB ( 1.8 MiB unpacked)
Retrieving:
lksctp-tools-1.0.16-lp151.2.3.x86_64.rpm .......................................................................................
[done (89.7 KiB/s)]
Retrieving package java-1_8_0-openjdk-headless-1.8.0.242-lp151.2.9.1.x86_64
(2/5), 49.4 MiB ( 84.7 MiB unpacked)
Retrieving:
java-1_8_0-openjdk-headless-1.8.0.242-lp151.2.9.1.x86_64.rpm ..................................................................
[done (652.4 KiB/s)]
Retrieving package java-1_8_0-openjdk-1.8.0.242-lp151.2.9.1.x86_64
(3/5), 475.2 KiB (569.4 KiB unpacked)
Retrieving:
java-1_8_0-openjdk-1.8.0.242-lp151.2.9.1.x86_64.rpm ...........................................................................
[done (162.4 KiB/s)]
Retrieving package java-1_8_0-openjdk-devel-1.8.0.242-lp151.2.9.1.x86_64
(4/5), 17.1 MiB ( 21.4 MiB unpacked)
Retrieving:
java-1_8_0-openjdk-devel-1.8.0.242-lp151.2.9.1.x86_64.rpm .....................................................................
[done (547.8 KiB/s)]
Retrieving package bazel-0.22.0-lp151.2.1.x86_64
(5/5), 109.2 MiB (111.5 MiB unpacked)
Retrieving:
bazel-0.22.0-lp151.2.1.x86_64.rpm .............................................................................................
[done (722.4 KiB/s)]

Checking for file
conflicts: ............................................................................................................................
[done]
(1/5) Installing:
lksctp-tools-1.0.16-lp151.2.3.x86_64 ..................................................................................................
[done]
(2/5) Installing:
java-1_8_0-openjdk-headless-1.8.0.242-lp151.2.9.1.x86_64 ..............................................................................
[done]
Additional rpm output:
update-alternatives: using /usr/lib64/jvm/jre-1.8.0-openjdk to
provide /usr/lib64/jvm/jre-1.8.0 (jre_1.8.0) in auto mode

(3/5) Installing:
java-1_8_0-openjdk-1.8.0.242-lp151.2.9.1.x86_64 .......................................................................................
[done]
(4/5) Installing:
java-1_8_0-openjdk-devel-1.8.0.242-lp151.2.9.1.x86_64 .................................................................................
[done]
Additional rpm output:
update-alternatives: using /usr/lib64/jvm/java-1.8.0-openjdk/bin/javac to
provide /usr/bin/javac (javac) in auto mode
update-alternatives: using /usr/lib64/jvm/java-1.8.0-openjdk to
provide /usr/lib64/jvm/java-openjdk (java_sdk_openjdk) in auto mode
update-alternatives: using /usr/lib64/jvm/java-1.8.0-openjdk to
provide /usr/lib64/jvm/java-1.8.0 (java_sdk_1.8.0) in auto mode

(5/5) Installing:
bazel-0.22.0-lp151.2.1.x86_64 .........................................................................................................
[done]
Executing %posttrans
scripts ............................................................................................................................
[done]
@06:17:31,root@pinto
/usr/local/tensorflow-master
█ ./configure
Extracting Bazel installation...
WARNING: detected http_proxy set in env, setting no_proxy for localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil
(file:/root/.cache/bazel/_bazel_root/install/73e92e4d21505d8fa81af652d5c23b82/_embedded_binaries/A-server.jar)
to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of
com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=159
INFO: Reading rc options for 'version'
from /usr/local/tensorflow-master/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
ERROR: Unrecognized option: --experimental_repo_remote_exec
WARNING: --batch mode is deprecated. Please instead explicitly shut down your
Bazel server using the command "bazel shutdown".
INFO: Invocation ID: d085ef2a-b0f8-409d-a59e-19f918a71993
Traceback (most recent call last):
File "./configure.py", line 1548, in
main()
File "./configure.py", line 1368, in main
_TF_MAX_BAZEL_VERSION)
File "./configure.py", line 483, in check_bazel_version
['bazel', '--batch', '--bazelrc=/dev/null', 'version'])
File "./configure.py", line 159, in run_shell
output = subprocess.check_output(cmd, stderr=stderr)
File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError:
Command '['bazel', '--batch', '--bazelrc=/dev/null', 'version']' returned
non-zero exit status 2

[Leslie-Fang]

I have used bazel-1.2.1 and get the same error:

ERROR: Unrecognized option: --experimental_repo_remote_exec

[jowagner]

@jlturriff and @Leslie-Fang Please open an issue "Build process requires too recent build tools" on https://github.com/tensorflow/tensorflow . The tensorflow developers are likely to close it quickly with "wontfix" but it is good to move the discussion from here to a dedicated space. Please cross-link here, e.g. by referring to this conversation in your bug report. (Github automatically creates a back-link.)

[jlturriff]

Done, except for the cross-link. (How do I do that?)