Bazel: Bazel release 0.17.2

[laurentlb]

This issue is for tracking a patch release on top of 0.17.1 (https://github.com/bazelbuild/bazel/issues/5059).

Release policy: https://bazel.build/support.html#policy

we do not cherry-pick new features into it. Moreover, if we discover that a new feature is buggy, we might decide to roll it back from a release candidate. Only critical, high-impact bugs will be fixed in a release candidate.

[laurentlb]

Request 1 by @jin is https://github.com/bazelbuild/bazel/commit/2e677fb6b8f309b63558eb13294630a91ee0cd33
Is there a tracking bug about it? How is it critical?

Request 2 by @steeve is https://github.com/bazelbuild/bazel/issues/6151
This seems to be a regression, but it is not breaking users. @cushon, can you comment on the impact on user-experience?

[jin]

Request 1 by @jin is 2e677fb
Is there a tracking bug about it? How is it critical?

Brief summary: 0.17 introduces a regression where Android apps can't be built at the desugaring step when used in conjunction with a manual --host_javabase flag pointing to a JDK 8 installation.

If an Android target can't be built using a user-accessible manner that was previously possible, I consider this a critical regression.

2e677fb fixes this regression.

Tracking bug: #6159

[cushon]

Request 2 by @steeve is #6151
This seems to be a regression, but it is not breaking users. @cushon, can you comment on the impact on user-experience?

I don't think #6151 is a regression since 0.16x, since that release also used an embedded JDK 9 which would have generated the same warnings. The issue results in some warning spam in the build log, which is annoying but ultimately harmless.

[jin]

@laurentlb FYI, looks like the fix for this issue with 0.17.0rc1 didn't make it into 0.17.1: https://github.com/bazelbuild/bazel/issues/5974

[aehlig]

@laurentlb FYI, looks like the fix for this issue with 0.17.0rc1 didn't make it into 0.17.1: #5974

...which is technically correct according to our policy. .bazelignore is a new feature, strictly opt-in, as so far no one had a reason to have a file called .bazelignore in their work space. And that new feature happened to be broken and will be useful only in 0.18; but it is not technically a regression, as it didn't work in any earlier release either. Unfortunate, but following our policies.

[laurentlb]

Correct. The goal of patch releases is to make sure that users can upgrade from 0.16 to 0.17. So if a change in 0.17 blocks you from upgrading, we can cherrypick the fix. In other cases, you'll have to wait for the next release (but it's not too bad, I think: we release once a month on average).

I'm going to create a candidate with one cherrypick: https://github.com/bazelbuild/bazel/commit/2e677fb6b8f309b63558eb13294630a91ee0cd33

[cushon]

@laurentlb consider including 1d956c707e1c843896ac58a341c335c9c149073d for #6180, which is a regression in 0.17.1. (cc @mhlopko @iirina)

[laurentlb]

0.17.2 rc1 is available here: https://releases.bazel.build/0.17.2/rc1/index.html

scripts/release/release.sh create 0.17.2 7bb9e0fe91fc40d0b4537ecc805313fb0bc88a76 1d956c707e1c843896ac58a341c335c9c149073d 2e677fb6b8f309b63558eb13294630a91ee0cd33

[steeve]

This version outputs these warnings when installing, FYI:

Uncompressing......Extracting Bazel installation...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/private/var/tmp/_bazel_steeve/install/2269f5a83016b62a56b593c692184ca4/_embedded_binaries/A-server.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".

[steeve]

This version fails with:

ERROR: /Users/steeve/go/src/github.com/steeve/project/BUILD.bazel:79:1: Sharding classes for dexing for //:myandroidapp failed (Exit 1): mapper failed: error executing command
  (cd /private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/execroot/__main__ && \
  exec env - \
    PATH=/bin:/usr/bin \
  bazel-out/host/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ziputils/mapper --output_jar bazel-out/x86-fastbuild/bin/android/_dx/zenlycore_split/classes.jar --output_resources bazel-out/x86-fastbuild/bin/myandroidapp_files/java_resources.jar --split_dexed_classes --input_jar bazel-out/x86-fastbuild/bin/myandroidapp_resources.jar.dex.zip)
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
    at com.google.devtools.build.android.ziputils.ZipIn.loadEndOfCentralDirectory(ZipIn.java:491)
    at com.google.devtools.build.android.ziputils.ZipIn.endOfCentralDirectory(ZipIn.java:96)
    at com.google.devtools.build.android.ziputils.SplitZip.run(SplitZip.java:294)
    at com.google.devtools.build.android.ziputils.DexMapper.main(DexMapper.java:70)

in .bazel.rc:

build                   --javabase=@local_jdk//:jdk
build                   --host_javabase=@local_jdk//:jdk
build                   --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build                   --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8

0.16.1 works

[cushon]

@steeve I tried building //examples/android/... with 0.17.2 rc1 and JDK 8 and wasn't able to observe that problem, can you share the repro?

[steeve]

I won't have my computer until the morning but until I do you can try https://github.com/steeve/bazel_issue_5978

[steeve]

With --config jdk8

[jin]

I'm getting a different error with 0.17.2rc1 with https://github.com/steeve/bazel_issue_5978 now:

$ bazel build //:sample --config=jdk8 --strategy=Javac=sandboxed
INFO: Analysed target //:sample (0 packages loaded).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_jingwen/8b561bac257f9f14a669861c2ced5081/external/androidsdk/BUILD.bazel:8:1: Building external/androidsdk/apksigner.jar () failed (Exit 1)
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
-Xbootclasspath/p is no longer a supported option.
Target //:sample failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.470s, Critical Path: 0.23s
INFO: 1 process: 1 darwin-sandbox.
FAILED: Build did NOT complete successfully

[cushon]

@jin what's your local JDK version / JAVA_HOME? The -Xbootclasspath/p error looks like you have a local JDK 9, which isn't going to work with --config=jdk8.

I can build https://github.com/steeve/bazel_issue_5978 with all of the following configurations:

JAVA_HOME=$JAVA8_HOME bazel-0.17.2rc1-linux-x86_64 build --config=jdk8 //:sample
JAVA_HOME=$JAVA8_HOME bazel-0.17.2rc1-linux-x86_64 build //:sample
JAVA_HOME=$JAVA9_HOME bazel-0.17.2rc1-linux-x86_64 build //:sample

[jin]

That explains it. Sorry for the noise!

On Thu, Sep 20, 2018 at 2:20 PM Liam Miller-Cushon [e-mail]
wrote:

@jin https://github.com/jin what's your local JDK version / JAVA_HOME?
The -Xbootclasspath/p error looks like you have a local JDK 9, which
isn't going to work with --config=jdk8.

I can build https://github.com/steeve/bazel_issue_5978 with all of the
following configurations:

JAVA_HOME=$JAVA8_HOME bazel-0.17.2rc1-linux-x86_64 build --config=jdk8 //:sample
JAVA_HOME=$JAVA8_HOME bazel-0.17.2rc1-linux-x86_64 build //:sample
JAVA_HOME=$JAVA9_HOME bazel-0.17.2rc1-linux-x86_64 build //:sample

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/6164#issuecomment-423054071,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVPDkPv_x7dmKWC3OkSen2gjr_RbJV_ks5uczOQgaJpZM4WrqxA
.

[cushon]

I think I have an explanation for the NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; error reported in https://github.com/bazelbuild/bazel/issues/6164#issuecomment-422980111.

I was trying to reproduce on linux, and the version of com.google.devtools.build.android.ziputils.ZipIn in bazel-0.17.1-darwin-x86_64 doesn't match the linux binary:

wget https://releases.bazel.build/0.17.2/rc1/bazel-0.17.2rc1-{linux,darwin}-x86_64
JAR=embedded_tools/src/tools/android/java/com/google/devtools/build/android/all_android_tools_deploy.jar
CLASS=com.google.devtools.build.android.ziputils.ZipIn
unzip -o bazel-0.17.2rc1-darwin-x86_64 $JAR && javap -v -p -cp $JAR $CLASS | grep ByteBuffer.position
unzip -o bazel-0.17.2rc1-linux-x86_64 $JAR && javap -v -p -cp $JAR $CLASS | grep ByteBuffer.position

Archive:  bazel-0.17.2rc1-darwin-x86_64
  inflating: embedded_tools/src/tools/android/java/com/google/devtools/build/android/all_android_tools_deploy.jar  
  #124 = Methodref          #208.#382     // java/nio/ByteBuffer.position:(I)Ljava/nio/ByteBuffer;
       230: invokevirtual #124                // Method java/nio/ByteBuffer.position:(I)Ljava/nio/ByteBuffer;
Archive:  bazel-0.17.2rc1-linux-x86_64
  inflating: embedded_tools/src/tools/android/java/com/google/devtools/build/android/all_android_tools_deploy.jar  
  #124 = Methodref          #208.#382     // java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;
       230: invokevirtual #124                // Method java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;

The issue affecting the darwin binary looks identical to https://github.com/bazelbuild/bazel/issues/5978#issuecomment-415907412, which we fixed by adding a step to the release pipeline to build the release with itself: https://github.com/bazelbuild/bazel/issues/5978#issuecomment-416197158.

@philwo any chance that change was only made for the linux release pipeline and not for darwin?

[cushon]

any chance that change was only made for the linux release pipeline and not for darwin?

I think that's what happened. Diffing the linux [1] and darwin [2] release configs, I see:

--- darwin   2018-09-19 23:45:10.380907192 -0700
+++ linux   2018-09-19 23:45:15.880890494 -0700
@@ -3,5 +3,5 @@
 mkdir output
 cp bazel-bin/src/bazel output/bazel

-bazel build --define IPHONE_SDK=1 -c opt --stamp --embed_label \"${release_name}\" --workspace_status_command=scripts/ci/build_status_command.sh src/bazel scripts/packages/with-jdk/install.sh
+output/bazel build -c opt --stamp --embed_label \"${release_name}\" --workspace_status_command=scripts/ci/build_status_command.sh src/bazel scripts/packages/with-jdk/install.sh scripts/packages/debian/bazel-debian.deb scripts/packages/debian/bazel.dsc scripts/packages/debian/bazel.tar.gz bazel-distfile.zip

We should be using output/bazel on both for that step, not just for linux.

[1] https://buildkite.com/bazel/release/builds/105#6874f846-9cff-469b-9095-a692db9f3006
[2] https://buildkite.com/bazel/release/builds/105#dcf156e2-41d0-4538-922f-a898c68ab849

[philwo]

Arghh, sorry :( Indeed, I must have missed that when updating the pipeline. I'll make the fix right now.

[philwo]

Fixed. All builds in the "release pipeline" from now on will use the fixed configuration.

[laurentlb]

I'm a bit confused with the messages.
Is there anything to cherrypick on top of 0.17.2 rc1? Is there still a blocking issue?

[philwo]

AFAIU, we don't have to cherrypick anything in order to fix the error that @cushon mentioned above, as the problem was only in our CI pipeline configuration (which I fixed now).

The next time you or anyone else builds a release candidate or a final release, the release binary on macOS will be built in the correct way (the same as on Linux and Windows) and that should fix the error.

[cushon]

Thanks @philwo!

@laurentlb can you build another release candidate (with no additional cherry-picks) so the fix to the release pipeline can be verified before the final release?

[steeve]

Is it possible to make an 0.17.2rc2 then? Or overwrite the 0.17.2rc1 with the new release pipeline.

[laurentlb]

Done.
https://releases.bazel.build/0.17.2/rc2/index.html

[cushon]

Thanks, looks good to me:

Archive:  bazel-0.17.2rc2-darwin-x86_64
  inflating: embedded_tools/src/tools/android/java/com/google/devtools/build/android/all_android_tools_deploy.jar  
  #124 = Methodref          #208.#382     // java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;
       230: invokevirtual #124                // Method java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;
Archive:  bazel-0.17.2rc2-linux-x86_64
  inflating: embedded_tools/src/tools/android/java/com/google/devtools/build/android/all_android_tools_deploy.jar  
  #124 = Methodref          #208.#382     // java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;
       230: invokevirtual #124                // Method java/nio/ByteBuffer.position:(I)Ljava/nio/Buffer;

@steeve can you verify that fixes the problem you were seeing?

[steeve]

I can confirm that rc2 fixes the jdk8 issue!

[laurentlb]

Bazel 17.2 released: https://github.com/bazelbuild/bazel/releases/tag/0.17.2

(cc @vbatts @petemounce)

[vbatts]

https://copr.fedorainfracloud.org/coprs/vbatts/bazel/build/801272/
I tried it locally, and it compiled fine for centos. Hopefully #6150 is
not an issue any longer.

[petemounce]

Pushed to chocolatey

[dimsuz]

Hmm, I still see this with bazel-0.20.0 on Arch Linux. I'm trying to follow an official Android Tutorial.

The command bazel build //src/main:app gives me a 'Can't start JVM' error, so I figured that's because I use java 8 (because Android):

$ javac -version
javac 1.8.0_192

So I try to tell bazel about this:

bazel build --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 //src/main:app

And get an error in response:

INFO: Invocation ID: 6850ce2f-2b64-4627-8377-966538caba8d
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //src/main:app (0 packages loaded, 957 targets configured).
INFO: Found 1 target...
ERROR: /home/dima/tmp/bazel-examples/android/tutorial/src/main/BUILD:1:1: Sharding classes for dexing for //src/main:app failed (Exit 1) mapper failed: error executing command bazel-out/host/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ziputils/mapper --output_jar bazel-out/k8-fastbuild/bin/src/main/_dx/app/classes.jar --output_resources ... (remaining 6 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
    at com.google.devtools.build.android.ziputils.ZipIn.loadEndOfCentralDirectory(ZipIn.java:491)
    at com.google.devtools.build.android.ziputils.ZipIn.endOfCentralDirectory(ZipIn.java:96)
    at com.google.devtools.build.android.ziputils.SplitZip.run(SplitZip.java:294)
    at com.google.devtools.build.android.ziputils.DexMapper.main(DexMapper.java:70)
Target //src/main:app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.184s, Critical Path: 4.01s
INFO: 23 processes: 13 processwrapper-sandbox, 10 worker.
FAILED: Build did NOT complete successfully

Any hints on how to fix this? (Seeing how all relevant issues are closed I suspect I might be doing something wrong)

[jin]

@dimsuz could you please file a new issue and tag me in it? This issue is for the 0.17.2 release.