Bazel: incompatible_remove_native_maven_jar

[dkelmer]

The native maven_jar rule will no longer be available with this flag set to true.

---

(Update: managed by @jin as of July 12)

A migration tool will be provided to automatically convert maven_jar targets to rules_jvm_external's maven_install rule, and modify BUILD files to use the new labels.

See https://docs.google.com/document/d/1CqxPv-TwvFWBHgtg7_QhVfck4vH4rnmCU_IuMdC7O2M/edit# for a design document of this migration tool.

---

Proposal by @dkelmer as of Nov 28, which is another valid migration solution that is done manually:

Use

load("@bazel_tools//tools/build_defs/repo:java.bzl", "java_import_external")

or the convenience wrapper

load("@bazel_tools//tools/build_defs/repo:jvm.bzl", "jvm_maven_import_external")

instead.

Given a WORKSPACE file that looks like the following:

maven_jar(
    name = "truth",
    artifact = "com.google.truth:truth:0.30",
    sha1 = "9d591b5a66eda81f0b88cf1c748ab8853d99b18b",
)

It will need to look like this after updating:

load("@bazel_tools//tools/build_defs/repo:jvm.bzl", "jvm_maven_import_external")
jvm_maven_import_external(
    name = "truth",
    artifact = "com.google.truth:truth:0.30",
    artifact_sha256 = "59721f0805e223d84b90677887d9ff567dc534d7c502ca903c0c2b17f05c116a",
    server_urls = ["http://central.maven.org/maven2"],
    licenses = ["notice"],  # Apache 2.0
)

Notably

• the licenses attribute is mandatory
• sha1 is no longer supported, only sha256 is
• the server_urls attribute is mandatory. If your maven_jar rule did not specify a url
  then you should use the default server ("http://central.maven.org/maven2"). If your rule
  did specify a url then keep using that one.

[meteorcloudy]

This flag will break many projects in downstream, you can see the details at
https://buildkite.com/bazel/bazel-at-release-plus-incompatible-flags/builds/48

Please help them migrate (by filing issues and sending PRs and pinging the owners), thanks!

[meisterT]

cc @vladmos this is a classical example where buildifier could help with fixing the projects, because it's a mostly mechanical change

Danna: it would be great if we could prioritize this migration, because the maven rules are one of the two "users" of the java.desktop module which contributes to 1/3 of the embedded JDK size.

[dkelmer]

merged: https://github.com/bazelbuild/rules_gwt/pull/20
merged: https://github.com/bazelbuild/rules_appengine/pull/96
merged: https://critique.corp.google.com/#review/229611659&tab=a (intellij, SOT is internal)
merged: https://github.com/grpc/grpc-java/pull/5327
merged: https://github.com/googlesamples/android-testing/pull/245
filed issue: https://github.com/bazelbuild/BUILD_file_generator/issues/48 (they use bazel-deps and updating requires updating their custom scripts so I'll leave that to them)

rules_k8s is breaking because of grpc, so marking it as fixed

[meteorcloudy]

Since this flag is still not flipped in 0.22, 0.22 should still be its migration window.

[guw]

Going through the examples above, how would one specify a Maven server which requires authentication?

The native.maven_rule has a server attribute which takes a name of a server. Authentication credentials for such a server would be specified in ~/.m2/settings.xml. The location of the settings file was configurable with native.maven_server rule.

[dkelmer]

Thanks for bringing this up, the native maven_jar rule will not be deprecated before there is support for authentication with a local maven server

[dapengzhang0]

@dkelmer How did you get the sha256 values? As the default server ("http://central.maven.org/maven2") only provides sh1. Is there an automated tool that can get sha256 values for a lot of dependent maven jars?

[dkelmer]

@dapengzhang0 There is not an automated tool. You need to independently fetch the jars and calculate the sha256. One thing you can do is omit the artifact_sha256 attribute and run bazel fetch. Bazel will then tell you what sha it expected for each jar.

[keith]

Similar to https://github.com/bazelbuild/bazel/issues/3414#issuecomment-475561244 I believe the authentication features are all fixed by moving to rules_jvm_external

[guw]

@keith I disagree. The authentication features of rules_jvm_external do not support proper basic authentication. They only allow using the URL scheme which isn't supported by all Maven repository servers.

https://github.com/bazelbuild/rules_jvm_external/issues/80

[meisterT]

Note that with my recent commits I worked around being blocked on this with the removal of java.desktop, so I'm removing it from that milestone.

[laurentlb]

What's the status of this? Is anything blocking the flag flip?

[dslomov]

@jin authentication is an issue for full migration here. Do we have a plan for addressing this? (I believe all the needed APIs for basic authentication exist already)

[jin]

The migration path from settings.xml to the authentication APIs doesn't exist yet. I also consider these two issues for rules_jvm_external to be blockers:

https://github.com/bazelbuild/rules_jvm_external/issues/80
https://github.com/bazelbuild/rules_jvm_external/issues/186

[JLLeitschuh]

As a note. This issue should be considered blocking this security report from being closed:

https://issuetracker.google.com/issues/130163464

Should this issue adopt the P2 label of the original security vulnerability report #8880?

Cheers!

[jin]

Prototype script for migrating from maven_jar to rules_jvm_external: https://github.com/bazelbuild/rules_jvm_external/issues/216

I've managed to automate migration for the Copybara project from maven_jar to rules_jvm_external using the above script.

[jin]

This incompatible change is not happening for Bazel 1.0. The ecosystem is still heavily dependent on maven_jar, with many having built infrastructure on top of the rule, like Gerrit.

The way forward for the near future is to continue recommending usage of resolver tools like rules_jvm_external, and discourage usage of maven_jar.

[JLLeitschuh]

@jin can you warn users in the logs about their insecure usage.

In Gradle, we will be shipping a release in 6.0 that will begin to warn users about formally deprecating the use of HTTP to resolve dependencies and plan to require explicit opt in in Gradle 7.0.

[jin]

@JLLeitschuh see https://github.com/bazelbuild/bazel/pull/9235 to disallow plain HTTP URLs with maven_server and maven_jar without checksums.

[jin]

Coming in #9237, maven_jar will compute the sha256 checksum for you, if the new sha256 checksum is not specified. This should help with migration to jvm_maven_import_external or similar rules..

[JLLeitschuh]

@jin will Google be the CNA for the CVE issued for this vulnerability or should I begin the issuance of the CVE number via MITRE?

[jin]

@JLLeitschuh I do not have the answer for you at the moment. I am following up with folks from the security team with your question on issuing a CVE, the next steps, and also to verify our remediation approaches (#9237, #9235, https://github.com/bazelbuild/bazel/commit/b065b1318641db2e75b12ca6a29c89d9265f3389, original sha1 issue ref: https://github.com/bazelbuild/bazel/issues/8880)

[jin]

I have verified with the security team that the referenced remediation approaches for the SHA-1 issue are sufficient.

1) The docs for the maven_jar rule have been updated to discourage use of sha1, and use sha256 instead. https://docs.bazel.build/versions/master/be/workspace.html#maven_jar

"A SHA-1 hash of the desired jar.

If the downloaded jar does not match this hash, Bazel will error out. It is a security risk to download a file without verifying cryptographic secure hash. Note that SHA-1 is no longer considered a secure cryptographic hash function, but specifying the hash is still marginally better than no check at all. This attribute is kept here for legacy support purposes. Please either use the 'sha256' attribute or migrate to rules_jvm_external and pin your Maven artifacts with their SHA-256 checksums."

2) As in 1), we added sha256 attributes to the maven_jar rule in the next Bazel release (1.0). We are also actively recommending usage of sha256 if we detect if no checksum or sha1 is used. https://github.com/bazelbuild/bazel/pull/9237

Sample warning message: "WARNING: /usr/local/google/home/jingwen/code/copybara/WORKSPACE:192:1: maven_jar rule @junit//jar: Not using a checksum to verify the integrity of the artifact or the usage of SHA-1 is not secure (see https://shattered.io) and can result in an non-reproducible build. Please specify the SHA-256 checksum with: sha256 = "90a8e1603eeca48e7e879f3afbc9560715322985f39a274f6f6070b43f9d06fe","

@JLLeitschuh Google is a CNA only for Chrome/Chromium issues.

[JLLeitschuh]

+1 on the mention of shattered in your warning message.

Do you want me to move forward with the CVE issuance process via MITRE or will you?

If you want me to do it, I need to know the first 'fixed' version as well as the commit hash/hashes or PR that fixes this issue.

Some context here: I'm actually not a Bazel user, I do work for Gradle though & in my free time I focus my software security research on build infrastructure and the Java supply chain.

[JLLeitschuh]

Do you want me to move forward with the CVE issuance process via MITRE or will you?

Ping: @jin

[jin]

Here are the commits that Google Security have verified to be sufficient as remediation:

https://github.com/bazelbuild/bazel/commit/bc9ff0c5f1dc1c592fd8e604e81de0357c78e47a
https://github.com/bazelbuild/bazel/commit/06d79dd21801caca08bda3c281c7bba04dbcdbb7
https://github.com/bazelbuild/bazel/commit/b065b1318641db2e75b12ca6a29c89d9265f3389

Please go ahead with the issuance.

[JLLeitschuh]

@jin Can you confirm the first fixed version of Bazel for the CVE disclosure?

[jin]

That would be 1.0.0rc1. https://releases.bazel.build/1.0.0/rc1/index.html

[JLLeitschuh]

Hey @jin unless Google plans to make an official disclosure for this, the CVE to use is the generic one for SHA-1 https://nvd.nist.gov/vuln/detail/CVE-2005-4900

[jin]

Please see this document for a design proposal for a migration tool from maven_jar to rules_jvm_external.

[JLLeitschuh]

Thanks! Cool to see progress on this!

[jin]

The automated migration tool is available here: https://github.com/bazelbuild/rules_jvm_external/tree/master/migration

If you don't wish to migrate to rules_jvm_external, please follow the alternative instructions to manually migrate to jvm_maven_import_external.