Bazel: bootstrapping 0.14.0rc1 requires network access

[aehlig]

As opposed to earlier versions of bazel, bootstrapping bazel with bash compile.sh from the distribution artifact (as described in https://docs.bazel.build/versions/master/install-compile-source.html) requires network access. The reason is that //tools/android:embedded_tools pulls in @desugar_jdk_libs which is an http_archive.

Depending on our policy this may or may not block #5056, i.e. the 0.14.0 release.

Possible solutions include

• adding the lib to the distribution archive
• not embedding the lib in android tools

[jin]

+@kevin1e100

[jin]

https://github.com/google/desugar_jdk_libs is 5.7MB with git clone depth=1, which is about 3% of the size of the distribution archive.

[laszlocsomor]

Possible solutions include

• adding the lib to the distribution archive
• not embedding the lib in android tools

For the second option: requiring the user adds a http_archive rule to their WORKSPACE file feels like the right approach, given how they also have to uncomment (or add) android_sdk_repository and android_ndk_repository rules.

[kevin1e100]

+@ahumesky

IIUC, having users add the http_archive rule will require users to have network access when building (any?) Android artifacts, is that realistic? Could you please point us to any background why needing network access for building Bazel is problematic?

[aehlig]

Could you please point us to any background why needing network access for building Bazel is problematic?

When building open-source software it is standard practice, especially when building binary packages, to first fetch the needed files and compare checksums, and then do the actual build in an offline fashion. Reasons for this include (among other reasons)

• being sure to have all sources, so that rebuilding at a later point in time is possible, even if files disappear from the web,
• easier audit for known vulnerabilities, if all source code is explicit, and
• compliance with licenses that require redisitribution of source files.

That is also how I discovered that network dependency; I was doing a test build for updating the bazel port for FreeBSD. Of course, for that I can work around it by listing the desugar-jdk library as another source archive needed, but it is still considered bad upstream packaging if a build does an unsolicited call out to the network.

[A related feature request is #5175, so @vrothberg might also be interested in this discussion.]

[ahumesky]

If I recall correctly, we based this decision on there already being 3 other http_archive rules in bazel's workspace file. I guess those aren't actually used when building bazel?

We could move this dependency around so that the user has to add it to their workspace file (that is, we'd grab the dependency when the user builds their android target, instead of grabbing the dependency when we build bazel), but that's not a great user experience I think. It also means you couldn't do an offline android build without understanding where the dependency is coming from and knowing how to override it in your workspace file.

The simplest solution is just to add the library to bazel. It's a size cost, but one that we're already sort of paying by embedding the library into bazel (now we just make it part of the codebase too).

And this will all go away once we've moved the android rules to skylark and to a separate repository.

[lfpino]

Do we know which commit introduced the problem?

[aehlig]

Do we know which commit introduced the problem?

I would guess bfd89d6393fd56c92c0b31b19d7ec78d8da9141f. But a quick forward fix like https://bazel-review.googlesource.com/c/bazel/+/56490 is probably less pain than a rollback.

[ahumesky]

I have a change in progress that puts the external dependency into bazel's repo under third_party. I'll send it out tomorrow.