Bazel: Error when http_archive has wrong sha256 now reports prefix not found in the zip

[alexeagle]

Maybe a caching bug?

I changed my WORKSPACE to update the version in my http_archive

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "e9bc013417272b17f302dc169ad597f05561bb277451f010043f4da493417607",
    strip_prefix = "rules_nodejs-0.7.0",
    url = "https://github.com/bazelbuild/rules_nodejs/archive/0.7.0.zip",
)

now I get

ERROR: error loading package '': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_nodejs//': Prefix rules_nodejs-0.7.0 was given, but not found in the zip

I tried releasing a new tag just to get around this, but it still fails.

Sorry, at ng-conf, don't have time to make a minimal repro right now. Possible this is not reproducible, will try to follow up later.

[laszlocsomor]

Please provide some repro, the repo you were working in, and the Bazel version.

[alexeagle]

I will try to repro (have seen this twice now) - one hint (maybe) is that the sha256 was also wrong. Next time I see this happen I'll try removing the sha256 attribute and see if that makes any difference.

[alexeagle]

Okay, now I understand it a bit better.

http_archive(
    name = "build_bazel_rules_nodejs",
    url = "https://github.com/bazelbuild/rules_nodejs/archive/0.7.0.zip",
    strip_prefix = "rules_nodejs-0.7.0",
    sha256 = "7406bea7954e1c906f075115dfa176551a881119f6820b126ea1eacb09f34a1a",
)

-> ERROR: error loading package '': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_nodejs//': Prefix rules_nodejs-0.7.0 was given, but not found in the zip

I expected the error message to tell me the sha256 is wrong, and what the actual one is, so I can update myself. That's what used to happen.

If I put the correct sha256sum, then it works.

[alexeagle]

and it's reproducible on linux too, so it's not platform-specific, sorry for the false lead there

[aehlig]

Looking into the issue, I'm pretty sure it is the same cache issue/user error (depending
on your point of view) as #5144. A minimal example that explains what is going on is the
following (on a bazel with empty repository cache).

  WRKDIR=$(mktemp -d "${TEST_TMPDIR}/testXXXXXX")
  cd "${WRKDIR}"

  mkdir ext_repo
  touch ext_repo/WORKSPACE
  touch ext_repo/BUILD
  tar cvf ext.tar ext_repo
  rm -rf ext_repo

  touch empty

  mkdir main
  cd main
  cat > WORKSPACE <<EOF
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")

http_archive(
  name = "ext",
  url = "file://${WRKDIR}/ext.tar",
  strip_prefix = "ext_repo",
  # actually the sha1 sum of the empty string
  sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
)

http_file(
  name = "empty",
  urls = ["file://${WRKDIR}/empty"],
  sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
)
EOF

Then, the first bazel build @ext//... fails with the expected error message
Checksum was ... but wanted .... Then doing a bazel build @empty/... succeeds
(as expected) and has the side effect that bazel now knows how to obtain a file
with SHA256 sum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
So, the next bazel build @ext//... will not try to download the file (we have
it in cache anyway!) but instead try to unpack it---and fail with Prefix ext_repo was given, but not found in the archive which is the correct description for
the archive with the given SHA256 sum.

As mentioned in #5144, in my opinion, the long-term solution would be to make
progress on the WORKSPACE.resolved proposal. Once hashes are no longer added
by hand, the problem of manually written WORKSPACES with hashes referring to
other files (e.g., earlier versions) will go away. I am not sure, we need a
quick interim solution.

--
Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado

[alexeagle]

Sure, I'm getting used to downloading the file and running sha256sum myself, this is low severity.

[qzmfranklin]

I am getting this same issue after upgrading to bazel 0.16

I remembered that in a previous version (possibly 0.12), it will report mismatched sha256. I liked that behavior better.

[ittaiz]

I have a feeling there always be some need for hashes to be added by hand
even if solely for the use case of quickly iterating by myself
(WORKSPACE.resolved IIRC is now discussed with respect to sync which is
time consuming).
I think the more accurate error message is indeed better.
On Mon, 8 Oct 2018 at 16:51 Erick J [e-mail] wrote:

"I am not sure, we need a quick interim solution."

to echo @qzmfranklin https://github.com/qzmfranklin - reverting to the
more accurate error message of a mismatching shasum would be extremely
helpful

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/5045#issuecomment-427974860,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF42n7lp22KwFYge7nZf8zUOU9oDXks5ui7q5gaJpZM4TaeGz
.

[aehlig]

I think the more accurate error message is indeed better.

Sure, but as explained in #6089, I don't see a way of improving the error message without degrading the non-error case by unwanted fetches.

[ittaiz]

I agree
On Wed, 7 Nov 2018 at 10:59 Klaus Aehlig [e-mail] wrote:

I think the more accurate error message is indeed better.

Sure, but as explained in #6089
https://github.com/bazelbuild/bazel/issues/6089, I don't see a way of
improving the error message without degrading the non-error case by
unwanted fetches.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/5045#issuecomment-436551881,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF7NbTRnHJJKv0pI1ZnhJKh_LIwElks5usqD_gaJpZM4TaeGz
.

[jrajahalme]

Just hit this by changing the archive I'm depending on to a newer one, but failing to update the SHA256, so in the hindsight it is obvious that the newer archive was not found from the old zip file.

@aehlig Would it be too much to ask to have some more context in the error message, e.g., "Existing zip with SHA256 ... was used but the contents does not match with the expected, maybe use a different SHA256?"

[dslomov]

Let's attempt to redownload the file and check the checksum if the prefix is not found

[aehlig]

The current state (after aff8319e25) is that in that case (URL and prefix changed, but hash not updated), bazel will

• report that the prefix wasn't found and show the available prefixes,
• report which files where taken from cache based on which provided hashes, and
• will fail the build.

After offline discussion with @dslomov, we decided to leave it at this state for the moment, and not attempt to proactively redownload the file.