Bazel: query, performance: running the same query twice should be fast

[laszlocsomor]

On Windows 10, with Bazel 0.8.1, I run the following query:

bazel --output_user_root=c:\tmp2 query "tests(//...)-tests(//src:all_windows_tests)"

then immediately I run it again without anything changing. The second time the query seems to be just as slow as the first time, even though all packages should already be loaded.

[laszlocsomor]

Apparently this also happens on Linux. Adjusting the bug description.

  $ time bazel query "tests(//...)-tests(//src:all_windows_tests)"
(...)
real    0m49.732s
user    0m0.020s
sys     0m0.020s

  $ time bazel query "tests(//...)-tests(//src:all_windows_tests)"
(...)
real    0m44.593s
user    0m0.008s
sys     0m0.028s

[laszlocsomor]

For the record I didn't profile what exactly takes 45 seconds in this query, whether it's loading the packages or calculating the query results or something else.

[laszlocsomor]

/cc @haxorz @ericfelly @michajlo @janakdr

[janakdr]

This is a pretty bananas regression. Internal Blaze finishes the ~equivalent query in 0.2 seconds. Looking...

[janakdr]

Looks like all the work is being done in re-fetching bazel_j2objc repository, happening unconditionally on every query. It's an http_archive. I'm going to have to hand this off to someone who knows about repositories.

[janakdr]

That rule is silently failing with:
Error downloading [https://github.com/google/j2objc/releases/download/2.0.3/j2objc-2.0.3.zip] to .../janakr/.cache/bazel/_bazel_janakr/6264bddbcc0465383fcc9614dbc9d23a/external/bazel_j2objc/j2objc-2.0.3.zip: Checksum was a36bac432d0dbd8c98249e484b2b69dd5720afa4abb58711a3c3def1c0bfa21d but wanted 529ee99e6f0e3f88edef61aeae4f13dc6e5eb8183993ced191338422b0e1fbeb

I don't know (a) why the checksum is wrong, and (b) why this error isn't being reported anywhere.

[laszlocsomor]

@dslomov owns repositories now.

[meisterT]

This seems fixed now:

$ time bazel query "tests(//...)-tests(//src:all_windows_tests)" | wc -l
Loading: 0 packages loaded
WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/3.1.0.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
DEBUG: /usr/local/google/home/twerth/.cache/bazel/_bazel_twerth/f01bc937da326f5bb0feb15c854c110c/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:9: rbe_ubuntu1804_java11 not using checked in configs as detect_java_home was set to True 
DEBUG: /usr/local/google/home/twerth/.cache/bazel/_bazel_twerth/f01bc937da326f5bb0feb15c854c110c/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:9: rbe_ubuntu1604_java8 not using checked in configs as detect_java_home was set to True 
Loading: 0 packages loaded
Loading: 0 packages loaded
415

real    0m0.304s
user    0m0.048s
sys 0m0.052s