Bazel: shallow_since appears to be broken

[mehrdadn]

Description of the problem / feature request:

shallow_since appears to be broken.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Try to build //:BUILD.boost from https://github.com/nelhage/rules_boost/commit/df908358c605a7d5b8bbacde07afbaede5ac12cf. You'll receive a recommendation for shallow_since. Insert that recommendation, and suddenly Bazel fails to find that commit at all.

What operating system are you running Bazel on?

Windows

What's the output of bazel info release?

release 1.1.0

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

Create an empty BUILD.bazel alongside the following WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(name="rules_boost", commit="df908358c605a7d5b8bbacde07afbaede5ac12cf", remote="https://github.com/nelhage/rules_boost")

Now run bazel --ignore_all_rc_files query "deps(@rules_boost//:BUILD.boost)":

bazel --ignore_all_rc_files query "deps(@rules_boost//:BUILD.boost)"
@bazel --ignore_all_rc_files clean --expunge

It outputs something like the following (after formatting/redactions):

INFO: Writing tracer profile to 'command.profile.gz'
DEBUG: Rule 'rules_boost' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1569511515 +0200"
DEBUG: Call stack for the definition of repository 'rules_boost' which is a git_repository (rule definition at external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
 - WORKSPACE:2:1
ERROR: no such target '@rules_boost//:BUILD.boost': target 'BUILD.boost' not declared in package ''; however, a source file of this name exists.  (Perhaps add 'exports_files(["BUILD.boost"])' to /BUILD?) defined by external/rules_boost/BUILD
Loading: 1 packages loaded

Now add shallow_since="1569511515 +0200" as recommended and re-run the query:

INFO: Writing tracer profile to 'command.profile.gz'
INFO: Call stack for the definition of repository 'rules_boost' which is a git_repository (rule definition at external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
 - WORKSPACE:2:1
ERROR: An error occurred during the fetch of repository 'rules_boost':
error running 'git reset --hard df908358c605a7d5b8bbacde07afbaede5ac12cf' while working with @rules_boost:
fatal: Could not parse object 'df908358c605a7d5b8bbacde07afbaede5ac12cf'.
ERROR: no such package '@rules_boost//'
Loading: 0 packages loaded

Suddenly Bazel seems to be unable to find the commit altogether.

[thii]

We got the same issue on macOS, but it seems to be a problem with git (tried with git clone --shallow-since).

[mehrdadn]

Yeah, seems like it. Though I'd say Bazel is still responsible for avoiding recommending shallow_since unless/until this is fixed (a version check on git might make sense). Otherwise it's really confusing for the user to be recommended an option that's broken.

[laurentlb]

@aehlig Do you know what causes it?

[nelhage]

I did some digging here, and reported an upstream bug to git

The upshot is that --shallow-since is subtly incorrect when the provided timestamp would cause us to include _some_ parents of a merge, but not others. In the case provided here, nelhage/rules_boost@df90835 has a commit timestamp of 1569511515. That commit is the second part of a merge, nelhage/rules_boost@76f3064; that merge's first parent is nelhage/rules_boost@a83197e, with an older commit timestamp of 1569487118

The bug I reported results in git discarding _all_ parents of the merge (nelhage/rules_boost@76f3064), including the desired commit.

For now, the workaround for bazel _users_ -- where available -- is to switch your commit= line to point to a commit on the first-parent spine, e.g. nelhage/rules_boost@76f3064. If you add the suggested shallow_since line for that commit, things work fine.

For Bazel itself, it might be nice to recognize this situation and adjust the shallow_since recommendation, at least until the upstream bug can be sorted out. I'm not certain offhand what the right/easiest heuristic to detect it is, though.