Hub: Hub sync exit status 128

[UnifyBadPirate]

git version 2.24.0
hub version 2.13.0

Trying to run hub sync always fails with: exit status 128

[mislav]

Can you please run HUB_VERBOSE=1 hub sync and see which is the last $ git line of debug output, i.e. which exact git command failed?

[UnifyBadPirate]

$ git rev-parse -q --git-dir
$ git remote -v
$ git fetch --prune --quiet --progress origin
$ git config --get-regexp branch.*.remote
$ git branch --list
$ git rev-parse --symbolic-full-name access-prod@{upstream}
$ git rev-parse -q refs/heads/access-prod refs/remotes/origin/access-prod
$ git rev-parse --symbolic-full-name access-staging@{upstream}
$ git rev-parse -q refs/heads/access-staging refs/remotes/origin/access-staging
$ git rev-parse --symbolic-full-name badpirate/SDK-1286-KeyboardPOC@{upstream}
$ git rev-parse -q refs/heads/badpirate/SDK-1286-KeyboardPOC refs/remotes/origin/badpirate/SDK-1286-KeyboardPOC
$ git rev-parse --symbolic-full-name badpirate/fastlane@{upstream}
$ git rev-parse -q refs/heads/badpirate/fastlane refs/remotes/origin/badpirate/fastlane
$ git rev-parse --symbolic-full-name badpirate/tasty-dev@{upstream}
$ git rev-parse -q refs/heads/badpirate/tasty-dev refs/remotes/origin/badpirate/tasty-dev
$ git rev-parse --symbolic-full-name badpirate/version-bump@{upstream}
$ git rev-parse -q refs/heads/badpirate/version-bump refs/remotes/origin/dmz

[UnifyBadPirate]

version-bump seemed to point at a non-existent branch in remote not sure how it happened, but force removing badpirate/version-bump allowed the hub sync to succeed. Going to leave open as likely the error could be recognized, or better logged.

[UnifyBadPirate]

Okay, this reproduces regularly and requires manual repair each time. (note I'm on a Gitlab server)

Repro

1. Locally create a branch, and push to origin
2. Merge that branch and check the box for "delete branch after merge"
3. Merge branch -> Deletes the remote branch reference but not local reference
4. hub sync -> status 128

In this case it seems the rev parse git rev-parse -q refs/heads/badpirate/version-bump refs/remotes/origin/dmz points the removed local branch back at the wrong branch (dmz)... this was a branch we had previously had selected as the root of our tree, but switched back to master in GitLab and dumped the old dmz branch after merging.

I'm going to remove my local repo and re-clone to see if the problem goes away, and I'll update this task.

[mislav]

Interesting, thank you for looking into this.

I'm guessing the relevant part of .git/config looked like:

[branch "badpirate/version-bump"]
        remote = origin
        merge = refs/heads/dmz

then hub did this:

git rev-parse --symbolic-full-name badpirate/version-bump@{upstream}
#=> "refs/remotes/origin/dmz"

When it got refs/remotes/origin/dmz as result, it had assumed that origin/dmz remote tracking branch _exists_, and proceeded to use it in the next rev-parse lookup, which failed because this tracking branch does not in fact exists, as it was deleted by GitLab on the remote and cleaned up by git fetch origin --prune which hub sync runs early in its invocation.

Now the question is, how come git rev-parse --symbolic-full-name badpirate/version-bump@{upstream} succeeded when the upstream tracking branch was deleted?

When you manage to reproduce again, please check git rev-parse --symbolic-full-name BRANCH@{upstream} and whenever it returns a result successfully, check that the resulting ref actually exists. If it doesn't, then please share with us your branch configuration (the [branch "BRANCH"] section of .git/config)

[mislav]

FWIW, I've tried with a GitLab repo and I couldn't reproduce.

[UnifyBadPirate]

Reproduced again - Case is as before, Gitlab merge with "delete branch after merge" checked deletes remote branch, local branch badpirate/SDK-1661-cognito-thread still exists:

HUB_VERBOSE=1 hub sync

$ git rev-parse -q --git-dir
...
$ git rev-parse -q refs/heads/badpirate/SDK-1661-cognito-thread refs/remotes/origin/dmz
exit status 128

The offending git command git rev-parse -q refs/heads/badpirate/SDK-1661-cognito-thread refs/remotes/origin/dmz:

93c00aa5a4af5c1e32739dd94659036ba501f4c8
refs/remotes/origin/dmz
fatal: ambiguous argument 'refs/remotes/origin/dmz': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

The related entry in .git/config:

[branch "badpirate/SDK-1661-cognito-thread"]
        remote = origin
        merge = refs/heads/badpirate/SDK-1661-cognito-thread

No mention of the old master branch dmz anywhere in .git/config or git/* for that matter --odd, as it's got to be "gitting" it from somewhere. Verified the default branch in gitlab is correctly set to master, and that git ls-remote | grep "dmz" returns no results...

I'm going to try deleting and re-cloning now (that I've got nothing locally stashed) and will add more details if it reproduces after the re-clone.

[mislav]

I would be really interested if you investigated if git rev-parse --symbolic-full-name badpirate/SDK-1661-cognito-thread@{upstream} returned refs/remotes/origin/dmz, and if so, how did it come to that point.

Thank you for all the debugging so far! 🙇

[UnifyBadPirate]

My apologies @mislav , didn't grab that command while I was reproducing, I removed the repo and re-cloned and haven't been seeing the issue any more. Going to leave that as a way to workaround this issue (while kind of a pain) in case anyone else sees it and close.

[hermanbanken]

I also had this. Also caused by branches that I had locally that were tracking remote branches that no longer existed (due to GitHub "delete branch" feature). Luckily I found this issue for the HUB_VERBOSE=1. Would have helped if hub sync --help referenced that... I was looking for a -v instead.