Git-subrepo: Can't Branch at "Check for rebase" step

[TrevorMills]

I'm working in my Main Repo and I'm tracking a Feature Repo as subrepo in the path/to/subrepo directory.

I run:

git subrepo push -v -d path/to/subrepo/

and get:

>>> git rev-parse --verify HEAD
* Fetch the upstream: [email protected]:feature-repo.git (my_branch).
  * Fetch '[email protected]:feature-repo.git' (my_branch).
>>> git fetch --quiet [email protected]:feature-repo.git my_branch
  * Get the upstream subrepo HEAD commit.
>>> git rev-parse FETCH_HEAD^0
  * Create ref 'refs/subrepo/path/to/subrepo/fetch'.
>>> git update-ref refs/subrepo/path/to/subrepo/fetch f80cf735a245d569ff873bcd119165b217e5e0c1
* Check upstream head against .gitrepo commit.
* Create subrepo branch 'subrepo-push/path/to/subrepo'.
  * Check if the 'subrepo-push/path/to/subrepo' branch already exists.
  * Subrepo parent: 7db63688c180c6dc151e0be0ab1cf325cb88b8c4
  * Create new commits with parents into the subrepo fetch
>>> git rev-list --reverse --first-parent 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD

...snip...

  * Find out the .gitrepo reference
>>> git config --blob c803a777d5f5d4635b69ab163bef20f9ff1d1874:path/to/subrepo/.gitrepo subrepo.commit
  * Check for rebase
git-subrepo: Can't branch: 'refs/subrepo/path/to/subrepo/fetch' doesn't contain f7784853f5461a98aa19f68bb70c320f82bb1ecd.

Note that I'm tracking the my_branch branch of the Feature Repo.

Commit c803a777d5f5d4635b69ab163bef20f9ff1d1874 in my Site Repo was done back when I was tracking the development branch of my Feature Repo. The commit contains the following change to path/to/subrepo/.gitrepo:

[subrepo]
    remote = [email protected]:feature-repo.git
    branch = development
-   commit = 54fc175ccc6f3fc5961105a477632d9f41c5ac51
+   commit = f7784853f5461a98aa19f68bb70c320f82bb1ecd
    parent = cd65da93895bd7fa4eaacb61901f5aa5173c643d
    cmdver = 0.3.1
    method = merge

So, commit f7784853f5461a98aa19f68bb70c320f82bb1ecd is in the development branch of my Feature Repo.

Can you help guide me on how to resolve this so I can push to my_branch of my Feature Repo? Is it possible to resolve without merging development back into my_branch?

[grimmySwe]

@TrevorMills
First, note that 0.3.1 has some pitfalls, if possible I would recommend you to try out release_0.4.0 branch of git-subrepo. Not released yet but it's getting closer.

So if I understand correctly you have tracked 'development' done some changes and now you want to push that to 'my_branch'?

Do you want to overwrite the 'my_branch' with what you have locally?

• I would say that git subrepo push -f -u -b my_branch should do the trick but I am not sure if that works.

or do you want to fetch the changes from 'my_branch' so you don't lose anything there?

• First do a git subrepo pull -u -b my_branch
• Then you should be able to push

[TrevorMills]

Hi Magnus,

Thank you for the response. Note, I am tracking the issue/216-G branch of git-subrepo. I tried release/0.4.0 branch and had the same behaviour.

Your understanding is almost correct. I used to track the development branch, and successfully pushed and pulled changes. Then, a while back, I switch to track the my_branch branch. I did the switch by running this command:

git subrepo clone --force --branch=<branch> <remote> <directory>

This may be the first time I've tried to push since the changeover - I can't recall for sure.

At the end of it all, yes, the remote my_branch should look exactly like what I have locally, though I'd like there to be the commit history for all of the changes.

I tried the -f -u pushing and it didn't work. I received the same error:

git-subrepo: Can't branch: 'refs/subrepo/path/to/subrepo/fetch' doesn't contain f7784853f5461a98aa19f68bb70c320f82bb1ecd.

I tried doing a pull first and then a push and still the same error message.

I wonder if the thing to do is to checkout development again, and push directly to that branch.

Unless you have any other ideas?

[grimmySwe]

Hi Trevor,

Ok, that sounds strange. The clone command should have erased any tracks of the development branch.

What if you to a git subrepo clean path/to/subrepo? This can remove some old local traces that might disturb.

Then I would suggest doing these steps:

$ git subrepo fetch path/to/subrepo
$ git subrepo branch path/to/subrepo
$ git log subrepo/path/to/subrepo

Then you can compare the log output with the one in you subrepo and see if there are any strange differencies.

[TrevorMills]

Alas, no

me$ git clean path/to/subrepo
Removed branch 'subrepo/path/to/subrepo'.
Removed remote 'subrepo/path/to/subrepo'.
me$ git fetch path/to/subrepo
me$ git branch path/to/subrepo
git-subrepo: Can't branch: 'refs/subrepo/path/to/subrepo/fetch' doesn't contain f7784853f5461a98aa19f68bb70c320f82bb1ecd.

I wonder if it's because that commit in the main repo (c803a777d5f5d4635b69ab163bef20f9ff1d1874 ) introduces the change to the .gitrepo file:

[subrepo]
    remote = [email protected]:feature-repo.git
    branch = development
-   commit = 54fc175ccc6f3fc5961105a477632d9f41c5ac51
+   commit = f7784853f5461a98aa19f68bb70c320f82bb1ecd
    parent = cd65da93895bd7fa4eaacb61901f5aa5173c643d
    cmdver = 0.3.1
    method = merge

So subrepo thinks , with that file in the main repo, that it's on the development branch, and it looks for commit f7784853f5461a98aa19f68bb70c320f82bb1ecd in the history, but since we're actually now tracking my_branch that commit doesn't exist.

[TrevorMills]

Is there a way to tell subrepo to only look so far back into the history of the main repo? Maybe the commit just immediately after c803a777d5f5d4635b69ab163bef20f9ff1d1874, so it doesn't try to go back so far?

[grimmySwe]

Have you edited/merged the .gitrepo manually?

As you say commit in the .gitrepo should point out the latest commit that has been pulled into your subrepo. so if that is something that doesn't exist then subrepo can't perform any valid operation.

[grimmySwe]

Is there a way to tell subrepo to only look so far back into the history of the main repo? Maybe the commit just immediately after c803a777d5f5d4635b69ab163bef20f9ff1d1874, so it doesn't try to go back so far?

There is a squash option to push that simply pushes the latest state but I think that it will still require you to build up a valid history.

[TrevorMills]

I haven't edited it manually ( not yet, but I'm wondering now if that's something that I need to try doing ).

The thing that I did that has got me here, I think, was when I switched branches from development to my_branch.

My .gitrepo file currently looks like:

[subrepo]
    remote = [email protected]:feature-repo.git
    branch = my_branch
    commit = f80cf735a245d569ff873bcd119165b217e5e0c1
    parent = 7db63688c180c6dc151e0be0ab1cf325cb88b8c4
    cmdver = 0.3.1
    method = merge

Commit 7db63688c180c6dc151e0be0ab1cf325cb88b8c4 is from August 14. Commit c803a777d5f5d4635b69ab163bef20f9ff1d1874 ( which is the one that git-subrepo gets to when it tries to rebase and we get the can't branch error ) is from July 21. Is it surprising to you that the push command is going back further in history than the parent commit from the .gitrepo file? The picture I have in my head is that it only goes back as far as the parent, looking for changes to the path/to/subrepo directory. Maybe that's not quite right.

[grimmySwe]

You are right that the parent commit should be the end of it. It indicates when the branch was last pushed (or cloned)

What about the subrepo commit f80cf735a245d569ff873bcd119165b217e5e0c1is that the expected one?

There are some troubles with branches that fan out a lot, because we try to find a good history path to recreate.

Could you check the .gitrepo file in 7db63688c180c6dc151e0be0ab1cf325cb88b8c4 and its child? Because what subrepo does is that it traverses through the old commits extracting information from the .gitrepo files to try and recreate the history and make it possible to merge in the information at the right spot.

[TrevorMills]

A ha,

The .gitrepo file in 7db63688c180c6dc151e0be0ab1cf325cb88b8c4 contains:

[subrepo]
    remote = [email protected]:feature-repo.git
    branch = development
    commit = 54fc175ccc6f3fc5961105a477632d9f41c5ac51
    parent = cd65da93895bd7fa4eaacb61901f5aa5173c643d
    cmdver = 0.3.1
    method = merge

So, at that point, it thinks it's on development. The commit immediately following ( after, in time ) 7db63688c180c6dc151e0be0ab1cf325cb88b8c4 was where I switched branches:

commit 657fb621b32bf9b045f35bdeb2dcccc9a5da6574
Author: Trevor Mills <[email protected]>
Date:   Mon Aug 14 13:36:08 2017 -0400

    git subrepo push --branch=my_branch path/to/subrepo

    subrepo:
      subdir:   "path/to/subrepo"
      merged:   "f80cf73"
    upstream:
      origin:   "[email protected]:feature-repo.git"
      branch:   "my_branch"
      commit:   "f80cf73"
    git-subrepo:
      version:  "0.3.1"
      origin:   "https://github.com/ingydotnet/git-subrepo"
      commit:   "e37942c"

Now, that obviously worked.

What do you think if I changed my current .gitrepo file to point to 657fb621b32bf9b045f35bdeb2dcccc9a5da6574 as the parent instead of
7db63688c180c6dc151e0be0ab1cf325cb88b8c4?

[TrevorMills]

And, f80cf735a245d569ff873bcd119165b217e5e0c1 is the latest commit to my_branch on the remote.

[grimmySwe]

When you git subrepo clone the parent should be the current commit BEFORE you commit the new change. So that seems ok. Because you can't include the hash of the commit where it is changed because it doesn't exist yet.

What happens if you run git rev-list --reverse --first-parent 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD on your branch?
From the log in the original post it looks like we somehow get to
657fb621b32bf9b045f35bdeb2dcccc9a5da6574 is that included in the output above?
The command is what subrepo uses internally to find out which commits in the main repo to process.

Is there a different output with:
git rev-list --reverse --ancestry-path 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD

ancestry-path will always get to the right spot but can get confused by other stuff, thats why we use first-parent right now.

[TrevorMills]

Hi Magnus,

git rev-list --reverse --first-parent 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD does NOT include 657fb621b32bf9b045f35bdeb2dcccc9a5da6574

git rev-list --reverse --ancestry-path 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD is a much shorter list and DOES include 657fb621b32bf9b045f35bdeb2dcccc9a5da6574, right at the top.

[grimmySwe]

Trevor, I must have had a short circuit in my brain or a copy paste error...

The culprit commit is actually: c803a777d5f5d4635b69ab163bef20f9ff1d1874

So the same question again, is c803a777d5f5d4635b69ab163bef20f9ff1d1874 included in any of the rev-list commands?

The result from your previous check indicates that there are differences and that the --first-parent seems to go of in another direction.

[TrevorMills]

Yes, git rev-list --reverse --first-parent 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD DOES include c803a777d5f5d4635b69ab163bef20f9ff1d1874

No, git rev-list --reverse --ancestry-path 7db63688c180c6dc151e0be0ab1cf325cb88b8c4..HEAD does NOT include c803a777d5f5d4635b69ab163bef20f9ff1d1874

[grimmySwe]

ok, if you want to try, you can test and change the code lib/git-subrepo, just replace --first-parent with --ancestry-path and see if that does it.

I will dive deeper into this and see if I can come up with a solution that solves both cases.

[grimmySwe]

Trevor, would it be possible to supply some git logs from your case?
I am interested in the structure of your repo and how I can reproduce it.
So if possible git log --oneline --graph and the output of the two git rev-lists above

If you don't want to expose it here you can email it to [email protected].

[TrevorMills]

Hi Magnus,

Thank you so much for excellent support and attention to this issue. I really appreciate it.

Changing lib/git-subrepo to use --ancestry-path instead of --first-parent did the trick. I was then able to run git subrepo push path/to/subrepo. Huzzah! I've changed it back to --first-parent now as I trust that's the better one to use in most cases.

I will still send you the log files as you've requested to your email.

[grimmySwe]

Thanks Trevor for all your help!

I think I have come up with a solution to the problem and added it to the release_0.4.0 branch. So now we perform an --ancestry-path but from that result we only follow one path from start to end.

[TrevorMills]

Hi Magnus,

Just looping back to this to say thank you once more. git-subrepo has been working flawlessly, as usual.

Thanks,
Trevor