Dvc: cannot rebase if a new file is created which is not present in .gitignore in the base branch

[ankurrajdev]

Please provide information about your setup
DVC version(i.e. dvc --version), Platform and method of installation (pip, homebrew, pkg Mac, exe (Windows), DEB(Linux), RPM(Linux))

Note: checking out '171d6c61b9a643be0d2d998f2d3b644c33bc4e4c'.
I created some new files namely cavchon.parquet and cavachon_result.csv.
I am getting the following error when I am trying to rebase it to the base branch.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at 171d6c6 ENH: add the device modifier based on adgroup type and device typ
WARNING: Cache 'd273ac33145468f344dd2a08b5dae827' not found. File 'data/cavachon_result.
WARNING: data 'data/beagle_predictions.parquet' exists. Removing before checkout.
WARNING: Cache '78e5b1b8677dec94d2f2f64b5c7b649f' not found. File 'data/cavachon.parquet
WARNING: data 'data/beagle.parquet' exists. Removing before checkout.
WARNING: data 'models/beagle.joblib' exists. Removing before checkout.
WARNING: data 'data/beagle_train.parquet' exists. Removing before checkout.
WARNING: data 'data/beagle_test.parquet' exists. Removing before checkout.
ERROR: unexpected error - Checkout failed for following targets:
data/cavachon_result.csv
data/cavachon.parquet
Did you forget to fetch?

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
could not detach HEAD

[efiop]

Hi @ankurrajdev !

Could you provide dvc version output, please?

Do you have our git hooks installed?

[ankurrajdev]

Hi @ankurrajdev !

Could you provide dvc version output, please?

Do you have our git hooks installed?

dvc version is 0.80.0.

Yes I have installed the git hooks.

[ghost]

I'm not able to understand the problem :see_no_evil:

@efiop , @ankurrajdev , could you provide a reproduction script for it?

[efiop]

@mroutis When rebasing git runs git checkout, which triggers dvc checkout, which is not able to checkout and looks like that is interrupting the rebase. Need to take a closer look.

[efiop]

@ankurrajdev For now a workaround would be to disable (remove) .git/hooks/post-checkout.py , it would prevent dvc checkout from being run automatically after git checkout, but you could still run dvc checkout by-hand.

[ankurrajdev]

@ankurrajdev For now a workaround would be to disable (remove) .git/hooks/post-checkout.py , it would prevent dvc checkout from being run automatically after git checkout, but you could still run dvc checkout by-hand.

Got it thanks. I copied everything again and created a new branch and it's working right now. If it happens again, I'll let you know.

Thanks for the support.

[ghost]

@efiop , I think git rebase with --no-verify option should also work.

[ankurrajdev]

@efiop @mroutis
I am getting the same error, but this time I have a clear picture.
For instance the base branch is A. My teammate and I branched out to branches B and C respectively. My teammate merged branch B to A, so I need to rebase. In branch C, on which I was working, I deleted a few files which were present in both branch B and C. So, when I am trying to rebase, I am getting an error 'File not found'.

In the above screenshot, all the parquet files are present in branch A and B but not in C.

I tried rebasing with --no-verify as well.

Any suggestions how to deal with it?

[efiop]

@ankurrajdev Thanks for clarifying! As I've noted before, the workaround is to remove .git/hooks/post-checkout, so that this hook won't run at all. As to proper solution, we need to make our checkout hook smarter and not run checkouts on rebases. Need to take a closer look.

[ankurrajdev]

Okay thanks. Just wanted to clarify the issue :)

[IamGianluca]

@efiop Is this bug supposed to be sorted with v0.83.0? I'm asking because I still have this issue with the most recent version of DVC.

[efiop]

@IamGianluca Yes, it should've been fixed in 0.83.0 (even earlier). Reopening. Could you provide some details?

@ankurrajdev Sorry, forgot to ping you earlier to try it out. Could you please give it a try and let us know if it is fixed for you?

[IamGianluca]

You're too fast @efiop ! I was going to remove my previous message and say it works. I forgot to manually remove the hooks and run again dvc install.

I can confirm that v0.83.0 git rebase works now!

Thank you again for that :+1:

[efiop]

@IamGianluca Phew, glad to hear that :slightly_smiling_face: Closing for now.

@ankurrajdev Would appreciate if you could also give it a try and confirm that it works for you :slightly_smiling_face:

Thanks for the feedback, guys! :pray: