Git-lfs: Encountered 1 file(s) that should have been pointers, but weren't

[cburanicz]

Hi, this is my first time posting, and has been my first time using git, so please let me know if I'm making some embarrassing mistakes :)

We're using git lfs for a company project, and my computer has been the only one where lfs checkouts hang indefinitely. The branch status is marking a bunch of files dirty that dont exist. I'm getting the error;
Encountered 1 file(s) that should have been pointers, but weren't when attempting to checkout an individual file.

This issue seems similar to #1900 that is still being addressed...so there doesnt seem to be an obvious solution yet. What can I do to resolve this issue?
Doing a checkout --force with GIT_TRACE_PACKET flag returns a HUGE dump of what looks like an ASCI representation of the file ( hundreds of lines ). The command simply hangs after that.

Thank you!

[technoweenie]

Hey, thanks for the report. The problem is that you have a file that _should_ be using LFS (according to a pattern in .gitattributes), but it isn't. The dirty git status is just converting your file to use LFS, so it should be safe to commit.

The hanging command is fixed with https://github.com/git-lfs/git-lfs/pull/1932, which is slated for LFS 2.0.0. I wanted to backport that for a new 1.5.x bugfix release though.

[cburanicz]

hey, thanks for getting back to me. I actually want to files entirely, so I can checkout another branch. I usually do a git stash, and that gives me a clean git status. but this time, stash just hangs indefinitly.
If I want to checkout another branch, but dont have a clean status, what should I do?
Thanks

[technoweenie]

Run git lfs uninstall and then git reset --hard (assuming there are _no_ changes you want to keep!). Once you're on the branch you want, you can run git lfs install && git lfs pull to reinstate the internal git lfs hooks, and then replace any LFS pointers with the actual files from the LFS server.

[cburanicz]

ok. I'll try that next time. Funny, we couldn't find lfs uninstall, and ran into your post from almost two years ago saying LFS uninstall didnt exist. Either way, I'm sure your solution will work. I think I can close this issue. thanks so much!

[technoweenie]

Thanks for the feedback! I'll be really curious to hear your thoughts after v2.0 launches :) But don't be shy about filing further issues in the mean time.

Funny, we couldn't find lfs uninstall

Couldn't find it where? Do you mean under "High level commands" when you run just git lfs?

[cburanicz]

Tpying in "git lfs" shows the install command, but not the uninstall command in git bash!! However, I can confirm that lfs uninstall and install worked for us!

[SychevIgor]

Issue still exists...
my 2 cents:
From my perspective - issue should be solved in an app, not a workaround like "turn off/on"

[ricky26]

This issue is still a problem - running in CI, if anyone doesn't install LFS and checks in an LFS-marked file as a normal binary file, the build machine gets stuck because it can't swap branch any more. There needs to be a way to checkout 'clean' (i.e. without fixing up pointers). Uninstalling and reinstalling LFS every build sounds like it could create issues for other build operations running in parallel.

[pthurmond]

Uninstall and re-install just ain't working. My co-worker committed without having LFS installed. I think it broke everything. Now I am trying to fix this mess.

[trejkaz]

Same situation here. Someone committed without LFS installed, uninstall command runs without complaint but then when I try to checkout the file I get the same error again.

[ttaylorr]

git lfs migrate import over the range of broken commits should fix the problem.

[jloudermilk]

i had this problem and git lfs migrate import --include="*.file extension that is the problem file " worked!

[rajasekharnukala]

Is there a permanent fix? Is there a way to push(git lfs migrate import) it to the repository hosted in github?

[ttaylorr]

Is there a permanent fix? Is there a way to push(git lfs migrate import) it to the repository hosted in github?

Yes! git lfs migrate import will repair your repository by migrating any blobs that should be stored with LFS to be so. You can push this to a remote via git push --all origin, or git push --all --force origin if you migrated commits that are already present on the remote.

[trejkaz]

Is there an equivalent for people migrating away from lfs?

[ttaylorr]

Is there an equivalent for people migrating away from lfs?

@trejkaz: no, not currently. Something like git lfs migrate export would be certainly possible. My main concern is that it would require downloading all LFS objects that are in the migration range, which could be potentially large (to the extent that they don't fit on disk).

That said, if the total size of your objects doesn't fit on disk, I am not sure if this is a likely scenario given what I imagine the typical use-cases of LFS to be. I wonder if it would be sufficient to download all objects in range, and then do the migration.

Another thing that people would have to be aware of is that there is currently no way to communicate "I want to remove this object" over the LFS API. I think that we would have to add this (potentially in a way that is extensible or supports the idea of "object expiration") before adding a git lfs migrate export command would be useful.

@technoweenie what do you think?

[technoweenie]

I'm all for an lfs migrate export command :+1:

The idea of a delete operation in the batch api kind of scares me though. It'd be really easy to blow your data away, and I don't know what the backup/undo status of other LFS servers is. I'd just rely on LFS servers providing these capabilities.

[ttaylorr]

I'd just rely on LFS servers providing these capabilities.

I think that that's fair. What do you think about shipping this in v2.5.0?

[sandersaares]

git lfs migrate import will repair your repository by migrating any blobs that should be stored with LFS to be so

This does not seem to be the case.

Encountered 2 file(s) that should have been pointers, but weren't:
        SetupDocker/docker-ubuntu16.deb

...

git lfs track *.deb
"*.deb" already supported

...

git lfs migrate import
migrate: Fetching remote refs: ..., done
migrate: Sorting commits: ..., done
migrate: Rewriting commits: 100% (0/0), done
migrate: Updating refs: ..., done
migrate: checkout: ..., done

...

git push --all
Locking support detected on remote "origin". Consider enabling it with:
  $ git config lfs.https://xxx.visualstudio.com/xxx.git/info/lfs.locksverify true
Everything up-to-date
...

[ttaylorr]

@sandersaares presently, you must specify the patterns that you wish git lfs migrate to pick up, i.e., git lfs migrate import --everything --include='*.deb'. I think that reading the .gitattribute(s) in a repository if no --include argument is given would be an interesting pull request, and I'd be happy to mentor someone through that if anyone has interest.

[subatomicglue]

Is there a way to have *.png for git-lfs, but exclude a single myfile.png to go into normal git? My problem is that the htmlpreview on github.io doesn't work with git-lfs, and I want one of my images to show in the preview (but still want .png rules for other images, if I ever add them)...

So I added/pushed myfile.png to git without the *.png .gitattribute, then re-added the *.png .gitattribute for future .png adds. So now myfile.png works in htmlpreview... :)

Encountered 1 file(s) that should have been pointers, but weren't

Is getting annoying for this file, because I actually want it in git, not in LFS... :( I can't ignore this error. Normally, I can see how useful this helpful message would be though.

[ttaylorr]

Is there a way to have *.png for git-lfs, but exclude a single myfile.png to go into normal git?

Certainly. You can accomplish this with:

$ git lfs migrate import --everything --include='*.png' --exclude='myfile.png'

[subatomicglue]

That helped, I ended up adding myfile.png -filter -diff -merge -text to the end of my .gitattributes. order seems to matter, and adding to the end overrides the general filter. Thanks!

To avoid this hell:

Encountered 1 file(s) that should have been pointers, but weren't

I just had to git rm the file and commit/push (or probably git add, but I didn't want to add it to lfs), otherwise I was caught in a weird merge hell loop, where I could never rebase --continue or --skip.

[PointedEars]

@sandersaares presently, you must specify the patterns that you wish git lfs migrate to pick up, i.e., git lfs migrate import --everything --include='*.deb'. I think that reading the .gitattribute(s) in a repository if no --include argument is given would be an interesting pull request, and I'd be happy to mentor someone through that if anyone has interest.

Thanks a lot. I do not know if I have time for the pull request, but I have written a working solution using (ba)sh and (g)awk:

git lfs migrate import --everything$(awk '/filter=lfs/ {printf " --include='\''%s'\''", $1}' .gitattributes)

(You can test it with echo git lfs migrate import --everything$(awk '/filter=lfs/ {printf " --include='\''%s'\''", $1}' .gitattributes).)

Can this be integrated?

[ttaylorr]

Can this be integrated?

Nice. That's a good solution, although it does not migrate locations
that were supposed Git LFS pointers historically (and were marked
appropriately as such), but have since been exported to no longer be,
and are thus no longer corrupt.

We shipped in v2.5.0 a new option to 'git lfs migrate import' called
'--fixup' which does just that. Before migrating a new commit, we read
all of the .gitattributes file(s) that we need to determine which
files at that point in history need to be converted.

Then, we do the conversion one by one, in effect simulating what you
wrote above at _every_ point in history (e.g., where the awk(1) command
is invoked at each historical view of the repository instead of the
tip).

So, I think that the one-liner that you're looking for is:

$ git lfs migrate import --fixup --everything

Does that suit your purposes?

[indirectlylit]

For future searchers:

There is now a --no-rewrite option for migrate import (see https://github.com/git-lfs/git-lfs/issues/2910 and https://github.com/git-lfs/git-lfs/pull/3029):

git lfs migrate import --no-rewrite -m "[new_commit_message]" [path_to_file]

This is helpful for fixing a shared git branch whose history should not be re-written.

[A-Sam]

lfs importing didn't work with me. The only thing worked
git rm --cached -rf .
git reset HEAD --hard

[Hemaolle]

Found a fairly simple solution

git rm .gitattributes
git reset --hard HEAD

https://stackoverflow.com/questions/11383094/unstaged-changes-left-after-git-reset-hard

[joneshf-cn]

git lfs migrate import over the range of broken commits should fix the problem.

Could anyone supply an example of this syntax? I'm having a hard time specifying to only import a range of commits. Any commands I attempt to run either migrate every commit in history or nothing at all.

[bk2204]

While you can use git lfs migrate import to fix this if you want to fix the history (and you can specify branches to include with their name and to exclude with ^BRANCHNAME), the easiest way to fix this if you're just going forward is to run git add --renormalize . and then commit.

If you do want to fix the history, and you want to include the master branch but not develop, you can write git lfs migrate import --include='*.png' master ^develop.

[joneshf-cn]

Sorry, I can be more explicit. The commits look like this:

*   G <- This is the latest commit
|\
| |
| |
| * F
| * E
|/
*   D
|\
| * C <- This is the problem commit
|/
*   B
*   A <- This is an older commit

The problem commit is C. I'd like to re-write everything from C to G without affecting past commits like A or B. I can't figure out that command. I'd even take a command that would require B to be re-written so long as A (or older) didn't have to be.

Is there a way to do that?

[joneshf-cn]

Also, thanks for the quick response!

[bk2204]

Sure, you can specify the branches as G ^B. That means to include G and its history and exclude B and its ancestors. You can verify that this does what you want by passing those arguments to git log, which will show you which commits will be walked (we use git rev-list internally, which uses the same syntax).

[joneshf-cn]

Sorry, I can't seem to get this syntax to work. I'm running something like this:

$ git lfs migrate import --include '*.png' G ^B   
migrate: override changes in your working copy? [Y/n] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done                                                                                                                                          
migrate: Sorting commits: ..., done                                                                                                                                               
migrate: Rewriting commits: 100% (0/0), done                                                                                                                                      
migrate: Updating refs: ..., done                                                                                                                                                 
migrate: checkout: ..., done

git rev-list G ^B works like you suggest, but not the above command.

[joneshf-cn]

Also, this is the version I'm on:

$ git lfs version  
git-lfs/2.5.2 (GitHub; darwin amd64; go 1.11.5)

Is this something that only works in a newer version of git-lfs?

[joneshf-cn]

Wait, do they have to be actual branches? Or do commit SHAs work? I've been specifying SHAs, but you've been saying branches.

[joneshf-cn]

Just tried making some branches, but no change.

[bk2204]

They do have to be branches. If you don't use --include-ref and --exclude-ref, then all remote refs are also implicitly excluded, which may prevent the history you want from being traversed. Instead of using G ^B, try --include-ref=refs/heads/G --exclude-ref=refs/heads/B, which doesn't exclude remote refs from consideration.

This should work in older versions, since we haven't changed things there recently.

[joneshf-cn]

That works. Thanks for the help!

[yaxili96]

Found a fairly simple solution

git rm .gitattributes
git reset --hard HEAD

https://stackoverflow.com/questions/11383094/unstaged-changes-left-after-git-reset-hard

God, this is just the answer for me. I don't know if anyone else has the same issue:

Environment:

Windows 10 + Windows cmd.exe + WSL

Scenario:
I installed git lfs on windows cmd.exe and everything works perfectly. However, one day I opened my repo with WSL and run git status. Instantly, the bash just hung there, and then if I ran git status in cmd.exe, there would be a bunch of dirty modified message to the blobs tracked by git lfs (for sure it's caused by EOF). No matter what I tried (git lfs migrate, or git lfs uninstall && git reset --hard), git just gave me:

Encountered xx file(s) that should have been pointers, but weren't:

Anyone has an idea why this would happen?

[kagJacky]

Found a fairly simple solution

git rm .gitattributes
git reset --hard HEAD

https://stackoverflow.com/questions/11383094/unstaged-changes-left-after-git-reset-hard

This one worked for me. Thanks.

[bk2204]

An easy solution to fix this, which will make its way into the Git FAQ soon, is to do this:

$ git add --renormalize .
$ git commit -m "Fix broken LFS files"

Note that that requires a decently recent Git for the --renormalize option. If you don't want to rewrite history, this is the easiest solution. You also want to do this on an otherwise clean working tree.