Git-lfs: lfs tracked files are sometimes incorrectly shown as modified in "git status" output

[AndrewJDR]

I relatively frequently run into a situation where git status will show one or more lfs-tracked files as being modified when they are not in fact modified. I have yet to nail down exact reproduction steps, but I do have some bits of information about it:

• git checkout -- <the lfs tracked files> Will not resolve the issue; The files will still be shown as modified in an immediately subsequent git status output.
• It can be "fixed" by doing the following:
  
  
  
  • Emptying the .gitatttributes file (i.e. run cp /dev/null .gitattributes or similar)
  
  
  • Running git status
  
  
  • Running git checkout -- .gitattributes (Restores the .gitattributes file)
  
  
  • Running git status again. The lfs-tracked files are now not shown as being modified anymore, as expected.
  
  
  • (Note that these steps did not include a step like git checkout -- <lfs tracked files>, which tells me the files were indeed completely unmodified the entire time.)
  
  
• It seems like problem might occur immediately after a git reset --hard <tree-ish> or git checkout <tree-ish>, but I am not 100% sure on this.

Has anyone else encountered this?

[AndrewJDR]

I have noticed #1544, but that issue seems to only happen after a merge conflict on the .gitattributes file, so I'm not sure that is the same issue.

I can confirm that this has happened to me many times even when there was no merge conflict on .gitattributes.

Also, next time this happens I can certainly try to gather some more information. If anyone has some suggestions as to what information I should be gathering, please let me know.

[AndrewJDR]

Also, just to be clear --

The lfs tracked files in question in my example have already been committed as lfs-tracked files long ago and have been that way for quite some time and working completely properly and as expected.

I just want to clarify that don't have any confusion about how to use git-lfs, and how to initially get files tracked/checked in as lfs files, etc. :)

fwiw, I've observed a similar problem with other tools that use git smudge/clean filters, such as git-fat. This therefore might a problem in git-core (as #1544 appears to be), but it would be very nice to get it tracked down and resolved/worked around one way or the other, because less experienced users really have no idea how to fix this problem when it happens, and it prevents them from being able to merge, reset, checkout, or do any other action that requires a clean working directory/checkout.

[AndrewJDR]

I'll post an example of one of my .gitattributes files here just for completeness, but I'll also note that I've experienced this problem on many different repositories with varying .gitattributes files:

*.dll filter=lfs diff=lfs merge=lfs -text
*.lib filter=lfs diff=lfs merge=lfs -text
*.ilk filter=lfs diff=lfs merge=lfs -text
*.pdb filter=lfs diff=lfs merge=lfs -text
*.pch filter=lfs diff=lfs merge=lfs -text
*.exp filter=lfs diff=lfs merge=lfs -text
*.so.* filter=lfs diff=lfs merge=lfs -text
*.so filter=lfs diff=lfs merge=lfs -text
*.a filter=lfs diff=lfs merge=lfs -text

[AndrewJDR]

I noticed https://github.com/git-lfs/git-lfs/pull/1104 which sounds similar, but it seems to only apply to files that were at one time not being tracked by git-lfs and later become tracked.

I can 100% confirm that the lfs-tracked files in question were always being tracked by lfs in my repositories, since the very beginning of history in those repositories, so I don't think #1104 applies here.

[larsxschneider]

Thanks for the report! Can you post the Git version, Git LFS version, file system type, and OS (incl. version) of the system that you see the error on?

[AndrewJDR]

git version 2.10.2
git lfs 1.4.4
ext4
Linux mint (Trusty 14.04 based) w/ kernel 4.6.2

[technoweenie]

Also, next time this happens I can certainly try to gather some more information. If anyone has some suggestions as to what information I should be gathering, please let me know.

From my basic understanding, a file showing up in git status means content in the git index or working directory differs from what is in the current commit. Clean filters and LFS pointers add a level of indirection that makes this more complicated. Maybe we could build some kind of debug command that shows us:

1. Is the git blob in the recent commit an LFS pointer or the actual blob?
2. Is the git blob in the index the LFS pointer of the actual blob? Does it differ from the commit?
3. Is the actual content in the working tree different?

[ttaylorr]

Maybe we could build some kind of debug command that shows us:

I think we should be able to do this with the commands that Git gives us. Some of these (mainly the first one) are a little awkward, but this can at least give us a starting point for debugging this problem.

As an aside, I definitely agree that packaging these together into an LFS command would be much easier than running these individually 👍

1. Is the git blob in the recent commit an LFS pointer or the actual blob?

$ git cat-file -p "$(git ls-tree $REV $FILE | awk '{ print $3 }')" | tee file.recent

2. Is the git blob in the index the LFS pointer of the actual blob? Does it differ from the commit?

$ git show ":$FILE"
$ diff -u <(git show ":$FILE") <(cat file.recent)

3. Is the actual content in the working tree different?

$ cat "$FILE" # for object contents
$ git cat-file -p "$FILE" # for pointer contents

[AndrewJDR]

Thanks for the pointers everyone. When this happens again, I'll run these commands and report back.

[jpschwinghamer]

@AndrewJDR I'm encountering this issue as well in our team's repo.

[noderat]

Yeah this seems to be an issues for my entire team, we've now experienced it with two separate repos.

[rob-moore]

I'm running into this issue as well. It's still prevalent even after wiping and re-cloning my repo.

[ttaylorr]

Possible for anyone to run the commands above and tell me where the discrepancy is?

[noderat]

@ttaylorr Okay so performing those commands with $REV set to last commit, and $FILE to one of the files in question:

1. It spits out the binary blob
2. Again spits out the blob and diff produces no output and exits with exit-code 0
3. cat returns binary blob, cat-file returns an error message:

$ git cat-file -p "$FILE"
fatal: Not a valid object name path/to/file/in/question.jpg

[ttaylorr]

3. cat returns binary blob, cat-file returns an error message:

$ git cat-file -p "$FILE"
fatal: Not a valid object name path/to/file/in/question.jpg

Hm, an argument to cat-file without the : prefix should print out the contents in your working tree (which I'd expect to be equal to the binary blobs above).

[technoweenie]

Try git cat-file -p ":$FILE". That will print the contents of the git blob for that file. It should be the LFS pointer.

[AndrewJDR]

I've continued to try, but I've had difficulty reproducing this in recent days.

I'm hoping @noderat and/or @rob-moore can shed light by running the commands listed above on their offending repos, as well as the more recently mentioned git cat-file -p ":$FILE" mentioned by @technoweenie

Given that this bug is a little bit elusive at times, I also have one other request to make of those that have been able to reproduce this (@noderat / @rob-moore) -- please make a backup of your offending repo first rather than fixing it in-place and going along your merry way :) This way you can refer back to it easily when the git-lfs devs ask more questions about it. Thanks! Hope this can be tracked down soon.

[noderat]

@AndrewJDR @ttaylorr Many repos, same issue, multiple machines. I'll stumble upon it again sometime during my shift today and I'll post results then and have them back up their repo when it occurs. I'm pretty sure it happens right after git lfs clone so I should know shortly.

[AndrewJDR]

@noderat Another thing to try when this comes up again:
git checkout -- <offending lfs tracked files>
In my case, this seems to do nothing -- the lfs tracked files still appear as modified in the git status output after running this command. It'd be good to get confirmation that the same thing happens for you.

[AndrewJDR]

I have identified one way of reproducing the issue, and it seems to happen when symlink files are involved whose filename matches the pattern of a git-lfs entry in .gitattributes. When such a repo is created/pushed from linux and cloned on windows, the issue will occur. Here's how to reproduce it.

On Windows with latest git for windows, 2.11.0 (download):

$ git --version
git version 2.11.0.windows.1

$ git-lfs version
git-lfs/1.5.3 (GitHub; windows amd64; go 1.7.4; git 23455e48)

$ git config --get core.symlinks
false
 # ^ This might be important; it should print 'false', or you may have trouble reproducing this.
# Though to be fair, on my system, setting it to 'true' did not fix the issue :)

$ git clone https://github.com/AndrewJDR/git-lfs-symlink-issue.git
Cloning into 'git-lfs-symlink-issue'...
remote: Counting objects: 9, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 9 (delta 1), reused 9 (delta 1), pack-reused 0
Unpacking objects: 100% (9/9), done.
Downloading file_1.a (1024 B)

$ cd git-lfs-symlink-issue/

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   file_1_symlink.a

no changes added to commit (use "git add" and/or "git commit -a")

$ git diff file_1_symlink.a
diff --git a/file_1_symlink.a b/file_1_symlink.a
index 8c9108a..233c527 120000
--- a/file_1_symlink.a
+++ b/file_1_symlink.a
@@ -1 +1,3 @@
-file_1.a
\ No newline at end of file
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cfd7662ccdb6718b9b44ad36430b6fc935bbfc30c412d9416201c8d1170efe2
+size 8

$ # Now that we're on windows, git has checked out a symlink as a plain text
file that contains the path to the file being pointed to. This is normal
behavior for git on windows unless you have core.symlinks set to true, which
is not very common (requires special privileges in windows). Let's note the
checksum of the file.

$ md5sum file_1_symlink.a
d21e41d834267c53db52b2eb40f7beb9 *file_1_symlink.a

$ git checkout -- file_1_symlink.a

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   file_1_symlink.a

no changes added to commit (use "git add" and/or "git commit -a")

$ md5sum file_1_symlink.a
d21e41d834267c53db52b2eb40f7beb9 *file_1_symlink.a

$ # Note that the 'git checkout' command did not change the file, and did not
change the output of 'git status'. Essentially it didn't resolve the issue.
However...

$ cp /dev/null .gitattributes

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   .gitattributes

no changes added to commit (use "git add" and/or "git commit -a")

$ git checkout -- .gitattributes

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean

$ # Now it's 'fixed'. The file is no longer showing up as being modified. So
file_1_symlink.a must've changed, I'm thinking? No...

$ md5sum file_1_symlink.a
d21e41d834267c53db52b2eb40f7beb9 *file_1_symlink.a

# The file has the exact same checksum as it did prior to the 'git status /
git checkout .gitattributes / git status' step. If I'm able to get git to be in a 
'good' state with a symlink involved by doing the status/checkout/status
steps, why couldn't it be in that 'good' state to start out with (after the clone)?
I'm not exactly sure what's going on, but hopefully this is helpful.

Here are the full steps I used to create the repository from linux, just in case it's useful:

$ mkdir git-lfs-symlink-issue
$ cd git-lfs-symlink-issue
$ git init
Initialized empty Git repository in ~/git-lfs-symlink-issue/.git/
$ git --version
git version 2.10.2
$ git lfs version
git-lfs/1.5.3 (GitHub; linux amd64; go 1.7.4)
$ git lfs track "*.a"
Tracking *.a
$ dd if=/dev/urandom of=file_1.a bs=1024 count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.000116807 s, 8.8 MB/s
$ git add .gitattributes
$ git commit -m "Tracking .a files"
[master (root-commit) fa4caca] Tracking .a files
 1 file changed, 1 insertion(+)
 create mode 100644 .gitattributes
$ git add file_1.a
$ git diff --cached file_1.a
diff --git a/file_1.a b/file_1.a
new file mode 100644
index 0000000..6d21b9e
--- /dev/null
+++ b/file_1.a
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f96a134e5fce598bc2ecd1bdcad838023beca9207dfff727e6e4abbefba23a8
+size 1024
$ git commit -m "Added file_1.a"
[master 8aed1ef] Added file_1.a
 1 file changed, 3 insertions(+)
 create mode 100644 file_1.a
$ ln -s file_1.a file_1_symlink.a
$ git add file_1_symlink.a
$ git diff --cached file_1_symlink.a
diff --git a/file_1_symlink.a b/file_1_symlink.a
new file mode 120000
index 0000000..8c9108a
--- /dev/null
+++ b/file_1_symlink.a
@@ -0,0 +1 @@
+file_1.a
\ No newline at end of file
$ echo "Note that (presumably because it's a symlink), it's not added as a git-lfs/oid style file even though I'm tracking .a files..."
$ git commit -m "Add symlink 'file_1_symlink.a'"
[master 0f21e57] Add symlink 'file_1_symlink.a'
 1 file changed, 1 insertion(+)
 create mode 120000 file_1_symlink.a
$ git remote add origin [email protected]:AndrewJDR/git-lfs-symlink-issue.git
$ git push origin master
Git LFS: (1 of 1 files) 1024 B / 1024 B
Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (9/9), 863 bytes | 0 bytes/s, done.
Total 9 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), done.
To github.com:AndrewJDR/git-lfs-symlink-issue.git
 * [new branch]      master -> master
$

[AndrewJDR]

Just to head off any "Why are you tracking symlinks in git-lfs?" type questions:

I often have repositories where i check in large numbers of libraries (.so files, .a files, etc). It's very common for these .so/.a files to be accompanied by symlinks (e.g. libaspell.so.15 -> libaspell.so.15.2.0, etc) so that any programs depending upon those libraries can more easily find the libraries they're looking for.

The build scripts for these libraries generate many dozens of these library symlinks, and they change over time when I build new versions, so manually maintaining which .a and .so files to track and which not to track (because they're symlinks) would be very cumbersome -- The easiest thing is to just git lfs track "*.a" and git lfs track "*.so" and call it a day.

So that's what I do :)

In any case, I'd argue that one should be able to commit a symlink file with an extension that happens to be tracked by git-lfs and get sensible behavior if that repo gets cloned onto a windows machine (i.e. after git clone, it should be in a clean state and not have modified files in its git status output, etc.)

[AndrewJDR]

Also, pinging some of the lfs devs now that I have an actual repro - @ttaylorr @technoweenie

[ttaylorr]

@AndrewJDR I was able to reproduce the issue in #1749, but only when my global core.symlinks was set to false. I have a few other things on my plate right now, but I'll look into this further shortly.

[AndrewJDR]

@ttaylorr right, that does make sense. For me, it happens even when my global core.symlinks is true, though. As far as I understand, the symlink functionality in Windows requires an additional permission on the user account, so perhaps Git is running into a lack of that permission when it tries to create the symlink and falls back to the symlinks=false behavior. I can't be sure without investigating more.

[aevernon]

I can reproduce this problem at will, even after deleting my working directory and re-cloning.

git version 2.7.4
git-lfs 1.5.3
Fedora 24

Contents of my .gitconfig:

[color]
    ui = auto
[merge]
    tool = meld
[diff]
    check = true
[help]
    autocorrect = 1
[push]
    default = matching
[gpg]
    program = gpg2
[difftool]
        prompt = false
[filter "lfs"]
    smudge = git-lfs smudge -- %f
    process = git-lfs filter-process
    required = true
    clean = git-lfs clean -- %f

Contents of my .gitattributes:

model/** filter=lfs diff=lfs merge=lfs -text

Here are the results of the debugging commands that @ttaylorr suggested:

1. It prints the actual blob.
2. The Git blob in the index is the actual blob. diff does not show any differences.
3. The outputs of cat and git cat-file are identical, but I was expecting git cat-file to print a LFS pointer.

[AndrewJDR]

@aevernon is the file a symlink by chance?

[aevernon]

No, it is not a symlink.

[AndrewJDR]

@aevernon Ok, there are probably two separate issues happening here. I'm not surprised, because I seem to recall running into this problem even when symlinks weren't involved.

It's nice that you can so easily reproduce it. Hopefully one of the devs, like @ttaylorr or @technoweenie will chime in with some additional questions for you.

[aevernon]

I've discovered that my test file, phones.txt, is stored as a LFS pointer in our master branch, but my collaborator committed and pushed phones.txt to his branch as a regular file. Thus, when I check out his branch, git status shows files as being modified, presumably because Git LFS is expecting pointers.

If I run git lfs uninstall followed by git lfs install, the problem goes away.

[aevernon]

The root cause in our case was that .git/hooks/pre-push was missing in my collaborator's working directory. Thus, when he pushed his commits, it sent the actual files instead of storing them in Git LFS. I assume the hook was missing because he did not run git lfs install.

[gilprime]

We have the same in our repository, and the given workaround _(by emptying the .gitattributes then checkout it after)_ work on our repository.

In our case :
$ git lfs version --> git-lfs/1.5.2 (GitHub; linux amd64; go 1.7.3; git 547a06a)
$ git version --> git version 2.9.3

But, in our case, the modified files are png's, gif's... And the modified shown files are not the same from a developper machine to an another...

[aevernon]

@gilprime Is .git/hooks/pre-push missing for any of your collaborators?

[githubkusi]

I'm affected by the same problem:

Here are the results of the debugging commands that @ttaylorr suggested:
1) It prints the actual blob.
2) The Git blob in the index is the actual blob. diff does not show any differences.
In order to compare the binary files, I've compared by calling
git cat-file -p :bla.mat | od -c and
cat bla.mat | od -c

3) the working tree contains the blob, git cat-file returns the lfs pointer

[AndrewJDR]

Curious, can someone being impacted by this (e.g. @githubkusi, @gilprime, @noderat) provide a full copy of their local repo to the git-lfs devs as a zip file or the like? Obviously it would be important that the repo is in the "bad" state at the time that you zip it up.

[aevernon]

I upgraded to Git LFS v1.5.4, and now when I switch to the problematic branch I see this:

Pointer file error: Unable to parse pointer at: "xxx/yyy"

xxx/yyy is the same file that incorrectly shows as modified in git status.

For each file indicating a pointer file error, I called git rm --cached followed by git add. This converted blobs to Git LFS pointers.

[githubkusi]

@AndrewJDR I unfortunately am not allowed to provide you with my local git repo, but I almost always get the issue during interactive rebasing and shuffling commits with LFS assets. If time allows, I try to recreate the issue on a test repo which I could provide you with.

[AJStacy]

Confirmed that I am also experiencing this issue. Half of our development team is on Mac and half is on Windows. When a Windows user clones our codebase git believes that all of the images we have tracked in LFS are modified. Are there any workarounds besides having the Windows users check in one file at a time?

[ChrisKitching]

An extra tidbit that may help: I can reproduce this problem reliably by having a file in a repo that starts out not tracked by LFS, but then (at some point in git history) is added to LFS. The history of this file, then, is a pointer file some of the time, but go back far enough and it's a binary checked into git directly.

Such a repo is, of course, broken, but it probably shouldn't derail LFS. The "delete your gitattributes and then checkout" workaround fixes this situation.

When cloning such a repo using git-lfs 1.5.6, it now prints the following message about such files:

Encountered 5 file(s) that should have been pointers, but weren't:
    angry.model
    disgusted.model
    fearful.model
    happy.model
    neutral.model

The repo is then in the "dirty" state, showing those files as modified. The only way to make them stop showing up as modified is to apply the aforementioned workaround.

[ChrisKitching]

Oh, slight correction to my earlier comment:

These steps to reproduce only work when the repo is being cloned as a submodule of another repo. The superrepo, in my case, does not use LFS (I do not know if that makes a difference).

Cloning the repo with the binary-but-then-later-pointer files by itself does work as expected.

[ttaylorr]

An extra tidbit that may help: I can reproduce this problem reliably by having a file in a repo that starts out not tracked by LFS, but then (at some point in git history) is added to LFS.

This will only happen when the file is still a large binary (i.e., hasn't been re-staged into the index) since after being tracked by LFS. In other words, LFS will only complain about a malformed pointer if it was unable to parsed as a pointer in the latest checkout.

[michaelpalumbo]

Hi There,

I am also encountering this issue, persistently on only one of 4 machines in our lab -- always occuring on this same machine.

We do not currently have lfs installed on this machine, but it is installed on one of the other machines.

It seems to occur whenever a push is made from another machine, this machine will report changes in the local working directory on those same files in response to a 'git status' call.

This Machine is running Mac OS 10.11.3, and git version 2.10.1

[technoweenie]

@michaelpalumbo It sounds like the change is just storing the file in LFS. Git registers it as a change, because it's storing the LFS pointer in Git, instead of the actual contents. This is something we plan to address in v2.1.

[michaelpalumbo]

Thanks @technoweenie Hmm. Here's the thing, these files are very small!

Is there any way to remove LFS from my repo? Several months ago we decided just not to store large files in the git anyway, so thats why this problem also strikes me as odd.

[theholla]

Confirming that @AndrewJDR's solution fixes the problem for me. This has only happened to me twice, when I cloned the repo on a new machine, and then the next time I pulled from my original machine.

[michaelpalumbo]

Hi gang, so the problem still persists for us. As we have no need for LFS for the foreseeable future, do you think its worth trying to simply reinstall git on each machine?

[ttaylorr]

As we have no need for LFS for the foreseeable future, do you think its worth trying to simply reinstall git on each machine?

Instead of reinstalling Git on each machine, I think you'd only need to rewrite the history to replace the LFS pointers with the objects themselves.

That being said, I'm planning to work on an enhanced git lfs status command that will be able to tell why certain LFS files are marked as persistently modified.

[michaelpalumbo]

Thanks @ttaylorr ! I'll look into rewriting history for now and will report if anything novel to this thread occurs.

[Nonameghost]

Just to bump this:

git version 2.10.2.windows.1
git-lfs/2.0.1 (GitHub; windows amd64; go 1.8; git 678cdbd4)

Having this issue after my coworker who didn't have LFS set-up committed a .dll to our master branch.
I'm assuming the dll was committed as a file instead of a pointer, and now the malformed pointer is throwing off LFS and keeping the dll in our status. At least, that's what I'm getting from this thread.

The "fix" posted up top with clearing the .gitattributes and restoring it works for removing it temporarily, but it shows up again if I merge that branch or check it out it seems.

If I recompile and commit it, will it become a pointer again and stop causing this issue? I'm assuming the blob will still exist in our history but future commits would be good?

[Nonameghost]

Quick update. Recompiling the dll and pushing it seems to have corrected everything. I'm assuming that commit my coworker made would cause the issue if I checked it out.

Hopefully something I've said is helpful. Thanks for the thread, helped me out a lot.

[ttaylorr]

I just merged #2070, which should shed much more light on why particular items are showing up as changed/modified in the output of git lfs status.

[ttaylorr]

Here's a pre-release build of 2.1.0 that contains the new changes to the git lfs status command which should shed more light on why certain files are modified. If you'd like to download these and give them a shot, please let me know if there's anything that could be improved or made clearer 😄

git-lfs-darwin-386-2.0.0.tar.gz
git-lfs-darwin-amd64-2.0.0.tar.gz
git-lfs-freebsd-386-2.0.0.tar.gz
git-lfs-freebsd-amd64-2.0.0.tar.gz
git-lfs-linux-386-2.0.0.tar.gz
git-lfs-linux-amd64-2.0.0.tar.gz
git-lfs-windows-386-2.0.0.zip
git-lfs-windows-amd64-2.0.0.zip

[pascalrobert]

We had a similar problem with only one file. In our case, looks like it's because the original file was only 209 bytes long. I applied the solution from @aevernon, git rm --cached/git add, and the file switched to be a pointer.

[rikur]

I had the same issue and it was solved by: git lfs install followed by git reset --hard. My co-worker was simply missing the git hook.

[amacara1]

I'm still having the issue, consistently. Now I'm on git-lfs/2.1.0 (GitHub; windows amd64; go 1.8.1; git bd2c9987) and git version 2.13.0.windows.1
It does not happen on all binary files (tracked by lfs), all my coworkers see the problem and the set of modified files is not the same every time.
I've tried, with no success this time (during past, these solved, one at a time):

• git lfs uninstall, re-checkout .gitattributes after deletion, git lfs install
• git lfs uninstall, git checkout -- file-with-problems (at this stage I'm getting the same, "pointers, but weren't" error for the file I'm freshly checking out)
• git lfs uninstall, git reset --hard (at this stage I'm getting the same, "pointers, but weren't" error, on some files)

git lfs status shows no files to be pushed or committed, but a list of files in the "objects not staged for commit" - the same list that is generating the "pinters, but weren't" error.

LE: Wild ask: could it be from the fact that I'm jumping all over the repository checking out various branches in the same workspace?

LLE: I've switched to lfs 2.1.1 and the empying-gitattributes workaround works intermittently.

[TTimo]

Seeing exactly the problems @amacara1 describes above. That has been going on for a while now.

[ttaylorr]

Hi all, sorry that you're having trouble. I think that this issue is caused by having large objects accidentally committed into the history of your repository. Upon trying to check those files out, LFS will notice that they are large, not parse-able as a pointer, and then return an error.

One way to diagnose this is by running the improvements added to the git-lfs-status(1) in v2.1.0. If you see something like (after staging a file, therefore running it through the LFS filters):

Git LFS objects to be committed:

        a.dat (Git: decaf00 -> LFS: decaf00)

this is an indication that the file a.dat is unmodified, but was committed as a large file (indicated by the Git source on the left side of the blob entry meta information).

To fix this, you can rewrite history by using the git-lfs-migrate(1) 'import' subcommand, as described in the manpages.

[TTimo]

I can't say if 'git lfs migrate import' did anything. Guess I'll wait and see if this happens again. Running the command did throw all file extensions in my repository (even .h .cpp) into LFS .. so beware ..

[ttaylorr]

Running the command did throw all file extensions in my repository (even .h .cpp) into LFS .. so beware ..

Make sure that you have --include or --exclude set to determine what goes into LFS. If the migration touched something you didn't want it to, it creates a reflog entry so you can git reset --hard {SHA} at a place before the migration took place.

Please feel free to open a new issue if you have any comments/questions/feedback.

[bengaiser]

I now have modified source files that show under "Git LFS objects not staged for commit" but there is no reflog entry for any lfs actions to revert...

Also, none of these source files show up in the git attributes file either.

Edit: happens in a clean repo too... probably nothing.

[AndrewJDR]

I don't understand why this was closed :(

[mrkita]

This is a real problem for my team as well at the moment. This really shouldn't be closed, because it's to the point that I will have to turn off GIT LFS support.

We had some ZIP files in our Code-Repo, which I started tracking with GIT LFS. It converted them and added them into the history as it should.

However, now randomly like described above, when doing a reset to an older state, or even merging into branches from an updated master branch (master post conversion), causes this problem. They can not be reset, except by a
"git lfs uninstall"
"git reset --hard"
"git lfs install".

This however DOES NOT check them out again as proper objects and instead just leaves the pointers on the filesystem. Then doing a "git checkout" to another branch magically copies in the real files.

Please don't close this issue, because it's really blocking for people who have it.

[TTimo]

@mrkita - have you tried @ttaylorr's proposed solution? Using git-lfs-migrate implies rewriting your repository history, which means once you push the change, other teammates may have to do a fresh clone, so it's generally pretty nasty, but that may address the issue.

[amacara1]

My problem also persists, and there are cases in which "expecting pointers, but weren't" does NOT go away in any way known to me - we basically get a fresh clone then, which is not best at all :(
Before I even consider rewriting history, if possible, I would like to understand the root cause of this fix and whether this applies to my case. Would it be possible to convey some dumps/info in order to debug 'my' situation?

Git lfs status does show same ID: (Git: d1863f1 -> File: d1863f1)

[technoweenie]

I would like to understand the root cause of this fix and whether this applies to my case.

That message just means a file matching a filter=lfs pattern in .gitattributes is not an LFS pointer. A very common cause of this is that the pattern is added to a repository with existing files matching that pattern.

In most cases, this isn't really a problem. It does present an opportunity to migrate your repository to LFS, shrinking your Git repository size. Though, this shouldn't be required to use LFS. I think the message should instead be written to GIT_TRACE=1, or some other hidden way, so users aren't constantly seeing noise about it.

The only time it IS a problem is if something wrote an invalid LFS pointer that Git LFS can't parse. Hopefully this is pretty rare -- I personally have not seen any reports of this happening.

[technoweenie]

@mrkita:

$ git lfs uninstall
$ git reset --hard
$ git lfs install

This however DOES NOT check them out again as proper objects and instead just leaves the pointers on the filesystem.

That's because you turned Git LFS off. You can check out the proper objects after that with git lfs pull.

[amacara1]

Please let me know if I should stop writing these, here. I'm trying to present my current issue.
$ git status
Your branch is up-to-date
Changes not staged from commit:
modified: a.docx

$ git checkout -- a.docx
Encountered 1 file(s) that should have been pointers, but weren't:
a.docx

$ git --version
git version 2.13.0.windows.1

$ git lfs version
git-lfs/2.2.1 (GitHub; windows amd64; go 1.8.3; git 621d1f82)

$ cat .gitattributes
*.docx filter=lfs diff=lfs merge=lfs -text

$ git log --follow a.docx
!!one single entry, with date older than enabling LFS
As weird as it sounds, this cannot be true. On another clone, history looks different.

[amacara1]

And here is the failed attempt for reset:
$ git lfs uninstall
Hooks for this repository have been removed.
Global Git LFS configuration has been removed.

$ git reset --hard
Encountered 1 file(s) that should have been pointers, but weren't:
a.docx
HEAD is now at ...

$ git lfs install
Updated git hooks.
Git LFS initialized.

[mrkita]

Unfortunately a history rewrite is not really acceptable in our case, or at least would not be worth it.

Next time it happens I will try to do a git lfs pull to see if it correctly resolves the point after the uninstall,reset,install.

That still is only a poor workaround, and honestly I too would like to know what is causing this. We don't at the moment have a large number of these files. As an alternative, would it be a valid fix to delete all of these files from git, and then readd them directly as lfs_objects, or would that likely still cause problems when resetting?

If not then I think I would have to turn off git lfs for this repo, which would be very disappointing, as it has helped a lot for other repos.

EDIT:
I tried adding a "git lfs pull" after reinstalling the hooks, and it sets the points to the right binaries, but they are still "modified". In fact they are automatically added to the stage for some reason. A reset kicks them back to modified off the stage, but I am in the same circle.

So changing branches after uninstalling, and reinstalling, sometimes sets them randomly to the right files without marking them as modified in the tree.

[technoweenie]

When you track *.zip in your .gitattributes file, LFS will eagerly convert any zip files in Git to LFS, leaving them modified.

It seems like there is a legit issue with symlinks in Git for Windows (https://github.com/git-lfs/git-lfs/pull/1749). If not that, then available fixes are:

1. Confirm the modification via git lfs status as suggested in https://github.com/git-lfs/git-lfs/issues/1726#issuecomment-314505357 and commit the change.
2. Rewrite your history with git lfs migrate, BFG, etc.

If you think you're having a different problem, it'd really help to have read access to a repository exhibiting the behavior so we can investigate. I realize it's a big request, but the alternative is a lot of back and forth through an Issue. We can do that too, but it'll likely take much longer to resolve, as I don't even know what to look for yet. If you're interested in that, please open a new issue so that everyone on this thread isn't included in it. Thanks.

[mrkita]

Thank you for the reply.

Having read that discussion, I'm still confused. The suggestion is that symlinks do not work in relation to git lfs objects, or that it must be turned on in the git config for them to work in windows?

Rewriting our history is not an option, but next time it pops up I will try committing them (They do in fact show up with git lfs status as the same exact sha.)

Sadly I can not give you access to our repository, it's corporate.

I will open a new issue when I get a chance, but if you could please clarify the point about symlinks?

[awmcclain]

Having a helpful suggestion such as Perhaps these files were already checked into git before they were added to LFS. Run 'git lfs migrate import --include *.XXX' to rewrite your git history. For more information, see man git-lfs-migrate on the error message would go a long way to fixing a pretty common LFS pattern.

[ttaylorr]

@awmcclain some of the improvements I posted in: https://github.com/git-lfs/git-lfs/issues/1726#issuecomment-314505357 may help, since they can indicate if files need to be migrated.

Adding an explicit Run 'git lfs migrate import ...' suggestion seems like a good idea as well. I don't have anything like this on my roadmap, but would gladly help a contributor through a pull request.

[awmcclain]

@ttaylorr Sounds great! I'll look into that next week and see if I can use some of the meat from there to determine whether to suggest it or not.

[amacara1]

Problem (expecting pointers, but weren't) hit again, same old repository. Running git lfs fsck results in:
rename .git\lfs\objects\47\ C:\NXPrsdk_clean.git\lfsbad\: The system cannot find the path specified. Of course, lfs uninstall, reset, install and pull does NOT solve the problem,
A fresh clone does NOT expose the issue. Can I debug this somehow?
git version 2.13.0.windows.1
git-lfs/2.2.1 (GitHub; windows amd64; go 1.8.3; git 621d1f82)

[ttaylorr]

Problem (expecting pointers, but weren't) hit again, same old repository.
Running git lfs fsck results in:

This sounds like a similar issue to the "large files committed in place of files
that should be tracked with Git LFS" as above.

rename .git\lfs\objects\47\ C:\NXPrsdk_clean.git\lfsbad\: The system cannot find the path specified.
Can I debug this somehow?

I'm curious what's causing this. Is your repository open-source? I'd love to
take a look.

[amacara1]

@ttaylorr , the repo is not open source :(
You seem to be right, someone added the file OUTsite of lfs. But I am confused:

• how come I was able to checkout the file in the first place?
• what can I do now to continue my work in that clone (where I get the 'expecting pointers, but weren't' message)?

Thank you!

[ttaylorr]

• how come I was able to checkout the file in the first place?

LFS will treat "malformed pointers" (usually large files that should have been committed with LFS but weren't, as in your case) as just that, and copy their contents from the index into your working tree, warning you along the way. In other words, LFS will try its best to degrade gracefully, but it isn't a perfect solution.

what can I do now to continue my work in that clone (where I get the 'expecting pointers, but weren't' message)?

You can run the git lfs migrate import command to find and fix those malformed pointers in the affected range of commits. For example:

git lfs migrate import --include="your-file-name" --include-ref="your-branch"

[rdstevens]

I'm having the malformed pointers problem too.

git lfs migrate import isn't fixing it:

git lfs migrate import
migrate: Sorting commits: ..., done
migrate: Rewriting commits: NaN% (0/0), done
migrate: Updating refs: ..., done

git status still shows modified files.

[ttaylorr]

git lfs migrate import isn't fixing it:

@rdstevens it looks like there are no commits in range of your migration. You may have better luck specifying an explicit reference (i.e., git lfs migrate import --include-ref=...) or with the --everything option releasing in v2.3.0 later this week.

Since this is a fairly old thread, I am going to lock this conversation and recommend that people who are encountering this issue open a new issue instead.