Vscode: Bug: Code + Git = deleted files

[rogierschouten]

Several times now I've had mysteriously deleted files when doing a git pull in a console while Visual Studio Code is running. I've seen this roughly since version 0.10.

It seems that when auto-merging, Git is able to delete a file but not able to write the merged file back if it's open in Code. However that is just a hypothesis.

Seen the issue last with git version 1.9.5.msysgit.1 and Code 0.10.2.

[alexdima]

@rogierschouten What OS are you on? [I guess Windows from the msysgit reference]
@joaomoreno fyi

(This might be the file watcher?)

[rogierschouten]

I'm on Windows 7 Professional SP1

[joaomoreno]

It definitely sounds like what we were hitting a lot before: running status during a rebase in Windows would simply cause rebase to fail.

@rogierschouten Please update to the latest git (2.6.3) and let us know if you continue hitting the issue. Closing this for now.

[rogierschouten]

Will do. Thanks for the quick response guys!

[rogierschouten]

@joaomoreno please re-open this issue - I've just experienced the problem with the latest git version 2.6.3.windows.1

[joaomoreno]

What do you get with git config pull.rebase?

When it happens please send me the git output log (open the git view in Code, press the ... in the top right corner, Show Git Output), a screenshot and a list of the names of the files that are missing but should actually be there.

[rogierschouten]

Note that I wasn't using Code's built-in git functionality: I used git pull --rebase from a separate console window.

Could it be that it is indeed that Code is running git status while I'm doing the git pull --rebase independently? The link you posted above seems to suggest that Code should not use git status:

Do not run git status from this application: it writes the index. Use 
git diff-files to detect changed files. See require_clean_work_tree in 
git-sh-setup for an example. 

-- Hannes 

[rogierschouten]

Now I get permission denied errors from Git - I don't think code should be blocking Git from doing anything?

(note that git up is a synonym for git pull --rebase)

$ git stash && git up && git stash pop
Saved working directory and index state WIP on master: 2811dac Issue #6272: webapp: bugfix after enum rewrite.
error: unable to create file lib/server/dbal/dbal-base.ts (Permission denied)
fatal: Could not reset index file to revision 'HEAD'.
lib/server/dbal/dbal-base.ts: Permission denied
lib/server/dbal/dbal-base.ts: Permission denied
First, rewinding head to replay your work on top of it...
Applying: Issue #6272: webapp: bugfix after enum rewrite.
Removing lib/server/dbal/history.ts
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   lib/server/dbal/dbal-base.ts
        deleted:    lib/server/dbal/history.ts

no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (90803eefd2da3da61892b8c7d1e2ae898f3fd92d)

[rogierschouten]

For now I've disabled Git in the user settings, let's confirm that it goes away then.

"git.enabled": false,

[joaomoreno]

Good idea. Let me know how that goes. Meanwhile, I'm looking into alternative ways to poke around the git status without side effects.

[joaomoreno]

Here are my latest findings:

We could avoid calling git status since that writes to the index. There are a few commands which would give us the same results:

• git ls-files --others --exclude-standard, for untracked files
• git diff-index HEAD, for the changes between the working tree and HEAD
• git diff-index --cached HEAD, for the changes between the index and HEAD

The reason why git status writes to the index, though, is very important: if you touch a file without modifying its contents, git status will hash the file, cache it in the index and not report it as modified.

git diff-index HEAD will incorrectly report it as modified since it doesn't hash the file but merely looks at the last modified timestamp. The consequence would be that _touched_ files would appear in the git viewlet, unless VSCode does the hashing itself or calls git to do it. This can get very expensive since on every git view refresh these hashes would be recomputed... because they don't get cached in the git index.

It's a pickle.

[rogierschouten]

How do other IDEs do it? Is there an efficient way to determine a subset of files to hash (e.g. with their modified timestamps)?

[joaomoreno]

Well, that subset is exactly what you get from git diff-index HEAD. One solution would be to simply implement the hashing only on Windows... I'm not super happy about it though.

[rogierschouten]

Wouldn't the number of files to hash be proportional to what the user touched (typically manually)? Would it be that much of a performance issue? Over here everybody has by himself walked into this issue already so I don't think the current way is the way to go.

[joaomoreno]

Sure, I'm not saying we should keep it this way :+1: Let me sketch up something.

[martinwebrant]

Any progress on ".git/index.lock': File exists."?
It's a mess to fix a large rebase that fails when you forget to close vscode.

[mrsnl]

Just sneaking by saying that vscode 0.10.11 (on OS X Yosemite) stages all files for deletion while you least expect it. While browsing through todays diffs the counter suddenly stepped up from 22 modfied files to 99.

Checked console and $ git status showed all files staged for deletion. Happened at least 5 times now. Luckily $ git reset HEAD * solves the problem.

Files are sshfs mounted from a ubuntu linux (webserver).
vscode are not used for git actions besides that amazing diff.

[ricardobeat]

Just had a rebase blow up with fatal: Unable to create '.git/index.lock': File exists. with VSCode open. Had to recover one of the commits from reflog. I tested again a few times and the rebase fails consistenly unless I close the application.

Also on OSX Yosemite.

[ricardobeat]

@joaomoreno I don't see git diff-index HEAD reporting a touched file as modified. That would be a bug in git itself, no?

[joaomoreno]

@ricardobeat is your repo open source by any chance? Since you can repro consistently, I would love to give it a try. Which git are you on?

[ricardobeat]

@joaomoreno not open-source unfortunately. The repo has >50k files, git v2.7.1 on OSX.

[timwieman]

As another data point, just yesterday I had Visual Studio 2015 Update 1 stage 3 files for deletion. Git version 2.7.2. Windows 10 Version 1511 (OS Build 10.586.218). I had modified and committed 3 files in git locally, and I did a pull (within VS 2015's Team Explorer window) that also had the 3 files modified. Instead of merging them, it deleted them from the local file system. I DID have VS Code open at the same time, but I don't know if that is related. "git status" locally had this output:

$ git status
On branch cross-cloud
Your branch and 'origin/cross-cloud' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        deleted:    shared/db_common.sh
        deleted:    shared/vpn_common.sh
        deleted:    shared/vpn_final_updates.sh

no changes added to commit (use "git add" and/or "git commit -a")

I used "git checkout -- " from the command line to fix this immediate issue. Then doing a Pull from within VS 2015 did the exact same thing again. I actually used VS Code to fix this by doing a pull in VS Code, which used its "merge" functionality to allow me to merge my commit changes with the remote commit changes.
BTW, I had VS 2013 do this same thing to me once long before VS Code was around.

[joaomoreno]

Let's:

1. Disable auto status when application is not in foreground and,
2. Provide setting (or checkbox in viewlet) to disable auto status altogether.

[joaomoreno]

Verifier, read previous comment.

[didacrios]

If it helps, I also experienced problems with git and VSC 1.7 in Windows 7 32 bit:

1. I opened the cmd in VSC
2. git checkout . In order to discard some changes
3. The cmd output errors regarding saying __permission denied__ with the files modified, the same with modified fields that were in the git index with --assume-unchanged
4. The files appear as deleted in git, besides the file is shown in the explorer, but without content, and I cannot back to previous commits nor checkout to other branches. Always showing problems of permission denied.

I think I will remove the git functions from VSC, also had problems with git clean

Hope it helps

[Masatoshi]

I am experiencing too. With VCS 1.9.1 and git 2.10.1 (Apple Git-78) on Mac OS X 10.12.3
And more, I am suspicious of VCS overwriting files as another existing files.

[01binary]

There seems to be a separate issue that's caused by this issue.

When VSCode runs certain git operations while files are locked, it corrupts the cached index because GIT has to convert line endings from CRLF to LF when it's storing the files in its database. If the file was locked, I guess the hash isn't updated properly and then GIT thinks the file is different even if I delete the file and restore it, check it out, force check it out, etc. GIT diff reveals that it thinks the whole file is modified because every line has CR in GIT's database, but CRLF when checked out (i.e. file locking causes de-sync). If you then try git diff with ignore whitespace, it says the file isn't different, thus proving that the de-sync with whitespace transform is what's causing the issue.

TL;DR

git rm --cached -r .
git reset --hard

...will fix the de-sync issue with line endings if that's what you are experiencing due to the file locking bug. This is easier than re-cloning.

Warning: as you can tell by --hard, make sure you don't have any changes you don't want to loose. If you do, stage them and commit, excluding any 'fake' changes to be fixed.

[martinlindhe]

I came from google looking to resolve this issue. There is also a setting to have vscode stop polling the current repository (i see it does it every 120 second).

in settings.json:

"git.autofetch": false,

[fredizzimo]

It doesn't seem like the fix works, I got index file lock problem multiple times when trying to do a rebase with Visual Studio Code open.

I didn't use the integrated terminal, but a normal git bash terminal window. Visual Studio Code was not focused, but maximized behind the terminal window. I didn't try to minimize it, so I can't tell if that would have helped.

This is on Windows 10, with Creators Update.

[fredizzimo]

I also just noticed that the the blue "clock" on top of the version control icon flashes when I focus the terminal window, and run some long git command, like git status. This also happens within the integrated terminal.

I assume that this means that Git is busy and vscode tries to do some operation. So it means that it's trying to do stuff while not focused. It also happens very frequently, which seems to indicate that vscode also tries to access Git frequently.

[joelhock]

I agree with @fredizzimo

i'm using vscode 1.13.1 on mac osx 10.12.4 and brew git 2.12.2.

i can put vscode in the background, edit a file in the project in emacs in a separate terminal, and when i save in emacs, i see vscode adding the clock overlay to the git icon, and it updates the git status. this doesn't happen if i use "git.autorefresh": false

the background git operations cause the git index file lock problem that started this whole thread. Perhaps the protections added in faeb58e were lost when the git code was refactored into an extension?

[alexjeffburke]

Just to chime in and say I've started noticing this on my mac over the last couple of days but I don't remember this being a problem before - commits through git gui have failed. On the plus side those can be retried, unfortunately I also had it happen during an interactive rebase where it broke applying the commits half way and git would not continue.

vscode 1.13.1 macOS 10.12.5 brew git 2.13.1

[benesch]

Yep, this is happening to me too on macOS. @joaomoreno, would you mind re-opening this issue?

[oskopek]

I can confirm, still happens. vscode 1.14.1, git 2.14.0.

[awinogradov]

+1

Uncomfortable thing in the middle of rebase

[caesay]

Why is this closed? This is still an issue. I have git.enabled: false, git.autorefresh: false and git.autofetch: false, and if I try to do any git operations from another terminal while vs code is open it fails due to locked files - and then when I close vs code, those locked files get immediately deleted. It's caused me to lose changes several times.

[lukephills]

This is happening for me in version 1.16.0

[joelhock]

@lukephills threads about this issue were consolidated under https://github.com/Microsoft/vscode/issues/11918
fix is committed in 475fb0e
i believe the fix will be included in 1.17

[hvdklauw]

Still happening in 1.17.. big merge, found the conflicts using VScode (they are nice at the top). Fixed them and switched to my terminal to add and commit..

Kept getting the error because VScode was still updating in the background, even when not looking at the GIT Activity sidebar.

[joelhock]

1.17 is correctly not updating status when vscode is in the background for me.

are you using any other extensions? eamodio.gitlens seemed to be causing a similar issue for me at one point.

[hvdklauw]

Ahh, yes.. that might be the culprit

[joelhock]

gotcha; that's affected a few people in our group also.

i opened an issue on GitLens to address this problem
https://github.com/eamodio/vscode-gitlens/issues/163

and you can workaround most of GitLens locking by setting gitlens.gitExplorer.includeWorkingTree": false