Github: Disable locking (observe only by default)

[Anillusion]

I prefer doing my commits and pushes from the terminal, because for instance I can use my template for commits in terminal, where the plugin just adds simple commit messages. But when the plugin is running, even if I only use it to observe, or not visible at all, it creates a .lock file in the .git folder which prevents me from doing anything in the commandline. When I remove the lockfile, it gets recreated by the plugin. Since Atom now installs this plugin by default this is a problem. Simple solution for now was to disable the plugin completely, because closing atom for a commit doesn't work for me. A good solution would be to be able to observe by default and only when you want to use its staging, committing, pushing, whatever feature that requires that lockfile, that you actually need to tell the plugin you want to use it. I don't want to be forecd to soly use this plugin. I don't mind it being installed and running in the background and use it for simple commits.

[BinaryMuse]

@Anillusion What file is getting created in the .git folder? Is it index.lock?

[Anillusion]

Had to look it up in my terminal history but yes, index.lock.
I'm sorry if I offended anybody for not using the template, I'm still quite new at github commenting. Also, I'm in the middle of a huge development project, this is why I turned off the plugin for now and didn't provide steps to reproduce. I wanted to get rid of the issue and move on with my work, but also let you know about what I experienced. Thanks :)

[BinaryMuse]

Okay. This file is placed there by Git when it runs status, so it's actually the observing (and not the writing!) that's creating the lock file! That said, there are a few things that are interesting here:

1. The index.lock file should be deleted by Git once the status call finishes. How long does it take for git status to run in the repository you saw this behavior in? (If you're on a macOS/*nix system, the last line of time git status could be helpful)
2. We actually heavily throttle status calls (by several seconds) when the app loses focus, so the app stays up to date in the background, but there is a delay of several seconds. It's possible that Git operations are taking long enough that our timer isn't long enough.

I'm curious about the following:

• What OS are you running?
• If you switch to the terminal and wait for a bit, does the index.lock file eventually go away?
• Are there processes running in the background which might be automatically touching files (e.g. a server writing to a log file that's tracked by Git, or a linter doing stuff automatically) that may be triggering filesystem change events?
• How long does status take to run in this repo?

[Anillusion]

That might explain it then.
• I'm using a mac (OSX Sierra, 10.12.5) as my workstation, but I'm connected through samba (v4.3.11-Ubuntu) to my testserver, which is an older machine that runs Ubuntu Server 16.04.02 LTS.
• I will test it again during my commits the coming days. My experience was that doing the git status on the terminal was fast. The index.lock file didn't disappear after a while, I had to manually delete it.
• I do have linter plugins installed, I usually close the files I'm done with before committing though, but I will keep an eye out for that.

Thanks again, I will get back to you if I have more information.

[viddo]

Just wanted to chime in that I observe the same issue.
Disabled this plugin for the time being which seems to make the err go away.

I'm also using mac (OSX Sierra, 10.12.5).

If you switch to the terminal and wait for a bit, does the index.lock file eventually go away?

Yes, it appears to happen "randomly". Usually the lock file is removed after a few seconds.

Are there processes running in the background which might be automatically touching files (e.g. a server writing to a log file that's tracked by Git, or a linter doing stuff automatically) that may be triggering filesystem change events?

Not that I'm aware of. Using some linter plugins in Atom (linter, linter-ui-default).
Any recommended way of checking this?

How long does status take to run in this repo?

My git repo I work in is fairly big ("monorepo", ~4GB), but the git commands are usually fast enough:

time git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean

real    0m1.064s
user    0m0.248s
sys 0m3.946s

[adampope]

I'm experiencing this issue when using atom as my git editor and performing a git rebase -i. The rebase todo opens in a new atom window, I make my changes and close the window to perform the rebase, atom then returns to my project window. However, the rebase fails with the error:

fatal: Unable to create '/Users/adampope/project/.git/index.lock': File exists.

I'm guessing that when atom switches back to my project window it tries to get the current git status, whilst the rebase is simultaneously happening in my terminal, creating the lock conflict. If I set my git editor to vi then the rebase succeeds.

My repo isn't very big ~100Mb. git status takes 0.05s.

OS X 10.11.6, atom 1.18

[agarciamontoro]

In case it helps: I'm experiencing the exact same issue than @adampope, but I'm using emacs as the git editor and the error persists.

I'm on Arch Linux, Atom 1.18.0 x64, git status takes 0.745s.

[UziTech]

I am running into this issue when I use context-git with the github package enabled.

It seems like there is a race condition between the two packages.

[UziTech]

It seems like this is affecting other git packages.

When another git package runs git add then git commit the github packages detects the status change after git add and it becomes a race to see whether github can run a git status command or the other git package can run git commit

The GitRepository class in atom seems to get around this issue but I'm not sure exactly how.

Could we add a setting to delay status checking more?

[719media]

I'm running into this problem as well and I use virtualbox mapped to a drive on my mac. Basically, over time, the lock file ends up being created and never removed, and I'm not sure why. So occasionally I will have to manually delete the lockfile, or quit Atom, so be able to commit or run git commands from the command line. To answer the above questions from @BinaryMuse

*Running Atom on OS Sierra, mapped drive to virtualbox CentOS over samba
*Often, the index lock will go away within a second or two when I run into this problem. Sometimes, it will never go way and I have to quit Atom, and then manually remove the lockfile myself as a last resort.
*No processes in the background should be doing anything with git, other than Atom and my manually running of commands
*status takes 0.024s

[joshribakoff]

I'm having this issue as well. Particularly bad when it happens during an interactive rebase, for example. It messes up the whole rebase history & can mess up your commits. Seems like this should be an upstream feature request to git? Git should be changed to allow running multiple concurrent "git status" (one from Atom, and one from me on the CLI)

[sladex]

Running into the same issue (never had this problem before github plugin were released). I prefer not to disable the plugin completely because of some of some useful features.

We actually heavily throttle status calls (by several seconds) when the app loses focus, so the app stays up to date in the background, but there is a delay of several seconds.

Might help an option which disables status calls completely when the app loses focus. This way when I am in terminal doing some crazy rebase it won't be disturbed by Atom's status calls. At least probability would be way lower.

[Ehekatl]

Thanks to this awesome logic, I lost my commits when rebase, the second time 😃 😃 😃
I don't need git in atom at all.

[50Wliu]

@Ehekatl Thanks for the feedback! If you find Git integration unhelpful, you can always disable it by disabling the github package.

[viddo]

@Ehekatl you can also always git rebase --abort to get back to prior state in case something goes awry during the rebase. Alternatively git reflog could also help recovering commits, see https://git-scm.com/book/en/v2/Git-Internals-Maintenance-and-Data-Recovery

[Ehekatl]

@50Wliu @viddo thanks, git reflog does help. In my case, when I do rebase it says index.lock is already exist, and if I keep retry, it will works sometime, but it may failed while it's rebasing, get permission denied error when create files, and rebase abort. After that, if I do git rebase --continue it succeed but those files just went missing. The missing file issue happen rarely, but index.lock happens all the time . I'm on windows 10.

[paulleader]

Just adding that I'm having the same experience as @adampope. Having github enabled in Atom makes doing interactive rebases a very hit and miss process. If I just do one or possibly two changes I will probably be ok, but any attempt to do more than that, e.g. squashing a bunch of different commits, will result in a lock collision and I'll have to bail on the rebase.

[the-j0k3r]

Ill join this discussion with a "Me too" type comment. after posting https://github.com/atom/github/issues/762#issuecomment-328839686 I was pointed here.

I do my interactive rebases via terminal, but 9 out of 10 times, it fails claiming there is an ongoing operation or some other times it claims that the previous process may have crashed.
The only resolution is to restart atom.

Clearly not the goto solution in the middle of everything.

After reading the comments here, Issue happens when.

[x] squashing commits.
[x] interactive rebase

I love atom and I love that Im able to dicth umpteen other programs for one.

Would be nice to have a solution and will try git reflog next time and report back

[the-j0k3r]

What a mess indeed.

I tried the git reflog and it was a disaster, when I was doing the squash, suddenly the text from the reflog appeared on terminal and could no longer see what the hell was going on...

Ended up in a detached head state for blindingly trying to fix it had to close atom due to making things more problematic. Easy fix but what the heck 😅

[leepowelldev]

Same issue as others have stated.

[BinaryMuse]

Just to provide a quick status update here, since we talk a LOT about this issue internally but I haven't said much publicly:

This is really tricky to solve properly at the application level. As I mentioned earlier, the thing that's causing the locking is simply running git status in order to get the current list of files that have changed and/or have been staged. Trying to determine when to skip this call due to the desire to work with the command line is nearly impossible in all cases — and if we guess wrong, we either (1) cause lock contention, or (2) leave the Git UI in an incorrect (and unusable) state. So we've been brainstorming other ways to solve this.

One promising solution exists in Git for Windows: a --no-lock-index flag that prevents locking the index. We're working on getting this ported to the other distributions of Git. If it is, we can make status calls without locking the index at all.

Apologies to everyone having this issue, and thanks for your patience.

[the-j0k3r]

@BinaryMuse, thanks for update,

Im using Windows and Git for windows is what I have installed, yet this problem exists.

Or do I misunderstand something?

[BinaryMuse]

@the-j0k3r The package bundles its own version of Git for various reasons, so this change (if it lands) will need to make its way into our Git distribution.

[the-j0k3r]

@BinaryMuse something else is wrong then (at least here), I had to install Git for Windows else cant make the any of the terminals packages for Atom detect any Git program installed and then spew errors to that effect.

Thx for the explanation, but now I have to find out why the built in Git dont work for me in terminals.

[Arcanemagus]

@the-j0k3r The built in git that @BinaryMuse is talking about is internal to the github package _only_. It's not available on your system, which is what would be required to work in the terminal packages.

[the-j0k3r]

Well isnt that simple confusing? Why would then the git for windows trip on this bug when Im doing things in terminal only, thus using Git For Windows directly?

OK, Im definitely confused.

[BinaryMuse]

@the-j0k3r The built-in Git is only meant to be used internally by the GitHub package. It does not provide you with a command-line GIt. We include it so that (1) the GitHub package works even if the user does not have Git installed, and (2) so that we can carefully control the version of Git that is bundled (so we can correctly predict supported flags, output formats, error codes, etc.)

As for your second question, the "bug" is that the version of Git embedded in the GitHub package is taking the index lockfile when it runs status. Then, when you try to run certain operations on the terminal, you're unable to because the GitHub package has locked the index. We try to avoid running Git commands when Atom isn't focused, but this strategy isn't foolproof in all cases. So it's not your Git for Windows exhibiting the bug, it's the Git inside the GitHub package — which is why the change needs to make its way to the other distributions of Git so we can include it in the package.

[the-j0k3r]

@BinaryMuse understood, thanks for explanation.

After a hand full altercations every day with Atom, Im starting to understand how complex it is to keep it behaving in a manner you can predict given the volume of components involved, At least you have identified the problem,

For me its starting to make me go back to using previous tools just because elementary stuff like this is long been resolved. Shame, I really love Atom and friends.

[UziTech]

@the-j0k3r I feel like it's not so much that old IDEs have solved this problem but that old IDEs just don't have this functionality. It is a pain to get things working right, but at least Atom is trying.

[GreenAsJade]

I'm having this problem with sourcetree and atom running at the same time.

I make a change in Atom, then go to sourcetree to commit and it fails due to index.lock.

Strange thing is I'm coming back to this dev environment after a break of some months - it never used to happen like this before.

Note that I'm reluctant to disable the git feature in Atom, because knowing which files are changed is very valuable, but sourcetree is definitely better for looking at the repo state overall.

[UziTech]

@GreenAsJade disabling the github package won't disable all the git features just the github dock pane.

[the-j0k3r]

@UziTech git (both Linux and Windows) works just fine fine for years and years( where this is concerned) , Even the old GitHub for Desktop App heck even the new, take a pick there's no shortage of tools using Git that dont stumble on this bug, ever.

[UziTech]

@the-j0k3r I was referring to Atom making an easily hackable editor. No other text editor makes it as easy to create an IDE from it.

This git issue has a very simple fix right now, disable the github package, since the issue only presents itself when using git outside of the github package this shouldn't be a problem for most people.

If you still want to use the guthub package you can fork this repo, fix the issue in your own way, and use your fork instead of the core package.

I can't think of any other editors that allow you to change core functionality. Yes that does mean more things will go wrong, which is why other editors don't allow it, but again, at least Atom is trying.

[the-j0k3r]

@UziTech You confuse me even further, at this moment your latest interjection is just really most confusing the actual issue from what I was explained and understood from https://github.com/atom/github/issues/961#issuecomment-331497436

So thank you for trying to help anyway, please refer back to https://github.com/atom/github/issues/961#issuecomment-331502909 for my only comment regarding your latest reply..

Ill wait for more positive progress in regards to this bug report and look forward for a expert solution, since Im not.

[BinaryMuse]

The proposed change to add an option to Git to disable locking has landed in Git as https://github.com/git/git/commit/eaffe9c638defbbf0728cbd704a2bff610fc265e and will make its way to master. Once it's been released in an actual version of Git, we can update dugite-native and solve this issue in a complete and robust way.

We'll continue to think of ways we may be able to further mitigate the issues caused by locking the index, but the new option to Git will be the final solution.

[srkaeppler]

I have also noticed this bug, which was rather annoying at first. I ended up disabling all of the internal github setting and that seemed to fix things. I like controlling git over command line - just my preference.

[BinaryMuse]

The referenced fix is in version 0.8.0 of this package, which is slated to go out with the next Atom beta (Atom 1.23).

[the-j0k3r]