Stack: Failed to reset to commit, deleting and re-cloning.

[mdibaiee]

Hey,

I'm using a specific commit of hmatrix, and I randomly get this error when trying to build:

Failed to reset to commit 42a88fbcb6bd1d2c4dc18fae5e962bd34fb316a1, deleting and re-cloning.

I _guess_ it might be caused in new versions as I was using the same stack.yaml file on my macOS before, and it worked perfectly.

stack.yaml

flags: {}
extra-package-dbs: []
packages:
- ../sibe
- location:
    git: [email protected]:albertoruiz/hmatrix.git
    commit: 42a88fbcb6bd1d2c4dc18fae5e962bd34fb316a1
  subdirs:
  - packages/base
- '.'
extra-deps:
- Chart-1.8
- Chart-cairo-1.8
- ansi-wl-pprint-0.6.7.3
- stemmer-0.5.2
- unix-2.7.1.0
- ansi-terminal-0.6.2.3
- ncurses-0.2.16
- random-1.1

resolver: lts-6.13

Verbose output

$ stack build --verbose
Version 1.1.3, Git revision 63f203d7886801c9ec3c303db0c459f08687a3c8 x86_64 hpack-0.14.0
2016-09-03 14:54:26.853372: [debug] Checking for project config at: /home/mahdi/documents/workshop/flappy-rl/stack.yaml
@(Stack/Config.hs:828:9)
2016-09-03 14:54:26.853570: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:846:13)
2016-09-03 14:54:26.854866: [debug] Checking whether stack was built with libgmp4
@(Stack/Config.hs:332:5)
2016-09-03 14:54:26.855067: [debug] Run process: /usr/bin/ldd /home/mahdi/.local/bin/stack
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.863476: [debug] Process finished in 8 ms: /usr/bin/ldd /home/mahdi/.local/bin/stack
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.863671: [debug] Stack was not built with libgmp4
@(Stack/Config.hs:336:14)
2016-09-03 14:54:26.863759: [debug] Trying to decode /home/mahdi/.stack/build-plan-cache/x86_64-linux/lts-6.13.cache
@(Data/Store/VersionTagged.hs:68:5)
2016-09-03 14:54:26.871585: [debug] Success decoding /home/mahdi/.stack/build-plan-cache/x86_64-linux/lts-6.13.cache
@(Data/Store/VersionTagged.hs:72:13)
2016-09-03 14:54:26.874927: [debug] Getting system compiler version
@(Stack/Setup.hs:350:17)
2016-09-03 14:54:26.875076: [debug] Run process: /usr/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.914747: [debug] Process finished in 39 ms: /usr/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.916594: [debug] Getting global package database location
@(Stack/GhcPkg.hs:54:5)
2016-09-03 14:54:26.916718: [debug] Run process: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.917029: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:101:13)
2016-09-03 14:54:26.917148: [debug] Run process: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.917805: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:171:5)
2016-09-03 14:54:26.917946: [debug] Run process: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.938871: [debug] Process finished in 22 ms: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.939705: [debug] Process finished in 21 ms: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.940009: [debug] Process finished in 22 ms: /home/mahdi/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-09-03 14:54:26.940088: [debug] Resolving package entries
@(Stack/Setup.hs:232:5)
2016-09-03 14:54:26.940614: [debug] Creating process: /usr/bin/git clone --recursive [email protected]:albertoruiz/hmatrix.git /home/mahdi/documents/workshop/flappy-rl/.stack-work/downloaded/Uc3OLtiS5s09
@(System/Process/Run.hs:103:5)
Cloning into '/home/mahdi/documents/workshop/flappy-rl/.stack-work/downloaded/Uc3OLtiS5s09'...
Enter passphrase for key '/home/mahdi/.ssh/id_rsa': 
remote: Counting objects: 9586, done.
remote: Total 9586 (delta 0), reused 0 (delta 0), pack-reused 9586
Receiving objects: 100% (9586/9586), 2.86 MiB | 117.00 KiB/s, done.
fatal: unable to open /home/mahdi/documents/workshop/flappy-rl/.stack-work/downloaded/Uc3OLtiS5s09/.git/objects/pack/tmp_pack_uaQiyn: No such file or directory
fatal: index-pack failed
Process exited with ExitFailure 128: /usr/bin/git clone --recursive [email protected]:albertoruiz/hmatrix.git /home/mahdi/documents/workshop/flappy-rl/.stack-work/downloaded/Uc3OLtiS5s09

The last error after re-cloning is also random, sometimes re-cloning works, sometimes errors.

Stack version

$ stack --version
Version 1.1.3, Git revision 63f203d7886801c9ec3c303db0c459f08687a3c8 x86_64 hpack-0.14.0

Method of installation

pacman -S stack and stack upgrade

[Blaisorblade]

The underlying failure happens in git. This might be due to stack invoking git in an unsupported scenario (not sure which it could be), or to a git misconfiguration/bug/.... Do you have a change to investigate?

Questions:

• does the git clone command work reliably if _you_ launch it?
• are you using a stable git version?
• does the target directory exist when git is launched?

[mdibaiee]

@Blaisorblade: I'll try the steps once I get the issue again (I don't know the steps to reproduce, I get them randomly) :+1:

[jonascarpay]

I am getting this error fairly consistently (almost every build) with the following yaml (and other projects with extra deps from git, but not as consistently):

resolver: lts-6.23
packages:
  - .
  - location:
      git: https://github.com/AccelerateHS/accelerate
      commit: 7f419367b303af021fb4b7d29bc296f04b462e31
    extra-dep: true
  - location:
      git: https://github.com/tmcdonell/cuda
      commit: bfa5b024fbf10a312f58c5d46d766d6bd2d01e04
    extra-dep: true
  - location:
      git: https://github.com/AccelerateHS/accelerate-cuda
      commit: ee9c4c13e9ef5d364f476f97c5ced05645d0a919
    extra-dep: true
  - location:
      git: https://github.com/AccelerateHS/accelerate-io
      commit: 4e3637469972993a4b160af39b00ea7a7ec6f1e2
    extra-dep: true
extra-deps:
  - ratio-int-0.1.2
  - repa-devil-0.3.2.6
  - transformers-0.4.3.0
  - unique-0
flags: {}
extra-package-dbs: []
extra-include-dirs:
  - /usr/local/Cellar/devil/1.7.8_3/include
extra-lib-dirs:
  - /usr/local/Cellar/devil/1.7.8_3/lib

A hopefully related issue is that changing a single line in my project source often causes stack to recompile all extra dependencies. If you think this is the same issue, I'd be happy to supply any information I can.

[mdibaiee]

I switched to Linux a little after I reported this bug and I haven't had this anymore and I can't help on this issue anymore. Sorry about that.

[mgsloan]

I suspect this has to do with git versions. Which version of git are y'all using? I have version 2.7.4

[Blaisorblade]

@mgsloan we might want to add the git version to logs on git failures (especially with stack --verbose, or maybe not even) so we get it automatically in bug reports.

[Blaisorblade]

@jonascarpay I think correct Git URLs here should end in .git, e.g. https://github.com/AccelerateHS/accelerate.git instead of just https://github.com/AccelerateHS/accelerate (and so on for other projects). GitHub does some magic to not require this but I don't know how robust it is. So, beyond the git version, can you make sure adding .git to the end of the these Git URLs?

[jonascarpay]

This the output taken from two stack build --verbose, in the first of which I get a Failed to reset to commit, followed by a ExitFailure 128. The second build completes without problems.

@mgsloan I'm on 2.10.1, the version from homebrew

@Blaisorblade Appending .git might have made the error somewhat rarer, but I'm not sure. I don't recall getting the ExitFailure 128 before, could this be related? I'll do some more testing after the weekend.

[Blaisorblade]

So it's hard to tell what's going on, especially without multiple traces to see what's in common and what isn't. Here's a few (somewhat desperate) hypotheses.

• _git_ itself is failing, for some reason (it'd be good to figure out what 128 means, but git doesn't document its exit codes)
• the projects use submodules, and I'm not sure that's relevant or a distraction. (In particular, git reset seems to work fine with submodules in the second run).
• stack's logs omit information that could be useful.

Is there any chance there's a permission problem involved—somehow git can't write to the repo, so that removing and cloning it helps? Is _anything_ touching the downloaded repo, except git as invoked by stack? (I've surely written my share of cron scripts doing funky stuff with permissions). Are you accessing the repo as a different user sometimes?

This might well be instead a bug in stack itself, but I lack the faintest hypothesis—except if some race condition when executing processes were happening—but this seems pretty absurd in this code (I remember some undiagnosed symptoms in strange cases, but that was while building packages concurrently, IIRC, not fetching them)...

[jonascarpay]

So I had some time on my hands and did some testing. I created a number of empty stack projects from the simple template. One (called test) had all the others (called extradep[1-6]) as extra git dependencies, like this. I triggered the error by repeatedly adding or removing a single line in test/src/Main.hs and running stack build. Here are some of the things I found, I hope they can provide some clue as to what is happening:

• The error still occurs, about once every 10 builds. The problem does not seem to be related to submodules (the extradeps have none) or network issues (all dependencies are local repos).
• The error cannot be triggered by running stack clean && stack build
• The error only ever occurs for the first listed package, the one below '.' in stack.yaml. More specifically, _when_ an error is reported I only ever see Failed to reset to commit [commit of first listed package]. For the stack.yaml linked above that gives Failed to reset to commit eb19e515002fd8e5853f90802682604af6505df8. Changing the order of the packages changes the error.
• Manually entering the folder and doing a git reset never fails.
• I don't think the files are being changed in the meanwhile somehow. I shouldn't have any scripts running that touch anything, I don't see git reporting any changes, and my timestamps don't change.
• I don't think there's a permissions issue, or at least, chmodding everything to 777 doesn't help.
• I still haven't found a way to reliably reproduce the error. The number of extra dependencies does not increase the frequency, and as far as I can tell occurrences are completely random.

This is the verbose build output of a successful build: http://pastebin.com/raw/afjEjFLX. There are some ignored exceptions, is that normal?
Failing build: http://pastebin.com/raw/btNUAV4u
If there's anything else you'd like to have me try, let me know

[mgsloan]

One thing to note is that you can get this message simply when changing the commit to something that has not yet been pulled. Doesn't quite match the description here, but may be related. See https://github.com/commercialhaskell/stack/issues/2762

This may also be related to https://github.com/commercialhaskell/stack/issues/2748

[kgadek]

Hmm, I got into the same issue on my playground project after this change: kgadek:novelist@dc009a31be

I've noticed that (full log at the bottom):

1. re-save any file,
2. first runhaskell invocation just fails
3. second invocation: Cloning into & works
4. third invocation just works
5. GOTO 1

OS X 10.11.6, git 2.10.2, stack 1.2.0. Ping me if you need any debugging.

[you@here novelist]$ stack runhaskell --verbose -- -isrc -itest -iqc-gen test/Spec.hs
Version 1.2.0 x86_64 hpack-0.14.0
2016-11-22 21:43:15.881614: [debug] Checking for project config at: /Users/konrad/prv/git/novelist/stack.yaml
@(Stack/Config.hs:792:9)
2016-11-22 21:43:15.882298: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:810:13)
2016-11-22 21:43:15.883718: [debug] Trying to decode /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:68:5)
2016-11-22 21:43:15.892942: [debug] Success decoding /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:72:13)
2016-11-22 21:43:15.906074: [debug] Getting system compiler version
@(Stack/Setup.hs:354:17)
2016-11-22 21:43:15.906937: [debug] Run process: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:15.992251: [debug] Process finished in 85 ms: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:15.994233: [debug] Getting global package database location
@(Stack/GhcPkg.hs:54:5)
2016-11-22 21:43:15.994449: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:15.994964: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:101:13)
2016-11-22 21:43:15.995116: [debug] Run process: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:15.995595: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:171:5)
2016-11-22 21:43:15.995737: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:16.032505: [debug] Process finished in 36 ms: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:16.032736: [debug] Process finished in 38 ms: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:16.066713: [debug] Process finished in 71 ms: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:16.066877: [debug] Resolving package entries
@(Stack/Setup.hs:234:5)
2016-11-22 21:43:16.067686: [debug] Run process: /usr/local/bin/git reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:16.071266: [info] Failed to reset to commit cf30558c3da375868a54600c6f796d2c7cd60565, deleting and re-cloning.
@(Stack/Config.hs:580:21)
2016-11-22 21:43:16.073669: [debug] Creating process: /usr/local/bin/git clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
@(System/Process/Run.hs:103:5)
Cloning into '/Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj'...
error: could not lock config file /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj/.git/config: No such file or directory
fatal: could not set 'remote.origin.fetch' to '+refs/heads/*:refs/remotes/origin/*'
Process exited with ExitFailure 128: /usr/local/bin/git clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj







[you@here novelist]$ stack runhaskell --verbose -- -isrc -itest -iqc-gen test/Spec.hs
Version 1.2.0 x86_64 hpack-0.14.0
2016-11-22 21:43:20.233018: [debug] Checking for project config at: /Users/konrad/prv/git/novelist/stack.yaml
@(Stack/Config.hs:792:9)
2016-11-22 21:43:20.233814: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:810:13)
2016-11-22 21:43:20.235191: [debug] Trying to decode /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:68:5)
2016-11-22 21:43:20.244288: [debug] Success decoding /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:72:13)
2016-11-22 21:43:20.257865: [debug] Getting system compiler version
@(Stack/Setup.hs:354:17)
2016-11-22 21:43:20.258585: [debug] Run process: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.341067: [debug] Process finished in 82 ms: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.343040: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:101:13)
2016-11-22 21:43:20.343249: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:171:5)
2016-11-22 21:43:20.343393: [debug] Getting global package database location
@(Stack/GhcPkg.hs:54:5)
2016-11-22 21:43:20.343525: [debug] Run process: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.343939: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.344278: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.380457: [debug] Process finished in 36 ms: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.381147: [debug] Process finished in 36 ms: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.413398: [debug] Process finished in 69 ms: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:20.413556: [debug] Resolving package entries
@(Stack/Setup.hs:234:5)
2016-11-22 21:43:20.414280: [debug] Creating process: /usr/local/bin/git clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
@(System/Process/Run.hs:103:5)
Cloning into '/Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj'...
remote: Counting objects: 368, done.
remote: Total 368 (delta 0), reused 0 (delta 0), pack-reused 368
Receiving objects: 100% (368/368), 57.60 KiB | 43.00 KiB/s, done.
Resolving deltas: 100% (166/166), done.
2016-11-22 21:43:23.222631: [debug] Run process: /usr/local/bin/git reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:23.228155: [debug] Process finished in 5 ms: /usr/local/bin/git reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:23.228687: [debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:166:18)
2016-11-22 21:43:23.228895: [debug] Run process: /usr/local/bin/runghc -isrc -itest -iqc-gen test/Spec.hs
@(Stack/Exec.hs:65:5)




(…snip: test output here…)







[you@here novelist]$ stack runhaskell --verbose -- -isrc -itest -iqc-gen test/Spec.hs
Version 1.2.0 x86_64 hpack-0.14.0
2016-11-22 21:43:27.459717: [debug] Checking for project config at: /Users/konrad/prv/git/novelist/stack.yaml
@(Stack/Config.hs:792:9)
2016-11-22 21:43:27.460536: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:810:13)
2016-11-22 21:43:27.461988: [debug] Trying to decode /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:68:5)
2016-11-22 21:43:27.471702: [debug] Success decoding /Users/konrad/.stack/build-plan-cache/x86_64-osx/lts-7.8.cache
@(Data/Store/VersionTagged.hs:72:13)
2016-11-22 21:43:27.481904: [debug] Getting system compiler version
@(Stack/Setup.hs:354:17)
2016-11-22 21:43:27.482574: [debug] Run process: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.569727: [debug] Process finished in 87 ms: /usr/local/bin/ghc --info
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.571652: [debug] Getting global package database location
@(Stack/GhcPkg.hs:54:5)
2016-11-22 21:43:27.571874: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.572390: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:101:13)
2016-11-22 21:43:27.572541: [debug] Run process: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.572908: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:171:5)
2016-11-22 21:43:27.572978: [debug] Run process: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.609372: [debug] Process finished in 36 ms: /usr/local/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.609538: [debug] Process finished in 37 ms: /usr/local/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.642743: [debug] Process finished in 70 ms: /usr/local/bin/ghc --numeric-version
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.642951: [debug] Resolving package entries
@(Stack/Setup.hs:234:5)
2016-11-22 21:43:27.643736: [debug] Run process: /usr/local/bin/git reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.649389: [debug] Process finished in 5 ms: /usr/local/bin/git reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
@(System/Process/Read.hs:277:3)
2016-11-22 21:43:27.649912: [debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:166:18)
2016-11-22 21:43:27.650181: [debug] Run process: /usr/local/bin/runghc -isrc -itest -iqc-gen test/Spec.hs
@(Stack/Exec.hs:65:5)




(…snip: test output here…)

[kgadek]

I have a suspect: race condition. My vim is heavily plugged, some stack exec into ghc-mod, hlint on each save, i.e. at the same time.

How do you think, putting a simple lock around Stack/Config.hs: cloneAndExtract could fix this?

[Blaisorblade]

@kgadek We've been wondering about race conditions for forever, but have been unable to confirm. But you're saying you suspect a race condition between different stack processes? I know there are locking mechanisms for that, but I don't think they're simple.

Also, what's the stack exec command line? stack exec does not necessarily fetch sources... (unless you use for instance --package X to ask it to build X).

[kgadek]

Yes, I suspect 2+ stack processes race condition. I replaced git binary with a little bash script appending some output to some log file and the output was mixed.

As far as I know, this is a solved problem assuming a sane filesystem (lock-files).

I don't run stack exec directly, I use stack runhaskell.

Wondering how could I confirm this thesis. Add withLockFile around that call and try to reproduce? (PoC: use one lockfile per .stack-work. Final solution: one lockfile per each downloaded package repo)

[Blaisorblade]

I replaced git binary with a little bash script appending some output to some log file and the output was mixed.

Could you also get their parent hierarchy? That should be easy with ps or pstree and would probably tell us if those git processes were started by the same stack process or by different ones, what are their command lines, and so on.

Re lockfiles I stand corrected—I assumed "simple lock" could only mean a threading thing. (I've also long conjectured there's some race condition in a single stack process). I'm not the most qualified to answer, but your idea certainly sounds clearly worth a try.

Locking the whole .stack-work for everything modifying it might even be a long-term solution (I've done stupid stuff like #2797), though we don't want to do too much locking.

I don't run stack exec directly, I use stack runhaskell.

I suspect the same comment applies: runhaskell without --package shouldn't be building anything per se, which is why I'm asking. But given your git experiment, it's likely something is racing.

[kgadek]

I've already done exactly what you're asking :) More precisely: mv /usr/local/bin/git{,_orig} and the following the new /usr/local/bin/git. Please forgive me hardcoding the PID 3784, my shell uses git for prompt, wanted to filter that out.

#!/bin/bash


ppid () { ps -p ${1:-$$} -o ppid=; }

pzsh=true
while true; do
  if [[ $curr -eq 1 ]]; then
    break;
  elif [[ $curr -eq 3784 ]]; then
    pzsh=false
  fi
  curr=$(ppid $curr)
done

if [ $pzsh = true ]; then
    echo "OHAI FROM GIT" >> ~/tmp/debug.log
    echo "  PWD: $(pwd)" >> ~/tmp/debug.log
    echo "  CMD: $@" >> ~/tmp/debug.log

    curr=$PPID
    while true; do
    echo "  CALL: $curr: $(ps -fp $curr | awk 'NR==2 {for (i=8; i<NF; i++) printf $i " "; print $NF; }')" >> ~/tmp/debug.log
      if [[ $curr -eq 1 ]]; then
        break;
      fi
      curr=$(ppid $curr)
    done

fi

/usr/local/bin/git_orig "$@"

Part of the output (full log is 8k+ lines, though the pattern reoccurring):

OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 30294: /usr/local/bin/stack path
  CALL: 30293: ghc-mod browse -o Data.Functor.Classes
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 30362: stack --verbosity silent exec -- hlint /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 30361: stack --verbosity silent exec --no-ghc-package-path -- hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 30360: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 30532: /usr/local/bin/stack path
  CALL: 30531: stack path --config-location
  CALL: 30360: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 30361: /Users/konrad/.local/bin/hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CALL: 31337: stack --verbosity silent exec -- hlint /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 31336: stack --verbosity silent exec --no-ghc-package-path -- hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 31335: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 31816: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 31821: stack --verbosity silent exec --no-ghc-package-path -- hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 31824: stack --verbosity silent exec -- hlint /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 32189: /usr/local/bin/stack path
  CALL: 32188: stack path --config-location
  CALL: 31821: /Users/konrad/.local/bin/hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 31816: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 32808: stack --verbosity silent exec --no-ghc-package-path -- hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 32807: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 32809: stack --verbosity silent exec -- hlint /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 32978: stack path --config-location
  CALL: 32979: /usr/local/bin/stack path
  CALL: 32808: /Users/konrad/.local/bin/hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 32807: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CALL: 32979: /usr/local/bin/stack path
  CALL: 32807: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 32979: /usr/local/bin/stack path
  CALL: 32807: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
OHAI FROM GIT
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 33265: stack --verbosity silent exec -- hlint /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 33264: stack --verbosity silent exec --no-ghc-package-path -- hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 33263: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CALL: 33263: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 33458: stack path --config-location
  CALL: 33264: /Users/konrad/.local/bin/hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded
  CMD: clone --recursive [email protected]:ekmett/recursion-schemes.git /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CALL: 33458: stack path --config-location
  CALL: 33264: /Users/konrad/.local/bin/hdevtools check -g-Wall /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 33263: stack --verbosity silent exec -- ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd
OHAI FROM GIT
  PWD: /Users/konrad/prv/git/novelist/.stack-work/downloaded/rfaC46YBapDj
  CMD: reset --hard cf30558c3da375868a54600c6f796d2c7cd60565 --
  CALL: 33730: /usr/local/bin/stack path
  CALL: 33263: /Users/konrad/.local/bin/ghc-mod check /Users/konrad/prv/git/novelist/src/System/Novelist/NovelistF.hs
  CALL: 27660: nvim src/System/Novelist/NovelistF.hs
  CALL:  1141: -zsh
  CALL:  1140: login -fp konrad
  CALL:  1139: /Applications/iTerm.app/Contents/MacOS/iTerm2 --server login -fp konrad
  CALL:   822: /Applications/iTerm.app/Contents/MacOS/iTerm2
  CALL:     1: /sbin/launchd

Not sure if it's relevant, but I use neovim with following config: https://github.com/kgadek/neovim-config

I'll write/try writing locks tomorrow, will see if it helps.

PS. Shall I use gists for such snippets?

[Blaisorblade]

Kudos for the find!

Without having proof-read the shell script and assuming you're navigating the "ppid chain", amazing findings I see there and I pass on to @mgsloan:

• stack path (a query command) resets repos?! That looks like a bug to me (though there might be valid reasons I'm missing).
• Overall, you're not the first to use ghc-mod, so while what you're seeing might not explain all reports in this issue, other unexplained reports might be due to this. At least when I triage a bug, it doesn't occur to me that concurrent queries on the repo might write to it.
• stack exec, in configurations that don't build code, still resets repos (unsurprising since it does need info that conceptually comes from stack path). While I was a 100% wrong above, this still looks like a bug although I'm less sure than with stack path.
• even if neither command reset repos, a lock-file for repos (or .stack-work) is certainly a good idea.

PS. Shall I use gists for such snippets?

A bit a matter of taste — if you posted the full log, sure. This one is borderline but I'm happy it was inline.

[mgsloan]

Interesting! I've split off https://github.com/commercialhaskell/stack/issues/2811 and also pushed a fix. So now stack path and stack exec won't cause packages to get cloned / reset. Does this resolve the issue?

[mgsloan]

I've also pushed a commit which goes back to the old approach. This means that instead of doing frequent git reset, it will just do an initial clone + reset each time the commit is changed.

[kgadek]

With current stack (Version 1.3.1, Git revision 164cb5529623c35249b7146d0bb8aaa66c502ecd), after stack setup (previously I was using system one; re # 2221), I can't reproduce the issue 😄 Thanks!

[jonascarpay]

Same here, thank you!

[mgsloan]

Great, closing!