Glide: Running `glide up` does not update glide.lock with latest revision of private repo

[ryanwalls]

I have a private repo that I have as a dependency of my project. When I push new code to the private repo and then run glide up in the project, the glide.lock file is not updated to the latest revision. I've worked around it by manually updating the lock file.

Glide version: 0.9.1
Go version: 1.6

[russmatney]

I'm seeing similar behavior, just want to add that I'm seeing Cannot detect VCS errors on the private repo's subpackages. The error is fair, but I wouldn't expect that to prevent a glide.lock update for the rest of the packages, especially when vendor/ updates just fine.

[technosophos]

I have seen this error, and have worked around it by using version: master in the glide.yaml file. That, however, is not the intended behavior. We need to get a fix for this into Glide.

[technosophos]

@russmatney Are you setting the VCS fields (vcs, repo) in the glide.yaml file?
https://github.com/Masterminds/glide/blob/master/docs/glide.yaml.md

[russmatney]

@technosophos glide.yml is as such:

package: myproj.domain/namespace/proj
import:
- package: myproj.domain/namespace/firstdep
  vcs: git
//etc

The errors/warnings are for firstdep's subpackages, which may not have a detectable vcs as they belong to firstdep's git repo. Ideally they'd throw "soft" warnings rather than fail the whole glide.lock update – all I need is firstdep to update itself, assuming that's the intended behavior. For now I'm updating the version in the .lock file by hand, but can move to the version: master fix.

[technosophos]

I think @mattfarina may know why the VCS error is showing up. But the reason it's a stopping condition is that once a VCS can't be updated, we can't guarantee the consistency of the version tree anymore.

Let me know if setting version: master works.

[mattfarina]

@russmatney Glide keeps compatibility with go get on remote package names. If you go to the import section on the glide.yaml docs you'll see the rules listed under the package property for a name.

@technosophos should we keep go get compatibility? This is one of those places I'd love to have a lively conversation on.

[sdboyer]

@mattfarina sorry, why wouldn't we keep keep godep compat? the only reason i can imagine is because that means including the .git infix for custom domains, and that feels redundant with also having the vcs: git property. what am i missing?

[mattfarina]

@sdboyer godep compat? Do you mean go get compat?

[sdboyer]

@mattfarina sorry, yes, butterfingers - go get compat

[russmatney]

@technosophos @mattfarina my mistake – I misread the errors and jumped to the wrong conclusion, namely that the private repo VCS errors were preventing the glide.lock write. The issue is caused by an actual error in resolving x/net/context, same as https://github.com/Masterminds/glide/issues/373.

The private repo VCS warnings is a result of the state of gogs and go-import metadata: https://github.com/gogits/gogs/issues/2825

I'm seeing proper glide.lock updates now (if the x/net version is set to an earlier commit, that is).

[atombender]

@technosophos @mattfarina: We're seeing something that looks like this problem, but it doesn't happen for me, only for my colleagues. Here are the outputs of the exact same commands on my machine (good.txt) and one of theirs (bad.txt).

We're using the same version of Go and Glide. It's all on OS X. The only difference seems to be that my colleagues check out Git repos using the git protocol, whereas I use https.

Setting version: master fixes the issue.

[atombender]

I can reproduce it consistently now, but I'm not sure if it represent the original failure scenario exactly. First, I create this glide.yaml in a clean directory, with one package:

package: github.com/t11e/glidebug
import:
- package: github.com/aws/aws-sdk-go
  version: 6f62fc3ff9aa98c629b350376825cda566658d75
  subpackages:
  - aws
  - aws/awserr
  - aws/session
  - service/s3

Then I run glide install. Everything good so far.

Then I remove version: 6f62... and run glide up. Now my lock file is:

hash: 1ac04a04efbc1f1153bf98b3876e0e722b49e5b702a87bf0dde64e0212d1e702
updated: 2016-05-06T14:35:39.336813296-04:00
imports:
- name: github.com/aws/aws-sdk-go
  version: 6f62fc3ff9aa98c629b350376825cda566658d75
  subpackages:
  - aws
  - aws/awserr
  - aws/session
  - service/s3
devImports: []

Notice old version still remains. glide up does not refresh anything.

Still doesn't explain why it's otherwise working fine for me on my own repo and not my colleagues'.

[atombender]

Great. After deleting my vendor directory and doing glide i; glide up, I now get the same exact same issue as my colleagues, ie. no changes on update.

[atombender]

Here's another data point:

$ cd vendor/github.com/aws/aws-sdk-go
$ cat .git/HEAD
6f62fc3ff9aa98c629b350376825cda566658d75
$ git status
HEAD detached at 6f62fc3
nothing to commit, working directory clean

I see in git.go that if HEAD is detached, an update does nothing. Could this be the cause?

[atombender]

@mattfarina Could this be related to #360?

[atombender]

@technosophos @mattfarina: Can you spare some time to look at this?

[mattfarina]

@atombender A couple quick questions...

1. In the directory vendor/github.com/aws/aws-sdk-go do you have a .git directory?
2. Do you have the external dependencies checked into your projects repo (vendored)?

[atombender]

@mattfarina: (1) Yes. (2) No.

[atombender]

(You may have missed this since you're asking whether it has a .git directory.)

[atombender]

Here's another data point: I find that I generally have to run glide up twice to make it pick up a new upstream Git version. Here's my starting point:

$ grep -A1 glidetest glide.yaml
- name: github.com/t11e/glidetest
  version: branch_x

$ (cd vendor/github.com/t11e/glidetest && git rev-parse HEAD)
399832009187ce47fde554d6d174b83ce58d13e2

$ grep -A1 glidetest glide.lock
- name: github.com/t11e/glidetest
  version: 399832009187ce47fde554d6d174b83ce58d13e2

Now I edit glide.yaml to move to a different branch:

$ grep -A1 glidetest glide.yaml
- name: github.com/t11e/glidetest
  version: branch_y

$ glide up
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/t11e/glidetest.
[INFO] Setting version for github.com/t11e/glidetest to branch_y.
[...]

$ grep -A1 glidetest glide.lock
- name: github.com/t11e/glidetest
  version: 399832009187ce47fde554d6d174b83ce58d13e2

$ (cd vendor/github.com/t11e/glidetest && git rev-parse HEAD)
399832009187ce47fde554d6d174b83ce58d13e2

Second try:

$ glide up
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/t11e/glidetest.
[INFO] Setting version for github.com/t11e/glidetest to branch_y.
[...]

$ grep -A1 glidetest glide.lock
- name: github.com/t11e/glidetest
  version: f28c1858bf5a4b0c4a23c33b09a97dc6cc6683bb

Perhaps git fetch is not being done? It ought to always be doing a fetch + git reset --hard to move HEAD to the new version, but evidently not.

[atombender]

@mattfarina I have a working dir with the problem, where I can reproduce the problem consistently — ie., glide up does nothing even though HEAD is behind, no matter how many times I run it. This is a private repo, so I can't share the code, though. How can we debug this?

[atombender]

Never mind, I don't have a repo. But I may be able to create one.

[atombender]

@mattfarina So can we expect anything to happen here? I can't imagine I'm the only one hit by this problem.

[sdboyer]

@atombender i know it's not much solace for an immediate bug you're having now, but this will _probably_ be resolved by the switch to the vsolver engine (i don't know exactly what's going on with this, so i can't be completely sure).

that's progressing in #384, if you want to follow along. with any luck, it'll be at a point soon where you could test this issue against it, and see if it's still a problem.

[oryband]

Anything new with this issue? Been 3 months and no progress. This is a serious bug.

[oryband]

PING

[sdboyer]

Are all of these issues still occurring with the 0.12 release?

[ryanwalls]

@sdboyer Yes, it is for me. glide up does not update the commit hash of my private repo in my glide.lock file.

[sdboyer]

@ryanwalls it seems quite strange to me that the dividing line here would be on private vs. public repos.

I'm working through some related parts of the code now as part of #384 - maybe I'll run across something.

[adangert]

👍 same here, having problems with glide.lock not updating to the glide.lock version.

[cep21]

I have the same issue. I removed the repository inside ~/.glide/cache/src and rerun glide up, and noticed I now had the latest version. The code in ~/.glide/cache/src/ was not getting updated to master.

[radomirml]

Same issue here. As noted by @cep21 , deleting repo from the cache helps.

[sdboyer]

the original issue here is most likely a separate problem - if clearing the cache is solving your problems, then the fix we're working on for #592 should be what you need.