Yarn: upgrading packages from git repos not working

[omnidan]

I have a dependency that loads from a git repo directly via git+ssh://.

Before upgrading:

• yarn.lock contains the package with the correct commit hash
• node_modules contains the git repo with the correct commit hash

After upgrading via yarn upgrade, I'd expect it to update the commit in yarn.lock to the latest commit of the repo, as well as updating the repo in node_modules.

However, what happens is:

• yarn.lock contains the package with the updated commit hash
• node_modules does not get updated, it still contains the git repo with the state of the old commit

After removing node_modules and running yarn install again:

• node_modules still does not get updated (probably because it got loaded from the cache)

Only when removing the folder manually from ~/.yarn-cache, removing node_modules again, then running yarn install, it pulls the latest version of the repository.

Please note that the package.json version did not change, only the commit hash did. yarn.lock seems to handle this correctly, while the cache doesn't. This also works fine with npm install as it just pulls the repo every time.

• node.js version: v6.5.0
• yarn version: 0.16.1
• OS version: macOS 10.12.1

[gertjvr]

Had a similar issue, where upgrade changed the yarn.lock file correctly but node_modules upgraded to the latest npm package rather than my git repo branch reference

node.js v6.9.1
yarn version 0.16.1
OS version: windows 10

[jshthornton]

Same issue.
Updated the hash of the commit, yarn lock has the correct hash, removed node modules, yet still getting old version.

debian jessie64
node v6.6.0
yarn v0.16.1

[wyze]

Could someone please provide a package.json I could use to try and reproduce this?

[omnidan]

@wyze this is a bit hard to reproduce as you need to add a git+ssh repo, then push to it, and then try to upgrade to the latest version via yarn. If you use an existing repo, yarn always uses the latest commit (even when the lock file has a different commit specified).

I can summarize the reproduction steps for you:

• create a new git repo testrepo
• add it to package.json dependencies: "testrepo": "git+ssh://[email protected]/wyze/testrepo.git"
• yarn
• add a file to the testrepo and push it
• run yarn upgrade
• file is not available in node_modules/testrepo/
• rm -rf ~/.yarn-cache/npm-testrepo-1.0.0/
• now installing via yarn should add the file to node_modules/testrepo/

I hope this helps :grin:

PS: it might have to do with the fact that repos from git+ssh are treated the same way as npm packages (by their package version, I assume). Maybe it would make sense to store them like this instead: ~/.yarn-cache/git-testrepo-COMMITHASH/ Treating git repos in a special way will also solve the issue mentioned above: if the lock file specifies a commit hash, this commit will be installed.

[wyze]

I'll try those steps, thanks. That is enough information for me to try and reproduce the issue.

[whitecolor]

The same for me:
1) I installed package from NPM using yarn add repo it saved in the cache say version 1.2.3,
2) then I decided to replace package with the version from github (as it was updated in the repo, but not published and the version in package.json was not changed).
3) I did yarn add github:user/repo - it replaced it in package json but took version from cache
4) I tried yarn upgrade, yarn --force but I still got version from cache.
5) So have to remove manually npm-repo-1.2.3 dir from cache.

[nitinbansal1989]

I have the similar error:

1. I forked an repo and tried to install it via github url.
2. i did yarn cache clean and yarn clean.
3. Added the github url in package.json
4. Still the package is downloaded from the npm repository.

[gijoehosaphat]

Seeing the same issues, but cleaning cache does resolve this for us. Unfortunately any time saved with yarn is lost due to this bug and we do not trust it for production use.

[aaronjensen]

fwiw, just like with npm, I typically bump the version on git hosted packages when I change them to work around this. If it's a fork, I just add -1 to the version (1.1.3-1) and increment it if I make additional changes. Obviously this requires access to the repo, so if it's someone else's, you'd need to fork it as well.

It's great that this is being treated like a bug on yarn, hopefully it is fixed so that it works similar to bundler.

[Fudoshiki]

1. remove node_modules folder
2. yarn cache clean
3. yarn
   work

[bestander]

Should be fixed now #2074.
Please reopen if the issue is still there

[rally25rs]

This is still not working for me in v0.19.1, but it is hard to tell if #2074 is in that build.

For me;

yarn upgrade does not get new code from any dependencies that use a github url
yarn upgrade pkgname where pkgname is a name of a package that uses a github url only checks real published NPM packages and fails if it's not a real published package, or downloads the published package not the github repo ( see issue #2501 )
yarn upgrade my-pkg@git+ssh://[email protected]/MyCompany/my-pkg.git#master appears like it does something, but again leaves the previous version of the code in node_modules instead of replacing it with the latest version.

[bestander]

To see if a PR is in a release:

• find the merge message where it went into master branch
  
• click on the commit and see in which branches this commit is present
  

In this case it is only available in master branch, so it is coming in the 0.20 release.

[behrad]

waiting for this :(

[juanca]

I opened a pull request which mirrors npm upgrade behavior.

[juanca]

This should be fixed. Please double check.

[erasmo-marin]

Still not working in v0.21.3, I need this for my workflow. Without this, I can not get the benefits of using yarn.

[juanca]

Please update to the latest version.

[erasmo-marin]

I didn't realize that the RC was available, Installed RC v0.22.0 and it's working now! 😄

[dunnock]

weird, it seems works for some people, but still does not work for me. After pushing to git repo master yarn upgrade <package> does not pick new version, while if I specify yarn upgrade <git url>#<commit> it does.

>yarn -version
yarn install v0.22.0

[bestander]

@dunnock, feel free to open a new issue with steps to reproduce it

[heyitsjames]

How do I install a RC? I'd like to move to v0.22.0 but I can't figure out how to install RC versions. homebrew is on 0.21.3

[dunnock]

@heyitsjames I've managed to install on OSX by instruction on this page :

curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version v0.22.0

After I've spent some time cleaning yarn build istalled by homebrew, make sure to clean it before install.

[whitecolor]

Here I too describe the issue with upgrading from git repo.

[apexskier]

https://github.com/yarnpkg/yarn/issues/3198 -- related

[pmunin]

Just reproduced the issue on latest Yarn. Still does not work. Git repository was updated with new commit, yarn does not upgrade it saying it's up to date, resolving previous commit :-1:

[rally25rs]

Yarn 0.27.5 yarn upgrade worked fine for me with a Github repo.

$ yarn add git+ssh://[email protected]:my-org/my-proj#test
yarn add v0.27.5
info No lockfile found.
...
Done in 19.42s.

$ cat node_modules/my-proj/test.txt
1

(push new commit to my-proj that updates test.txt)

$ yarn upgrade
yarn upgrade v0.27.5
...
Done in 10.83s.

$ cat node_modules/my-proj/test.txt
2

[g-sam]

broken for me with v1.0.2

[krzkaczor]

@bestander I think there is a regression or something. I also needed to do yarn clean cache to make it work

[bestander]

@krzkaczor this is a year old bug and was reported to be fixed, please open a new issue with repro steps

[andreialecu]

@bestander Also seeing this bug, I think it is a regression because it used to work fine until recently.

It happens in my case when the git dependency is to a particular branch. yarn upgrade never sees the new commits.

Probably related: https://github.com/yarnpkg/yarn/issues/4722

[shtse8]

I have the same problem when pointing the package to my forked repo. i have pushed a new commit and yarn again. nothing was upgraded and even removed the node_modules and then yarn again still failed.

[felipeplets]

Same here. Did someone opened a new issue for this? I do not believe many people will find it here.

[apengstrom]

I am still experiencing this. It will not update to the new commit I pushed to my forked repo.

[whitecolor]

@apengstrom it is better to make repo and instructions to reproduce, or no one will believe you here =)

[apengstrom]

Instructions are already given, scroll up. I was just commenting that the issue still exists.

[rally25rs]

@apengstrom I am unable to reproduce the problem on v1.3.2. I also use private repos daily at work and have not had a problem, so I can't dive in and try to debug it.

Here is a quick example:

~/Projects/yarn-test 🐒   cat package.json
{
  "name": "yarn-test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "yarn-scripts-test": "git+ssh://[email protected]:rally25rs/yarn-scripts-test.git"
  }
}

~/Projects/yarn-test 🐒   yarn
yarn install v1.3.2
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 4.28s.

~/Projects/yarn-test 🐒   cat node_modules/yarn-scripts-test/file.txt
1

Note that the dependency's file.txt contains the text 1.

Now I'll update that dependency:

~/Projects/yarn-scripts-test (master) : echo 2 > file.txt

~/Projects/yarn-scripts-test (master *) : git commit -am "updated to 2"
[master 570351d] updated to 2
 1 file changed, 1 insertion(+), 1 deletion(-)

~/Projects/yarn-scripts-test (master) : git push origin master
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 317 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To github.com:rally25rs/yarn-scripts-test.git
   247243f..570351d  master -> master

and back to the original project, run an upgrade

~/Projects/yarn-test 🐒   yarn upgrade yarn-scripts-test
yarn upgrade v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Rebuilding all packages...
success Saved lockfile.
success Saved 1 new dependency.
└─ [email protected]
✨  Done in 3.99s.

~/Projects/yarn-test 🐒   cat node_modules/yarn-scripts-test/file.txt
2

The dependency has been updated.

---

Keep in mind we are pretty much all volunteers working on this open source project. If this is a reproducible and blocking issue for you, we encourage you to help out debugging the issue and opening a PR to fix it.

[andreialecu]

@rally25rs I can reproduce this every time. I have several projects, but let's assume projectA and projectB both as git dependencies.

If I run:
yarn upgrade projectB

It will properly upgrade projectB but it will also downgrade projectA every time. It will actually downgrade it from the already up-to-date version.

If however, I run yarn upgrade with no packages specified, it works properly - but it is much slower.

Edit: It may be relevant that projectA in this context is a git dependency to a specific branch (eg. #debug)

[rally25rs]

@andreialecu does projectB have a dependency on projectA? Is projectA required as a dependency from more than one place? IIRC upgrade should behave roughly equivalent to yarn remove projectB && yarn add projectB so if projectA is only required by projectB then it would be removed and re-added.

If you are willing to provide more detail, like what the package.json entries are and what branches/tags exist on the projects that it should resolve to, and the before and after yarn.lock entries, then it might help debug.
Or if you can somehow rig up some minimal reproduction example.
Otherwise it's really hard to debug issues with people's private repos.

[andreialecu]

@rally25rs projectA in this case has no dependencies to anything else in the main project's list of dependencies.

Actually, something else happens now that I looked into it.

It doesn't downgrade projectA itself, but it downgrades all of projectA's dependencies to some weird previous version.

Here's yarn.lock:

On the left is the current version of that particular package, with the current dependencies. This is after running just yarn upgrade.

On the right is what running yarn upgrade projectB (which is not public) does to the completely unrelated projectA (which is andreialecu/dpd-apn#debug in this case, which is available on github)

Notice how it is deciding to just revert the dependency changes that occured during the last 4 commits, even though the main package itself resolves to the same exact commit hash.

I was seeing problems because of one of the subdependencies of that package reverting every time and reintroducing a bug.

yarn -v is 1.3.2

I can reproduce it every time by doing yarn upgrade then yarn upgrade projectB (even without updating projectB at all

[rally25rs]

@andreialecu since this issue is already marked "closed" for almost a year, would you mind opening a new issue and copy/pasting your details above into it? You can try to assign me to it, or just mention me if you can't assign me.

Thanks!

[carlost]

this issue just bit us as well.

yarn 0.27.5 on ubuntu

we were using the url of https://github.com/repo_owner/forked_repo_name.git#branch_name. even though we updated the yarn.lock to point to an updated SHA1 hash, yarn install did not recognize it as different.

we have to update the version # in the branch for yarn install to recognize the difference and pull down the update.

[juanca]

Happening to me too. Though, I'm not on latest yarn, but definitely 1+.

Could a sworn I had committed some regression tests... Will try to take a look later today.

[rally25rs]

@carlost

we have to update the version # in the branch for yarn install to recognize the difference and pull down the update.

A lot of bugs have been fixed since v0.27.5, and yarn upgrade has been mostly rewritten. In Yarn v1, running yarn upgrade {package_name} should automatically update yarn.lock to the new/latest SHA for the branch.

[Hobart2967]

Please reopen this issue, as it exists in 1.3.2!

Steps to repro:

1. Add repository to package.json
2. yarn install
3. Push file to repo
4. yarn install
   => Result: Pulled package is still the old version.

[rally25rs]

@Hobart2967 yarn install will respect the commit hash that is tracked in yarn.lock. This is fundamental to providing deterministic builds. To change to the latest commit, you can run yarn upgrade {package_name} and it will go get the latest commit hash and update yarn.lock (in other words, your step 4 should be yarn upgrade {package})

[davidgg]

I'm still facing a similar problem with private reps but I've found a workaround,

Using yarn upgrade {private_package_url} (yarn upgrade git+https://...) yarn is not updating yarn.lock to the last SHA.

Using yarn upgrade {package_name} updates to the last SHA.