Yarn: Yarn doesn't fetch all remote refs from git repositories

[a-rodin]

Do you want to request a feature or report a bug?

Incompatibility with npm (probably a bug)

What is the current behavior?

By default yarn doesn't fetch all remote refs from git repositories. When it takes a git URL with a commit hash it does something equivalent to

git clone git://github.com/LearnBoost/websocket.io
cd websocket.io
git checkout 5f0b3799f50944bc4d1eb58acdef61c38883d6cf # fails

The commit 5f0b3799f50944bc4d1eb58acdef61c38883d6cf is from a pull request and by default it is not fetched by git, so this command would fail.

On the other hand, npm does something equivalent to

git clone git://github.com/LearnBoost/websocket.io
cd websocket.io
git fetch --all
git checkout 5f0b3799f50944bc4d1eb58acdef61c38883d6cf # works

If the current behavior is a bug, please provide the steps to reproduce.

$ yarn global add "git://github.com/LearnBoost/websocket.io#5f0b3799f50944bc4d1eb58acdef61c38883d6cf"
yarn global v0.21.3
warning No license field
[1/4] Resolving packages...
error Couldn't find manifest in "git://github.com/LearnBoost/websocket.io".
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.

Originally I found this by trying to do yarn global add markserv which uses this as a dependency.

What is the expected behavior?

$ npm install -g  "git://github.com/LearnBoost/websocket.io#5f0b3799f50944bc4d1eb58acdef61c38883d6cf"
/home/user/.nvm/versions/node/v6.10.0/lib
└── [email protected]  (git://github.com/LearnBoost/websocket.io.git#5f0b3799f50944bc4d1eb58acdef61c38883d6cf)

(installs without errors)

Please mention your node.js, yarn and operating system version.

$ yarn --version
0.21.3
$ node --version
v6.10.0
$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

[XaserAcheron]

I managed to run into the same error through the following sequence of events:

• yarn add https://github.com/XaserAcheron/xample.git#1.0.0
• Push a new tag, v1.0.1 (with corresponding package.json) to the xample repo
• yarn add https://github.com/XaserAcheron/xample.git#1.0.1

The second yarn add fails with"error Couldn't find manifest in "https://github.com/XaserAcheron/xample.git", presumably because it doesn't re-fetch the tags. The symptoms match the OP's at least.

yarn cache clean fixes the problem, but that's nuclear.

[note: xample isn't a real repo; just an anonymized [e]xample ;]

[josephfrazier]

I was also able to reproduce this:

$ yarn add https://github.com/mozilla/webextension-polyfill#99faff94ffbf03094361875a10e1692acc4b8662
yarn add v0.21.3
[1/4] 🔍  Resolving packages...
error Couldn't find manifest in "https://github.com/mozilla/webextension-polyfill".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

npm works as expected:

$ npm install --save https://github.com/mozilla/webextension-polyfill#99faff94ffbf03094361875a10e1692acc4b8662
/private/tmp/package
└── [email protected]  (git+https://github.com/mozilla/webextension-polyfill.git#99faff94ffbf03094361875a10e1692acc4b8662)

[arcanis]

We closed the related pull request, so I'm going to close this issue as well. Short story is: you probably should use forks urls rather than the main repo urls 😃

[XaserAcheron]

It still happens with tags on the main repo (see my previous comment), which is particularly problematic for private Git repositories since there's no other way to reference a particular tagged version (that I know of).

[arcanis]

Hm, I see. It seems it's a different issue from the one described in the OP (if I understand correctly, initial tags are correctly fetched, but not correctly updated, right?). Would you mind opening a new issue?

[XaserAcheron]

I can't seem to reproduce it in a test repo. Not sure if it was sneakily fixed in v0.23.3 or if it's a very odd set of circumstances that lead up to it.

I'll post an issue if I run across it again, but not until then. If anyone else has some insight, that'd be helpful.

[recaro]

With yarn v0.24.6 issue is still present:

$ yarn add git://github.com/LearnBoost/websocket.io#5f0b3799f50944bc4d1eb58acdef61c38883d6cf --dev
yarn add v0.24.6
[1/5] Resolving packages...
error Couldn't find manifest in "git://github.com/LearnBoost/websocket.io".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

[mrsum]

Got same error last week. Yarn v0.24.4

[BYK]

@recaro @mrsum can you try one of the latest nightlies to see if it fixes your problem?

[cherealnice]

I ran into the same issue, yarn cache clean seemed to fix it, @recaro have you tried that?