Yarn: `yarn` fails to install with git url `git+ssh` - `npm install` works

[javinor]

Do you want to request a feature or report a bug?
Bug - I looked through the issues, and couldn't find one with is exactly the same, only closely related ones.

What is the current behavior?
yarn install fails with the following message:

yarn install v0.27.5
info No lockfile found.
[1/4] Resolving packages...
error Command failed.
Exit code: 128
Command: git
Arguments: clone [email protected]:lodash/lodash.git /Users/javinor/Library/Caches/Yarn/v1/.tmp/bbb8205227284249bc4db0af5b50715f
Directory: /Users/javinor/workspace/broken-yarn-install
Output:
Cloning into '/Users/javinor/Library/Caches/Yarn/v1/.tmp/bbb8205227284249bc4db0af5b50715f'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

If the current behavior is a bug, please provide the steps to reproduce.
In an empty dir, create the following package.json and run yarn install:

{
  "name": "broken-yarn-install",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "lodash": "git+ssh://[email protected]:lodash/lodash.git#4.17.4"
  }
}

What is the expected behavior?
Expected yarn to successfully install lodash in node_modules. npm install works fine

• I tried to change the colon : in the git-url to a slash /
• I tried adding ~/.ssh/config:

Host github.com
    User git

I tried all sorts of work arounds from similiar issues, to no avail.
* This works with yarn 0.24.6 *

Please mention your node.js, yarn and operating system version.
yarn - 0.27.5
node - 6.11.0
OSX - Sierra 10.12

[Volune]

I'm not reproducing (Yarn 0.27.5, Node 6.9.4, Windows 10, Git 2.13.0.windows.1)

• What git version do you have?
• With yarn 0.24.6, do you need to input a password or something?
• Can you try to set GIT_TRACE environment variable, with an absolute path, like
  GIT_TRACE=/tmp/git-trace.log yarn install
  Then check/share the logs

[javinor]

current git version 2.8.2, I don't input passwords, I use ssh keys for github.

git-trace.log:

10:54:57.864888 git.c:350               trace: built-in: git 'pull'
10:54:57.867729 run-command.c:336       trace: run_command: 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'master'
10:54:57.868525 exec_cmd.c:120          trace: exec: 'git' 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'master'
10:54:57.873664 git.c:350               trace: built-in: git 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'master'
10:54:57.877060 run-command.c:336       trace: run_command: 'fetch' '--update-head-ok'
10:54:57.877739 exec_cmd.c:120          trace: exec: 'git' 'fetch' '--update-head-ok'
10:54:57.882868 git.c:350               trace: built-in: git 'fetch' '--update-head-ok'
10:54:57.888053 run-command.c:336       trace: run_command: 'ssh -oBatchMode=yes' '[email protected]' 'git-upload-pack '\''lodash/lodash.git'\'''
10:54:57.888778 run-command.c:195       trace: exec: '/bin/sh' '-c' 'ssh -oBatchMode=yes "$@"' 'ssh -oBatchMode=yes' '[email protected]' 'git-upload-pack '\''lodash/lodash.git'\'''

I upgraded git (brew) to version 2.13.3, and reran the trace:

10:58:54.810790 git.c:322               trace: built-in: git 'pull'
10:58:54.813867 run-command.c:626       trace: run_command: 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'master'
10:58:54.819373 git.c:322               trace: built-in: git 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'master'
10:58:54.825204 run-command.c:626       trace: run_command: 'fetch' '--update-head-ok'
10:58:54.830997 git.c:322               trace: built-in: git 'fetch' '--update-head-ok'
10:58:54.839204 run-command.c:626       trace: run_command: 'ssh -oBatchMode=yes' '[email protected]' 'git-upload-pack '\''lodash/lodash.git'\'''

output of yarn config:

yarn config v0.27.5
info yarn config
{ 'version-tag-prefix': 'v',
  'version-git-tag': true,
  'version-git-sign': false,
  'version-git-message': 'v%s',
  'init-version': '1.0.0',
  'init-license': 'MIT',
  'save-prefix': '^',
  'ignore-scripts': false,
  'ignore-optional': false,
  registry: 'https://registry.yarnpkg.com',
  'strict-ssl': true,
  'user-agent': 'yarn/0.27.5 npm/? node/v6.11.0 darwin x64',
  lastUpdateCheck: 1500278025543 }
info npm config
{}
Done in 0.04s.

When changing the package json to:

{
  "name": "broken-yarn-install",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "lodash": "[email protected]:lodash/lodash.git#4.17.4"
  }
}

I get prompted by yarn, asking for which version to install, and once selecting, everthing install fine.

Any clues / ideas / leads?

[robinduckett]

I have the same issue, I am just after whatever release is in the master branch of a repo on github and it just pops up. Annoyingly, I have one that works, and one that doesn't. Maybe the tag name needs to match exactly? I tried #master and #{commit} and it didn't work

[BYK]

@arcanis - I think this is fixed on master now, right?

[arcanis]

I'm not sure - I can't reproduce this on 0.28.1, which is apparently what @robinduckett is using. Is it correct? Can you check the output of yarn --version?

[robinduckett]

0.27.5 -- I didn't say I was using any particular version

[arcanis]

Sorry then, I misunderstood :) Can you try using the 0.28.1?

[robinduckett]

Confirmed that it still fails at the 'Fetching packages' stage with 0.28.1. Anyway I can enable debugging and show you a log?

[arcanis]

I'm not sure which command would tell me enough to reproduce this on my side - and if I can't reproduce it, I won't be able to debug or validate my fixes 😞 Maybe two things:

• It seems like your issue is different than the one reported by @javinor - they had issue during the resolution step (the first one), whereas you seem to be affected during the fetching step (the second one).

• I would suggest trying to run ssh -oBatchMode=yes [email protected] git-upload-pack lodash/lodash.git manually and see if it works.

[deathweaselx86]

I am on OS X 10.12 Sierra and Yarn 0.28.0. I was getting this problem fetching modules from private repos. I was able to fix it by adding a new SSH key to Github and then adding the following to my .ssh/config

```
Host *
UseKeychain yes
AddKeysToAgent yes
IdentityFile ~/.ssh/mykey_rsa

[javinor]

@Volune , @BYK: git bisect run and I go way back. I cloned and bisected the project. This is the problematic commit: 27e9922e97f28db9d0151478e39133ba0cb46926

I tried using the above suggestion and added an ~/.ssh/config:

Host github.com
    UseKeychain yes 
    AddKeysToAgent yes 
    IdentityFile ~/.ssh/<mykey>_rsa

Any thing else I can do from here?

[javinor]

bisect log (for reference):

git bisect start
# good: [351384eca0999cec6bb5d0c4f6071eff91a146c3] 0.24.6
git bisect good 351384eca0999cec6bb5d0c4f6071eff91a146c3
# bad: [f7aa742776e4689c040fed7664fd8f4b695903b2] Fix various commands asking for a password (#3862)
git bisect bad f7aa742776e4689c040fed7664fd8f4b695903b2
# good: [19122357fb8f03896582c0d0f3de674adce872c5] v0.24.0
git bisect good 19122357fb8f03896582c0d0f3de674adce872c5
# bad: [f78fecc2d173599ed8413b87fad78041f35b5130] Update: Better comment around symlinks and workspaces (#3672)
git bisect bad f78fecc2d173599ed8413b87fad78041f35b5130
# good: [d57cce98efc3bb1b8a9768998cadb961a4381f91] Add explicit check for tarball files in mirror files directory (#3467)
git bisect good d57cce98efc3bb1b8a9768998cadb961a4381f91
# good: [f94bef3fdf6724f1310d069c49781cd6e0951f43] 0.26.0
git bisect good f94bef3fdf6724f1310d069c49781cd6e0951f43
# bad: [27e9922e97f28db9d0151478e39133ba0cb46926] Fix: prevent Git from showing a password prompt (#3633)
git bisect bad 27e9922e97f28db9d0151478e39133ba0cb46926
# good: [7515772eabc91b741244a4f19aa34dce0a5dc5f4] Fix: Don't do range collapsing when using a frozen lockfile (#3604)
git bisect good 7515772eabc91b741244a4f19aa34dce0a5dc5f4
# good: [0817ded0eedd1719ef96a3dd7e711bd3a8bf7fce] Enhancement: slightly optimize how we update status from running scripts (#3629)
git bisect good 0817ded0eedd1719ef96a3dd7e711bd3a8bf7fce
# good: [0361edf4b4bd330370bff1d57803f2d0ecd94273] Color fix (#3607)
git bisect good 0361edf4b4bd330370bff1d57803f2d0ecd94273
# first bad commit: [27e9922e97f28db9d0151478e39133ba0cb46926] Fix: prevent Git from showing a password prompt (#3633)

[BYK]

@javinor - Then I'd ask you the values of the following environment variables: GIT_ASKPASS, GIT_TERMINAL_PROMPT, `GIT_SSH_COMMAND'.

Also, we had merged #3885 recently which fixed an issue with the commit you raised. Try using that may be?

[javinor]

I don't have any of those values defined in my env.

Still not working from master - SHA f7aa742776e4689c040fed7664fd8f4b695903b2

[BYK]

@javinor then I honestly cannot understand how this can be related to that commit. Are you sure you haven't changed your GitHub credentials in the mean time? May be your tokens have expired etc.?

[arcanis]

Does running ssh -oBatchMode=yes [email protected] git-upload-pack lodash/lodash.git work?

[javinor]

I tried running the ssh command and it failed. Maybe you can help me decipher the debug log? I'm not sure exactly where things are going wrong...

$ ssh -v -oBatchMode=yes [email protected] git-upload-pack lodash/lodash.git
OpenSSH_7.2p2, LibreSSL 2.4.1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to github.com [192.30.253.113] port 22.
debug1: Connection established.
debug1: identity file /Users/javinor/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/javinor/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2
debug1: Remote protocol version 2.0, remote software version libssh-0.7.0
debug1: no match: libssh-0.7.0
debug1: Authenticating to github.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /Users/javinor/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/javinor/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 535
debug1: Trying private key: /Users/javinor/.ssh/id_dsa
debug1: Trying private key: /Users/javinor/.ssh/id_ecdsa
debug1: Trying private key: /Users/javinor/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).

[BYK]

I'd honestly try following these steps first: https://help.github.com/articles/error-permission-denied-publickey/

Looks like there's a confusion between your public and private SSH keys though.

[javinor]

When I omit the -oBatchMode flag, it works fine. I ran through the github troubleshooting you sent me, and for some reason, after running ssh-add -l it just started working.

Thanks very much for your quick help!

[BYK]

Glad we could help and thanks for trying all those and reporting back here!

[adambullmer]

To anyone else from the future like myself ssh-add -l -E md5 (for OpenSSH >= 6.8) seems to have cleared the issue up, while proclaiming that no identities were added. ¯\_(ツ)_/¯

Got the instructions from the above github ssh troubleshooting guide as well.

$ ssh-add -l -E md5
The agent has no identities.

[joris-rotteveel]

Adding the following to the ~/.shh/config did the trick for me

Host github.com
    User git

EDIT:

After a reboot, the above failed again. Colleague of mine suggested to ssh-add eacht ime you reboot.

[keego]

I believe I'm getting this issue as well. Just upgraded from yarn 1.6.0 to 1.9.4 and that fixed it for me

output

$ yarn
yarn install v1.6.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error Command failed.
Exit code: 128
Command: git
Arguments: pull
Directory: /Users/keegomyneego/Library/Caches/Yarn/v1/.tmp/0c3099ce0049074b4afd330c4eb871d8
Output:
fatal: Not a git repository (or any of the parent directories): .git
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

relevant package.json dependency

"ros3d": "[email protected]:keego/ros3djs#ES6_ONLY",

Also tried these formats:

"ros3d": "git+ssh://[email protected]:keego/ros3djs#ES6_ONLY",
"ros3d": "ssh://[email protected]:keego/ros3djs#ES6_ONLY",
"ros3d": "github.com:keego/ros3djs#ES6_ONLY",
"ros3d": "keego/ros3djs#ES6_ONLY",

Also tried the above formats with ros3djs.git to no avail.

versions

yarn - 1.6.0
node - v8.9.4
OSX - Sierra 10.12

[nicollecastrog]

@keego which format did you use in the end?