Yarn: Bug: Yarn fails to install when a custom SSH command is used for Git via GIT_SSH

[matthijs-bakker]

Do you want to request a feature or report a bug?

Report a bug

What is the current behavior?
Yarn fails on dependencies on our private gitlab server.

Error message:

yarn install v0.27.5
[1/4] Resolving packages...
error Command failed.
Exit code: 128
Command: git
Arguments: clone ssh://[email protected]/my-package/my-package.git C:\Users\matthijsba\AppData\Local\Yarn\cache\v1\.tmp\3abb1dfe087a6eb779c097328e5514bb
Directory: C:\source\my-app
Output:
Cloning into 'C:\Users\matthijsba\AppData\Local\Yarn\cache\v1\.tmp\3abb1dfe087a6eb779c097328e5514bb'...
Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

If the current behavior is a bug, please provide the steps to reproduce.
No real reproduction steps. If I run:

git clone ssh://[email protected]/my-package/my-package.git this works without any issues

If I run npm i this will install the dependencies.

What is the expected behavior?
The dependencies will be installed

Please mention your node.js, yarn and operating system version.
Node: 8.3.0 / 6.11.2 (tested both)
Yarn: 0.27.5 / 0.28.1 (tested both)
OS: Windows 10 version 1703

I have read other issues like #513 and #686 but I could not find a solution yet.

[migeorge]

I a similar error with Gitlab and macOS, I solved it by adding my ssh key to the ssh agent (via ssh-add). Maybe there is something similar on Windows?

Unfortunately after fixing this error I was greeted with the error described in #4172

[matthijs-bakker]

Hi @migeorge I think I have a different problem; I have had the same problem as you describe on a Mac. But on my windows machine I use pageant / plink to authenticate and I have no authentication issues when I git pull or when I run the same git command which leads to yarn's error.

[matthijs-bakker]

I just downgraded to 0.23.4 (since a colleague is using this version) and I can yarn without issues.

[BYK]

@matthijs-bakker can you try one of the latest nightlies because there were changes around this both in 0.28.x line and afterward.

[matthijs-bakker]

Hi @BYK,

Thanks for your reply. 0.28.4 did not have this issue. Thanks!

Trolled by cache; 0.28.4 has this issue and the issue is also not resolved in nightly: v1.0.0-20170815.1354.

[matthijs-bakker]

Last version that does not have this issue is 0.26.1

[BYK]

@matthijs-bakker closing since this is fixed on latest master and latest release.

[matthijs-bakker]

Hi @BYK , sorry for the confusion but this is not fixed. See the test results for different versions below:
✅ 0.23.4
✅ 0.26.1
❌ 0.27.5
❌ 0.28.4
❌ v1.0.0-20170815.1354
❌ v1.0.0-20170817.1516

[BYK]

@matthijs-bakker I think this is due to running SSH/Git in batch mode. Are you sure your credentials are automatically usasble by SSH when it is invoked in batch mode?

[matthijs-bakker]

@BYK sorry I have no knowledge or experience about batch mode. It looks like yarn is unable to use plink to authenticate due to some sort of isolation.

[BYK]

@matthijs-bakker may be this will help: https://serverfault.com/questions/420526/auto-storing-server-host-key-in-cache-with-plink?

[matthijs-bakker]

plink [email protected] and plink -batch [email protected] both work; I am not sure if I am testing the right cases. I am happy to test more but I don't have much knowledge about plink/ssh.

[BYK]

@matthijs-bakker this may be us forcing using ssh instead of plink for older versions of git. What's your git version?

[matthijs-bakker]

2.14.1.windows.1

[matthijs-bakker]

@BYK let me know if there is anything else I can do to test / analyze this issue.

[BYK]

@matthijs-bakker so I think the culprit is #3633. That said there were a few follow ups like #3885.

If you can figure anything out from that, great!

[matthijs-bakker]

@BYK I can confirm that after running
git config core.sshCommand "ssh -oBatchMode=yes" my git commands stop working with our repo's

I immediately get:

git pull
Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[BYK]

@matthijs-bakker awesome, thanks for the help! What is the original value for that field?

[matthijs-bakker]

@BYK The field did not exist.

[BYK]

@matthijs-bakker there should be a way to tell git to use plink instead of ssh. May be we can get it with git config --list in your project directory?

Be careful when sharing the output to not reveal any confidential information about your project or company tho :)

[matthijs-bakker]

Hi @BYK ,

Git wil use plink if an environment variable GIT_SSH is set with the path to plink.exe. So in my case the value is C:\Program Files (x86)\PuTTY\plink.exe.

I think the issue is that yarn does not use this variable but explicitly calls ssh -oBatchMode=yes

[BYK]

I think the issue is that yarn does not use this variable but explicitly calls ssh -oBatchMode=yes

Yes, seems like it. I'll try to find a good workaround for this. Thanks a lot for all the help with the investigation.

[BYK]

@matthijs-bakker okay, so this is the crucial part:

$GIT_SSH_COMMAND takes precedence over $GIT_SSH

So we are simply overriding this. I need to add an explicit check for GIT_SSH. I'll get this ready for 1.0. In the meantime, you can try setting $GIT_SSH_COMMAND to your $GIT_SSH variable.

[matthijs-bakker]

Hi @BYK , good news that a solution is coming! Settting the GIT_SSH_COMMAND system variable does not work.

[BYK]

Settting the GIT_SSH_COMMAND system variable does not work.

Interesting. Anyway, the fix is at #4302. If it is not too hard for you, I'd appreciate giving that branch a try. You can try this single-file build: https://5062-49970642-gh.circle-artifacts.com/0/home/ubuntu/yarn/artifacts/yarn-legacy-1.0.0.js

[matthijs-bakker]

@BYK

Just tested; it seems to get further but still errors:

λ  yarn
yarn install v1.0.0
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
error Command failed.
Exit code: 128
Command: git
Arguments: ls-remote --tags --heads ssh://[email protected]/my-package/my-package.git
Directory: C:\source\my-package
Output:
C:\Program Files (x86)\PuTTY\plink.exe -batch: -c: line 0: syntax error near unexpected token `('
C:\Program Files (x86)\PuTTY\plink.exe -batch: -c: line 0: `C:\Program Files (x86)\PuTTY\plink.exe -batch "$@"'
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[BYK]

@matthijs-bakker thanks! I'm pretty sure this is due to the path having spaces in it. I'll wrap it in quotes and see what happens.

[matthijs-bakker]

Windows 😍

[BYK]

❤️ Windows

Kind of agree but I'd say, we'd have had the same issue if there were spaces on Unix systems since seems like $GIT_SSH is used as the process path and $GIT_SSH_COMMAND is treated as a shell command.

[BYK]

@matthijs-bakker how about this one: https://5064-49970642-gh.circle-artifacts.com/0/home/ubuntu/yarn/artifacts/yarn-legacy-1.0.0.js

[matthijs-bakker]

@BYK works! 💯

[BYK]

@matthijs-bakker thank you so much since I couldn't have done this without your help in debugging the issue!

You'll be helping so many people with this fix :)