Vscode-remote-release: New Update Seems to Have Broken my Remote SSH Connection

[JSkylerNuth]

Issue Type: Bug

Still trying to test if the update was indeed the culprit but all signs point to it since it was working fine Monday and only change has been the automatic update that apparently took place that same day and would've restarted with the reboot it did over the night.

VS Code version: Code 1.43.0 (78a4c91400152c0f27ba4d363eb56d2835f9903a, 2020-03-09T19:47:57.235Z)
OS version: Windows_NT x64 10.0.17763
Fetching remote diagnostics for 'SSH' failed: Could not resolve hostname

Extensions: none

[vscodebot[bot]]

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• Cant open remote ssh (#92606)
  

[roblourens]

Can you share the log from the Remote-SSH output channel?

Does this help?

"remote.SSH.useLocalServer": false,
"remote.SSH.windowsRemotes": ["<your remote hostname>"] // only if the remote is Windows

[biltongza]

I got this as well. I went down a wild goose chase trying to figure out what went wrong. My SSH host is a Linux VM running on my own machine and I wound up resetting its network completely. I eventually just added its IP to my hosts file to get it to connect.

However, now public key authentication is not working unless I set remote.SSH.useLocalServer to false. If I set that to true, it always asks me for my password on the host (which it always rejects even with the correct password). Something is very unhappy in this latest update.

Logs with useLocalServer set to false:

[14:10:06.308] Log Level: 2
[14:10:06.330] [email protected]
[14:10:06.330] win32 x64
[14:10:06.342] SSH Resolver called for "ssh-remote+archvm", attempt 1
[14:10:06.343] SSH Resolver called for host: archvm
[14:10:06.343] Setting up SSH remote "archvm"
[14:10:06.419] Using commit id "78a4c91400152c0f27ba4d363eb56d2835f9903a" and quality "stable" for server
[14:10:06.424] Install and start server if needed
[14:10:06.435] Checking ssh with "ssh -V"
[14:10:06.588] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
[14:10:06.591] Using SSH config file "C:\Users\logan.dam\.ssh\config"
[14:10:06.592] Running script with connection command: ssh -T -D 52520 -F "C:\Users\logan.dam\.ssh\config" archvm bash
[14:10:06.603] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[14:10:07.132] > 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ]0;C:\WINDOWS\System32\cmd.exe
[14:10:07.132] Got some output, clearing connection timeout
[14:10:07.151] > 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
[14:10:07.408] > Warning: Permanently added the ECDSA host key for IP address 'fe80::215:5dff:fec
> a:8%13' to the list of known hosts.
> 
[14:10:07.608] > Warning: Permanently added the ECDSA host key for IP address 'fe80::215:5dff:feca:8%13' to the list of known hosts.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
[14:10:08.656] > ee42a7308182: running
> 
[14:10:08.970] > Acquiring lock on /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a
> 
[14:10:08.988] > \ln /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a.target /home/logan/.vscode-server/bin/78a4c
> 91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a
> 
[14:10:09.001] > Found existing installation at /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a...
> 
[14:10:09.075] > SHELL=/usr/bin/zsh
> PWD=/home/logan
> LOGNAME=logan
> XDG_SESSION_TYPE=tty
> HOME=/home/logan
> VSCODE_AGENT_FOLDER=/home/logan/.vscode-server
> SSH_CONNECTION=fe80::9e2:3a01:f362:6c42%eth0 52522 fe80::215:5dff:feca:8%eth0 22
> XDG_SESSION_CLASS=user
> USER=logan
> SHLVL=1
> XDG_SESSION_ID=1
> XDG_RUNTIME_DIR=/run/user/1000
> SSH_CLIENT=fe80::9e2:3a01:f362:6c42%eth0 52522 22
> PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
> DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> MAIL=/var/spool/mail/logan
> OLDPWD=/home/logan
> _=/usr/bin/printenv
> 
[14:10:09.082] > Starting server with command... /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/server.sh --host=127.0.0.1 --enable-remote-auto-shutdown  --port=0 &> "/home/loga
> n/.vscode-server/.78a4c91400152c0f27ba4d363eb56d2835f9903a.log" < /dev/null
> Waiting for server log...
> 
[14:10:11.366] > Waiting for server log...
> Waiting for server log...
> Waiting for server log...
[14:10:11.379] > 
> 
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
> 
[14:10:11.380] > 
> ee42a7308182: start
> sshAuthSock====
> agentPort==40699==
> osReleaseId==arch==
> arch==x86_64==
> webUiAccessToken====
> tmpDir==/run/user/1000==
> platform==linux==
> ee42a7308182: end
[14:10:11.381] Received install output: 
sshAuthSock====
agentPort==40699==
osReleaseId==arch==
arch==x86_64==
webUiAccessToken====
tmpDir==/run/user/1000==
platform==linux==

[14:10:11.382] Remote server is listening on port 40699
[14:10:11.383] Parsed server configuration: {"agentPort":40699,"osReleaseId":"arch","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1000","platform":"linux"}
[14:10:11.386] Starting forwarding server. localPort 52525 -> socksPort 52520 -> remotePort 40699
[14:10:11.388] Forwarding server listening on 52525
[14:10:11.388] Waiting for ssh tunnel to be ready
[14:10:11.391] > 
> 
[14:10:11.393] Tunneled remote port 40699 to local port 52525
[14:10:11.393] Resolved "ssh-remote+archvm" to "127.0.0.1:52525"
[14:10:11.395] [Forwarding server 52525] Got connection 0
[14:10:11.403] ------




[14:10:11.422] [Forwarding server 52525] Got connection 1
[14:10:11.425] [Forwarding server 52525] Got connection 2
[14:11:02.155] > Starting server with command... /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/server.sh --host=127.0.0.1 --enable-remote-auto-shutdown  --port=0 &> "/home/loga
> n/.vscode-server/.78a4c91400152c0f27ba4d363eb56d2835f9903a.log" < /dev/null
> Waiting for server log...
> Waiting for server log...
> Waiting for server log...
> Waiting for server log...
> 
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
> 
> ee42a7308182: start
> sshAuthSock====
> agentPort==40699==
> osReleaseId==arch==
> arch==x86_64==
> webUiAccessToken====
> tmpDir==/run/user/1000==
> platform==linux==
> ee42a7308182: end
> 

Logs with useLocalServer true:

[14:18:29.195] Log Level: 2
[14:18:29.203] [email protected]
[14:18:29.203] win32 x64
[14:18:29.205] SSH Resolver called for "ssh-remote+archvm", attempt 1
[14:18:29.206] SSH Resolver called for host: archvm
[14:18:29.206] Setting up SSH remote "archvm"
[14:18:29.216] Acquiring local install lock: C:\Users\logan.dam\AppData\Local\Temp\vscode-remote-ssh-archvm-install.lock
[14:18:29.262] Looking for existing server data file at c:\Users\logan.dam\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-ssh\vscode-ssh-host-archvm-78a4c91400152c0f27ba4d363eb56d2835f9903a\data.json
[14:18:29.306] Using commit id "78a4c91400152c0f27ba4d363eb56d2835f9903a" and quality "stable" for server
[14:18:29.316] Install and start server if needed
[14:18:29.332] Checking ssh with "ssh -V"
[14:18:29.451] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
[14:18:29.452] Checking ssh with "C:\WINDOWS\System32\OpenSSH\ssh.exe -V"
[14:18:29.521] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
[14:18:29.522] Checking ssh with "C:\Program Files\Git\usr\bin\ssh.exe -V"
[14:18:29.620] > OpenSSH_8.2p1, OpenSSL 1.1.1d  10 Sep 2019
[14:18:29.622] Using SSH config file "C:\Users\logan.dam\.ssh\config"
[14:18:29.625] askpass server listening on \\.\pipe\vscode-ssh-askpass-4a8901f94694ce7d76ccafef224d46ee075a4a3d-sock
[14:18:29.626] Spawning local server with {"ipcHandlePath":"\\\\.\\pipe\\vscode-ssh-askpass-bd93d123af9080817c540d4a3a3cec303c23ecb5-sock","sshCommand":"C:\\Program Files\\Git\\usr\\bin\\ssh.exe","sshArgs":["-v","-T","-D","52872","-F","C:\\Users\\logan.dam\\.ssh\\config","archvm"],"dataFilePath":"c:\\Users\\logan.dam\\AppData\\Roaming\\Code\\User\\globalStorage\\ms-vscode-remote.remote-ssh\\vscode-ssh-host-archvm-78a4c91400152c0f27ba4d363eb56d2835f9903a\\data.json"}
[14:18:29.627] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"c:\\Users\\logan.dam\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.50.0\\out\\local-server\\askpass.bat","VSCODE_SSH_ASKPASS_NODE":"C:\\Program Files\\Microsoft VS Code\\Code.exe","VSCODE_SSH_ASKPASS_MAIN":"c:\\Users\\logan.dam\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.50.0\\out\\askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"\\\\.\\pipe\\vscode-ssh-askpass-4a8901f94694ce7d76ccafef224d46ee075a4a3d-sock"}
[14:18:30.513] Spawned 12068
[14:18:30.650] > local-server> Spawned ssh: 17140
[14:18:30.721] stderr> OpenSSH_8.2p1, OpenSSL 1.1.1d  10 Sep 2019
[14:18:30.813] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:BZ8q/NnCGN0bMYLAPnk9Efbsqn3thhtD8j+9u1cMzjY
[14:18:30.816] stderr> Warning: Permanently added the ECDSA host key for IP address 'fe80::215:5dff:feca:8%13' to the list of known hosts.
[14:18:31.862] Got askpass request: {"request":"logan@arch's password: "}
[14:18:31.863] Showing password prompt
[14:18:31.865] Listening for interwindow password on \\.\pipe\vscode-ssh-askpass-ae16c508cce1046dd9b5b7137c732d0ac8d572a8-sock
[14:18:31.865] Writing password prompt to globalState
[14:18:59.025] Got password response
[14:18:59.025] Interactor gave response: ********
[14:18:59.026] Cleaning up other-window auth server
[14:18:59.095] stderr> Authenticated to arch ([fe80::215:5dff:feca:8%13]:22).
[14:18:59.157] > ready: c54f591ab12c
[14:18:59.161] > Linux 5.5.9-2-ck-skylake #1 SMP PREEMPT Sat, 14 Mar 2020 20:58:46 +0000
[14:18:59.164] Platform: linux
[14:18:59.182] > c54f591ab12c: running
[14:18:59.233] > Acquiring lock on /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a
> \ln /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a.target /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a/vscode-remote-lock.logan.78a4c91400152c0f27ba4d363eb56d2835f9903a
[14:18:59.236] > Found existing installation at /home/logan/.vscode-server/bin/78a4c91400152c0f27ba4d363eb56d2835f9903a...
[14:18:59.247] > Found running server...
>  
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
>  
[14:18:59.265] > Checking server status on port 40699 with curl
[14:18:59.274] > c54f591ab12c: start
> sshAuthSock====
> agentPort==40699==
> osReleaseId==arch==
> arch==x86_64==
> webUiAccessToken====
> tmpDir==/run/user/1000==
> platform==linux==
> c54f591ab12c: end
[14:18:59.274] Received install output: 
sshAuthSock====
agentPort==40699==
osReleaseId==arch==
arch==x86_64==
webUiAccessToken====
tmpDir==/run/user/1000==
platform==linux==

[14:18:59.276] Remote server is listening on port 40699
[14:18:59.276] Parsed server configuration: {"agentPort":40699,"osReleaseId":"arch","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1000","platform":"linux"}
[14:18:59.282] Persisting server connection details to c:\Users\logan.dam\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-ssh\vscode-ssh-host-archvm-78a4c91400152c0f27ba4d363eb56d2835f9903a\data.json
[14:18:59.286] Starting forwarding server. localPort 52878 -> socksPort 52872 -> remotePort 40699
[14:18:59.288] Forwarding server listening on 52878
[14:18:59.288] Waiting for ssh tunnel to be ready
[14:18:59.290] Tunneled remote port 40699 to local port 52878
[14:18:59.290] Resolved "ssh-remote+archvm" to "127.0.0.1:52878"
[14:18:59.292] [Forwarding server 52878] Got connection 0
[14:18:59.310] ------




[14:18:59.342] [Forwarding server 52878] Got connection 1
[14:18:59.344] [Forwarding server 52878] Got connection 2

[tsquillario]

rm -rf ~/.vscode-server/ on the remote machine fixed it for me

[jghal]

I just started seeing it and believe the issue is this bit

[16:45:43.849] Checking ssh with "ssh -V"
[16:45:44.140] > OpenSSH_for_Windows_8.0p1, LibreSSL 2.6.5
[16:45:44.140] Checking ssh with "C:\Windows\System32\OpenSSH\ssh.exe -V"
[16:45:44.142] Got error from ssh: spawn C:\Windows\System32\OpenSSH\ssh.exe ENOENT
[16:45:44.142] Checking ssh with "C:\Program Files\Git\usr\bin\ssh.exe -V"
[16:45:45.078] > OpenSSH_8.2p1, OpenSSL 1.1.1d  10 Sep 2019
[snip]
[16:45:45.455] Spawned 1752
[16:45:45.900] > local-server> Spawned ssh: 8332
[16:45:46.235] stderr> OpenSSH_8.2p1, OpenSSL 1.1.1d  10 Sep 2019

So it's using the ssh executable bundled in Git for Windows, when I actually want to use the native OpenSSH for Windows (the _C:\Program Files\OpenSSH-Win64ssh.exe_ 8.0 version that is the only version in my PATH and that was discovered via ssh -V). I have my agent and .ssh\config setup for that native OpenSSH and instead it's using the Git for Windows copy and asking for my key passphrase and mostly failing, sometimes it connects but then it's not forwarding my agent like I have configured so Git operations don't work.

Setting remote.SSH.useLocalServer: false fixes it but that is a rather unintuitive way to control what SSH _client_ is being used to connect to the remote.

[biltongza]

@jghal I can confirm that using the ssh client from git for windows doesn't appear to use my public key for auth. SSH bundled with windows does.

C:\Users\logan.dam> cd '..\..\Program Files\Git\usr\bin'
C:\Program Files\Git\usr\bin> .\ssh.exe archvm
logan.dam@archvm's password:

Update: Using suggestion from https://github.com/microsoft/vscode-remote-release/issues/2547#issuecomment-597967652 has resolved this for me:

1. Specify the absolute path of your config file in the settings of Remote SSH extension
2. Specify the absolute path of your OpenSSH ssh.exe executable in the settings of Remote SSH extension. For me the path was C:\Windows\System32\OpenSSHssh.exe

[nick-walt]

Since the latest update (April 2020) VSCode has been unable to connect to a remote host. It seems to be wanting to re-install itself on the Ubuntu host and I guess that might be due to the update. I'm wondering if updating its server installation is failing.

I've rebooted the Ubuntu host without any success.

Final output:

[12:22:29.216] > c59bc771ae6d: running
> 
[12:22:29.225] > Acquiring lock on /home/ubuntu/.vscode-server/bin/ff915844119ce9485abfe8aa9076ec
> 76b5300ddd/vscode-remote-lock.ubuntu.ff915844119ce9485abfe8aa9076ec76b5300ddd   
> \ln /home/ubuntu/.vscode-server/bin/ff915844119ce9485abfe8aa9076ec76b5300ddd/vsc
> ode-remote-lock.ubuntu.ff915844119ce9485abfe8aa9076ec76b5300ddd.target /home/ubu
> ntu/.vscode-server/bin/ff915844119ce9485abfe8aa9076ec76b5300ddd/vscode-remote-lo
> ck.ubuntu.ff915844119ce9485abfe8aa9076ec76b5300ddd
> Installation already in progress...
> c59bc771ae6d##24##
> ln: failed to create hard link '/home/ubuntu/.vscode-server/bin/ff915844119ce948
> 5abfe8aa9076ec76b5300ddd/vscode-remote-lock.ubuntu.ff915844119ce9485abfe8aa9076e
> c76b5300ddd': File exists
> 
[12:22:29.225] Received install output: c59bc771ae6d##24##
[12:22:29.225] Server installation process already in progress - waiting and retrying
[12:22:29.494] "install" terminal command done
[12:22:29.494] Install terminal quit with output: 
[12:22:30.226] Resolver error: 
[12:22:30.228] ------

[github-actions[bot]]

Hey @roblourens, this issue might need further attention.

@JSkylerNuth, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[github-actions[bot]]

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!