Vscode-cpptools: debugee fails to launch with native vscode-cpptools on WSL

[therealkenc]

VSCode 1.29.1 and vscode-cpptools 0.20.1. No other extensions.
Confirmed with builds 18282 and 17134.

Ref #2811 and WSL#3679. This is with pure native Linux VS Code on WSL. Application fails to launch.

1. Go to Debug
2. Click on Start Without Debugging

launch.json here and test code here for pedantry. gcc -g hello.c.

Continuing from #2811 ...

When I read the issue you reference, you are running VS Code and extension natively inside WSL correct?

Correct. There is no win32 VSCode or extension involved in the scenario.

If so, then OpenDebugAD7 should work fine as it would think its running on Linux and download the correct bits and go through its Ubuntu checks.

You'd think so. But vscode-cpptools stopped working on WSL recently. I haven't been able to dissect exactly what "recently" means yet, but I suspect it is 0.20.0+ (or we would have heard about it earlier).

This is going to be due to either:

1. a change in vscode-cpptools awoke some gap in the WSL syscall emulation
2. vscode-cpptools is now making some assumptions about the Ubuntu userspace where Ubuntu on WSL isn't _exactly_ like Ubuntu on Real Linux [this can happen unfortunately, f.e. we don't have systemd(1) or real login(1)] or
3. something I haven't thought of yet.

I'll start bisecting and get you logs. I haven't deep dove yet. In the mean time any wild ass guesses as to what might have changed are appreciated. All things being equal, yes, this is a "it should work" situation. It did, now it doesn't. Plot is figuring out why.

[ed: Outline of installing native VSCode on WSl [here](https://gist.github.com/therealkenc/c7912035d57eb9df18225cd529461ed9) if you feel like playing along.]

[pieandcakes]

@therealkenc Can you try and change externalConsole to false in your launch.json and see if you get the same experience?

difference here would be we are now relying on VS Code to do all the terminal work such as launching it. I'm wondering if this is part of the problem and would at least give us some indication on which path its going down.

[pieandcakes]

[ed: Outline of installing native VSCode on WSl here if you feel like playing along.]

I'll take a look when I have some time to set up an environment. Thanks for the instructions.

[therealkenc]

Can you try and change externalConsole to false

I've tried all the usual suspects. Ditto on the "some time" to get you whatever logs are available (I haven't enabled logging before and need to look into it). Appreciate the reply; thanks.

[therealkenc]

I think I have a half-educated guess. In launch.json there is the externalConsole: true/false dial to tweak, but nothing to specify where the external console should be found. My (entirely speculative) guess is that VSCode faceplants if it can't find or spawn the terminal emulator.

To be clear, it doesn't matter whether externalConsole is set to false for me (unlike some but not all of the folks in #2842). But the details of the terminal emulator is a variable along the lines of possibility (2) in my OP. Something like that could different for users of not-Ubuntu distributions like Fedora or OpenSUSE as well. It isn't likely that the VSCode peeps tried all the distros out there and thought nothing of it when it all worked for them on Ubuntu. In any case what we are seeing here is with almost near certainty going to be the same as #2842. As you said earlier, VSCode, and for that matter all of Ubuntu userspace, doesn't know it is running in WSL.

The only problem with this line of reasoning is I tried installing gnome-terminal and xterm on WSL to align as close as possible with a typical Ubuntu install Real Linux. /usr/bin/x-terminal-emulator from the bash commandline (technically in Windows Console) launches fine. Didn't help. Sigh.

[ed] To put it maybe a better way: The people over in #2842 who managed to work around the problem by setting externalConsole: false don't have an answer as to why : true isn't working for them (when presumably it used to two weeks ago). Find the answer to that question and I wager you've got the answer to my woes. That I'm on 'WSL' doesn't matter.

[pieandcakes]

@therealkenc the code to use gnome-terminal within our extension has been removed as of 0.20.0 in favor of relying on VS Code to handle terminal launching. If VS Code has a requirement for gnome-terminal or xterm that would be surprising.

[therealkenc]

They wouldn't. Speculated maybe they would use x-terminal-emulator which is the common apt-alternative path for the default terminal emulator. But even that is a Ubuntu/Debian construct, which wouldn't necessarily apply to other distros which wouldn't know an apt-alternative from a deep hole in the ground, because they use rpm or pacman or zypper (or please someone shoot me).

[pieandcakes]

@therealkenc ah ok. I understand now. for #2842, that is something I need to investigate also. I'll let you know what I find there.

[therealkenc]

I'll let you know what I find there.

Thanks. I think maybe ping the people upstream who wrote the "handle terminal launching" to which you are now deferring but were doing yourself previously (Daniel maybe?). Perhaps this isn't even your wheelhouse. But one unanswered question is why this isn't taking down _all_ the debug extensions, but demonstrably affects cpptools. The guy who submitted WSL#3679 claims Python debugging is fine (I don't 'do' Python and couldn't tell you).

But in any case, I am pretty convinced there is going to be a pretty straightforward explanation if the right people are asked.

[I'm also pretty convinced the same underlying problem is what is borking the #2811 scenario. I am not a big believer in coincidences, and these reports all came out within days of the 1.29 VS Code release. But we can revisit 2811 later.]

[therealkenc]

Rolling back to a previous version of vscode-cpptools addresses the problem, so the WSL peeps have a work-around for now. I posted details in the WSL tracker. I'm going to sit on this for a bit because me deep-diving the cause is going to be a lot less time efficient than someone familiar with the launch mechanism. Thanks for all your help Pierson.

[tommyvct]

As of January 8, 2019, there is still no newer version of 'vscode-cpptools', hence the issue still there.

[therealkenc]

Yes it is the New Year. We're presently at an impasse on this one (and #2842) because turning on debug traces doesn't yield any meaningful insight. At launch the control flow black-holes inside vscode proper and vscode-cpptools doesn't even get the chance to fail.

In the absence of someone familiar with the changes made in 0.20 doing a deep dive, it would perhaps be constructive to roll back the launch changes and go back to the "old" launch method temporarily (whatever it was) for the next roll. Other language debug extensions (nodejs, Python) seem to be okay. At least that will unblock folks, in the absence of another solution.

[sunyinqi0508]

This issue persists on the latest version of cpptools built directly from GitHub code with code-insiders 1.31.0. However I found some clue by debugging the extension sources. The exception is listed below. It seems the debug adapter failed to spawn.

bootstrap-window.js:202 [uncaught exception]: Error: spawn /c/Users/sunyi/vscode-cpptools/Extension/debugAdapters/OpenDebugAD7 ENOENT
onUnexpectedError @ bootstrap-window.js:202
(anonymous) @ bootstrap-window.js:38
emit @ events.js:182
window.onerror @ /usr/share/code-insi…enderer/init.js:148

bootstrap-window.js:205 Error: spawn /c/Users/sunyi/vscode-cpptools/Extension/debugAdapters/OpenDebugAD7 ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:229)
at onErrorNT (internal/child_process.js:406)
at process._tickCallback (internal/process/next_tick.js:63)
onUnexpectedError @ bootstrap-window.js:205
(anonymous) @ bootstrap-window.js:38
emit @ events.js:182
window.onerror @ /usr/share/code-insi…enderer/init.js:148

My current workaround is by using a third-party extension called Native Debug for C++ debugging. Don't know if anyone have better solutions.

[therealkenc]

bootstrap-window.js:205 Error: spawn /c/Users/sunyi/vscode-cpptools/Extension/debugAdapters/OpenDebugAD7 ENOENT

_Please_ do your test inside /home. If you invite /c or /mnt/c to the party it is going to confuse everyone. WSL should look exactly like a Real Linux box.

This said, are you getting the same behavior with your private build -- which is a no error popup just the red stop square sitting there but no launch (contrast some kind of error notification)? You have a window.onerror() there, but in the behavior reported there is no popup. No anything.

The problem has to do with launching the terminal (internal _or_ external) not OpenDebugAD7. That's what changed in ~0.20. That's the change I am hoping will be rolled back absent resolution. Cpptools shouldn't have any problem _finding_ OpenDebugAD7, unless the mechanism for finding it changed.

To reiterate in case it is being lost: Launching _used to work_ for a year until the ~0.20 release. It is not vscode proper that changed either, because if I hold at cpptools 0.18.1 it is fine, despite VS Code marching forward for over a month.

[sunyinqi0508]

I figured it out. The error that I was getting is probably due to my build configuration.

Good news is I tried the latest insider build (0.21.0-insider3) of this cpptools extension, it seems to work correctly for me with "externalConsole" set to false. However, it will get the same problem ( no error, red stop button, blue indicator running ) when using external terminal.

[sunyinqi0508]

The 0.21.0 update was released. This issue seems to be solved.

[therealkenc]

The _internal_ console appears to be working again in 0.21.0. Thanks.

The _external_ console may still result in the reported behavior if vscode can't launch the default terminal program (typically gnome-terminal on Ubuntu). The reported behavior being the small red square (ie stop) button with no error reported. This is going to still be the case on other distributions where the external console can't be launched under whatever assumptions VS Code makes about launching it, per #2842.

You can reproduce this fairly easily on 0.21.0 with VS Code 1.30.3 on Real Ubuntu. Temporarily do:

$ sudo mv /usr/bin/gnome-terminal /usr/bin/gnome-terminal.bak
$ sudo touch /usr/bin/gnome-terminal

Start debugging with the external console. It will fail (duh) but there is no indication to the user as to why. I believe VS Code proper (contrast the extension) thinks the debugger is running. Please include some feedback in the UX so the user knows what went wrong. In the synthesized case above, that would be something like a popup indicating /usr/bin/gnome-terminal: Permission denied and the debug session (that never started) closing.

I'll post a follow-up in WSL#3679 (which isn't Pierson's problem), but it would be a useful to improve the failure path for the external debug console.

[tommyvct]

Oh hell yeah! It worked on my WSL Ubuntu 1810!
I updated the new 239 version of 'systemd', and update the 'cpptools' to version 0.21.0, it now works!

[pieandcakes]

@therealkenc when we made our runInTerminal change, we removed any requirement for gnome-terminal. If you are still seeing the requirement, you may be missing a setting in VS Code. If you change your VS Code settings to use a different terminal does it work?

[therealkenc]

That link is about the shell not the terminal. There is no setting in VS Code (that I know of, anyway) that specifies the path or arguments to the terminal program. I am not even sure how VS Code determines the default terminal. The default _happens_ to be gnome-terminal, typically, on Ubuntu but that is not the issue. The user might have configured their default terminal incorrectly, causing it to fail to launch. They might not have a default terminal installed at all, because they forgot - which is uncommon for typical folks who installed from a Ubuntu desktop ISO, but an easy enough situation to get oneself into when running code in headless remote scenarios. This is the case with WSL, but applies to anyone setting up their remote X11 rig from scatch.

The specific ask is that they get some kind of feedback when their _external console terminal_ fails to launch. The _shell_ (be it /bin/bash or /usr/bin/fish) is a separate topic and not the cause of the red stop square hanging there when the external terminal (sic) fails to launch. [ed: specifics for WSL posted [here](https://github.com/Microsoft/WSL/issues/3679#issuecomment-458346128)]

[pieandcakes]

@therealkenc I agree that there should be some feedback when it can't be launched and I'll go open an issue on their side.

The terminal app can be changed in the VS Code settings:

[pieandcakes]

Filed: https://github.com/Microsoft/vscode/issues/67296

[therealkenc]

Awesome. That is very good to know - thank you so much for your help Pierson. So the applicable settings are:

    "terminal.explorerKind": "external",
    "terminal.external.linuxExec": "x-terminal-emulator"

You'll find that is the magic answer to everyone's #2842 problems as well. That linuxExec line needs to be changed by the user to something that actually runs on their rig, say xterm or whatever suits their distribution of choice. It is commonly going to be x-terminal-emulator (which is in turn typically a symlink to gnome-terminal), but not necessarily.

[therealkenc]

Calling this a wrap.

[therealkenc]

[back here from [#2998](https://github.com/Microsoft/vscode-cpptools/issues/2998#issuecomment-459565516) so the setting doesn't get more burried than it needs to....]

TBH I didn't know about the setting either until we started talking about this issue

I kind of figured; the non-obviousness isn't on you. It also apears that this was all a side-effect of addressing two-digit #35. No good deed goes unpunished.

[pieandcakes]

Correct. The original complaint is that we were rolling our own terminal support and only supporting xTerm and gnome-terminal so when we switched, it seems the users of our extension who expected it to just "work" were now stuck trying to figure out the "new" VS Code angle of it.