Vscode-powershell: Different result of `$?` when stepping over code PS ISE versus VSCode PS Extension

[t-l-k]

System Details

System Details Output

### VSCode version: 1.40.2 f359dd69833dd8800b54d458f6d37ab7c78df520 x64

### VSCode extensions:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]


### PSES version: 1.13.1.0

### PowerShell version:

Name                           Value
----                           -----
PSVersion                      5.1.18362.145
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.145
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

I had these settings:

    "powershell.powerShellExePath": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
    "powershell.enableProfileLoading": false,
    "powershell.debugging.createTemporaryIntegratedConsole": true

Issue Description

Following the 100 IAC example at https://github.com/starkfell/100DaysOfIaC/blob/master/articles/day.20.azure.cli.logging.build.pipes.redirects.exit.codes.win.md
appears to yield the following inconsistencies within the Powershell ISE versus the VSCode PS extension. Example reproduced here:

Example Script

The script below is going to be the focus of today's topic and is designed to fail on purpose.

$ShowGroup = az group show 2>&1

# Variation 1 - Check Exit Code of the previous command.
if ($?)
{
    Write-Output "[---success---] Azure CLI command ran as intended."
}
else
{
    Write-Output "[---fail------] Azure CLI command failed."
    Write-Output "[---fail------] $ShowGroup."
}

Expected Behaviour

When _stepping over_ the lines of the example in Powershell ISE, $? evaluates as False. This is correct as per the 100 IAC example.

Actual Behaviour

When _stepping over_ the lines of the example in VSCode, $? evaluates as True. This is incorrect as per the 100 IAC example.

The behaviour of both is the same when _resume execution_ is used and breakpoints are set in the relevant if statement blocks.

Additional Info

The behaviour of $LASTEXITCODE in the example, when loading profile is disabled, appears to work normally. I would expect the example to use $LASTEXITCODE as a call out to the AZ CLI is equivalent (?) to calling out to a win32 executable.

The example may be wrong, but the discrepancy was making it difficult to debug when stepping over, so I believe the behaviour should be consistent with PS ISE.

[rjmholt]

If the cause of this is the PSReadLine prompt or us executing things in the background to provide intellisense, then this may be very difficult behaviour to change.

Fundamentally, PowerShell is single threaded, but user interaction involves several concurrent phenomena. To provide this concurrency (so that the prompt can block while you get intellisense, etc), we must rehost the prompt and perform calls to things in PSReadLine for each prompt. When those calls succeed, PowerShell sets $?, overwriting your command's original $? value.

It might be possible to use some reflection to preserve and reset this value while debugging, but I can make no promises there. I would highly recommend you use $LASTEXITCODE in contexts where a native utility is used.

[rjmholt]

As it happens, I just tried your example out in the preview extension using PowerShell 7 and it doesn't reproduce there.

That suggests this this is likely a Windows PowerShell issue, so I'd suggest trying out PowerShell 7. Some behaviours, particularly those where we depend on unusual hosting features like nested prompts, are buggy in Windows PowerShell but have since been fixed, and PowerShell Editor Services has no workaround for them on the older platform.

[t-l-k]

Oh okay thanks for your suggestion, I think for the work I am currently doing I can probably use PowerShell 7 for everything.

Thanks

[ggirard07]

@rjmholt in a previous post you said

Fundamentally, PowerShell is single threaded

Just to be sure, the value presents in $? variable is dedicated to the current PowerShell host process. This is not shared among multiple Powershell command prompt shell opened in parallel.

[rjmholt]

This is not shared among multiple Powershell command prompt shell opened in parallel

This is correct.

the value presents in $? variable is dedicated to the current PowerShell host process.

However, this is not synonymous with that. A given PowerShell process may host multiple threads. Normally (in a standard interactive session), one of these threads is where the PowerShell pipeline is being run and others represent background worker threads within PowerShell that you don't need to worry about. However, you can run pipelines in different runspaces in PowerShell, and they will each have their own concept of pipeline success ($?).

Concretely, variables in PowerShell are not scoped to a process, they are scoped to a runspace.

The problem comes in when the PowerShell extension must use the pipeline thread to do things like prompt the user. Because of how PowerShell's state is runspace-local, it must run operations within that runspace to get everything to work. When it does that, the engine sets $?, overwriting the previous value of $? from the debugger execution.

I'm not sure of this, but we go to the effort of running the debug prompt in a nested prompt and I suspect a fix was made in PowerShell 7 to prevent this issue from occurring. I could be wrong about that though.

A possible solution would be to use reflection to preserve and reset $? around executions that we know are internal, using this property of execution context.

[ggirard07]

@rjmholt thanks for the detailed answer. Really appreciated.

In my specific case, I use $? mostly when invoking external executable so I don't have to manage error based on both $LASTEXITCODE and whether the executable wrote to stderr, as I assume $? already handles all of this for me.

If I just can't trust $? for this purpose, then I will simply revert back to $LASTEXITCODE until we make the move to PowerShell 7 😄

[rjmholt]

as I assume $? already handles all of this for me.

Actually, $? just relies on $LASTEXITCODE too. There have been discussions about this in the PowerShell repo that I wasn't able to find immediately, but basically there are applications (on Windows as well as on *nix) that write to stderr even when they succeed.

$?'s virtue is that it also captures cmdlet/function failure.