Powershell: pipeline operator don't work with function ?

[p0W3RH311]

steps to reproduce

PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Microsoft Windows 6.3.9600
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

PS C:\>

PS C:\> function ex1 {tasklist.exe externalProgram}
PS C:\>
PS C:\> ex1 || echo false
# display error but not execute "echo false"
PS C:\>

in other shell like bash it work correctly exemple with bash:

/-> function ex1() {false}
/-> ex1 || echo "false"

# display "false"

[mklement0]

The behavior is certainly surprising [_update_: to users of POSIX-like shells such as Bash]. /cc @rjmholt

The root cause is that executing your function doesn't result in $? being set to $false, so the RHS of || isn't invoked.

Unlike in a script, where you can use exit $LASTEXITCODE, which indirectly sets $? to $false, there is currently no way to do that from a function.

The need to provide this ability has been recognized - see https://github.com/PowerShell/PowerShell/issues/10917#issuecomment-550550490 - but, as far as I know, is not yet implemented.

Find a - cumbersome and suboptimal - workaround at the bottom.

---

Note that what will _not_ change - for the sake of backward compatibility - is that you must _explicitly_ set $? (using exit in scripts, and the not-yet-implemented mechanism in functions).

This differs from POSIX-compatible shells such as Bash, where - in the absence of an explicit exit or return statement - the _last_ statement's exit code determines the enclosing script / function's exit code.

However, in the context of the PowerShell _CLI_ this logic _does_ exist - with a twist:

• with -File, the script's exit code (whether implicit or explicit) becomes the PowerShell process' exit code.

  • Note that the _implicit_ exit code of a script is always 0, except if a script-terminating (runspace-terminating) error occurs, in which case it is always 1.

• with -Command, the _last statement's_ success status ($?) is mapped to exit code 0 ($true) or 1 ($false) and reported as the PowerShell process' exit code; note that this means that while success/failure information is preserved in the abstract, the _specific_ exit code a call to an external program may have reported (as the last statement in the -Command string) is _lost_.

---

Workaround:

The only way I know of to currently (indirectly) set $? to $false is to make your function an _advanced_ function and use $PSCmdlet.WriteError() to write a non-terminating error.

Apart from being cumbersome, this has the obvious downside that an (extra) error message will display (and an extra error will be logged in $Error):

function foo { 
  [cmdletbinding()] param() 
  /bin/ls nosuch  # Call to external program that may set a nonzero exit code.
  if ($LASTEXITCODE) { 
    $PSCmdlet.WriteError([System.Management.Automation.ErrorRecord]::new('foo failed', $null, 'InvalidOperation', $null)) 
  }  
}; foo || 'NO'  # -> 'NO'

Note that the workaround is only effective with $PSCmdlet.WriteError(), not with Write-Error - see #3629.

[rjmholt]

The behavior is certainly surprising.

I disagree there. The function succeeds, since failure does not propagate up the call stack. $? encodes pipeline success. When the calling pipeline (ex1) finishes, it sets its success (it successfully executed a pipeline, which itself failed) and washes away the old value of $?. Call stack traversal is what exceptions are designed for.

The concept of && and || is very simple and reuses existing concepts. Trying to make it do magical things in various scenarios is just going to make it more and more complex and eventually you'll end up with the situation that PowerShell errors are in.

The need to provide this ability has been recognized - see #10917 (comment) - but, as far as I know, is not yet implemented.

Yes, that request is effectively up for grabs.

The workaround I would suggest is to use the pipeline chain operator inline with the actual failing native command, and if you need the information at caller's level, promote the failure to an exception.

[mklement0]

I disagree there.

Yes, I should have said: It's surprising _if you come from a POSX shell background_.

The concept of && and || is very simple and reuses existing concepts.
Trying to make it do magical things in various scenarios

While best suited for direct invocation of external programs, it also works for PowerShell scripts, _assuming they set their exit code intentionally_.

Implementing https://github.com/PowerShell/PowerShell/issues/10917#issuecomment-550550490 isn't about trying to do magical things - it's about providing the same feature available to _scripts_ analogously to _functions_ - but the tension with PowerShell's native, largely fundamentally different error handling remains (unavoidably so).

That said, for a function to be able to set $? intentionally can also be useful in pure PowerShell scenarios.

That use of Write-Error doesn't set $? to false _automatically_, the way use of PSCmdlet.WriteError() does, is a related problem (#3629).

[SeeminglyScience]

That use of Write-Error doesn't set $? to false _automatically_, the way use of PSCmdlet.WriteError() does, is a related problem (#3629).

Maybe this is a rant for another time, but the reason for it (and all other inconsistencies with those cmdlets) is probably because every Write-* command is it's own command. I know I'm not stating new facts here, but because they are they're own command they have their own MshCommandRuntime, their own PSCmdlet instance, etc. I've always found it strange that they don't reach into their caller's command processor for all that, ships probably sailed for that though. Either way worth noting.

[jhoneill]

When the calling pipeline (ex1) finishes, it sets its success (it successfully executed a pipeline, which itself failed)

This is the key part and it is not understood as widely and well as it should be.
The title of this is issue is the symptom, not the issue. A function normally leaves $? as true so in this case && and II are shown "success". People have (generally) learned this and build functions which return something to say they didn't run properly. In this case $lastExitCode is global and set to 1 but $? is true because the function ran to completion.

I'm not really a fan of these operators but the documentation should be clear what they intended for and what their limits are. Changing the behaviour to what people first expect might be a big ask; consider:

>tasklist foo
ERROR: Invalid argument/option - 'foo'.
Type "TASKLIST /?" for usage.
>$LASTEXITCODE
1
>$?
True
>$LASTEXITCODE
1

The command runs, it sets last exist code to 1 instead of 0.
But getting the value of last exit code was successful so $? is now true
However Last exit code will remain as 1 until another external command changes it....

[mklement0]

I think there's a simple way to frame this for end users that is true to the - ultimately conceptually simple, as @rjmholt notes above - implementation:

• && and || act on the value of automatic success-status variable $? after the LHS operand executes, in the context of a _single statement_:
  
  
  
  • <pipeline1> && <pipeline2> is the same as <pipeline1>; if ($?) { <pipeline2> },
  
  
  • <pipeline1> || <pipeline2> equals <pipeline1>; if (-not $?) { <pipeline2> }
  
  

• && and || are _left_-associative:
  
  
  
  • e.g., in <pipeline1> || <pipeline2> && <pipeline3>, <pipeline1> || <pipeline2> is executed first, and whatever $? is afterwards (as set by whatever pipeline is executed (last)) becomes the LHS of &&.
  
  

Pitfalls:

• Use with _expressions_ as the LHS instead of _commands_ / pipelines is pointless, because expressions always set $? to $true (unless a _terminating_ error occurs, in which case the entire statement or runspace is terminated); the only exception is when a command is wrapped in (...), $(...) or @(...) (without making it part of a larger expression), which technically makes it an expression, yet $? is (sensibly) still set based on the wrapped command's status.

  • && and || fundamentally do not work meaningfully with PowerShell's Test-* cmdlets such as Test-Path, because the latter _output_ a Boolean to report the test result rather than indicating it via $? - see #10917.

  • Syntax pitfall: Flow-control statements exit, return and throw cannot be used as-is as the RHS of && and ||, they must be wrapped in $(...) - see #10967.

How $? is set / how to _control_ how it is set:

Note: Technically, what $? reflects is a _pipeline_'s success, even if that pipeline is composed of only a single command.
Applying && and || to a _single_ command on the LHS is by far the most common use case, but it also works with multi-command pipelines, in which case the non-success of _any_ involved command causes $? to be set to $false for the pipeline as a whole; e.g.,
'/', 'nosuch' | Get-Item | % FullName || 'NO' outputs 'NO'.

• Call to an _external program_:

  • $? is set to $true if that program's (process') _process exit code_ (as reflected in LASTEXITCODE) is 0, and to $false for any other value.

  • /bin/ls nosuch || 'NO'

• Call to a _PowerShell script_ (*.ps1):

  • $? is set to $true _unless_ the script uses exit $n to explicitly set a _nonzero_ exit code.

  • '/bin/ls nosuch; exit $LASTEXITCODE' > tmp.ps1; ./tmp.ps1 || 'NO'

  • Note that this means that, with respect to && and ||, scripts behave more like external programs than cmdlets or functions (as they do if you run them as self-contained shell script in a child process with a shebang line on Unix).

• Call to a _cmdlet_ (a _binary_ cmdlet, as opposed to a written-in-PowerShell (advanced) _function_):

  • $? is set to $false only if the cmdlet emits at least one _non-terminating_ error - irrespective of whether the error is _silenced_ or _ignored_ (that is, neither -ErrorAction SilentlyContinue nor -ErrorAction Ignore affect how $? is set).

  • Get-Item NoSuch || 'NO'

• Call to a _function_ - the currently problematic case:

  • $? is currently _never_ set to $false - even if you use Write-Error, for the reasons explained by @SeeminglyScience above.

  • The only current - very cumbersome - workaround is to use $PSCmdlet.WriteError(), as shown above.

  • The problem will go away once a mechanism is provided so that functions too can set $? intentionally, as agreed in https://github.com/PowerShell/PowerShell/issues/10917#issuecomment-550550490.

    • There arguably is no strict need to provide a mechanism to set $? _directly_: For consistency with cmdlets, functions too could be required to signal (partial) non-success via _non-terminating_ errors, but making Write-Error do that _by default_ is no longer option due to backward compatibility concerns, as noted.

  • In that vein, maybe something like Write-Error -SignalFailure ... would suffice, analogous to the previously pondered ability to write a _terminating_ error with Write-Error -Throw.

    • Conceivably, Write-Error -SignalFailure _without additional arguments_ could then be implemented to _only_ set $? to $False, without also writing an error record, though Cmdlet.WriteError() should then be extended to accept null in lieu of an error record to achieve the same effect).

[jhoneill]

@mklement0 I think that should be taken as the first draft of a help item :-)

[mklement0]

I actually just took a closer look at about_pipeline_chain_operators:

It is technically commendably detailed (judging by the British spelling, @rjmholt may have authored it / contributed to it [_Never mind: a topic's contributors can now easily be seen in the online topic pages themselves_]), but to me could additionally use the systematic and pragmatic guidance that I've tried to provide above.

Unfortunately, a condensed summary such as the above is at odds with the way PowerShell documentation is written.