Powershell: Return semantics

[SRGOM]

More of a feature request:

Are you guys looking at improving the return semantics anyhow? Or maybe a new operator to capture the returned value and not the output? This is probably the only gripe I have from PS as a scripting langauge.

Once again, I (and all others I know) are a big fan of how well-designed PS is so I'd rather not have anything than something tacked on.

[rkeithhill]

If you use PowerShell classes, the methods therein behave more like a traditional programming language. They only return what you explicitly return:

13> class Foo {
>>     [string] ToString() {
>>         'a'
>>         'b'
>>         'c'
>>         return 'foo'
>>     }
>> }
>>
>> [Foo]::new().ToString()
foo

[vors]

Thank you for the warm words!
We aware of the return semantic concerns. As you can imagine it's quite hard to make any reasonable non-breaking change in the existing semantic to alternate it for functions.

Our approach to address these concerns are PS classes as @rkeithhill pointed out.
Classes are relatively new and not a fully-baked feature (input is welcome!).
One of their goals is to provide more developer-oriented experience for people who are used to languages like C#. They should help people write more safe and maintainble PS code faster.

[rkeithhill]

@SRGOM Also, the return semantics you're posting about is very typical (expected even) for a shell scripting language. Korn shell and Bash behave this way.

The idea is that you can copy paste lines you execute from the console into a script file inside a function and then executing that function would behave just as if you executed those lines - one at a time - at the console.

I think the return semantics difference between PowerShell functions and PowerShell class methods is a pretty clever way of A) not breaking thousands of existing functions B) appealing to shell scripters (often sys admins - not devs) and C) giving devs with more experience in a traditional/GP programming language something they're more comfortable with.

[SRGOM]

Thank you for discussing this. I will check out classes. I should note that comparison with ksh/bash is a disservice to PS. It's a nightmare to write maintainable scripts in them and the reason that I'm exploring PS on Linux looking to migrate some of that stuff. Devs nor ops like those sells for scripting, there just are no options. Given what PS offers, there's a real chance at setting the direction so I'd request you to not use them as a bar.

[rkeithhill]

@SRGOM I'm not on the PowerShell team. I'm just a PowerShell user (for 10+ years) and a Korn shell user before that. I'm just saying a shell scripting language is not the same beast as a general purpose programming language. There's a reason PowerShell uses -gt for the greater than operator instead of > (commonly used for file redirection in a shell). Likewise you can invoke an exe like Git without have to resort to an api like Process.Start, exec, spawn, etc. PoweShell's "shell nature" is why you do not have to quote arguments e.g. Copy-Item foo.txt bar.txt instead of Copy-Item "foo.txt" "bar.txt".

That said, the bar definitely is already much higher than ksh/bash. You have a real type system with support for ints, floats, bools, DateTime, etc and oh yeah, strings. You have recognizable and powerful control flow constructs like foreach ($item in $coll) { }, while (1} { Stop-Process StupidItAppThatSucksMyCpuDry -ea 0; Start-Sleep -Sec 5}, try { ...something that might throw ...} catch { ... handle error ... } finally { ...always execute cleanup script here ...}, if (...) { } elseif (...) {} else {}.

Then you have destructive commands that provide a WhatIf parameter that causes the command to show you what it would do (but not actually do it). You have two types of errors in PowerShell: non-terminating and terminating. You have a built-in parameter parsing engine with various validation attributes, you have comment-based help, a unit test framework (Pester), a linter (PS ScriptAnalyzer), etc, etc, etc. You get the idea. :-)

The PowerShell team looked at various shells out there and took the best ideas from each (pipes from Unix, common naming from VAX DCL, hostable engine from TK/TCL/Windows Scripting engine) and synthesized those ideas into a really freakin' awesome shell, language and automation platform.

[vors]

There is one practical way to improve code maintainability in regard to returns that I find useful. It's a convention to use # yeild comment to indicate explicit intention to write output to the pipeline.

Add comment "# yeild" on subroutine calls that write values to pipeline.
It would help keep code maintainable and simplify ramp up for others.

I don't know does anybody else use something similar or it's just my thing.

[SRGOM]

So I'm not saying this to continue arguing but just making a point for othes who come here.

Both the cases @rkeithhill points out and many others are well thought out and in cases where they deviate from well-established dev principles, they are usually following well-established ops principles where the cost and intuitiveness isn't lost.

Case in point, > is already an established operator for shell redirection. Once you know about this and know of -gt, you will likely never have trouble about this.

Similarly processes. In this case, Sysadmins have been ahead of the curve to make process launching straightforward. The programming language way of doing it is not canonical but given that process launching isn't really the bread and butter, they can get away with it. Although at least one language that I know of- Scala, has process related APIs that make them as easy as shell to handle.

import scala.ProcessSomethingSomething
val x = "myCommand arg1 arg2".!! 

Speaking of return, besides a "convention" that most on the bash/ksh side don't even think of at first usage and nobody likes, I don't see anything going for it in the current form. I use $(...) all the time knowing fully well that it captures output. Yet when I write return statement on a function and see in my shell status that the function return value is in shell but the assigned value is output, I scratch my head. Good thing I'm not alone. I see 0.7 million google hits for bash return value and going so far back as page 10, I find a question that deals with exactly this: 'http://stackoverflow.duapp.com/questions/28080307/either-getting-original-return-value-from-xargs-or-simulate-xargs'. At some point you have to break from convention if it helps make progress. One of the USPs of Microsoft is it doesn't break backwards compatibility which is what makes it so reliable. So I wouldn't want them to break anything but creating a very straightforward simple operator would hurt nobody (besides conventions, of course ;)).

@vors Maybe take inspiration from Scala process API?

I haven't done this for a while so its probably even easier than this but quick intro: You can launch process like x="program arg1 arg2".!!, which is the equivalent of X=(program arg1 arg2) in powershell. You can also do (some variation of this, not exactly) x= "program arg1 arg2" ! ProcessLogger( handlerForStdOut, handlerForStdErr)" and now x has the return value and the handlers have captured output. Clean, very elegant, breaks no assumptions.

[vors]

Sorry, I just realized I was talking about a different problem with return from the very beginning.
It's a bias: expect people ask questions that you already thought thru. :)

Let me step back and ask you to make sure we are all on the same page.

Or maybe a new operator to capture the returned value and not the output?

Are you talking about native executables invocations? PowerShell scripts invocations? Powershell functions invocations? Can you elaborate the original problem one more time.

UPD: for powershell function returned value and output are the same things. Hence my confusion and questioning my understanding.

I.e. these two function are indistinguishable for the caller

function foo()
{
   1
   2
   return 3
}

function foo()
{
   1..3
}

There is no such thing as outputting object anywhere besides the pipeline.
All pipelines are ending up in the console, where they are rendered by the formatting system.
But the important thing is that when you write 1 vs return 1, both these expressions are sending data to the current output pipeline, none of them is sending anything directly to the screen.
Hence, all return statements could be replaced by argument-less returns.
I.e. return 1 by 1; return
The return <arg> form is just a convenient syntax sugar.

Note: it's all applicable to functions and scripts only, classes are different, as described above.

[SRGOM]

I see your edit and you have understood the question as I intended to ask.

There is no such thing ...

I understand. But I contend there should be. And going by the discussions I see on this, a lot of people agree.

The question becomes, do you append Out-Null to all commands or do you let those write whatever they are writing want so you can read it later but return whatever single value you choose to.

function X(){
  ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted | Out-Null

ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted2 | Out-Null

ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted3 | Out-Null

if( ThingsHappendMyWay ) echo "yay!" else echo "nay!"
}

vs

function X(){
  ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted 

  ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted2 

  ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted3  

  return ( if( ThingsHappendMyWay )  0 else 1 )
}

[vors]

Ok, then classes :)

[dragonwolf83]

@SRGOM this was confusing to me at first too, partly because a lot of bad example scripts out there show a function with a return value. Instead of doing Out-Null, just assign the command to a variable.

function X(){
  $test = ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted 
  $test2 = ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted2 
  $test3 = ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted3  

  if( $test3 )  { 1 } else  { 0 )
}

I think that makes more sense to a programmer than using Out-Null. The output is captured in that variable and never sent to Output. You can then do something with the results or add to it, then output it later.

I never use return unless I'm breaking out of the function based on some check.

[mklement0]

@vors: As an aside: The "yeild" commenting convention is helpful, but I suggest you fix the typo: it's "yield", not "yeild".

(The mess that is English orthography has few discernible rules, but this one works in almost all cases for words that have a long E sound represented with the letters I and E: "I before E, except after C".)

[mklement0]

capture the returned value and not the output?

I think this is the crux: POSIX-like shells, despite their unfortunate choice of keyword return for it, provide a useful distinction that PowerShell currently doesn't have:

A _separate output stream_ that carries _invisible success-status/error-code information_.

That is, a POSIX shell return sets an _exit code_, which is invisible and distinct from _stdout_ output created with commands such as echo.

You can query that exit code explicitly with $?, but more importantly, the shell offers _control operators_ that can _directly act on exit codes, without interfering with the output stream_:

if a; then # .... conditional is only true if a's exit code is 0, but its stdout/stderr output is passed through

a && b  # ... execute b only if a succeeds (has exit code 0) - again, stdout/stderr output is passed through

a || b  # execute b only if a fails (has nonzero exit code) - stdout/stderr output is passed through

These very expressive constructs are sorely missed by anyone coming from the world of Unix shell scripting, and there have been repeated requests to add them, dating back many years.
I've recently created #3241 in summary.

Conceivably, PowerShell could build these control operators (but not the if behavior) on top of the automatic $? variable, but that variable is currently read-only.
(Also, $? is a _Boolean_, not an integer, but this could be considered a helpful simplification, given that even in the POSIX shell world the _specific_ nonzero value is rarely used.)

Thus, the equivalent of this Bash code:

foo() {
  echo 'oh no'  # write to stdout
  return 1  # signal failure via exit code
}

would be:

function foo {
  'oh no'  # write to success stream
  $? = $False  # signal failure via $?
}

(Perhaps a more explicit mechanism is called for, given that $? is reset after every command. Setting $? could only work as the very last statement in a function/script.)

In effect, this would be a _silent_ way of signaling a _non-terminating_ error.

Note that, arguably, calling Write-Error should also set $? to $False, which, however, is currently not the case:

> function foo { Write-Error 'oh no' }; $?
$True

[mklement0]

(Stumbling across this again many moons later, I realize I should have listened to @vors' cautionary tale about mistakenly answering a question one _thought_ was being asked: my previous comment related to yet a different use of return, namely for _exit codes_ in functions in POSIX-like shells.)

Let me try to summarize the debate:

• We've established that return in PowerShell script blocks / scripts / function is just syntactic sugar (its main purpose is to _exit_ a script block), and that anything not captured or redirected goes to the success output stream _by default_ - no operator or output command needed - and that is something that mustn't change.

  • The only way to get traditional return semantics in PowerShell is via _class_ methods.

• Assigning statement output to _variables_, as @dragonwolf83 has shown, can be used to prevent unintended implicit output.

  • If suppressing output is the only goal, i.e., if you know you're not interested in the output, you can simply assign to $null (e.g., $null = $someArrayList.Add('foo')) - this is arguably more readable than ... | Out-Null and typically also faster, though only marginally so.

• However, if I understand you correctly, @SRGOM, your desire is to have a way to _display_ statement output without it _becoming part of the output_ (with you conceiving of output-producing statements as limited to return calls, but we've established that we cannot limit it to that).

  • That's what the various Write-* cmdlets are for, which target either a _different_ output stream (e.g., Write-Verbose or the fraught Write-Host) or print _directly to the host_, _bypassing_ PowerShell's stream system altogether (Out-Host and, indirectly, Out-Default)

  • Specifically, if the intent is just to pass any statement's output through _for display only_, without interfering with a script/function/script block's "return value" (success output), use Write-Host or Out-Host:

function X(){

  ComplexCommandWhoseOutputIWouldIdeallyLikeToReadWhenItIsExecuted | Out-Host

  'regular output'
}

The advantage of Out-Host is that it applies PowerShell's usual output formatting, which Write-Host and the other different-stream-targeting Write-* cmdlets do not.

The disadvantage is that it's fundamentally impossible to capture or suppress Out-Host output from within PowerShell, whereas this is possible for the other Write-* cmdlets, including Write-Host (since v5, via stream 6, the information stream).

If you want both friendly output _and_ the ability to suppress / redirect, the somewhat cumbersome approach is to insert an Out-String call: e.g., ... | Out-String | Write-Host

---

At this point I'm unclear on what is actionable here and whether any enhancement is needed.
@SteveL-MSFT, can you clarify what, specifically, is up for grabs?

[SteveL-MSFT]

@mklement0 I believe you've succinctly summarized the original question and provided the existing PowerShell-way of solving those problems. It doesn't appear that any code work needs to happen here.