Powershell: $Null, $VariableThatDoesntExist result in pipeline execution whereas @() doesn't

[ChrisMagnuson]

Steps to reproduce

@() | % {1}
$ArrayList = New-Object System.Collections.ArrayList
$ArrayList | % {1}
$null | % {1}
$VarNotDefined | % {1}

Expected behavior

PS > @() | % {1}
PS > $ArrayList = New-Object System.Collections.ArrayList
PS > $ArrayList | % {1}
PS > $null | % {1}
PS > $VarNotDefined | % {1}
PS >

Actual behavior

PS > @() | % {1}
PS > $ArrayList = New-Object System.Collections.ArrayList
PS > $ArrayList | % {1}
PS > $null | % {1}
1
PS > $VarNotDefined | % {1}
1

Environment data

Name                           Value
----                           -----
PSVersion                      6.0.2
PSEdition                      Core
GitCommitId                    v6.0.2
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

To follow the principle of least astonishment I would expect all of these to do nothing.

[BrucePay]

Consider $null, $null, $null | %{ 1 }. What would you expect to happen? How about $null, $null | % { 1 }? Now given the first two, what would the consistent behaviour for $null | %{ 1 } be?

[ChrisMagnuson]

@BrucePay With $null, $null, $null | %{ 1 } and $null, $null | % { 1 } the behavior is driven by the fact that an array is sent through the pipeline, irrespective of what is in the array, so the fact that those examples produce a 1 for each element of the array makes sense.

If I was saying that @($null) | %{ 1 } shouldn't produce output then your examples would be directly comparable but in my examples no array with given size (regardless of what it contains) is being created.

To be fair though my biggest beef is with the $VarNotDefined | % {1} scenario though I suspect that under the hood somewhere the $null | % {1} scenario is related to $VarNotDefined | % {1} and it still also feels wrong.

[BrucePay]

@ChrisMagnuson When sending the result of an expression evaluation into the pipeline, there is effectively an implicit @( ) around the input such that 1 | % { 1 } is equivalent to @( 1 ) | % { 1 }.

To be fair though my biggest beef is with the $VarNotDefined | % {1} scenario

By default, referencing an undefined variable results in $null. You can change this behaviour using the Set-StrctMode cmdlet.

[ChrisMagnuson]

@BrucePay If I understand properly, what Set-StrictMode will do is generate an error where as the behavior that is desired is to do nothing when a variable that doesn't exist is piped into the pipeline.

Thanks for the explanation about the implicit @( ).

I still feel that the expected behavior listed initially is best for least astonishment.

Do you have examples of where it is desirable that $null | % {1} executes the pipeline?

[mklement0]

@ChrisMagnuson: Indeed, Set-StrictMode -Version 1 and higher cause a _statement-terminating error_ when you reference an uninitialized variable, but don't allow you to change the _enumeration behavior_.

The only way to avoid enumeration of $null is to use the foreach _loop_ instead of the pipeline (e.g. foreach($v in $null) { <# never entered #> }) - a discrepancy that is in itself problematic, let alone that substituting one for the other is not always an option.

One way out of this would be to let uninitialized variables default to the "null collection", [System.Management.Automation.Internal.AutomationNull]::Value rather than to $null, given that this null collection is enumerated in neither scenario.

That would still enumerate _explicit_ $null values (in the pipeline), but that is less problematic, given that commands must go out of their way to output $null (not producing any output implicitly results in [System.Management.Automation.Internal.AutomationNull]::Value).

However, that would clearly be a breaking change.

[vexx32]

Actually, that's not quite correct. @Mklement0 -- the first reference you make, to having the foreach loop use a straight $null, isn't quite analogous to the pipeline examples.

foreach ($n in @($null)) { 1 }
# is equivalent to
$null | ForEach-Object { 1 }

Because the pipeline is incapable of determining the difference between an array and a single object (because by default all arrays are passed as individual objects) it has to assume that the $null it's receiving is intentional and part of a collection, so it has to operate on it just like anything else. To the pipeline, there's no difference between @($null) and $null, so it has to behave the same for both.

[mklement0]

@vexx32:

What this comes down to is _consistency_ and, as a corollary, _predictability_:

Both <stmt> | ... and foreach ($var in <stmt>) { ... } are _enumeration contexts_.

In PowerShell, this implies:

• If <stmt> evaluates to something that already _is_ enumerable, enumerate it.

• If it doesn't, _treat it like an enumerable_, and treat that something as the one and only member of the pretend-enumerable, _except_ if that "something" is the PS-specific "null collection" value whose very purpose is to signal that there is _nothing to enumerate_ (see below).

Not treating these two enumeration contexts the same is an inconsistency that invites confusion and is hard to remember.

---

Arguably, in the context of PowerShell:

• $null is a _something_ that happens to _represent_ a "single nothing" and is therefore _enumerable_ - it corresponds to null in C#.

• By contrast, [System.Management.Automation.Internal.AutomationNull]::Value is a PS-specific representation of an _enumerable_ that is _nothing in itself_ and _has no elements_: It is the "null _collection_" whose purpose is to signal "I represent nothing - I am not an object myself (unless I'm forced to act as one (as a scalar), in which case I'll pretend to be $null), and enumerating me results in _no_ iterations".
  It it is the "value" that commands that produce _no output_ implicitly "return".

---

Therefore, _both_ the foreach loop _and_ the pipeline:

• should treat $null as an enumerable and result in a _single iteration_ whose iteration variable is $null.

• should result in _no_ iterations for [System.Management.Automation.Internal.AutomationNull]::Value, given that its very purpose is to signal that there is _nothing to enumerate_.

---

Yet, these two contexts currently act differently with respect to _uninitialized variables_ (which default to $null), as the following example demonstrates:

PS> foreach ($var in $noSuchVar) { 'inside the foreach loop' }; $noSuchVar | ForEach-Object { 'inside ForEach-Object' }
inside ForEach-Object

The foreach loop didn't enumerate anything, but the pipeline did:

• It is the _pipeline_ that acts as expected here: given that uninitialized variables default to $null, it enumerates that $null.

• Unexpectedly, the foreach loop does _not_ enumerate the $null.

---

The same discrepancy happens with an _initialized_ variable that _happens to contain $null_:

PS> $varThatIsNull = $null; foreach ($var in $varThatIsNull) { 'inside the foreach loop' }; $varThatIsNull | ForEach-Object { 'inside ForEach-Object' }
inside ForEach-Object

---

The only time the discrepancy does _not_ surface is by _direct use of a command_ that outputs $null:

PS> foreach ($var in & { $null }) { 'inside the foreach loop' }; & { $null } | ForEach-Object { 'inside inside the foreach loop' }
inside the foreach loop
inside inside the foreach loop

Note how the foreach loop now too enumerated the $null output by the script block.

This is the subject of #5674, though the OP there is looking for the opposite behavior: they want foreach not to enumerate in this case either.

---

Again: These inconsistencies could be resolved if:

• both the foreach loop _and_ the pipeline consistently enumerate $null

• if uninitialized variables default to [System.Management.Automation.Internal.AutomationNull]::Value rather than $null.

Remember that $null -eq $noSuchVar would continue to work, because, as stated, [System.Management.Automation.Internal.AutomationNull]::Value acts like $null when forced into a scalar context (try $null -eq (& {}) - & {} being the simplest statement that produces a [System.Management.Automation.Internal.AutomationNull]::Value value).

Note that reversing the operands should arguably work differently, given that [System.Management.Automation.Internal.AutomationNull]::Value should be considered an _enumerable_ (array-valued) LHS, but (& {}) -eq $null does currently return $True, i.e., treats [System.Management.Automation.Internal.AutomationNull]::Value as a _scalar_

https://github.com/PowerShell/PowerShell/issues/3866#issuecomment-304764165 discusses this and shows that the current behavior is inconsistent in that the behavior varies depending on the operator used.

While it would be warranted from a consistency perspective, making $noSuchVar -eq $null return @() - i.e., treating the [System.Management.Automation.Internal.AutomationNull]::Value LHS as array-valued - would be a massively breaking change, because existing code that relies on such conditionals to return $True (whether explicitly or not) would break.

---

As another (farther) aside: #6823 is another example of inconsistent use of $null vs. [System.Management.Automation.Internal.AutomationNull]::Value.