Powershell: Do delay-bind scriptblocks / scriptblocks in calculated properties run in a child variable scope by design?

[mklement0]

The current behavior may well be by design, but it would be helpful to understand why - and document it.

• Script blocks passed to dedicated [scriptblock]-typed parameters such as in the context of ForEach-Object and Where-Object run directly in the _caller's scope_.

• Delay-bind script-block arguments / script blocks in calculated properties, by contrast, run in a _child_ scope, so that direct attempts to modify variables in the caller's scope create new local instances instead.

Contrast the following two commands:

# $i in the script block is the *caller's* $i, so modifying it works across invocations.
PS> $i = 0; 'a', 'b' | ForEach-Object { (++$i) }
1
2

# The script block runs in a *child* scope, so modifying $i modifies a local copy that
# goes out of scope with the script block.
PS> $i = 0; 'a', 'b' | Select-Object { (++$i) }

 (++$i) 
--------
       1
       1

Note how $i didn't increment across calls.

---

If you do want $i to increment across calls, you currently need a somewhat obscure workaround based on [ref]:

PS> $iRef = [ref] 0; 'a', 'b' | Select-Object { (++$iRef.Value) }

 (++$iRef.Value) 
-----------------
                1
                2

---

Changing the behavior to consistently execute script blocks in the caller's would be a breaking change, though presumably of type Bucket 3: Unlikely Grey Area.

Environment data

Written as of:

PowerShell Core v6.1.0-preview.3

[GeeLaw]

This behavior is so subtle that people might have been implicitly taking advantage of the fact that delay-bind scripts doesn't pollute the caller's scope.

My understanding is that pipeline-extraction scripts (ValueFromPipeline(ByPropertyName)) are mapping that are supposed to be pure (free of side-effect), while clearly ForEach-Object is more on the side of imperative paradigm, resembling the foreach statement. The choice for Where-Object is somehow arbitrary, but is useful for the case

$i = 0;
Get-Items | Where-Object { (++$i) % 2 }

One might ask why such usage wouldn't benefit the pipeline-extraction scenario, i.e., what's wrong if we execute the extraction scripts in the caller's scope. The answer is that there might be multiple parameters delay-bound. If the extraction script blocks have side effects, the order of extraction would affect the result. E.g., consider

# Imaginary
$i = 0;
Get-SomeItem | Use-SomePipeline -IndexMod2 { <# ??? #> } -IndexMod3 { <# ??? #> }

# Currently possible version
$i = [pscustomobject]@{ 'Value' = 0 };
Get-SomeItem | Use-SomePipeline -IndexMod2 { <# ??? #> } -IndexMod3 { <# ??? #> }

It is impossible to write the script correctly without knowing the order of delay-binding -- should I increase $i in the script for IndexMod2 or IndexMod3? I'm not sure whether PowerShell has the order of which parameter is bound before which documented. If that's undocumented, that's undefined to me, and if the extraction scripts had side effects, the whole script wouldn't be useful because of undefined behavior.

The invocation order for ForEach-Object and Where-Object is clear, so it makes sense to allow execution in the caller's scope.

[daxian-dbw]

clearly ForEach-Object is more on the side of imperative paradigm

Agreed. For Foreach-Object and Where-Object, running in the caller's scope allows us to do more than just writing results to the pipeline.
Also, Foreach-Object has -Begin, -Process and -End three script blocks, and they need to run in the same scope.

As for delay-bind script blocks in parameter binding, the purpose is simply to do some quick transformation/calculation on the value from pipeline, and it makes sense to isolate the execution to not pollute the caller's scope.

[mklement0]

This behavior is so subtle

The behavioral _distinction_ is indeed so subtle that it invites everlasting confusion.
Even if you understand the rationale - and the multiple-delay-bind-script-block scenario is a good argument _for_ the distinction _in principle_, but, due to its exoticness, to my mind _not in practice_ - _remembering_ it is a challenge.

it makes sense to isolate the execution to not pollute the caller's scope.

To paraphrase a common saying: One person's pollution is another person's treasure.

Given:

PS> $i = 0; 'a', 'b' | ForEach-Object { (++$i) }
1
2

why shouldn't a delay-bind script block with Rename-Item work the same?

# Does NOT work as intended, because a *local* $i variable is created on every invocation.
PS>  $i = 0; $null = New-Item -Force tf-a, tf-b; Get-Item tf-? |
           Rename-Item -NewName { '{0}-{1}' -f ++$i, $_.Name  } -WhatIf
What if: Performing ... "Rename File" on target "Item: .../tf-a Destination: .../1-tf-a".
What if: Performing ... "Rename File" on target "Item: .../tf-b Destination: .../1-tf-b".

That is not a hypothetical example, by the way: it's a real-world scenario for renaming files with sequence numbers that I've come across multiple times on Stack Overflow.

[mklement0]

P.S.: The flip side of this issue is that it's currently extremely cumbersome to implement your own cmdlet / advanced function that processes a given script block in the _caller's_ scope ("dot-sourced") with $_ support, the way that ForEach-Object / Where-Object / Measure-Command do - see #3581

[vexx32]

Yeah, I've hit that one too @mklement0. Takes me a few attempts to realize what I'm doing wrong, and thankfully the solution is relatively easy -- just scope-bind the variable properly (either to $script: or $global: usually works OK.) 😄

[wisemoth]

the solution is relatively easy -- just scope-bind the variable properly (either to $script: or $global: usually works OK.) 😄

@vexx32 Would you please be able to expand on this? Pretty sure I'm up against this issue, so wanted to explore your suggestion further.

[vexx32]

@mklement0's not-working example was:

$i = 0
$null = New-Item -Force tf-a, tf-b; Get-Item tf-? |
    Rename-Item -NewName { '{0}-{1}' -f ++$i, $_.Name  } -WhatIf

In that case you can workaround it with something like:

$script:i = 0
$null = New-Item -Force tf-a, tf-b; Get-Item tf-? |
    Rename-Item -NewName { '{0}-{1}' -f ++$script:i, $_.Name  } -WhatIf

Generally you should only run into this kind of issue if you've a need to _set_ or change the variable value from inside the delay-bind scriptblock. 🙂

[mklement0]

Thanks, @vexx32. Note that there's also the more obscure [ref]-based workaround from the OP, which, however, does have that advantage that it predictably targets the _parent_ scope (the caller's specific scope), whatever it may be (you could alternatively implement this via Get-Variable / New-Variable -PassThru in the calling scope):

[ref] $iRef = 0; $null = New-Item -Force tf-a, tf-b; Get-Item tf-? |
           Rename-Item -NewName { '{0}-{1}' -f ++$iRef.Value, $_.Name  } -WhatIf