Powershell: Presumptive DLR optimization bugs cause compound assignments not to read parent-scope variables.

[mklement0]

Note: This issue was discovered by @PetSerAl and has been a problem since v3.
Future resolution of #2230 may have an impact.

^{Note: Updated based on feedback from @PetSerAl below.}

Steps to reproduce

$a='a'

# Optimization bug: child-scope PS variable $a is created, but the parent-scope $a 
# isn't seen when `+=` is applied, resulting only in the RHS getting assigned.
# Variable type is LocalVariable rather than PSVariable
& {$a; $a+='b'; $a; (Get-Variable a).GetType().FullName }

'-'

# NO optimization bug, due to inclusion of a call to Remove-Variable.
# The mere *presence* of Remove-Variable in the block is enough, even if it is not actually called.
# Using the *built-in* alias `rv`, works too, but not *custom* aliases - see below.
# The outer-scope $a reappears after the Remove-Variable call.
& { $a; $a+='b'; $a; (Get-Variable a).GetType().FullName; Remove-Variable a; $a } 

'-'

# Optimization bug: Due to use of a *custom* alias, the call to Remove-Variable is not recognized, 
# and the problem surfaces again.
Set-Alias rvar Remove-Variable; &{ $a; $a+='b'; $a; (Get-Variable a).GetType().FullName; rvar a; $a } 

Expected behavior

a
ab
System.Management.Automation.PSVariable
-
a
ab
System.Management.Automation.PSVariable
a
-
a
ab
System.Management.Automation.PSVariable
a

Actual behavior

a
b # !!
System.Management.Automation.LocalVariable # !!
-
a
ab
System.Management.Automation.PSVariable
a
-
a
b # !!
System.Management.Automation.LocalVariable  # !!
rvar : Cannot remove variable a because the variable has been optimized and is not removable. Try using the Remove-Variable cmdlet (without any aliases), or dot-sourcing the command that you are using to remove the variable. # !!
...
b # !!

Environment data

PowerShell Core v6.0.0-beta.4 on macOS 10.12.5
PowerShell Core v6.0.0-beta.4 on Ubuntu 16.04.2 LTS
PowerShell Core v6.0.0-beta.4 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.413 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)

[PetSerAl]

You are not correct in first case. You should replace $a+='a' to $a+='b'. Then you will see, that child-scope variable is created, but parent-scope variable is not read by compound assignment.

Also, I do not see any easy fix for third case, unless you gave up on optimization entirely. It does not matter of detecting custom aliases, but deep problem with dynamic nature of PowerShell language:

&{ $a='asd'; &(Read-Host) a; $a }

And I type rv at runtime.

[mklement0]

Thanks for the correction, @PetSerAl; I've updated the initial post - see if it's accurate now.

Based on what you say, the title of this issue may be mistaken too - what do you suggest instead?

Re 3rd case:

That's an intriguing edge case.

Pragmatically speaking, asking innocently : how important are these optimizations?

[PetSerAl]

Something like this: "Compound assignment do not read parent-scope variable".

[PetSerAl]

P.S.
Actual behavior, second case: aa -> ab.

[mklement0]

@PetSerAl: Thanks, fixed.

[iSazonov]

$a='a'
& {$a; $a+='b'; $a; (Get-Variable a).GetType().FullName }

a
ab

I see the suggested fix is that $a is Global.
What about $a is Local? (Is it the optimization goal?) And the output:

<null>
b

[PetSerAl]

@iSazonov If there are no variable $a in parent scope, then you still have different behavior:

Set-StrictMode -Version Latest
& { $a; $a+='a'; $a; if(0){rv} }
& { $a; $a+='a'; $a }

[cfinnberg]

I just closed a dup of this. But just to note that, $private:var seems to have same effect as remove-variable. :var can be any variable, even a non-existent one.

```powershell
$a='a'
& { $a += 'b'; $a; if(0){$private:c} } # ab
& { $a += 'b'; $a } # b

[mikeclayton]

Real world example just found on Stack Overflow here:

https://stackoverflow.com/questions/56452820/handling-array-in-powershell-script/56457580#56457580

which basically boils down to:

$myArray = @( "xxx" );

function Invoke-MyFunction
{
    write-host "before function = '$($myArray | ConvertTo-Json)'";
    $myArray += "aaa";
    $myArray += "bbb";
    write-host "after function = '$($myArray | ConvertTo-Json)'";
}

write-host "before script = '$($myArray | ConvertTo-Json)'";
Invoke-MyFunction;
write-host "after script = '$($myArray | ConvertTo-Json)'";

gives output

before script = '"xxx"'
before function = '"xxx"'
after function = '"aaabbb"'
after script = '"xxx"'

i.e. in the context of a child scope, if $myArray += "aaa" is the first assignment to a parent scope variable, doesn't add an item to an array like $myArray = $myArray + "aaa" - it does string concatenation instead $myArray = $null + "aaa"

some workarounds that bypass the problematic optimisations are:

don't use "assignment by addition operator" in first assignment

function Invoke-MyFunction
{
    write-host "before function = '$($myArray | ConvertTo-Json)'";
    # don't use "assignment by addition operator" in first assignment
    $myArray = $myArray + "aaa";
    $myArray += "bbb";
    write-host "after function = '$($myArray | ConvertTo-Json)'";
}

do an assignment first to intialise the local variable*

function Invoke-MyFunction
{
    # do an assignment first to intialise the local variable
    $myArray = $myArray ;
    $myArray += "aaa";
    $myArray += "bbb";
}

both of these then give this output:

before script = '"xxx"'
before function = '"xxx"'
after function = '[
    "xxx",
    "aaa",
    "bbb"
]'
after script = '"xxx"'