Powershell: Precedence of comma and addition operators doesn't make sense

[vexx32]

The comma operator has the highest precedence of any operator (excepting casts and enclosed statements like subexpressions and script blocks), causing (for example) + to behave as though concatenating arrays rather than adding values as one would more reasonably expect (in my opinion).

Steps to reproduce

$a = 0..10
$x = 5
$a[$x, $x + 1, $x + 2]

Expected behavior

Addition should be more similar to multiplication, taking place before the comma operator in the absence of explicit parentheses.

5
6
7

Actual behavior

5
5
1
5
2

As it is not completely clear, what's actually happening is that PS is treating the resolution more like it was entered like this:

$a[ @($x, $x) + @(1, $x) + 2 ]

Environment data

Name                           Value
----                           -----
PSVersion                      6.1.0
PSEdition                      Core
GitCommitId                    6.1.0
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[MaximoTrinidad]

Hum!
I don't think anything is wrong. I would use parentheses to contain the expressions results.
That's me!

PS [40] > $a[($x), ($x + 1), ($x + 2)]
5
6
7

PS [44] > $b = 5; $c = 6; $d = 7;
PS [45] >
PS [46] > $a[$b, $c, $d]
5
6
7
PS [50] >

:)

[vexx32]

Sure, you _can_. But in most languages folks are used to, simple math operations generally evaluate before something like concatenating reference objects like arrays. 😄

In my opinion, I think requiring parentheses to evaluate as "add arrays together" is _much_ more sensible, and simple math should be default.

[SteveL-MSFT]

@vexx32 if we were starting from scratch, I would agree with you. However, this would be a breaking change.

[mklement0]

@vexx32:

I get that the current behavior may be surprising, but changing it would be a massively breaking change.

The current behavior _is_ documented, namely in about_operator_precedence.

As @SteveL-MSFT states, such a change would definitely be a vZeroTechnicalDebt-related proposal - see #6745.

[vexx32]

@SteveL-MSFT @mklement0 I would be _very_ surprised indeed if anyone actually depended on the current behaviour. But knowing how PowerShell users are, I fully expect to be surprised pretty shortly as someone turns up depending on the current behaviour. 😏

That said, I agree this is the sort of fundamental language precedence that is... _difficult_ at best to ever have changed.

I still think it should be a noted and tracked issue, just in case we ever _do_ get any kind of zero-tech-debt version in the works as @mklement0 mentions. 😄

[mklement0]

@vexx32:

it should be a noted and tracked issue,

It is: in about_operator_precedence.
And, as long as the surprising behavior can't be changed, that's the best we can do.

That said, I encourage you track the issue with an eye toward the future by adding a comment at #6745.

if anyone actually depended on the current behaviour. But knowing how PowerShell users are, I fully expect to be surprised pretty shortly as someone turns up depending on the current behaviour.

I hear you, but changing something as fundamental as operator precedence is bound to break existing code, especially if users have read the docs and have come to rely on what they find there; e.g.:

PS> , 1   +   2, 3
1
2
3

[vexx32]

😬

Haha, if I had someone hand me code written like _that_, it'd be getting rewritten on the first pass. 😄

I'll make a note over there, thanks!

[BrucePay]

@vexx32 The addition operator always has lower precedence than the comma operator however not all commas are operators e.g. in the top level of method calls, the comma lexically separates arguments, not array elements. Note that indexing does not do this - in an index expression, the commas are operators (see below).

The way PowerShell uses the comma for arrays is intended to be consistent with the way arrays are passed as a single argument to a command i.e. as comma-separated lists without delimiters. For example:

Invoke-MyCommand 1, 2, 3

This example is passing one argument which is a three element array. Here's the same thing using a variable:

$arr = 1, 2, 3
Invoke-MyCommand $arr

Again - one argument with three elements. Contrast this with splatting

$arr = 1, 2, 3
Invoke-MyCommand @arr

which passes three arguments, each of which has a single scalar value.

We chose to favour array concatenation over addition on individual elements as the construction of lists is implicit in most PowerShell operations:

$result1 =  echo 1, 2  
$result1 += echo 3, 4
$result2 =  1,2 + 3, 4
$result3 =  $result1 + $result2
$result3 +=  4, 5, 6 

Mixing and matching commands and expression just works with no extra parentheses.

Now let's look at array indexing. In PowerShell, and array index expression takes a _single_ index which is either a scalar or array value so:

PS[1] (17) >   $a = 1..10
PS[1] (18) >   $a[1,3,5,7]
2
4
6
8

is equivalent to

PS[1] (19) >   $index = 1,3,5,7
PS[1] (20) >   $a[$index]
2
4
6
8

which is effectively equivalent to splatting without actually splatting.

Method invocation is where things get inconsistent because commas are lexical elements that separate arguments (as opposed to commands where arguments are separated by spaces.) This means that the following works:

PSCore (1:127) >  "hello world".substring(0+1, 2+3)
ello

Although in this example, it appears that addition has higher precedence than comma, this is, in fact not the case because the comma here is not an operator, it's a lexical token that separates arguments. Ultimately this approach was the one that most people preferred despite the inconsistency. For one thing, it made transcribing C# code fragments trivial.

As an aside, some alternatives we considered included parsing methods like commands where spaces separated arguments: "hello world".substring(1 (2+3)) but the feedback on this was negative. We also considered using commands for member invocation

Invoke-Object -InputObject "Hello world" SubString 0 3

But this was felt to be too verbose. (though this is pretty much what we implemented with ForEach-Object

PSCore (1:128) >  ForEach-Object -InputObject "Hello world" SubString 0 3
Hel

Overall, taking everything in context, I'm reasonably satisfied with most of the decisions we made in this area. That said, many of these decisions were made well over a decade ago. Given a decade and a half of broad user experience it would be interesting to see how things might be refactored. But the hard part, then and as now, is keeping the entire system in mind and consistent.

[vexx32]

Appreciate you taking the time to lay that all out, @BrucePay! 😄

You're right, it is complicated somewhat by the fact that the comma operator sometimes just isn't an operator, which maybe is a bit unfortunate given that method invocation is a separate parsing mode _anyway_ and perhaps other delimiters _might_ have been a slightly more effective choice given the easy confusion of arrays vs method arguments as it is.

But really I guess the "issue" here is that it doesn't really match up to expectations of operator precedence that I would generally expect... and maybe, at the end of the day, that's just me. 😄

[jzabroski]

@vexx32 Can you please edit your original post to say that the comma operator has the highest precedence of all PowerShell operators except for casts and property or array references?

Quote from PowerShell in Action (emphasis mine):

5.4.1 The comma operator
You’ve already seen many examples using the comma operator to build arrays. We covered this topic in some detail in chapter 3, but there are a couple of things we still need to cover. In terms of precedence, the comma operator has the highest precedence of any operator except for casts and property or array references. This means that when you’re building an array with expressions, you need to wrap those expressions in parentheses. In the next example, you’ll build an array containing the values 1, 2, and 3. You’ll use addition to calculate the final value. Because the comma operator binds more strongly than the plus operator, you won’t get what you want:

PS (1) > 1,2,1+2
1
2
1
2

[jzabroski]

@SteveL-MSFT Did you close this by accident?

[mklement0]

@jzabroski, this issue is mentioned and linked to from #6745, which should be sufficient.