Powershell: Performance: -join slower than [string]::Join()

[HumanEquivalentUnit]

Steps to reproduce

Joining 100kb random numbers, the operator is much slower, ~13x here:

[string[]]$nums = Get-Random -Minimum 1 -Maximum 100 -Count 100kb

measure-command { $nums -join ','            } |% TotalMilliseconds
measure-command { [string]::join(',', $nums) } |% TotalMilliseconds

234.82690000000002
14.497100000000001

Stranger, with a big wordlist text file, loading it two different ways seems to make a difference:

$w1 = Get-Content d:\test\wordlist.txt;   $w1.Count
$w2 = ${D:\test\wordlist.txt};            $w2.Count

measure-command { $w1 -join "`r`n" } |% TotalMilliseconds
measure-command { $w2 -join "`r`n" } |% TotalMilliseconds

measure-command { [string]::join("`r`n", $w1) } |% TotalMilliseconds
measure-command { [string]::join("`r`n", $w2) } |% TotalMilliseconds

172823
172823        # same word count

254.6534
423.63100000000003    # <-- slower?

183.01510000000002
16.148400000000002    # <-- how come this one is SO fast?

Environment data

PS D:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.0.0-preview.1
PSEdition                      Core
GitCommitId                    7.0.0-preview.1
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[iSazonov]

I can confirm.

measure-command { for ($i=1; $i -le 100; $i++) { $w1 -join "`r`n" } } | Select-Object -ExpandProperty TotalMilliseconds
measure-command { for ($i=1; $i -le 100; $i++) { $w2 -join "`r`n" } } | Select-Object -ExpandProperty TotalMilliseconds
measure-command { for ($i=1; $i -le 100; $i++) { [string]::join("`r`n", $w1) } } | Select-Object -ExpandProperty TotalMilliseconds
measure-command { for ($i=1; $i -le 100; $i++) { [string]::join("`r`n", $w2) } } | Select-Object -ExpandProperty TotalMilliseconds

8171,9807
10849,47
8893,1133
6298,2773

I wonder there is very large deviations.

[vexx32]

I have similar results. Very odd.

 3  Z:  Root  PS>  $w1 = Get-Content Z:\enable1.txt; $w1.Count
>> $w2 = ${Z:\enable1.txt}; $w2.Count
>> measure-command { $w1 -join "`r`n" } |% TotalMilliseconds
>> measure-command { $w2 -join "`r`n" } |% TotalMilliseconds
>> measure-command { [string]::join("`r`n", $w1) } |% TotalMilliseconds
>> measure-command { [string]::join("`r`n", $w2) } |% TotalMilliseconds
172823
172823
355.6246
320.0242
242.2418
16.8466

[BrucePay]

The -join operator does more "stuff" - it works on enumerables of objects not just arrays of strings - so there's conversion logic overhead, etc. but I'm surprised to see such a big difference.

[BrucePay]

The source for the -join operator implementation is here.

[BrucePay]

Hmm - did some profiling for -join. Lotta time spend in PSObject.ToString()...

[iSazonov]

I very wonder that results for $w1 and $w2 notably differ

[SeeminglyScience]

I very wonder that results for $w1 and $w2 notably differ

The former is a object[] containing PSObject's, the latter is a object[] containing string's.

[iSazonov]

I pulled PR for $w2 = ${D:\test\wordlist.txt}; scenario but it seems also fix the first scenario for random numbers.

[daxian-dbw]

By _first scenario for random numbers_, do you mean [string[]]$nums = Get-Random -Minimum 1 -Maximum 100 -Count 100kbB in the PR repro? $num is also a string array.

[iSazonov]

$num is also a string array.

Yes, my comment for the issue description only.

[iSazonov]

After discussion in #10389 we close this as "by-design".

The bottom line is we don’t want to lose ETS although it makes us to wrap all in PSObject-s that cause a performance degradation. Recommendation is to use [string]::join() if needed.

We could add an option to -join operator to bypass ETS (PSObject wrapping). If you have an interest please open new issue with feature request.

Also we discovered an possible inconsistency in string operators (-join, -like, -match, etc.)
in how they relate to ETS. Perhaps @mklement0 will be interested to investigate this.