Powershell: Very high function call overhead

[strawgate]

Steps to reproduce

function test {}
measure-command { foreach($i in 1..1000000) { test }}

Expected behavior

Function calls do not exhibit high overhead

Actual behavior

measure-command { foreach($i in 1..1000000) { test }}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 23
Milliseconds      : 547
Ticks             : 235471577
TotalDays         : 0.000272536547453704
TotalHours        : 0.00654087713888889
TotalMinutes      : 0.392452628333333
TotalSeconds      : 23.5471577
TotalMilliseconds : 23547.1577

Function calls (even empty functions) exhibit very high overhead

Environment data

Name                           Value
----                           -----
PSVersion                      6.0.4
PSEdition                      Core
GitCommitId                    v6.0.4
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[strawgate]

This kind of overhead makes powershell unsuitable for code which requires hundreds of thousands or millions of simple function calls.

In-lining the function call shows the difference in overhead of function call vs inline in these scenarios:

measure-command { foreach($i in 1..1000000) { $i = 0; $i++ }}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 63
Ticks             : 10638564
TotalDays         : 1.23131527777778E-05
TotalHours        : 0.000295515666666667
TotalMinutes      : 0.01773094
TotalSeconds      : 1.0638564
TotalMilliseconds : 1063.8564

function test {$i = 0; $i++}
measure-command { foreach($i in 1..1000000) { test }}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 24
Milliseconds      : 23
Ticks             : 240235646
TotalDays         : 0.000278050516203704
TotalHours        : 0.00667321238888889
TotalMinutes      : 0.400392743333333
TotalSeconds      : 24.0235646
TotalMilliseconds : 24023.5646

[msftrncs]

This is a scripting system (high latency), not a programming language that is compiled to native code (low latency). I found better mileage if I can keep loops to small counts, by utilizing the cmdlets or operators to do all the work. I had an instance where I needed to break a Base64String in to 80 character indented lines in to an XML document. A typical programming loop construct took 20 seconds to break the blob in to approximately 40,000 lines. A REGEX (-replace) pattern did it in under a second in a single command.

[strawgate]

A similar test in python shows overhead of about 0.11s over 1 million calls so id argue that the overhead is significantly higher than comparable languages.

There are ways around it like inlining functions and using native C# calls

[powercode]

You have run into the parameter binder. Some of the features that make PowerShell nice to use as a shell makes it harder to create efficient functions.

But your example is using the simplest and oldest of the parameter binders, and it should be possible to make at least that one faster.

In the general case, where the parameter binder has to handle input from the pipeline etc., there will always be overhead compared to much simpler environments, such as Python.

That said, I think there are things that could be done to speed up those scenarios to, but the code is complex, to say the least.

A workaround you can try is to create a class, and call methods on it - that is much faster than invoking functions:

class Test { 
    static [void] TestMethod(){}
}
measure-command { foreach($i in 1..1000000) { [Test]::TestMethod() }} | % TotalSeconds

5,0407504

[HumanEquivalentUnit]

@msftrncs but PowerShell is JIT compiled, for loops which happen more than 16 times ( https://stackoverflow.com/a/34236617 )

[powercode]

Parameter set resolution is not part of that. It still happens at runtime.

We are looking into improving the performance of the parameter binder, but it's quite a complex piece of code.

[eythort-mm]

I've stumbled on this Powershell performance bug two many times, and it's quite embarrassing. Calling a function within a large loop absolutely kills the performance of the script, and I'm forced to copy the contents of the function into the for loop to restore the performance.
Fixing embarrassing performance issues like this one (and the ProgressPreference on Invoke-WebRequest/RESTMethod) is necessary if you want to drive further adoption of Powershell

[iSazonov]

and the ProgressPreference on Invoke-WebRequest/RESTMethod

@eythort-mm It was fixed. If you still see the problem in latest builds please open new issue with repo steps.

[eythort-mm]

yeah it's probably fixed in the latest builds, but that was more of a side note, being an equally embarrassing performance issue