Runtime: for loop little slower on .NET Core 3.0
The (very) basic benchmark below shows that a for loop on .NET Core 3.0 is a little slower than on .NET Core 2.1.5. What surprised me most, is that a foreach is "a lot" faster than a for loop on .NET Core 3.0.
using BenchmarkDotNet.Attributes;
using System.Linq;
namespace Benchmarks
{
public class LoopBenchmark
{
private int[] _ints;
private string[] _strings;
[Benchmark]
public void Int32_ForEach()
{
foreach (var s in _ints)
{
if (s == -1)
break;
}
}
[Benchmark]
public void Int32_For()
{
for (var i = 0; i < _ints.Length; i++)
{
if (_ints[i] == -1)
break;
}
}
[Benchmark]
public void String_ForEach()
{
foreach (var s in _strings)
{
if (s == null)
break;
}
}
[Benchmark]
public void String_For()
{
for (var i = 0; i < _strings.Length; i++)
{
if (_strings[i] == null)
break;
}
}
[GlobalSetup]
public void GlobalSetup()
{
_ints = Enumerable.Range(1, 100).ToArray();
_strings = _ints.Select(p => p.ToString()).ToArray();
}
}
}
.NET Core 3.0
BenchmarkDotNet=v0.11.2, OS=Windows 10.0.17134.345 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914070 Hz, Resolution=255.4885 ns, Timer=TSC
.NET Core SDK=3.0.100-alpha1-009640
[Host] : .NET Core 3.0.0-preview1-27004-04 (CoreCLR 4.6.27003.04, CoreFX 4.6.27003.02), 64bit RyuJIT
DefaultJob : .NET Core 3.0.0-preview1-27004-04 (CoreCLR 4.6.27003.04, CoreFX 4.6.27003.02), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|--------------- |---------:|----------:|----------:|
| Int32_ForEach | 40.81 ns | 0.2533 ns | 0.2369 ns |
| Int32_For | 56.18 ns | 0.8982 ns | 0.8402 ns |
| String_ForEach | 41.19 ns | 0.2446 ns | 0.2169 ns |
| String_For | 56.52 ns | 0.3497 ns | 0.3271 ns |
.NET Core 2.1.5
BenchmarkDotNet=v0.11.2, OS=Windows 10.0.17134.345 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914070 Hz, Resolution=255.4885 ns, Timer=TSC
.NET Core SDK=3.0.100-alpha1-009640
[Host] : .NET Core 2.1.5 (CoreCLR 4.6.26919.02, CoreFX 4.6.26919.02), 64bit RyuJIT
DefaultJob : .NET Core 2.1.5 (CoreCLR 4.6.26919.02, CoreFX 4.6.26919.02), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|--------------- |---------:|----------:|----------:|
| Int32_ForEach | 52.81 ns | 0.3626 ns | 0.3392 ns |
| Int32_For | 54.38 ns | 0.2734 ns | 0.2557 ns |
| String_ForEach | 52.76 ns | 0.3920 ns | 0.3274 ns |
| String_For | 54.35 ns | 0.3005 ns | 0.2811 ns |
I always (wrongly?) assumed that a for loop would produce far (?) more optimized code.
category:cq
theme:loop-opt
skill-level:intermediate
cost:medium
drieseng
All 15 comments
Try using a local copy of the array reference e.g.
public void Int32_For()
{
var ints = _ints;
for (var i = 0; i < ints.Length; i++)
{
if (ints[i] == -1)
break;
}
}
benaadams
on 7 Nov 2018
I think it would be caused by https://github.com/dotnet/coreclr/pull/15756?
If we are fetching an Array Length for an array ref that came from global memory
then for CSE safety we must use the conservative value number for both
/cc @mikedn
benaadams
on 7 Nov 2018
@benaadams I already tried before (been stalking you :p), but I since get more comparable results (when do we get that developer mode?)…
The code that's produced by .NET Core 2.1.5 and 3.0 is quite different though (as captured by BDN):
.NET Core 2.1.5
00007ffb`a5924370 Benchmarks.LoopBenchmark.Int32_ForEach()
00007ffb`a5924370 488b4108 mov rax,qword ptr [rcx+8]
00007ffb`a5924374 33d2 xor edx,edx
00007ffb`a5924376 8b4808 mov ecx,dword ptr [rax+8]
00007ffb`a5924379 85c9 test ecx,ecx
00007ffb`a592437b 7e14 jle 00007ffb`a5924391
00007ffb`a592437d 4c63c2 movsxd r8,edx
00007ffb`a5924380 468b448010 mov r8d,dword ptr [rax+r8*4+10h]
00007ffb`a5924385 4183f8ff cmp r8d,0FFFFFFFFh
00007ffb`a5924389 7406 je 00007ffb`a5924391
00007ffb`a592438b ffc2 inc edx
00007ffb`a592438d 3bca cmp ecx,edx
00007ffb`a592438f 7fec jg 00007ffb`a592437d
00007ffb`a5924391 c3 ret
00007ffb`a5914370 Benchmarks.LoopBenchmark.Int32_For()
00007ffb`a5914370 488b4108 mov rax,qword ptr [rcx+8]
00007ffb`a5914374 488bd0 mov rdx,rax
00007ffb`a5914377 33c9 xor ecx,ecx
00007ffb`a5914379 8b5208 mov edx,dword ptr [rdx+8]
00007ffb`a591437c 85d2 test edx,edx
00007ffb`a591437e 7e14 jle 00007ffb`a5914394
00007ffb`a5914380 4c8bc0 mov r8,rax
00007ffb`a5914383 4c63c9 movsxd r9,ecx
00007ffb`a5914386 43837c8810ff cmp dword ptr [r8+r9*4+10h],0FFFFFFFFh
00007ffb`a591438c 7406 je 00007ffb`a5914394
00007ffb`a591438e ffc1 inc ecx
00007ffb`a5914390 3bd1 cmp edx,ecx
00007ffb`a5914392 7fec jg 00007ffb`a5914380
00007ffb`a5914394 c3 ret
.NET Core 3.0 Preview1
00007ffb`a88f3eb0 Benchmarks.LoopBenchmark.Int32_ForEach()
00007ffb`a88f2533 488b4510 mov rax,qword ptr [rbp+10h]
00007ffb`a88f2537 488b4008 mov rax,qword ptr [rax+8]
00007ffb`a88f253b 488945e8 mov qword ptr [rbp-18h],rax
00007ffb`a88f253f 33c0 xor eax,eax
00007ffb`a88f2541 8945e4 mov dword ptr [rbp-1Ch],eax
00007ffb`a88f2544 eb25 jmp 00007ffb`a88f256b
00007ffb`a88f2546 488b45e8 mov rax,qword ptr [rbp-18h]
00007ffb`a88f254a 8b55e4 mov edx,dword ptr [rbp-1Ch]
00007ffb`a88f254d 483b5008 cmp rdx,qword ptr [rax+8]
00007ffb`a88f2551 732c jae 00007ffb`a88f257f
00007ffb`a88f2553 488d449010 lea rax,[rax+rdx*4+10h]
00007ffb`a88f2558 8b00 mov eax,dword ptr [rax]
00007ffb`a88f255a 8945e0 mov dword ptr [rbp-20h],eax
00007ffb`a88f255d 837de0ff cmp dword ptr [rbp-20h],0FFFFFFFFh
00007ffb`a88f2561 7414 je 00007ffb`a88f2577
00007ffb`a88f2563 8b45e4 mov eax,dword ptr [rbp-1Ch]
00007ffb`a88f2566 ffc0 inc eax
00007ffb`a88f2568 8945e4 mov dword ptr [rbp-1Ch],eax
00007ffb`a88f256b 8b45e4 mov eax,dword ptr [rbp-1Ch]
00007ffb`a88f256e 488b55e8 mov rdx,qword ptr [rbp-18h]
00007ffb`a88f2572 3b4208 cmp eax,dword ptr [rdx+8]
00007ffb`a88f2575 7ccf jl 00007ffb`a88f2546
00007ffb`a88f2577 488d65f0 lea rsp,[rbp-10h]
00007ffb`a8913ea0 Benchmarks.LoopBenchmark.Int32_For()
00007ffb`a8912527 488b4510 mov rax,qword ptr [rbp+10h]
00007ffb`a891252b 488b4008 mov rax,qword ptr [rax+8]
00007ffb`a891252f 488945f8 mov qword ptr [rbp-8],rax
00007ffb`a8912533 33c0 xor eax,eax
00007ffb`a8912535 8945f4 mov dword ptr [rbp-0Ch],eax
00007ffb`a8912538 eb23 jmp 00007ffb`a891255d
00007ffb`a891253a 488b4510 mov rax,qword ptr [rbp+10h]
00007ffb`a891253e 488b4008 mov rax,qword ptr [rax+8]
00007ffb`a8912542 8b55f4 mov edx,dword ptr [rbp-0Ch]
00007ffb`a8912545 483b5008 cmp rdx,qword ptr [rax+8]
00007ffb`a8912549 7324 jae 00007ffb`a891256f
00007ffb`a891254b 488d449010 lea rax,[rax+rdx*4+10h]
00007ffb`a8912550 8338ff cmp dword ptr [rax],0FFFFFFFFh
00007ffb`a8912553 7414 je 00007ffb`a8912569
00007ffb`a8912555 8b45f4 mov eax,dword ptr [rbp-0Ch]
00007ffb`a8912558 ffc0 inc eax
00007ffb`a891255a 8945f4 mov dword ptr [rbp-0Ch],eax
00007ffb`a891255d 8b45f4 mov eax,dword ptr [rbp-0Ch]
00007ffb`a8912560 488b55f8 mov rdx,qword ptr [rbp-8]
00007ffb`a8912564 3b4208 cmp eax,dword ptr [rdx+8]
00007ffb`a8912567 7cd1 jl 00007ffb`a891253a
00007ffb`a8912569 488d6500 lea rsp,[rbp]
Shall I close this issue?
drieseng
on 7 Nov 2018
/cc @AndyAyersMS
benaadams
on 7 Nov 2018
COMPlus_TieredCompilation?
omariom
on 7 Nov 2018
@omariom Tiered JIT was indeed still enabled by default in the preview build I used. With Tiered JIT disabled, the generate code is identical.
With Tiered JIT disabled, _foreach_ is way faster than _for_ (including the change @benaadams suggested) in .NET Core 3.0:
.NET Core 2.1.5:
| Method | Mean | Error | StdDev |
|-------------- |---------:|----------:|----------:|
| Int32_ForEach | 51.96 ns | 0.0162 ns | 0.0136 ns |
| Int32_For | 52.21 ns | 0.3361 ns | 0.2979 ns |
.NET Core 3.0 Preview1
| Method | Mean | Error | StdDev |
|-------------- |---------:|----------:|----------:|
| Int32_ForEach | 36.86 ns | 0.0186 ns | 0.0174 ns |
| Int32_For | 53.79 ns | 0.0984 ns | 0.0872 ns |
As you can see, _foreach_ has gotten quite a bit faster with .NET Core 3.0.
drieseng
on 8 Nov 2018
can you post the relevant assembly with tiered jit off? I'm curious as to what's different between these two cases.
mburbea
on 8 Nov 2018
; Benchmarks.LoopBenchmark.Int32_ForEach()
mov rax,qword ptr [rcx+8]
xor edx,edx
mov ecx,dword ptr [rax+8]
test ecx,ecx
jle M00_L01
M00_L00:
movsxd r8,edx
mov r8d,dword ptr [rax+r8*4+10h]
cmp r8d,0FFFFFFFFh
je M00_L01
inc edx
cmp ecx,edx
jg M00_L00
M00_L01:
ret
; Total bytes of code 34
; Benchmarks.LoopBenchmark.Int32_For()
mov rax,qword ptr [rcx+8]
mov rdx,rax
xor ecx,ecx
mov edx,dword ptr [rdx+8]
test edx,edx
jle M00_L01
M00_L00:
mov r8,rax
movsxd r9,ecx
cmp dword ptr [r8+r9*4+10h],0FFFFFFFFh
je M00_L01
inc ecx
cmp edx,ecx
jg M00_L00
M00_L01:
ret
; Total bytes of code 37
C# converts a foreach over an array to a for loop
benaadams
on 9 Nov 2018
@benaadams Why do I consistently get better results with the foreach?
drieseng
on 9 Nov 2018
Do you happen to know the address of the first instruction in the respective loops?
While instructions in the loop are a bit different the likely suspect is loop head alignment … see dotnet/runtime#11409 for an example of the kind of impact that can have.
We can make the jitted code more resilient here by padding (#11607).
AndyAyersMS
on 9 Nov 2018
00007ffb`381c9110 Benchmarks.LoopBenchmark.Int32_ForEach()
00007ffb`381c9115 488b4108 mov rax,qword ptr [rcx+8]
00007ffb`381c9119 33d2 xor edx,edx
00007ffb`381c911b 8b4808 mov ecx,dword ptr [rax+8]
00007ffb`381c911e 85c9 test ecx,ecx
00007ffb`381c9120 7e14 jle 00007ffb`381c9136
00007ffb`381c9122 4c63c2 movsxd r8,edx
00007ffb`381c9125 468b448010 mov r8d,dword ptr [rax+r8*4+10h]
00007ffb`381c912a 4183f8ff cmp r8d,0FFFFFFFFh
00007ffb`381c912e 7406 je 00007ffb`381c9136
00007ffb`381c9130 ffc2 inc edx
00007ffb`381c9132 3bca cmp ecx,edx
00007ffb`381c9134 7fec jg 00007ffb`381c9122
00007ffb`381c9136 c3 ret
00007ffb`381d9110 Benchmarks.LoopBenchmark.Int32_For()
00007ffb`381d9115 488b4108 mov rax,qword ptr [rcx+8]
00007ffb`381d9119 488bd0 mov rdx,rax
00007ffb`381d911c 33c9 xor ecx,ecx
00007ffb`381d911e 8b5208 mov edx,dword ptr [rdx+8]
00007ffb`381d9121 85d2 test edx,edx
00007ffb`381d9123 7e14 jle 00007ffb`381d9139
00007ffb`381d9125 4c8bc0 mov r8,rax
00007ffb`381d9128 4c63c9 movsxd r9,ecx
00007ffb`381d912b 43837c8810ff cmp dword ptr [r8+r9*4+10h],0FFFFFFFFh
00007ffb`381d9131 7406 je 00007ffb`381d9139
00007ffb`381d9133 ffc1 inc ecx
00007ffb`381d9135 3bd1 cmp edx,ecx
00007ffb`381d9137 7fec jg 00007ffb`381d9125
00007ffb`381d9139 c3 ret
Let me know if and how I can provide more info.
drieseng
on 9 Nov 2018
Looks like in the faster ForEach case the first four instructions in the loop are all in the same 16 byte bundle, while in the slower For case they split across two bundes (the cmp hangs over just a bit). So loop alignment still seems like the best guess.
If you have access to the hardware perf counters (say via vtune) you could confirm, you should see similar instructions retired but higher clocks (and hence lower CPI).
Alternatively you can try adding some padding by hand to see if you can influence results (either for better or worse) -- say add an out param to the method and store to it above the loop (or store to it more than once, etc). If doing that drastically alters perf for either variant then the code is highly alignment senstitive.
AndyAyersMS
on 9 Nov 2018
Why do I consistently get better results with the foreach?
I have seen this issue before when I was measuring looping over a span versus an array. As you can see, I added a random static variable at the beginning of the method which helped.
Try @AndyAyersMS's suggestion.
```C#
[DisassemblyDiagnoser(printAsm: true, printPrologAndEpilog: true)]
public class Program
{
// Work-around for loop head alignment issue since this micro-benchmark is sensitive to it.
public static int zero = 0;
public static int ArraySum(byte[] data)
{
int sum = zero;
for (int i = 0; i < data.Length; i++)
{
sum += data[i];
}
return sum;
}
public static int SpanSum(Span<byte> data)
{
int sum = zero;
for (int i = 0; i < data.Length; i++)
{
sum += data[i];
}
return sum;
}
private byte[] data;
[GlobalSetup]
public void Setup()
{
data = new byte[10_000];
new Random(42).NextBytes(data);
}
[Benchmark(Baseline = true)]
public int ArraySum() => ArraySum(data);
[Benchmark]
public int SpanSum() => SpanSum(data);
static void Main(string[] args)
{
byte[] data = { 1, 2, 3, 4, 5, 6, 7 };
ArraySum(data); // returns 28
SpanSum(data); // returns 28
Summary summary = BenchmarkRunner.Run<Program>();
}
}
```
ahsonkhan
on 16 Nov 2018
Adding my simple benchmark result here:
Version used: 3.0.100
| Method | Mean | Error | StdDev | Ratio |
|------------- |----------:|----------:|----------:|------:|
| ArrayForeach | 34.64 ns | 0.0661 ns | 0.0619 ns | 1.00 |
| ArrayFor | 41.62 ns | 0.0606 ns | 0.0506 ns | 1.20 |
| ListForeach | 163.54 ns | 0.4555 ns | 0.3556 ns | 4.72 |
| ListFor | 54.17 ns | 0.4392 ns | 0.4108 ns | 1.56 |
| ArrayAsSpan | 46.85 ns | 0.0278 ns | 0.0232 ns | 1.35 |
The codegen of span looks suboptimal.
huoyaoyuan
on 25 Oct 2019
Related issues
btecu
·
3Comments
noahfalk
·
3Comments
nalywa
·
3Comments
jchannon
·
3Comments
omajid
·
3Comments