Runtime: Overloaded Operator performance diff when using Fields or Auto Properties in Structs

[sonnemaf]

After reading this great Is C# a low-level language blog from @mattwar I found an performance issue with the * operator.

There is a difference in performance when using the overloaded operators on a struct with Fields compared to Auto Properties.

Have a look at the following Benchmark.

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;

namespace ConsoleApp1 {
    class Program {
        static void Main(string[] args) {
            BenchmarkRunner.Run<BM>();
        }
    }

    public class BM {

        [Benchmark(Baseline = true)]
        public VecWithFields TestVecWithFields() {
            var p = new VecWithFields(12f, 5f, 1.5f);
            var r = new VecWithFields();
            for (int i = 0; i < 100000; i++) {
                r = p + r;
                //r = p * r;
            }
            return r;
        }

        [Benchmark]
        public VecWithProperties TestVecWithProperties() {
            var p = new VecWithProperties(12f, 5f, 1.5f);
            var r = new VecWithProperties(); ;
            for (int i = 0; i < 100000; i++) {
                r = p + r;
                //r = p * r;
            }
            return r;
        }

        [Benchmark]
        public ReadOnlyVecWithProperties TestReadOnlyVecWithProperties() {
            var p = new ReadOnlyVecWithProperties(12f, 5f, 1.5f);
            var r = new ReadOnlyVecWithProperties(); ;
            for (int i = 0; i < 100000; i++) {
                r = p + r;
                //r = p * r;
            }
            return r;
        }

        [Benchmark]
        public VecWithProperties TestVecWithPropertiesNoOperator() {
            var p = new VecWithProperties(12f, 5f, 1.5f);
            var r = new VecWithProperties(); ;
            for (int i = 0; i < 100000; i++) {
                r = new VecWithProperties(p.X + r.X, p.Y + r.Y, p.Z + r.Z);

                //r = p * r;
                //r = new VecWithProperties(p.X * r.X, p.Y * r.Y, p.Z * r.Z);
            }
            return r;
        }
    }

    public struct VecWithFields {
        public float X;
        public float Y;
        public float Z;

        public VecWithFields(float x, float y, float z) {
            this.X = x;
            this.Y = y;
            this.Z = z;
        }

        public static VecWithFields operator *(VecWithFields q, VecWithFields r) {
            return new VecWithFields(q.X * r.X, q.Y * r.Y, q.Z * r.Z);
        }

        public static VecWithFields operator +(VecWithFields q, VecWithFields r) {
            return new VecWithFields(q.X + r.X, q.Y + r.Y, q.Z + r.Z);
        }
    }


    public struct VecWithProperties {
        public float X { get; set; }
        public float Y { get; set; }
        public float Z { get; set; }

        public VecWithProperties(float x, float y, float z) {
            this.X = x;
            this.Y = y;
            this.Z = z;
        }

        public static VecWithProperties operator *(VecWithProperties q, VecWithProperties r) {
            return new VecWithProperties(q.X * r.X, q.Y * r.Y, q.Z * r.Z);
        }

        public static VecWithProperties operator +(VecWithProperties q, VecWithProperties r) {
            return new VecWithProperties(q.X + r.X, q.Y + r.Y, q.Z + r.Z);
        }

    }

    public readonly struct ReadOnlyVecWithProperties {
        public float X { get; }
        public float Y { get; }
        public float Z { get; }

        public ReadOnlyVecWithProperties(float x, float y, float z) {
            this.X = x;
            this.Y = y;
            this.Z = z;
        }

        public static ReadOnlyVecWithProperties operator *(ReadOnlyVecWithProperties q, ReadOnlyVecWithProperties r) {
            return new ReadOnlyVecWithProperties(q.X * r.X, q.Y * r.Y, q.Z * r.Z);
        }

        public static ReadOnlyVecWithProperties operator +(ReadOnlyVecWithProperties q, ReadOnlyVecWithProperties r) {
            return new ReadOnlyVecWithProperties(q.X + r.X, q.Y + r.Y, q.Z + r.Z);
        }

    }
}

The TestVecWithProperties() method is almost 5 times slower when using the + opeator. I also tested the * operator and it is more than 3 times slower. The C# 7.2 readonly struct is also slower.

BenchmarkDotNet=v0.11.4, OS=Windows 10.0.18348
Intel Core i7-2640M CPU 2.80GHz (Sandy Bridge), 1 CPU, 4 logical and 2 physical cores
.NET Core SDK=2.2.200
  [Host]     : .NET Core 2.2.2 (CoreCLR 4.6.27317.07, CoreFX 4.6.27318.02), 64bit RyuJIT
  DefaultJob : .NET Core 2.2.2 (CoreCLR 4.6.27317.07, CoreFX 4.6.27318.02), 64bit RyuJIT


|                          Method |      Mean |     Error |    StdDev | Ratio | RatioSD |
|-------------------------------- |----------:|----------:|----------:|------:|--------:|
|               TestVecWithFields |  95.46 us | 1.0436 us | 0.9251 us |  1.00 |    0.00 |
|           TestVecWithProperties | 465.63 us | 7.7676 us | 6.8857 us |  4.88 |    0.09 |
|   TestReadOnlyVecWithProperties | 463.03 us | 4.7100 us | 3.9331 us |  4.85 |    0.07 |
| TestVecWithPropertiesNoOperator |  93.72 us | 0.8890 us | 0.8316 us |  0.98 |    0.02 |

Is this by design or a bug?

category:cq
theme:inlining
skill-level:expert
cost:large

[mikedn]

Inlining issue. The use of properties makes the operator code larger and the JIT is reluctant to inline the operator method:

INLINER impTokenLookupContextHandle for VecWithProperties:op_Addition(struct,struct):struct is 0x00007FFF6460DBB1.
*************** In fgFindBasicBlocks() for VecWithProperties:op_Addition(struct,struct):struct
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight=-12 : state  76 [ add ]
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight=-12 : state  76 [ add ]
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight= 77 : state  16 [ ldarga.s ]
weight= 79 : state  40 [ call ]
weight=-12 : state  76 [ add ]
weight=227 : state 103 [ newobj ]
weight= 19 : state  42 [ ret ]

multiplier in methods of promotable struct increased to 3.
Inline candidate callsite is in a loop.  Multiplier increased to 6.
calleeNativeSizeEstimate=1146
callsiteNativeSizeEstimate=155
benefit multiplier=6
threshold=930
Native estimate for function size exceeds threshold for inlining 114.6 > 93 (multiplier = 6)
Inline expansion aborted, inline not profitable


**************** Inline Tree
Inlines into 0600001B PostcardSizedRaytracerDotNet.Program:Test():struct
  [1 IL=0017 TR=000006 06000028] [profitable inline] VecWithProperties:.ctor(float,float,float):this
    [2 IL=0002 TR=000061 06000023] [below ALWAYS_INLINE size] VecWithProperties:set_X(float):this
    [3 IL=0009 TR=000067 06000025] [below ALWAYS_INLINE size] VecWithProperties:set_Y(float):this
    [4 IL=0016 TR=000073 06000027] [below ALWAYS_INLINE size] VecWithProperties:set_Z(float):this
  [0 IL=0036 TR=000029 0600002A] [FAILED: unprofitable inline] VecWithProperties:op_Addition(struct,struct):struct
Budget: initialTime=228, finalTime=264, initialBudget=2280, currentBudget=2280
Budget: initialSize=1408, finalSize=1598

cc @AndyAyersMS

[AndyAyersMS]

Unfortunately there's something of a discontinuity in the inliner's behavior when you cross the 16 byte IL size boundary. This is boundary is a tricky part of the heuristic to tune because once we get into this IL size range the impact of inlining can vary widely, and our modelling is pretty crude.

I still have ambitions of improving the inlining heuristics someday. In the meantime, you can work around this by adding AggressiveInlining to your operators.

Auto properties tend not to be great for perf in general, as the jit has to do more work to uncover what is really going on (and is sometimes unable to justify doing that work, as in the above). They can also slow down the ability of the jit to recognize facts needed for some optimizations. So use them judiciously in code that is perf sensitive.

[sonnemaf]

I was afraid that I had to use the AggressiveInlining solution. It works but looks ugly and it is easy to forget. Thanks for your help. I hope the Future is near by.

[bencyoung]

Unfortunately there's something of a discontinuity in the inliner's behavior when you cross the 16 byte IL size boundary. This is boundary is a tricky part of the heuristic to tune because once we get into this IL size range the impact of inlining can vary widely, and our modelling is pretty crude.

I still have ambitions of improving the inlining heuristics someday. In the meantime, you can work around this by adding AggressiveInlining to your operators.

Auto properties tend not to be great for perf in general, as the jit has to do more work to uncover what is really going on (and is sometimes unable to justify doing that work, as in the above). They can also slow down the ability of the jit to recognize facts needed for some optimizations. So use them judiciously in code that is perf sensitive.

@AndyAyersMS Could the compiler add perf hints (perhaps via an attribute) to let the JIT know that a property is trivial and should always be inlined? It seems easy for the compiler to be able to tell that a property is a simple mapping to a field and so add the attribute AOT?

[jkotas]

I was afraid that I had to use the AggressiveInlining solution. It works but looks ugly and it is easy to forget.

If you are building your own Vector, you would be better off to just use fields. Just like the Vector types in the framework: https://github.com/dotnet/corefx/blob/master/src/System.Numerics.Vectors/ref/System.Numerics.Vectors.cs#L354

Could the compiler add perf hints (perhaps via an attribute) to let the JIT know that a property is trivial

The property code is not the problem. The bottleneck is in code that calls these properties. Fetching a property via property getter generates bigger more complex IL than fetching a field.

Another side-effect of auto properties is that they create a copy of the value. The copy of the value may have a semantics difference to the program. It tends to be hard to perform the analysis to prove that the copy of the value is not necessary, especially if the value is a large struct.

I think (auto) properties are overused and folks do not realize that they are paying for them.

[stephentoub]

I think (auto) properties are overused and folks do not realize that they are paying for them.

@jaredpar, could we revisit https://github.com/dotnet/roslyn/issues/15644?

[bencyoung]

I think (auto) properties are overused and folks do not realize that they are paying for them.

@jkotas I think people probably have a mental model of what the JIT can do which is wrong! I'd always imagined these types of methods would always be trivially inlined without cost, so used them without thinking much. Similarly I was surprised reading the issues about structs with one struct member being expensive as again I'd assumed the JIT would trivally replace the outer with inner (assuming alignment etc).

The JIT can do loads of amazing optimizations so I think people are surprised when an "accumulation" of simple stuff causes surprising results!