Runtime: Consider making Interlocked.And and friends into JIT intrinsics

[stephentoub]

From https://github.com/dotnet/runtime/pull/32216#discussion_r378817293

https://github.com/dotnet/runtime/pull/32216 adds new methods like Interlocked.And. Some of these could be done as JIT intrinsics and replaced by better implementations/instructions in some cases, like lock and on platforms that have it.

cc: @tannergooding

category:cq
theme:jit-intrinsics
skill-level:intermediate
cost:medium

[BruceForstall]

@AndyAyersMS

[GrabYourPitchforks]

For context, it turns out the JIT already optimizes this for the existing Interlocked.Add and similar APIs.

using System;
using System.Threading;

public class C {
    public static int M(ref int i) {
        return Interlocked.Add(ref i, 1);
    }

    public static void N(ref int i) {
        Interlocked.Add(ref i, 1);
    }
}

; C.M(Int32 ByRef)
    L0000: mov eax, 0x1
    L0005: lock xadd [rcx], eax
    L0009: inc eax
    L000b: ret

; C.N(Int32 ByRef)
    L0000: lock add dword [rcx], 0x1
    L0004: ret

[stephentoub]

Nice. This can be closed then, right?

Oh.. you said Add, not And. Those are way too closely named :-) I was very impressed it was able to reverse the pattern in Interlocked.And :-)

[AndyAyersMS]

These should be done via new-style intrinsics. We may also want to convert some of the existing intrinsified interlocked ops over to this mechanism as well.

[AndyAyersMS]

@stoub we're triaging proposed 5.0 codegen work -- any thoughts on the priority of this issue?

[stephentoub]

From my perspective, "nice to have", but I expect there's other higher priority JIT work that'll be more impactful. These APIs are brand new.

[AndyAyersMS]

@GrabYourPitchforks can you make a case for addressing this in 5.0?

[GrabYourPitchforks]

I agree with Steve's assessment. Optimizing these APIs might save a few cycles in a handful of places, such as below.

https://github.com/dotnet/runtime/blob/ef2ea8a6a241431a209beab36236444de837dcbf/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs#L739-L740

https://github.com/dotnet/runtime/blob/ef2ea8a6a241431a209beab36236444de837dcbf/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/SafeHandle.cs#L104-L105

These are hot code paths, but not _so_ hot that a few extra cycles are going to kill us. :) If you have other more pressing matters, so be it.

[AndyAyersMS]

Ok, I'm going to move this to future.

[JulieLeeMSFT]

@echesakovMSFT Possible candidate for .NET 6.

[EgorBo]

this is a bit tricky for the cases when return value of And/Or is used 🙂

[GrabYourPitchforks]

Wouldn't it be able to use the same logic Interlocked.Add and friends use? See my earlier comment at https://github.com/dotnet/runtime/issues/32239#issuecomment-586462094.

[EgorBo]

@GrabYourPitchforks here is what LLVM emits for us (mono-llvm): https://godbolt.org/z/vbsYz3

Quoting LLVM:

      // There are two different ways of expanding RMW instructions:
      // - into a load if it is idempotent
      // - into a Cmpxchg/LL-SC loop otherwise

Or/And aren't idempotent, Add is.

[EgorBo]

better demo: https://godbolt.org/z/cMj5Tx

[echesakovMSFT]

When using Armv8.1 instructions the clang/llvm generated code seems much better than its Armv8.0 counterpart https://godbolt.org/z/Yvjcqb and there is no loops required.

Armv8.0

InterlockedAdd(int*, int):                  // @InterlockedAdd(int*, int)
.LBB0_1:                                // =>This Inner Loop Header: Depth=1
        ldaxr   w8, [x0]
        add     w8, w8, w1
        stlxr   w9, w8, [x0]
        cbnz    w9, .LBB0_1
        mov     w0, w8
        ret
InterlockedAnd(int*, int):                  // @InterlockedAnd(int*, int)
.LBB1_1:                                // =>This Inner Loop Header: Depth=1
        ldaxr   w8, [x0]
        and     w8, w8, w1
        stlxr   w9, w8, [x0]
        cbnz    w9, .LBB1_1
        mov     w0, w8
        ret
InterlockedOr(int*, int):                   // @InterlockedOr(int*, int)
.LBB2_1:                                // =>This Inner Loop Header: Depth=1
        ldaxr   w8, [x0]
        orr     w8, w8, w1
        stlxr   w9, w8, [x0]
        cbnz    w9, .LBB2_1
        mov     w0, w8
        ret

Armv8.1

InterlockedAdd(int*, int):                  // @InterlockedAdd(int*, int)
        ldaddal w1, w8, [x0]
        add     w0, w8, w1
        ret
InterlockedAnd(int*, int):                  // @InterlockedAnd(int*, int)
        mvn     w8, w1
        ldclral w8, w8, [x0]
        and     w0, w8, w1
        ret
InterlockedOr(int*, int):                   // @InterlockedOr(int*, int)
        ldsetal w1, w8, [x0]
        orr     w0, w8, w1
        ret