Runtime: HW intrinsics: Poor codegen for operations requiring imm8 when the value is calculated to constant during JIT
In the repro code below sizeof(TChar) is known during JIT, still code is suboptimal in disasm - some helper (?) method is invoked instead of instruction and SIMD vector value is placed on stack due to this.
This issue is quite notable with shifts and typecasting during concrete value calculation, but to my memory there were similar cases with other instructions.
Repro code:
[MethodImpl(MethodImplOptions.NoInlining)]
static void Test<TChar>() where TChar : unmanaged
{
Vector128<byte> v = Sse2.SetZeroVector128<byte>();
v = Sse2.ShiftRightLogical128BitLane(v, (byte)sizeof(TChar)); // <======
Console.WriteLine(Sse41.Extract(v, 0));
}
static void Main(string[] args)
{
Test<byte>();
Test<ushort>();
}
Disasm:
--- ...\Program2.cs ---
Vector128
000007FE75244A20 sub rsp,58h
000007FE75244A24 xor eax,eax
000007FE75244A26 mov qword ptr [rsp+40h],rax
000007FE75244A2B mov qword ptr [rsp+48h],rax
000007FE75244A30 pxor xmm0,xmm0
000007FE75244A34 movaps xmmword ptr [rsp+40h],xmm0
v = Sse2.ShiftRightLogical128BitLane(v, (byte)sizeof(TChar));
000007FE75244A39 movaps xmm0,xmmword ptr [rsp+40h]
000007FE75244A3E movaps xmmword ptr [rsp+30h],xmm0
000007FE75244A43 lea rcx,[rsp+40h]
000007FE75244A48 lea rdx,[rsp+20h]
000007FE75244A4D mov r8,qword ptr [rsp+30h]
000007FE75244A52 mov qword ptr [rdx],r8
000007FE75244A55 mov r8,qword ptr [rsp+38h]
000007FE75244A5A mov qword ptr [rdx+8],r8
000007FE75244A5E lea rdx,[rsp+20h]
000007FE75244A63 mov r8d,1
000007FE75244A69 call 000007FE752421A0 <=================
Console.WriteLine(Sse41.Extract(v, 0));
000007FE75244A6E movaps xmm0,xmmword ptr [rsp+40h]
000007FE75244A73 pextrb ecx,xmm0,0
000007FE75244A79 call 000007FE75241AC8
000007FE75244A7E nop
000007FE75244A7F add rsp,58h
000007FE75244A83 ret
--- No source file -------------------------------------------------------------
category:cq
theme:hardware-intrinsics
skill-level:expert
cost:large
voinokin
All 25 comments
That's because of the cast to byte.
mikedn
on 11 Sep 2018
...which makes things a bit funny in the context of C#/CLR due to roundtrips of most calculations on byte thru int ;-)
voinokin
on 11 Sep 2018
That's because of the cast to byte.
Yup, importer fails to recognize arg2 as const bcs it does not walk beyond cast:
[000008] ------------ arg2 \--* CAST int <- ubyte <- int
[000007] ------------ \--* CNS_INT int 1
Whole importer dump
*************** In impImport() for Repro.Program:Test()
impImportBlockPending for BB01
Importing BB01 (PC=000) of 'Repro.Program:Test()'
[ 0] 0 (0x000) call 2B000001
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
[ 1] 5 (0x005) stloc.0
[000005] ------------ * STMT void (IL 0x000... ???)
[000001] ---XG------- | /--* HWIntrinsic simd16 ubyte SetZeroVector128
[000004] -A-XG------- \--* ASG simd16 (copy)
[000002] D----------- \--* LCL_VAR simd16 V00 loc0
[ 0] 6 (0x006) ldloc.0
[ 1] 7 (0x007) sizeof 1B000002
[ 2] 13 (0x00d) conv.u1
[ 2] 14 (0x00e) call 0A000010
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
Known type Vector128<byte>
Calling impNormStructVal on:
[000006] ------------ * LCL_VAR simd16 V00 loc0
Known type Vector128<byte>
resulting tree:
[000012] x----------- * OBJ(16) simd16
[000011] L----------- \--* ADDR byref
[000006] ------------ \--* LCL_VAR simd16 V00 loc0
[ 1] 19 (0x013) stloc.0
[000017] ------------ * STMT void (IL 0x006... ???)
[000009] S-C-G------- \--* CALL void System.Runtime.Intrinsics.X86.Sse2.ShiftRightLogical128BitLane
[000015] L----------- arg0 +--* ADDR byref
[000014] ------------ | \--* LCL_VAR simd16 V00 loc0
[000012] x----------- arg1 +--* OBJ(16) simd16
[000011] L----------- | \--* ADDR byref
[000006] ------------ | \--* LCL_VAR simd16 V00 loc0
[000008] ------------ arg2 \--* CAST int <- ubyte <- int
[000007] ------------ \--* CNS_INT int 1
4creators
on 11 Sep 2018
After simple fix:
Instructions as they come out of the scheduler
G_M44985_IG01: ; func=00, offs=000000H, size=0005H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref, nogc <-- Prolog IG
IN0006: 000000 4883EC28 sub rsp, 40
IN0007: 000004 90 nop
G_M44985_IG02: ; func=00, offs=000005H, size=0017H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref
IN0001: 000005 C4E179EFC0 vpxor xmm0, xmm0, xmm0
IN0002: 00000A C4E17973D802 vpsrldq xmm0, xmm0, 2
IN0003: 000010 C4E37914C100 vpextrb ecx, xmm0, 0
[F582F0A8] ptr arg pop 0
IN0004: 000016 E845000000 call System.Console:WriteLine(int)
IN0005: 00001B 90 nop
G_M44985_IG03: ; func=00, offs=00001CH, size=0005H, epilog, nogc, emitadd
IN0008: 00001C 4883C428 add rsp, 40
IN0009: 000020 C3 ret
Allocated method code size = 33 , actual size = 33
No more poor codegen for HW intrinsics 馃槃
4creators
on 11 Sep 2018
Will this fix also handle the following code ? (attempt to work this issue around)
[MethodImpl(MethodImplOptions.NoInlining)]
static void Test<TChar>()
where TChar : unmanaged
{
Vector128<byte> v = Sse2.SetZeroVector128<byte>();
byte C;
if (sizeof(TChar) == sizeof(byte))
{
C = 1;
}
else if (sizeof(TChar) == sizeof(ushort))
{
C = 2;
}
else
{
throw new InvalidOperationException();
}
v = Sse2.ShiftRightLogical128BitLane(v, C);
Console.WriteLine(Sse41.Extract(v, 0));
}
Nope. It's too complex for codegen right now, but most importantly simple and equivalent approach works.
4creators
on 11 Sep 2018
...once you know it :-)
voinokin
on 11 Sep 2018
..which makes things a bit funny in the context of CLR due to roundtrips of most calculations on byte thru int ;-)
Yeah. Making those parameters byte instead of int was likely a bad idea.
But then JIT's importer could and should probably do some trivial constant folding. constant/cast sequences do appear in other situations as well. Most commonly I've seen this with long constants where the C# compiler decides to save space and emits ldc.i4 and conv.i8. The importer obeys and generates a cast node, that then the optimizer removes.
Usually it's not a good idea to mix IL importing with optimizations but there are some cases where that's reasonable, especially when dealing with IL's own encoding specifics/limitations.
mikedn
on 11 Sep 2018
But then JIT's importer could and should probably do some trivial constant folding.
In this fix importer does not do any const folding just recognizes that cast is done on const. Later stages take care of const folding sufficiently well to let them do the job. The problem here is that Roslyn does constant folding and evidently it missed arg2.
4creators
on 11 Sep 2018
...once you know it :-)
Yeah, right, it's not best of worlds we can have here.
4creators
on 11 Sep 2018
In this fix importer does not do any const folding just recognizes that cast is done on const.
Until someone tries something like 2 * sizeof(TChar).
mikedn
on 11 Sep 2018
Until someone tries something like 2 * sizeof(TChar).
This should be handled by Roslyn. Will check.
4creators
on 11 Sep 2018
This should be handled by Roslyn. Will check.
It's impossible, sizeof(TChar) is not a compile time constant.
mikedn
on 11 Sep 2018
@mikedn You are right. Added function to fold const expressions during import and it works as expected:
Test methods
```C#
[MethodImpl(MethodImplOptions.NoInlining)]
private static unsafe void Test
{
Vector128
v = Sse2.ShiftRightLogical128BitLane(v, (byte)sizeof(TChar));
Console.WriteLine(Sse41.Extract(v, 0));
}
[MethodImpl(MethodImplOptions.NoInlining)]
private static unsafe void Test1<TChar>() where TChar : unmanaged
{
Vector128<byte> v = Sse2.SetZeroVector128<byte>();
v = Sse2.ShiftRightLogical128BitLane(v, (byte)((3 * sizeof(TChar)) / 2 + 4 - sizeof(TChar)));
Console.WriteLine(Sse41.Extract(v, 0));
}
[MethodImpl(MethodImplOptions.NoInlining)]
private static unsafe void Test2<TChar>() where TChar : unmanaged
{
Vector128<byte> v = Sse2.SetZeroVector128<byte>();
byte C;
if (sizeof(TChar) == sizeof(byte))
{
v = Sse2.ShiftRightLogical128BitLane(v, 1);
}
else if (sizeof(TChar) == sizeof(ushort))
{
v = Sse2.ShiftRightLogical128BitLane(v, 2);
}
else
{
throw new InvalidOperationException();
}
Console.WriteLine(Sse41.Extract(v, 0));
}
Method Test
```asm
Instructions as they come out of the scheduler
G_M44985_IG01: ; func=00, offs=000000H, size=0005H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref, nogc <-- Prolog IG
IN0006: 000000 4883EC28 sub rsp, 40
IN0007: 000004 90 nop
G_M44985_IG02: ; func=00, offs=000005H, size=0017H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref
IN0001: 000005 C4E179EFC0 vpxor xmm0, xmm0, xmm0
IN0002: 00000A C4E17973D802 vpsrldq xmm0, xmm0, 2
IN0003: 000010 C4E37914C100 vpextrb ecx, xmm0, 0
[6A96D338] ptr arg pop 0
IN0004: 000016 E87DE6FFFF call System.Console:WriteLine(int)
IN0005: 00001B 90 nop
G_M44985_IG03: ; func=00, offs=00001CH, size=0005H, epilog, nogc, emitadd
IN0008: 00001C 4883C428 add rsp, 40
IN0009: 000020 C3 ret
Allocated method code size = 33 , actual size = 33
Method Test1
Instructions as they come out of the scheduler
G_M17352_IG01: ; func=00, offs=000000H, size=0005H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref, nogc <-- Prolog IG
IN0006: 000000 4883EC28 sub rsp, 40
IN0007: 000004 90 nop
G_M17352_IG02: ; func=00, offs=000005H, size=0017H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref
IN0001: 000005 C4E179EFC0 vpxor xmm0, xmm0, xmm0
IN0002: 00000A C4E17973D805 vpsrldq xmm0, xmm0, 5
IN0003: 000010 C4E37914C100 vpextrb ecx, xmm0, 0
[6A97D9A8] ptr arg pop 0
IN0004: 000016 E845000000 call System.Console:WriteLine(int)
IN0005: 00001B 90 nop
G_M17352_IG03: ; func=00, offs=00001CH, size=0005H, epilog, nogc, emitadd
IN0008: 00001C 4883C428 add rsp, 40
IN0009: 000020 C3 ret
Allocated method code size = 33 , actual size = 33
Method Test2
Instructions as they come out of the scheduler
G_M17451_IG01: ; func=00, offs=000000H, size=0005H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref, nogc <-- Prolog IG
IN0006: 000000 4883EC28 sub rsp, 40
IN0007: 000004 90 nop
G_M17451_IG02: ; func=00, offs=000005H, size=0017H, gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref
IN0001: 000005 C4E179EFC0 vpxor xmm0, xmm0, xmm0
IN0002: 00000A C4E17973D802 vpsrldq xmm0, xmm0, 2
IN0003: 000010 C4E37914C100 vpextrb ecx, xmm0, 0
[6A97DE18] ptr arg pop 0
IN0004: 000016 E855FEFFFF call System.Console:WriteLine(int)
IN0005: 00001B 90 nop
G_M17451_IG03: ; func=00, offs=00001CH, size=0005H, epilog, nogc, emitadd
IN0008: 00001C 4883C428 add rsp, 40
IN0009: 000020 C3 ret
Allocated method code size = 33 , actual size = 33
And in the importer we get:
Method Test:
*************** In impImport() for Repro.Program:Test()
impImportBlockPending for BB01
Importing BB01 (PC=000) of 'Repro.Program:Test()'
[ 0] 0 (0x000) call 2B000001
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
[ 1] 5 (0x005) stloc.0
[000005] ------------ * STMT void (IL 0x000... ???)
[000001] ------------ | /--* HWIntrinsic simd16 ubyte SetZeroVector128
[000004] -A---------- \--* ASG simd16 (copy)
[000002] D----------- \--* LCL_VAR simd16 V00 loc0
[ 0] 6 (0x006) ldloc.0
[ 1] 7 (0x007) sizeof 1B000002
[ 2] 13 (0x00d) conv.u1
[ 2] 14 (0x00e) call 0A000010
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
Folding operator with constant nodes into a constant:
[000008] ------------ * CAST int <- ubyte <- int
[000007] ------------ \--* CNS_INT int 2
Bashed to int constant:
[000008] ------------ * CNS_INT int 2
Known type Vector128<byte>
[ 1] 19 (0x013) stloc.0
[000013] ------------ * STMT void (IL 0x006... ???)
[000008] ------------ | /--* CNS_INT int 2
[000009] ------------ | /--* HWIntrinsic simd16 ubyte ShiftRightLogical128BitLane
[000006] ------------ | | \--* LCL_VAR simd16 V00 loc0
[000012] -A---------- \--* ASG simd16 (copy)
[000010] D----------- \--* LCL_VAR simd16 V00 loc0
Method Test1
*************** In impImport() for Repro.Program:Test1()
impImportBlockPending for BB01
Importing BB01 (PC=000) of 'Repro.Program:Test1()'
[ 0] 0 (0x000) call 2B000001
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
[ 1] 5 (0x005) stloc.0
[000005] ------------ * STMT void (IL 0x000... ???)
[000001] ------------ | /--* HWIntrinsic simd16 ubyte SetZeroVector128
[000004] -A---------- \--* ASG simd16 (copy)
[000002] D----------- \--* LCL_VAR simd16 V00 loc0
[ 0] 6 (0x006) ldloc.0
[ 1] 7 (0x007) ldc.i4.3 3
[ 2] 8 (0x008) sizeof 1B000002
[ 3] 14 (0x00e) mul
[ 2] 15 (0x00f) ldc.i4.2 2
[ 3] 16 (0x010) div
[ 2] 17 (0x011) ldc.i4.4 4
[ 3] 18 (0x012) add
[ 2] 19 (0x013) sizeof 1B000002
[ 3] 25 (0x019) sub
[ 2] 26 (0x01a) conv.u1
[ 2] 27 (0x01b) call 0A000010
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
Folding operator with constant nodes into a constant:
[000008] ------------ /--* CNS_INT int 2
[000009] ------------ * MUL int
[000007] ------------ \--* CNS_INT int 3
Bashed to int constant:
[000009] ------------ * CNS_INT int 6
Folding operator with constant nodes into a constant:
[000010] ------------ /--* CNS_INT int 2
[000011] ------------ * DIV int
[000009] ------------ \--* CNS_INT int 6
Bashed to int constant:
[000011] ------------ * CNS_INT int 3
Folding operator with constant nodes into a constant:
[000012] ------------ /--* CNS_INT int 4
[000013] ------------ * ADD int
[000011] ------------ \--* CNS_INT int 3
Bashed to int constant:
[000013] ------------ * CNS_INT int 7
Folding operator with constant nodes into a constant:
[000014] ------------ /--* CNS_INT int 2
[000015] ------------ * SUB int
[000013] ------------ \--* CNS_INT int 7
Bashed to int constant:
[000015] ------------ * CNS_INT int 5
Folding operator with constant nodes into a constant:
[000016] ------------ * CAST int <- ubyte <- int
[000015] ------------ \--* CNS_INT int 5
Bashed to int constant:
[000016] ------------ * CNS_INT int 5
Known type Vector128<byte>
[ 1] 32 (0x020) stloc.0
[000021] ------------ * STMT void (IL 0x006... ???)
[000016] ------------ | /--* CNS_INT int 5
[000017] ------------ | /--* HWIntrinsic simd16 ubyte ShiftRightLogical128BitLane
[000006] ------------ | | \--* LCL_VAR simd16 V00 loc0
[000020] -A---------- \--* ASG simd16 (copy)
[000018] D----------- \--* LCL_VAR simd16 V00 loc0
Method Test2
*************** In impImport() for Repro.Program:Test2()
impImportBlockPending for BB01
Importing BB01 (PC=000) of 'Repro.Program:Test2()'
[ 0] 0 (0x000) call 2B000001
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
[ 1] 5 (0x005) stloc.0
[000005] ------------ * STMT void (IL 0x000... ???)
[000001] ------------ | /--* HWIntrinsic simd16 ubyte SetZeroVector128
[000004] -A---------- \--* ASG simd16 (copy)
[000002] D----------- \--* LCL_VAR simd16 V00 loc0
[ 0] 6 (0x006) sizeof 1B000002
[ 1] 12 (0x00c) ldc.i4.1 1
[ 2] 13 (0x00d) bne.un.s
Folding operator with constant nodes into a constant:
[000007] ------------ /--* CNS_INT int 1
[000008] N--------U-- * NE int
[000006] ------------ \--* CNS_INT int 2
Bashed to int constant:
[000008] ------------ * CNS_INT int 1
The conditional jump becomes an unconditional jump to BB03
impImportBlockPending for BB03
Importing BB03 (PC=025) of 'Repro.Program:Test2()'
[ 0] 25 (0x019) sizeof 1B000002
[ 1] 31 (0x01f) ldc.i4.2 2
[ 2] 32 (0x020) bne.un.s
Folding operator with constant nodes into a constant:
[000011] ------------ /--* CNS_INT int 2
[000012] N--------U-- * NE int
[000010] ------------ \--* CNS_INT int 2
Bashed to int constant:
[000012] ------------ * CNS_INT int 0
The block falls through into the next BB04
impImportBlockPending for BB04
Importing BB04 (PC=034) of 'Repro.Program:Test2()'
[ 0] 34 (0x022) ldloc.0
[ 1] 35 (0x023) ldc.i4.2 2
[ 2] 36 (0x024) call 0A000010
In Compiler::impImportCall: opcode is call, kind=0, callRetType is struct, structSize is 16
Known type Vector128<byte>
Known type Vector128<byte>
[ 1] 41 (0x029) stloc.0
[000020] ------------ * STMT void (IL 0x022... ???)
[000015] ------------ | /--* CNS_INT int 2
[000016] ------------ | /--* HWIntrinsic simd16 ubyte ShiftRightLogical128BitLane
[000014] ------------ | | \--* LCL_VAR simd16 V00 loc0
[000019] -A---------- \--* ASG simd16 (copy)
[000017] D----------- \--* LCL_VAR simd16 V00 loc0
I also had a similar problem with the following code:
private static void Test()
{
Vector128<float> vec = Sse.SetZeroVector128();
vec = VectorInsertSingle(vec, 0, 1.0f);
vec = VectorInsertSingle(vec, 1, 1.5f);
vec = VectorInsertSingle(vec, 2, 2.0f);
vec = VectorInsertSingle(vec, 3, 2.5f);
}
public static Vector128<float> VectorInsertSingle(Vector128<float> vector, byte index, float value)
{
//Produces sub-optimal code (jump table fallback,
//JIT doesn't notice that the value is constant after inline).
return Sse41.Insert(vector, value, (byte)(index << 4));
}
After inline, the index value is constant, however the jump table fallback is still called. This is the assembly produced for Test:
00007FFD4B14361B vmovss xmm2,dword ptr [7FFD4B1436E0h]
00007FFD4B143624 xor r9d,r9d
00007FFD4B143627 call 00007FFD4B143360
00007FFD4B14362C mov rcx,qword ptr [rsp+70h]
00007FFD4B143631 mov rdx,qword ptr [rsp+78h]
00007FFD4B143636 lea r9,[rsp+60h]
00007FFD4B14363B lea rax,[rsp+20h]
00007FFD4B143640 mov qword ptr [rax],rcx
00007FFD4B143643 mov qword ptr [rax+8],rdx
00007FFD4B143647 mov rcx,r9
00007FFD4B14364A lea rdx,[rsp+20h]
00007FFD4B14364F vmovss xmm2,dword ptr [7FFD4B1436E4h]
00007FFD4B143658 mov r9d,10h
00007FFD4B14365E call 00007FFD4B143360
00007FFD4B143663 mov rcx,qword ptr [rsp+60h]
00007FFD4B143668 mov rdx,qword ptr [rsp+68h]
00007FFD4B14366D lea r9,[rsp+50h]
00007FFD4B143672 lea rax,[rsp+20h]
00007FFD4B143677 mov qword ptr [rax],rcx
00007FFD4B14367A mov qword ptr [rax+8],rdx
00007FFD4B14367E mov rcx,r9
00007FFD4B143681 lea rdx,[rsp+20h]
...
The values are constant (0, 0x10 etc), but it still produces the call.
Trying to workaround the issue I wrote another version using a switch case that use the constant values directly instead of relying on the JIT to do constant folding:
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static Vector128<float> VectorInsertSingle(Vector128<float> vector, byte index, float value)
{
switch (index)
{
case 0: return Sse41.Insert(vector, value, 0x00);
case 1: return Sse41.Insert(vector, value, 0x10);
case 2: return Sse41.Insert(vector, value, 0x20);
case 3: return Sse41.Insert(vector, value, 0x30);
default: throw new ArgumentOutOfRangeException(nameof(index));
}
}
This also doesn't work well because the JIT is not capable of inlining methods with switch cases so the code produced is also sub-optimal.
Finally we have this version using if/else:
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static Vector128<float> VectorInsertSingle(Vector128<float> vector, byte index, float value)
{
if (index == 0)
{
return Sse41.Insert(vector, value, 0x00);
}
else if (index == 1)
{
return Sse41.Insert(vector, value, 0x10);
}
else if (index == 2)
{
return Sse41.Insert(vector, value, 0x20);
}
else if (index == 3)
{
return Sse41.Insert(vector, value, 0x30);
}
else
{
throw new ArgumentOutOfRangeException(nameof(index));
}
}
This version produces more or less the expected code (this is the same Test method, calling the above VectorInsertSingle method):
00007FFD4B1235EC vxorps xmm0,xmm0,xmm0
00007FFD4B1235F1 vmovss xmm1,dword ptr [7FFD4B123668h]
00007FFD4B1235FA vinsertps xmm0,xmm0,xmm1,0
00007FFD4B123600 vmovapd xmmword ptr [rsp+50h],xmm0
00007FFD4B123607 vmovapd xmm0,xmmword ptr [rsp+50h]
00007FFD4B12360E vmovss xmm1,dword ptr [7FFD4B12366Ch]
00007FFD4B123617 vinsertps xmm0,xmm0,xmm1,10h
00007FFD4B12361D vmovapd xmmword ptr [rsp+40h],xmm0
00007FFD4B123624 vmovapd xmm0,xmmword ptr [rsp+40h]
00007FFD4B12362B vmovss xmm1,dword ptr [7FFD4B123670h]
00007FFD4B123634 vinsertps xmm0,xmm0,xmm1,20h
00007FFD4B12363A vmovapd xmmword ptr [rsp+30h],xmm0
00007FFD4B123641 vmovapd xmm0,xmmword ptr [rsp+30h]
00007FFD4B123648 vmovss xmm1,dword ptr [7FFD4B123674h]
00007FFD4B123651 vinsertps xmm0,xmm0,xmm1,30h
00007FFD4B123657 vmovapd xmmword ptr [rsp+20h],xmm0
00007FFD4B12365E add rsp,68h
00007FFD4B123662 ret
The method was properly inlined, however there are still some register moves that I believe that are not necessary (for example XMM0 is spilled then immediately moved back). Furthermore this instruction also accepts a memory location, so the whole method after inline could be just a single instruction, something like this:
vxorps xmm0,xmm0,xmm0
vinsertps xmm0,xmm0,dword ptr [7FFD4B123668h],0
vinsertps xmm0,xmm0,dword ptr [7FFD4B12366Ch],10h
vinsertps xmm0,xmm0,dword ptr [7FFD4B123670h],20h
vinsertps xmm0,xmm0,dword ptr [7FFD4B123674h],30h
I know there are more efficient ways to load a vector, this was just to demonstrate the usage of the VectorInsertSingle method.
Those tests were made with dotnet version 2.2.100-preview2-009404.
Another thing that may be worth pointing out is that the index argument name on the Vector128<float> overload of the Insert intrinsic method is a bit misleading, since it is not really index, the low bits are a zero mask and only the high bits are the actual index. One could assume it is just the destination index but that's not the case. Maybe its a good idea to rename it, I don't know if its worth opening a issue for that.
fyi @fiigii
Sorry if this is not the right place for that, but the issue is similar so I felt that opening a separate one was not necessary.
gdkchan
on 22 Sep 2018
Those tests were made with dotnet version 2.2.100-preview2-009404.
All new work on HW intrinsics is in master branch what means that it is not available in 2.2 preview2 and will not be available in 2.2 RTM
HW Intrinsic with immediate operand expansion
The method below cannot be handled correctly by Jit even with current not accepted solution from dotnet/coreclr#19898.
C#
public static Vector128<float> VectorInsertSingle(Vector128<float> vector, byte index, float value)
{
//Produces sub-optimal code (jump table fallback,
//JIT doesn't notice that the value is constant after inline).
return Sse41.Insert(vector, value, (byte)(index << 4));
}
The first problem is that Jit needs to recognize index variable as a const to be able to trigger HW intrinsic expansion and generate efficient code. Currently this happens only when value passed into intrinsic is a constant. Constant expressions in the form (byte)(index << 4) like in your code are not folded currently but they are handled correctly dotnet/coreclr#19898 under the assumption that index is a const or const expression. Unfortunately method parameter syntactically is not a const. There is a language proposal which could solve this https://github.com/dotnet/csharplang/pull/886 but it is not accepted.
There is still a second problem here caused by method inlining. Inlining happens after decision on intrinsic expansion is made and Jit currently will not reconsider if immediate HW intrinsic should be expanded after it is inlined and arguments semantics changes like in your code. As a result your method despite working on semantically valid assumptions for Jit is not syntactically equivalent to intrinsic with imm8 constant.
I have created issue to track the second problem as this may be important in many situations as long as we do not have const parameters implemented.
4creators
on 22 Sep 2018
@gdkchan Thanks for reporting the codegen issue. The problem is different from this issue but also detected in https://github.com/dotnet/coreclr/issues/17108.
@CarolEidt @AndyAyersMS @mikedn @tannergooding
Currently, all the codegen issues of imm-intrinsic are from two aspects:
- the importer does not fold some constant expressions, like
GT_CAST - the const arguments flow from the function parameters and the function can be inlined, but we expend intrinsic before inlining.
The first issue can be solved by morphing the expressions in the importer (like https://github.com/dotnet/coreclr/pull/19898), but both the two issues can be naturally solved by https://github.com/dotnet/coreclr/issues/17108. I know there is some difficulty to introduce call-node in lowering, can we just try to expend intrinsic behind inlining and constant propagation/folding?
fiigii
on 22 Sep 2018
but both the two issues can be naturally solved by dotnet/runtime#9989.
IMO we should reevaluate several problems:
We have mismatch between semantic requirements of imm intrinsics and C#/VB/F# semantics which will be impacting all future development of any intrinsic with immediate operands.
Current implementation despite providing ability to generate code for non const immediate parameters is undesired and avoided by users in other words it solves problem for compiler writers but does not solve users problems and users want high quality and fast code. The problem of codegen for non-const immediate parameters is raised in at least couple of issues.
We have deficiencies in importer and later compiler stages related to HW intrinsics handling. We do not fold constant expressions in importer, we do not reevaluate HW intrinsics expansion at later stages i.e. during morphing, value numbering, all optimizations happening before lowering and during lowering.
IMHO there is no single simple solution to problems which will be haunting imm HW intrinsic implementation, whether it would be dotnet/coreclr#19898 or dotnet/runtime#9989 or https://github.com/dotnet/csharplang/pull/886 or current implementation of non-const parameter handling.
Scenarios which as turns out should be supported are much more complex:
- support complex const expression folding
- support immediate parameters which are jit time const
- support semantic changes introduced by inlining
- support bona fide immediate non-const parameters
IMHO we should do:
(i) const expression folding in importer,
(ii) reevaluate HW intrinsic expansion once or more times after morphing, value numbering and following optimizations,
(iii) const parameter semantics in C#/VB/F# from https://github.com/dotnet/csharplang/pull/886,
(iv) have HW intrinsic overloads with non-const parameters what would allow for supporting of const expression which are constant only to Jit.
What could be helpful for (ii) it would be HW intrinsic support implementation for value numbering and following optimizations what could form a basis for further optimizations and even in more distant future autovectorization support.
4creators
on 22 Sep 2018
I know there is some difficulty to introduce call-node in lowering, can we just try to expend intrinsic behind inlining and constant propagation/folding?
Basically what you want is to expand on import and then attempt to expand again during lowering, hoping that non-const parameters have become const. Not expanding on import at all is not an option, you really don't want to keep a bunch of call nodes if you can avoid it. Allways expanding on import and falling back to call in lowering is not an option as well, at least not a trivial one. So you really don't have much of a choice.
Well, there's also the choice of telling people - hey, that parameter must be a constant. Period.
mikedn
on 23 Sep 2018
So, it turns out we already do some rewriting in rationalization. For GT_INTRINSIC nodes where there is the RewriteIntrinsicAsUserCall method, which calls down into RewriteNodeAsCall.
I had taken a look at utilizing the same logic for HWIntrinsics, but ran into a few problems, namely due to dealing with struct returns and having struct arguments as inputs:
- GT_HWINTRINSIC nodes need to become large nodes, as they need to carry the method handle
- Nodes also need to carry the entry point for the R2R compiler
- Remorphing the call fails because the
lvaRefCountStateis expected to beEARLY, but it is insteadNORMAL - Just remorphing the call args isn't sufficient, as there are some additional flags and the like that need to be handled for
TreatAsHasRetBufArg
- Due to having a ret buffer, the trees need to be shaped a bit differently and you can't consume the call node directly (since it has a void ret type)
I've given some examples of some code below and included the tree we have at rationalization when carrying intrinsics through vs when carrying the call through. In the "ideal" scenario, we would rewrite the intrinsic tree to be approx equivalent to the "call" tree.
tannergooding
on 4 Nov 2019
[MethodImpl(MethodImplOptions.AggressiveOptimization | MethodImplOptions.NoInlining)]
static Vector128<float> PermuteNoInline(Vector128<float> x, int control)
{
return Sse.Shuffle(x, x, (byte)control);
}
Intrinsic
***** BB01
STMT00000 (IL 0x000...0x009)
N013 ( 23, 19) [000014] -A-X----R--- * ASG simd16 (copy) $VN.Void
N012 ( 3, 2) [000013] D--X---N---- +--* IND simd16
N011 ( 1, 1) [000012] ------------ | \--* LCL_VAR byref V00 RetBuf u:1 $80
N010 ( 19, 16) [000011] ------------ \--* HWINTRINSIC simd16 float Shuffle $240
N009 ( 18, 15) [000010] ------------ \--* LIST void <l:$204, c:$203>
N002 ( 7, 5) [000007] n----------- +--* OBJ(16) simd16 <l:$140, c:$180>
N001 ( 1, 1) [000000] ------------ | \--* LCL_VAR byref V01 arg0 u:1 $81
N008 ( 10, 9) [000009] ------------ \--* LIST void <l:$202, c:$201>
N004 ( 7, 5) [000005] n----------- +--* OBJ(16) simd16 <l:$140, c:$181>
N003 ( 1, 1) [000001] ------------ | \--* LCL_VAR byref V01 arg0 u:1 (last use) $81
N007 ( 2, 3) [000008] ------------ \--* LIST void $200
N006 ( 2, 3) [000003] ------------ \--* CAST int <- ubyte <- int $1c0
N005 ( 1, 1) [000002] ------------ \--* LCL_VAR int V02 arg1 u:1 (last use) $c0
***** BB01
STMT00001 (IL ???... ???)
N002 ( 2, 2) [000016] ------------ * RETURN byref $2c0
N001 ( 1, 1) [000015] ------------ \--* LCL_VAR byref V00 RetBuf u:1 (last use) $80
Call
***** BB01
STMT00000 (IL 0x000...0x009)
N020 ( 47, 32) [000004] SACXG------- * CALL void System.Runtime.Intrinsics.X86.Sse.Shuffle $VN.Void
N003 ( 1, 3) [000017] -A------R-L- arg0 SETUP +--* ASG byref $80
N002 ( 1, 1) [000016] D------N---- | +--* LCL_VAR byref V06 tmp3 d:1 $80
N001 ( 1, 1) [000009] ------------ | \--* LCL_VAR byref V00 RetBuf u:1 $80
N007 ( 7, 5) [000013] -A--G---R-L- arg1 SETUP +--* ASG simd16 (copy) $VN.Void
N006 ( 3, 2) [000012] D------N---- | +--* LCL_VAR simd16(AX) V04 tmp1
N005 ( 3, 2) [000008] *---G--N---- | \--* IND simd16 <l:$180, c:$1c0>
N004 ( 1, 1) [000000] ------------ | \--* LCL_VAR byref V01 arg0 u:1 $81
N011 ( 7, 5) [000015] -A--G---R-L- arg2 SETUP +--* ASG simd16 (copy) $VN.Void
N010 ( 3, 2) [000014] D------N---- | +--* LCL_VAR simd16(AX) V05 tmp2
N009 ( 3, 2) [000006] *---G--N---- | \--* IND simd16 <l:$181, c:$1c1>
N008 ( 1, 1) [000001] ------------ | \--* LCL_VAR byref V01 arg0 u:1 (last use) $81
N013 ( 1, 1) [000018] ------------ arg0 in rcx +--* LCL_VAR byref V06 tmp3 u:1 (last use) $80
N015 ( 3, 3) [000020] ------------ arg1 in rdx +--* ADDR byref $141
N014 ( 3, 2) [000019] -------N---- | \--* LCL_VAR simd16(AX) V04 tmp1 $182
N017 ( 3, 3) [000022] ------------ arg2 in r8 +--* ADDR byref $142
N016 ( 3, 2) [000021] -------N---- | \--* LCL_VAR simd16(AX) V05 tmp2 $183
N019 ( 2, 3) [000003] ------------ arg3 in r9 \--* CAST int <- ubyte <- int $2c0
N018 ( 1, 1) [000002] ------------ \--* LCL_VAR int V02 arg1 u:1 (last use) $c0
***** BB01
STMT00001 (IL ???... ???)
N002 ( 2, 2) [000011] ------------ * RETURN byref $143
N001 ( 1, 1) [000010] ------------ \--* LCL_VAR byref V00 RetBuf u:1 (last use) $80
[MethodImpl(MethodImplOptions.AggressiveOptimization | MethodImplOptions.NoInlining)]
static void PermuteNoInlineNoReturn(Vector128<float> x, int control)
{
Sse.Shuffle(x, x, (byte)control);
}
Intrinsic
STMT00000 (IL 0x00A...0x00A)
N001 ( 0, 0) [000012] ------------ * RETURN void $c0
Call
***** BB01
STMT00000 (IL 0x000...0x009)
N021 ( 49, 32) [000004] SACXG------- * CALL void System.Runtime.Intrinsics.X86.Sse.Shuffle $VN.Void
N004 ( 3, 3) [000021] -A------R-L- arg0 SETUP +--* ASG long $1c0
N003 ( 1, 1) [000020] D------N---- | +--* LCL_VAR long V06 tmp4 d:1 $1c0
N002 ( 3, 3) [000010] ------------ | \--* ADDR long $1c0
N001 ( 3, 2) [000009] ----G--N---- | \--* LCL_VAR simd16(AX) V03 tmp1 $180
N008 ( 7, 5) [000017] -A--G---R-L- arg1 SETUP +--* ASG simd16 (copy) $VN.Void
N007 ( 3, 2) [000016] D------N---- | +--* LCL_VAR simd16(AX) V04 tmp2
N006 ( 3, 2) [000008] *---G--N---- | \--* IND simd16 <l:$181, c:$240>
N005 ( 1, 1) [000000] ------------ | \--* LCL_VAR byref V00 arg0 u:1 $80
N012 ( 7, 5) [000019] -A--G---R-L- arg2 SETUP +--* ASG simd16 (copy) $VN.Void
N011 ( 3, 2) [000018] D------N---- | +--* LCL_VAR simd16(AX) V05 tmp3
N010 ( 3, 2) [000006] *---G--N---- | \--* IND simd16 <l:$182, c:$241>
N009 ( 1, 1) [000001] ------------ | \--* LCL_VAR byref V00 arg0 u:1 (last use) $80
N014 ( 1, 1) [000022] ------------ arg0 in rcx +--* LCL_VAR long V06 tmp4 u:1 (last use) $1c0
N016 ( 3, 3) [000024] ------------ arg1 in rdx +--* ADDR byref $1c1
N015 ( 3, 2) [000023] -------N---- | \--* LCL_VAR simd16(AX) V04 tmp2 $183
N018 ( 3, 3) [000026] ------------ arg2 in r8 +--* ADDR byref $1c2
N017 ( 3, 2) [000025] -------N---- | \--* LCL_VAR simd16(AX) V05 tmp3 $184
N020 ( 2, 3) [000003] ------------ arg3 in r9 \--* CAST int <- ubyte <- int $300
N019 ( 1, 1) [000002] ------------ \--* LCL_VAR int V01 arg1 u:1 (last use) $c0
***** BB01
STMT00002 (IL 0x00A...0x00A)
N001 ( 0, 0) [000015] ------------ * RETURN void $380
[MethodImpl(MethodImplOptions.AggressiveOptimization | MethodImplOptions.NoInlining)]
static Vector128<float> SqrtPermuteNoInline(Vector128<float> x, int control)
{
return Sse.Sqrt(Sse.Shuffle(x, x, (byte)control));
}
Intrinsic
***** BB01
STMT00000 (IL 0x000...0x00E)
N014 ( 24, 20) [000015] -A-X----R--- * ASG simd16 (copy) $VN.Void
N013 ( 3, 2) [000014] D--X---N---- +--* IND simd16
N012 ( 1, 1) [000013] ------------ | \--* LCL_VAR byref V00 RetBuf u:1 $80
N011 ( 20, 17) [000012] ------------ \--* HWINTRINSIC simd16 float Sqrt $241
N010 ( 19, 16) [000011] ------------ \--* HWINTRINSIC simd16 float Shuffle $240
N009 ( 18, 15) [000010] ------------ \--* LIST void <l:$204, c:$203>
N002 ( 7, 5) [000007] n----------- +--* OBJ(16) simd16 <l:$140, c:$180>
N001 ( 1, 1) [000000] ------------ | \--* LCL_VAR byref V01 arg0 u:1 $81
N008 ( 10, 9) [000009] ------------ \--* LIST void <l:$202, c:$201>
N004 ( 7, 5) [000005] n----------- +--* OBJ(16) simd16 <l:$140, c:$181>
N003 ( 1, 1) [000001] ------------ | \--* LCL_VAR byref V01 arg0 u:1 (last use) $81
N007 ( 2, 3) [000008] ------------ \--* LIST void $200
N006 ( 2, 3) [000003] ------------ \--* CAST int <- ubyte <- int $1c0
N005 ( 1, 1) [000002] ------------ \--* LCL_VAR int V02 arg1 u:1 (last use) $c0
***** BB01
STMT00001 (IL ???... ???)
N002 ( 2, 2) [000017] ------------ * RETURN byref $2c0
N001 ( 1, 1) [000016] ------------ \--* LCL_VAR byref V00 RetBuf u:1 (last use) $80
Call
***** BB01
STMT00000 (IL 0x000...0x00E)
N021 ( 49, 32) [000004] SACXG------- * CALL void System.Runtime.Intrinsics.X86.Sse.Shuffle $VN.Void
N004 ( 3, 3) [000023] -A------R-L- arg0 SETUP +--* ASG long $1c0
N003 ( 1, 1) [000022] D------N---- | +--* LCL_VAR long V07 tmp4 d:1 $1c0
N002 ( 3, 3) [000010] ------------ | \--* ADDR long $1c0
N001 ( 3, 2) [000009] ----G--N---- | \--* LCL_VAR simd16(AX) V04 tmp1 $180
N008 ( 7, 5) [000019] -A--G---R-L- arg1 SETUP +--* ASG simd16 (copy) $VN.Void
N007 ( 3, 2) [000018] D------N---- | +--* LCL_VAR simd16(AX) V05 tmp2
N006 ( 3, 2) [000008] *---G--N---- | \--* IND simd16 <l:$181, c:$240>
N005 ( 1, 1) [000000] ------------ | \--* LCL_VAR byref V01 arg0 u:1 $81
N012 ( 7, 5) [000021] -A--G---R-L- arg2 SETUP +--* ASG simd16 (copy) $VN.Void
N011 ( 3, 2) [000020] D------N---- | +--* LCL_VAR simd16(AX) V06 tmp3
N010 ( 3, 2) [000006] *---G--N---- | \--* IND simd16 <l:$182, c:$241>
N009 ( 1, 1) [000001] ------------ | \--* LCL_VAR byref V01 arg0 u:1 (last use) $81
N014 ( 1, 1) [000024] ------------ arg0 in rcx +--* LCL_VAR long V07 tmp4 u:1 (last use) $1c0
N016 ( 3, 3) [000026] ------------ arg1 in rdx +--* ADDR byref $1c1
N015 ( 3, 2) [000025] -------N---- | \--* LCL_VAR simd16(AX) V05 tmp2 $183
N018 ( 3, 3) [000028] ------------ arg2 in r8 +--* ADDR byref $1c2
N017 ( 3, 2) [000027] -------N---- | \--* LCL_VAR simd16(AX) V06 tmp3 $184
N020 ( 2, 3) [000003] ------------ arg3 in r9 \--* CAST int <- ubyte <- int $300
N019 ( 1, 1) [000002] ------------ \--* LCL_VAR int V02 arg1 u:1 (last use) $c0
***** BB01
STMT00001 (IL ???... ???)
N005 ( 8, 6) [000015] -A-XG---R--- * ASG simd16 (copy) $VN.Void
N004 ( 3, 2) [000014] D--X---N---- +--* IND simd16
N003 ( 1, 1) [000013] ------------ | \--* LCL_VAR byref V00 RetBuf u:1 $80
N002 ( 4, 3) [000012] ----G------- \--* HWINTRINSIC simd16 float Sqrt $283
N001 ( 3, 2) [000011] ----G------- \--* LCL_VAR simd16(AX) V04 tmp1 $185
***** BB01
STMT00002 (IL ???... ???)
N002 ( 2, 2) [000017] ------------ * RETURN byref $1c3
N001 ( 1, 1) [000016] ------------ \--* LCL_VAR byref V00 RetBuf u:1 (last use) $80
Related issues
omariom
路
3Comments
noahfalk
路
3Comments
omajid
路
3Comments
EgorBo
路
3Comments
v0l
路
3Comments
Most helpful comment
Yeah. Making those parameters
byteinstead ofintwas likely a bad idea.But then JIT's importer could and should probably do some trivial constant folding. constant/cast sequences do appear in other situations as well. Most commonly I've seen this with long constants where the C# compiler decides to save space and emits
ldc.i4andconv.i8. The importer obeys and generates a cast node, that then the optimizer removes.Usually it's not a good idea to mix IL importing with optimizations but there are some cases where that's reasonable, especially when dealing with IL's own encoding specifics/limitations.