Roslyn: Proposal: Compiler intrinsics

Created on 21 May 2016 · 50Comments · Source: dotnet/roslyn

Background

C# and VB languages aren't able to express certain IL instructions and patterns (e.g. ldtoken, calli, ldftn, etc.). That's ok, it's not the job of a language to express all possible patterns of the underlying VM. However, there are scenarios where these patterns are very useful. In these cases users are currently left with options that are painful to use, break debugging, break IDE experience (refactorings etc.), are hard to maintain. These include various forms of IL rewriting, writing code in plain IL and compiling it to a separate assembly, runtime code generation, etc.

A few examples of such scenarios:

the CLR's tool for generating native interop stubs (MCG) would greatly benefit from the ability to emit IL instructions like calli, ldftn, etc.
infoof feature could be implemented as a library method if it was possible to directly emit ldtoken instruction
@joeduffy's implementation of slices currently needs some IL helpers compiled into a separate library.

If only Roslyn compilers could provide a way to emit these commonly needed IL patterns without changing the language specification ...

Proposal

A _compiler intrinsic_ is a static extern method declared in compilation source code that is marked with CompilerIntrinsicAttribute and its name is well-known to the compiler.

Each intrinsic provides a way to emit certain IL instruction or pattern that otherwise can’t be expressed in the C# language, using _existing syntax_ and _preserving standard evaluation stack behavior_. These intrinsics can thus be used in the middle of ordinary statements and expressions with no adverse effect on debugging, EnC, IntelliSense, refactorings, etc. Intrinsics have well-known names that determine the IL instruction pattern to emit and imply a signature pattern. The specific signature declared by the user along with the call-site arguments provide the compiler all the information it needs to emit the requested IL.

The declaration of an intrinsic won't be emitted to metadata. It can't be, as the type loader would not be able to load the containing type due to missing implementation for the intrinsics.

Additional constraints might need to be enforced on intrinsic call-sites and corresponding errors reported during binding phase. One of the constraints is that some arguments passed to the call-site of an intrinsic method must evaluate to a compile-time constant. This constraint is needed when the generated IL depends on the value of such arguments (e.g. mappedData in AddressOfMappedData).

Currently proposed intrinsics

```C#
namespace System.Runtime.CompilerServices
{
[AttributeUsage(AttributeTargets.Method)]
internal class CompilerIntrinsicAttribute : Attribute { }
}

class Program
{
[CompilerIntrinsic]
static unsafe extern void* LoadFunctionPointer(DelegateType target); // e.g. DelegateType = Func

[CompilerIntrinsic]
static unsafe extern void* LoadVirtualFunctionPointer(DelegateType target);

[CompilerIntrinsic]
static unsafe extern ReturnType CallIndirect(Type1 p1, ..., TypeN pN, void* managedFunctionPointer);

[CompilerIntrinsic]
static unsafe extern ReturnType CallIndirectCDecl(Type1 p1, ..., TypeN pN, void* nativeFunctionPointer);

[CompilerIntrinsic]
static unsafe extern ReturnType CallIndirectStdCall(Type1 p1, ..., TypeN pN, void* nativeFunctionPointer);

[CompilerIntrinsic]
static unsafe extern ReturnType CallIndirectThisCall(Type1 p1, ..., TypeN pN, void* nativeFunctionPointer);

[CompilerIntrinsic]
static unsafe extern DataType* AddressOfMappedData(DataType[] mappedData); // adds entry to FieldRVA table

}

## Generic Intrinsics

It might be useful to allow the declarations to be generic in order to reduce the amount of declarations required in the code, for example

```C#
class Program
{
    [CompilerIntrinsic]
    static unsafe extern void* LoadFunctionPointer<T1, ..., TN, TRet>(Func<T1, ..., Tn, TRet> target);

    [CompilerIntrinsic]
    static unsafe extern TRet CallIndirect<T1, ..., TN, TRet>(T1 p1, ..., TN pN, void* managedFunctionPointer);
}

Local Intrinsics

Provided that C# 7 allows _local functions_ to be declared as extern and custom attributes applied to them it would be possible to declare intrinsics "inline" (see https://github.com/dotnet/roslyn/issues/11731), for example:

``` C#
unsafe void IL()
{
[CompilerIntrinsic]
extern int CallIndirectCDecl(int arg0, void* nativeFunctionPointer);

Console.WriteLine(CallIndirectCDecl(123, GetAddressOfNativeFunction()));

}

## Possible future intrinsics

The compiler can define any number of intrinsics and add more as needed. The following example demonstrates what would be possible. The exact set of intrinsics to introduce is up for a debate. 

Note that the following code compiles and works in the current version of C# compiler and IDE services. It just doesn't compile to the desired IL.

``` C#
class Program
{
    [CompilerIntrinsic]
    static unsafe extern int CallIndirect(string arg0, bool arg1, void* functionPointer);

    [CompilerIntrinsic]
    static unsafe extern int CallIndirectCDecl(int arg0, void* functionPointer);

    [CompilerIntrinsic]
    static unsafe extern int TailCallIndirect(string arg0, bool arg1, void* functionPointer);

    [CompilerIntrinsic]
    static unsafe extern void TailCallIndirectHasThis(void* functionPointer);

    [CompilerIntrinsic]
    static unsafe extern void* LoadFunctionPointer(Func<string, bool, int> target);

    [CompilerIntrinsic]
    static unsafe extern void* LoadFunctionPointer(Action target);

    [CompilerIntrinsic]
    static unsafe extern void* LoadVirtualFunctionPointer(Func<int, int> target);

    [CompilerIntrinsic]
    static extern RuntimeTypeHandle LoadTypeToken<T>();

    [CompilerIntrinsic]
    static extern RuntimeMethodHandle LoadMethodToken(Func<int, int> target);

    [CompilerIntrinsic]
    static extern RuntimeMethodHandle LoadGetterToken<T>(T targetProperty);

    [CompilerIntrinsic]
    static extern RuntimeMethodHandle LoadSetterToken<T>(T targetProperty);

    [CompilerIntrinsic]
    static extern RuntimeFieldHandle LoadFieldToken<T>(T targetField);

    [CompilerIntrinsic]
    static extern int LoadTypeTokenInt32<T>();

    [CompilerIntrinsic]
    static extern int LoadMethodTokenInt32(Func<int, int> target);

    [CompilerIntrinsic]
    static extern int LoadFieldTokenInt32<T>(T targetField);

    [CompilerIntrinsic]
    static extern int LoadGetterTokenInt32<T>(T targetProperty);

    [CompilerIntrinsic]
    static extern int LoadSetterTokenInt32<T>(T targetProperty);

    [CompilerIntrinsic]
    static extern string LoadMvidString();


    abstract class C<T>
    {
        public C(int a) { }
        public abstract int F(int a);
    }

    public void F() { }
    public static T G<T>(T a) => default(T);
    public static int H(string s, bool b) => 1;

    public static string fld;

    public static int P { get; set; }

    private unsafe void* GetAddressOfNativeFunction() => null;

    void IL()
    {
        // ldvirtftn C<string>.F
        // pop
        unsafe { LoadVirtualFunctionPointer(default(C<string>).F); }

        // ldstr "str"
        // ldc.i4.1
        // ldftn H
        // calli int(string, bool)
        // box
        // call Console.WriteLine(object)
        unsafe 
        {
            Console.WriteLine(CallIndirect("str", true, LoadFunctionPointer(H))); 
         }

        // ldc.i4 123
        // call GetAddressOfNativeFunction()
        // calli int(int)
        // box
        // call Console.WriteLine(object)
        unsafe 
        {
            Console.WriteLine(CallIndirectCDecl(123, GetAddressOfNativeFunction())); 
        }

        LoadTypeToken<Program>();                 // ldtoken Program
        LoadMethodToken(default(C<bool>).F);      // ldtoken C<bool>.F
        LoadMethodToken(G<int>);                  // ldtoken G<int>
        LoadFieldToken(fld);                      // ldtoken fld
        LoadGetterToken(P);                       // ldtoken get_P
        LoadSetterToken(P);                       // ldtoken set_P

        LoadTypeTokenInt32<Program>();            // ldc.i4 <token of Program>
        LoadMethodTokenInt32(default(C<bool>).F); // ldc.i4 <token of C<bool>.F>
        LoadFieldTokenInt32(fld);                 // ldc.i4 <token of fld>
        LoadGetterTokenInt32(P);                  // ldc.i4 <token of get_P>
        LoadSetterTokenInt32(P);                  // ldc.i4 <token of set_P>
        LoadMvidString();                         // ldstr "<containing module mvid>"

        // ldarg.0
        // ldftn F
        // tail.calli void()
        unsafe
        {
           TailCallIndirectHasThis(LoadFunctionPointer(F)); 
         }
    }
}

0 - Backlog Area-Language Design Feature Request

Source

tmat

👍47 ❤8

Most helpful comment

After a few hours of work, here we go: http://xoofx.com/blog/2016/05/25/inline-il-asm-in-csharp-with-roslyn/ :tada:

xoofx on 25 May 2016

👍24

All 50 comments

@jkotas @yizhang82 @smosier @stephentoub @MadsTorgersen @gafter @dotnet/roslyn-compiler

tmat on 21 May 2016

Looks exactly how I had imagined, @jaredpar and I were talking about this afternoon, so I'm super happy to see this!

mjsabby on 21 May 2016

How do we feel about not taking a dependency on CoreFX defining the CompilerIntrinsic attribute, instead allowing the compiler to make a decision about which attribute to look for via an argument, or (and this may be heavy handed) making a keyword out of this.

I'd hate for this feature to be gated on your need to upgrade the compiler AND your standard library.

mjsabby on 21 May 2016

There is no dependency on CoreFX defining the attribute. The attribute can be defined in source as it is demonstrated in the example above (notice the internal visibility). The compiler doesn't require any attribute to be defined in a specific assembly. It just looks for its namespace qualified name and particular signature of the attribute constructor.

In other words you can just define the attribute anywhere in your project.

tmat on 21 May 2016

Gotcha, thanks.

mjsabby on 21 May 2016

Event adders/removers seem to be missing from the example.

0xd4d on 21 May 2016

How would LoadMethodToken work with overload resolution? For example, if I wanted the tokens of both Console.WriteLine(object) and Console.WriteLine(string), would the following work?

``` c#
class Program
{
[CompilerIntrinsic]
static unsafe extern void* LoadFunctionPointer(Action

Roslyn: Proposal: Compiler intrinsics

Background

Proposal

Currently proposed intrinsics

Local Intrinsics

Most helpful comment

All 50 comments

Related issues