Runtime: Helper class for dealing with native shared libraries and function pointers

[mellinoe]

Approved Proposal

From https://github.com/dotnet/corefx/issues/17135#issuecomment-365364216

```C#
public sealed class NativeLibrary
{
public static bool TryLoad(string name, Assembly caller, DllImportSearchPaths paths, out NativeLibrary result);

public IntPtr Handle { get; }

public bool TryGetDelegate<TDelegate>(string name, out TDelegate result) where TDelegate : class;
public bool TryGetDelegate<TDelegate>(string name, bool exactSpelling, out TDelegate result) where TDelegate : class;
public bool TryGetSymbolAddress(string symbolName, out IntPtr result);

}

# Original proposal

Note: Proposal updated based on discussion: https://github.com/dotnet/corefx/issues/17135#issuecomment-353571556

## Background

Many popular C-callable shared libraries exist which do not have a consistent name across platforms. Examples include:

* SDL2
* OpenAL
* Vulkan
* Many more (the above are just ones I've used personally)

CoreCLR does not support any notion of "remappable" PInvokes, a la DllImport in Mono. The name of the shared library must be encoded directly in the `DllImport` attribute, which makes it impossible to create a single wrapper assembly which works on many platforms. This imposes an unnecessary development and deployment burden on library developers who could otherwise ship a single, simple DLL. Many third-party libraries which rely on PInvoke's only work on Mono platforms because of the DllMap feature. These libraries do not work on .NET Core at all, even if they are otherwise completely compatible with its profile.

The alternative to using `[DllImport]` is to write logic which manually opens the shared library, discovers function pointers, and converts them to managed delegates. In a sense, this very similar to what we are doing with our "Portable Linux" version of the runtime and native shims. However, this code can be complicated, tedious, and error-prone. This proposal is for a small helper library, with optional cooperation by the runtime, to make this scenarion easier.

## Proposed API

```C#
namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public class NativeLibrary : IDisposable
    {
        // The default search paths
        public static IEnumerable<string> NativeLibrarySearchDirectories { get; }

        // The operating system handle of the loaded library.
        public IntPtr Handle { get; }

        // Constructs a new NativeLibrary using the platform's default library loader.
        public NativeLibrary(string name);
        public NativeLibrary(string name, DllImportSearchPath paths);
        public static NativeLibrary Open(string name, DllImportSearchPath paths, bool throwOnError);

        // Loads a native function pointer by name.
        public IntPtr LoadFunction(string name);

        // Loads a function whose signature matches the given delegate type's signature.
        // This is roughly equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        public T LoadDelegate<T>(string name);

        // Lookup a symbol (not a function) from the specified dynamic library
        public IntPtr SymbolAddress(string name);

        // Frees the native library. Function pointers retrieved from this library will be void.
        public void Dispose();
    }
}

Usage

```C#
public static class Sdl2
{
private static NativeLibrary s_sdl2NativeLib = new NativeLibrary(GetSdl2LibraryName());
// Determine library name at runtime. This could be influenced by a DllMap-like policy if desired.
private static string GetSdl2LibraryName();

private delegate IntPtr SDL_CreateWindow_d(string title, int x, int y, int w, int h, int flags);
private static SDL_CreateWindow_d SDL_CreateWindow_ptr = s_sdl2NativeLib.LoadFunctionPointer<SDL_CreateWindow_d>("SDL_CreateWindow");
public static IntPtr CreateWindow(string title, int x, int y, int w, int h, uint flags)
    => SDL_CreateWindow_ptr(title, x, y, w, h, flags);

}

public static void Main()
{
IntPtr window = Sdl2.CreateWindow("WindowTitle", 100, 100, 960, 540, 0);
}
```

Open Design Questions

Probing Path Logic

CoreCLR applies particular probing logic when discovering shared libraries listed in a DllImport. Ideally, calling new NativeLibrary("lib") would follow the same probing logic as [DllImport("lib")], so that PInvoke's can be easily converted. This could potentially be solved with some internal or public runtime helper function exposed from System.Private.CoreLib. Alternatively, the constructor forNativeLibrary` could include another parameter which controlled probing logic, allowing someone to plug their own logic in.

Generic delegate types

The simplest way to convert from a native function pointer to a clean managed delegate is through the use of Marshal.GetDelegateForFunctionPointer<T>(IntPtr). Unfortunately, this cannot be used with generic delegates, e.g. Func<string, int, int, int, int, int, IntPtr> in the above example. Given that many native function signatures include pointer types, which cannot be used as generic type arguments, other than as IntPtr, this is not a major problem. However, many cases would benefit from being able to use Action and Func types in the LoadFunction<T> method. We could probably allow this using Reflection.Emit, but it would most likely be ugly, complicated, and slower than if custom delegate types were used. If Marshal.GetDelegateForFunctionPointer<T>(IntPtr) accepted generic delegate types, this feature would greatly benefit.

Disposability and lifetime tracking

The constructor of NativeLibrary involves opening an operating system handle to a native shared library, via LoadLibrary, dlopen, etc. I have proposed that NativeLibrary be disposable, which would involve calling FreeLibrary, dlclose, etc. Is this desirable or useful?

A related issue is how the runtime currently tracks and manages these handles. Should handles opened by NativeLibrary be coordinated and tracked in the same way that other handles (via PInvoke, etc.) are?

Prototype

I have a prototype version of this library implemented here: https://github.com/mellinoe/nativelibraryloader, and I have successfully used the pattern described here in a few projects.

@yizhang82 @janvorli @danmosemsft @conniey

[danmosemsft]

You've eliminated the option of mimicing Mono's solution (DllMaps, as you mention)?
Would this API ultimately work for Mono, making DllMap unnecessary?

[akoeplinger]

Related discussion (which you're probably aware of): https://github.com/dotnet/coreclr/issues/930

[mellinoe]

@danmosemsft See the link that @akoeplinger shared.

[akoeplinger]

@mellinoe since my thread is pretty old by now, do you know what the current status of MCG (which was proposed as the solution by .NET runtime folks) is?

[mellinoe]

I don't know. I'd be interested to hear an update from @yizhang82 about it.

[IllidanS4]

Nobody uses unsafe pointers in extern methods, only IntPtr or managed equivalents. Usage of generic delegates should be allowed in _GetDelegateForFunctionPointer_, and frankly I don't see what difficulties would implementing it have. On the other hand, I think that the "LoadFunction" doesn't allow for as much marshalling options as the DllImport system can. What if you could specify "instance extern" methods on types inheriting from _NativeLibrary_, which would be automatically imported from the library with the specified marshalling options?

[mellinoe]

Nobody uses unsafe pointers in extern methods, only IntPtr or managed equivalents.

It is very common to use unsafe pointers in extern methods, in my experience. I certainly do it in a lot of my libraries, and I've seen it in many libraries from others, as well. Wrapping things into IntPtr's is unnecessarily verbose when you're already doing fundamentally unsafe things. Now, it's also very common to "wrap" the PInvokes into methods will fully-safe signatures, but those call the unsafe version in turn.

What if you could specify "instance extern" methods on types inheriting from NativeLibrary, which would be automatically imported from the library with the specified marshalling options?

That sounds like it would involve a lot of external machinery (IL rewriting), or actual C# language support. Certainly an interesting idea, but probably outside of scope here.

@yizhang82 Do you have any feedback for this proposal?

[akoeplinger]

Another problem with the proposed API is that calling dlopen() with arbitrary native libraries is prohibited on iOS and recent Androids so this wouldn't work there (i.e. unusable by Xamarin).

[mellinoe]

Another problem with the proposed API is that calling dlopen() with arbitrary native libraries is prohibited on iOS and recent Androids so this wouldn't work there (i.e. unusable by Xamarin).

That is a good point, and it may also be the case for UWP applications.

[qmfrederik]

Just my 2 cents on this:

• On operating systems where you cannot load a native library, this could throw a PlatformNotSupportedException, right? (Although I believe that at least on iOS, you can load native libraries which are part of your application bundle.)
• I like the idea to keep a dll-map like functionality separate from this API. It makes sense to me that this is a lowel-level API which is close to dlopen/LoadLibrary where you can pass an explicit, full path to the library you want to load. Nothing prevents you from implementing dllmap-like functionality in a separate API, which you can use to resolve the path to the library you want to load
• I'd suggest adding a static property NativeLibrarySearchDirectories which exposes AppContext.GetData("NATIVE_DLL_SEARCH_DIRECTORIES"), hence:

namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public class NativeLibrary : IDisposable
    {
+       // The default search paths
+       public string[] NativeLibrarySearchDirectories { get; }

        // The operating system handle of the loaded library.
        public IntPtr Handle { get; }

        // Constructs a new NativeLibrary using the platform's default library loader.
        public NativeLibrary(string name);

        // Loads a native function pointer by name.
        public IntPtr LoadFunction(string name);

        // Loads a function whose signature matches the given delegate type's signature.
        // This is roughly equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        public T LoadDelegate<T>(string name);

        // Frees the native library. Function pointers retrieved from this library will be void.
        public void Dispose();
    }
}

[migueldeicaza]

This wrapper looks good, additionally, I would add the following API, that would lookup a symbol (not a function) from the specified dynamic library:

IntPtr SymbolAddress (string name);

This would be useful to access global variables and other global symbols in the loaded library.

[tmds]

@mellinoe can this be used to perform lookups using dlvsym instead of dlsym on Linux? dlvsym is a Linux specific mechanism that allows symbols to be versioned. It allows a library to provide different versions of the same symbol. The dlsym lookup (that is used in coreclr) always gives you the oldest version of the symbol. See https://github.com/dotnet/coreclr/issues/8721 for more info.

[qmfrederik]

@tdms Looking at https://linux.die.net/man/3/dlvsym and https://www.gnu.org/software/gnulib/manual/html_node/dlvsym.html, dlvsym is a glibc-extension. This means it's not available on non-glibc Linux, BSD, macOS and Windows (GetProcAddress doesn't seem to take a version parameter, either).

I'm wondering whether it makes sense to add support for a glibc-extension in the core API?

[tmds]

I'm wondering whether it makes sense to add support for a glibc-extension in the core API?

I'm not requesting dlvsym to be baked in .NET Core. I tried this with https://github.com/dotnet/coreclr/issues/8721 (+PR) and it was not accepted.
I got feedback this should be done via some extension mechanism. I believe this proposal could fit that use-case.

[qmfrederik]

@tmds Well with this API you should be able to do something like this, right:

using(var dl = new NativeLibrary("dl"))
{
    var dlvsym = dl.LoadDelegate<dlvsym_delegate>("dlvsym");
}

I do hope that the NativeLibrary API will make it into .NET core, though ;-)

[tmds]

I think in this proposal, I'm looking to override LoadFunction. So e.g. LoadFunction("foo@VERS_1.1") calls dlvsym(Handle, "foo", "VERS1_1");.

[tmds]

The constructor of NativeLibrary involves opening an operating system handle to a native shared library, via LoadLibrary, dlopen, etc. I have proposed that NativeLibrary be disposable, which would involve calling FreeLibrary, dlclose, etc. Is this desirable or useful?

What happens when someone uses a delegate after disposing the library?

A related issue is how the runtime currently tracks and manages these handles. Should handles opened by NativeLibrary be coordinated and tracked in the same way that other handles (via PInvoke, etc.) are?

Are libraries used via PInvoke ever closed? When?

[ghost]

What happens when someone uses a delegate after disposing the library?

simple ObjectDisposedException?

[qmfrederik]

Adding the area owners @russellhadley @luqun @shrah to this thread.

One of the issues we're trying to solve is how managed code can load native libraries in a cross-platform way.

For example, System.Drawing.Common depends on libgdiplus which ships in various forms in various distros. So the code probes for a couple of files and loads the correctly library using dlopen.

One of the limitations we've hit is that dlopen also lives in separate libraries - libdl.so on most Unixes but sometimes also libdl.so.2 (CentOS) or libc (FreeBSD).

I've tried to address that in dotnet/corefx#25134 by adding dlopen to the PAL but the consensus on that issue was that System.Drawing.Common cannot call the PAL directly and any wrapper around dlopen needs to be exposed in a separate API which is part of corefx.

Hence this issue 😄 .

Can you give your feedback on this API proposal?

[jkotas]

We may need a constructor overload that takes DllImportSearchPath to control the probing paths.

[qmfrederik]

@jkotas Something like this?

namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public class NativeLibrary : IDisposable
    {
+       // The default search paths
+       public static IEnumerable<string> NativeLibrarySearchDirectories { get; }

        // Constructs a new NativeLibrary using the platform's default library loader.
        public NativeLibrary(string name);

+       public NativeLibrary(string name, DllImportSearchPath paths);

+       // Constructs a new NativeLibrary using user-provided search libraries.
+       public static NativeLibrary Open(string name, DllImportSearchPath paths, bool throwOnError)

        // The operating system handle of the loaded library.
        public IntPtr Handle { get; }

        // Loads a native function pointer by name.
        public IntPtr LoadFunction(string name);

+       // Lookup a symbol (not a function) from the specified dynamic library
+       public IntPtr SymbolAddress (string name);

        // Loads a function whose signature matches the given delegate type's signature.
        // This is roughly equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        public T LoadDelegate<T>(string name);

        // Frees the native library. Function pointers retrieved from this library will be void.
        public void Dispose();
    }
}

[jkotas]

I meant something like this:

public class NativeLibrary : IDisposable
{
    public NativeLibrary(string name);

+   public NativeLibrary(string name, DllImportSearchPath paths);

...
}

DllImportSearchPath is existing enum, used together with regular DllImport today.

[jkotas]

Also, to allow implementation custom probing logic like the one in https://github.com/dotnet/corefx/blob/35d0838c20965c526e05e119028dd7226084987c/src/System.Drawing.Common/src/System/Drawing/GdiplusNative.Unix.cs#L39 we may want to have non-throwing factory like:

public class NativeLibrary : IDisposable
{
+   public static NativeLibrary Open(string name, DllImportSearchPath paths, bool throwOnError);

...
}

[qmfrederik]

@jkotas Ah, that makes sense. I updated the API spec.

So the libgdiplus probing logic could be rewritten to something like this:

NativeLibrary libgdiplus = null;
string libraryName = null;

if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
    libraryName = "libgdiplus.dylib";
    libgdiplus = NativeLibrary.Open("libgdiplus.dylib", DllImportSearchPath.SafeDirectories, throwOnError: false);
}
else
{
    // Various Unix package managers have chosen different names for the "libgdiplus" shared library.
    // The mono project, where libgdiplus originated, allowed both of the names below to be used, via
    // a global configuration setting. We prefer the "unversioned" shared object name, and fallback to
    // the name suffixed with ".0".
    libraryName = "libgdiplus.so";
    libgdiplus = NativeLibrary.Open("libgdiplus.so", DllImportSearchPath.SafeDirectories, throwOnError: false);
    if (libgdiplus == null)
    {
        libgdiplus = NativeLibrary.Open("libgdiplus.so.0", DllImportSearchPath.SafeDirectories, throwOnError: false);
    }
}

// If we couldn't find libgdiplus in the system search path, try to look for libgdiplus in the
// NuGet package folders. This matches the DllImport behavior.
if (libgdiplus ==null)
{
    libgdiplus = NativeLibrary.Open(libraryName, DllImportSearchPath.ApplicationDirectory,, throwOnError: false);
}

Assuming SafeDirectories and ApplicationDirectory are the DllImportSearchPath values to to express "search in system paths only" vs "search in NuGet packages"

[karelz]

@qmfrederik did you leave out @jkotas's constructor overload on purpose?

Also, should NativeLibrarySearchDirectories be IEnumerable<string> instead of string[] to prevent mutation of the array?

[joelday]

@karelz the interface alone doesn't really enforce immutability. There a standard pattern for this in corefx? I should read up on that. :)

[karelz]

I don't think there is a standard pattern documented. But in general I think we try to expose interfaces instead of underlying collections (at least I vaguely remember that being discussed in API reviews).
Evil users can cast to the underlying collection - that is not something we typically guard against. Breaking such users might not be considered a breaking change though.
Technically speaking we could create internal type that encapsulates the collection and exposes its items, but that is usually too heavy-weight for us to use.

[qmfrederik]

@karelz That was an oversight on my side, although having both a constructor and a static Open method may be a bit of overkill.

Good point on IEnumerable vs string[]. I changed that; it does help communicate the values are read-only.

[karelz]

Top post updated with latest proposal.
Naming-wise we should probably rename NativeLibrarySearchDirectories to DefaultSearchDirectories.

@jkotas can you mark it api-ready-for-review when you are ok with the API shape?

[jkotas]

My comments about API shape:

• How are LoadFunction and SymbolAddress different, except for their name? Is their implementation going to differ in any way?

• LoadFunction is not loading anything. It is just doing lookup in what was loaded already. Should it be rather called GetFunction?

• Add non-throwing variant of LoadFunction to allow probing for entrypoints that may not exist?

This is roughly equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer

Is it roughly equivalent, or exactly equivalent? It would be nice it LoadDelegate kept the delegates created by it alive so that people do not have to do it manually.

[qmfrederik]

• LoadFunction vs SymbolAddress: I think we can drop LoadFunction now that we have SymbolAddress.
• Yes, probably better names would be GetSymbolAddress and Get(Function)Delegate
• I always assumed (although it was never made explicit) that the functions would return IntPtr.Zero/null if they fail. I added a throwOnError, this way the caller can decide which behavior they want.

Do you think we need both the static Open method and the public constructors?

So something like this:

namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public class NativeLibrary : IDisposable
    {
       // Returns the default search paths
       public static IEnumerable<string> NativeLibrarySearchDirectories { get; }

        // Constructs a new NativeLibrary using the platform's default library loader.
        public NativeLibrary(string name);

        // Constructs a new NativeLibrary using the search paths defined by the value of paths
       public NativeLibrary(string name, DllImportSearchPath paths);

       // Constructs a new NativeLibrary using user-provided search libraries, using the search paths
       // defined by the value of paths and optionally returning null instead of throwing an error.
       public static NativeLibrary Open(string name, DllImportSearchPath paths, bool throwOnError)

        // The operating system handle of the loaded library.
        public IntPtr Handle { get; }

       // Lookup a symbol (including functions) from the specified dynamic library and returns the
       // the address of that symbol. Returns IntPtr.Zero if the symbol cannot be found and throwOnError
       // is false; throws an exception otherwise.
       public IntPtr GetSymbolAddress (string name, bool throwOnError = false);

        // Gets a function whose signature matches the given delegate type's signature.
        // This is equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        // Returns null if the function cannot be found and throwOnError is false; throws an exception
        // otherwise.
        public T GetDelegate<T>(string name, bool throwOnError = false);

        // Frees the native library. Function pointers retrieved from this library will be void.
        public void Dispose();
    }
}

[jkotas]

Do you think we need both the static Open method and the public constructors?

I do not think we need the public constructors. (If you remove them, you can also make the type sealed.)

[qmfrederik]

Great! I also renamed NativeLibrarySearchDirectories to SearchDirectories as it would just duplicated NativeLibrary. I think it looks much better now.

namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public sealed class NativeLibrary : IDisposable
    {
        // Returns the default search paths
        public static IEnumerable<string> SearchDirectories { get; }

        // Constructs a new NativeLibrary using user-provided search libraries, using the search
        // paths defined by the value of paths and optionally returning null instead of throwing
        // an error.
        public static NativeLibrary Open(string name, DllImportSearchPath paths,
            bool throwOnError = true);

        // The operating system handle of the loaded library.
        public IntPtr Handle { get; }

        // Lookup a symbol (including functions) from the specified dynamic library and returns
        // the address of that symbol. Returns IntPtr.Zero if the symbol cannot be found and
        // throwOnError is false; throws an exception otherwise.
        public IntPtr GetSymbolAddress(string name, bool throwOnError = false);

        // Gets a function whose signature matches the given delegate type's signature.
        // This is equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        // Returns null if the function cannot be found and throwOnError is false; throws an
        // exception otherwise.
        public T GetDelegate<T>(string name, bool throwOnError = false);

        // Frees the native library. Function pointers and delegates retrieved from this library will be void.
        // Using function pointers and delegates returned by this library after calling dispose will
        // result in undefined behavior.
        public void Dispose();
    }
}

[tmds]

The current proposal doesn't support turning a pointer returned by dlvsym into a delegate. (https://github.com/dotnet/corefx/issues/17135#issuecomment-343844607)

It's also not clear whether/how the behavior changes of the delegates returned by GetDelegate once the NativeLibrary.Dispose method is called. (https://github.com/dotnet/corefx/issues/17135#issuecomment-344507664)

[qmfrederik]

The current proposal doesn't support turning a pointer returned by dlvsym into a delegate.

That's correct, and because dlvsym is glibc-specific I don't know how to create a general API that supports that. You can use this API to P/invoke into dlvsym, though (https://github.com/dotnet/corefx/issues/17135#issuecomment-343850978).

It's also not clear whether/how the behavior changes of the delegates returned by GetDelegate once the NativeLibrary.Dispose method is called.

When you call NativeLibrary.Dispose(), the underlying native library is freed. All pointers to symbols are void, using delegates returned by GetDelegate will result in undefined behavior (most likely access violations).

I've updated the Dispose comments.

[tmds]

That's correct, and because dlvsym is glibc-specific I don't know how to create a general API that supports that. You can use this API to P/invoke into dlvsym, though (#17135 (comment)).

I'm looking for a way to turn the pointer returned by dlvsym into to a delegate, e.g.
C# public T GetDelegate<T>(IntPtr address, bool throwOnError = false);

Using delegates returned by GetDelegate will result in undefined behavior (most likely access violations). I've updated the Dispose comments.

:+1:

[tmds]

I'm looking for a way to turn the pointer returned by dlvsym into to a delegate, e.g.

After some googling, I learned this is already available: Marshal.GetDelegateForFunctionPointer :smile:

[qmfrederik]

@tmds Cool, looks like we're all set then ;-)

[karelz]

FYI: The API review discussion was recorded - see https://youtu.be/dOpH6bUNbcw?t=2391 (5 min duration)

Key takeaways:

• We need @jkotas in the room next time, to talk about motivation details, etc.
• Unloading opens door to security vulnerabilities - either the delegates have to detect unloading, or we should not allow unloading at all

[jkotas]

We need @jkotas in the room next time

Happy to come to the next API review meeting - send me invite.

either the delegates have to detect unloading, or we should not allow unloading at all

This was answered in https://github.com/dotnet/corefx/issues/17135#issuecomment-353719848 by @qmfrederik

This API is proposed System.Runtime.InteropServices namespace. Every other method in this namespace turns into security vulnerability when used incorrectly. If you are using the Dispose method, you have to know that it is safe to do.

[morganbr]

Dispose is not normally an unsafe operation, rather tooling strongly pushes developers toward using it. There's a difference between allowing something unsafe and encouraging it. Do we really have to support unloading at all? If this is just a way to better parameterize P/Invokes, I don't see a need to support it and we can avoid the problem.

[jkotas]

Dispose is often unsafe operation for interop types: System.Runtime.InteropServices.CriticalHandle.Dispose, System.Runtime.InteropServices.GCHandle.Free or recently introduced System.Buffers.MemoryHandle.Dispose are about as unsafe as this one - you have to make sure that the stuff they are tracking is not in use.

Do we really have to support unloading at all?

I think it should be supported for completeness. There are certainly situations where it is required.

[jkotas]

We can consider calling the method Free to make it clear that it is not an ordinary safe Dispose.

[jkotas]

And/or we can rename the type to NativeLibraryHandle to make it clear that it is just a handle to the unmanaged library.

[morganbr]

Would it make sense to make it derive from SafeHandle and to have the delegate go through an extra indirection that makes it keep the SafeHandle alive? That would make it safe as long as you only use the delegate. Changing GetSymbolAddress to UnsafeGetSymbolAddress (or DangerousGetSymbolAddress) would likely match our other standards if the delegate was safe.

[jkotas]

Would it make sense to make it derive from SafeHandle

I have no problem with that.

to have the delegate go through an extra indirection that makes it keep the SafeHandle alive?

What kind of indirection you have in mind? There are number of different options, none of them is complete solution. The best we can do is to ref-count on the way in and on the way-out, that would kill the performance of this - nobody would want to use this wrapper. And even that does not completely guarantee safety because of it ignores the library state. There can be code running in the library code even when all delegates are gone. You have to know what the library is and where it is safe to unload it.

If the GetDelegate<T> convenience helper is the most contentious point of this proposal, we can drop it from the proposal and just let people call Marshal.GetDelegateForFunctionPointer or use unmanaged calli themselves to make it clear there is no magic.

Our current naming standards is to not add Unsafe or Dangerous for APIs to APIs in System.Runtime.InteropServices namespace. It is implied that APIs in this namespace are unsafe.

[morganbr]

What kind of indirection you have in mind?

I don't have the entire picture due to signature matching, but assuming we have some way to generate the arbitrary signatures this API needs anyway, something like pointing the delegate to the NativeMethod.Invoke method below.

class NativeMethod
{
    NativeLibraryHandle m_libraryHandle;
    IntPtr m_nativeMethodPointer;

    SignatureMagic Invoke(MoreSignatureMagic)
    {
        InvokeNativeMethod(m_nativeMethodPointer); // Probably a stdcall intrinsic or something
        GC.KeepAlive(m_libraryHandle);
    }
}

Interop code could probably do something roughly equivalent in the VM if that's more convenient on CoreCLR.

I agree it could still be a problem if a call kicks off a background thread and returns and then the handle gets disposed/finalized.

I'm also ok with Free (or perhaps Unload) idea since libraries shouldn't necessary be freed in the same way other native resources are.

[qmfrederik]

assuming we have some way to generate the arbitrary signatures this API needs anyway, something like pointing the delegate to the NativeMethod.Invoke method below.

Most projects I know of use clang to generate C# code based on C headers.
Examples of projects doing this for .NET Core are https://github.com/Ruslan-B/FFmpeg.AutoGen and https://github.com/libimobiledevice-win32/imobiledevice-net; and I believe the Xamarin team does this as well.

Changing the class to inherit from SafeHandle makes sense, so we end up with something like this:

namespace System.Runtime.InteropServices
{
    // Exposes functionality for loading native shared libraries and function pointers.
    public sealed class NativeLibrary : SafeHandle
    {
        // Returns the default search paths
        public static IEnumerable<string> SearchDirectories { get; }

        // Constructs a new NativeLibrary using user-provided search libraries, using the search
        // paths defined by the value of paths and optionally returning null instead of throwing
        // an error.
        public static NativeLibrary Open(string name, DllImportSearchPath paths,
            bool throwOnError = true);

        // The operating system handle of the loaded library. (Was a property previously)
        // (Inherited from SafeHandle)
        public IntPtr DangerousGetHandle()

        // Lookup a symbol (including functions) from the specified dynamic library and returns
        // the address of that symbol. Returns IntPtr.Zero if the symbol cannot be found and
        // throwOnError is false; throws an exception otherwise.
        public IntPtr GetSymbolAddress(string name, bool throwOnError = false);

        // Gets a function whose signature matches the given delegate type's signature.
        // This is equivalent to calling LoadFunction + Marshal.GetDelegateForFunctionPointer
        // Returns null if the function cannot be found and throwOnError is false; throws an
        // exception otherwise.
        public T GetDelegate<T>(string name, bool throwOnError = false);

        // Frees the native library. Function pointers and delegates retrieved from this library will be void.
        // Using function pointers and delegates returned by this library after calling dispose will
        // result in undefined behavior.
        // (Inherited from SafeHandle)
        public void Dispose();
    }
}

[terrajobst]

Video

• IDisposable. We're concerned that by making this type IDisposable we lead people down the path of disposing these instances over-aggressively (due to static code analysis rules), resulting in undefined behavior. However, we expect most people never having to unload the library. Thus, we believe it's more consistent with other concepts, such as AssemblyLoadContext, by calling it Unload.

• throwOnError. Since failing seems to be the default here, we've changed the APIs to follow the Try-pattern.

• SearchDirectories. We've removed SearchDirectories as there seems to be no scenario where someone would need to probe that; most probing should be handled by calling TryOpen.

public sealed class NativeLibrary
{
    public static bool TryOpen(string name, DllImportSearchPath paths, out NativeLibrary result);

    public IntPtr Handle { get; }
    public bool TryGetSymbolAddress(string name, out IntPtr result);
    public bool TryGetDelegate<T>(string name, out T result);
    public void Unload();
}

[qmfrederik]

This issue is both marked as up-for-grabs and assigned to @GrabYourPitchforks .

@GrabYourPitchforks Do you plan to work on this? If not, I can probably at least provide an initial PR with implements NativeLibrary based on the work done in dotnet/corefx#25134.

[GrabYourPitchforks]

@qmfrederik Yes, I plan on tackling this in the near future. Will remove the up-for-grabs label.

[GrabYourPitchforks]

I found some usability issues while working on this. Bringing it back through API review. New proposed API surface:

public sealed class NativeLibrary
{
    // Changed: removed Try* pattern since failure values (null, IntPtr.Zero) are easy to determine

    // Changed: renamed Open -> Load so that we can add Unload method in future
    // Changed: added Assembly-based overload so that we can mimic [DllImport] logic
    public static NativeLibrary Load(string name, DllImportSearchPaths paths);
    public static NativeLibrary Load(string name, Assembly caller, DllImportSearchPaths? paths = null);

    // Changed: renamed Handle to RawHandle so that we can add a real handle type in future
    public IntPtr RawHandle { get; }

    // Changed: added 'exactName' parameter so that we can mimic [DllImport] logic
    public TDelegate GetDelegate<TDelegate>(string name, bool exactName = false) where TDelegate : class;
    public IntPtr GetSymbolAddress(string name);

    // Changed: removed Unload method since we can't yet reliably unload
}

[dotMorten]

I'd also love to make this architecture specific, so I can build an AnyCPU application, but it automatically picks the right DLL based on what it's running under. Ie. if it's an x86 process, it could be pinvoking into /x86/mylibrary.dll or /mylibrary32.dll, and if it's x64 into /x64/mylibrary.dll or /mylibrary64.dll

[daniel-white]

With the DllImportSearchPath being specified, how would one specify a path to a native library?

[dotMorten]

One more thing to consider: A NuGet package would xcopy deploy the native resources, but during design time in Visual Studio we might not know what the path is, as the xcopy doesn't happen until build. So we need a way to specify a path relative to something in the nuget package when the design time check returns true.

[terrajobst]

We just reviewed this and settled on the following API shape:

```C#
public sealed class NativeLibrary
{
public static bool TryLoad(string name, Assembly caller, DllImportSearchPaths paths, out NativeLibrary result);

public IntPtr Handle { get; }

public bool TryGetDelegate<TDelegate>(string name, out TDelegate result) where TDelegate : class;
public bool TryGetDelegate<TDelegate>(string name, bool exactSpelling, out TDelegate result) where TDelegate : class;
public bool TryGetSymbolAddress(string symbolName, out IntPtr result);

}
```

[terrajobst]

Video

@dotMorten

The name argument for TryLoad allows relative paths but behavior depends on the value of paths.

[qmfrederik]

@dotMorten Unless I'm wrong, specifying DllImportSearchPath.ApplicationDirectory would include the native assemblies from the NuGet package folders.

You can also do this manually, see GdiplusNative.Unix.cs for an example.

[mellinoe]

I haven't been following this, but I'm happy to see it's been approved.

My only thought: I think there should be a TryLoad overload without the Assembly parameter. I think the vast majority of calls will not be interested in it. It's unclear whether you can pass null for it, but I think there should just be an overload where it is absent.

[Joe4evr]

public bool TryGetDelegate<TDelegate>(string name, out TDelegate result) where TDelegate : class;

Wouldn't it be a better idea to piggyback off of the upcoming delegate constraints?

[luqunl]

Why do we need to add a new class? Could we just reuse Marshal class?
The reason is that people may already familiar with Interop stuff and all of these API stuff lives under Marshal class.

[qmfrederik]

@GrabYourPitchforks I missed this being moved to Future since there are no notifications for label changes.

Can I assume you don't plan to work on this in the near future? If so, could you add the up-for-grabs tag?

I'd very much like to see this in 2.1 as it impacts a couple of my scenarios and I'm willing to spend time on this.

[GrabYourPitchforks]

Hi @qmfrederik, we're still set on getting to this in a future release, and in fact we're even code-complete on the feature (see https://github.com/dotnet/coreclr/pull/16409). We found some usability issues while testing, and these issues will require API tweaks. We weren't able to get the necessary changes in under the scheduling deadline.

But the implementation in the referenced PR _is_ otherwise complete and tested, and it works in the scenarios we've tried. We plan on bringing it back to API review once the feature window opens back up. So there's not much that's up for grabs, to be honest.

[qmfrederik]

@GrabYourPitchforks This is the first time I've heard about those usability issues. The API is/was supposed to be a very thin wrapper around dlopen/LoadLibrary. Can you elaborate a bit more on the usability issues you've found?

[tannergooding]

My biggest complaint about this API is that it will require a lot more effort to implement than DllImport.

The API certainly provides benefits and opens the possibilities for more complicated scenarios. However, it also requires a lot more explicit calls, declaring delegates with UnmanagedFunctionPointer attributes, and manually tracking the delegate/IntPtr combo and the eventual cleanup of the NativeLibrary (vs just declaring the function with a DllImport attribute and calling it).

In many cases, all a user needs is the ability to say: look for this library name on Platform X and this platform name on Platform Y. DllImport already works today when the difference between the platforms is name and libname, but there are some libraries that don't follow this pattern between languages and extending this to be able to explicitly specify alternative search names would also be greatly beneficial for a number of cases (e.x. Vulkan-1 and libVulkan).

[qmfrederik]

@tannergooding Absolutely, System.Drawing.Common is a good example of a library where the native wrapper went from [DllImport] to something significantly more complicated.

What you're describing sounds very much like Mono's dllmap (or a variant thereof), which would also work for most cases.

Having said that,

1. This is a real pain for lots of libraries
2. We're now stuck with having to resort to dlopen. But dlopen _also_ suffers from the same problem (dlopen can be located in libdl or libc, with or without version suffix,...). At least this API would give us consistent access to dlopen
3. There has been a lot of back and forth on this, so I was glad to finally see an approved API that would make it 2.1. Now it looks we'll be stuck for some time to come.

[Nihlus]

I've recently released a library which solves most (if not all) of the aforementioned issues, and bundles a cross-platform delegate-based approach into a simple and easy-to-use APIs, which also supports Mono DllMaps, as well as custom library search implementations. The backend is superficially similar to the proposed API, but is abstracted away for ease of use.

On top of that, it allows some more interesting extensions to the P/Invoke system, such as direct marshalling of T? and ref T?.

https://github.com/Firwood-Software/AdvanceDLSupport

[qmfrederik]

@Nihlus Looks interesting, thanks! Would be awesome if you could license it under something more permissive than GPLv3 and publish it on NuGet ;-).

[Nihlus]

@qmfrederik It's on nuget already - there's custom licensing options available if GPLv3 doesn't fit your project. Drop me an email and I'm sure we can iron something out :)

[dotMorten]

People still use GPL these days? Especially for a library that seems exceptionally silly, which is what LGPL was meant for. No one could ever use this, without releasing all their source code as GPL as well. So you better not be using any other 3rd party library that aren't GPL too.

[Nihlus]

@dotMorten The reasoning behind choosing GPL over LGPL has been better said in other channels: http://www.gnu.org/philosophy/why-not-lgpl.html

And as for being able to use it, you're incorrect - as I've previously stated, if GPLv3 doesn't fit your project or product, there are custom licensing options available to you. I have rent to pay and a mouth to feed, man.

[xoofx]

Hey, I'm super interested also in something similar, as I have being writing xplat LoadLibrary/dlopen countlessly and this is very annoying that we don't get this through the underlying .NET framework. Just wondering, I can see that this proposal is going beyond what we are usually using (dlopen...etc.) and what if we could at least get the few core methods to System.Runtime.InteropServices.Marshal:

c# IntPtr LoadNativeLibrary(string libraryPath); IntPtr GetNativeSymbolAddress(IntPtr nativeLibraryHandle, string symbolName); void CloseNativeLibrary(IntPtr nativeLibraryHandle);

They would be a one-to-one mapping to a native function, so it is very easy to get them working on a bunch of platforms (and at least all the platforms supported by CoreCLR). Then if someone wants to build a higher level like NativeLibrary above, I would not mind, but having core methods is really what we need first.
Thoughts?

[luqunl]

@xoofx , Your idea is similar to our potential plan for x-platform Pinvoke support

1. Expose few methods in Marshal as builder block
2. [Optional] Enable others higher level support in libraries: such as DllMap etc..

[xoofx]

@xoofx , Your idea is similar to what We are going to do next for x-platform Pinvoke support

Good to know! 👍

The only thing I'm uncertain of the behavior is, how far IntPtr LoadNativeLibrary(string libraryPath); handles prefix and postfix (extension) automatically? I would expect it to handle this automatically if you don't pass a valid extension for shared library for the current platform, something like this:

• If you pass myShared.dll on Windows, it would use myShared.dll directly
• If you pass myShared on Windows, it would try to load myShared.dll
• If you pass myShared on Linux, it would try to load libmyShared.so
• If you pass libmyShared on Linux, it would try to load libmyShared.so
• If you pass myShared.dll on Linux, it would try to load libmyShared.dll.so
• ...etc.

The logic is platform dependent (specially with the .so, .so.0 variations...etc.), but it allows to keep the same C# managed assembly with a native dependency (haven't checked CoreCLR implementation of DllImport for this actually...) which is quite important.

[luqunl]

Current CoreCLR implementation of DllImport will append .dll/.so/.dylib if the file name itself doesn't contain '.'. see https://github.com/dotnet/coreclr/issues/17150

[luqunl]

@qmfrederik, Sorry for confusion. My original thinking is that If We can't make a good API for this scenario, how about expose some basic builder blocks as API -- at least it can unblock others.

[Ruslan-B]

Folks, I would like to bring to attention that in real life library naming could be a little bit more complicated. As for example ffmpeg avcodec library is named avcodec-57.dll on Windows, libavcodec.57.so on Linux and libavcodec.57.dylib on macOS. So dash is used on Windows and dot is used other platforms. Right now we have to handle all discrepancies manually per platform so DllImport in this case quite useless. For the reference Ruslan-B/FFmpeg.AutoGen/issues/71

[qmfrederik]

@luqunl I'm all for exposing the basic building blocks as an API. Anything which is a tiny wrapper around dlopen/LoadLibrary will work for me, and I guess for @xoofx and @Ruslan-B as well. In particular, it will also help fix a couple of issues related to System.Drawing.Common on platforms such as CentOS.

Do you have an idea of what the API could look like (either an API speclet or sample code using the API)? That may help up some of the confusion I have :).

[xoofx]

Folks, I would like to bring to attention that in real life library naming could be a little bit more complicated. As for example ffmpeg avcodec library is named avcodec-57.dll on Windows, libavcodec.57.so on Linux and libavcodec.57.dylib

First time I'm seeing this. Do you know why they have been using - instead of . on Windows? If this is something that common, that could be supported, otherwise it is not really realistic to support any naming inconsistencies.

[xoofx]

@qmfrederik, Sorry for confusion. My original thinking is that If We can't make a good API for this scenario, how about expose some basic builder blocks as API -- at least it can unblock others.

I concur. This basic block functionality is what will unlock the more complex scenarios. And if it can help also to get this more easily into the PR process for corefx, that's a double win. 😉

[mellinoe]

First time I'm seeing this. Do you know why they have been using - instead of . on Windows? If this is something that common, that could be supported, otherwise it is not really realistic to support any naming inconsistencies.

It's quite common. Vulkan binaries follow this pattern and have other inconsistencies:

https://github.com/mellinoe/vk/blob/master/src/vk/Commands.cs#L23

The other common inconsistency is many libraries append their name with a ".version" suffix. Some platforms/distributions omit the suffix in some cases, others don't. Some have different suffixes for the same library. Vulkan has a little bit of this (see link), as does SDL2:

https://github.com/mellinoe/veldrid/blob/master/src/Veldrid.SDL2/Sdl2.cs#L10

Exposing Marshal.LoadLibrary and Marshal.GetProcAddress would be fine for my scenarios. It would at least help me resolve this issue: https://github.com/mellinoe/nativelibraryloader/issues/2.

[xoofx]

It's quite common. Vulkan binaries follow this pattern and have other inconsistencies:

Thanks, so that's a good example: I'm fine with inconsistencies, as long as it is handled by the caller, and not by the underlying Marshal.LoadLibrary. The vulkan bindings shows that this can't be covered in a generic way. The LoadLibrary should at least cover the common pattern. Also I would not mind having a helper method that returns a list of possible shared lib extensions for the running platform - like string[] Marshal.GetCommonNativeExtensions() if this can help some more complex/dynamic paths scenarios while still being xplat.

[xoofx]

One thing about the versions used in dylib names (so.1 ...etc.)... I would expect that a 1 or 2 would provide a different API (likely only an extension but not breaking the previous API?)... but can we expect that the LoadLibrary could choose whatever is available.? I don't know well the different Linux discrepancies, but how this is working? Is there cases where you can have a so.1 without a plain so?

[ghost]

@xoofx, yes it is possible https://github.com/dotnet/corefx/blob/1c974c975d7c354c8dadd96b449847b6ad01180f/src/Native/Unix/System.Security.Cryptography.Native/opensslshim.cpp#L18 and https://github.com/dotnet/coreclr/blob/e5a17bac5f0fd0b0137be7c8cf23391adc59958f/src/pal/src/loader/module.cpp#L1636-L1655 etc. Same lib, different OS, same or different kernel having different numeric suffix.
Usually there is a non-numeric-suffixed .so symlink created by package installers, but not always.
I like your suggestion of wrapping up numeric suffix logic into an API.

[ghost]

(even libc doesn't have consistent extension across the unices)

[xoofx]

Fair enough, all good cases that confirm that this can't be realistic to support all these discrepancies. This can be only part of higher level usage of Marshal.LoadLibrary and this will work as well in this case (you could have also different loading path...etc.)

[ghost]

In fact, that numeric format is actually libFOO.so.MAJOR.MINOR (from https://unix.stackexchange.com/a/481 &
https://www.ibm.com/developerworks/linux/library/l-shlibs/index.html (section Compatibility's not just for relationships))

[qmfrederik]

The LoadLibrary should at least cover the common pattern.

Please, no. There is no common pattern across distro's or even within a single distro. There will be at least one scenario in which the "common pattern" will have unexpected side-effects and you want to access the underlying API.

I think most of us are fine with writing our own probing mechanism (be it for FFmpeg, Vulkan or libgdiplus) as long as we have a way to invoke dlopen and GetFunctionPtr.

Can we expect that the LoadLibrary could choose whatever is available.? I don't know well the different Linux discrepancies, but how this is working? Is there cases where you can have a so.1 without a plain so?

Yes, in fact that's the common scenario. For example, on Ubuntu libdl is part of libc6 and named libdl.so.2. libld.so exists but only as part of libc6-dev, so it will be installed on developer machines only.

For Debian/Ubuntu, section 8. Shared Libraries in the Debian Policy gives a detailed overview of how .so files are named in that family of Linux distro's.

It states

It is usually of the form name.so.major-version (for example, libz.so.1). The version part is the part which comes after .so., so in that example it is 1. The soname may instead be of the form name-major-version.so, such as libdb-5.1.so, in which case the name would be libdb and the version would be 5.1.

Long story short, don't generalize & let the caller decide :)

[tmds]

+1 @qmfrederik The numbers version the API, the caller must understand and deal with the semantics.

[xoofx]

Please, no. There is no common pattern across distro's or even within a single distro. There will be at least one scenario in which the "common pattern" will have unexpected side-effects and you want to access the underlying API.
Long story short, don't generalize & let the caller decide :)

I would tend to agree with that... but my main grip is that for the common case where you develop and deploy your own native library (and you control the way the naming is performed), we should be able to write xplat code without having to make a if (platform_blabla) { use_stupid_extension_naming} else {...}. I still think that Marshal.LoadLibrary should provide a minimalist default behavior that facilitate this (but we can disagree on this, that's fine too 😉 )

[qmfrederik]

For the common case where you develop and deploy your own native library (and you control the way the naming is performed), we should be able to write xplat code without having to make a if (platform_blabla) { use_stupid_extension_naming} else {...}

I agree, and the good news is you can already do so today - actually .NET Core has a pretty good story there.

If you control the naming of your native library, you can something like [DllImport("mylibrary")]| and that will automagically resolve to libmylibrary.so, libmylibrary.dylib and mylibrary.dll.

Since you control the naming, you also control the distribution, in which case you can ship your native libraries in a NuGet package and .NET Core will "do the right thing".

That's how I ship imobiledevice-net, for example; and ASP.NET ships libuv IIRC.

[xoofx]

If you control the naming of your native library, you can something like [DllImport("mylibrary")]| and that will automagically resolve to libmylibrary.so, libmylibrary.dylib and mylibrary.dll.

Yep, and that was my proposal just above (to make LoadLibrary acting similar to DllImport behavior) So you agree a bit then? 😅

[qmfrederik]

So you agree a bit then? 😅

Yes!

[chmorgan]

It sounds like there is almost consensus here, what are the next steps? I'm itching to use whatever the solution is to fix cross platform support with dotnet that previously worked with .net and mono.

[GrabYourPitchforks]

@chmorgan the next steps are for us to finish the existing PR for this feature when the checkin window opens back up.

[dotMorten]

What about the fact that on iOS callbacks must be static? That's where all my interop code diverges the most. For us to be able to create truly reusable interop code, these sort of differences needs to be addressed too.
Second when running on .NET Framework as AnyCPU we need to be able to point to two different DLLs (or 3 if we get ARM64 support) based on which architecture the app is running under.

[ghost]

https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks#architecture-specific-folders

we need to restore packages on each system

[dotMorten]

@kasper3 Good point. I really like the \runtime\[TFM]-[arch]\native\ approach that UWP uses. The other platforms need to support this as well - currently we have to create .targets files and either embed or xcopy deploy the dynamic libraries.

[Ruslan-B]

@kasper3, @dotMorten no offence, but I guess warping the native to OS packaging systems is a bit of topic, as in case you are the owner of the native library - you can maintain a library naming in a dotnet friendly way. So you need this functionality only in case you are not the owner of the original library(s) which is exposing C-API. To summarize this is needed to cover discrepancies between different OS and habits for naming these libraries.

[dotMorten]

@Ruslan-B I fail to understand the relevance to whether you own the original library or not.

The way you deploy your native libs via NuGet is quite different on each platform.
On Android you have to embed you .so files as <EmbeddedNativeLibrary/> or <AndroidNativeLibrary/> depending on whether its embedded in a library or an app (why they didn't make it one build action is beyond me). On UWP and WPF you have to xcopy deploy the DLL, same on iOS but there you also have to define several MtouchExtraArgs.

Having the \runtimes\ folder part be part of the nuget story across all platforms would simplify and unify how you get the native libs deployed in your app. While I agree that part of the discussion is probably more appropriate in the NuGet repo, it's completely valid in the context of unifying how we use native code on the various platforms.

[Ruslan-B]

@dotMorten in this case I would suggest to reread the original proposal.

[dotMorten]

@Ruslan-B Read the entire thing, and you'll find the discussion has expanded quite beyond that.

[Ruslan-B]

@dotMorten true, however, I think the only reason we need this, is to handle discrepancies between different OS with stable habits of naming things.

[qmfrederik]

This issue is an API proposal issue. It concerns an API for loading native libraries and using the functions they expose. It was created because we need to load (lib)gdiplus for System.Drawing across platforms.

How you package these native libraries with your app (and whether you should package them at all or use a different acquisition mechanism) is related but not strictly in scope for the API proposal.

I don't know about iOS and Android, but the .NET Core SDK supports the runtimes folder structure and that works very well for the projects and I'm involved in - YMMV.

[qmfrederik]

@karelz @joshfree This API didn't make it for 2.1, any way we can track it for 2.2.0 (it's currently marked as Future)?

[karelz]

Currently there is not much difference between Future and 2.2. I moved it to 2.2, but it depends if we get to agreement and the result passes code reviews (there has been quite a few people with strong opinions).

[xoofx]

Currently there is not much difference between Future and 2.2. I moved it to 2.2, but it depends if we get to agreement and the result passes code reviews (there has been quite a few people with strong opinions).

@karelz How should we proceed from there? Based on my previous pseudo-proposal should I open a new issue?

[karelz]

I will defer to @GrabYourPitchforks who worked on it and has PR ready (https://github.com/dotnet/corefx/issues/17135#issuecomment-373878407), he should be able to suggest best next steps (once he is back from vacation - one more week).

[xoofx]

What about the fact that on iOS callbacks must be static? That's where all my interop code diverges the most. For us to be able to create truly reusable interop code, these sort of differences needs to be addressed too.

@dotMorten btw, what do you mean by static? Static linking required by iOS with mono? Isn't dlopen being supported starting from iOS8+? (or you need to support older iOS?)

[joshfree]

I will defer to @GrabYourPitchforks who worked on it

Actually we should defer this to @jeffschwMSFT and the Interop team, who are interested in driving this feature area forward in .NET Core. @jeffschwMSFT could you share your team's current thoughts on direction / shape?

[jeffschwMSFT]

@qmfrederik we are taking a step back and considering the broader scenario (exploring a dllmap like mechanism). Right now we don't have a concrete design, but once we do we will share and update this thread.

[qmfrederik]

So, based on all the feedback, perhaps we should split the issue in two.

The core gist of this is that we want a way to call dlopen/LoadLibrary and dlsym/GetProcAddress; most of us can take it from there. The API @xoofx proposed as well as the original proposal for this issue seem fine.

Making cross-platform library loading (i.e. dllmap & friends) would be orthogonal to that, no?

[ghost]

Currently for 2.2, there is one inconsistency left in DllImport tracked by https://github.com/dotnet/coreclr/issues/17604.
This approved API is definitely not related to DllImport enhancement with DllMap like mechanism (which is actually tracked by @akoeplinger's https://github.com/dotnet/coreclr/issues/930). So we would love to see @GrabYourPitchforks's PR https://github.com/dotnet/coreclr/pull/16409 getting resurrected.

[xoofx]

Making cross-platform library loading (i.e. dllmap & friends) would be orthogonal to that, no?

Yeah, I agree, maybe not completely orthogonal, but it is more an implementation detail or an enhancement to the resolution. It should not change the API shape, nor the default behavior if a dllmap file is not present (which is good). Dllmap is a super nice feature/addition and it should be considered, but maybe we should split first an implementation without, which was almost done it seems, and then proceed on the full dllmap story (which will take likely several months of study+shape+implem+tests...etc.)

[swaroop-sridhar]

Closing this issue, in favor of dotnet/corefx#32015