Iot: Include the major version when referencing *.so libraries

[Tragetaschen]

Describe the bug

Linux distributions typically split libraries like libgpiod into multiple packages.

One specific split is between libgpiod and libgpiod-dev, where the former contains the necessary binaries and symlinks to run programs while the latter is meant for development, where you (traditionally) invoke a linker to produce a binary. The latter typically uses the unversioned symlink to simplify the build and the binary referencing the library then includes the versioned reference. In this split, the full version (libgpiod.so.2.1.3) and a symlink with the major version (libgpiod.so.2) are in the main package, the unversioned symlink (libgpiod.so) is in the -dev package, which doesn't get installed per default.

For example for Ubuntu:

• https://packages.ubuntu.com/bionic/amd64/libgpiod1/filelist
• https://packages.ubuntu.com/bionic/amd64/libgpiod-dev/filelist

Embedded distributions like Yocto per default do not install the -dev package into the image when you declare a runtime dependency on a library. Including the -dev package in the image can have unwanted side-effects of pulling in other development dependencies (for example I ended up with a Python interpreter in my image for a different -dev package in the past)

My own Yocto image doesn't include any -dev packages, so in order to use the LibGpiodDriver in my code, I have to "translate" the DllImport string by

static MyType()
    => NativeLibrary.SetDllImportResolver(typeof(GpioDriver).Assembly, libgpiodFixup);

private static IntPtr libgpiodFixup(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
{
    if (libraryName == "libgpiod")
        if (NativeLibrary.TryLoad("libgpiod.so.2", assembly, searchPath, out var result))
            return result;
    return IntPtr.Zero;
}

However, this "trick" only works exactly once per assembly (only one resolver per assembly), and the NativeLibrary API is meant to be used within the library requiring a specialized resolver. If System.Device.Gpio would decide to start using that functionality in the future, my own fixup would be broken.

Since the C# interop code typically relies on a specific major ABI version when declaring methods from an external library like libgpiod, that major version should be included in the DllImport instead of the unversioned .so.

Steps to reproduce

Run code using LibGpiodDriver on a Linux image without the libgpiod-dev package.

Expected behavior
Run without exception.

Actual behavior
DllNotFoundException

Versions used

Add following information:

• dotnet --info on the machine being used to build

.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.101/

Host (useful for support):
  Version: 3.1.1
  Commit:  a1388f194c

.NET Core SDKs installed:
  2.2.402 [/usr/share/dotnet/sdk]
  3.0.102 [/usr/share/dotnet/sdk]
  3.1.101 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

• dotnet --info on the machine where app is being run

  It was not possible to find any installed .NET Core SDKs
  Did you mean to run .NET Core SDK commands? Install a .NET Core SDK from:
      https://aka.ms/dotnet-download

Host (useful for support):
  Version: 3.1.3
  Commit:  4a9f85e9f8

.NET Core SDKs installed:
  No SDKs were found.

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

• Version of System.Device.Gpio package: 1.0.0

[joperezr]

Thanks for logging the issue. While you are absolutely right that we currently require the -dev package and that this could be fixed by simply changing our DLL imports and providing the version, it would also mean that our library would only work if that specific version is installed, once a new version of libgpiod comes out we would be broken as we wouldn't be able to find the library. We (.NET Core org) have done direct PInvokes with versions very few times, and in the times we have is when it is very unlikely for the native library to get versioned or if one specific version is widely available in most distros, but the problem with libgpiod is that this one is still on development so we will definitely get new versions updates.

The way that .NET usually deals with these is by building a native shim that we control so that we can do DLLImports to that instead and know we will always bind, and then just write simple code on that native shim which will resolve the right version of libgpiod and make the calls at runtime. This however does increase a lot of problems and complexity to our package, and it would also mean that it would be much harder to onboard new platforms as we would require a native shim per platform you want to run in. That is why for now we opted to depend on the -dev package.

[Tragetaschen]

Now that you mention it:

• Ubuntu 18.04 has libgpiod.so.1 (libgpiod 0.*)
• My Yocto image has libgpiod.so.2 (libgpiod 1.4.3)

Is the latter tested in any of the CI builds or am I just outright lucky?

[joperezr]

I'm almost sure that the one we test with is libgpiod.so.2, although I could be wrong. We would have to check the PCBs we have in our lab to see which version do they have to know for sure. Perhaps it would be interesting to add a test to just return the version of libgpiod that is being used for future reference.

[Tragetaschen]

A couple of random thought:

The proposal is to reference just the major version, not the minor or the patch, so any future backward-compatible update to the binary will be resolved successfully.

If the update to the library is not backwards compatible and the major version is increased, then it's also likely that the P/Invokes will be broken by that change. Using the unversioned .so from the -dev package won't help then and you'll end up with a missing entry point, or—worse—a corrupt program that hopefully AVs.

By depending on .NET Core directly (netcoreapp TFM instead of netstandard), the NativeLibrary API could be used to test-load one major version after the other and choose the right internal implementation without breaking or requiring a native shim.

[joperezr]

I see, yeah I think depending on the major version seems reasonable to me.