Core: .net standard library with dylib - DllNotFoundException

[stoneman]

I previously posted this on Stack Overflow but that's had no responses and I think there is a fair chance it's a problem with .Net Core or the .Net Core documentation.

I'm using .Net Core 1.0.

I have a dylib (libextern.dylib) that I've created which contains the following:

int extern_test()
{
    return 123;
}

I have a library which targets netstandard1.5 and net451 and which contains:

[DllImport("libextern.dylib", EntryPoint="extern_test")]
public static extern int ExternTest();

Finally, I have a .net core CLI app which calls the library's ExternTest(). It has a project reference to the library and allowUnsafe in it's buildOptions:

"dependencies" : {
  "MyLibrary": {
    "version": "0.0.1",
    "target": "project"
  }
},
"buildOptions": {
    "allowUnsafe": true
}  

If I place libextern.dylib into the root of the CLI project then ExternTest() returns 123 as expected.

Rather than putting it into my CLI project, I want to include libextern.dylib within my library. I have followed the How to Use Native Dependencies instructions and the KestrelHttpServer/Libuv example it links to (as well as various other guides which I believe have become out of date) but I get a DllNotFoundException when calling ExternTest().

The library's project.json contains:

"buildOptions": {
  "allowUnsafe": true
},
"packOptions": {
  "files": {
    "mappings": {
      "runtimes/osx/native/": "libextern.dylib"
    }
  }
}

The How to Use Native Dependencies instructions omit "mappings" but I have tried both ways and found it was required for libextern.dylib to be included in the nuget package.

I have also tried packaging libextern.dylib into another package (Option 2 in the How to Use Native Dependencies instructions) but this did not work for me either.

What might I be doing wrong here? Might this actually be an issue with .net core?

[yizhang82]

@schellap Could this be a problem with the host?

[schellap]

@jonstoneman, would you be able to share the entry for this "MyLibrary" in your deps.json file in your app base?

I'm suspecting an instance of https://github.com/dotnet/core-setup/issues/194

[schellap]

Sorry, I would need the "type" in the "libraries" section for this library.

[brthor]

@jonstoneman try changing packOptions to publishOptions

Like this:

"publishOptions": {
  "files": {
    "mappings": {
      "/": "libextern.dylib"
    }
  }

Not entirely sure this will work but packOptions seems wrong.
After doing this do a dotnet publish and take a look at the output directory to see if the .dylib file was binplaced correctly.

@cartermp The documentation @jonstoneman is referencing appears incorrect for project.jsons attempting to reference a native dll.

[brthor]

@cartermp Specifically

For the first option, you'll need to include the following in your project.json file:+  
1.Setting allowUnsafe to true in a buildOptions section.
2.Specifying the path to the native .dll(s) with a Runtime Identifier (RID) under files in the packOptions section.

Should packOptions here be perhaps buildOptions or publishOptions

@jonstoneman There is some additional info for these option types in the original announcement here:
https://github.com/aspnet/Announcements/issues/175

[cartermp]

@ajaybhargavb Which would be the correct place to list a native dependency? What I had documented is now broken with the project.json changes.

[ajaybhargavb]

If the dll need to be copied to the output directory, then it needs to be included in copyToOutput in buildOptions.

"buildOptions":{
        "copyToOutput":{
            "runtimes/win7-x64/native/":"native-lib.dll"
        }
    }

[cartermp]

@ajaybhargavb I see. Looking at the Libuv packOptions is being used, but this is specifically for packaging it as a NuGet package. Is that still valid?

Is there a doc or matrix of valid configurations for placing files under sections? Specifically:

• Put your native .dll dependency under buildOptions in these ways to accomplish X, Y, and Z
• Put your native .dll dependency under publishOptions in these ways to accomplish X, Y, and Z
• Put your native .dll dependency under packOptions in these ways to accomplish X, Y, and Z

This is what I can gather based on the information I've found:

• ??? To build with a native dependency, associate the path with the RID(s) you want under mappings under files under packOptions. This will not copy to the output directory.
• To package a native dependency with your NuGet package, associate the path with the RID(s) you want under mappings under files under packOptions
• To publish the native dependency with your application, associate the path with the RID(s) you want under mappings under files under packOptions

In addition to what you just said above:

• To build with the native dependency and copy the .dll to the output directory, associate the path with the RID you want under copyToOutput under buildOptions

[stoneman]

@schellap: MyApp.deps.json contains:

{
  "runtimeTarget": {
    "name": ".NETCoreApp,Version=v1.0",
    "signature": "da39a3ee5e6b4b0d3255bfef95601890afd80709"
  },
  "compilationOptions": {},
  "targets": {
    ".NETCoreApp,Version=v1.0": {
      "MyApp/1.0.0": {
        "dependencies": {
          "MyLibrary": "1.0.0"
        },
        "runtime": {
          "MyApp.dll": {}
        }
      },
      "MyLibrary/1.0.0": {
        "runtime": {
          "MyLibrary.dll": {}
        }
      }
    }
  },
  "libraries": {
    "MyApp/1.0.0": {
      "type": "project",
      "serviceable": false,
      "sha512": ""
    },
    "MyLibrary/1.0.0": {
      "type": "project",
      "serviceable": false,
      "sha512": ""
    }
  }
}

[stoneman]

@brthor: I can't use dotnet publish with a library

[stoneman]

@ajaybhargavb: copyToOutput resulted in the dylib being copied into the output directory of the library project but I still get the DllNotFoundException when calling the test method from the app.

[stoneman]

I've put example code here: https://github.com/jonstoneman/nativelibrepro

The master branch has the dylib in the MyApp project and works.

The feature/packaged_lib branch demonstrates the problem.

[brthor]

@jonstoneman I didn't realize this was a library project.

I think you are hitting this issue: https://github.com/dotnet/cli/issues/2902

To make this work you'll need to do a dotnet pack on that library (also going back to packOptions in your project.json). Then target the output nupkg with the consuming app rather than the project.

You can do this by using dotnet restore -f C:\Directory\Where\The\Nupkg\Sits where -f adds a source to the restore call, so the nupkg can be picked up from a local directory.

For packOptions you'll want the original string that you had:

"buildOptions": {
  "allowUnsafe": true
},
"packOptions": {
  "files": {
    "mappings": {
      "runtimes/osx/native/": "libextern.dylib"
    }
  }
}

Let me know if this works out.

[stoneman]

Thanks @brthor - that does work! I had previously tried that but using a NuGet.config file instead of dotnet restore -f. The NuGet.config seemed to be working (MyLibrary appeared in ~/.nuget/packages) so I guess I must have had something else different without realising.

Is there any way of getting this working with project references? I found this aspnet/dnx sample but I don't know if it worked. dotnet restore certainly complains about dependencies without version numbers.

Without project references working then presumably I'll need to do this each time I make a change to my library project:

• rm -rf ~/.nuget/packages/MyLibrary
• dotnet pack ../src/MyLibrary
• dotnet restore -f ../src/MyLibrary/bin/Debug

[brthor]

Unfortunately due to the linked bug above I don't believe this will currently work for P2P references.

an alternative to doing this, each time you update your library project:

rm -rf ~/.nuget/packages/MyLibrary
dotnet pack ../src/MyLibrary
dotnet restore -f ../src/MyLibrary/bin/Debug

is to version your library higher with each change, by putting this in your library project.json : "version": "1.0.0-*"

Then using dotnet pack --version-suffix 000X ../src/MyLibrary where X increases with each change.

For the cli we use the commit count of the repo to keep versions increasing.

[blackdwarf]

@jonstoneman is the workaround provided by @brthor working for you? If so, can we close this issue?

[stoneman]

@blackdwarf The workaround works but is no faster than what I was already doing and actually less convenient. I still need to call dotnet restore -f ../src/MyLibrary/bin/Debug but I also need to increment the --version-suffix. With my method I can just rerun the same commands (which I have all on one line so can just up arrow and enter).

In any case, using package references is still a pretty bad experience compared to project references. I'd suggest keeping this issue open until project references work but won't object if you feel that https://github.com/dotnet/cli/issues/2902 really covers it and want to close this one.

[blackdwarf]

@jonstoneman I will actually close this one since we have dotnet/cli#2902. It is preferable to keep the product issues in the actual product repos.

[SeanSnyders]

I'm also struggling getting to specify and use a native library with my .Net Core project.
In my traditional Net461 VS solution, the native lib is compiled and project-referenced as a dependency.

I've tried all recommendations here (except making a NuGet package of it) and nothing seems to work as one would expect.

I literally just want to specify the .dll file as a dependency and have the Core app find and load it appropriately. Even copying the specific native dll into all folders in my output path (incl the folder where the .exe is build to), I still get that the dll fails to be found.
Is there some new magic in .Net Core runtime to how it is finding assemblies? I've fiddled with the additionalProbingPaths setting in *.runtimeconfig.json as well to no joy.

Where is the _de facto_ reference doc on is stuff?

Is making a NuGet package of my native dll the only way it can work? Seems a bit overkill....

[schellap]

@SeanSnyders, Could you also mention the version of the SDK and the runtime (Microsoft.NETCore.App) you are using?

dotnet.exe appname.dll (shared FX app) and appname.exe (standalone app) find the DLLs referenced in the appname.deps.json files. Do you have your library in the appname.deps.json?

[SeanSnyders]

@schellap It is netcoreapp1.1 and I'm targeting runtime win10-x64.

I ended up making a NuGet package of my native lib and added a local folder as a NuGet source where this package is deployed.

I can see my dependency in appname.deps.json:

But I still get a DllNotFoundException:

[SeanSnyders]

On the topic of referenced native libraries, how is the c++ runtime referenced? Does the dotnet app run in a sandbox (implicating it should be specced as a dependency) or does an installed runtime redist get picked up? Or both?

[SeanSnyders]

@schellap I found the problem. It was in the end my own error as I was pulling in an incorrect dll into my NuGet package as a dependency of my top-level native dll. I monitored my running app with SysInternal's Process Monitor (procmon.exe) and found the loading was working for the one dll but not for the other.

I would still like to know what is the recommended procedure for specifying vc runtime dependencies in .Net Core projects. Can we make a SCD that includes the runtimes as well. What of other platforms like linux, osx, etc....?

[schellap]

@piotrpMSFT, @blackdwarf, @livarcocc, can you help @SeanSnyders understand how native libraries are referenced in the project file? (i.e., as project to project dependency instead of making nupkgs for them)

[czifro]

@blackdwarf I am experiencing this exception as well. I can only assume that I am doing something wrong. I outlined all details in this StackOverflow post: http://stackoverflow.com/questions/41432671/unable-to-load-dll-dotnet-core . Any help would greatly be appreciated.

[blackdwarf]

@cartermp can you take a look at @czifro question since it is F#?

[cartermp]

No idea what the status of pinvoke on .NET Core is, let alone for F#. Tagging @KevinRansom

[yizhang82]

@cartermp p/invoke works in .NET Core. We've been using it in CoreFX implementations without issues. The problem folks are having here are packaging / project related.
@czifro Can you post another GitHub issue and provide more detail? Are you loading dylib that you package yourself or supplied by the OS?

[czifro]

@yizhang82 post another issue on this repo? or a different one? Just want to make sure I post it in the right location.

[yizhang82]

@czifro You can start with the coreclr repo first. Thanks.