Runtime: Unable to load DLL 'System.Security.Cryptography.Native' on OS X

[aplex]

We're using .Net Core RC2 to build a desktop app with Web UI. It works fine on developer mac, but does not start on a clean OS X 10.11, producing an exception:

System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Interop.Crypto.GetMaxMdSize()
   at Interop.Crypto..cctor()
TypeInitializationException: The type initializer for 'Crypto' threw an exception.
Interop.Crypto.EvpSha1()
Internal.Cryptography.HashProviderDispenser.CreateHashProvider(String hashAlgorithmId)
System.Security.Cryptography.SHA1.Implementation..ctor()
Microsoft.AspNetCore.Razor.RazorTemplateEngine.ComputeChecksum(Stream inputStream)
Microsoft.AspNetCore.Razor.RazorTemplateEngine.GenerateCode(Stream inputStream, String className, String rootNamespace, String sourceFileName)
Microsoft.AspNetCore.Mvc.Razor.Internal.RazorCompilationService.Compile(RelativeFileInfo file)
Microsoft.AspNetCore.Mvc.Razor.Internal.CompilerCache.CreateCacheEntry(String normalizedPath, Func`2 compile)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Razor.Internal.CompilerCache.GetOrAdd(String relativePath, Func`2 compile)
Microsoft.AspNetCore.Mvc.Razor.Internal.DefaultRazorPageFactoryProvider.CreateFactory(String relativePath)
Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.CreateCacheResult(HashSet`1 expirationTokens, String relativePath, Boolean isMainPage)
Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.OnCacheMiss(ViewLocationExpanderContext expanderContext, ViewLocationCacheKey cacheKey)
Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.LocatePageFromViewLocations(ActionContext actionContext, String pageName, Boolean isMainPage)
Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
Microsoft.AspNetCore.Mvc.ViewEngines.CompositeViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ViewResultExecutor.FindView(ActionContext actionContext, ViewResult viewResult)
Microsoft.AspNetCore.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.FilterActionInvoker.<InvokeResultAsync>d__44.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.FilterActionInvoker.<InvokeResultFilterAsync>d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNetCore.Mvc.Internal.FilterActionInvoker.<InvokeAllResultFiltersAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.FilterActionInvoker.<InvokeResourceFilterAsync>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
Microsoft.AspNetCore.Mvc.Internal.FilterActionInvoker.<InvokeAsync>d__32.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.MvcRouteHandler.<InvokeActionAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.<Invoke>d__7.MoveNext()

Looks like this is caused by openssl dependency. Can it be bundled with the app somehow, so that users don't have to install openssl separately?

The app is built using

dotnet build --runtime osx.10.11-x64 --framework netcoreapp1.0
dotnet publish --runtime osx.10.11-x64 --framework netcoreapp1.0

[bartonjs]

@aplex You are correct that it is the OpenSSL dependency. While it does cause an less-than-ideal experience on a clean machine, we don't want to bundle OpenSSL with us because that would leave the machines in a place where they aren't getting security updates as OpenSSL releases new fixes (and/or we'd need a whole update stream for carrying our partner dependencies).

If you have any recommendations for where we can better inform users of the dependency (and their manual step of fulfilling it) I'd be quite happy to hear them.

[aplex]

For server environment, installing openssl may be ok (although I don't think there are a lot of people who would use OS X as a server). But, for desktop application that does not make sense. End-user wouldn't install or update some library that he does not know or care about. Also from what I read, openssl of different versions are not binary compatible, so the app would not be able to use the new version in some cases anyway.

Again, the problem is not in the installing of the dependency on dev machine, but that all thousands of end-user need to install it on their machines through some wired stuff like homebrew.

A separate issue would be publishing .Net Core apps to mac store that will not allow the app to escape the sandbox and install homebrew or openssl with admin permissions.

RC1 was working fine, and looks like everything was bundled in it (or at least it used libraries already present in the OS).

The expectation is that stand-alone application should be stand-alone, without dependencies on the latest version of OS with recent updates.

So I'm looking for some workaround for this in a short-term. I've already tried the following without any luck and the same exception:

1. copying libssl.1.0.0.dylib and libcrypto.1.0.0.dylib to app folder
2. in addition to 1, patching System.Security.Cryptography.Native.dylib to use the copied dylibs instead of the ones in /usr/local/opt/openssl/lib/

[bartonjs]

Very recent builds should already have System.Security.Cryptography.Native.dylib using \@rpath binding for libcrypto and libssl; which should allow for the copy-local-to-distribute scenario.

This was dotnet/runtime#17140, fixed after RC2 (present in -rc3 packages 24117 and later). Hopefully it eases some of the burden.

[aplex]

I can try that. What package should I reference from RC3? Is it on myget?

[bartonjs]

https://dotnet.myget.org/feed/dotnet-core/package/nuget/runtime.native.System.Security.Cryptography/4.0.0-rc3-24204-00 is the relevant package; but you'd get it by just changing all your rc2-based references to rc3-based references (24117+ for this particular problem's fix, 24204 is the newest build).

I don't know that it'll be happy letting you reference that package directly (since it's a runtime asset with no compile assets) so you might be better off referencing System.Security.Cryptography.Algorithms (https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Algorithms/4.2.0-rc3-24204-00) and letting it pull up all its dependencies.

[aplex]

Using RC3 package didn't went quite smooth, as there were a bunch of error during dotnet restore. I guess I need to upgrade all those "preview1" packages too, will try that tommorow.

So I tried just pulling System.Security.Cryptography.dylib from https://dotnet.myget.org/feed/dotnet-core/package/nuget/runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography/1.0.1-rc3-24204-00 and replacing the existing one in the built app with this one. Unfortunately I got the same exception.

otool shows that it is using rpath for libssl and libcrypto

otool -L System.Security.Cryptography.Native.dylib 
System.Security.Cryptography.Native.dylib:
    @rpath/System.Security.Cryptography.Native.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    @rpath/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

And I copied the whole contents of /usr/local/opt/openssl/lib to the same folder as my app binary. So the app, System.Security.Cryptography.Native.dylib, libcrypto.1.0.0.dylib and libssl.1.0.0.dylib are all in the same folder. But, i still get the same exception:

System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Interop.Crypto.GetMaxMdSize()
   at Interop.Crypto..cctor()
TypeInitializationException: The type initializer for 'Crypto' threw an exception.
Interop.Crypto.EvpSha1()
Internal.Cryptography.HashProviderDispenser.CreateHashProvider(String hashAlgorithmId)
System.Security.Cryptography.SHA1.Implementation..ctor()
Microsoft.AspNetCore.Razor.RazorTemplateEngine.ComputeChecksum(Stream inputStream)
...

[aplex]

Looks like I managed to hack around this. Turns out, it was not enough for System.Security.Cryptography.Native.dylib to have rpath dependencies.

libssl.1.0.0.dylib also had a hard-coded full path dependency to /usr/local/Cellar/openssl/.../libcrypto.1.0.0.dylib

I took libssl and libcrypto from openssl installed with Hombrew.

To get this working, i replaced this libcrypto dependency with rpath in libssl, and added an rpath to it, in a way similar to https://github.com/dotnet/corefx/blob/9823256decb60d6824121fb4578e3780698ccd4e/src/Native/System.Security.Cryptography.Native/CMakeLists.txt

[bartonjs]

@aplex Cool, I like it when things work the way they're supposed to :).

It seems to me that we're happy here, with the only actionable thing being to make sure we have documentation... but I wanted to be sure that you agreed with that state of affairs.

[aplex]

Yep, I thinks this can be closed, and some documentation on how to produce a stand-alone app on OS X would be useful. It's not really that hard to add that reference patching to a build script, but it was quite time consuming to understand what needs to be done.

Maybe in some future version of dotnet there could be an option to bundle openssl with the app, with all those dependency copying and patching done by dotnet tooling.

[bartonjs]

Release note contents:

OS X has an external dependency on OpenSSL

.NET Core uses OpenSSL as the provider for cryptographic primitives and the SSL/TLS protocol. While there is a pre-installed version of OpenSSL on OS X 10.11, that version is no longer supported, and is not used by .NET Core. In order to satisfy the OpenSSL dependency, libcrypto.1.0.0.dylib and libssl.1.0.0.dylib must be loadable via rpath probing. One such way of satisfying this requirement is via Homebrew:

brew install openssl

# without this next step Homebrew will not register a symlink in a standard rpath location,
# so .NET Core will still be unable to find the installed libraries.
brew link --force openssl

When this dependency is not met, an application making direct or indirect use of cryptography will get an exception similar to
System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native': The specified module could not be found..

A note on producing standalone OS X applications

Since .NET Core loads libcrypto and libssl via rpath probing these libraries can be copied into the working directory of an application before being copied to another machine. But when trying to use this configuration users should be advised that the Homebrew version of libssl has an absolute path dependency on libcrypto. The local copy of libssl may need to be modified to search for libcrypto via rpath with the install_name_tool utility.

[lynxluna]

I'm using macports, and because DYLD_LIBRARY_PATH sometimes does not work in El Capitan, I manage to do workaround by making libcrypto and libssl symlink to corefx directory.

sudo ln -s /opt/local/lib/libcrypto.1.0.0.dylib \
  /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/
sudo ln -s /opt/local/lib/libssl.1.0.0.dylib \
  /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/

And dotnet cli runs as expected. Here's my version

% dotnet --info                                                                                                                                                                                        
.NET Command Line Tools (1.0.0-preview2-003119)

Product Information:
 Version:            1.0.0-preview2-003119
 Commit SHA-1 hash:  0708fe095e

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

[bartonjs]

@lynxluna Does MacPorts not have a "make symlinks in /usr/local/lib" command (like Homebrew does)?

According to the docs I found (https://developer.apple.com/library/prerelease/content/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryUsageGuidelines.html#//apple_ref/doc/uid/TP40001928-SW12) the default fixed locations are $HOME/lib, /usr/local/lib, /usr/lib (after the current directory... and I feel like "the directory that the library which called (directly or indirectly) dyopen" is supposed to be in there somewhere)

[lynxluna]

Hi @bartonjs, Macports doesn't meddle with the /usr or /usr/local. They create a whole new and isolated rootfs consisting of their packages in /opt/local and add environment variables to PATH to prioritese /opt/local before /usr. So for example openssl is in here:

% which openssl
/opt/local/bin/openssl
% openssl version
OpenSSL 1.0.2h  3 May 2016

We can still access OSX installed tools by using full path.

% /usr/bin/openssl version
OpenSSL 0.9.8zh 14 Jan 2016

Macports also sets all packages installed via macports to have --prefix onto /opt/local
If we use pkg-config then it will give us flags onto Macports' installed libs. I thought homebrew also do the same thing except setting --prefixes to /usr/local

% pkg-config --cflags --libs openssl
-I/opt/local/include -L/opt/local/lib -lssl -lcrypto

If we want to get rid of macports we'll just nuke /opt/local and be done with it.

[c-garcia]

Hi

The command below works for me 100% of the time, for the moment, with MacOS X 10.11.6, macports and dotnet core version 1.0.0-preview2-003131:

DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib:${HOME}/lib:/usr/local/lib:/lib:/usr/lib dotnet --version

Please, note the variable only affects the dotnet command.

HTH!

[weegeekps]

If you are a terminal newbie and want to replace your dotnet command with the command that @c-garcia posted, you can just add this alias to your .profile/.bashrc/.zshrc below.

alias dotnet="DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib:${HOME}/lib:/usr/local/lib:/lib:/usr/lib dotnet"

That will prevent you from having to make a potentially dangerous link from the macports rootfs.

[danmosemsft]

@bartonjs @steveharter Lots of tests are failing in the offical test runs due to this.

eg https://mc.dot.net/#/product/netcore/master/source/official~2Fcorefx~2Fmaster~2F/type/test~2Ffunctional~2Fcli~2F/build/20161024.01/workItem/System.Security.Cryptography.Algorithms.Tests

What should we do here -- have some setup step that gets openssl?

[bartonjs]

@danmosemsft Yep, there should already be a step that does it, but it probably calls brew link --force openssl, which doesn't work anymore. (The fact that machine setup isn't failing when it does that is a sign of a bug in infrastructure). Instead it needs to follow the steps on https://www.microsoft.com/net/core#macos (though with the mkdir /usr/local/lib that we haven't added on that page).

[jaredcnance]

I'm not sure if this is the right place to post this, but I'm having this same issue. However, it seems isolated to a single project. The symlinks are in place (see below).

$ dotnet --info
.NET Command Line Tools (1.0.0-preview2-003121)

Product Information:
 Version:            1.0.0-preview2-003121
 Commit SHA-1 hash:  1e9d529bc5

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

Trying to link:

$ ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib
ln: /usr/local/lib/libcrypto.1.0.0.dylib: File exists
$ ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib
ln: /usr/local/lib/libssl.1.0.0.dylib: File exists

$ which dotnet
/usr/local/bin/dotnet

$ which openssl
/usr/bin/openssl
$ openssl version
OpenSSL 0.9.8zh 14 Jan 2016

$ brew info openssl
openssl: stable 1.0.2j (bottled) [keg-only]
SSL/TLS cryptography library
https://openssl.org/
/usr/local/Cellar/openssl/1.0.2j (1,695 files, 12M)
  Poured from bottle on 2016-10-27 at 15:34:33
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/openssl.rb
==> Dependencies
Build: makedepend ✔
==> Options
--universal
    Build a universal binary
--without-test
    Skip build-time tests (not recommended)
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

This formula is keg-only, which means it was not symlinked into /usr/local.

Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries

Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include

The project restores perfectly fine, but crashes at runtime with the following error:

System.Data.SqlClient.SqlException: A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: SSL Provider, error: 31 - Encryption(ssl/tls) handshake failed) ---> System.TypeInitializationException: The type initializer for 'Crypto' threw an exception. ---> System.TypeInitializationException: The type initializer for 'CryptoInitializer' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native.OpenSsl': The specified module could not be found.

The project.json of the offending application:

{
  "version": "1.0.0-*",
  "buildOptions": {
    "emitEntryPoint": true,
    "copyToOutput": [
      "**/*.py"
    ]
  },

  "dependencies": {
    "Microsoft.NETCore.App": {
      "type": "platform",
      "version": "1.0.1"
    },
    "Microsoft.Extensions.DependencyInjection": "1.1.0-preview1-final",
    "Autofac.Extensions.DependencyInjection": "4.0.0",
    "Microsoft.Extensions.Logging.Console": "1.0.0",
    "Microsoft.EntityFrameworkCore.Tools": "1.0.0-preview2-final",
    "Npgsql.EntityFrameworkCore.PostgreSQL": "1.0.0"
  },

  "frameworks": {
     "netcoreapp1.0": {
      "imports": ["dnxcore50", "portable-net45+win8"]
    }
  },

  "tools": {
    "Microsoft.EntityFrameworkCore.Tools": {
      "version": "1.0.0-preview2-final",
      "imports": [
        "portable-net45+win8+dnxcore50",
        "portable-net45+win8"
      ]
    },
    "Microsoft.DotNet.Watcher.Tools": "1.0.0-*"
  },

  "publishOptions": {
    "include": "**/*.py"
  },

  "packOptions": {
    "files": {
      "include": "**/*.py"
    }
  }
}

I'm really unsure why it would happen in this application and no other app. I have also tried switching to a different machine which can also run other dotnet core applications fine with no luck. Also, if I cd /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/ I can see these:

...
-rwxr-xr-x    1 jcnance  admin     61652 Oct 27 16:30 System.Security.Cryptography.Native.dylib
-rwxr-xr-x    1 jcnance  admin    106496 Oct 12 16:41 System.Security.Cryptography.OpenSsl.dll
...
lrwxr-xr-x    1 jcnance  admin        33 Oct 27 16:22 libssl.1.0.0.dylib -> /opt/local/lib/libssl.1.0.0.dylib

but no System.Security.Cryptography.Native.OpenSsl as the error would suggest.

Also, I noticed the mismatching openssl versions between which openssl and brew info openssl. So, I created an alias to the brew version:

which openssl
openssl: aliased to /usr/local/Cellar/openssl/1.0.2j/bin/openssl

but this didn't help either.

Please let me know if this needs to be moved to a different issue.

Update: I was able to reproduce the error in a different project by restoring the dependencies. With no other changes, this project was working fine earlier this week.

[bartonjs]

@jaredcnance I think you're hitting dotnet/corefx#12517. If you check in your project.lock.json file you'll probably see that you're needing System.Security.Cryptography.Algorithms 4.3.0. If so, it's the other issue.

[jaredcnance]

Thanks! That does appear to be the issue.

[damilola-dealdey]

Hello!
I have this same issue but i am running on an Ubuntu 16 system.

Please Help!

[bartonjs]

@damilola-dealdey Are you having trouble loading System.Security.Cryptography.Native, or System.Security.Cryptography.Native.OpenSsl? I'm assuming the latter, which is more like https://github.com/dotnet/corefx/issues/12517. Psychic debugging says you have a mix of the 1.0 shared framework and a new 1.1-based individual package. The easiest answer is to upgrade to 1.1.

[kaykayes]

brew install openssl did it but it stopped after upgrading to macOs High Sierra. OpenSSL libraries were not supported.
installing aspnet core 2 sorted the issue. Thanks everyone