Sdk: Unhandled Exception: System.DllNotFoundException: Unable to load DLL 'libproc': The specified module could not be found.

[Ox5f3759df]

Steps to reproduce

Install dotnet core
Update openssl with homebrew and update symlinks as per instructions:
cd /usr/lib/
ln -sf /usr/local/Cellar/openssl/1.0.2k/lib/libcrypto.1.0.0.dylib libcrypto.1.0.0.dylib
ln -sf /usr/local/Cellar/openssl/1.0.2k/lib/libssl.1.0.0.dylib libssl.1.0.0.dylib
dotnet new console --language c#

Running new works however when it tries to restore I get this error:
Failed to create prime the NuGet cache. restore failed with: 134

Unhandled Exception: System.DllNotFoundException: Unable to load DLL 'libproc': The specified module could not be found.
(Exception from HRESULT: 0x8007007E)
at Interop.libproc.proc_pidpath(Int32 pid, Byte* buffer, UInt32 bufferSize)
at Interop.libproc.proc_pidpath(Int32 pid)
at System.Diagnostics.Process.ResolvePath(String filename)
at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at Microsoft.DotNet.Cli.ForwardingApp.Execute()
at Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingApp.Execute()
at Microsoft.DotNet.Tools.Restore.RestoreCommand.<>c__DisplayClass0_0.b__0()
at Microsoft.DotNet.Cli.CommandLine.CommandLineApplication.Execute(String[] args)
at Microsoft.DotNet.Tools.Restore.RestoreCommand.Run(String[] args)
at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
at Microsoft.DotNet.Cli.Program.Main(String[] args)
Abort trap: 6

Expected behavior

Actual behavior

Environment data

dotnet --info output:
.NET Command Line Tools (1.0.1)

Product Information:
Version: 1.0.1
Commit SHA-1 hash: 005db40cd1

Runtime Environment:
OS Name: Mac OS X
OS Version: 10.11
OS Platform: Darwin
RID: osx.10.11-x64
Base Path: /usr/local/share/dotnet/sdk/1.0.

openssl version -a
OpenSSL 1.0.2h 3 May 2016 (Library: OpenSSL 1.0.2k 26 Jan 2017)
built on: reproducible build, date unspecified
platform: darwin64-x86_64-cc
options: bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) idea(int) blowfish(idx)
compiler: clang -I. -I.. -I../include -fPIC -fno-common -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
OPENSSLDIR: "/usr/local/etc/openssl"

[livarcocc]

@gkhanna79 can you help here?

[gkhanna79]

@danmosemsft or @karelz should be able to help.

[karelz]

@bartonjs does it ring a bell?

[karelz]

cc @janvorli

Can't find any hit in CoreFX repo or in internet search ... @Ox5f3759df any chance to try another Mac machine to rule out one-machine problems?

[janvorli]

CoreFX has been depending on libproc from the very beginning of its port to OSX. The library name is defined here:
https://github.com/dotnet/corefx/blob/master/src/Common/src/Interop/OSX/Interop.Libraries.cs
And the pinvokes here:
https://github.com/dotnet/corefx/blob/master/src/Common/src/Interop/OSX/Interop.libproc.cs

So, I think there can be several possible issues:

• The libproc is not installed on the target machine - that would be very strange, libproc seems to be part of the OSX.
• Some change in coreclr has broken lookup for interop with libraries from the system
• Some external change causes the libproc lookup to fail

@Ox5f3759df could you please check if the following file exists?
/usr/lib/libproc.dylib
If it does, could you please run the following command and dump the result here?
otool -L /usr/lib/libproc.dylib

Could you also run the following commands and dump their output here? I expect all of the env vars to be empty, if they are not, it might be causing the problem.
echo $LD_LIBRARY_PATH
echo $DYLD_LIBRARY_PATH
echo $DYLD_FALLBACK_LIBRARY_PATH
echo $DYLD_FRAMEWORK_PATH
echo $DYLD_FALLBACK_FRAMEWORK_PATH

[Ox5f3759df]

@janvorli
Thank you! I had these env vars set in my bash profile which was giving me the problem
$DYLD_LIBRARY_PATH
$DYLD_FALLBACK_LIBRARY_PATH

I am able to run dotnet restore now. Cheers!

[janvorli]

@Ox5f3759df I am glad to hear that!

[m2b]

I hate to ask. What did you set the env variables ($DYLD*) to? Where am I suppose to find libproc? Is it libproc.dylib?

[tarurar]

The same issue.
The output from otool -L /usr/lib/libproc.dylib:
/usr/lib/libproc.dylib:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
/usr/lib/system/libcache.dylib (compatibility version 1.0.0, current version 79.0.0)
/usr/lib/system/libcommonCrypto.dylib (compatibility version 1.0.0, current version 60092.50.5)
/usr/lib/system/libcompiler_rt.dylib (compatibility version 1.0.0, current version 62.0.0)
/usr/lib/system/libcopyfile.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/system/libcorecrypto.dylib (compatibility version 1.0.0, current version 442.50.19)
/usr/lib/system/libdispatch.dylib (compatibility version 1.0.0, current version 703.50.37)
/usr/lib/system/libdyld.dylib (compatibility version 1.0.0, current version 433.5.0)
/usr/lib/system/libkeymgr.dylib (compatibility version 1.0.0, current version 28.0.0)
/usr/lib/system/liblaunch.dylib (compatibility version 1.0.0, current version 972.60.2)
/usr/lib/system/libmacho.dylib (compatibility version 1.0.0, current version 898.0.0)
/usr/lib/system/libquarantine.dylib (compatibility version 1.0.0, current version 85.50.1)
/usr/lib/system/libremovefile.dylib (compatibility version 1.0.0, current version 45.0.0)
/usr/lib/system/libsystem_asl.dylib (compatibility version 1.0.0, current version 349.50.5)
/usr/lib/system/libsystem_blocks.dylib (compatibility version 1.0.0, current version 67.0.0)
/usr/lib/system/libsystem_c.dylib (compatibility version 1.0.0, current version 1158.50.2)
/usr/lib/system/libsystem_configuration.dylib (compatibility version 1.0.0, current version 888.60.2)
/usr/lib/system/libsystem_coreservices.dylib (compatibility version 1.0.0, current version 41.4.0)
/usr/lib/system/libsystem_coretls.dylib (compatibility version 1.0.0, current version 121.50.4)
/usr/lib/system/libsystem_dnssd.dylib (compatibility version 1.0.0, current version 765.50.9)
/usr/lib/system/libsystem_info.dylib (compatibility version 1.0.0, current version 503.50.4)
/usr/lib/system/libsystem_kernel.dylib (compatibility version 1.0.0, current version 3789.60.24)
/usr/lib/system/libsystem_m.dylib (compatibility version 1.0.0, current version 3121.6.0)
/usr/lib/system/libsystem_malloc.dylib (compatibility version 1.0.0, current version 116.50.8)
/usr/lib/system/libsystem_network.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/system/libsystem_networkextension.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/system/libsystem_notify.dylib (compatibility version 1.0.0, current version 165.20.1)
/usr/lib/system/libsystem_platform.dylib (compatibility version 1.0.0, current version 126.50.8)
/usr/lib/system/libsystem_pthread.dylib (compatibility version 1.0.0, current version 218.60.3)
/usr/lib/system/libsystem_sandbox.dylib (compatibility version 1.0.0, current version 592.60.1)
/usr/lib/system/libsystem_secinit.dylib (compatibility version 1.0.0, current version 24.50.4)
/usr/lib/system/libsystem_symptoms.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/system/libsystem_trace.dylib (compatibility version 1.0.0, current version 518.60.2)
/usr/lib/system/libunwind.dylib (compatibility version 1.0.0, current version 35.3.0)
/usr/lib/system/libxpc.dylib (compatibility version 1.0.0, current version 972.60.2)

The only variable which is not empty is $DYLD_FALLBACK_LIBRARY_PATH. The value is:
/Users/atarutin/.dnx/packages/Microsoft.AspNet.Server.Kestrel/1.0.0-rc1-final/runtimes/osx/native:

Any ideas?

[tarurar]

Anybody can help with this issue or, at least, point towards right direction?

[karelz]

Sadly both our experts are on vacation now. @Ox5f3759df would you be able to provide any guidance?

[esquivalient]

I the same issue as @tarurar and wasn't really in a position to unset DYLD_FALLBACK_LIBRARY_PATH. Instead, I appended the directory where libproc lives to the fallback path, like this

export DYLD_FALLBACK_LIBRARY_PATH=$DYLD_FALLBACK_LIBRARY_PATH:/usr/lib/

I did not investigate _why_ this was necessary, which I would probably do if I was going to make it a permanent part of my environment (which I'm not, I'm just over here working a support ticket for a dotnet customer of ours) so caveat emptor.

[tarurar]

@esquivalient thanks, it solved my issue )

[tarurar]

@esquivalient, can I make it globally? it works only until restart, then I need to execute this command again.

[tarurar]

Anybody else has the idea how to make it work constantly?
I have Rider and it works fine, it builds and runs dotnet core applications. Once I try to build it myself via terminal, it shows me the exception:

System.DllNotFoundException: Unable to load DLL 'libproc': The specified module or one of its dependencies could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Interop.libproc.proc_pidpath(Int32 pid, Byte* buffer, UInt32 bufferSize)
   at Interop.libproc.proc_pidpath(Int32 pid)
   at System.Diagnostics.Process.ResolvePath(String filename)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.DotNet.Cli.Utils.ProcessStartInfoExtensions.Execute(ProcessStartInfo startInfo)
   at Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingApp.Execute()
   at Microsoft.DotNet.Tools.Restore.RestoreCommand.Run(String[] args)
   at Microsoft.DotNet.Tools.RestoringCommand.Execute()
   at Microsoft.DotNet.Tools.Build.BuildCommand.Run(String[] args)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

@esquivalient?

[rmorriso]

The advice to verify the contents of the various LD_LIBRARY_PATHS was the key to solving this for me: https://github.com/dotnet/cli/issues/6225.

Ensure that the directories on these paths do in fact exist. In my case the environment variable was being set to support an old MySQL database install that had long since been removed. Seems that the dotnet stuff doesn't deal well with library paths that contain references to directories that don't exit.

[danmosemsft]

@janvorli this is a runtime failure -- is LD_LIBRARY_PATHS used at runtime? Is a fix necessary here to be more robust?

[janvorli]

@danmosemsft we don't explicitly use any of these (please note that there is no LD_LIBRARY_PATHS on OSX, @rmorriso uses the name as a reference to several env variables with similar purpose). The OS dynamic library loader uses them to find shared libraries. See here for the reference on the vars: https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/dyld.1.html
If they are misconfigured so that they don't contain the path where the libproc is located, then apps cannot load this library

[rmorriso]

Just to clarify, LD_LIBRARY_PATH and any of the other OSX DYLD_* variants do not need to be set. The issue I encountered was an invalid setting (if I recall it was DYLD_FALLBACK_FRAMEWORK_PATH) that referenced a directory that did not exist. Some old MySQL install that needed it at the time.

So it's not that libproc wasn't found on any of these load library paths but rather that an invalid path component on one of the PATHs was confusing matters and causing complaints about not finding libproc. I currently have NO library paths set at all and that is working fine. libproc will be found without specifying anything in the load library paths.

[danmosemsft]

@janvorli ah that's what I thought, there is no improvement possible on our side.

[janvorli]

What we could possibly do is to include some kind of trouble-shooting option to the dotnet tool that would check all libraries that coreclr, corefx and core-setup stuff can possibly use and see if it can load them. It could print the result of such check and also some hint on what we think might the user do to fix it (install something, fix the env vars, ...)

[rmorriso]

I'm not sure exactly what's happening with your code, but I think it might be worth checking the behaviour when the load libraries are being scanned. Probably best to skip an invalid directory with a log to STDERR rather than throw an exception and quit. (If that is indeed what's happening).

[janvorli]

@rmorriso we don't do anything with those variables, that's the OS loader reading and using them. We just ask it to load a library of the specified name.

[rmorriso]

hmm - pretty sure this isn't the OS loader failing - absolutely everything in OSX uses the OS loader and this error in the paths has been present forever (until today)

are you quite sure that dotNet itself doesn't perform some magic to allow DLLs to be mixed in with vanilla Darwin/Linux libraries?

[janvorli]

The shared library libproc.dylib we fail to load is an OSX component, it is not part of dotnet. The fact that the error message calls it "DLL" means "dynamically loaded library", it has nothing to do with the file extension. What dotnet does though is that it tries various permutations of the name until it succeeds loading it. We try to prepend "lib" and append ".dylib" (all possible combinations of those) to the name supplied in the pinvoke. So, in this case, the managed code uses "libproc" as the name, so we first try libproc and then liblibproc.dylib, libproc.dylib and liblibproc names. As for the locations, we first try locations requested by the host (dotnet executable), then the DefaultDllImportSearchPathsAttribute on the managed method, then DefaultDllImportSearchPathsAttribute of the module, then the application directory and finally if all has failed, we let the OS loader find it using its means.