Vstest: "dotnet test" fails to find .so file in output folder on Ubuntu when LD_LIBRARY_PATH is blank.

[nguerrera]

_From @mpp-oliverh on July 11, 2018 21:29_

I have a cross-platform project which uses [DllImport] to load a native binary. I have equivalent .dll and .so placed in the application root, set to CopyAlways and they appear in the output folder.

The application runs correctly on both Windows and Linux (Ubuntu) both when published and from the command line with "dotnet run".

I have a test which calls a method from the loaded library. On Windows the test runs correctly, on a Linux system when LD_LIBRARY_PATH is not set (such as the visualstudio.com hosted Linux build agent) the test fails with a DllNotFoundException.

Setting the LD_LIBRARY_PATH to : (just a colon) causes the tests to pass. I believe an incorrect assumption is made somewhere during the test procedure about LD_LIBRARY_PATH.

Further details below, including further tests.

Many thanks!

Steps to reproduce

1. Create a project with a [DllImport] method from a library.so file.
2. Create a test for the method.
3. Set the library.so file to CopyAlways so it appears in the output folder alongside the normal application dlls.
4. Use the VSTS hosted linux build agent or host your own linux vsts-agent which has no LD_LIBRARY_PATH by default.
5. Build the project and run it - the method successfully loads the library.so file from the application folder and runs the imported method properly.
6. Run "dotnet test" against the project and you get DllNotFoundException.

Expected behavior

The tests run successfully - the .so file is found and loaded by the method under test.

Actual behavior

Error Message:
System.DllNotFoundException : Unable to load shared library 'library.so' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibrary: cannot open shared object file: No such file or directory

Additional info

The fact that LD_LIBRARY_PATH is blank is relevant.
If LD_LIBRARY_PATH either starts or ends with a : then the test runs correctly.

export LD_LIBRARY_PATH=
dotnet test
fails with error above

export LD_LIBRARY_PATH=:
dotnet test
passes

export LD_LIBRARY_PATH=:blah
dotnet test
passes

export LD_LIBRARY_PATH=blah:
dotnet test
passes

export LD_LIBRARY_PATH=blah
dotnet test
fails with error above

export LD_LIBRARY_PATH=path_to_application_output_directory
dotnet test
passes

Environment data

dotnet --info output:

root@vsts-agent-5c87858b7b-dsbjf:/vsts# dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 2.1.301
Commit: 59524873d6

Runtime Environment:
OS Name: ubuntu
OS Version: 16.04
OS Platform: Linux
RID: ubuntu.16.04-x64
Base Path: /usr/share/dotnet/sdk/2.1.301/

Host (useful for support):
Version: 2.1.1
Commit: 6985b9f684

.NET Core SDKs installed:
2.1.301 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download

_Copied from original issue: dotnet/cli#9649_

[XiaoFaye]

Even set the LD_LIBRARY_PATH as your suggestions, I still got the error in Ubuntu 18.04. The dotnet core version is 2.1.302

[cltshivash]

@nguerrera Do you see this issue on a simple console app ? (to narrow down the root cause)

[nguerrera]

I haven't reproduced this myself. I moved @mpp-oliverh's issue because they indicated that it only repros in dotnet test. Perhaps they can answer your question...

[abhishkk]

@mpp-oliverh
Are you seeing this issue on simple console app as well?

[nigurr]

Closing this issue as there's no further response. Please feel free to re-open this if you are still having an issue.

[ashkaun]

I am still running into this issue (on CentOS rather than Ubunutu). The workaround mentioned in the original post works for me as well.

I am also able to reproduce the issue on macOS, however it refers to the DYLD_LIBRARY_PATH rather than LD_LIBRARY_PATH due to the difference in loading linked libraries between the two operating systems.

To reiterate:

The application runs correctly on both Windows and Linux (Ubuntu) both when published and from the command line with "dotnet run".

I have a test which calls a method from the loaded library. On Windows the test runs correctly, on a Linux system when LD_LIBRARY_PATH is not set (such as the visualstudio.com hosted Linux build agent) the test fails with a DllNotFoundException.

Setting the LD_LIBRARY_PATH to : (just a colon) causes the tests to pass. I believe an incorrect assumption is made somewhere during the test procedure about LD_LIBRARY_PATH.

[nigurr]

@ashkaun thanks for updating the thread. This seems to be an issue with dotnet coreclr. Test platform doesn't use any of these variables to load dependencies. They are resolved by the coreclr.
I came across this active issue in coreclr https://github.com/dotnet/coreclr/issues/12707. Are they related? If not, can you please open new issue in dotnet/coreclr repo.

[ashkaun]

@nigurr From what I can tell this issue is unique to dotnet test, there are no problems running with the *.so file when running it with dotnet run. Additionally the *.so files are in the output & publish directories for the test projects as well as the solution itself.

I'll see if the issue you linked provides additional info in isolating the behaviour I'm experiencing.

[mpp-oliverh]

@cltshivash the issue report was prompted by a problem with testing a dotnet console application.

@nigurr the issue isn't present at run time, only ever during test. that issue you linked doesn't seem directly related.

this issue was originally posted to the dotnet-cli repository and was moved while I was away, sorry for the delay responding.

[karanjitsingh]

Sorry for the delayed response. This seems to be a coreclr issue since we don't internally check for or set LD_LIBRARY_PATH. Our recommendation would be to set this externally.

Considering the time that has passed since this issue was opened, maybe they've already fixed it? If not, please report this issue at dotnet/cli

[karanjitsingh]

Closing this issue for now.

[oliverholliday]

Hmm they moved it to here from there!

The problem wasn't fixed as of last week, I'll follow up with them thanks.

[nguerrera]

I'll move this to coreclr. This isn't a CLI issue either.

@oliverholliday Is it possible to supply a complete repro project and steps?

[nguerrera]

This issue was moved to dotnet/coreclr#21176