Nunit-console: NUnit Console Runner fails when test assembly dynamically loads a DLL

[sansjunk]

I have a test assembly that runs perfectly fine when run in VisualStudio using the NUnit3 Test Runner. However the same fails when I try to run using the NUnit Console Runner. I suspect this has something to do with my test DLL statically linked with another DLL which in turn dynamically loads a third DLL.

There is a discussion about a similar error here but no solution. Is there some options I need to use for this kind of scenario?

Here is the console output:

nunit3-console.exe ......CommonTestFramework.sln
NUnit Console Runner 3.6.1
Copyright (C) 2017 Charlie Poole

Runtime Environment
OS Version: Microsoft Windows NT 10.0.15063.0
CLR Version: 4.0.30319.42000

Test Files
......CommonTestFramework.sln

Errors, Failures and Warnings

1) Invalid : CommonTestFramework.dll
Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

Run Settings
ProcessModel: InProcess
DisposeRunners: True
WorkDirectory: packagesNUnit.ConsoleRunner.3.6.1tools
DebugTests: True
NumberOfTestWorkers: 0
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.5.2
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False

[CharliePoole]

From what you indicate I see two possible problems;

1. You are forcing the tests to run in-process with a command-line option rather than letting NUnit decide how to run it. Do you have a reason for that? The general rule of thumb: if you don't know or don't have a good reason, then let NUnit decide what to do.

2. You are running the solution file. The runner is trying to run tests in every assembly you build. Based on it's name, CommonTestFramework.dll doesn't sound like a test framework. Why not just run the actual test assemblies rather than the solution? When you use the VS adapter, VS passes all the assemblies to it, but the adapter is designed to deal with that. It tries to figure out which assemblies are tests. The console runner assumes you would not be giving it an assembly unless it's a test and tries to run it.

If neither of these helps, please provide the command-line you are using to run the console runner.

[sansjunk]

@CharliePoole , I played around with various ways and have actually tried both of what you are suggesting. I was trying a hand at debug and therefore that output was when I was running in-process. And I tried passing the solution file to see if the runner could better figure out the dependencies that way.

Here is another run with the options you have suggested:

D:WorkWorkSpacesVisualStudioCommonTestFrameworkCommonTestFrameworkpackagesNUnit.ConsoleRunner.3.6.1tools>nunit3-console.exe ..\..\..\CommonTestFrameworkbinDebugCommonTestFramework.dll
NUnit Console Runner 3.6.1
Copyright (C) 2017 Charlie Poole

Runtime Environment
OS Version: Microsoft Windows NT 10.0.15063.0
CLR Version: 4.0.30319.42000

Test Files
..\..\..\CommonTestFrameworkbinDebugCommonTestFramework.dll

Errors, Failures and Warnings

1) Invalid : D:WorkWorkSpacesVisualStudioCommonTestFrameworkCommonTestFrameworkCommonTestFrameworkbinDebugCommonTestFramework.dll
Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

Run Settings
DisposeRunners: True
WorkDirectory: D:WorkWorkSpacesVisualStudioCommonTestFrameworkCommonTestFrameworkpackagesNUnit.ConsoleRunner.3.6.1tools
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.5.2
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 4

Test Run Summary
Overall result: Failed
Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2017-06-13 13:03:05Z
End time: 2017-06-13 13:03:06Z
Duration: 0.937 seconds

Results (nunit3) saved as TestResult.xml

[ChrisMaddock]

What version of the NUnit Framework are you using? There was an update in v3.7 which improves that error message - although there are still certain cases which don't seem to display the improvement.

See https://github.com/nunit/nunit/issues/1238

[CharliePoole]

I can guarantee that passing the solution file will not improve what NUnit does because each assembly is processed individually. It could cause spurious errors as well, but didn't seem to in your case since you have the same error.

There is some reason that your assembly cannot be loaded. Is it possible that the test is referencing a 32-bit assembly? An assembly with a higher target runtime than the test assembly itself?

Next step would be to get an internal trace using --trace:Debug.

[sansjunk]

@ChrisMaddock , updating NUnit to 3.7.1 produced a more helpful error message. It told me the DLL which couldn't be loaded. This coupled with @CharliePoole 's hint about the architecture helped me trace the issue. The dependent DLL was in fact 32-bit and needs to be so since it's calling into a JVM. So after changing the test framework DLL to 32-bit I got everything working fine!

Thanks to both of you for helping me track this to a solution. However I wonder how it was working fine via VisualStudio using NUnit3TestAdapter?

[CharliePoole]

Glad it worked!

You raise a really good question about why it worked in VS. Do you have a .runsettings file that tells it to do 32-bit? @OsirisTerje can you shed any light on this? How does VS decide the nature of the process it will create to run the tests?

[sansjunk]

I do not have any .runsettings file. The target "Platform target" of the test DLL earlier was "Any CPU" and I changed it to "x86" to get it to run fine with the Console Runner.

[CharliePoole]

That's probably the best thing to do. Using the --x86 option of the console runner is also possible.

As for VS it's still a mystery as to why it is able to work.

[VinS101]

Worked for me! I set my NUnit test runner in Teamcity to x86.

[304NotModified]

It had the same issue. --x86 worked, but also changing the project to x64 or any cpu.

Please add to the console:

• that it's running in 64 bits
• that --x86 or changing the project could work.

Thanks!

[CharliePoole]

@304NotModified Please create a new issue... posting a new request on a closed issue will generally not get anybody working on that request! 😃

[kditrj2d]

Thanks. I have opened #444. Issue #315 does appear to be Open to me.

From: CharliePoole [e-mail]
Sent: 27 July 2018 16:04
To: nunit/nunit-console [e-mail]
Cc: Subscribed [e-mail]
Subject: Re: [nunit/nunit-console] NUnit Console Runner fails when test assembly dynamically loads a DLL (#241)

Please create a new issue... posting a new request on a closed issue will generally not get anybody working on that request! 😃

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/nunit/nunit-console/issues/241#issuecomment-408446135, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AabFuIWg-rG0lGUjfcjGLQb49405kMX6ks5uKyvPgaJpZM4N4YnI.

[CharliePoole]

@kditrj2d Sorry, my comment was intended for @304NotModified who asked for some new information to be displayed by the console. I think your new issue #444 is a duplicate of #315 isn't it?

[kditrj2d]

Yes, it is.

From: CharliePoole [e-mail]
Sent: 27 July 2018 16:32
To: nunit/nunit-console [e-mail]
Cc: David Razzetti [e-mail]; Mention [e-mail]
Subject: Re: [nunit/nunit-console] NUnit Console Runner fails when test assembly dynamically loads a DLL (#241)

@kditrj2dhttps://github.com/kditrj2d Sorry, my comment was intended for @304NotModifiedhttps://github.com/304NotModified who asked for some new information to be displayed by the console. I think your new issue #444https://github.com/nunit/nunit-console/issues/444 is a duplicate of #315https://github.com/nunit/nunit-console/issues/315 isn't it?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/nunit/nunit-console/issues/241#issuecomment-408455052, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AabFuN0e0b3MsKXBnVP9SplJLbpqfdbGks5uKzJ-gaJpZM4N4YnI.