Testfx: Test run failed with Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' or one of its dependencies. Unspecified error (Exception from HRESULT: 0x80004005 (E_FAIL))

[smadala]

Description

Not able to run tests

Steps to reproduce

Create MSTest Desktop Unit test project
Add app.config file with content similar to following

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.2" />
    </startup>
  <appSettings>
    <?ap-config target="/configuration/appSettings/add[@key='AzureKeyVaultConnectionString']/@value" value="VaultUrl=https://TestAnBKV02.vault.azure.net:443; ServicePrincipalId=https://TestDocumentAdjustment; StoreName=My; StoreLocation=LocalMachine; ThumbPrint=‎72372785dab2b10f353046ba5785c64a3df426f8;" when="@environment='Redmond-CI'" ?>
  </appSettings>
</configuration>

Run tests

Expected behavior

Tests should run.

Actual behavior

test output pane content:

[6/20/2017 6:42:11 PM Informational] ------ Load Playlist started ------
[6/20/2017 6:42:11 PM Informational] ========== Load Playlist finished (0:00:00.0050034) ==========
[6/20/2017 6:43:59 PM Informational] ------ Discover test started ------
[6/20/2017 6:43:59 PM Warning] Could not load file or assembly 'file:///C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\ALM\ENTERPRISE\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\TESTWINDOW\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' or one of its dependencies. Unspecified error (Exception from HRESULT: 0x80004005 (E_FAIL))
[6/20/2017 6:43:59 PM Informational] ========== Discover test finished: 0 found (0:00:00.2929448) ==========
[6/20/2017 7:05:18 PM Informational] ------ Discover test started ------
[6/20/2017 7:05:18 PM Informational] ========== Discover test finished: 0 found (0:00:00.0510373) ==========

EqtTrace content:

W, 51832, 14, 2017/06/20, 14:59:10.852, 1686621383696, vstest.executionengine.exe, MSTestDiscoverer.TryGetTests: Failed to discover tests from C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\QTestRerun_new\ClientCenter.Common.Test.dll\ClientCenter.Common.Test.dll. Reason:System.IO.FileLoadException: Could not load file or assembly 'file:///C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\Microsoft.VisualStudio.Test.14.0.25420.25\BinaryDeployment\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' or one of its dependencies. Unspecified error (Exception from HRESULT: 0x80004005 (E_FAIL))
File name: 'file:///C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\Microsoft.VisualStudio.Test.14.0.25420.25\BinaryDeployment\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile, Evidence securityEvidence)
   at System.Activator.CreateInstanceFromInternal(String assemblyFile, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstanceFrom(String assemblyFile, String typeName)
   at System.AppDomain.CreateInstanceFromAndUnwrap(String assemblyName, String typeName)
   at System.AppDomain.CreateInstanceFromAndUnwrap(String assemblyName, String typeName)
   at Microsoft.VisualStudio.TestPlatform.ObjectModel.EqtTrace.SetupRemoteEqtTraceListeners(AppDomain childDomain)
   at Microsoft.VisualStudio.TestPlatform.MSTestFramework.AppDomainIsolatedHost.CreateEnumerator(String assemblyFileName, IRunSettings runSettings)
   at Microsoft.VisualStudio.TestPlatform.MSTestFramework.AssemblyEnumeratorWrapper.GetTests(String assemblyFileName, IDiscoveryContext discoveryContext, ICollection`1& warnings)
   at Microsoft.VisualStudio.TestPlatform.Extensions.VSTestIntegration.UnitTestDiscoverer.DiscoverTests(String source, IDiscoveryContext discoveryContext, IMessageLogger logger, ITestCaseDiscoverySink discoverySink)

Repro project:
AssemblyLoadUnitTestProject.zip

Enable fusion logs show following failure:

<meta http-equiv="Content-Type" content="charset=unicode-1-1-utf-8"><!-- saved from url=(0015)assemblybinder: --><html><pre>
*** Assembly Binder Log Entry  (6/20/2017 @ 6:10:24 PM) ***

The operation failed.
Bind result: hr = 0x80004005. Unspecified error

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\Microsoft.VisualStudio.Test.14.0.25420.25\BinaryDeployment\vstest.executionengine.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: Where-ref bind. Location = C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\Microsoft.VisualStudio.Test.14.0.25420.25\BinaryDeployment\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll
LOG: Appbase = file:///C:/Users/samadala/Desktop/DRI/cloudbuild/MSASG_AnB_retail/QTestRerun_new/ClientCenter.Common.Test.dll
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = NULL
Calling assembly : (Unknown).
===
LOG: This bind starts in LoadFrom load context.
WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
LOG: Download of application configuration file was attempted from file:///C:/Users/samadala/Desktop/DRI/cloudbuild/MSASG_AnB_retail/QTestRerun_new/ClientCenter.Common.Test.dll/ClientCenter.Common.Test.dll.config.
LOG: Found application configuration file (C:\Users\samadala\Desktop\DRI\cloudbuild\MSASG_AnB_retail\QTestRerun_new\ClientCenter.Common.Test.dll\ClientCenter.Common.Test.dll.config).
ERR: There was an error parsing XML (hr = 0x80004005).
ERR: Parse XML memory stream failed.
ERR: Unrecoverable error occurred during pre-download check (hr = 0x80004005).

</pre></html>

[AbhitejJohn]

We need to figure out if/how we can better message this to users instead of just throwing a blanket exception below:

Could not load file or assembly 'file:///C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\ALM\ENTERPRISE\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\TESTWINDOW\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' or one of its dependencies. Unspecified error (Exception from HRESULT: 0x80004005 (E_FAIL))

@smadala ,
I'm gonna assign this to you to take a look.

[dukeofharen]

For anyone running in this issue with netcoreapp2.0 and MSTest, I updated the following packages to an older version (and the tests run again after this):

• Microsoft.NET.Test.Sdk: 15.3.0
• MSTest.TestAdapter: 1.1.18
• MSTest.TestFramework: 1.1.18

BTW, only had this on Bash on Windows, not on Windows itself.

[smadala]

@dukeofharen Can you share logs by following here. Do you have app.config for unit test project? Can you share sample repro?

[dukeofharen]

Hi @smadala,

It might be a totally different problem, but I did investigate a little bit.

I tried running unit tests with the --diag set, but get no log files whatsoever.
The repository that contain the tests that fail is https://github.com/dukeofharen/armyknife.
A lot of unit tests worked anyway, but the ones that fail are mostly located in https://github.com/dukeofharen/armyknife/tree/master/src/Armyknife.Tests/Integration/Tools.
The exception occurred in the method decorated with the TestInitialize attribute (which is located in another class and project). The original exception was:

Failed   Armyknife.Business.Tests.DependencyRegistrationFacts.DependencyRegistration_ResolveExecutor_HappyFlow
Error Message:
 Initialization method Armyknife.Business.Tests.DependencyRegistrationFacts.Initialize threw exception. System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information..
Stack Trace:
    at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Armyknife.Utilities.AssemblyHelper.<>c__0`1.<GetImplementations>b__0_0(Assembly s) in /mnt/f/Code/priveprojecten/armyknife/src/Armyknife.Utilities/AssemblyHelper.cs:line 13
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray()
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Armyknife.Utilities.AssemblyHelper.GetImplementations[TInterface]() in /mnt/f/Code/priveprojecten/armyknife/src/Armyknife.Utilities/AssemblyHelper.cs:line 11
   at Armyknife.Business.DependencyRegistration.RegisterDependencies(IServiceCollection serviceCollection) in /mnt/f/Code/priveprojecten/armyknife/src/Armyknife.Business/DependencyRegistration.cs:line 20
   at Armyknife.Business.Tests.DependencyRegistrationFacts.Initialize() in /mnt/f/Code/priveprojecten/armyknife/src/Armyknife.Tests/Business/DependencyRegistrationFacts.cs:line 19

Only when I caught this exception and checked the LoaderExceptions, this is what came up:

Standard Output Messages:
 System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.ObjectModel.Navigation.DiaSourceClass' from assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

I'm using .NET Core, so I'm not using an app.config file.

I suspect this is a different problem, but maybe you can do something with my information.

[smadala]

@dukeofharen I couldn't setup solution on BOW. From logs, Microsoft.VisualStudio.TestPlatform.ObjectModel.Navigation.DiaSourceClass is new class added in Microsoft.TestPlatform.ObjectModel -15.5.0. I clouldn't figure out why it was looking for it even projects have reference to Microsoft.NET.Test.Sdk-15.3.0. Can you try updating Microsoft.Net.Test.Sdk to 15.5.0? Can you figure out from where the Microsoft.TestPlatform.ObjectModel assembly getting loaded?

[cltshivash]

@dukeofharen Please reopen if you are still running into this issue

[david-x-chen]

Really?! Closed already? 'Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' does not exist in nuget package. Why the unit test of dotnet core still looking for this file?

[smadala]

@david-x-chen
Microsoft.NET.Test.Sdk dependent on Microsoft.TestPlatform.TestHost which is dependent on Microsoft.TestPlatform.ObjectModel (contains Microsoft.VisualStudio.TestPlatform.ObjectModel.dll).

Can you explain your scenario or share project?

[derhally]

Ran into this issue today trying to build and run unit tests on a .net standard 2.0 project using the microsoft/dotnet:sdk container image.

NUnit3TestExecutor converted 21 of 21 NUnit test cases

Test Run Aborted.
Failed   LogMessageIsCorrect
Error Message:
 System.Reflection.ReflectionTypeLoadException : Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
  ----> System.TypeLoadException : Could not load type 'Microsoft.VisualStudio.TestPlatform.ObjectModel.Navigation.DiaSourceClass' from assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Stack Trace:
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeAssembly.get_DefinedTypes()
   at Atlas.Auditing.AuditLogToLogLogger.<>c__6`1.<GetAllTypesOf>b__6_0(Assembly x) in /temp/Atlas/Auditing/AuditLogToLogLogger.cs:line 85
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Atlas.Auditing.ObjectPropertyAccessCache`1..ctor(IEnumerable`1 types) in /temp/Atlas/Auditing/ObjectPropertyAccessCache.cs:line 18
   at Atlas.Auditing.AuditLogToLogLogger..ctor(ILogger`1 logger, IOptions`1 options) in /temp/Atlas/Auditing/AuditLogToLogLogger.cs:line 31
   at Atlas.Auditing.Tests.AuditLogToLogLoggerTest.LoggerTests(LogLevel logLevel, Boolean loggingEnabled, Boolean expectedLogCalled, LogLevel expectedLogLevel, String expectedMessage) in /temp/Tests/Atlas.Auditing.Tests/Auditing/AuditLogToLogLoggerTest.cs:line 65
--TypeLoadException

Here are my package refs

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.0.0" />
    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="2.0.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="2.0.0" />
    <PackageReference Include="Microsoft.Extensions.Options" Version="2.0.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
    <PackageReference Include="Moq" Version="4.8.0" />
    <PackageReference Include="NUnit" Version="3.9.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
  </ItemGroup>

[smadala]

@derhally The actual issue (please see "Steps to reproduce" ) is different. Can you please log different issue with detail repro steps?

[smadala]

Reopening it because actual issue(having ?ap-config element in app.config fails the test run.) yet to address.

[FrankSzendzielarz]

Getting this too. Actually, this started to happen only after changing .net standard library up to 2.0 in the unit test project. Somehow a binding redirect was added into app.config file in the unit test project (System.Reactive...)

[smadala]

@FrankSzendzielarz Your problem sounds like not similar to this issue(refer Steps to reproduce
). If so can you please create separate issue?

[SidShetye]

FYI, this is also an issue in Apache Avro when porting it's C# language library and unit tests to .net standard 2.0.

Workaround

Downgrade Microsoft.NET.Test.Sdk to 15.3.0

Expectations

Behavior should be consistent across platforms. At the very minimum, reflection should be exception-free on non-windows OS's.

Details

It happens during deserialization when you want to reflect on all current assemblies to find the type required to materialize; specifically at assembly.GetTypes() (source) when assembly is Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

The exception is only seen on Linux and Mac (not on Windows) and is:

{System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()

Specifically, LoaderExceptions is

Could not load type 'Microsoft.VisualStudio.TestPlatform.ObjectModel.Navigation.DiaSourceClass' from assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

[kfrajtak]

Maybe an different problem, but I was getting the same error thanks to an invalid app.config file.

[cltshivash]

Microsoft.VisualStudio.TestPlatform.ObjectModel has few types specific for windows platform that is causing issues when doing something like "assembly.gettypes()". Recommend to filter assemblies based on what is needed as we won't be fixing this issue in the near future.