Vstest: 'No test is available' using 16.9.1

[JDCain]

Description

netcoreapp3.1 test sets updated to 16.9.1 which also use xUnit will not discover tests when being run on a build server with only VS 2019 Build Tools installed. Downgrading to 16.8.3 allows the tests to run fine.

Test sets with 16.9.1 work fine if run on a machine with full VS 2019 Enterprise.

Enabling xUnit diagnostics shows no data so it looks like it never loads the xUnit adaptor at all.

Updating projects with 16.9.1 to net5.0 allows them to run.

Steps to reproduce

Run a test dll using: dotnet test name.dll

Expected behavior

Tests run as normal

Actual behavior

No test is available in path\name.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

When comparing a failing log to a passing this error does not take place in a passing log:

TpTrace Warning: 0 : 7576, 4, 2021/03/05, 12:55:20.036, 4566936487095, testhost.dll, TestPluginDiscoverer: Failed to load extensions from file '*******.\xunit.runner.visualstudio.dotnetcore.testadapter.dll'.  Skipping test extension scan for this file.  Error: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.ComponentModel.Primitives.resources, Version=4.2.2.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.ComponentModel.Primitives.resources, Version=4.2.2.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.ComponentModel.Primitives.resources, Version=4.2.2.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.ComponentModel.Primitives.resources, Version=4.2.2.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, RuntimeAssembly assemblyContext, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformAssemblyResolver.AssemblyResolverEvent(Object sender, Object eventArgs)
   at System.Runtime.Loader.AssemblyLoadContext.InvokeResolveEvent(ResolveEventHandler eventHandler, RuntimeAssembly assembly, String name)
   at System.Runtime.Loader.AssemblyLoadContext.OnAssemblyResolve(RuntimeAssembly assembly, String assemblyFullName)


   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, RuntimeAssembly assemblyContext, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(CultureInfo culture, Version version, Boolean throwOnFileNotFound)
   at System.Resources.ManifestBasedResourceGroveler.InternalGetSatelliteAssembly(Assembly mainAssembly, CultureInfo culture, Version version)
   at System.Resources.ManifestBasedResourceGroveler.GetSatelliteAssembly(CultureInfo lookForCulture)
   at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary`2 localResourceSets, Boolean tryParents, Boolean createIfNotExists)
   at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents)
   at System.Resources.ResourceManager.GetString(String name, CultureInfo culture)
   at System.Resources.ResourceManager.GetString(String name)
   at System.SR.GetResourceString(String resourceKey, String defaultString)
   at System.ComponentModel.CategoryAttribute.GetLocalizedString(String value)
   at System.ComponentModel.CategoryAttribute.get_Category()
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.Utilities.TestDiscovererPluginInformation.GetAssemblyType(Type testDiscovererType)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.Utilities.TestDiscovererPluginInformation..ctor(Type testDiscovererType)


   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
   at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
   at System.Activator.CreateInstance(Type type, Object[] args)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionFromType[TPluginInfo](Type type, Type extensionType, Dictionary`2 extensionCollection)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromAssembly[TPluginInfo,TExtension](Assembly assembly, Dictionary`2 pluginInfos)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromFiles[TPluginInfo,TExtension](String[] files, Dictionary`2 pluginInfos)

Diagnostic logs

log.host.21-03-05_12-55-18_74667_6.txt
log.txt

Environment

Windows 10 VM
VS 2019 Build Tools
VS2017 Enterprise

[Shortcheese]

Same problem here.
Run tests with 16.9.1 and net5.0 gives us the same error on a machine with only Sdk installed on.

No test is available in path\name.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

TpTrace Warning: 0 : 12100, 4, 2021/03/08, 16:25:16.433, 9789094603132, testhost.dll, TestPluginDiscoverer: Failed to load extensions from file 'C:\**\xunit.runner.visualstudio.dotnetcore.testadapter.dll'. Skipping test extension scan for this file. Error: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.ComponentModel.Primitives.resources, Version=5.0.0.0, Culture=nl-NL, PublicKeyToken=b03f5f7f11d50a3a'.

[mattfrear]

I came here to +1.

In my case the tests aren't found on my local machine and also on our Azure DevOps build server.
16.8.3 still works. Just wasted an hour figuring that out.

[dwjobling]

Spent a few hours on this one. Seems to be a conflict issue with versions of Microsoft.TestPlatform.PlatformAbstractions.dll. The publish command is copying over a version with timestamp 11:57 whereas the compilation is using one time stamped at 12:25, the same as the other libraries in the package. Logs are rough to churn though but seems to be picking up a core 2.1 library. Replacing this assembly in the publish directory allows the tests to be found.

Also rolled back to 16.8.3

[vytas-dauksa]

Same here, rolled-back to 16.7.1 fixed it (didn't try 16.8.3)

[Kharos]

Still not working in 16.9.4.

I diffed the bin/Release/netcoreapp3.1 folder with the publish folder after running dotnet publish, and I can confirm that Microsoft.TestPlatform.PlatformAbstractions.dll is different between these two folders. As a workaround, replacing the dll in the publish folder with the one from the bin folder will fix the issue. Is this an issue in dotnet publish?

Edit: We are seeing this issue on Windows 10. On Mac it works without any problems.

[SeijiSuenaga]

Same problem here, and rolling back to 16.8.3 fixed it.

[josemedina760]

Getting this issue running Xunit test on my pipeline and rolling back to 16.8.3 fixed it. Spent so much time on this... Why would someone not test this on a pipeline prior to release.

[mattfrear]

I'm concerned that this has been open for one month and no one at Microsoft has responded.
@nohwnd are you aware of this?

.NET Core 3.1 is still on LTS, so this library should work with 3.1, but as you can see lots of us are having problems.

[tommarkosi]

+1!! Hanging back on 16.8.3 and watching this issue for any activity. Using .NET 3.1, and Azure DevOps pipelines

[JDCain]

I no longer see this issue with v16.10.0 and netcoreapp3.1.

[mattfrear]

Thanks for letting us know @JDCain.