Vstest: Unable to find tests with FileNotFoundException

Created on 12 May 2018 · 5Comments · Source: microsoft/vstest

Description

IMPORTANT: if the defect is reproduced only in a workflow from within the Visual Studio IDE then do not report the issue here - instead, please report it using Visual Studio's "Send Feedback" option that can be accessed from the Help menu OR using this link https://developercommunity.visualstudio.com.

For a defect reproducable from the vstest command line, describe the issue you've observed.

I have a test which run fine locally but always meet issue in cloudtest complaint about FileNotFoundException and unable to find tests.

Steps to reproduce

What steps can reproduce the defect?
Please share the setup, commandline for vstest.console, sample project, target
framework etc.

dotnet.exe vstest HttpServiceE2ETests.dll /Platform:X64 /Logger:TRX /Diag:diag.log

Expected behavior

Share the expected output

Expect test to discover 4 tests and run fine.

Actual behavior

What is the behavior observed?

System.IO.FileNotFoundException: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable1 sources, IDictionary2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
Test Run Aborted.

Diagnostic logs

Please share test platform diagnostics logs. Instructions to collect logs are here.
The logs may contain test assembly paths, kindly review and mask those before sharing.

TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.434, 41206697267, vstest.console.dll, Using platform:X64
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.451, 41206856385, vstest.console.dll, TestPluginCache: Discovering the extensions using extension path.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.453, 41206866978, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.BlameDataCollector: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.454, 41206883854, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.BlameDataCollector: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.455, 41206894334, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.EventLogCollector: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.458, 41206917201, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.EventLogCollector: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.459, 41206933409, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.460, 41206941147, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.472, 41207060728, vstest.console.dll, AssemblyResolver: Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.473, 41207068940, vstest.console.dll, AssemblyResolver: Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger: Resolved assembly.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.475, 41207087983, vstest.console.dll, TestPluginCache: Discovered the extensions using extension path ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.475, 41207091793, vstest.console.dll, TestPluginCache: Discoverers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476, 41207098029, vstest.console.dll, TestPluginCache: Executors are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476, 41207100664, vstest.console.dll, TestPluginCache: Setting providers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476, 41207103351, vstest.console.dll, TestPluginCache: Loggers are ''.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.481, 41207149838, vstest.console.dll, RunTestsArgumentProcessor:Execute: Test run is starting.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.481, 41207152870, vstest.console.dll, RunTestsArgumentProcessor:Execute: Queuing Test run.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.481, 41207156447, vstest.console.dll, TestRequestManager.RunTests: run tests started.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.496, 41207303495, vstest.console.dll, AssemblyMetadataProvider.GetFrameWork: Determined framework:'.NETCoreApp,Version=v2.0' for source: 'D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll'
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.497, 41207307048, vstest.console.dll, Determined framework for all sources: .NETCoreApp,Version=v2.0
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.500, 41207339994, vstest.console.dll, AssemblyMetadataProvider.GetArchitecture: Determined architecture:X64 info for assembly: D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.503, 41207370166, vstest.console.dll, Determined platform for all sources: X64
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.506, 41207406088, vstest.console.dll, Compatible sources list :
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.507, 41207408817, vstest.console.dll, D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.510, 41207440141, vstest.console.dll, InferRunSettingsHelper.IsTestSettingsEnabled: Unable to navigate to RunSettings/MSTest. Current node: RunSettings
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.519, 41207529497, vstest.console.dll, TestPluginCache: Discovering the extensions using extension path.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520, 41207537107, vstest.console.dll, TestPluginCache: Discovered the extensions using extension path ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520, 41207540189, vstest.console.dll, TestPluginCache: Discoverers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520, 41207544095, vstest.console.dll, TestPluginCache: Executors are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520, 41207546282, vstest.console.dll, TestPluginCache: Setting providers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.521, 41207548930, vstest.console.dll, TestPluginCache: Loggers are 'logger://Microsoft/TestPlatform/Extensions/Blame/v1,logger://Microsoft/TestPlatform/TrxLogger/v1'.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.522, 41207562217, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.Extensions.TrxLogger.TrxLogger
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.527, 41207613337, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.528, 41207618399, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.528, 41207624248, vstest.console.dll, TestEngine: Initializing Parallel Execution as MaxCpuCount is set to: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.529, 41207629241, vstest.console.dll, InferRunSettingsHelper.IsTestSettingsEnabled: Unable to navigate to RunSettings/MSTest. Current node: RunSettings
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.539, 41207734909, vstest.console.dll, TestRequestSender is acting as server
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.541, 41207754697, vstest.console.dll, TestRunRequest.ExecuteAsync: Creating test run request.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.542, 41207758917, vstest.console.dll, TestRunRequest.ExecuteAsync: Starting.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.542, 41207762681, vstest.console.dll, TestRunRequest.ExecuteAsync: Starting run with settings:TestRunCriteria:
KeepAlive=False,FrequencyOfRunStatsChangeEvent=10,RunStatsChangeEventTimeout=00:00:01.5000000,TestCaseFilter=,TestExecutorLauncher=
Settingsxml=

D:\data\CHT\6311c6f0\TestResults
X64
.NETCoreApp,Version=v2.0
False
False

TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.542, 41207765198, vstest.console.dll, TestRunRequest.ExecuteAsync: Wait for the first run request is over.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.543, 41207770918, vstest.console.dll, ParallelProxyExecutionManager: Start execution. Total sources: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.544, 41207780458, vstest.console.dll, ProxyParallelExecutionManager: Triggering test run for next source: D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.544, 41207785363, vstest.console.dll, ProxyParallelExecutionManager: No sources available for execution.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.544, 41207785703, vstest.console.dll, ParallelProxyExecutionManager: Execution started. Started clients: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.545, 41207788524, vstest.console.dll, TestRunRequest.ExecuteAsync: Started.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.545, 41207793429, vstest.console.dll, ProxyExecutionManager: Test host is always Lazy initialize.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.545, 41207793719, vstest.console.dll, TestRunRequest.WaitForCompletion: Waiting with timeout -1.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.546, 41207797629, vstest.console.dll, TestRequestSender.InitializeCommunication: initialize communication.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.559, 41207935771, vstest.console.dll, SocketServer: Listening on end point : 127.0.0.1:1745
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564, 41207977683, vstest.console.dll, DotnetTestHostmanager: Full path of dotnet.exe is C:\Program Files\dotnet\dotnet.exe
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564, 41207981307, vstest.console.dll, DotnetTestHostmanager: Adding --runtimeconfig "D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.runtimeconfig.json" in args
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564, 41207984800, vstest.console.dll, DotnetTestHostmanager: Adding --depsfile "D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.deps.json" in args
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.565, 41207988440, vstest.console.dll, AssemblyResolver: Microsoft.Extensions.DependencyModel: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.565, 41207995596, vstest.console.dll, AssemblyResolver: Microsoft.Extensions.DependencyModel: Resolved assembly.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.566, 41208000186, vstest.console.dll, DotnetTestHostmanager: Reading file D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.deps.json to get path of testhost.dll
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.612, 41208466391, vstest.console.dll, DotnetTestHostmanager: Relative path of testhost.dll with respect to package folder is microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.618, 41208520372, vstest.console.dll, DotnetTestHostmanager: Looking for path microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll in folder C:\Users_qcloud2.dotnet\store\|arch|\|tfm|
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.618, 41208523685, vstest.console.dll, DotnetTestHostmanager: Looking for path microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll in folder d:\dbs\Cx\iri.N
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.619, 41208526910, vstest.console.dll, DotnetTestHostManager: Assume published test project, with test host path = D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\testhost.dll.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.619, 41208531429, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.619, 41208535348, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Failed to load assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.620, 41208543424, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.621, 41208546738, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolved from cache.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.621, 41208551878, vstest.console.dll, DotnetTestHostmanager: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
TpTrace Error: 0 : 5108, 5, 2018/05/11, 23:17:40.694, 41209277739, vstest.console.dll, ProxyExecutionManager.StartTestRun: Failed to start test run: System.IO.FileNotFoundException: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable1 sources, IDictionary2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.770, 41210038717, vstest.console.dll, TestRunRequest:SendTestRunMessage: Starting.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.770, 41210043829, vstest.console.dll, TestRunRequest:SendTestRunMessage: Completed.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.771, 41210048734, vstest.console.dll, ParallelProxyExecutionManager: HandlePartialRunComplete: Total completed clients = 1, Run complete = True, Run canceled: False.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:41.842, 41220764369, vstest.console.dll, TestRunRequest:TestRunComplete: Starting. IsAborted:True IsCanceled:False.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:41.843, 41220769490, vstest.console.dll, TestLoggerManager.HandleTestRunComplete: Ignoring as the object is disposed.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:41.843, 41220773172, vstest.console.dll, TestRunRequest:TestRunComplete: Completed.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:41.843, 41220773621, vstest.console.dll, TestRunRequest.Dispose: Starting.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844, 41220778683, vstest.console.dll, TestRunRequest.Dispose: Completed.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844, 41220780969, vstest.console.dll, TestRequestManager.RunTests: run tests completed.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844, 41220783765, vstest.console.dll, RunTestsArgumentProcessor:Execute: Test run is completed.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:41.844, 41220786027, vstest.console.dll, Executor.Execute: Exiting with exit code of 1

Environment

Please share additional details about the test environment.
Operating system, Build version of vstest.console
.NET Core SDK (reflecting any global.json):
Version: 2.1.300-preview2-008530
Commit: 822ae6d43a

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\2.1.300-preview2-008530\

Host (useful for support):
Version: 2.1.0-preview2-26406-04
Commit: 6833f3026b

question

Source

chuanboz

Most helpful comment

@chuanboz
For running test projects that target NetCore, Testplatform needs to find testhost.dll for hosting.
The appropriate testhost.dll is searched based on the information in the deps file. In this case, the location assumed in the deps file does not exist on the cloudtest machine.

Solution for you is to Publish your test project, so that all the dependencies (this will include testhost.dll) get copied next to the test dll and hence gets resolved.
Please try it out and let us know if you are still hitting the issue.

singhsarab on 28 May 2018

👍5 ❤2

All 5 comments

Microsoft (R) Test Execution Command Line Tool Version 15.7.0-preview-20180221-13

here is httpservicee2etests.deps (3).json that indicate Microsoft.NET.Test.Sdk.
"Microsoft.NET.Test.Sdk/15.3.0": {
"dependencies": {
"Microsoft.TestPlatform.TestHost": "15.3.0"
}
},

chuanboz on 12 May 2018

singhsarab on 28 May 2018

👍5 ❤2

thanks @singhsarab for the tips, I've figured it out after looking into the source code what it is doing. The solution is the same as you have said, to publish the test project.

If #1585 could be fixed to provide better and more clear error message, that will help end user to resolve this by themselves quicker and easier.

chuanboz on 29 May 2018

Hi,

I'm facing the same issue when trying to execute some Selenium test as part of the release pipeline...
I have enabled diagnostics and a file seems to have been created, but not sure how I can access it.

Logging Vstest Diagnostics in file: D:\a\r1\a\diag.log

But how do I access that?

Thanks!

X00158293 on 28 Nov 2018

Hello @singhsarab I am having this same problem and after following your advice I have not been able to correct the issue.

I have several VsTest tasks in my AzureDevOps Release Pipeline, I have ensured that I have published all of the respective test binaries, json files and dependencies as Build artefacts. But no matter what I do I cannot run my Integration or Acceptance tests without seeing the following error:

2019-01-16T21:18:14.5234936Z vstest.console.exe 
2019-01-16T21:18:14.5235596Z "D:\a\r1\a\drop\Tests.Integration\Tests.Integration.dll"
2019-01-16T21:18:14.5235789Z /logger:"trx"
2019-01-16T21:18:14.5235976Z /TestAdapterPath:"D:\a\r1\a\drop\Tests.Integration"
2019-01-16T21:18:19.6372101Z Starting test execution, please wait...
2019-01-16T21:18:24.2975659Z System.IO.FileNotFoundException: Unable to find tests for D:\a\r1\a\drop\Tests.Integration\Tests.Integration.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
2019-01-16T21:18:24.2977430Z    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
2019-01-16T21:18:24.2977592Z    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources)
2019-01-16T21:18:24.2977657Z    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)