Runtime: Running Unit Tests fail with .net Core 2.1 when using System.Memory 4.5.0-preview3 package

[nockawa]

I have a very simple case:

• A .net Core 2.1 (Preview 2) library project containing the library to test. Referencing the System.Memory NuGet Package (4.5.0-preview3)
• A .net Core 2.1 (Preview 2) library project for the unit test, referencing the first library.

I've tried, xUnit, nUnit frameworks and dotnet test, Visual Studio Runner (15.7 Preview 1) and Resharper Runner (2018.1 EAP 3) and I can't get my unit test executed.
I've used "Microsoft.NET.Test.Sdk" Version="15.6.1" and "Microsoft.NET.Test.Sdk" Version="15.7.0-preview20180307".

I have this error:

Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Testhost process exited with error: Unhandled Exception: System.TypeLoadException: Could not load type 'System.MemoryExtensions' from assembly 'System.Memory, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
   at System.Net.IPAddress.Parse(String ipString)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.GetIPEndPoint(String value)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.SocketClient.Start(String endPoint)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.InitializeCommunication()
   at Microsoft.VisualStudio.TestPlatform.TestHost.DefaultEngineInvoker.Invoke(IDictionary`2 argsDictionary)
   at Microsoft.VisualStudio.TestPlatform.TestHost.Program.Main(String[] args)

   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)

Test Run Aborted.

Apparently the Microsoft.NET.Test.Sdk doesn't get along well with the System.Memory NuGet package (System.Memory 4.5.0-preview3-26318-03) that I use in my tested library because it can't load type 'System.MemoryExtensions' from assembly 'System.Memory, Version=4.0.1.0

Any idea?

[nockawa]

Please tell me if there's anything I can do to help troubleshooting/fixing the issue.
I've just jumped on the .net Core bandwagon but I think I can help!

[ahsonkhan]

@nockawa, can you please share your library project csproj and your test project csproj?

A .net Core 2.1 (Preview 2) library project containing the library to test.

What do you mean by that? When you create a new class library project, Visual Studio sets the TargetFramework to netstandard2.0, by default. What is your library's tfm?

I was able to create a Class Library Project along with an xUnit Test Project (targeting .NET Core 2.1) and run the tests, just fine.

```C#
using System;

namespace ClassLibrary6
{
public class Class1
{
public static bool MyTest(Span span)
{
return span.Length > 10;
}
}
}

```csproj
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.Memory" Version="4.5.0-preview3-26319-04" />
  </ItemGroup>

</Project>

```C#
using Xunit;

namespace XUnitTestProject1
{
public class UnitTest1
{
[Fact]
public void Test1()
{
Assert.False(ClassLibrary6.Class1.MyTest(new byte[5]));
Assert.True(ClassLibrary6.Class1.MyTest(new byte[15]));
}
}
}

```csproj
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0-preview-20170628-02" />
    <PackageReference Include="xunit" Version="2.2.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
    <!-- Package upgrade also works fine -->
    <!--<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0-preview-20180307-01" />
    <PackageReference Include="xunit" Version="2.4.0-beta.1.build3958" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0-beta.1.build3958" />-->
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\ClassLibrary6\ClassLibrary6.csproj" />
  </ItemGroup>

</Project>

I was also able to use dotnet build, dotnet test via the command line.

>dotnet test
Build started, please wait...
Build completed.

...
Microsoft (R) Test Execution Command Line Tool Version 15.7.0-preview-20180221-13
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.4825 Seconds

[ektrah]

@ahsonkhan I'm seeing the same problem with System.Memory preview2 packages running on the preview1 runtime (2.1.0-preview1-26216-03).

That is, if the test run starts at all. Running your code above just gets stuck at "Starting test execution, please wait...".

I assumed it's because preview2 packages simply don't run on preview1 runtimes.

[nockawa]

@ahsonkhan I use Visual Studio 2017 15.7 Preview 1, and I update the .csproj to set the projects (both lib and test) to <TargetFramework>netcoreapp2.1</TargetFramework>

I've created a repo here that demonstrate the bug.

A dotnet test gives the following results

dotnet test
Build started, please wait...
Build completed.

Test run for C:\GitHub\Mine\corefxUnitTestBug\xUnitTest\bin\Debug\netcoreapp2.1\xUnitTest.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.6.0-preview-20180109-01
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Testhost process exited with error: Unhandled Exception: System.TypeLoadException: Could not load type 'System.MemoryExtensions' from assembly 'System.Memory, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
   at System.Net.IPAddress.Parse(String ipString)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.GetIPEndPoint(String value)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.SocketClient.Start(String endPoint)
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler.InitializeCommunication()
   at Microsoft.VisualStudio.TestPlatform.TestHost.DefaultEngineInvoker.Invoke(IDictionary`2 argsDictionary)
   at Microsoft.VisualStudio.TestPlatform.TestHost.Program.Main(String[] args)

   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)

Test Run Aborted.

I've tried with many different combination of dependencies but I end up with always the same result.

Here's the .net core version:

dotnet --version
2.1.300-preview2-008261

[ahsonkhan]

@nockawa, can you please add the full output from dotnet --info as well? I suspect there is some version mismatch happening between preview1/preview2 that is causing this issue. I am going to try and repro with the same setup you have, so I need to know the shared framework version that you are referencing (dotnet --info will show that).

I'm seeing the same problem with System.Memory preview2 packages running on the preview1 runtime (2.1.0-preview1-26216-03).

@ektrah, mixing preview2 runtime with preview1 packages might lead to unexpected behaviors, especially for System.Memory since we moved the MemoryExtensions type down to corelib recently. Is there a particular reason why you can't use only preview1 bits (or only preview2 bits)? I haven't yet tried mixing the versions. Not sure if this scenario is relevant.

[nockawa]

@ahsonkhan
here you go:

dotnet --info
.NET Command Line Tools (2.1.300-preview2-008261)

Product Information:
 Version:            2.1.300-preview2-008261
 Commit SHA-1 hash:  c28f4a2e34

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.300-preview2-008261\

Microsoft .NET Core Shared Framework Host

  Version  : 2.1.0-preview2-26131-06
  Build    : b13a0d5c331f374afd35ded57b9a4b4ab128864c

[ektrah]

@ahsonkhan No reason. I just remember seeing exactly the same error message as @nockawa, who's apparently trying to mix a preview3 package with a preview2 runtime.

[nockawa]

@ektrah aligning versions leads to the same result.

[ahsonkhan]

I installed that exact version of the dotnet cli and was able to reproduce the issue that you guys reported.
The issue is not reproducible on the latest dotnet cli (2.1.300-preview3-008391).

Here are some workarounds (any of these should work):
1) If your library and tests are targeting netcoreapp2.1, you don't need to explicitly reference System.Memory at all (since it is inbox for 2.1). If you remove the package reference (you can continue to use Span/etc.), but still use the existing cli version, you won't see the issue.
2) Downgrade your System.Memory package to 4.5.0-preview2-26221-05 or lower (for instance, reference the System.Memory package from NuGet: 4.5.0-preview1-26216-02), OR upgrade your dotnet cli to latest 2.1.300-preview3-*. There was a change in System.Memory around end of February (where we moved MemoryExtensions down to corelib when applications referencing System.Memory are targeting .NET Core 2.1) which ended up causing this transient issue if you mix and match certain preview versions. _Note that these are nightly preview builds, which sometimes can have such issues._
3) Update just the runtime and add a <RuntimeFrameworkVersion> property specifying the version in your xUnit csproj, from https://github.com/dotnet/core-setup#daily-builds. We do something similar in corefxlab.

Let me know if any of these don't work out for you or if you see other packaging related issues. Otherwise, please close the issue as it is already fixed as far as I can tell.

Other notes:
If you look in your C:\Program Files\dotnet\shared\Microsoft.NETCore.App\\ directory and open up System.Private.CoreLib.dll and System.Memory.dll in ildasm, you should see MemoryExtensions in one of these places (under System - see image below). Going forward, it will be in System.Private.CoreLib.dll (for .NET Core 2.1+). In the version of CLI you are currently using, you won't see MemoryExtensions in S.P.C but still in System.Memory (it has since moved to S.P.C). However, if you reference a newer version of System.Memory explicitly in your application or library, the default in-box version gets overriden and now neither dlls have the type we need which results in missing MemoryExtension error.
System.Private.Corelib.dll:

System.Memory:

Is this something we need to fix for specific, previous preview versions of the product (like the version that @nockawa is referencing)? This isn't an issue in master nor in what is available on NuGet today (as part of preview1).

cc @livarcocc, @weshaggard

[joshfree]

/cc @joperezr

[ahsonkhan]

Let me know if any of these don't work out for you or if you see other packaging related issues. Otherwise, please close the issue as it is already fixed as far as I can tell.

@nockawa, did the suggested workarounds fix the issue you were seeing?

[nockawa]

sorry, I forgot to answer, yes it did. Aligning all the versions are working fine now. Thanks!

[ahsonkhan]

Since this issue is not reproducible on any sdk/package versions we have already shipped (nor on what's in master today), I am closing it. Feel free to re-open if that isn't the case.