Sdk: dotnet test with xUnit and full .NET framework causes System.BadImageFormatException

[jlandersen]

Steps to reproduce

Create a web application that targets full .NET framework, e.g. with:
"frameworks": { "net461": { } },

Create a class library with xUnit, testhost and web application as dependencies - e.g.:

{
  "version": "1.0.0-*",
  "testRunner": "xunit",
  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc2-build10015",
    "Microsoft.AspNetCore.TestHost": "1.0.0-rc2-final",
    "WebApp":  "*" 
  },
  "frameworks": {
    "net461": { }
  }
}

Create a unit test in class library that uses the web application:

public class IntegrationTests
{
    [Fact]
    public void Test()
    {
        var builder = new WebHostBuilder();
        builder.UseStartup<Startup>();
    }       
}

Expected behavior

Tests are executed properly

Actual behavior

Project WebApp (.NETFramework,Version=v4.5.1) was previously compiled. Skipping compilation.
Project WebApp.Tests (.NETFramework,Version=v4.6.1) will be compiled because inputs were modified
Compiling WebApp.Tests for .NETFramework,Version=v4.6.1

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:01.4138420


xUnit.net .NET CLI test runner (32-bit win81-x86)
  Discovering: WebApp.Tests
  Discovered:  WebApp.Tests
  Starting:    WebApp.Tests
      System.BadImageFormatException : Could not load file or assembly 'WebApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An attempt was made to load a program with an incorrect format.
      Stack Trace:
           at WebApp.Tests.IntegrationTests.Test()
  Finished:    WebApp.Tests
=== TEST EXECUTION SUMMARY ===
   WebApp.Tests  Total: 1, Errors: 0, Failed: 1, Skipped: 0, Time: 0,193s

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.3.9600
 OS Platform: Windows
 RID:         win81-x64

[Bob55]

Also experiencing this.

[joergjo]

I ran into the same issue today on Windows 10 x64. It seems the dotnet test runner _always_ runs as 32 bit executable and hence expects x86 binaries. Forcing both the ASP.NET Core and the xUnit test project to build x86 assemblies works for me as a stopgap measure.

"buildOptions": { "platform": "x86" }

[joergjo]

Ouch, now the web app crashes with anoter BadImageFormatException in Kestrel. See issue 4663 in MVC and issue 3091 in CLI.

[Bob55]

joergjo, I am experiencing the exact same thing if I force both projects to build x86 binaries. Though the xunit runner now executes, "dotnet run" fails in the main project with the following stack trace:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
at Microsoft.AspNetCore.Server.Kestrel.Networking.Libuv.NativeMethods.uv_loop_size()
at Microsoft.AspNetCore.Server.Kestrel.Networking.Libuv.loop_size()
at Microsoft.AspNetCore.Server.Kestrel.Networking.UvLoopHandle.Init(Libuv uv)
at Microsoft.AspNetCore.Server.Kestrel.KestrelThread.ThreadStart(Object parameter)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.KestrelEngine.Start(Int32 count)
at Microsoft.AspNetCore.Server.Kestrel.KestrelServer.StartTContext
at Microsoft.AspNetCore.Hosting.Internal.WebHost.Start()
at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host, CancellationToken token, String shutdownMessage)
at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)
at Test.Api.Program.Main(String[] args)
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[5]
Hosting shutdown

[joergjo]

Just out of curiosity: Did you use the standalone .NET Core SDK installer or the integrated setup for VS 2015? I'm using the latter.

[Bob55]

I'm using the standalone version.

[joergjo]

It seems dotnet-test-xunit 1.0.0-rc2-build10015 is the culprit. The NuGet package contains only the 32 bit runner, even in runtimes\win7-x64. If I replace the runner in bin\Debug\net461\win7-x64 with a x64 RC3 version the unit tests do work.

[macgyver76]

I'm also facing the same problem. Full framework 4.6.1 + win7-x64. @joergjo Where did you found x64 RC3 version runner?

[joergjo]

I used the MusicStore sample dev branch to compare a supposedly working unit test setup with mine, that's how I stumbled across it. If you download a copy of MusicStore and run dotnet restore, it'll download RC3.

[macgyver76]

Thanks but anyway - it should be fixed.

[joergjo]

Just found this: dotnet-test-xunit 1.0.0-rc2-build10015 has a 32 bit binary in the runtimes/net451/win7-x64 directory. So it is an xUnit issue after all.

[macgyver76]

@joergjo It works if you download RC3 of "dotnet-test-xunit": "1.0.0-rc3-*". Hovewer VS itegration (eg. discovering tests in Test Explorer) should be provided (am I wrong?) by the MS guys.

[joergjo]

After upgrading to RC3, both dotnet test and VS integration work for me. I also didn't need to muck about with any dependencies for the net461 TFM.

[joergjo]

I take that back. On on of my machines, I do need

"net461": { "dependencies": { "Microsoft.NETCore.Platforms": "1.0.1-rc2-24027" } }

in the test project in order to get VS2015 discover them.

[wpostma]

I saw a report on stackoverflow that the upstream package problem that triggers this condition is now fixed (a new xunit runner build in nuget)?

Seems a good idea to make the visual studio integration show a nicer error message than this, if for some reason, your test runner is missing or invalid, right?

[macgyver76]

"dotnet-test-xunit": "1.0.0-rc2-*" works for me as well.

[RehanSaeed]

Visual Studio Fix

Menu Bar -> Test -> Test Settings -> Default Processor Architecture -> X64

[joergjo]

What you are referring to updates the target platform in Visual Studio's .runsettings. These are not even supported by xUnit last time I've checked, and even if they were, you would have ended up with more or less the same error, since there was no x64 binary shipped as part of that particular xUnit version...

[kirillta]

Same on VS Code 1.4.0-insider, xunit 2.2.0-beta2-build3300, dotnet-test-xunit 2.2.0-preview2-build1029 and net461

[LeeChenelerJS]

@joergjo switching builds to x86 also resolved this for me.

[rclabo]

@RehanSaeed THANK YOU! A thousand times THANK YOU ! The Visual Studio setting you pointed out fixed got things working here!

[TheRealPiotrP]

Glad this got fixed :)

[thangchung]

@RehanSaeed You are my savior 👍

[andresdsep]

The Visual Studio settings fix didn't work for me, but upgrading the xunit packages did:

"dependencies": {
    "dotnet-test-xunit": "2.2.0-preview2-build1029",
    "xunit": "2.2.0-beta4-build3444"
},

[felschr]

After upgrading my .NET Core project which targets net461 to csproj I have this issue again when trying to run dotnet test in the command line it always runs the x86 build.

[DemoBytom]

I can confirm that after moving a project to VS17 and csproj I constantly get System.BadImageFormatException when running from the command line.
It works with no problem from Visual Studio 2017. Seems to be a problem with the dotnet-xunit CLI tool?

[RehanSaeed]

Did you try my fix? I actually googled this myself last week and found my own post from months earlier :).

Menu Bar -> Test -> Test Settings -> Default Processor Architecture -> X64

[DemoBytom]

Did you try my fix? I actually googled this myself last week and found my own post from months earlier :).

It works in Visual Studio for me. It doesn't work when I run it from cmd using dotnet xunit
Running dotnet test does work though, but I need xml output which I don't think I can set with dotnet test anymore.

[DemoBytom]

I did a little testing today. The problem is if you try to test an executable project/web app, referencing Microsoft.AspNetCore.Server.Kestrel (or any project that indirectly references that package, like a Microsoft.AspNetCore package for example)

Steps to reproduce:

Create ConsoleApp with this as a ConsoleApp1.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net461</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Server.Kestrel" Version="1.1.2" />
  </ItemGroup>
</Project>

Test class in the console app that will be tested:

namespace ConsoleApp1
{
    public class ConsoleClass
    {
        public int Method()
        {
            return 123;
        }
    }
}

Create xUnit test project with this as .csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net461</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="xunit" Version="2.3.0-beta3-build3705" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.0-beta3-build3705" />
    <DotNetCliToolReference Include="dotnet-xunit" Version="2.3.0-beta3-build3705" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\ConsoleApp1\ConsoleApp1.csproj" />
  </ItemGroup>  
</Project>

The test method class:

using ConsoleApp1;
using Xunit;

namespace XUnitTestProject1
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {
            var consoleTest = new ConsoleClass();
            var returned = consoleTest.Method();
            Assert.Equal(123, returned);
        }
    }
}

Expected behavior

running dotnet xunit causes test to pass.

Actual behavior

running dontet xunit causes the test to fail
dotnet xunit output:

Detecting target frameworks in XUnitTestProject1.csproj...
Building for framework net461...
  ConsoleApp1 -> d:\michal\Documents\Visual Studio 2017\Projects\CoreXUnitNet461\ConsoleApp1\bin\Debug\net461\ConsoleApp1.exe
  XUnitTestProject1 -> d:\michal\Documents\Visual Studio 2017\Projects\CoreXUnitNet461\XUnitTestProject1\bin\Debug\net461\XUnitTestProject1.dll
Running desktop CLR tests for framework net461...
xUnit.net Console Runner (64-bit Desktop .NET 4.0.30319.42000)
  Discovering: XUnitTestProject1
  Discovered:  XUnitTestProject1
  Starting:    XUnitTestProject1
    XUnitTestProject1.UnitTest1.Test1 [FAIL]
      System.BadImageFormatException : Nie można załadować pliku lub zestawu 'ConsoleApp1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' lub jednej z jego zależności. Próbowano załadować program w niepoprawnym formacie.
      Stack Trace:
           w XUnitTestProject1.UnitTest1.Test1()
  Finished:    XUnitTestProject1
=== TEST EXECUTION SUMMARY ===
   XUnitTestProject1  Total: 1, Errors: 0, Failed: 1, Skipped: 0, Time: 0,137s

Removing the Microsoft.AspNetCore.Server.Kestrel reference from the Console App causes the test to pass
Changing the project type from Console Application to Class Library, even with the Microsoft.AspNetCore.Server.Kestrel reference present, causes the test to pass
Running dotnet test causes the tests to pass as well

Environment Data

dotnet info output:

.NET Command Line Tools (1.0.4)

Product Information:
 Version:            1.0.4
 Commit SHA-1 hash:  af1e6684fd

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.4

Additional notes

Running tests in Visual Studio 2017 works properly every time, only the cli tools seem to be broken

I tried running tests with different versions of a dotnet-xunit package as well as different versions of Microsoft.AspNetCore.Server.Kestrel package. The results were always the same.

I made a simple test repo that has 5 projects, showcasing different project type/references combination:

• 3 executable projects, 1 without Kestrel reference
• 1 Class library project with Kestrel reference
• xUnit test project with 4 test methods

Running tests with dotnet xunit causes 2 out of 4 test methods to fail with System.BadImageFormatException

[DemoBytom]

The fix to the problem was changing projects to x64 or Any CPU as per this response.