Aspnetcore: WebApplicationFactory - Exception not being caught by HttpClient

[D3niz]

Describe the bug

When using WebApplicationFactory, and using CreateHostBuilder to set it up, Then in a test HttpClient from _factory.CreateClient() is used to call an endpoint and that endpoint method has an exception within its code and the HttpClient should catch that exception and a bad request StatusCode is expected, what is happening though is the exception is bubbling up to the test and the test fails from whatever the exception is during the endpoint method and no response is ever returned from the client.

This is only the case when using CreateHostBuilder(), if using something like ConfigureWebHost() then it works fine and a bad request response from the client is returned as expected.

To Reproduce

A test base class with

private class WeatherForecastApplicationFactory : WebApplicationFactory<Startup>
{
    protected override IHostBuilder CreateHostBuilder() =>
        Host.CreateDefaultBuilder()
            .ConfigureWebHost(w => w.UseStartup<Startup>());
}

Then in a unit test
var response = await Client.SendAsync(new HttpRequestMessage(HttpMethod.Get, "/api/test"));

And then the method that it is calling, just throw an exception
e.g.

[HttpGet]
public IActionResult Get()
{
    throw new Exception();
}

Further technical details

netcoreapp3.1
Microsoft.AspNetCore.Mvc.Testing: 3.15.1
nunit: 3.12.0
NUnit3TestAdapter: 3.15.1
Microsoft.NET.Test.Sdk: 16.4.0

Microsoft Visual Studio Professional 2019
Version 16.4.5
ReSharper Ultimate 2019.3.1

[D3niz]

Example
WebAppFacTesting.zip

[javiercn]

@D3niz thanks for contacting us.

We'll look into this and get back to you.

[pwen090]

Just wanted to echo that we had the same issue and this behavior was unexpected. We were implementing unit tests to generically validate that our asp.net core app was giving detailed exception errors in Developer mode and not in Production. We were surprised that SendAsync returned directly the Exception generated by our asp.net core web api and not the 500 error response that you would get if you tested the same scenario via chrome/curl/network... Wondering now what are all the scenarios that the factory will give different responses than what you would actually get via "real world" integration testing against a live API set?

[estephinson]

The more and more I look into this, I get the feeling it's intended behaviour. The exception jumps from the server thread to the client and presents itself, looking at the stack traces I'm seeing right now, it can only be intentional.

[estephinson]

For those poor souls who have spent their evening looking at a non-issue like me, if you're expected a 500 to display from your test client rather than the exception be thrown in the test run, you need the following line in your Configure() method in Startup.cs:

app.UseDeveloperExceptionPage();

Otherwise, it's pretty cool to have the stack trace accessible straight from the test client, go team

[D3niz]

That does work, good stuff.
Though doesn't seem like just because it works its the right thing.
Being that you are using a webapplicationfactory for testing it would be good to have a setup in that class with a more meaningful name to what it behaviour is and not have to change code within your main app (in this case startup) to make tests work.
Also to add to that having a different default behaviour on 2 different ways to setup is a little annoying e.g. CreateHostBuilder() or ConfigureWebHost().

[estephinson]

Yeah I hate adding test-specific code to the main project, luckily for us it's what we wanted to happen in our dev environment anyway, so we have it guarded by an environment check and it just happens to do what we want in the test world so I consider it acceptable in our case.

In your case, i'd use the following method in your ConfigureWebHost method under your custom web application factory
c# builder.Configure(app => { app.UseDeveloperExceptionPage(); }

Or, if you're using TestServer, you can call the Configure method on that chain

[msftbot[bot]]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.