Aspnetcore.docs: WebApplicationFactory with MSTest?

[ghost]

The above tutorial uses XUnit, but we're using MSTest in Visual Studio. Is there a similar tutorial on how to use WebApplicationFactory with MSTest?

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 77889d43-fedb-f8b4-8fea-379ec946555c
• Version Independent ID: f499b131-8b14-7e10-1728-543fdd5a5656
• Content: Integration tests in ASP.NET Core
• Content Source: aspnetcore/test/integration-tests.md
• Product: aspnet-core
• GitHub Login: @guardrex
• Microsoft Alias: riande

[guardrex]

Hello @darrentraffk ... There's no tutorial; but according to the engineer who wrote the framework code for testing, there's no significant difference between setting up the tests with MSTest versus xUnit. Both test frameworks should work with WebApplicationFactory. I didn't try MSTest when writing the topic, so I can't say myself if there are any "gotchas" along the way.

If you run into trouble setting up testing with MSTest, open an issue on the aspnet/Home repo issues or Stack Overflow. If you find that the docs are missing critical information, please report back here.

[asadsahi]

@guardrex usage of NUnit is more than xUnit and MsTest I believe. So a tutorial of integration tests with NUnit will really help teams kick start with NUnit. Its good that we have a tutorial of NUnit Unit tests but I haven't managed to get NUnit working for integration test.

[guardrex]

I haven't tried to use NUnit and MSTest. I'm not opposed to building additional sample apps for the topic that use them.

@javiercn Can u comment? Any known issues? Do you think we should have sample apps based on the other test frameworks?

[javiercn]

I don't think there's any issue that I know of other than having to instantiate the web application factory yourself. (In xUnit you can use a fixture)

[guardrex]

I'm not opposed to providing samples for both ... the problem is time. We can open an issue (use this issue, actually) to add them to the topic, but I can't say if the work would ever be done short of the community providing the samples.

[SWarnberg]

Yes, it would be great to get a direction on how to do this with MSTest. I'm struggling with it too.

[wadebee]

My concern is with disposal. I am looking at switching from my current usage of the TestServer class (which I understand is used within WebApplicationFactory).

I currently use the following pattern:

using(var server = new TestServer(MyHostBuilder())) 
{
     using (var client = server.CreateClient())
     {
         // add'l code here
     }
}

I see that WebApplicationFactory implements the Dispose method so am I to assume I should use the same pattern? In all the xUnit code samples you don't see any manual disposal code. Is this because of some "magic" happening in the xUnit implementation of IClassFixture

[guardrex]

@wadebee We recently added a bit to the topic on that point. See :point_right: https://docs.microsoft.com/aspnet/core/test/integration-tests#disposal-of-objects

[wadebee]

@guardrex The documentation addresses the XUnit implementation. The point of this thread is making use of WebApplicationFactory from MSTest (and therefore not having/needing the xUnit IClassFixture interface mentioned in your documentation link)

After the tests of the IClassFixture implementation are executed, TestServer and HttpClient are disposed when xUnit disposes of the WebApplicationFactory.

[guardrex]

Yes, but Javier said ...

instantiate the web application factory yourself

... so the principal is the same for MSTest, no? Can you handle your object disposal in your custom factory?

I see that WebApplicationFactory implements the Dispose method so am I to assume I should use the same pattern?

Sounds good. Can you post back if you have success? Idk if/when this issue will be worked, but it would be great to have your feedback here.

[wadebee]

This does not appear to work at all within MSTest. The test fails with:

Exception thrown: 'System.InvalidOperationException' in Microsoft.AspNetCore.Mvc.Testing.dll

when CreateClient() is called. Also when I set a breakpoint on the offending line and view the factory variable from the Locals window something is very wrong: factory expands to 5 members however name, value and type for each member is completely blank. I haven't ever seen that in my debugger before.

Here is my code..

        [TestMethod]
        public async Task RootView()
        {
            using (var factory = new TestFactory())
            {
                using (var client = factory.CreateClient())
                {
                }
            }
         }

TestFactory is simple subclass like this:
public class TestFactory : WebApplicationFactory<Startup> {}

[guardrex]

ping @javiercn

... and if this all doesn't end in tears :cry:, would you like me to prioritize a MSTest section for the topic (or a full SxS topic with MSTest)?

[willj]

I too have been struggling to get this working with MSTest today, however I've now managed to make it work.

Here's a link to a repo with a single basic test in both MStest and xUnit
https://github.com/willj/aspnet-core-mstest-integration-sample

I believe the issue I was having was related to the fact I was trying to do this on an older project, which has BuildWebHost instead of CreateWebHostBuilder in Program.cs (Which throws the InvalidOperationException @wadebee reports)

Additionally, make sure you add references to both Microsoft.AspNetCore.App and Microsoft.AspNetCore.Mvc.Testing in the MSTest project.

[guardrex]

Thanks for posting that @willj ... I'll get this scheduled for work in Q1-2019 if Javier would like me to pursue it. I think an exact copy of the current sample app that uses MSTest makes sense. We wouldn't need to expand the text much in that case. I can add a topic section that calls out the parallel sample, provide setup/config instructions, and list any caveats.

I appreciate it if you'll leave that GH sample online for me to :eyes: when the time comes. If you think you will pull it down, let me know here, and I'll look sooner.

[willj]

No problem, I don't delete anything so that will remain online 👍

In reality, I think as long as you File -> New Project this just works as expected, however I think it would be useful to have the sample and some notes as I couldn't find anything about this in my searches yesterday.

If I get chance over the next couple of weeks, would it be helpful for me to have a go at this?

[guardrex]

Sure ... that would be awesome. There are updates to the topic+sample in review on https://github.com/aspnet/Docs/pull/9887, which include taking the sample app to 2.2. After that merges, this issue can go forward.

You can provide the new sample or provide the sample and the updates necessary to host the changes in the topic.

@scottaddie Can we get permission to have tabs for the two test frameworks here?

# [xUnit](#tab/xunit)

XXX

# [MSTest](#tab/mstest)

XXX

---

If we don't, we'll have either of two undesirable scenarios:

• The topic will only show xUnit, and the MSTest will just be a tack-on sample.
• Tons of duplication by having two topics ... one for xUnit, one for MSTest.

[scottaddie]

@guardrex I'm working with the team to get approval for "xUnit", "MSTest", and "NUnit" tabs. This could be a slow-moving request due to the holidays. Proceed with the assumption that we have approval.

[scottaddie]

@guardrex These tabs are approved. The syntax will be:

• [MSTest](#tab/mstest)
• [NUnit](#tab/nunit)
• [xUnit](#tab/xunit)

You'll see build warnings until the tabs whitelist is updated.

[voroninp]

@willj interesting enough Microsoft.NET.Sdk.Web is not referenced in test project, though everything works. However this doc clearly states:

The Web SDK is required when referencing the Microsoft.AspNetCore.App metapackage.

[javiercn]

This was due to some issue we had during the 2.1 release that I believe got fixed in 2.2, but I don't remember the exact details

[Rick-Anderson]

Thanks for contacting us.
We don’t have the resources to invest in this area, so we are closing the issue. Should your request generate enough 👍 responses, we’ll reconsider.