Aspnetcore.docs: Integration Testing update to 3.0

[guardrex]

.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 77889d43-fedb-f8b4-8fea-379ec946555c
• Version Independent ID: f499b131-8b14-7e10-1728-543fdd5a5656
• Content: Integration tests in ASP.NET Core
• Content Source: aspnetcore/test/integration-tests.md
• Product: aspnet-core
• Technology: aspnetcore-test
• GitHub Login: @guardrex
• Microsoft Alias: riande

[SidShetye]

What's the status of this? 3.0 has production go ahead, so this needs an update ASAP.

[guardrex]

3.0 has production go ahead

3.0 isn't released ... it's in preview.

I'll get to this soon. I listed it tenatively on this month's (August) sprint, so I hope to get to it within the next two weeks. No guarantee on that, but I'll try to get to it soon.

[SidShetye]

Went by this announcement which says

NET Core 3.0 Preview 7 is supported by Microsoft and can be used in production

So the supported go-live is just for .NET Core 3.0 Preview 7+ and NOT for ASP.NET Core 3.0 Preview 7+ ?

[guardrex]

Perhaps so ... @danroth27 :point_up: :eye: Is .NET Core Preview supported for production and ASP.NET Core Preview not supported for production?

[danroth27]

ASP.NET Core is part of .NET Core, so yes, you can use ASP.NET Core 3.0 in production. However, there are some areas in .NET Core 3.0 that are excluded from this go-live recommendation due to anticipated churn: WPF, Windows Forms, Blazor and Entity Framework.

[guardrex]

Thanks @danroth27.

@SidShetye I'll get to this as soon as I can, but note that MS docs guidance is that we focus on the current release and work on documentation for previews in the latter stages of development, often close to or during the Release Candidate (RC) stage.

The practical reason for it is that there is a lot of engineering churn during early previews and it's not possible with our staffing to keep up with rapid preview changes in the framework.

No worries tho ... I put this on the August Sprint, and I think I'll be able to get to it next week.

[guardrex]

Working on this, but I need to get some feedback from engineering WRT using the in-memory dB with Identity. It used to work, and we've used the approach over releases going back to 1.0 IIRC. Might need to adopt the template-based approach with SQLite. I'll continue with this work after I hear back from engineering.

https://github.com/aspnet/AspNetCore/issues/13187

[guardrex]

@danroth27 @Rick-Anderson @tdykstra @wadepickett @scottaddie

We have an answer now, but it puts us in a bit of a tight spot: https://github.com/aspnet/EntityFrameworkCore/issues/16963

I'm not on nightly builds, but I could go for it and see what happens. 🎲🎲

How do you want to proceed with samples that use Identity and the in-memory dB?

• Try nightly builds ... hope that the :pray: server gods :pray: (ajcvickers :smile:) bless us with code soon and update the samples as soon as in-memory is working with Identity again.
• Wait until 3.0 RC and try again.
• Convert these apps over to using SQLite.
• Do something else?

[Rick-Anderson]

I'd wait until the next release and do a quick test. I prefer in-memory.

[stevo-knievo]

I’m sorry if this isn’t the right place for it.
I’m currently migrating my ASP.NET Core app from .NET Core 2.2 to Core 3.0 Preview 9.

Unfortunately I can’t get my integration tests working which use the customized WebApplicationFactory from the Integration tests in ASP.NET Core page to use the entity framework in memory database. Through debugging I noticed the test will execute the method ConfigureServices from the Startup class. In which the test will use the application DbContext.

I’m happy to provide a small demo app which demonstrates my problem.

Again I hope I’m posting this at the right spot.

Thanks in advance.

[guardrex]

@stevo-knievo It's a known issue, but I haven't had time to see what happens here with Preview 9. I'll get back to this as soon as I can. Hopefully, I'll have time later this week to see where things stand.

[stevo-knievo]

That would be amazing! Even a workaround would be great. It’s currently stopping me from finishing my migration to 3.0.

Thanks for getting back so quickly @guardrex! I much appreciate it.

[guardrex]

@SidShetye @stevo-knievo I think† that I have the sample app running. It's here ...

https://github.com/aspnet/AspNetCore.Docs/blob/guardrex/int-test-30/aspnetcore/test/integration-tests/samples/3.x/IntegrationTestsSample/

_I think_† = No warranties are expressed or implied. :smile:

Although it compiles, runs, and all of the tests pass, I can't guarantee that everything is perfectly in order. For example, I'm not sure if the seeding bits are working properly ... it's supposed to only seed one time with fresh data (no dups) for each test. I didn't check that, so I'm not sure if it's all good.

Also, I only have those sample app updates at this time. I haven't touched the topic ... haven't even looked at it yet.

I'll take a closer look at the sample app's testing behavior and get into fixing up the topic when I get back to the issue. It might be later this week. Depends on my workload ... priorities ... pings ... etc.

[stevo-knievo]

Thanks @guardrex!
I pulled your example and yes all the tests are passing. But I didn’t have time to really get into today. There is a lot going on and I think it would be much easier to have a real simple case to show the problem I’m facing so I pushed up a really simple app. Please have a quick look and play around with it.

https://github.com/stevo-knievo/IntegrationTestDotNetCore3.0

I much appreciate it. Thanks!

[guardrex]

You'll have to consult with devs on a support forum, such as Stack Overflow, or a support chat, such as Slack or Gitter. We just don't have the staffing to address problems in developer apps. There's only three of us here, and we maintain >300 topics with >100 samples and have >500 repo issues. 🏃 As far as product support goes, there's Microsoft Support, of course. If you find a bug in the framework, you can open it on the engineering repo at https://github.com/aspnet/AspNetCore/issues. Before you go any further tho, I recommend that you compare the working sample to your app and see what the differences are. You can probably find out what's wrong by comparing the code.

[stevo-knievo]

@guardrex fair enough and I appreciate you are taking the time. I forked your example. I added a SQLite DB provider for the application and I added a test that will count the messages on the index page. This test shows nicely that the integration tests aren't use the InMemory DB provider from the CustomWebApplicationFactory class.

Here are the changes I made:
https://github.com/aspnet/AspNetCore.Docs/compare/guardrex/int-test-30...stevo-knievo:guardrex/int-test-30

Here the link to my fork:
https://github.com/stevo-knievo/AspNetCore.Docs/tree/guardrex/int-test-30

[stevo-knievo]

@guardrex I merged all your changes into my fork. Same issues the integration test is using the application DB provider (SQLite) instead of the InMemory DB provider set up in the customized WebApplicationFactory class.

[guardrex]

I don't repro that here. I checked this earlier today when I was preparing the PR. The dB is seeded with the messages in RazorPagesProject.Tests.Utilities.GetSeedingMessages, which are prefixed with "TEST RECORD." The SUT's messages don't have the prefix (in RazorPagesProject.Data.ApplicationDbContext.GetSeedingMessages), so it was easy for me to see that the records came from the test app and not the SUT. In theory, SQLite in the SUT shouldn't matter ... in theory, of course. Stranger things have happened! 👽:smile: This seems like an engineering/product unit issue to me thus far, so .......

• Put up a repro project (not a fork of the docs repo) ... just a repo that contains your Integration Testing sample app ... just the contents of the folder under 3.0 named IntegrationTestsSample.
• Open an issue for engineering at https://github.com/aspnet/AspNetCore/issues with a link to your repro project.
• Tell them that you've made all of the updates on the 3.0 docs PR at https://github.com/aspnet/AspNetCore.Docs/pull/14265.
• Put the error message/behavior into the issue.
• Add a "cc: @guardrex" at the bottom ... I'll track on it and see what they say. If they respond quickly enough and there is something missing from the topic, we might be able to get it on that PR before it merges.

[stevo-knievo]

Thanks @guardrex for guiding me through the process. I open an issue. https://github.com/aspnet/AspNetCore/issues/13918

[SidShetye]

I guess we should be happy the initial lack of documentation pushed us down our path. Our integration tests do NOT use an in-memory database. We use “real” sql connection string pointed to a sql docker instance or local db or sql dev edition (depends on machine/config). More reliable, conditions closer to production and none of the quirks.

Would highly recommend that. EF core allows rapid database scaffolding too.