Aspnetcore.docs: Document improved support for in memory end to end testing of MVC apps

[javiercn]

We need to cover:

• Setting up a project for testing an MVC application in memory using TestServer and the new Microsoft.AspNetCore.Mvc.Testing package.
• How to customize the IWebHostBuilder instance from the app for testing.
  
  
  
  • How to override configured services on the application for testing. (e.g the database).
  
  
• How to configure where to find views and static files.

[Rick-Anderson]

Functional testing infrastructure
@javiercn would you be able to write the sample code?

[guardrex]

@javiercn I'll get this done. Don't sweat the sample code. I'll take care of it. I'll ping u on the PR.

[guardrex]

@Rick-Anderson

This is a document content swap ... a rewrite/overhaul of the current topic. I'd like to avoid a redirect file entry and two versions of the integration testing topic, if possible. Given that the current integration testing topic is soooo MVC focused (well, 100% MVC really), can I rewrite the whole topic in-place for 2.1 (with the preview notice, of course) focused on RP (but with MVC, too)? Then, I won't need to do a topic swap-a-roo later with a redirect. The only thing I'll need to do when 2.1 RTMs is kill the integration testing section in the RP testing topic.

[Rick-Anderson]

@guardrex sounds good to me. @scottaddie does that sound OK?

[Rick-Anderson]

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/secure-data?view=aspnetcore-2.1
See this PDF file for the ASP.NET Core MVC version.

If that PDF is OK, use it, otherwise create another date stamped PDF.

[guardrex]

Yes, that's great. This topic hasn't been touched since then except for metadata and a heading update.

[scottaddie]

Now that we have conceptual versioning, I recommend preserving the MVC content for folks using bits before 2.0. With this approach, we wouldn't need a PDF.

[Rick-Anderson]

Is the MVC content worth maintaining? Can't they apply the RP content pretty easily to MVC controllers?

If it's not much work, then use versioning.

[scottaddie]

My opinion is that the MVC content is worth maintaining. We recommend Razor Pages for new development, but there are still far more MVC users out there.

[Rick-Anderson]

OK, lets go with that - unless @guardrex can make a compelling argument.

[guardrex]

I can't use conceptual versioning with such an extensive overhaul/rewrite as this without putting large chunks of content into moniker tag sets. When I say "large," I mean just about all of it. It will still cover MVC, but it will be MVC using the 2.1 testing infrastructure.

If not outright cut in several spots, content will be heavily modified. I don't like this sample app; besides, the sample should be RP and closely aligned with the RP sample in the RP testing topic. Dan said to prioritize the RP coverage over the MVC coverage.

Also, we're not making 1.x, 2.0, and 2.1 samples. It's 1.x and 2.x, unless it's decided to go a different way here.

Moniker tags don't nest, correct? That's going to end badly at 2.2, 3.0, or .NET-future with a possible desire to split content within 2.1+ content (e.g., 2.1 vs. 2.2 (or 3.0) within 2.1+ because there's also 2.0 tagged content).

To preserve this topic and its content, then I suggest one of these options:

• Use the PDF to cover <=2.0 and rewrite it for >=2.1. Either launch the 2.1 version/PDF now or sit on the PR until 2.1 RTM.
• Go the parallel topic approach: Keep this. Add a 2nd topic for 2.1+. When 2.0 is depreciated a few years down the road, drop this topic.

My suggestion: The first one ... PDF.

btw - Can someone email me the CSAT and page views?

[Rick-Anderson]

I agree, monikers won't work. Why not keep the doc and create a new one for RP. The only problem is do we really want the maintenance?

I'll try to get the stats.

[Rick-Anderson]

Unless you did the entire doc in a moniker and the new doc below that.

[guardrex]

Why not keep the doc and create a new one for RP.

Dan said 'no' to that. He wants one integration testing topic that covers both RP (priority) and MVC.

Unless you did the entire doc in a moniker and the new doc below that.

Yeah, but that's where the nesting is going to :boom: with the next release (or 2.3 or 3.0) if we need to split content within monikers ... won't work with their current bits.

[guardrex]

PDF and full rewrite is the simplest, easiest way to go. We can sit on the PR until 2.1 lands if you want.

[Rick-Anderson]

Why won't the moniker < 2.1 work for the MVC content?

The new doc will be >= 2.1 and probably never need to change.

[guardrex]

The problem might just be training. I don't have access to the internal documents that explain conceptual versioning, and I wasn't in meetings where it was discussed/explained. Therefore, I might have significant knowledge gaps that prevent me from understanding how best to implement it in a case like this.

However, everything I listed is probably* true:

• Total topic overhaul/rewrite
• Topic will have MVC coverage, but it will focus and prioritize RP
• Different testing infrastructure than 2.0 bits
• 2.0 sample has to go in favor of a 2.1 replacement (only doing 1.x and 2.x ... not 1.x, 2.0, and 2.1)
• Sample will be RP (MVC will be code examples)
• Sample will be totally different than this one (it will be a lot like the RP testing topic sample)

probably* = I'll know more when I get into the writing.

Can conceptual versioning load in an entirely different document body (i.e., from two different files)?

[scottaddie]

@guardrex There are 3 types of conceptual versioning:

• Version by Folder - At the top level of your docset, you can use moniker ranges to set up folders representing content sets whose content is NOT shared across versions. This is primarily targeted at scenarios involving migration of legacy content.
• Version by File - Within a top-level versioned folder, you can use file metadata to associate specific files with a moniker range. This is primarily targeted at the scenario of documenting new product features and/or deprecating old product features.
• Version by File Zone - Within a file, you can use a Markdown extension to associate a "zone" of content with a moniker range. This is primarily targeted at extending existing topics for product features that have changed in incremental ways across versions.

Our repo is only configured to use the Version by File Zone strategy. If we have a strong enough need for it, we can configure the repo to enable other strategies too. They can be mixed together.

[guardrex]

I see. Thanks for explaining it out. I'm cool on Version by File Zone. Used it ... works great outside of having to duplicate entire tables (or lists until they get that fixed).

Version by File seems like it would be useful in total topic overhaul/rewrite scenarios. I'd like to just leave this current topic exactly as it is (perhaps making the sample snippets inline code) and then writing a whole new replacement topic that loads for 2.1+.

I'll try to make Version by File Zone work as best I can. I have a feeling that the 2.0 MVC coverage will be Just the facts, Sir!

[scottaddie]

@guardrex You can make that replacement doc appear only for 2.1+ by using the following metadata:

monikerRange: '>= aspnetcore-2.1'

[guardrex]

@scottaddie How does the file naming/TOC work in that case?

[scottaddie]

@guardrex The single piece of metadata I provided above takes care of toggling TOC visibility automatically. This is how we handle the SignalR docs.

[guardrex]

I see. Cool.

@Rick-Anderson I think we can cancel the meeting tomorrow. I simply didn't understand conceptual versioning.

[guardrex]

@scottaddie Sorry ... let me check just one more little thing on this.

I can't discern how it works exactly by looking at SignalR topics (/aspnetcore/signalr) because I don't see pairs of matching topics providing alternatively-versioned content.

I guess it works based on the matching UIDs. Let me give you my plan, and then you can say where I'm going right/wrong.

Today, we have this ...

razor-pages-testing
troubleshoot/_static
index.md 
integration-testing.md
razor-pages-testing.md
troubleshoot.md

The metadata for integration-testing.md is ...

title: Integration tests in ASP.NET Core
author: ardalis
description: How to use ASP.NET Core integration ...
manager: wpickett
ms.author: riande
ms.date: 09/25/2017
ms.prod: asp.net-core
ms.technology: aspnet
ms.topic: article
uid: testing/integration-testing

I'd like to add integration-testing-2-1.md here with this metadata ...

title: Integration tests in ASP.NET Core
author: guardrex
description: How to use ASP.NET Core integration ...
manager: wpickett
monikerRange: '>= aspnetcore-2.1'
ms.author: riande
ms.date: DATE
ms.prod: asp.net-core
ms.technology: aspnet
ms.topic: article
uid: testing/integration-testing

... and then :sparkles: _The Magic Sauce_:tm: :sparkles: will take care of the rest. Is that right?

[scottaddie]

@guardrex Your understanding is correct. The SignalR docs only apply to 2.1+, hence the reason why monikerRange: '>= aspnetcore-2.1' is applied to those docs. If you'd like integration-testing.md to only display for versions prior to 2.1, then you'll need to add the following metadata to that doc: monikerRange: '<= aspnetcore-2.0'. The versioning system handles toggling TOC visiblity based on the selected version from here.

[guardrex]

Gotcha . . thanks. I'm good. :+1: This should go well.

[guardrex]

@javiercn Three questions ...

Question 1: Host builder signature very specfic

It seems like it's very specific about the host builder signature in the SUT.

Seems like I can't use ...

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseStartup<Startup>()
        .Build();

... but instead need to use this ...

public static IWebHostBuilder CreateWebHostBuilder(string [] args)
{
    var hostBuilder = WebHost.CreateDefaultBuilder(args);

    hostBuilder.UseStartup<Startup>();

    return hostBuilder;
}

public static IWebHost BuildWebHost(string[] args)
{
    var hostBuilder =  CreateWebHostBuilder(args);

    return hostBuilder.Build();
}

... yielding the method sig of public static IWebHostBuilder CreateWebHostBuilder(string [] args) to get the SUT initialized. Or, do you prefer I go along the lines of ...

You can configure the WebHostBuilder yourself by overriding CreateWebHostBuilder on WebApplicationTestFixture<TStartup>.

... which might be preferable because changing the SUT's BuildWebHost method doesn't seem appealing.

Question 2: Compilation context failure?

Nevermind! I see it ... https://github.com/aspnet/Mvc/issues/7615. For now, I'm going to just explicitly list the packages out. At 2.1, I must return anyway to update versions ... I'll go back to the App package at that time.

Question 3: Can the dev still supply a relative path to the SUT

How to configure where to find views and static files.

Looks like there were some changes (or it was dropped) for passing a relative SUT path to the factory/fixture. What's the current way do it, if there is one?

Question 4: Disable shadow copy automatically for the dev?

Could ...

<ItemGroup>
  <Content Include="xunit.runner.json" CopyToOutputDirectory="Always" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>

... be added to the Microsoft.AspNetCore.Mvc.Testing.targets with the xunit.runner.json file supplied?

{
  "shadowCopy": false
}

Doesn't affect the topic here (I'll include it) ... I'm just curious if there was a reason why it wasn't done.

[guardrex]

@javiercn Do u have a minute to answer those questions :point_up:? Question 2 isn't really a problem, and the last question is just out of curosity and doesn't affect the topic. Therefore, it's just Questions 1 and 3 that I need help with.

[javiercn]

@guardrex Some answers:
1) CreateWebHostBuilder is the new recommended pattern. You can remove BuildWebHost all together (inline it)

2) It's a preview2 issue.

3) By default you don't need it, as we do some magic with MSBuild + assembly attributes to figure the content root path automatically for you in 99.99% of the cases. For the 00.01% remaining you can just override CreateWebHostBuilder and chain call .UseSolutionRelativeContentRoot() or do it in ConfigureWebHostBuilder.

4) This package is not xUnit specific, hence it should not contain xUnit opinions.

[guardrex]

ty @javiercn

1. Ah ... gotcha. We spend so much time documenting the current release that we don't frequently have good knowledge of what's next until it smacks us in the head.

2. Cool.

3. Cool.

4. Yes, silly of me to suggest it for that package. There could be a Microsoft.AspNetCore.Mvc.Testing.Xunit package with the bits (or a community-built flavor along those lines). Then, also a Microsoft.AspNetCore.Mvc.Testing.Nunit package for those folks. Anyway ... no worries. Since our samples adopt xUnit, the topic will cover it. We may need to strengthen the language regarding the use of other test frameworks. Let's take a look at that on review.

I think have what I need to finish the 1st draft. I should have it in no later than tomorrow mid-day.