Mvc: Productize and harden our functional testing infrastructure

[rynowak]

This is too hard for our users to get right and we currently have many copies of this. We should make this into a shipped package. This is not just the TestHost, it needs to include all of the MVC-specific things that we do.

This needs to include validation that some of the common problems are addressed:

• Shadow copy off
  
  
  
  • Look for missing deps files
  
  
  • Set working directory correctly
  
  
  • Prevent assembly crosstalk (through ApplicationPartManager)
  
  

And new stuff:

• Get a RequestDelegate or HttpClient

[Eilon]

See https://github.com/aspnet/Mvc/issues/5793 for other pointers, discussions, and scenarios.

[rynowak]

@javiercn - has started peeling the onion on this btw as part of his testing effort for identityservice

The strategy we were going with was to check in an MVP as a separate project and then file issues for all of the blockers we find trying to use his package in all of the places where we have a copy/fork of this code today.

[ferrx]

Any ETA on an MVP or updates? With regard to aspnet/testing#276, I'm not sure how to approach implementing TDD within an application that performs HasClaims checks on just about every HttpPost controller action.

[rynowak]

@javiercn any updates here?

[javiercn]

I'm working on other things at the moment, but I'll be able to come back to this sometime this week.

[Eilon]

Leaving this open because we won't officially have this done until 2.1.0.

[drauch]

Is this going to be fixed for 1.1.x as well? We are currently having the same hacks in place in order to have web tests with an in-process kestrel server.

[Eilon]

@drauch - this is a new feature we're building for the 2.1 release, so there isn't really anything to fix for 1.x because this feature wasn't there.

[drauch]

@Eilon : Maybe I'm mistaken here, but hosting Kestrel in-process to do integration testing doesn't seem to be a "new" feature. We do web testing this way using ASP.NET Core 1.1 and ran into various problems all described in the initial post here. We would very much appreciate it, to not have to turn off shadow copying, create the deps file with a special target in the build file, etc. Would be great if this thingy (i.e., whatever mitigates these problems) would be available for 1.1 too.

[Eilon]

@drauch - ah, well, the way we're fixing this is with new features we're adding in the 2.1 release, so we're not planning on back-porting this to the 1.x releases. Perhaps you can take what we've done so far in the 'dev' branch (or even the 'rel/2.0.0' branch) and see if that addresses the issues you have in 1.x?

[javiercn]

We need an extensibility point to configure the WebHostBuilder like real applications do. See: https://github.com/aspnet/MetaPackages/blob/396f413b389f5983e413be465f917cbfdbda26c9/src/Microsoft.AspNetCore/WebHost.cs#L152

[dustinmoris]

Hi, I got here from https://github.com/aspnet/Razor/issues/1212 and see this issue is closed but I am still experienced the original problem. Has this been tested before closing the issue?

[javiercn]

@dustinmoris I’ve perused through that issue and seems like it’s an issue with razor missing references to assemblies.

I put together a blog post a while ago on how to do this.
https://blogs.msdn.microsoft.com/webdev/2017/12/07/testing-asp-net-core-mvc-web-apps-in-memory/ can you try the instructions in the blog post (and related sample) and let me know if you still have issues?

[dustinmoris]

@javiercn Thanks that works, but that feels extremely complex for a simple unit test project for a razor MVC app. Hope this can be simplified one day...

[javiercn]

@dustinmoris This will actually be simplified in the next release. See https://github.com/aspnet/Announcements/issues/275 for details.

[drauch]

Shadow copying will still be required to be turned off? That's a bummer : (

[rynowak]

Shadow copying will still be required to be turned off? That's a bummer : (

MVC needs to be able to find the view files on disk to be able to do compilation. This is also relevant if you have any other file content that you are serving. If you don't need to do any of these things you should be OK to use shadow copying.

If you do need these things, let's talk, I'd love to understand what you think we could do to improve this.

[drauch]

View files and content is found by setting the content directory accordingly, that's not an inherent problem with shadow copying turned on. However, it is important that the compilation-time assembly references for Razor compilation are properly shadow copied. I hope this second issue is resolved by the infrastructure, at the moment, we have to do the copying ourselves.

We need it primarily to run all our web tests in the background.

[javiercn]

@drauch are you doing this already on your tests? If so, how are you setting up the content root?

[drauch]

We are simply setting the content root to the web app's project directory within the solution. We know that this may interfere with the tests, but it is better than the other way: always copying the static content and views to the shadow copy location.