Sdk: Improve testing of PreserveCompilationContext with approximation of what Razor does for runtime compilation

[nguerrera]

Follow-up to offline conversation around https://github.com/dotnet/cli/issues/10502#issuecomment-453690369

Historically, we have broken this scenario too frequently.

We have identified two things:

1. Code flow delays have been too long and we end up only trying end-to-end scenarios too close to release dates.

2. It would be nice to have a unit test in dotnet/sdk that can catch as many related issues as possible immediately.

As of today, we finally have all involved repos inserting up to core-sdk automatically, so (1) will hopefully get better soon. 🙏

This issue tracks (2). The idea we sketched out was to have a test that builds and runs a console app, which:

1. Has a P2P reference
2. References shared framework(s)
3. Has a snippet of code that substantially replicates how razor enumerates compilation references at runtime
4. Also enumerates runtime assemblies from deps file along the same lines

Let me know if I missed something here, and please help to define the snippets of code for (3) and (4)

@rynowak @pranavkm @mkArtakMSFT

[rynowak]

(3) https://github.com/aspnet/AspNetCore/blob/321327f84b2b22dcff2e9beb06a9a64236c5cced/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/ApplicationParts/AssemblyPart.cs#L52

I think this is simple enough... A good test for this would require the dependency context to be non-null, and would verify that all of the files returned from ResolveReferencePaths exist in the filesystem

@pranavkm - can you help out by providing a good snippet of test code (based on MVC) that tries to load the whole runtime graph? You know more about this than me 😁

[pranavkm]

It's a little more than a snippet, but I added a console app that emulates the two code paths that MVC uses DependencyContext today - https://github.com/pranavkm/compilation-context/blob/master/CompilationContextTest/Program.cs.

• The primary path is when MVC uses DependencyContext to determine what libraries reference MVC. We do this as a startup optimization to avoid loading assemblies that certainly will not contain MVC specific items such as controllers. Having MVC be in the shared runtime adds some quirks to this that wasn't common knowledge. I was planning on setting some time aside to talk about it before 3.0 ships.

• As of 3.0, runtime compilation isn't a feature that is delivered out of the box (it's a separate package reference). However, this is by far been the most common cause of bug reports.

An alternative to running the test app that I produced is to straight up run a shipped release of MVC. But it does add additional constraints such as having the shared runtime installed, which might be more work than necessary.

[nguerrera]

Noting that the determination of who references mvc is now done in the build without a dependency on PreserveCompilationContext/deps file, so the test needed is only https://github.com/dotnet/sdk/issues/2828#issuecomment-454608968

[rynowak]

Well that's true for 3.0. Earlier versions of MVC still use the DependencyContext and will continue to do so in tandem with newer SDK releases.

[nguerrera]

Ah, true.