Aspnetcore.docs: Need documentation on how to set environment during integration test

[davisnw]

The documentation only says what the default environment is. It should tell us how to change the environment for integration tests.

For example, in a single page app, a common setup is:
~~
if (env.IsDevelopment())
{
spa.UseAngularCliServer(npmScript: "start");
}
~~

For my integration test, I don't want to start npm - it's unnecessary for integration testing my api endpoints, and it causes errors without additional setup, and even if npm start works, it slows things down unnecessarily when just testing endpoints.

So an obvious solution would be to change the environment in integration testing to e.g. "IntegrationTest" - but the documentation doesn't mention how to accomplish that.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 77889d43-fedb-f8b4-8fea-379ec946555c
• Version Independent ID: f499b131-8b14-7e10-1728-543fdd5a5656
• Content: Integration tests in ASP.NET Core
• Content Source: aspnetcore/test/integration-tests.md
• Product: aspnet-core
• Technology: aspnetcore-test
• GitHub Login: @guardrex
• Microsoft Alias: riande

[guardrex]

Updates for config by environment were rejected on https://github.com/aspnet/Docs/issues/8917 and https://github.com/aspnet/Docs/pull/9060. You can look at the PR that I was pitching for what I proposed tho.

[guardrex]

Oh .... :point_up: look at the 1st, 2nd, and 3rd commits on #9060 ... not where we left the discussion. The early commits show the approaches that I was floating.

[davisnw]

Why not just update the documentation for the environment section to mention using WebHostBuilder? That's what I eventually figured out after looking at: https://github.com/aspnet/Mvc/issues/7976#issuecomment-400656032

Though for my purposes, I just hard-coded the environment, rather than using an environment variable.

[guardrex]

Yes, env var was one that I listed in those early commits on the PR.

I like this one from that PR ...

• Supply the environment for tests in code:

  • Using a CustomWebApplicationFactory, call UseEnvironment on the IWebHostBuilder:

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
      // The SUT consumes configuration from appsettings.staging.json or 
      // `if (_env.IsStaging())`/`if (Environment.IsStaging())` code 
      // in `Startup`
      builder.UseEnvironment("Staging")
      ...
    

  • When calling WithWebHostBuilder, call UseSetting on the Environment key:

    var client = _factory.WithWebHostBuilder(builder =>
    {
      // The SUT consumes configuration from appsettings.staging.json or 
      // `if (_env.IsStaging())`/`if (Environment.IsStaging())` code 
      // in `Startup`
      builder.UseSetting("Environment", "Staging");
      ...
    

IIRC That keeps the setting in the test project ... nothing to sweat on the SUT side.

Several devs have asked about this. I'll label it for the product unit ("PU") and leave it for triage on the Integration Testing project. IIRC, I didn't fully understand what Javier was telling me to do on that PR. I might have been slammed tho and not paying close enough attention. 🏃🏃🏃🏃😅

[EDIT] Now that I think of it, I think what he was saying was to directly override (e.g., services) from the test project. We may have been talking past each other a bit. What I proposed was more like the scenario where one just needs different config settings, for example. It would be easy enough to load into the SUT if only it knew what its environment was ... say ... Staging. After that, the app can configure itself however it likes. I think he was saying directly override things in the SUT ... a bit heavy-handed perhaps in some scenarios, but I think I get it now.

[AnthonyMastrean]

I'm trying to setup ASP.NET Core integrated tests for my "webapi" right now and I'm trying to stay "by the book". Except, I need _different_ configuration when I run integrated tests on my local machine vs. on my CI server.

On my local machine, I want to stick with the Development environment and configuration. For example, I'm running services via Docker and published ports at "localhost". But, on my CI server, I have to run those as remote services, like "rabbitmq" or "postgres" (not "localhost").

If the WebApplicationFactory just respected the normal environment selection and configuration system, it would work as normal when I changed ASPNETCORE_ENVIRONMENT.

As is, I can use @guardrex's solution, but I need to _reinvent_ the logic!

var env = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
if (!string.IsNullOrWhiteSpace(env))
{
    builder.UseEnvironment(env);
}

// else, Development is fine

[mkArtakMSFT]

We will try to handle this post 3.0 preview 9 CC date.

[guardrex]

@mkArtakMSFT To confirm our plan for any of the integration testing-related issues that appear: They've been added the Integration Test project for triage ...

https://github.com/aspnet/AspNetCore.Docs/projects/31

I don't work them until/if assigned. I only work a real doc 🐞 bug report 🐞 for the topic. For example, I worked this one ... https://github.com/aspnet/AspNetCore.Docs/pull/13058.

[mkArtakMSFT]

@guardrex an engineer will be handling this. We will create the PR and would then ping you for review. I hope this works.

[guardrex]

Yes ... that's what I was told ... i.e., don't work them unless it's a doc bug requiring immediate attention. So far, so good.

[javiercn]

Integration tests setup Development unconditionally as in https://github.com/aspnet/AspNetCore/blob/master/src/Mvc/Mvc.Testing/src/WebApplicationFactory.cs#L321

In order to support environment variables for multiple environments the way to achieve that is to re-register the environment variables config provider in your custom waf

        protected override IHostBuilder CreateHostBuilder() => 
            base.CreateHostBuilder()
                .ConfigureHostConfiguration(cb => cb.AddEnvironmentVariables("ASPNETCORE"));

@guardrex can we add this to the doc?

[guardrex]

Sure. I'm 🏃😅 for about the next two weeks with 3.0 priority issues, so I'll add this to the October sprint.

[javiercn]

@guardrex No problem.

[javiercn]

@mkArtakMSFT my part in this issue is handled.

[triple-vee]

I realize this is Closed, but I thought I'd add my 2 cents. Yes, WAF unconditionally sets ASPNETCORE_ENVIRONMENT to Development as javiercn notes with the link to the code. However javiercn, proposed override does not seem to have any effect. I am using 3.1 and WAF.