Aspnetcore: Improve automated browser testing with real server

[danroth27]

Feedback from @shanselman:

We should do better with testing. There's issues with moving from the inside out:

LEVELS OF TESTING

• GOOD - Unit Testing - Make a PageModel and call On Get
• GOOD - Functional Testing - Make a WebApplicationFactory and make in-memory HTTP calls
• BAD - Automated Browser Testing with Real Server - can't easily use Selenium or call a real server. I shouldn't have to do this. We should decouple the WebApplicationFactory from the concrete TestServer implementation. @davidfowl

public class RealServerFactory<TStartup> : WebApplicationFactory<Startup> where TStartup : class
{
    IWebHost _host;
    public string RootUri { get; set; }
    public RealServerFactory()
    {
        ClientOptions.BaseAddress = new Uri("https://localhost");
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseEnvironment("Development"); //will be default in RC1
    }

    protected override TestServer CreateServer(IWebHostBuilder builder)
    {
        //Real TCP port
        _host = builder.Build();
        _host.Start();
        RootUri = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.FirstOrDefault();

        //Fake Server we won't use...sad!
        return new TestServer(new WebHostBuilder().UseStartup<TStartup>());
    }

    protected override void Dispose(bool disposing) 
    {
         base.Dispose(disposing);
         if (disposing) 
        {
                _host.Dispose();
            }
        }
}

/cc @javiercn

[mkArtakMSFT]

As this is not a priority for 2.2 for now, moving to the backlog.

[davidfowl]

@mkArtakMSFT just to clarify the concrete change we should make in 2.2:

We should make it so that it's possible to boot up the WebApplicationFactory (or another derived type) without a test server. It makes functional testing of your application absolutely trivial and the changes required to do this should be small.

/cc @javiercn

[steveoh]

I'm not sure I follow. How would these changes make it easier to use something like puppeteer and chrome headless/selenium for automated browser testing. What is a real server?

[shanselman]

@steveoh the current set up allows for very convenient Unit Testing by spinning up the App/WebHost and talking to it 'in memory." So no HTTP, no security issue, you're basically talking HTTP without actually putting bytes on the wire (or localhost). Super useful.

But if you want to use Automated Browser Testing and have it driven by a Unit Test Framework, you are hampered by the concrete TestServer class and have to do a little dance (above) to fake it out and start up the WebHost yourself. I'd propose no regular person could/would figure that out.

David is saying that if WebApplicationFactory could be started without the fake TestServer we could easily and cleanly do Unit Testing AND Browser Automation Testing with the piece we have here.

[rgamage]

This is why we love Scott! Always the advocate for the mainstream developers out there wanting to use MS tools, but stymied by various obscure limitations. Thank you sir!

[giggio]

The provided workaround has problems.

The problems start with the fact that now we have 2 hosts. One from the TestServer, and another built to work with http. And WebApplicationFactory.Server references the first one, which we are not testing against. And to make things worse, calls to methods that configure the builder, such as WebApplicationFactory.WithWebHostBuilder will not work with the dummy TestServer.

Because of these problems we cannot easily interact with the real host, the one being tested. It is very common that I change some backend service and configure it before a test is run. Suppose I need to access some service that is not callable during development, only production. I replace that service when I configure the services collection with a fake, and then configure it to respond the way I want it to respond. I can't do that through WebApplicationFactory.Server.Host.Services.

The resulting code I have works, but it is ugly as hell, it is an ugly ugly hack.

I hope we can move this forward and do not require TestServer, maybe an IServer. I thought about forking the whole TestServer and WebApplicationFactory infrastructure, but as this is somewhat planning I'll wait. I hope it gets fixed soon. I am just commenting to complement that the provided workaround is not enough and to really work around you have to avoid WebApplicationFactory.Server and create a terrible work around it.

[bchavez]

One way I solved this is to stop using WebApplicationFactory<T> all together.

Refactored Program.Main() to:

public static Task<int> Main(string[] args)
{
   return RunServer(args);
}
public static async Task<int> RunServer(string[] args,
                                        CancellationToken cancellationToken = default)
{
    ...
    CreateWebHostBuilder()
          .Build()
          .RunAsync(cancellationToken)
}

So, my unit test fixtures new up a var cts = new CancellationTokenSource(), then pass the cancellation token by calling Program.RunServer(new string[0], cts.Token). The server starts up as normal without having to create a separate process.

Make real HTTP calls like normal. When your done and your unit test completes, call cts.Cancel() to clean up and shutdown the HTTP server.

One down side is you need to copy appsettings.json to an output directory from your test project; potentially managing two appsettings.json files (one in your server project, and one in your test project). Maybe a linked project file could help eliminate the issue.

YMMV.

:thinking: :sparkles: "Do you ponder the manner of things... yeah yeah... like glitter and gold..."

[giggio]

Hello everyone, this issue is still unresolved and seems to keep being postponed. Just so we know the planning, are you considering resolving this for ASP.NET Core 3.0? If not, do you have a workaround that does not incur on the problems I mentioned earlier (https://github.com/aspnet/AspNetCore/issues/4892#issuecomment-407569665)?

[BennyM]

This would be very welcome. The workaround mentioned here is great, except when you have html files and what not that you would also need to copy over.

[Tratcher]

WebApplicationFactory is designed for a very specific in-memory scenario. Having it start Kestrel instead and wire up HttpClient to match is a fairly different feature set. We actually do have components that do this in Microsoft.AspNetCore.Server.IntegrationTesting but we've never cleaned them up for broader usage. It might make more sense to leave WebApplicationFactory for in-memory and improve the IntegrationTesting components for this other scenario.

[giggio]

I'm fine with that, as long as we have a good end to end testing story.

[M-Yankov]

Thanks guys for this discussion and specially to @bchavez
I succeeded to run Selenium tests in Azure pipelines with the idea to start a local server on the agent, run tests and stop the server.

My test class:
```c#
public class SelenuimSampleTests : IDisposable
{
private const string TestLocalHostUrl = "http://localhost:8080";

private readonly CancellationTokenSource tokenSource;

public SelenuimSampleTests()
{
    this.tokenSource = new CancellationTokenSource();

    string projectName = typeof(Web.Startup).Assembly.GetName().Name;

    string currentDirectory = Directory.GetCurrentDirectory();
    string webProjectDirectory = Path.GetFullPath(Path.Combine(currentDirectory, $@"..\..\..\..\{projectName}"));

    IWebHost webHost = WebHost.CreateDefaultBuilder(new string[0])
        .UseSetting(WebHostDefaults.ApplicationKey, projectName)
        .UseContentRoot(webProjectDirectory) // This will make appsettings.json to work.
        .ConfigureServices(services =>
        {
            services.AddSingleton(typeof(IStartup), serviceProvider =>
            {
                IHostingEnvironment hostingEnvironment = serviceProvider.GetRequiredService<IHostingEnvironment>();
                StartupMethods startupMethods = StartupLoader.LoadMethods(
                    serviceProvider, 
                    typeof(TestStartup),
                    hostingEnvironment.EnvironmentName);

                return new ConventionBasedStartup(startupMethods);
            });
        })
        .UseEnvironment(EnvironmentName.Development)
        .UseUrls(TestLocalHostUrl)
        //// .UseStartup<TestStartUp>() // It's not working
        .Build();

    webHost.RunAsync(this.tokenSource.Token);
}

public void Dispose()
{
    this.tokenSource.Cancel();
}

[Fact]
public void TestWithSelenium()
{
    string assemblyLocation = System.Reflection.Assembly.GetExecutingAssembly().Location;
    string currentDirectory = Path.GetDirectoryName(assemblyLocation);

    using (ChromeDriver driver = new ChromeDriver(currentDirectory))
    {
        driver.Navigate().GoToUrl(TestLocalHostUrl);
        IWebElement webElement = driver.FindElementByCssSelector("a.navbar-brand");

        string expected = typeof(Web.Startup).Assembly.GetName().Name;

        Assert.Equal(expected, webElement.Text);

        string appSettingValue = driver.FindElementById("myvalue").Text;
        const string ExpectedAppSettingsValue = "44";
        Assert.Equal(ExpectedAppSettingsValue, appSettingValue);
    }
}

}
```

Very similar to the @bchavez 's example, I'm starting the CreateDefaultBuilder. That approach gives me a little bit more flexibility to set custom settings. For example using a TestStartup. the .UseStartup<TestStartUp>() wasn't working, so I used logic from WebHostBuilderExtensions.UseStartUp to set WebHostDefaults.ApplicationKey and configure services.

I'm sure there is better approach somewhere, but after а few days of research, nothing helped me. So I share this solution in case someone need it.

_The test method is just for an example, there are different approaches to initialize browser driver._

[arkiaconsulting]

@M-Yankov I had your code work by using IWebHostBuilder.UseStartup<>, and IWebHost.StartAsync as well as IWebHost.StopAsync, without CancellationTokenSource.
=> .NetCore 2.2

[M-Yankov]

The idea of the IWebHostBuilder.UseStartup<> is to simplify the code. But for me it was not enough: after applying the UseStartup<TestStartup>(), ChromeWEbDriver cannot open the home page.
I didn't mention that the TestStartup class inherits the real startup class with overriding two custom methods (_if it matters_).
That's why I've used the .ConfigureServices(services => ... approach.
About the IWebHost.StartAsync & IWebHost.StopAsync - it seems that they are working. Thanks.

[Sebazzz]

This is a solution that worked for me in the mean time. It ensures that everything on WebApplicationFactory, like the Services property keep working as expected:

    protected override void ConfigureWebHost(IWebHostBuilder builder) {
        if (builder == null) throw new ArgumentNullException(nameof(builder));

        IPEndPoint endPoint;
        // Assign available TCP port
        using (var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)){
            socket.Bind(new IPEndPoint(IPAddress.Loopback, 0));
            socket.Listen(1);
            endPoint = (IPEndPoint)socket.LocalEndPoint;
        }

        return builder
            .ConfigureKestrel(k => k.Listen(new IPEndPoint(IPAddress.Loopback, 0)));
    }

    protected override TestServer CreateServer(IWebHostBuilder builder)
    {
        // See: https://github.com/aspnet/AspNetCore/issues/4892
        this._webHost = builder.Build();

        var testServer = new TestServer(new PassthroughWebHostBuilder(this._webHost));
        var address = testServer.Host.ServerFeatures.Get<IServerAddressesFeature>();
        testServer.BaseAddress = new Uri(address.Addresses.First());

        return testServer;
    }

    private sealed class PassthroughWebHostBuilder : IWebHostBuilder
    {
        private readonly IWebHost _webHost;

        public PassthroughWebHostBuilder(IWebHost webHost)
        {
            this._webHost = webHost;
        }

        public IWebHost Build() => this._webHost;

        public IWebHostBuilder ConfigureAppConfiguration(Action<WebHostBuilderContext, IConfigurationBuilder> configureDelegate){
            TestContext.WriteLine($"Ignoring call: {typeof(PassthroughWebHostBuilder)}.{nameof(this.ConfigureAppConfiguration)}");
            return this;
        }

        public IWebHostBuilder ConfigureServices(Action<WebHostBuilderContext, IServiceCollection> configureServices) {
            TestContext.WriteLine($"Ignoring call: {typeof(PassthroughWebHostBuilder)}.{nameof(ConfigureServices)}");
            return this;
        }

        public IWebHostBuilder ConfigureServices(Action<IServiceCollection> configureServices)
        {
            TestContext.WriteLine($"Ignoring call: {typeof(PassthroughWebHostBuilder)}.{nameof(ConfigureServices)}");
            return this;
        }

        public string GetSetting(string key) => throw new NotImplementedException();

        public IWebHostBuilder UseSetting(string key, string value)
        {
            TestContext.WriteLine($"Ignoring call: {typeof(PassthroughWebHostBuilder)}.{nameof(this.UseSetting)}({key}, {value})");
            return this;
        }
    }

[JeroMiya]

Just a note: with .net core 3.0 apps using IHostBuilder, @danroth27's workaround no longer works.

Now I just have a test script that runs the server, then runs the tests, waits for them to finish, then kills the server. Have to start up the server manually when running in the Visual Studio test runner. Not a great experience.

[Sebazzz]

Have you looked at what I posted above?

Met vriendelijke groet,
Sebastiaan Dammann

---

Van: Jeremy [e-mail]
Verzonden: Friday, October 25, 2019 6:23:35 PM
Aan: aspnet/AspNetCore [e-mail]
CC: Sebastiaan Dammann [e-mail]; Comment [e-mail]
Onderwerp: Re: [aspnet/AspNetCore] Improve automated browser testing with real server (#4892)

Just a note: with .net core 3.0 apps using IHostBuilder, @danroth27https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdanroth27&data=02%7C01%7C%7C431d232f4a1e48a4fdae08d75967af9e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637076174165457064&sdata=nnjWPdeQbclhT8tqRJ0dTvT0%2B9%2BIVVy59xlR6YroZ9A%3D&reserved=0's workaround no longer works.

Now I just have a test script that runs the server, then runs the tests, waits for them to finish, then kills the server. Have to start up the server manually when running in the Visual Studio test runner. Not a great experience.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faspnet%2FAspNetCore%2Fissues%2F4892%3Femail_source%3Dnotifications%26email_token%3DAAK4FMKG45JYM75DPWDPXCDQQMMQPA5CNFSM4GKKY3G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECI3HWQ%23issuecomment-546419674&data=02%7C01%7C%7C431d232f4a1e48a4fdae08d75967af9e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637076174165457064&sdata=RBes58Dw7x44ts%2F0lcG4UoyiWSCrOtAruhhqNIdqmX0%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAK4FMIWUOFTNNP5HZ54KNDQQMMQPANCNFSM4GKKY3GQ&data=02%7C01%7C%7C431d232f4a1e48a4fdae08d75967af9e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637076174165467069&sdata=7%2BkYk6vmLkKcfCmQF23laO3E1b2%2FsE2bhc3t3DNufSI%3D&reserved=0.

[JeroMiya]

I'm not using IWebHostBuilder so CreateServer is never called.

[Sebazzz]

Yes but you might be able to use a similar approach 👍

Met vriendelijke groet,
Sebastiaan Dammann

---

Van: Jeremy [e-mail]
Verzonden: Friday, October 25, 2019 6:30:51 PM
Aan: aspnet/AspNetCore [e-mail]
CC: Sebastiaan Dammann [e-mail]; Comment [e-mail]
Onderwerp: Re: [aspnet/AspNetCore] Improve automated browser testing with real server (#4892)

I'm not using IWebHostBuilder so CreateServer is never called.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faspnet%2FAspNetCore%2Fissues%2F4892%3Femail_source%3Dnotifications%26email_token%3DAAK4FMJDHYG5EVV7DEXXI2TQQMNLXA5CNFSM4GKKY3G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECI35HQ%23issuecomment-546422430&data=02%7C01%7C%7C5c08ad3d4c9142a4bf7208d75968b3c0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637076178530399604&sdata=ssIMCSG9VHxrAhaDKP49o3cXXy79hOkd1k4rH8ZneJM%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAK4FMJJAN4VBVYAYJVKAGLQQMNLXANCNFSM4GKKY3GQ&data=02%7C01%7C%7C5c08ad3d4c9142a4bf7208d75968b3c0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637076178530409614&sdata=r7Id092T3ADSKoHnnTKJSzvBsmgis1Vu698UMYEWOn8%3D&reserved=0.

[JeroMiya]

There's an override you can define create a custom IHostBuilder and another to create a custom IHost with a builder passed in, and that's likely where the solution would be. However, it didn't work when I tried it. I don't know what part of the internals of WebApplicationFactory adds the in-memory restrictions but it might to be outside of those two overloads. I've already spent too much time on it, and running the app directly without WebApplicationFactory seems to work fine for now.

[ffMathy]

What is the status on this? @davidfowl did this ever make it to 2.2? In that case, how do we use it?

[Tratcher]

@ffMathy not much progress has been made on this scenario, it's still in the backlog.

[ffMathy]

Alright. Is there an ETA? Rough estimate?

[Tratcher]

@ffMathy this is an uncommitted feature, there's no ETA until we decide to move it from the backlog to a milestone.

[lukos]

@Sebazzz Your code above does not compile.

• You are returning ConfigureKestrel, even though the method is void.
• You are assigning a variable _webHost in CreateServer that is not in the listing. Is this just a class variable or is it supposed to be coming from somewhere else?
• You setup an endpoint in ConfigureWebHost and then you don't use it, you just create another one in the call to ConfigureKestrel.

[Sebazzz]

Look here for the full example: https://github.com/Sebazzz/Return/blob/1088ce98cc93fbb8f4f6863edf6fc9292fbff15a/tests/Return.Web.Tests.Integration/Common/CustomWebApplicationFactory.cs#L130

Met vriendelijke groet,
Sebastiaan Dammann

---

Van: Luke Briner [e-mail]
Verzonden: Thursday, March 5, 2020 6:41:02 PM
Aan: dotnet/aspnetcore [e-mail]
CC: Sebastiaan Dammann [e-mail]; Mention [e-mail]
Onderwerp: Re: [dotnet/aspnetcore] Improve automated browser testing with real server (#4892)

@Sebazzzhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSebazzz&data=02%7C01%7C%7Cd0cce2fffe774d406ebf08d7c12c6049%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637190268638247927&sdata=A2ZE3M0mz%2FPCCDvCmxpG9ZwR7rWaRtdE7bpZR5CLr6M%3D&reserved=0 Your code above does not compile.

• You are returning ConfigureKestrel, even though the method is void.
• You are assigning a variable _webHost in CreateServer that is not in the listing. Is this just a class variable or is it supposed to be coming from somewhere else?
• You setup an endpoint in ConfigureWebHost and then you don't use it, you just create another one in the call to ConfigureKestrel.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Faspnetcore%2Fissues%2F4892%3Femail_source%3Dnotifications%26email_token%3DAAK4FMKQE24FSZFJ2VIT5PLRF7P25A5CNFSM4GKKY3G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN6GWXA%23issuecomment-595356508&data=02%7C01%7C%7Cd0cce2fffe774d406ebf08d7c12c6049%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637190268638257908&sdata=R4B9jZqXN37M3Coba5lpBGgvsiklBe3%2FV2xp3y97Jsw%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAK4FMOXCMRNPOBK252VS3DRF7P25ANCNFSM4GKKY3GQ&data=02%7C01%7C%7Cd0cce2fffe774d406ebf08d7c12c6049%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637190268638267892&sdata=HK4S%2FeoimGA6KIsh8eC92%2BFaItr8Co5nXOsILjsPKL8%3D&reserved=0.

[shanselman]

Sorry to dump a bunch of code, but this is how I do it. I did it this way in 2.x and I do it like this in 3.1.

namespace hanselminutes_core_tests
{
    using hanselminutes_core;
    using Microsoft.AspNetCore.Hosting;
    using Microsoft.AspNetCore.Hosting.Server.Features;
    using Microsoft.AspNetCore.Mvc.Testing;
    using Microsoft.AspNetCore.TestHost;
    using OpenQA.Selenium;
    using OpenQA.Selenium.Chrome;
    using OpenQA.Selenium.Remote;
    using System;
    using System.Diagnostics;
    using System.Linq;
    using System.Net.Http;
    using Xunit;

    public static class AreWe
    {
        public static bool InDockerOrBuildServer { 
           get { 
               string retVal = Environment.GetEnvironmentVariable("DOTNET_RUNNING_IN_CONTAINER");
               string retVal2 = Environment.GetEnvironmentVariable("AGENT_NAME");
               return (
                    (String.Compare(retVal, Boolean.TrueString, ignoreCase: true) == 0)  
                    || 
                    (String.IsNullOrWhiteSpace(retVal2) == false));
               } 
            }
    }

    public class SeleniumServerFactory<TStartup> : WebApplicationFactory<Startup> where TStartup : class
    {
        IWebHost _host;
        public string RootUri { get; set; }
        Process _process;


        public SeleniumServerFactory()
        {
            if (AreWe.InDockerOrBuildServer) return;

            ClientOptions.BaseAddress = new Uri("https://localhost"); //will follow redirects by default

            _process = new Process() {
                StartInfo = new ProcessStartInfo {
                    FileName = "selenium-standalone",
                    Arguments = "start",
                    UseShellExecute = true,
                    CreateNoWindow = false                
                }
            };
            _process.Start();
        }

        protected override TestServer CreateServer(IWebHostBuilder builder)
        {
            //Real TCP port
            _host = builder.Build();
            _host.Start();
            RootUri = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.LastOrDefault(); //Last is ssl!

            //Fake Server we won't use
            return new TestServer(new WebHostBuilder().UseStartup<TStartup>());
        }

        protected override void Dispose(bool disposing) 
        {
            base.Dispose(disposing);
            if (disposing) {
                _host?.Dispose();
                _process?.CloseMainWindow();
            }
        }
    }

    [Trait("Category", "SkipWhenLiveUnitTesting")]
    public class SeleniumTests : IClassFixture<SeleniumServerFactory<Startup>>, IDisposable
    {
        public HttpClient Client { get; }
        public SeleniumServerFactory<Startup> Server { get; }
        public IWebDriver Browser { get; }
        public ILogs Logs { get; }

        public SeleniumTests(SeleniumServerFactory<Startup> server)
        {
            Console.WriteLine("In Docker?" + AreWe.InDockerOrBuildServer);
            if(AreWe.InDockerOrBuildServer) return;
            Server = server;
            Client = server.CreateClient(); //weird side effecty thing here. This shouldn't be required but it is.

            var opts = new ChromeOptions();
            //opts.AddArgument("--headless");
            opts.SetLoggingPreference(OpenQA.Selenium.LogType.Browser, LogLevel.All);

            var driver = new RemoteWebDriver(opts);
            Browser = driver;
            Logs = new RemoteLogs(driver); //TODO: Still not bringing the logs over yet?
        }

        [SkippableFact(typeof(OpenQA.Selenium.WebDriverException))]
        public void LoadTheMainPageAndCheckTitle()
        {
            Skip.If(AreWe.InDockerOrBuildServer);
            Browser.Navigate().GoToUrl(Server.RootUri);
            Assert.StartsWith("Hanselminutes Technology Podcast - Fresh Air and Fresh Perspectives for Developers", Browser.Title);
        }

        [SkippableFact(typeof(OpenQA.Selenium.WebDriverException))]
        public void ThereIsAnH1()
        {
            Skip.If(AreWe.InDockerOrBuildServer);
            Browser.Navigate().GoToUrl(Server.RootUri);

            var headerSelector = By.TagName("h1");
            Assert.Equal("HANSELMINUTES PODCAST by Scott Hanselman", Browser.FindElement(headerSelector).Text);
        }

        [SkippableFact(typeof(OpenQA.Selenium.WebDriverException))]
        public void KevinScottTest()
        {
            Skip.If(AreWe.InDockerOrBuildServer);
            Browser.Navigate().GoToUrl(Server.RootUri + "/631/how-do-you-become-a-cto-with-microsofts-cto-kevin-scott");

            var headerSelector = By.TagName("h2");
            Assert.Equal("How do you become a CTO - with Microsoft's CTO Kevin Scott", Browser.FindElement(headerSelector).Text);
        }

        [SkippableFact(typeof(OpenQA.Selenium.WebDriverException))]
        public void KevinScottTestThenGoHome()
        {
            Skip.If(AreWe.InDockerOrBuildServer, "In Docker!");
            Browser.Navigate().GoToUrl(Server.RootUri + "/631/how-do-you-become-a-cto-with-microsofts-cto-kevin-scott");

            var headerSelector = By.TagName("h1");
            var link = Browser.FindElement(headerSelector);
            link.Click();
            Assert.Equal(Browser.Url.TrimEnd('/'),Server.RootUri); //WTF
        }

        public void Dispose()
        {
            if (Browser != null)
                Browser.Dispose();
        }
    }
}

[lukos]

@shanselman thanks for the code, although this doesn't work correctly with IHostBuilder apps, which will not call CreateServer but CreateHost. That would be fine except the types then change and the journey through the factory is quite hard to understand e.g. IHost doesn't have ServeFeatures so we cannot get the RootUrl in the same way. I will continue to mess around though, I think you have provided the important bit.

[davidfowl]

IHost.Services.GetRequiredService\

[lukos]

@davidfowl thanks but the IServerAddressesFeature is no longer added to the server: https://github.com/aspnet/Hosting/issues/956

I have used UseUrls() on the ConfigureWebHostDefaults webbuilder param so I preset the address and don't need to query it but now I just need to add appsettings from the test project otherwise the app falls over.

[Tratcher]

@lukos that's not what that linked issue means, it was only taking about default values. Did you try it?

[lukos]

@Tratcher Yes I did and it didn't have the IServerAddressesFeature feature in IServer.Features.

In the end I have made it work but I need to blog about it, paste the link here and see what feedback I get before suggesting it as a proper solution. Basically, by using UseUrls() on my real server, I know what the RootUri is and can set it directly rather than allowing Kestrel to allocate something and having to ask what it is afterwards.

I also had to do a few things to the fake server. Part of the complication is that my web app won't run without configuration so I also had to set the content root for the fake server to allow it to startup and find appsettings using UseSolutionRelativeContentRoot and then call UseTestServer so that calling base.CreateHost() doesn't moan about not being able to cast KestrelServer to TestServer.

I also removed the Selenium standalone since we already have something that works using the normal nuget package.

[davidfowl]

@lukos That's not correct. IServerAddressesFeature is absolutely there. If you have a piece of code that reproduces the issue please paste it here.

[lukos]

The code has churned a bit since then but I think it was simply the following code, taken from Scott's example and modified for IHostBuilder that was the problem:

protected override IHostBuilder CreateHostBuilder()
{
    var builder = base.CreateHostBuilder();
    // Logging added here
    return builder;
}

protected override IHost CreateHost(IHostBuilder builder)
{
    host = builder.Build();
    host.Start();
    var features = host.Services.GetRequiredService<IServer>().Features;
    RootUri = features.Get<IServerAddressesFeature>().Addresses.FirstOrDefault();  // Null reference exception
}

[jepperaskdk]

I haven't tried with a more complex scenario (and I'm a complete noob) - but it seems to work fine with a KestrelServer if I simply ignore the type being wrong (both KestrelServer and TestServer implement IServer?): Repro

Am I missing the point?