Polly: Test Polly retry polly configured via Startup.ConfigureServices() with ASP.NET Core API

[PingPongSet]

I want to find out how Polly retry polly configured via Startup.ConfigureServices() can be tested.

ConfigureServices

Polly policy is configured within it

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
                 services.AddHttpClient<IHttpClientService, HttpClientService>()
                .SetWaitAndRetryPolicy1();     

     }
}

Below is the Polly policy:

 public static class IServiceCollectionExtension
    {

    public static void SetWaitAndRetryPolicy1(this IHttpClientBuilder clientBuilder)
        {
                             clientBuilder.AddPolicyHandler((service, request) =>
                            HttpPolicyExtensions.HandleTransientHttpError()
                                .WaitAndRetryAsync(3,
                                    retryCount => TimeSpan.FromSeconds(Math.Pow(2, retryCount)),
                onRetry: (outcome, timespan, retryCount, context) =>
                                {
                                    service.GetService<ILog>().Error("Delaying for {delay}ms, then making retry {retry}.",
                                        timespan.TotalMilliseconds, retryCount);
                                }
                            )

                        );        
        }
    }

Below is what I tried:

Integration test

The Polly policy is configured within the test.

  public class RetryPolicyTests : IClassFixture<WebApplicationFactory<Startup>>
    {
        private readonly WebApplicationFactory<Startup> _factory;

        public RetryPolicyTests(WebApplicationFactory<Startup> factory)
        {
            _factory = factory;
        }

        [Theory]
        [InlineData("http://localhost:1234/api/v1/car/")]
        public async Task Test3(string url)
        {
            // Arrange
            var client = _factory.WithWebHostBuilder(whb =>
                {
                    whb.ConfigureServices((bc, sc) =>
                    {
                        sc.AddOptions();
                        sc.AddHttpClient("test")
                          .SetWaitAndRetryPolicy1();         //Test the Polly policy             
                        sc.BuildServiceProvider();
                    });
                })
                .CreateClient();  //cannot get the named HttpClient

            // Act           
            var body = "{}";
            using (var content = new StringContent(body, Encoding.UTF8, "application/json"))
            {

                var response = await client.PostAsync(url, content);



            }

            //Assert: somewhy assert it
        }
    }
}

The problem is that

I cannot retrieve the HttpClient that has been configured with the Polly polly. Because WebApplicationFactory.CreateClient() has no overloads that returns the named HttpClient:

Any idea?

ASPS.NET Core API 2.2

Update After Comment from @reisenberger 4 Jan 2019

Updated Integration Test method
I updated my existing integration test method to below, but the retry policy is not activated. That is, it only sends request one time, not three times.

Queston 1: Am I missing something?

Please note the new name RetryPolicyTests2 .

```
public class RetryPolicyTests2 : IClassFixture>
{
private readonly WebApplicationFactory _factory;

        public RetryPolicyTests2(WebApplicationFactory<Startup> factory)
        {
            _factory = factory;
        }

[Theory]
[InlineData("http://localhost:1234/api/v1/car/")]
public async Task TestRetry(string url)
{
// Arrange
int count = 0;
var logMock = new Mock();
logMock.Setup(log =>
log.Log(It.IsAny(), It.IsAny>(), It.IsAny(), It.IsAny )
.Callback(() => {
count++;
});

       var client = _factory.WithWebHostBuilder(whb =>
        {
            whb.ConfigureServices((bc, sc) =>
            {
                sc.AddOptions();
                sc.AddSingleton<ILog>(logMock.Object);

                sc.AddHttpClient("test")
                    .SetWaitAndRetryPolicy()
                    .AddHttpMessageHandler(() => new StubDelegatingHandler(HttpStatusCode.InternalServerError));

                sc.BuildServiceProvider()
                    .GetRequiredService<IHttpClientFactory>()
                    .CreateClient("test");
            });
        }).CreateClient();

        // Act           
        var body = "{}";
        using (var content = new StringContent(body, Encoding.UTF8, "application/json"))
        {
            var response = await client.PostAsync(url, content);

            // Assert
            Assert.Equal(3, count);
        }

    }

}

**Updated Retry Policy method**

Retry setting is set via a config file in JSON (e.g. appsettings.json). Please note the new name SetWaitAndRetryPolicy2.

public static IHttpClientBuilder SetWaitAndRetryPolicy2(this IHttpClientBuilder clientBuilder)
{
Random random = new Random();
return clientBuilder.AddPolicyHandler((service, request) =>
{
var config = service.GetService>().Value; //get config from json file
return HttpPolicyExtensions.HandleTransientHttpError()
.WaitAndRetryAsync(config.Retry.RetryCount,
retryCount => TimeSpan.FromSeconds(Math.Pow(2, retryCount)) + //2,4,8,...
TimeSpan.FromMilliseconds(random.Next(100, 990)), //random in millisecs
onRetry: (outcome, timespan, retryCount, context) =>
{
service.GetService().Log(LogLevel.Error,
() =>
$"Delaying for {timespan.TotalMilliseconds}ms, then making retry {retryCount}.");
});
}

        );
    }

`` **Question 2**: How can Config be setup for Integration test within WithWebHostBuilder() inTestRetry()method if it is the correct method, and for unit test inHttpClientFactory_Polly_Policy_Test `class.

[reisenberger]

Hi @PingPongSet

WebApplicationFactory.CreateClient() has no overloads that returns the named HttpClient:

That makes sense: the Httpclient returned by WebApplicationFactory.CreateClient() is specifically geared to pass requests to your app from outside; the HttpClient instances configured within your app are (on the other hand) an internal concern to it.

I will answer the question at three different levels, and you can choose what suits best.

If you just want to adapt your existing test code to get the client configured with name "test"

(in response to "I cannot retrieve the HttpClient that has been configured with the Polly polly")

sc.BuildServiceProvider()
    .GetRequiredService<IHttpClientFactory>()
    .CreateClient("test");

If you want a full end-to-end integration test that your app uses the configured Policy

(to respond to the question title: "Test Polly retry polly configured via Startup.ConfigureServices() with ASP.NET Core API")

If you want to test the Polly policy configured on IHttpClientService within your app, _via an end-to-end integration test of your app orchestrated by_ WebApplicationFactory, then you will have to fire the whole request at http://localhost:1234/api/v1/car/ (as your test code is already doing), and somehow stub out whatever downstream call http://localhost:1234/api/v1/car/ is making through HttpClientService. To provide stub responses to that downstream call (not shown in the code posted in the question, I don't know what it is), you could use either:

• Mountebank via MBDotNet - fairly heavyweight as it involves installing node.js and binding a port for Http traffic
• HttpClientInterception, much more lightweight - it stubs responses to HttpClient calls by adding an intercepting DelegatingHandler to the delegating handler middleware of HttpClients used by your app

HttpClientInterception provides a good sample app which demonstrates how to set up HttpClientInterception to provide stub responses to outbound calls which your app makes. The app-under-test in their sample app is also using typed-clients from IHttpClientFactory; and is also using WebApplicationFactory to orchestrate the tests; so is a close fit for the test approach you have already started on.

You would use Mountebank or HttpClientInterception to stub the outbound call from HttpClientService to return something the policy handles eg HttpStatusCode.InternalServerError, in order to trigger the Polly retry policy.

Alternatively, you could write your own very short StubDelegatingHandler

    public class StubDelegatingHandler : DelegatingHandler
    {
        private HttpStatusCode stubHttpStatusCode;
        public StubDelegatingHandler(HttpStatusCode stubHttpStatusCode) => this.stubHttpStatusCode = stubHttpStatusCode;
        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) => Task.FromResult(new HttpResponseMessage(stubHttpStatusCode));
    }

and configure it after the Polly policy on the HttpClient ('inside' the Polly policy , it terms of the nested DelegatingHandlers).

sc.AddHttpClient("test")
    .SetWaitAndRetryPolicy1() // modify this to return IHttpClientBuilder, to allow chaining
    .AddHttpMessageHandler(() => new StubDelegatingHandler(HttpStatusCode.InternalServerError));

This means _every_ outbound call that the named-client "test" makes would return HttpStatusCode.InternalServerError; it's a minimal example of what HttpClientInterception does, but HttpClientInterception does more, does it with much more configurability, and with a nice fluent syntax.

Of course, you could make StubDelegatingHandler more sophisticated, to return the error only 2 times ... or whatever.

To test that the retry policy is invoked, you could make the test setup configure a fake/mock ILog implementation, and (for example) assert that the expected call .Error("Delaying for {delay}ms, ...") in your onRetry delegate is made on the fake logger. Then you would know the retry had been invoked.

---

If you want the shortest possible unit test that HttpClientFactory does configure a policy correctly on the client

... and that the policy you declare and configure with HttpClientFactory does handle errors as you want

I offer this variant in case you just want the shortest possible test of the functionality declared in a method like .SetWaitAndRetryPolicy1().

This is (almost) the shortest xUnit test I could write that HttpClientFactory does correctly configure and use a policy. There is no need for any WebApplicationFactory, IHost, IHostedService or anything from ASP.NET. The test simply proves that HttpClientFactory does configure the HttpClient to use the policy. It is possible simply to new up a ServiceCollection; configure a named client using HttpClientFactory; get the named client back from the IServiceProvider; and test if that client uses the policy.

public class HttpClientFactory_Polly_Policy_Test
{
    const string TestClient = "TestClient";

    [Fact]
    public async Task Given_a_retry_policy_configured_on_a_named_client_When_call_via_the_named_client_Then_the_policy_is_used()
    {
        // Given / Arrange 
        IServiceCollection services = new ServiceCollection();

        bool retryCalled = false;

        HttpStatusCode codeHandledByPolicy = HttpStatusCode.InternalServerError;

        services.AddHttpClient(TestClient)
            .AddPolicyHandler(HttpPolicyExtensions.HandleTransientHttpError()
                .RetryAsync(3, onRetry: (_, __) => retryCalled = true))
            .AddHttpMessageHandler(() => new StubDelegatingHandler(codeHandledByPolicy));

        HttpClient configuredClient =
            services
                .BuildServiceProvider()
                .GetRequiredService<IHttpClientFactory>()
                .CreateClient(TestClient);

        // When / Act
        var result = await configuredClient.GetAsync("https://www.doesnotmatterwhatthisis.com/");

        // Then / Assert
        Assert.Equal(codeHandledByPolicy, result.StatusCode);
        Assert.True(retryCalled);
    }

}

public class StubDelegatingHandler : DelegatingHandler
{
    private readonly HttpStatusCode stubHttpStatusCode;
    public StubDelegatingHandler(HttpStatusCode stubHttpStatusCode) => this.stubHttpStatusCode = stubHttpStatusCode;
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) => Task.FromResult(new HttpResponseMessage(stubHttpStatusCode));
}

It should be easy to expand this sample to test more sophisticated policies, for example to test .SetWaitAndRetryPolicy1().

[PingPongSet]

Hi @reisenberger

What a great comment and advice!

However, I still have problem getting the named HttpClient, and other questions.

Please refer to my updated comments at the bottom of OP.

Thanks again in advance.

[reisenberger]

Hi @PingPongSet . No problem, glad it could help.

Re:

Updated Integration Test method
I updated my existing integration test method to below, but the retry policy is not activated. That is, it only sends request one time, not three times.
Queston 1: Am I missing something?

One possible thing. This:

 var client = _factory.WithWebHostBuilder(whb =>
       {
           whb.ConfigureServices((bc, sc) =>
           {
              /* snip */

               sc.BuildServiceProvider()
                   .GetRequiredService<IHttpClientFactory>()
                   .CreateClient("test"); // This isn't doing anything because the return value of this call isn't assigned to anything you use.
           });
       }).CreateClient();

means the variable HttpClient client which the test posts on (await client.PostAsync(url, content);) is assigned the HttpClient returned from WebApplicationFactory, the HttpClient instance designed to invoke your webapp, not the "test" configuration from HttpClientFactory. So for the test to succeed, your app must be configured such that invoking the http://localhost:1234/api/v1/car/ endpoint eventually chains on internally to something (via HttpClientService?) invoking the "test" configuration from HttpClientFactory (as I believe it should, from what you have described as the code intention). In other words, it's a full end-to-end integration test. If that code expectation is not all wired up properly inside the app, it could be a cause of test failure.

However, if you intended the test to exercise more directly the "test" configuration from HttpClientFactory, you may want:

 HttpClient client;
 factory.WithWebHostBuilder(whb =>
       {
           whb.ConfigureServices((bc, sc) =>
           {
              /* snip */

               client = sc.BuildServiceProvider()
                   .GetRequiredService<IHttpClientFactory>()
                   .CreateClient("test");
           });
       }).CreateClient();

so that the variable client is assigned the "test" configuration from HttpClientFactory.

This makes it like a half-integration, half-unit test. The test uses WebApplicationFactory to exercise your normal app startup in configuring the HttpClient/policy to be tested; but then pull the "test" HttpClient configuration out for a tighter unit test.

Per my original post, if you just want a tight unit-test on the HttpClient "test" configured via HttpClientFactory, you can also do this with the "shortest-possible approach", without needing to involve WebApplicationFactory.

---

Re:

Question 2:
How can Config be setup for Integration test within WithWebHostBuilder() in TestRetry() method if it is the correct method, and for unit test in HttpClientFactory_Polly_Policy_Test class.

This is more general ASP.NET Core support rather than Polly, but some pointers: Options.Create<>() if you want the options to be entirely self-generated by a purely self-contained unit test; or use ConfigurationBuilder to read in external config (eg json settings file) if you want a more integration-style approach which reads in some version of your app's configuration. See these example links: 1; 2; 3; 4.

[PingPongSet]

Hi @reisenberger,

Thanks again for the prompt reply and the great answer.

I have another question on setting system clock.

Very much appreicate your help.

P.S. I posted the same question on StackOverflow a few weeks ago without any answer. I am getting answers right away here. :)