Aspnetcore: Controller returning IAsyncEnumerable consumed by HttpClient & GetAsync with ResponseHeadersRead not working on Windows with content type application/json

[rossbuggins]

Currently trying out the controller IAsyncEnumerable support. I have a controller returning IAsyncEnumerable. I have a client that it consuming this, and I thought i would be able to access the data before its all finished being received, but not currently seeing this - I'm using ReadAsStreamAsync & ReadAsync on the HttpClient.

[mkArtakMSFT]

Thanks for contacting us, @rossbuggins.
There is no protocol support for this. So we simply buffer this before rendering the results. All MVC is doing is removing the blocking call from controller actions.
Async serialization support for System.Text.Json is tracked as part of https://github.com/dotnet/corefx/issues/38523

[rossbuggins]

Thanks for getting back to me. When documentation is written for this it needs to be pretty clear this is the behaviour- as I think the general assumption would be that returning IAsyncEnumerable from a controller will stream the response back at each yield return? ie the though that its doing something more special that IEnumerable responses?

[rossbuggins]

I've quickly knocked up the below ActionResultExecutor to handle a proof of concept. Strange thing though, if i set content type to application/json then HttpClient doesn't return to ReadAsync until the whole request is finished, even though i can see with wireshark (and server memory) that the data is being sent, but if i use text/event-stream then HttpClient is responding each time the buffer is flushed on the server?

 public class AsyncStreamExecutor<T> : IActionResultExecutor<AsyncStreamResult<T>>
    {
        public async Task ExecuteAsync(ActionContext context, AsyncStreamResult<T> result)
        {
            context.HttpContext.Response.StatusCode = 200;
            context.HttpContext.Response.Headers.Add("Content-Type", "application/json");
           // context.HttpContext.Response.Headers.Add("Content-Type", "text/event-stream");
            await context.HttpContext.Response.Body.FlushAsync();

            var sw = new StreamWriter(context.HttpContext.Response.Body);

            await foreach (var obj in result.AsyncEnmerable)
            {
                var str = System.Text.Json.JsonSerializer.Serialize(obj);
                await sw.WriteAsync(str);
                await sw.FlushAsync();
            }

        }
    }

    public class AsyncStreamResult<T> : IActionResult
    {
        public IAsyncEnumerable<T> AsyncEnmerable { get; set; }
        public AsyncStreamResult(IAsyncEnumerable<T> asyncEnmerable)
            {
            this.AsyncEnmerable = asyncEnmerable;
            }

        public Task ExecuteResultAsync(ActionContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException(nameof(context));
            }

            var executor = context.HttpContext.RequestServices.GetRequiredService<IActionResultExecutor<AsyncStreamResult<T>>>();
            return executor.ExecuteAsync(context, this);
        }
    }

    public static class AsyncStreamerExtensionMethods
    {
        public static IServiceCollection AddAsyncEnumerableStreamer<T>(this IServiceCollection services)
        {
            services.TryAddSingleton<IActionResultExecutor<AsyncStreamResult<T>>, AsyncStreamExecutor<T>> ();
            return services;
        }
    }

[pranavkm]

@Tratcher would you know what's up with the HttpClient?

[Tratcher]

Please show the client code. It should look something like this:

var response = await client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
var responseStream = await response.ReadAsStreamAsync();

[rossbuggins]

yup, thats what I've got (tried with SendAsync with type post as well - see below), the latest code which works and deserializes the json objects "live" as they appear at the client is below, seems pretty neat - just not understanding why the variance in behaviour with the content type.

Although - I've just tested in chrome and I get this same behaviour, which, I would expect I think. I just thought the HTTP client would not have this behaviour, as the data is getting to the client (both in chrome and .net HTTP client) from looking at wireshark. It's almost as if when it sees text/event-stream it makes the data avaliable straight away but with application/json it keeps it in an internal buffer? Or is this happening somewhere deeper that HttpClient?

Server side:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.Encodings.Web;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http.Features;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;

namespace AsyncEnumNet3.Controllers
{
    public class AsyncStreamExecutor<T> : IActionResultExecutor<AsyncStreamResult<T>>
    {
        public async Task ExecuteAsync(ActionContext context, AsyncStreamResult<T> result)
        {
            context.HttpContext.Response.StatusCode = 200;
           //if you switch to the other content type then the HTTP client doesnt work as expected
           // context.HttpContext.Response.Headers.Add("Content-Type", "application/json");
            context.HttpContext.Response.Headers.Add("Content-Type", "text/event-stream");
            await context.HttpContext.Response.Body.FlushAsync();

            var sw = new StreamWriter(context.HttpContext.Response.Body);
            var writer = new Newtonsoft.Json.JsonTextWriter(sw);
            var setting = new JsonSerializerSettings() { TypeNameHandling = TypeNameHandling.All };
            var seri = Newtonsoft.Json.JsonSerializer.Create(setting);

            writer.WriteStartArray();
            await writer.FlushAsync();

            int count = 0;
            await foreach (var obj in result.AsyncEnmerable)
            {
                 seri.Serialize(writer, obj);
                await writer.FlushAsync();


                count++;
            }
            writer.WriteEndArray();
            await writer.FlushAsync();
        }
    }

    public class AsyncStreamResult<T> : IActionResult
    {
        public IAsyncEnumerable<T> AsyncEnmerable { get; set; }
        public AsyncStreamResult(IAsyncEnumerable<T> asyncEnmerable)
            {
            this.AsyncEnmerable = asyncEnmerable;
            }

        public Task ExecuteResultAsync(ActionContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException(nameof(context));
            }

            var executor = context.HttpContext.RequestServices.GetRequiredService<IActionResultExecutor<AsyncStreamResult<T>>>();
            return executor.ExecuteAsync(context, this);
        }
    }

    public static class AsyncStreamerExtensionMethods
    {
        public static IServiceCollection AddAsyncEnumerableStreamer<T>(this IServiceCollection services)
        {
            services.TryAddSingleton<IActionResultExecutor<AsyncStreamResult<T>>, AsyncStreamExecutor<T>> ();
            return services;
        }
    }

    public class MyStuff
    {
        public string MyTextThing { get; set; }

        public override string? ToString()
        {
            return MyTextThing;
        }
    }

    #nullable enable
    [ApiController]
    [Route("[controller]")]
    public class ValuesController : ControllerBase
    {
        byte[] data = new byte[0];
        Random r = new Random();

        public ValuesController()
        {


        }

        [HttpGet]
        public async Task<AsyncStreamResult<MyStuff>> Get()
        {

            return new AsyncStreamResult<MyStuff>(GetLines());
        }


        public async IAsyncEnumerable<MyStuff> GetLines()
        {
            for (int i = 0; i < 100; i++)
            {
                using (var sr = new StreamReader(new FileStream("TextFile.txt", FileMode.Open)))
                {
                    string? line = null;
                    while ((line = await sr.ReadLineAsync()) != null)
                    {
                        yield return await AddToString(line);
                    }
                }
            }
        }

        public async Task<MyStuff> AddToString(string indata)
        {
                var stringToTask = indata;
                await Task.Delay(r.Next(0, 100));
                return new MyStuff() { MyTextThing = stringToTask.ToUpper() };
        }
    }
}

Client code

using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Http;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace AsyncEnumClient
{

#nullable enable
    class Program
    {
        static async Task Main(string[] args)
        {
            var host = new HostBuilder();

           await  host.ConfigureServices((ctx, services) =>
            {
                services.AddHttpClient();
                services.AddHostedService<Worker>();
            })
               .RunConsoleAsync();
        }

        public class Worker : BackgroundService
        {
            IHttpClientFactory factory;
            public Worker(IHttpClientFactory factory)
            {
                this.factory = factory;
            }
            protected override async Task ExecuteAsync(CancellationToken stoppingToken)
            {
                await Task.Delay(5000);
                var client =  factory.CreateClient();


                 var reqMsg = new HttpRequestMessage(HttpMethod.Get, "http://127.0.0.1:5000/values");

                var req  = await client.SendAsync(reqMsg, HttpCompletionOption.ResponseHeadersRead);

                using var stream = await req.Content.ReadAsStreamAsync();
                 var str = new StreamReader(stream);

                var reader = new JsonTextReader(str);
                var setting = new JsonSerializerSettings() { TypeNameHandling = TypeNameHandling.All };
                var seri = Newtonsoft.Json.JsonSerializer.Create(setting);

                var  o = new List<object>();

                while (await reader.ReadAsync())
                {
                    if (reader.TokenType == JsonToken.StartObject)
                    {
                       var oo = seri.Deserialize(reader);
                        o.Add(oo);
                        Console.WriteLine(oo);
                    }

                }
            }

        }
    }
}

[Tratcher]

HttpClient doesn't check the content-type. HttpCompletionOption.ResponseHeadersRead is the main flag that enables this streaming behavior.

[rossbuggins]

at the moment the only difference i can see is switiching between the two

  // context.HttpContext.Response.Headers.Add("Content-Type", "application/json");
            context.HttpContext.Response.Headers.Add("Content-Type", "text/event-stream");

I've double checked fiddler and the headers are showing there with either content type selected. Whats HttpClient looking for when set for HttpCompletionOption.ResponseHeadersRead?

[rossbuggins]

just looking in HttpClient SendAsync and this is only place i can see a decision based on the completion option,

   return completionOption == HttpCompletionOption.ResponseContentRead && !string.Equals(request.Method.Method, "HEAD", StringComparison.OrdinalIgnoreCase) ?
                FinishSendAsyncBuffered(sendTask, request, cts, disposeCts) :
                FinishSendAsyncUnbuffered(sendTask, request, cts, disposeCts);

So the HttpClient then uses HttpClientHandler? So is it going to be something in there or the handlers pipeline?

[Tratcher]

at the moment the only difference i can see is switiching between the two

  // context.HttpContext.Response.Headers.Add("Content-Type", "application/json");
            context.HttpContext.Response.Headers.Add("Content-Type", "text/event-stream");

Which one is delayed? text/event-stream?

Are you using IIS Express? It's dynamic compression module looks for text/* and it partially buffers during compression.

[rossbuggins]

when using application/json client.GetAsync is delayed until all data is at the client.

Using .net core host:

    public class Program
    {
        public static void Main(string[] args)
        {
            CreateHostBuilder(args).Build().Run();
        }

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)

                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                });

    }

 public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {

            services.AddAsyncEnumerableStreamer<MyStuff>();
            services.AddControllers(options =>
            {
                options.MaxIAsyncEnumerableBufferLimit = 1000000;
            });
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            app.UseHttpsRedirection();
            app.UseRouting();
            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }
    }

[rossbuggins]

3.0.0-preview7.19365.7 in use

[rossbuggins]

Hi - so I've just tried this on Ubuntu running core 3 preview 7 and the behaviour is what I expected - i.e. no difference between setting content type to be application/json or text/event-stream.

This is not the case on windows, where when content type is application/json the call to HttpClient
SendAsync with HttpCompletionOption.ResponseHeadersRead set does not return until all the content is received

[rossbuggins]

Hi, Any thoughts on this? With different behaviour on different platforms?

[Tratcher]

No, there shouldn't be a difference here across platforms. I suggest collecting full client and server logs along with wireshark traces for both scenarios and we can compare them.

[rossbuggins]

So trace level logging in the apps for both server and client?

[i9ii9i]

I've quickly knocked up the below ActionResultExecutor to handle a proof of concept. Strange thing though, if i set content type to application/json then HttpClient doesn't return to ReadAsync until the whole request is finished, even though i can see with wireshark (and server memory) that the data is being sent, but if i use text/event-stream then HttpClient is responding each time the buffer is flushed on the server?

```

Thank you, I've been looking for this.
I was able to get your code working on a Blazor server-side proj, where I get a stream from a SQL DB and return to an HTML table rendering rows as they stream.

For some reason I cannot get this to work with Blazor Client-side (wasm). It seems like the client is buffering the full response before it starts rendering.

If you wouldn't mind maybe you can share some suggestions.
Best wishes.

Client-side code below: (sorry about the code formatting)

`protected override async Task OnInitializedAsync()
{
weatherForecasts = new List();

await foreach (var weatherForecast in GetDataAsync())
{
    weatherForecasts.Add(weatherForecast);
    records = weatherForecasts.Count();
    this.StateHasChanged();
}

}

public async IAsyncEnumerable GetDataAsync()
{
var serializer = new JsonSerializer();

using (var stream = await Http.GetStreamAsync("https://localhost:44334/api/values/i"))
{
    using (var sr = new StreamReader(stream))
    using (var jr = new JsonTextReader(sr))
    {
        while (await jr.ReadAsync())
        {
            if (jr.TokenType != JsonToken.StartArray && jr.TokenType != JsonToken.EndArray)
            {
                yield return serializer.Deserialize<SomeClass>(jr);
            }
        };
    }
}

}`

[msftbot[bot]]

Thank you for contacting us. Due to no activity on this issue we're closing it in an effort to keep our backlog clean. If you believe there is a concern related to the ASP.NET Core framework, which hasn't been addressed yet, please file a new issue.