Embedio: Large response exceeding Modules.FileModuleBase.ChunkSize not sent fully

[nkundu]

Describe the bug
A large response (exceeding Modules.FileModuleBase.ChunkSize) sent via the extension method JsonResponseAsync is not sent fully.

When inspecting in Wireshark I see a PSH then RST being sent before the whole response body is streamed.

I was able to fix by replacing the implementation of the extension method WriteToOutputStream with (in other words, let the framework do the copy)

        public static async Task WriteToOutputStream(
            this IHttpResponse response,
            Stream buffer,
            long lowerByteIndex,
            CancellationToken ct) {

            buffer.Seek(lowerByteIndex, SeekOrigin.Begin);
            await buffer.CopyToAsync(response.OutputStream);
        }

I think somewhere Flush might be getting called on the stream prematurely.

To Reproduce
Call return await controller.JsonResponseAsync(myString) where myString.Length exceeds 256 * 1024

I will include a test program and screenshots when I have a chance to remove private information from them.

Os Win10 x64, .net core 2
Edit: EmbedIO v 2.2.3

[nkundu]

Here is a sample test program that reproduces the bug.

To reproduce, you can run this and hit http://localhost:9697/bigresponse1 from Chrome or Postman.
Postman returns an error, and Chrome displays partial content and shows an error in console.

Replacing WriteToOutputStream with the code above eliminates the issue. If you think that's an acceptable solution I can prepare a PR.

TestController.cs

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
using Unosquare.Labs.EmbedIO;
using Unosquare.Labs.EmbedIO.Constants;
using Unosquare.Labs.EmbedIO.Modules;

namespace EmbedIOTestProgram {
    public class TestController : WebApiController {
        private IHttpContext context;

        public TestController(IHttpContext context) : base(context) {
            this.context = context;
        }

        [WebApiHandler(HttpVerbs.Get, "/bigresponse1")]
        public async Task<bool> BigResponse1() {
            // setting gzip = false so that the actual content-length is guaranteed to be > Modules.FileModuleBase.ChunkSize
            // even if the client accepts gzip
            return await this.JsonResponseAsync(MakeBigString(), useGzip: false);
        }

        private static string MakeBigString(int length = 256 * 1024 + 1) {
            // just make any garbage string that's big enough
            StringBuilder sb = new StringBuilder();
            sb.Append('"');
            while (sb.Length < length) {
                sb.Append(Guid.NewGuid().ToString());
                sb.Append(", ");
            }
            sb.Remove(sb.Length - 2, 2);
            sb.Append('"');

            return sb.ToString();
        }
    }
}

Program.cs

using System;
using System.Threading;
using Unosquare.Labs.EmbedIO;
using Unosquare.Labs.EmbedIO.Constants;
using Unosquare.Labs.EmbedIO.Modules;

namespace EmbedIOTestProgram {
    class Program {
        static void Main(string[] args) {
            Thread t = new Thread(RunServer);
            t.Start();

            Console.ReadLine();
        }

        private static async void RunServer() {
            using (var server = new WebServer("http://localhost:9697/", RoutingStrategy.Wildcard)) {

                server.RegisterModule(new WebApiModule());
                server.Module<WebApiModule>().RegisterController<TestController>();
                // Once we've registered our modules and configured them, we call the RunAsync() method.
                await server.RunAsync();
            }
        }
    }
}

[geoperez]

We released a new version, you can check nuget 2.2.5.

cc @bufferUnderrun

[bufferUnderrun]

it seems fixed after some tests.
I will deploy in prod for few users.

[geoperez]

@nkundu ??

[nkundu]

I pulled 2.2.5 but it did not resolve the issue (using the test program from my earlier comment). I've not had a chance to debug why, but the symptom is exactly the same. Wireshark shows a PSH then RST.

In my local development I'm just using a different extension method, which does not exhibit the problem.

````
public static async Task WriteToOutputStream(
this IHttpResponse response,
Stream buffer,
long lowerByteIndex,
CancellationToken ct) {

        buffer.Seek(lowerByteIndex, SeekOrigin.Begin);
        await buffer.CopyToAsync(response.OutputStream);
    }

````

Here is a screenshot

[k3z0]

Hi @nkundu

We just released a new nuget version (2.2.6), could you please give it a try and let us know is the issue is solved?

[nkundu]

Hi @k3z0

Confirmed 2.2.6 if working for me! I'll upgrade our main project to that version. Thank you for the excellent hosting framework, it's been fast and flexible for us!