Kestrelhttpserver: Kestrel send empty response over custom SSL

[Ph47]

I have constantly repeated exceptions:

Microsoft.AspNetCore.Server.Kestrel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60: Error: ConnectionFilter.OnConnection

System.AggregateException: One or more errors occurred. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionFilter.<OnConnectionAsync>d__3.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionFilter.<OnConnectionAsync>d__3.MoveNext()<---

Microsoft.AspNetCore.Server.Kestrel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60: Error: ConnectionFilter.OnConnection

System.AggregateException: One or more errors occurred. ---> System.IO.IOException: The handshake failed due to an unexpected packet format.
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionFilter.<OnConnectionAsync>d__3.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.IO.IOException: The handshake failed due to an unexpected packet format.
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionFilter.<OnConnectionAsync>d__3.MoveNext()<---

My Main is like:

        public static void Main(string[] args)
        {
            ContainerBuilder builder = new ContainerBuilder();
            string folder = Directory.GetCurrentDirectory() + "\\";

            // register configuration
            builder.RegisterInstance(
                new ConfigurationBuilder()
                    .SetBasePath(folder)
                    .AddJsonFile("config.json")
                    .AddEnvironmentVariables()
                    .AddCommandLine(args)
                    .Build()
                ).As<IConfiguration>();

            Container = builder.Build();

            X509Certificate2 httpsCert = new X509Certificate2(folder + Container.Resolve<IConfiguration>()["Hosting:PfxPath"], Container.Resolve<IConfiguration>()["Hosting:PfxPassword"]);


            // build and launch web server
            new WebHostBuilder()
                .UseConfiguration(Container.Resolve<IConfiguration>())
                .UseContentRoot(folder)                
                .UseKestrel(options => {
                    options.NoDelay = true;
                    options.UseHttps(new HttpsConnectionFilterOptions() {
                         ServerCertificate = httpsCert,
                         SslProtocols = SslProtocols.Tls12
                    });
                })
                .UseUrls(Container.Resolve<IConfiguration>()["Hosting:HttpUrl"], Container.Resolve<IConfiguration>()["Hosting:HttpsUrl"])
                .UseStartup<Startup>()
                .Build()
                .Run();
        }     

My project.json:

{
  "dependencies": {
    "Autofac": "4.0.0-*",
    "Hangfire": "1.5.4",
    "Hangfire.Autofac": "2.2.0",
    "HttpMultipartParser": "2.1.3",
    "ImageProcessor": "2.3.3",
    "Lucene.Net": "3.0.3",
    "AutoMapper": "4.2.1",
    "NReco.PdfGenerator": "1.1.12",
    "MySql.Data": "6.9.8",
    "Nancy": "1.4.3",
    "Nancy.Validation.DataAnnotations": "1.4.1",
    "Nancy.Owin": "1.4.1",
    "Nancy.Bootstrappers.Autofac": "1.4.1",
    "Newtonsoft.Json": "8.0.3",
    "Nancy.Serialization.JsonNet": "1.4.1",
    "MailKit": "1.2.23",
    "IdentityServer3": "2.5.0",

    "Microsoft.AspNetCore.StaticFiles": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Diagnostics": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Owin": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Server.Kestrel": "1.0.0-rc2-final",
    "Microsoft.AspNetCore.Server.Kestrel.Https": "1.0.0-rc2-final",

    "Microsoft.EntityFrameworkCore": "1.0.0-rc2-final",
    "Microsoft.EntityFrameworkCore.SqlServer": "1.0.0-rc2-final",

    "Microsoft.Extensions.Configuration": "1.0.0-rc2-final",
    "Microsoft.Extensions.Configuration.CommandLine": "1.0.0-rc2-final",
    "Microsoft.Extensions.Configuration.EnvironmentVariables": "1.0.0-rc2-final",
    "Microsoft.Extensions.Configuration.Json": "1.0.0-rc2-final",
    "Microsoft.Extensions.Logging": "1.0.0-rc2-final",
    "Microsoft.Extensions.Logging.Console": "1.0.0-rc2-final",
    "Microsoft.Extensions.Logging.Debug": "1.0.0-rc2-final",
    "Microsoft.Extensions.Options": "1.0.0-rc2-final",
    "Microsoft.Extensions.PlatformAbstractions": "1.0.0-rc2-final",
    "Autofac.Extensions.DependencyInjection": "4.0.0-*"
  },

  "tools": {
    "Microsoft.AspNetCore.Server.IISIntegration.Tools": {
      "version": "1.0.0-preview1-final",
      "imports": "portable-net45+win8+dnxcore50"
    }
  },

  "frameworks": {
    "net461": {
      "frameworkAssemblies": {
        "System.Runtime": "",
        "System.Runtime.Serialization": "",
        "System.Runtime.Serialization.Primitives": "",
        "System.Reflection": "",
        "System.Data": "",
        "System.ComponentModel": "",
        "System.Drawing": "",
        "System.Collections": ""
      }
    }
  },

  "buildOptions": {
    "emitEntryPoint": true,
    "preserveCompilationContext": true
  },

  "publishOptions": {
    "include": [
      "wwwroot",
      "web.config",
      "cert.pfx",
      "config.json"
    ]
  },

  "scripts": {
    "postpublish": [ "dotnet publish-iis --publish-folder %publish:OutputPath% --framework %publish:FullTargetFramework%" ]
  }
}

My config contains:

  "Hosting": {
    "HttpUrl": "http://localhost:4748/",
    "HttpsUrl": "https://localhost:4747/",
    "PfxPath": "cert.pfx",
    "PfxPassword": "testPassword"
  },

I trieed:

• many different certs include test one in samples folder;
• admin / user rights;
• install certs into different locations;
• all SslProtocols avaiable;
• VS and console dotnet run.
  My windows is 8 x64. I am using 4747 tcp port.

UPD: when i am trying UseConnectionLogging and Null Exception throw. It is strange behavior.
UPD: i tried different ports (and 443) and it is same exception.

[cesarblum]

What's in your Startup.cs? Can you upload the app (or a stripped down version of it) somewhere so we can try to reproduce the issue?

[halter73]

@Ph47 Does HTTPS work for you when you run the KestrelSampleApp?

If you can't provide use with a full repro, since this is an issue with the handshake, maybe you dould take a look at the connection with message analyzer or something similar?

[Ph47]

Sample works well. The example also works well with RC2 in project.json.
Purified project what runs and work not very well:
WebApp.zip
I am going to Troubleshooting TLS and investigating.

[Ph47]

I found one strange behavior. Copypasting test config into my project:

.UseKestrel(options =>
{
   options.NoDelay = true;
   options.UseHttps(httpsCert);
   options.UseConnectionLogging();
})

breaks with exception from DI "Value can not be null" at UseConnectionLogging.
This means there are differences in the projects that I have not noticed. So Autofac can be source of problem.

[davidfowl]

@Ph47 we have an issue with Autofac and Kestrel in RC2.

[Ph47]

I just figured it out. There are ways to get around?

[Ph47]

My SSL problem depends on this. Revert to standart DI solve primary problem. Link Autofac issue please.

[davidfowl]

https://github.com/autofac/Autofac/issues/755

[Ph47]

My workaround while issue in Autofac discussed listed below:

#region Autofac integration
   ContainerBuilder servicesUpdateBuilder = new ContainerBuilder();
   servicesUpdateBuilder.Populate(services.Reverse());
   servicesUpdateBuilder.Update(Container);
   return Container.Resolve<IServiceProvider>();
#endregion

https://msdn.microsoft.com/en-us/library/bb358497(v=vs.100).aspx

But don't close issue, workaround fix null exception but don't fix SSL problem. Fallback to "factory" DI fix problem.

[Ph47]

Dividing all DI registration into two flow (dotnet and others include my owns) and merging them at very end of registration process with inversion what i posted before temporary solve problem until Autofac updates.

[muratg]

@halter73 and @mikeharder have some ideas about this one.

[halter73]

@Ph47 I took a look at WebApp.zip, and I think I found the remaining problem. You populate Container twice with services from the IServiceCollection. Once in the "Autofac DI" region where you call builder.Populate(services.Reverse()); and then builder.Update(Containter). You then make the same two calls in the "Autofac integration" region, but with servicesUpdateBuilder instead of builder.

This causes your UseKestrel() callback gets invoked twice (Options callbacks are wrapped by a service and registered with DI) which in turn options.UseHttps(...) to also be called twice. The end result is that Kestrel attempting to double decrypt/encrypt leading to the exceptions you see an no response.

It's possible, but a little complicated, to try to catch this type of error a bit sooner so we can give a clearer error.

The lowest level we could probably catch this is inside of OptionsManager. If OptionsManager resolves multiple identical instances of IConfigureOptions<TOptions> that's almost surely a registration error. I think this is the best approach if we decide this double registration error is worth proactively identifying and reporting.

Alternatively, we could throw an error in the HttpsConnectionFilter if the previous IConnectionFilter is also an HttpsConnectionFilter. This wouldn't catch the issue if the UseKestrel() callback registers multiple filters (such as logging _and_ HTTPS like our sample app does) or if now connection filters were used at all.

P.S. You might want to look into using IHostingEnvironment.ContentRootPath instead of Directory.GetCurrentDirectory() to find the cert file. This would probably require calling options.ApplicationServices.GetService<IHostingEnvironment>() in the UseKestrel() callback.