Runtime: SslStream.AuthenticateAsServerAsync failing with "System.ComponentModel.Win32Exception (0x80004005): The credentials supplied to the package were not recognized"

[halter73]

This seems to be a possible reoccurrence of dotnet/runtime#20356 and/or dotnet/runtime#23221.

Kestrel's LoggingConnectionAdapterTests.LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter test failed with "System.ComponentModel.Win32Exception (0x80004005): The credentials supplied to the package were not recognized" being thrown from SslStream.AuthenticateAsServerAsync(). This was seen in a Windows x64 Jenkins run run to validate aspnet/KestrelHttpServer#2844.

This error was also reported by a customer, but is difficult to reproduce. This is my first time seeing this type of failure outside of the customer report. Below are the client and server side logs (both of which are using SslStream) including the raw bytes sent by the client to initiate the handshake.

` 15:55:59 Starting test group: UnitTests/net461 15:55:59 Microsoft (R) Test Execution Command Line Tool Version 15.7.0 15:55:59 Copyright (c) Microsoft Corporation. All rights reserved. 15:55:59 15:55:59 Starting test execution, please wait... 15:56:17 [xUnit.net 00:00:13.40] LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter [FAIL] 15:56:17 Failed LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter 15:56:17 Error Message: 15:56:17 System.IO.IOException : Authentication failed because the remote party has closed the transport stream. 15:56:17 Stack Trace: 15:56:17 at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) 15:56:17 at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result) 15:56:17 at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) 15:56:17 at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 at Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.TestTransport.InMemoryHttpClientSlim.<GetStream>d__9.MoveNext() 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 at Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.TestTransport.InMemoryHttpClientSlim.<GetStringAsync>d__3.MoveNext() 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 at Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.TestTransport.InMemoryHttpClientSlim.<GetStringAsync>d__2.MoveNext() 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 at Microsoft.AspNetCore.Testing.TaskExtensions.<TimeoutAfter>d__0`1.MoveNext() 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 at Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.LoggingConnectionAdapterTests.<LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter>d__0.MoveNext() 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 --- End of stack trace from previous location where exception was thrown --- 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 Standard Output Messages: 15:56:17 | [0.000s] TestLifetime Information: Starting test LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter at 2018-08-23T22:56:07 15:56:17 | [0.004s] Microsoft.AspNetCore.Hosting.Internal.WebHost Debug: Hosting starting 15:56:17 | [0.294s] Microsoft.AspNetCore.Hosting.Internal.WebHost Debug: Hosting started 15:56:17 | [0.294s] Microsoft.AspNetCore.Hosting.Internal.WebHost Debug: Loaded hosting startup assembly testhost.x86 15:56:17 | [0.298s] Microsoft.AspNetCore.Server.Kestrel Debug: Connection id "0HLG99EEA0QKF" started. 15:56:17 | [0.509s] LoggingConnectionAdapter Debug: ReadAsync[5] 16 03 03 00 A8 15:56:17 | \x16\x03\x03\0" 15:56:17 | [0.512s] LoggingConnectionAdapter Debug: ReadAsync[168] 01 00 00 A4 03 03 5B 7F 3B 88 EE D6 80 6D 86 95 B5 B8 64 3F C1 A8 D3 EE 7A 11 04 4D A3 97 29 45 E9 8B A4 2A D6 C4 00 00 34 C0 28 C0 27 C0 14 C0 13 00 9F 00 9E 00 39 00 33 00 9D 00 9C 00 3D 00 3C 00 35 00 2F C0 2C C0 2B C0 24 C0 23 C0 0A C0 09 00 6A 00 40 00 38 00 32 00 0A 00 13 01 00 00 47 00 00 00 0E 00 0C 00 00 09 6C 6F 63 61 6C 68 6F 73 74 00 0A 00 06 00 04 00 17 00 18 00 0B 00 02 01 00 00 0D 00 14 00 12 06 01 06 03 04 01 05 01 02 01 04 03 05 03 02 03 02 02 00 23 00 00 00 17 00 00 FF 01 00 01 00 15:56:17 | \x01\0\0\x03\x03[;?��?m??�,d?A"O�z\x11\x04M�?)E�?*��\0\04A(A'A\x14A\x13\0?\0?\09\03\0?\0?\0=\0<\05\0/A,A+A$A#A 15:56:17 A \0j\0@\08\02\0 15:56:17 \0\x13\x01\0\0G\0\0\0\x0e\0 \0\0 localhost\0 15:56:17 \0\x06\0\x04\0\x17\0\x18\0 \0\x02\x01\0\0 15:56:17 \0\x14\0\x12\x06\x01\x06\x03\x04\x01\x05\x01\x02\x01\x04\x03\x05\x03\x02\x03\x02\x02\0#\0\0\0\x17\0\0�\x01\0\x01\0 15:56:17 | [1.099s] Microsoft.AspNetCore.Server.Kestrel Error: Uncaught exception from the OnConnectionAsync method of an IConnectionAdapter. 15:56:17 | System.ComponentModel.Win32Exception (0x80004005): The credentials supplied to the package were not recognized 15:56:17 | at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) 15:56:17 | at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result) 15:56:17 | at System.Net.Security.SslStream.EndAuthenticateAsServer(IAsyncResult asyncResult) 15:56:17 | at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) 15:56:17 | --- End of stack trace from previous location where exception was thrown --- 15:56:17 | at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 | at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 15:56:17 | at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) 15:56:17 | at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionAdapter.<InnerOnConnectionAsync>d__10.MoveNext() 15:56:17 | --- End of stack trace from previous location where exception was thrown --- 15:56:17 | at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 15:56:17 | at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.HttpConnection.<ApplyConnectionAdaptersAsync>d__52.MoveNext() 15:56:17 | [1.178s] Microsoft.AspNetCore.Server.Kestrel Debug: Connection id "0HLG99EEA0QKF" disconnecting. 15:56:17 | [3.888s] Microsoft.AspNetCore.Server.Kestrel Debug: Connection id "0HLG99EEA0QKF" stopped. 15:56:17 | [6.926s] Microsoft.AspNetCore.Hosting.Internal.WebHost Debug: Hosting shutdown 15:56:17 | [9.902s] TestLifetime Information: Finished test LoggingConnectionAdapterCanBeAddedBeforeAndAfterHttpsAdapter in 9.9021088s 15:56:17 15:56:17 15:56:18 [xUnit.net 00:00:14.00] ContentLength_Received_MultipleDataFramesOverSize_Reset [SKIP] 15:56:18 Skipped ContentLength_Received_MultipleDataFramesOverSize_Reset 15:56:22 Results File: D:\j\workspace\windows-Confi---42185a5f\artifacts\logs\UnitTests-net461-156.trx 15:56:22 15:56:22 Total tests: 6478. Passed: 6476. Failed: 1. Skipped: 1. 15:56:22 Test Run Failed. 15:56:22 Test execution time: 20.4468 Seconds 15:56:22 C:\Users\dotnet-bot\.dotnet\buildtools\korebuild\2.2.0-preview1-20180807.2\modules\vstest\module.targets(108,5): error : Test group UnitTests/net461 failed with exit code '1'. [C:\Users\dotnet-bot\.dotnet\buildtools\korebuild\2.2.0-preview1-20180807.2\KoreBuild.proj]

[abatishchev]

Hi, please ping alexbat if you need more info on this. I tried to use high port (18020 rather than 8020) but still getting this error if run VS not under admin.

[davidsh]

System.ComponentModel.Win32Exception (0x80004005): The credentials supplied to the package were not recognized

Typically this error occurs on Windows when PFX files are used. In particular, importing the same PFX files in parallel will lead to these kind of errors due to bugs in Windows OS components. The workaround is to reduce parallelism in running tests (such as XUnit) which import PFX files. There is no known fix for this Windows bug/limitation.

but still getting this error if run VS not under admin.

The other possibility of this error is when PFX files are imported but the private key permissions don't allow non-admins to access the private key.

Both of these issues are not something that CoreFx can fix.