Sqlclient: Bug | Fix Connection Resiliency for Linux/MacOS clients

[aaomidi]

.NET Core 2.2

    <PackageReference Include="Microsoft.Data.SqlClient" Version="1.1.0" />
    <PackageReference Include="Microsoft.SqlServer.SqlManagementObjects" Version="160.1911221.0-preview" />

Repro: https://github.com/aaomidi/smo-test/blob/master/Program.cs

Repro of the issue:

1. Create a connection to a SQL Server on MacOS
2. Run a query using that connection.
3. Disconnect from internet.
4. Run a query again, wait for timeout.
5. Connect to internet after the timeout.
6. Using the same connection from the first step, run another query.
   
   
   
   • Expected: Recreates the connection and returns the result.
   
   
   • Actual: Fails with the following error:
   
   

Microsoft.SqlServer.Management.Smo.FailedOperationException: Execute with results failed for Database 'master'.  ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
   at System.Net.Sockets.Socket.set_ReceiveTimeout(Int32 value)
   at Microsoft.Data.SqlClient.SNI.SNITCPHandle.Receive(SNIPacket& packet, Int32 timeoutInMilliseconds) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SNI\SNITcpHandle.cs:line 453
   at Microsoft.Data.SqlClient.SNI.TdsParserStateObjectManaged.ReadSyncOverAsync(Int32 timeoutRemaining, UInt32& error) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObjectManaged.cs:line 136
   at Microsoft.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 2178
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 2137
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 1093
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte& value) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 1370
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 1773
   at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 1679
   at Microsoft.Data.SqlClient.TdsParser.DrainData(TdsParserStateObject stateObj) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 4187
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.ValidateConnectionForExecute(SqlCommand command) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnectionTds.cs:line 706
   at Microsoft.Data.SqlClient.SqlCommand.ValidateCommand(Boolean isAsync, String method) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 4622
   at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 3934
   at Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 1759
   at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet)
   at Microsoft.SqlServer.Management.Common.ConnectionManager.ExecuteTSql(ExecuteTSqlAction action, Object execObject, DataSet fillDataSet, Boolean catchException) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ConnectionManager.cs:line 742
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand, Boolean retry) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1325
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1261
   at Microsoft.SqlServer.Management.Smo.ExecutionManager.ExecuteWithResults(StringCollection query) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\ExecutionManager.cs:line 664
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2202
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2217
   at SmoTestPackage.Program.DoQuery(Database db) in /Users/amir/Projects/smo-test/Program.cs:line 45
   at SmoTestPackage.Program.Main(String[] args) in /Users/amir/Projects/smo-test/Program.cs:line 33

I have confirmed that this issue does not exist in Windows.

I have not confirmed if this issue repros in Linux.

[aaomidi]

This is the error I get when I let it timeout on MacOS (Step 4):

Microsoft.SqlServer.Management.Smo.FailedOperationException: Execute with results failed for Database 'master'.  ---> System.Net.Sockets.SocketException: Invalid argument
   at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, String callerName)
   at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
   at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
   at System.Net.Sockets.Socket.set_ReceiveTimeout(Int32 value)
   at Microsoft.Data.SqlClient.SNI.SNITCPHandle.Receive(SNIPacket& packet, Int32 timeoutInMilliseconds) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SNI\SNITcpHandle.cs:line 453
   at Microsoft.Data.SqlClient.SNI.TdsParserStateObjectManaged.ReadSyncOverAsync(Int32 timeoutRemaining, UInt32& error) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObjectManaged.cs:line 136
   at Microsoft.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 2178
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 2137
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 1093
   at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte& value) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParserStateObject.cs:line 1370
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 1773
   at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlDataReader.cs:line 1135
   at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlDataReader.cs:line 263
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 4519
   at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 4337
   at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 4035
   at Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlCommand.cs:line 1759
   at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet)
   at Microsoft.SqlServer.Management.Common.ConnectionManager.ExecuteTSql(ExecuteTSqlAction action, Object execObject, DataSet fillDataSet, Boolean catchException) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ConnectionManager.cs:line 742
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand, Boolean retry) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1325
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1261
   at Microsoft.SqlServer.Management.Smo.ExecutionManager.ExecuteWithResults(StringCollection query) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\ExecutionManager.cs:line 664
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2202
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2217
   at SmoTestPackage.Program.DoQuery(Database db) in /Users/amir/Projects/smo-test/Program.cs:line 45
   at SmoTestPackage.Program.Main(String[] args) in /Users/amir/Projects/smo-test/Program.cs:line 33

This is the error I get when I timeout on Windows (Step 4):

Microsoft.SqlServer.Management.Smo.FailedOperationException: Execute with results failed for Database 'master'.  ---> Microsoft.SqlServer.Management.Common.ExecutionFailureException: An exception occurred while executing a Transact-SQL statement or batch. ---> Microsoft.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 0 - No such host is known.) ---> System.ComponentModel.Win32Exception: No such host is known
   --- End of inner exception stack trace ---
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnection.cs:line 615
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 1183
   at Microsoft.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, Boolean withFailover, SqlAuthenticationMethod authType, SqlAuthenticationProviderManager sqlAuthProviderManager) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\TdsParser.cs:line 373
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnectionTds.cs:line 1741
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnectionTds.cs:line 1408
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnectionTds.cs:line 1311
   at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool, SqlAuthenticationProviderManager sqlAuthProviderManager) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlInternalConnectionTds.cs:line 439
   at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlConnectionFactory.cs:line 135
   at Microsoft.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Common\src\Microsoft\Data\ProviderBase\DbConnectionFactory.cs:line 98
   at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\ProviderBase\DbConnectionFactory.cs:line 50
   at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Common\src\Microsoft\Data\ProviderBase\DbConnectionInternal.cs:line 328
   at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlConnection.cs:line 1467
   at Microsoft.Data.SqlClient.SqlConnection.Open() in E:\agent1\_work\31\s\src\Microsoft.Data.SqlClient\netcore\src\Microsoft\Data\SqlClient\SqlConnection.cs:line 981
   at Microsoft.SqlServer.Management.Common.ConnectionManager.ExecuteTSql(ExecuteTSqlAction action, Object execObject, DataSet fillDataSet, Boolean catchException) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ConnectionManager.cs:line 780
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand, Boolean retry) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1325
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand, Boolean retry) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1330
   at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\ConnectionInfo\ServerConnection.cs:line 1261
   at Microsoft.SqlServer.Management.Smo.ExecutionManager.ExecuteWithResults(StringCollection query) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\ExecutionManager.cs:line 664
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2202
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.Smo.Database.ExecuteWithResults(StringCollection sqlCommands) in d:\a\1\s\src\Microsoft\SqlServer\Management\Smo\DatabaseBase.cs:line 2217
   at SmoTestPackage.Program.DoQuery(Database db) in C:\Users\amomidi\Documents\smo-test\Program.cs:line 45
   at SmoTestPackage.Program.Main(String[] args) in C:\Users\amomidi\Documents\smo-test\Program.cs:line 33

[cheenamalhotra]

Hi @aaomidi

Connection Resiliency was never implemented for MacOS/Linux with Managed SNI, so I guess this is expected. Ref comment: https://github.com/dotnet/SqlClient/pull/306#issuecomment-554027722

I'm going to convert it to Feature Request 😄

[TheDruidsKeeper]

In case the additional info is useful, I ran into the same error today with a slightly different stack trace:

System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 35 - An internal exception was caught) ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
   at System.Net.Sockets.Socket.set_NoDelay(Boolean value)
   at System.Data.SqlClient.SNI.SNITCPHandle..ctor(String serverName, Int32 port, Int64 timerExpire, Object callbackObject, Boolean parallel)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()

Using the (linux) mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim base docker image with the application using dotnet core 2.2.

I got the error after this other error that I didn't get a stack trace for:

System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (258): Unknown error 258

but once the socket error happened the sql connection pool got into an unusable state, all attempts to get a connection failing with: The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

[saurabh500]

There is no feature gap specific reason that connection resiliency shouldn't work on non-Windows platform. This is definitely a bug where we are trying to reuse a disposed socket. There was no feature parity gap with Connection Resiliency on the SqlClient driver on nix platform.

I am only referring to the original stack posted here.

[David-Engel]

I believe this is a duplicate of #304 which was fixed in #310 and ported to System.Data.SqlClient 4.8.1 released with .NET Core 3.1.2 last week.

[PashaPash]

Got similar issue on .NET Core 3.0 / Linux with Microsoft.Data.SqlClient 1.1.1:

An unhandled exception was thrown by the application.Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 35 - An internal exception was caught) ---> System.ObjectDisposedException: Cannot access a disposed object. Object name: 'System.Net.Sockets.Socket'. at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue) at Microsoft.Data.SqlClient.SNI.SNITCPHandle..ctor(String serverName, Int32 port, Int64 timerExpire, Object callbackObject, Boolean parallel) at Microsoft.Data.ProviderBase.DbConnectionPool.CheckPoolBlockingPeriod(Exception e) at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) at Microsoft.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()

For some reason, is does not fail immediately on .OpenAsync call. It hangs for 60+ seconds, and then fails with SqlException / ObjectDisposedException, so just retrying on error does not resolve the issue completely.

[frankyuan]

Hi, I met a similar issue, not sure if it's the same issue. If it is duplicated, let me know and I will remove this, thanks. The log is like below, only a little part of queries has this issue. The application runs in Linux docker.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.\n
---> System.ComponentModel.Win32Exception (258): Unknown error 258\n
at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction)\n at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\n at Microsoft.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)\n at Microsoft.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()\n at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()\n at Microsoft.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()\n at Microsoft.Data.SqlClient.TdsParserStateObject.TryReadByteArray(Span1 buff, Int32 len, Int32& totalRead)\n
at Microsoft.Data.SqlClient.TdsParser.TrySkipValue(SqlMetaDataPriv md, Int32 columnOrdinal, TdsParserStateObject stateObj)\n
at Microsoft.Data.SqlClient.TdsParser.TrySkipRow(_SqlMetaDataSet columns, Int32 startCol, TdsParserStateObject stateObj)\n
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\n
at Microsoft.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)\n
at Microsoft.Data.SqlClient.SqlDataReader.ReadAsync(CancellationToken cancellationToken)\n
--- End of stack trace from previous location where exception was thrown ---\n
at Dapper.SqlMapper.QueryAsyncT in /_/Dapper/SqlMapper.Async.cs:line 437\n

[cheenamalhotra]

Hi @frankyuan

Please provide a repro for your scenario to analyze. Please create a new ticket with it.
Thanks!

[frankyuan]

Sure, I will create a new ticket, thanks.