Aws-sdk-net: SSL Cert Invalid When Connecting To MCS (Managed Cassandra Service)

[classifieds-dev]

Hello,

I'm trying to connect to MCS (managed Cassandra service). The info out there is sparse but I believe I have cobbled together the proper code to do so with references from the aws python mcs and c# mongodb connection docs. That code can be found below.

            var pathToCAFile = $"{Directory.GetCurrentDirectory()}/AmazonRootCA1.pem";
            X509Certificate2[] certs = new X509Certificate2[] { new X509Certificate2(pathToCAFile, "amazon") };
            lambdaContext.Logger.LogLine($"CERT DATA: {certs[0]}");
            X509Certificate2Collection certificateCollection = new X509Certificate2Collection(certs);
            var options = new Cassandra.SSLOptions(SslProtocols.Tls11, true, ValidateServerCertificate);
            options.SetCertificateCollection(certificateCollection);
            Cluster cluster = Cluster
                                .Builder()
                                .WithCredentials("<username>", "<password>")
                                .WithPort(9142)
                                .AddContactPoint("cassandra.us-east-1.amazonaws.com")
                                .WithSSL(options)
                                .Build();
            try
            {
                var session = cluster.Connect("<TableName>");
                lambdaContext.Logger.LogLine("CONNECTED TO CASSANDRA!!!!");
            } catch(Exception e)
            {
                lambdaContext.Logger.LogLine(JsonConvert.SerializeObject(e));
            }

When this code is executed within aws lambda the following log entry is generated in CloudWatch.

{
    "ClassName": "Cassandra.NoHostAvailableException",
    "Message": "All hosts tried for query failed (tried 3.83.169.154:9142: AuthenticationException 'The remote certificate is invalid according to the validation procedure.')",
    "Data": null,
    "InnerException": null,
    "HelpURL": null,
    "StackTraceString": "   at Cassandra.Connections.ControlConnection.Connect(Boolean isInitializing)\n   at Cassandra.Connections.ControlConnection.InitAsync()\n   at Cassandra.Tasks.TaskHelper.WaitToCompleteAsync(Task task, Int32 timeout)\n   at Cassandra.Cluster.Init()\n   at Cassandra.Cluster.ConnectAsync(String keyspace)\n   at Cassandra.Tasks.TaskHelper.WaitToComplete(Task task, Int32 timeout)\n   at Cassandra.Tasks.TaskHelper.WaitToComplete[T](Task`1 task, Int32 timeout)\n   at Cassandra.Cluster.Connect(String keyspace)\n   at Chat.Handler.FunctionHandlerAsync(APIGatewayProxyRequest apiGatewayRequest, ILambdaContext lambdaContext) in /Users/***/Handler.cs:line 60",
    "RemoteStackTraceString": null,
    "RemoteStackIndex": 0,
    "ExceptionMethod": null,
    "HResult": -2146233088,
    "Source": "Cassandra",
    "WatsonBuckets": null
}

After going down a rabbit I eventually came to a post here regarding connecting Cassandra using c# on google groups.

https://groups.google.com/a/lists.datastax.com/forum/#!topic/csharp-driver-user/nID_q2Btxec

The response in that thread mentions using this method to validate / verify the cert.

https://docs.microsoft.com/en-us/dotnet/api/system.net.security.sslstream.authenticateasclient?redirectedfrom=MSDN&view=netframework-4.8#System_Net_Security_SslStream_AuthenticateAsClient_System_String_System_Security_Cryptography_X509Certificates_X509CertificateCollection_System_Security_Authentication_SslProtocols_System_Boolean_

Under the remarks section it is stated:

The value specified for targetHost must match the name on the server's certificate.

This is the cert data after it is loaded into c# environment and dumped as a string.

CERT DATA: [Subject]
  CN=Amazon Root CA 1, O=Amazon, C=US

[Issuer]
  CN=Amazon Root CA 1, O=Amazon, C=US

[Serial Number]
  XXXXXXXXXXXXXXXX

[Not Before]
  5/26/2015 12:00:00 AM

[Not After]
  1/17/2038 12:00:00 AM

[Thumbprint]
   XXXXXXXXXXXXXXXX

I believe that the cert validation is failing because that is not the case with the contact endpoint. I'm not familiar enough with this but should something in this cert reference amazonaws.com or something?

Has anyone been able to successfully connect to MCS using c# / dotnet or might this be a known problem?

A possible solution to all this might be to implement a Sig4v Auth Provider and Authorizer for c#. It seems that only a Java one exists at the moment. I was considering building one myself from the java code and converting it to c#. However, I also noticed that the java driver sets some jvm variables and didn't really know what the corresponding task would be to do so in c# / dotnet. So before I go down that rabbit whole I would like to see what others have to say about this.

I also know MCS is in beta. Considering the service in beta I suspect there are still issues that might relate to what I'm seeing here.

These are the docs for connecting to MCS using java and python.

https://docs.aws.amazon.com/mcs/latest/devguide/programmatic.drivers.html

Java requires that driver for Sig4v which is why I mentioned it earlier.

[classifieds-dev]

I have been able to successfully connect using sqlsh with the same pem file being used in the c# code.

username$ cqlsh cassandra.us-east-1.amazonaws.com 9142 -u "<username>" -p "<password>" --ssl
Connected to Amazon Managed Apache Cassandra Service at cassandra.us-east-1.amazonaws.com:9142.
[cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
username@cqlsh>

[classifieds-dev]

I think this might be related to the problem I'm having.

https://datastax-oss.atlassian.net/browse/CSHARP-881

[classifieds-dev]

I turned on tracking and the csharp driver emitted this error.

TcpSocket: 04/22/2020 23:48:16.817 +00:00 #ERROR: SSL connection: Can not resolve host name for address 3.17.136.35. Using the IP address instead of the host name. This may cause RemoteCertificateNameMismatch error during Cassandra host authentication. Note that the Cassandra node SSL certificate's CN(Common Name) must match the Cassandra node hostname.

[classifieds-dev]

I've spent more time than I care to admit on this. I actually began building middle layer to act as private gateway using the Java micronaut framework when I stubbled upon the docs here.

https://micronaut-projects.github.io/micronaut-cassandra/snapshot/guide/

In that guide what stood out to me was the load balancing policy.

So I added that to the c# connect and success was had.

Final code

            var pathToCAFile = $"{Directory.GetCurrentDirectory()}/AmazonRootCA1.pem";
            X509Certificate2[] certs = new X509Certificate2[] { new X509Certificate2(pathToCAFile, "amazon") };
            lambdaContext.Logger.LogLine($"CERT DATA: {certs[0]}");
            X509Certificate2Collection certificateCollection = new X509Certificate2Collection(certs);
            var options = new Cassandra.SSLOptions(SslProtocols.Tls11, true, ValidateServerCertificate);
            options.SetCertificateCollection(certificateCollection);
            Cluster cluster = Cluster
                                .Builder()
                                .WithCredentials("<username>", "<password>")
                                .WithPort(9142)
                                .AddContactPoint("cassandra.us-east-1.amazonaws.com")
                                .WithSSL(options)
                                .WithLoadBalancingPolicy(new DefaultLoadBalancingPolicy("us-east-1"))
                                .Build();
            try
            {
                var session = cluster.Connect("<TableName>");
                lambdaContext.Logger.LogLine("CONNECTED TO CASSANDRA!!!!");
            } catch(Exception e)
            {
                lambdaContext.Logger.LogLine(JsonConvert.SerializeObject(e));
            }

[ashishdhingra]

Hi @classifieds-dev,

Good afternoon.

I was going through issue backlog and came across this one. Looks like you have already found a solution to the issue. Please confirm if the issue could be closed.

Thanks,
Ashish

[github-actions[bot]]

This issue has not recieved a response in 2 weeks. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.