Etcd: ETCD with TLS showing error "transport: remote error: tls: bad certificate" for certificates generated using OpenSSL

Created on 25 Sep 2017  路  27Comments  路  Source: etcd-io/etcd

ETCD 3.2.5 started with openssl certificates as follows
etcdserver/api/v3rpc: Failed to dial 0.0.0.0:2379: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.

The ca-chain and server certificates were generated as mentioned here
https://jamielinux.com/docs/openssl-certificate-authority/appendix/root-configuration-file.html
as well as here
https://coreos.com/os/docs/latest/generate-self-signed-certificates.html

Please find the complete log here
[root@vm-188 etcd-v3.2.5-linux-amd64]# ./etcd --client-cert-auth --trusted-ca-file=/root/cfssl/ca.pem --cert-file=/root/cfssl/server.pem --key-file=/root/cfssl/server-key.pem --advertise-client-urls https://0.0.0.0:2379 --listen-client-urls https://10.53.70.188:2379 2017-09-25 19:31:46.145497 I | etcdmain: etcd Version: 3.2.5
2017-09-25 19:31:46.145663 I | etcdmain: Git SHA: d0d1a87
2017-09-25 19:31:46.145678 I | etcdmain: Go Version: go1.8.3
2017-09-25 19:31:46.145690 I | etcdmain: Go OS/Arch: linux/amd64
2017-09-25 19:31:46.145707 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2017-09-25 19:31:46.145734 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
2017-09-25 19:31:46.145827 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2017-09-25 19:31:46.146912 I | embed: listening for peers on http://localhost:2380
2017-09-25 19:31:46.147068 I | embed: listening for client requests on 10.53.70.188:2379
2017-09-25 19:31:46.151360 I | etcdserver: name = default
2017-09-25 19:31:46.151384 I | etcdserver: data dir = default.etcd
2017-09-25 19:31:46.151397 I | etcdserver: member dir = default.etcd/member
2017-09-25 19:31:46.151418 I | etcdserver: heartbeat = 100ms
2017-09-25 19:31:46.151430 I | etcdserver: election = 1000ms
2017-09-25 19:31:46.151441 I | etcdserver: snapshot count = 100000
2017-09-25 19:31:46.151464 I | etcdserver: advertise client URLs = https://0.0.0.0:2379
2017-09-25 19:31:46.197402 I | etcdserver: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 387
2017-09-25 19:31:46.197544 I | raft: 8e9e05c52164694d became follower at term 43
2017-09-25 19:31:46.197646 I | raft: newRaft 8e9e05c52164694d [peers: [], term: 43, commit: 387, applied: 0, lastindex: 387, lastterm: 43]
2017-09-25 19:31:46.244364 W | auth: simple token is not cryptographically signed
2017-09-25 19:31:46.269470 I | etcdserver: starting server... [version: 3.2.5, cluster version: to_be_decided]
2017-09-25 19:31:46.269551 I | embed: ClientTLS: cert = /root/cfssl/server.pem, key = /root/cfssl/server-key.pem, ca = , trusted-ca = /root/cfssl/ca.pem, client-cert-auth = true
2017-09-25 19:31:46.270799 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2017-09-25 19:31:46.271154 N | etcdserver/membership: set the initial cluster version to 3.2
2017-09-25 19:31:46.271269 I | etcdserver/api: enabled capabilities for version 3.2
2017-09-25 19:31:46.298335 I | raft: 8e9e05c52164694d is starting a new election at term 43
2017-09-25 19:31:46.298399 I | raft: 8e9e05c52164694d became candidate at term 44
2017-09-25 19:31:46.298440 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 44
2017-09-25 19:31:46.298474 I | raft: 8e9e05c52164694d became leader at term 44
2017-09-25 19:31:46.298494 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 44
2017-09-25 19:31:46.299064 I | etcdserver: published {Name:default ClientURLs:[https://0.0.0.0:2379]} to cluster cdf818194e3a8c32
2017-09-25 19:31:46.321000 I | embed: ready to serve client requests
2017-09-25 19:31:46.321265 E | etcdmain: forgot to set Type=notify in systemd service file?
2017-09-25 19:31:46.322924 I | embed: serving client requests on 10.53.70.188:2379
2017-09-25 19:31:46.328396 I | etcdserver/api/v3rpc: Failed to dial 10.53.70.188:2379: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.

Investigating arequestion aretls
Source
picture keyankay
👍8

Most helpful comment

I think the cause is missing or unclear documentation on what the server/peer & client certificates need to have in CN & SAN in order for it to work.

picture Constantin07 on 6 Apr 2018
👍5

All 27 comments

What is the output of openssl x509 -in /root/cfssl/server.pem -text -noout?

gyuho picture gyuho on 25 Sep 2017

And do not advertise default route address 0.0.0.0.

See https://github.com/coreos/etcd/blob/master/Documentation/faq.md#what-is-the-difference-between-listen-clientpeer-urls-advertise-client-urls-or-initial-advertise-peer-urls.

gyuho picture gyuho on 25 Sep 2017

Thank you for the link

Please find the output of server.pem

openssl x509 -in /root/cfssl/server.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            31:ea:e0:8e:fd:9a:d7:5a:0c:4b:2b:8f:4e:c4:74:fb:33:c6:5f:86
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
        Validity
            Not Before: Sep 25 13:47:00 2017 GMT
            Not After : Sep 24 13:47:00 2022 GMT
        Subject: C=US, ST=San Francisco, L=CA, CN=example.net
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:f4:04:d0:7e:4f:64:56:67:12:b5:57:9c:fd:5a:
                    af:f8:c4:92:d2:bd:11:41:ca:63:f2:82:04:8b:1e:
                    2c:d4:1b:ad:28:8e:f9:0a:b3:3a:4b:21:00:ed:3c:
                    4c:93:4a:21:cc:13:80:b4:9f:95:e6:9c:c1:c9:60:
                    23:f1:6b:d2:33
                ASN1 OID: prime256v1
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                68:AB:F6:C1:C7:F7:72:F2:1A:80:9C:E9:19:16:D6:3F:28:8F:F5:04
            X509v3 Authority Key Identifier:
                keyid:5D:9C:BB:4C:0C:2D:87:12:05:EC:92:C3:EC:30:6F:4F:50:D8:A2:E7

            X509v3 Subject Alternative Name:
                DNS:www.example.net, IP Address:10.53.70.188
    Signature Algorithm: sha256WithRSAEncryption
         00:f8:7e:f4:32:d5:ca:71:7a:9d:c4:c8:fc:29:94:e0:f0:d7:
         cb:03:17:f7:13:ed:c5:c8:1d:9b:c7:94:7c:05:4b:f2:7f:33:
         4a:00:e5:6d:40:5b:bb:54:dc:4f:03:74:88:89:c3:a2:f6:5f:
         ea:51:50:ce:28:06:84:b0:f1:c6:ea:e3:9b:55:34:47:b2:b8:
         af:bc:fa:d5:ab:cc:02:bb:f6:6e:36:8f:43:98:94:95:08:8e:
         9e:1b:44:11:1c:ed:fe:d0:bb:63:e7:ce:e4:cd:3c:d0:1f:4b:
         01:3c:13:9f:2d:05:62:51:82:63:ce:a6:ee:05:9c:6b:72:40:
         df:6b:62:71:e7:6a:cf:b3:4e:21:37:3b:18:05:93:04:dc:54:
         a5:e8:d4:63:6d:cb:4f:e9:53:0d:eb:7a:6d:b9:89:34:fb:88:
         8d:e6:69:c9:6d:93:62:a3:8d:b5:6c:3e:85:cb:45:be:8a:8b:
         4a:5e:69:c5:95:75:96:ad:6e:4f:23:aa:5c:66:c9:b3:cb:da:
         d5:3f:b0:3a:5a:e8:43:b7:6c:c5:2e:ee:f0:50:0f:76:ef:08:
         e3:e3:9b:4e:1a:e6:59:a3:33:b4:5d:ff:81:0a:c9:7f:ea:83:
         16:9e:0e:8d:56:af:eb:f7:64:35:61:d0:96:73:86:c2:c1:1e:
         10:d1:18:0c
keyankay picture keyankay on 26 Sep 2017

@keyankay SSL config looks good, but you are advertising the default route 0.0.0.0.

Try fixing:

  ---advertise-client-urls https://0.0.0.0:2379 \
  --listen-client-urls https://10.53.70.188:2379

To

  ---advertise-client-urls https://10.53.70.188:2379 \
  --listen-client-urls hhttps://0.0.0.0:2379

?

gyuho picture gyuho on 26 Sep 2017

I still get the same error (used 2 options with and without 0.0.0.0). Same issue both the times
./etcd --client-cert-auth --trusted-ca-file=/root/cfssl/ca.pem --cert-file=/root/cfssl/server.pem --key-file=/root/cfssl/server-key.pem --advertise-client-urls https://10.53.70.188:2379 --listen-client-urls https://0.0.0.0:2379
2017-09-26 09:29:50.594169 I | etcdmain: etcd Version: 3.2.5

2017-09-26 09:29:51.305220 I | etcdserver: published {Name:default ClientURLs:[https://10.53.70.188:2379]} to cluster cdf818194e3a8c32
2017-09-26 09:29:51.306228 I | embed: serving client requests on [::]:2379
2017-09-26 09:29:51.311492 I | etcdserver/api/v3rpc: Failed to dial 0.0.0.0:2379: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.

./etcd --client-cert-auth --trusted-ca-file=/root/cfssl/ca.pem --cert-file=/root/cfssl/server.pem --key-file=/root/cfssl/server-key.pem --advertise-client-urls https://10.53.70.188:2379 --listen-client-urls https://10.53.70.188:2379
2017-09-26 09:30:44.208540 I | embed: ClientTLS: cert = /root/cfssl/server.pem, key = /root/cfssl/server-key.pem, ca = , trusted-ca = /root/cfssl/ca.pem, client-cert-auth = true
2017-09-26 09:30:44.209672 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2017-09-26 09:30:44.209966 N | etcdserver/membership: set the initial cluster version to 3.2
2017-09-26 09:30:44.210189 I | etcdserver/api: enabled capabilities for version 3.2
2017-09-26 09:30:44.803166 I | raft: 8e9e05c52164694d is starting a new election at term 46
2017-09-26 09:30:44.803245 I | raft: 8e9e05c52164694d became candidate at term 47
2017-09-26 09:30:44.803366 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 47
2017-09-26 09:30:44.803410 I | raft: 8e9e05c52164694d became leader at term 47
2017-09-26 09:30:44.803431 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 47
2017-09-26 09:30:44.804112 I | etcdserver: published {Name:default ClientURLs:[https://10.53.70.188:2379]} to cluster cdf818194e3a8c32
2017-09-26 09:30:44.804173 I | embed: ready to serve client requests
2017-09-26 09:30:44.805257 E | etcdmain: forgot to set Type=notify in systemd service file?
2017-09-26 09:30:44.806964 I | embed: serving client requests on 10.53.70.188:2379
2017-09-26 09:30:44.811752 I | etcdserver/api/v3rpc: Failed to dial 10.53.70.188:2379: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.

keyankay picture keyankay on 26 Sep 2017

I also checked in 3.2.7, the problem exists.
I installed etcd 3.1.10 and i do not see the issue. I sense this is an etcd bug

[root@vm-188 etcd-v3.1.10-linux-amd64]# ./etcd --client-cert-auth --trusted-ca-file=/root/cfssl/ca.pem --cert-file=/root/cfssl/server.pem --key-file=/root/cfssl/server-key.pem --advertise-client-urls https://10.53.70.188:2379 --listen-client-urls https://10.53.70.188:2379
2017-09-26 10:07:16.392173 I | etcdmain: etcd Version: 3.1.10

2017-09-26 10:07:16.421663 I | embed: ClientTLS: cert = /root/cfssl/server.pem, key = /root/cfssl/server-key.pem, ca = , trusted-ca = /root/cfssl/ca.pem, client-cert-auth = true
2017-09-26 10:07:16.424200 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2017-09-26 10:07:16.424582 N | etcdserver/membership: set the initial cluster version to 3.1
2017-09-26 10:07:16.424711 I | etcdserver/api: enabled capabilities for version 3.1
2017-09-26 10:07:17.016066 I | raft: 8e9e05c52164694d is starting a new election at term 2
2017-09-26 10:07:17.016148 I | raft: 8e9e05c52164694d became candidate at term 3
2017-09-26 10:07:17.016214 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 3
2017-09-26 10:07:17.016257 I | raft: 8e9e05c52164694d became leader at term 3
2017-09-26 10:07:17.016281 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 3
2017-09-26 10:07:17.016918 I | etcdserver: published {Name:default ClientURLs:[https://10.53.70.188:2379]} to cluster cdf818194e3a8c32
2017-09-26 10:07:17.017012 I | embed: ready to serve client requests
2017-09-26 10:07:17.017274 E | etcdmain: forgot to set Type=notify in systemd service file?
2017-09-26 10:07:17.018073 I | embed: serving client requests on 10.53.70.188:2379

However, when i try to connect from client, i get an error

sudo curl -v --noproxy '*' --cacert /root/cfssl/ca.pem --cert /root/cfssl/client.pem --key /root/cfssl/client-key.pem -L https://10.53.70.188:2379/v2/keys/foo -XPUT -d value=bar -v

  • About to connect() to 10.53.70.188 port 2379 (#0)
  • Trying 10.53.70.188...
  • Connected to 10.53.70.188 (10.53.70.188) port 2379 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • CAfile: /root/cfssl/ca.pem
    CApath: none
  • unable to load client key: -8178 (SEC_ERROR_BAD_KEY)
  • NSS error -8178 (SEC_ERROR_BAD_KEY)
  • Peer's public key is invalid.
  • Closing connection 0
    curl: (58) unable to load client key: -8178 (SEC_ERROR_BAD_KEY)

ca.pem

  Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Validity
        Not Before: Sep 26 04:54:00 2017 GMT
        Not After : Sep 25 04:54:00 2022 GMT
    Subject: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
            Public-Key: (2048 bit)
            Modulus:
                00:c1:68:e4:0e:f1:a5:e2:57:b5:d6:d6:c1:ac:19:
                bf:2f:f5:50:a1:ba:75:d7:03:aa:d3:0c:6f:a0:2a:
                58:b7:ff:bd:c0:e7:82:c6:06:c2:57:9b:f9:23:0a:
                b6:15:b2:2e:ca:7e:d5:d9:0d:7f:83:b9:2b:bc:3c:
                0c:be:f6:3c:12:ad:5a:a7:71:26:2b:af:e6:af:14:
                66:79:98:3c:19:32:c0:1b:74:64:da:eb:9e:70:aa:
                8b:22:0d:03:5f:ff:76:de:c8:e0:73:f3:11:33:b6:
                dd:66:06:c8:58:b1:a5:5d:f5:e9:47:cd:fe:01:27:
                9b:07:d1:9b:bb:55:cb:bf:06:c0:3a:ef:c7:db:63:
                aa:79:6e:7d:0d:d3:58:45:48:09:3a:0b:c0:8e:76:
                aa:48:18:09:22:6d:0e:18:fc:f1:9f:d9:e4:f7:78:
                10:a4:e7:0a:d6:0c:95:2d:88:a2:0d:d8:3f:2f:89:
                ad:97:bd:68:fa:19:30:3e:f3:07:30:78:87:0d:2b:
                eb:ea:83:c4:e2:53:8e:f6:52:4b:ac:fb:67:ce:91:
                6d:e1:d3:b6:41:73:5d:a6:14:80:14:6d:6a:1c:49:
                6f:f6:34:31:64:6f:17:28:14:17:41:c9:7d:c2:14:
                3c:fe:81:78:1b:22:bb:4b:0a:6b:44:69:3b:67:22:
                19:f3
            Exponent: 65537 (0x10001)
    X509v3 extensions:
        X509v3 Key Usage: critical
            Certificate Sign, CRL Sign
        X509v3 Basic Constraints: critical
            CA:TRUE, pathlen:2
        X509v3 Subject Key Identifier:
            21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23
        X509v3 Authority Key Identifier:
            keyid:21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23

Signature Algorithm: sha256WithRSAEncryption
     88:88:5c:d5:cb:da:fc:9a:13:f0:50:f1:0e:3c:07:9b:2e:b8:
     98:03:6b:d3:7e:43:37:7a:e2:56:c1:c3:87:59:d9:96:28:42:
     65:cc:2d:bb:71:25:be:bd:10:d7:c4:1b:c4:9e:c9:fc:e1:81:
     b2:b3:3f:0c:99:25:66:c2:a3:a3:44:d9:66:05:80:42:1b:c4:
     e0:3e:96:fd:e0:19:6a:d2:5e:86:cc:2d:d7:1b:ca:7c:b2:34:
     22:93:a6:c9:7e:b0:07:de:79:48:e3:fc:9d:fc:09:1b:35:6b:
     8c:aa:ce:f3:c7:23:5e:1b:02:77:ed:e9:52:4b:1d:b0:e3:e2:
     2c:73:00:d2:5c:ba:c4:36:48:99:0c:9f:6f:62:f7:d7:e1:18:
     21:cb:00:a7:fd:fc:84:33:a5:0d:37:12:d6:07:0b:4a:8b:20:
     c1:c3:00:00:96:fe:a3:ce:53:d7:43:21:3f:a5:7e:f1:4f:22:
     69:15:55:8c:9e:b8:c9:f6:f9:4c:9f:4e:9f:2d:75:93:f7:8d:
     db:b6:99:f0:fd:84:30:ff:12:43:18:d4:b1:d0:e2:32:48:24:
     fa:5a:d0:01:39:8f:73:5e:9f:55:97:33:98:b2:c2:96:62:cb:
     be:9c:f7:f2:0e:ea:67:68:b8:af:19:67:18:d2:3b:7a:de:61:
     33:9c:5e:62

server.pem

Certificate:

    Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Validity
        Not Before: Sep 26 04:55:00 2017 GMT
        Not After : Sep 25 04:55:00 2022 GMT
    Subject: C=US, ST=San Francisco, L=CA, CN=example.net
    Subject Public Key Info:
        Public Key Algorithm: id-ecPublicKey
            Public-Key: (256 bit)
            pub:
                04:20:8c:19:15:0a:e6:58:e5:80:8b:3c:1f:09:05:
                e4:85:d8:2b:29:49:a1:28:d6:69:fc:d0:61:99:40:
                45:c5:3b:a4:a4:31:62:63:8c:87:77:43:87:4a:43:
                e2:2b:40:66:b1:fa:fa:8c:7b:fd:74:bc:25:60:7e:
                5f:6c:8a:44:27
            ASN1 OID: prime256v1
    X509v3 extensions:
        X509v3 Key Usage: critical
            Digital Signature, Key Encipherment
        X509v3 Extended Key Usage:
            TLS Web Server Authentication
        X509v3 Basic Constraints: critical
            CA:FALSE
        X509v3 Subject Key Identifier:
            57:4C:55:B4:D7:CF:7D:F6:5C:23:8D:B0:93:7F:8A:09:F0:87:05:91
        X509v3 Authority Key Identifier:
            keyid:21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23

        X509v3 Subject Alternative Name:
            DNS:www.example.net, IP Address:10.53.70.188
Signature Algorithm: sha256WithRSAEncryption
     81:a6:59:9c:ea:ee:c8:56:6f:c4:7a:aa:80:85:f4:71:f0:a0:
     ac:dd:1f:0c:95:57:f7:be:b2:2f:e8:08:74:f1:aa:2a:47:59:
     c4:ff:15:c2:3b:84:f2:26:48:51:4f:d3:f8:c1:46:28:c3:72:
     23:87:2c:bf:2c:2e:2e:53:d7:86:e9:2b:28:98:6c:01:ac:0b:
     9f:e5:86:55:47:87:fe:4a:82:55:23:36:ac:7a:9f:f0:76:7f:
     10:1e:92:01:a5:29:63:18:c6:af:1d:f9:b1:be:8c:32:87:7f:
     45:72:44:0e:c8:d5:a1:1f:23:71:4e:cf:ec:39:92:fb:da:44:
     34:b9:d2:dd:f8:75:68:ce:d8:f4:13:63:62:3c:e1:48:dc:34:
     81:cf:fb:90:04:13:50:a7:dc:5a:cf:e7:da:70:b5:05:f8:d7:
     44:98:19:4f:5d:a4:f7:3e:7d:1b:fc:b3:59:60:e0:07:6b:06:
     54:78:31:18:60:c0:92:7e:68:88:47:62:3d:eb:e8:ff:d8:13:
     82:53:59:84:dc:59:bb:fb:c3:6e:d9:14:d1:e1:8e:d9:03:2e:
     28:bd:23:09:09:88:ec:df:a5:6d:26:b4:a3:fc:96:d2:4b:05:
     cc:b0:d1:6e:fb:1e:b0:f7:3b:73:14:5e:11:49:e7:48:77:da:
     bd:b1:d3:71

client.pem

certificate:

    Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Validity
        Not Before: Sep 26 04:56:00 2017 GMT
        Not After : Sep 25 04:56:00 2022 GMT
    Subject: C=US, ST=San Francisco, L=CA, CN=client
    Subject Public Key Info:
        Public Key Algorithm: id-ecPublicKey
            Public-Key: (256 bit)
            pub:
                04:72:8a:ac:8a:fe:44:ad:09:51:1d:7c:f5:3d:6a:
                e0:5a:79:55:5e:e6:6b:10:38:40:4f:2e:1a:63:eb:
                46:0a:f4:35:29:ba:23:8d:96:c0:c4:a1:f7:8e:fb:
                ed:9e:5c:cc:9c:1f:3f:d7:4f:19:90:b4:d4:c7:93:
                de:e3:9f:03:ec
            ASN1 OID: prime256v1
    X509v3 extensions:
        X509v3 Key Usage: critical
            Digital Signature, Key Encipherment
        X509v3 Extended Key Usage:
            TLS Web Client Authentication
        X509v3 Basic Constraints: critical
            CA:FALSE
        X509v3 Subject Key Identifier:
            6B:9C:C5:14:C7:5D:05:34:C4:B3:39:C4:45:A0:61:C5:27:FC:72:0B
        X509v3 Authority Key Identifier:
            keyid:21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23

        X509v3 Subject Alternative Name:
            DNS:
Signature Algorithm: sha256WithRSAEncryption
     47:25:b3:b5:64:21:ad:2d:7d:9b:52:ae:4c:41:fb:7e:70:fe:
     60:44:8d:8e:50:fc:e0:76:09:8b:46:62:a1:d5:d6:91:d1:b9:
     93:cc:b2:91:32:cf:9e:82:e0:a3:24:e1:93:85:7f:6a:77:15:
     02:a8:d1:5d:18:3f:bb:24:92:0d:a9:6d:51:97:5e:d1:03:d9:
     21:a3:f3:e6:b0:2a:07:0c:5f:16:4f:63:4f:1d:1d:c3:09:ae:
     55:b9:b5:81:f3:78:70:9c:27:86:af:47:fe:be:91:f4:61:8a:
     f8:13:ad:04:9c:05:14:0e:4b:88:40:0b:e7:86:a4:45:12:a8:
     0e:66:c7:ca:46:a1:e8:de:b9:81:d1:7c:8f:f0:dc:7c:71:18:
     57:39:00:61:64:96:53:42:7f:65:50:45:44:e4:cd:1d:02:67:
     19:43:63:c3:73:a0:35:dd:0d:17:f1:f4:c7:de:20:a6:e7:d4:
     35:2f:e4:4f:c9:1f:c3:25:b8:05:bc:f7:0c:bf:bb:7c:65:31:
     cf:9b:cb:39:ef:fb:2b:d9:63:b9:ba:0b:bc:9b:a2:b7:17:d2:
     9c:69:20:9c:64:15:80:6a:de:09:dd:08:4d:0e:d4:a5:84:0c:
     7c:0b:54:2c:ff:34:3d:51:b2:37:9a:43:bc:e0:72:dc:41:8d:
     63:ae:7b:e1
keyankay picture keyankay on 26 Sep 2017

Can you regenerate certs with TLS Web Server Authentication, TLS Web Client Authentication for X509v3 Extended Key Usage? For our debug, we need a reproducible way.

gyuho picture gyuho on 26 Sep 2017

I did with another set with the extended key usage, but not self signed (With a intermediate CA). I get a different error when i do with intermediate and CA Chain

Here i am posting with all the relevant logs and information
ETCD Server start:

./etcd --client-cert-auth --trusted-ca-file=/root/etcd/server_certs/ca-chain.cert.pem --cert-file=/root/etcd/server_certs/server.cert.pem --key-file=/root/etcd/server_certs/server.key.pem --advertise-client-urls https://10.53.70.188:2379 --listen-client-urls https://0.0.0.0:2379
.
.

2017-09-26 12:14:29.212009 I | embed: ClientTLS: cert = /root/etcd/server_certs/server.cert.pem, key = /root/etcd/server_certs/server.key.pem, ca = , trusted-ca = /root/etcd/server_certs/ca-chain.cert.pem, client-cert-auth = true
2017-09-26 12:14:29.213857 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2017-09-26 12:14:29.214412 N | etcdserver/membership: set the initial cluster version to 3.1
2017-09-26 12:14:29.214539 I | etcdserver/api: enabled capabilities for version 3.1
2017-09-26 12:14:29.507364 I | raft: 8e9e05c52164694d is starting a new election at term 17
2017-09-26 12:14:29.507539 I | raft: 8e9e05c52164694d became candidate at term 18
2017-09-26 12:14:29.507585 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 18
2017-09-26 12:14:29.507622 I | raft: 8e9e05c52164694d became leader at term 18
2017-09-26 12:14:29.507647 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 18
2017-09-26 12:14:29.508665 I | etcdserver: published {Name:default ClientURLs:[https://10.53.70.188:2379]} to cluster cdf818194e3a8c32
2017-09-26 12:14:29.508721 I | embed: ready to serve client requests
2017-09-26 12:14:29.509057 E | etcdmain: forgot to set Type=notify in systemd service file?
2017-09-26 12:14:29.509508 I | embed: serving client requests on [::]:2379

ETCD client log:

[root@vm-188 cfssl]# curl -v --noproxy '*' --cacert /root/etcd/server_certs/ca-chain.cert.pem --cert /root/etcd/client_certs/client.cert.pem --key /root/etcd/client_certs/client.key.pem -L https://localhost:2379/v2/keys/foo -XPUT -d value=bar -v
About to connect() to localhost port 2379 (#0)
Trying ::1...
Connected to localhost (::1) port 2379 (#0)
Initializing NSS with certpath: sql:/etc/pki/nssdb
CAfile: /root/etcd/server_certs/ca-chain.cert.pem
CApath: none
Server certificate:
subject: CN=ServerKarthik,O=Alice Ltd,ST=England,C=GB
start date: Sep 25 08:28:06 2017 GMT
expire date: Oct 05 08:28:06 2018 GMT
common name: ServerKarthik
issuer: CN=IntermediateKarthik,O=Alice Ltd,ST=England,C=GB
NSS error -12276 (SSL_ERROR_BAD_CERT_DOMAIN)
Unable to communicate securely with peer: requested domain name does not match the server's certificate.

  • Closing connection 0
    curl: (51) Unable to communicate securely with peer: requested domain name does not match the server's certificate.

Please find the certificate details :
ca-chain Certificate:

Data:
    Version: 3 (0x2)
    Serial Number: 4096 (0x1000)
Signature Algorithm: sha256WithRSAEncryption
    Issuer: C=GB, ST=England, O=Alice Ltd, CN=Karthik
    Validity
        Not Before: Sep 25 08:17:22 2017 GMT
        Not After : Sep 23 08:17:22 2027 GMT
    Subject: C=GB, ST=England, O=Alice Ltd, CN=IntermediateKarthik
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
            Public-Key: (4096 bit)
            Modulus:
                00:d1:88:a4:be:34:68:d5:94:f7:aa:66:0e:10:43:
                e0:9f:cf:e9:97:71:9e:3d:a9:61:dc:de:fc:1e:ee:
                0f:74:f6:a5:6c:df:37:56:12:8c:ca:8a:b6:95:a6:
                44:23:a0:eb:c2:6a:67:63:09:31:c9:b8:01:69:a8:
                9a:1b:71:85:95:52:cc:22:45:f9:4c:42:77:b2:d2:
                60:6f:dc:2a:ec:bc:fe:f9:ae:23:7d:55:03:79:d4:
                de:27:15:6f:fe:42:1e:7c:db:fd:50:5f:a4:bd:ba:
                3c:8d:a3:7d:70:ae:b3:da:27:c8:28:4a:1b:74:83:
                a1:30:46:62:ab:77:eb:09:ed:d9:4d:4e:74:9e:bf:
                e4:cd:c1:99:14:bf:09:2f:69:09:28:b4:dc:6d:72:
                34:38:d8:c8:eb:ce:56:e3:f1:f9:e9:46:ed:ad:a7:
                df:3e:23:f6:60:84:5d:01:cf:4f:25:73:1a:ce:62:
                8e:a6:d1:94:9a:34:61:e5:a6:e1:2b:b7:bb:9f:b8:
                44:64:bf:fa:50:79:33:27:10:0a:15:ab:f3:b9:63:
                23:41:f9:12:0f:ab:3d:ee:b6:ca:44:fa:5f:f6:53:
                db:c6:aa:9b:b9:6e:7e:70:f0:dc:ac:60:da:90:66:
                7f:99:a2:68:53:68:88:bd:59:5e:57:35:f5:9d:6b:
                02:10:df:0f:93:97:74:ee:bb:e8:34:91:38:86:8b:
                ff:46:97:52:ba:a9:13:05:24:32:4a:3c:cf:b0:ee:
                4d:dc:f9:e3:b0:19:ad:ef:8d:cb:c5:e9:14:a5:60:
                83:1a:9f:b4:3f:57:ed:eb:32:4c:70:27:7a:f2:8b:
                20:01:cc:f5:7c:d4:87:0b:01:04:31:3d:0e:30:e9:
                e0:a6:56:0f:96:26:29:52:c6:d0:b8:63:da:27:f3:
                73:f4:78:f8:ce:04:29:3f:a5:a7:fa:0c:73:60:e5:
                22:73:7a:3d:aa:de:b0:3c:d8:ef:7b:1f:fc:96:a2:
                b3:71:f0:29:13:f2:01:0f:e1:f4:44:a0:5e:19:a0:
                39:98:62:31:59:3b:46:a5:75:a9:ac:68:72:6e:04:
                16:6f:d4:fd:57:5b:c5:cb:e8:28:2a:5b:76:fe:c8:
                f9:79:70:c0:00:21:ff:60:5e:b7:44:ea:dd:22:e7:
                a0:a7:a8:06:22:29:58:38:c8:02:a6:6b:89:fd:7e:
                63:6a:f4:fd:58:2d:5a:97:88:24:a2:3e:d7:c0:33:
                46:a2:f7:c7:53:ba:bc:0c:bf:b2:28:e5:e7:ca:93:
                01:0a:e8:a9:3b:cc:40:3e:07:6c:f0:64:b3:69:da:
                89:43:b3:fc:82:82:9c:c1:2f:33:be:59:df:ff:6d:
                0e:7e:df
            Exponent: 65537 (0x10001)
    X509v3 extensions:
        X509v3 Subject Key Identifier:
            F9:0C:48:44:36:E1:BA:8F:A7:B3:A5:EF:21:89:2F:01:D5:23:4A:99
        X509v3 Authority Key Identifier:
            keyid:6E:E2:BB:CD:84:35:F9:F7:F8:24:B6:57:9E:C5:BE:9D:C1:2D:2B:B2

        X509v3 Basic Constraints: critical
            CA:TRUE, pathlen:0
        X509v3 Key Usage: critical
            Digital Signature, Certificate Sign, CRL Sign
Signature Algorithm: sha256WithRSAEncryption
     8d:d0:02:30:c3:9b:a1:e9:fe:ff:74:0d:66:1a:43:85:c1:d6:
     96:c8:16:73:82:b8:6b:ba:db:a2:1e:74:05:ff:00:a1:53:16:
     50:c1:8b:33:96:02:b7:0f:ee:f8:38:24:7e:64:a8:bd:64:31:
     6f:73:68:af:07:c2:16:7e:46:7b:69:b6:20:f2:97:c0:0b:26:
     f6:86:61:a4:35:70:2e:71:1b:06:8c:ab:ea:fe:d0:bd:81:43:
     9d:c7:5d:7b:6a:53:27:ce:75:9c:7c:e3:00:4e:28:6d:85:64:
     a8:f1:e8:bf:cf:75:52:c6:2d:9c:5e:05:05:07:44:3b:54:04:
     fa:1c:72:16:9a:26:05:d3:31:d2:9e:5c:ed:24:05:d8:a7:c4:
     bf:d1:2a:ad:44:df:09:7b:3f:d5:b7:8a:db:58:9e:57:5d:a4:
     4c:cc:9e:dc:19:29:f9:30:f8:e4:ea:dd:2f:bf:9f:44:fb:e0:
     83:01:0c:f9:2a:f4:e0:e8:64:b4:2c:67:6a:8a:bb:a4:3e:b8:
     e6:41:93:8d:b0:c4:5d:95:eb:30:29:56:33:67:79:2e:4d:6c:
     57:fe:de:94:d4:ac:8e:d9:b3:53:13:9a:a8:04:c3:48:ad:7b:
     5c:70:7c:46:50:fd:ad:90:cb:47:d6:c3:ed:58:a2:07:66:9e:
     d5:1e:76:2b:54:cf:6b:79:6b:15:d0:a7:30:cd:47:87:9d:1e:
     f8:c8:1f:d9:46:bd:40:02:e1:f4:c6:12:1d:68:ed:9f:ab:a8:
     f1:c2:32:e8:5f:50:bc:e9:75:49:6e:13:f5:e3:95:22:af:34:
     23:6e:0a:46:bc:de:c8:de:1d:e3:c4:f4:bf:ba:b3:0a:d5:d2:
     39:ae:ee:2f:13:34:97:c4:20:66:4f:84:1c:3c:c7:84:b0:1c:
     ff:b0:c7:41:0c:47:94:69:19:18:b8:51:f1:da:af:69:2c:32:
     b3:c3:6e:61:fd:95:23:af:23:bb:d1:ac:33:b3:95:35:e5:07:
     43:85:d9:9d:b5:f4:84:a5:3f:d0:5e:cb:42:31:9d:9b:01:0b:
     0e:6f:8b:53:a0:b7:1a:28:47:8e:a0:43:06:b3:f1:22:1e:91:
     f9:fd:ba:11:20:1a:b0:1c:e4:43:dd:74:83:3a:1e:f0:4f:3b:
     fd:0d:09:55:dd:eb:5e:e2:ef:78:6c:18:31:8f:0e:0d:00:57:
     6f:96:7f:2d:0b:85:ff:3e:18:ed:3f:23:78:33:5e:8e:f3:0d:
     42:e9:ab:0e:95:c8:0c:bb:07:31:6b:9d:b3:28:16:72:46:99:
     46:fd:85:bd:40:d9:c5:c4:d0:13:a8:5c:4b:af:ae:c0:e5:34:
     42:89:92:ca:9e:8d:ec:eb

Server Certificate:

Data:
    Version: 3 (0x2)
    Serial Number: 4096 (0x1000)
Signature Algorithm: sha256WithRSAEncryption
    Issuer: C=GB, ST=England, O=Alice Ltd, CN=IntermediateKarthik
    Validity
        Not Before: Sep 25 08:28:06 2017 GMT
        Not After : Oct  5 08:28:06 2018 GMT
    Subject: C=GB, ST=England, O=Alice Ltd, CN=ServerKarthik
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
            Public-Key: (2048 bit)
            Modulus:
                00:c1:88:15:f3:e2:84:d6:ce:a0:82:33:60:f4:4e:
                53:7f:bc:4e:82:69:19:af:f6:44:4c:63:bb:7a:f1:
                43:ac:c7:6c:23:f5:ae:f9:66:76:ab:f3:5b:fb:af:
                a4:84:e8:3a:a7:44:ce:c8:cb:d2:a2:3b:4e:e6:5c:
                4f:cb:22:43:26:8e:3e:81:37:c7:83:d4:92:f7:de:
                f9:aa:5f:55:04:a2:61:bf:4f:51:61:63:51:31:78:
                ea:64:fa:b0:69:0a:5d:32:60:fe:68:0a:c8:f5:fd:
                de:a7:de:82:a7:a5:39:38:3b:7d:32:84:3c:0f:52:
                45:28:46:9b:54:d6:4f:16:15:c8:ac:b3:d6:99:c4:
                ad:2a:23:53:59:81:37:2e:7d:21:94:1c:20:45:75:
                dc:27:f4:48:ed:5e:0d:a3:00:91:91:e6:d0:59:fe:
                cf:cc:99:36:77:e0:cb:cd:22:7d:83:40:75:e7:db:
                75:e6:89:5b:77:80:b8:7f:67:03:30:d1:97:45:7e:
                8e:67:e5:90:92:05:70:da:7a:c8:85:93:0f:e0:67:
                66:e8:21:13:43:a6:43:b4:70:41:82:27:68:34:08:
                dd:ca:bb:38:59:a0:9d:81:13:0a:cb:d3:e7:06:b9:
                09:9d:da:c6:a6:6d:20:3f:4a:ea:80:2e:cb:bd:14:
                0e:d7
            Exponent: 65537 (0x10001)
    X509v3 extensions:
        X509v3 Basic Constraints:
            CA:FALSE
        Netscape Cert Type:
            SSL Server
        Netscape Comment:
            OpenSSL Generated Server Certificate
        X509v3 Subject Key Identifier:
            B0:7D:D0:56:FA:57:F4:96:2C:D1:ED:50:9A:3D:03:46:AF:AD:51:93
        X509v3 Authority Key Identifier:
            keyid:F9:0C:48:44:36:E1:BA:8F:A7:B3:A5:EF:21:89:2F:01:D5:23:4A:99
            DirName:/C=GB/ST=England/O=Alice Ltd/CN=Karthik
            serial:10:00

        X509v3 Key Usage: critical
            Digital Signature, Key Encipherment
        X509v3 Extended Key Usage:
            TLS Web Server Authentication
Signature Algorithm: sha256WithRSAEncryption
     25:c2:5d:72:14:07:9b:02:ed:bb:f5:57:d2:a9:a6:df:a3:a7:
     3c:ed:75:c6:eb:c6:06:3a:a8:10:b5:f0:a1:3b:66:d2:63:ba:
     e7:82:ae:ff:e8:48:5e:bc:c6:7a:d6:8d:be:57:0d:32:6f:13:
     68:c0:4f:8f:b9:7f:eb:e5:b8:0d:60:79:0c:2c:b5:d9:62:59:
     ba:e1:13:16:f4:e1:0e:74:c6:a0:65:1a:77:87:1e:ed:90:df:
     31:fe:30:a1:bb:68:3e:2f:b8:05:59:56:8c:76:cb:68:79:47:
     ba:38:6e:7c:64:27:17:fa:a0:93:cd:39:7e:4f:e4:c0:cc:40:
     96:73:e3:11:11:37:7a:b7:b6:10:be:a0:90:fe:87:e0:51:39:
     91:f0:94:71:d8:0d:83:c7:55:85:80:f9:f8:33:25:4b:9d:ed:
     64:79:50:0b:82:67:ec:e2:79:30:59:77:39:48:1e:9c:25:6b:
     9a:e7:a7:d0:59:0c:81:83:ed:e9:9a:5e:d3:b3:94:9f:d1:cf:
     2a:54:95:38:95:3f:48:d6:8a:c6:88:b8:13:ba:71:26:c0:58:
     c8:e2:5a:11:bc:2b:20:c0:a1:9b:ef:82:62:2c:10:be:36:2a:
     02:7c:b1:2d:4b:47:e0:c5:8d:51:68:a5:88:55:19:a9:db:3a:
     57:86:28:1f:b0:51:47:ef:c5:ba:ab:a2:72:9c:33:35:e0:c9:
     eb:d9:39:78:f8:b8:7c:aa:fb:9c:29:05:c0:64:cb:4c:7c:c8:
     b1:f8:96:d3:58:e2:e7:73:02:12:55:d0:81:cc:fe:f9:f2:fa:
     04:44:34:c5:08:f5:cb:6e:59:2c:cc:8e:34:e4:27:d6:61:59:
     b9:75:bd:e0:88:98:a9:f9:2d:86:f9:22:2f:86:a4:89:56:2e:
     96:10:57:f1:1c:74:14:0c:2c:9f:a8:69:22:20:fc:60:79:a2:
     05:4e:c1:80:4a:e0:1a:e0:27:2d:b3:21:25:43:3a:31:5c:74:
     e9:b7:e7:c1:ba:8d:4e:86:3f:13:38:ec:53:7b:f6:10:4c:38:
     60:12:cf:4c:31:86:3b:43:08:23:60:61:b3:2d:11:13:c1:f1:
     4e:3d:1e:38:2b:a1:df:22:8d:83:e3:bd:b5:b5:85:f7:b3:32:
     e1:44:f6:a3:3f:fe:68:1c:82:9d:8e:77:a1:d1:96:06:b6:f7:
     b3:c6:95:45:e0:75:47:72:81:36:cf:cb:dd:58:3a:97:17:07:
     be:37:dd:5c:15:d4:07:16:70:9a:ca:92:0d:82:8e:e7:8c:9f:
     1f:b8:f4:71:bc:6f:50:d9:c4:5b:36:84:56:1a:ef:6c:08:d4:
     4c:51:47:9c:47:a0:57:7b

ClientCertificate:

Data:
    Version: 3 (0x2)
    Serial Number: 4098 (0x1002)
Signature Algorithm: sha256WithRSAEncryption
    Issuer: C=GB, ST=England, O=Alice Ltd, CN=IntermediateKarthik
    Validity
        Not Before: Sep 25 08:45:02 2017 GMT
        Not After : Oct  5 08:45:02 2018 GMT
    Subject: C=GB, ST=England, O=Alice Ltd, CN=Client
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
            Public-Key: (2048 bit)
            Modulus:
                00:e4:84:57:46:c8:00:99:5e:5a:be:3b:77:d9:83:
                1d:05:e7:9d:d4:14:40:a1:fa:07:52:37:26:d6:31:
                25:fb:48:05:4f:e9:7d:4b:3a:c3:6c:5b:71:59:34:
                be:97:57:dd:3c:69:70:c4:92:1d:3b:65:b8:42:39:
                ad:e6:7d:16:3b:08:a9:71:ee:34:e2:78:19:32:89:
                2c:6c:f5:36:9c:36:ed:e1:93:0f:9d:3c:e1:66:ab:
                52:da:43:47:0a:9a:3c:4b:fe:08:9b:21:b3:22:2d:
                2c:32:e2:0a:e5:5a:0a:e9:83:9f:23:f4:b6:86:22:
                2c:96:08:f2:9f:06:92:3f:24:38:54:02:33:b3:c0:
                4e:73:23:ed:9c:f8:72:27:42:1b:f1:98:1e:0a:7f:
                7a:f2:2c:80:5d:9a:86:1e:3e:14:1f:1f:77:12:58:
                55:93:8b:b0:a1:0a:6d:cf:d6:38:3c:99:a7:ca:f5:
                19:b3:82:41:77:55:9c:69:12:27:5a:36:3d:1c:46:
                ea:00:5e:47:68:33:a4:c8:d0:f2:2d:3e:1a:95:52:
                25:ee:86:c4:95:01:fe:90:59:84:eb:1a:3e:77:3f:
                0b:f3:40:05:18:df:90:c9:c5:96:3e:64:33:a2:16:
                1b:0c:3c:9b:11:68:92:63:74:0e:2b:b9:02:ce:ad:
                2f:af
            Exponent: 65537 (0x10001)
    X509v3 extensions:
        X509v3 Basic Constraints:
            CA:FALSE
        Netscape Cert Type:
            SSL Client, S/MIME
        Netscape Comment:
            OpenSSL Generated Client Certificate
        X509v3 Subject Key Identifier:
            8F:CE:D1:2D:08:FA:84:57:8D:5F:16:69:CF:87:C3:16:01:41:7B:0C
        X509v3 Authority Key Identifier:
            keyid:F9:0C:48:44:36:E1:BA:8F:A7:B3:A5:EF:21:89:2F:01:D5:23:4A:99

        X509v3 Key Usage: critical
            Digital Signature, Non Repudiation, Key Encipherment
        X509v3 Extended Key Usage:
            TLS Web Client Authentication, E-mail Protection
Signature Algorithm: sha256WithRSAEncryption
     29:2d:f0:98:47:4e:8d:b1:62:fb:af:cb:fe:1c:56:22:51:ed:
     93:c5:93:05:a3:56:24:a3:d5:dc:65:5b:f0:ba:6b:04:d0:55:
     08:c5:86:38:36:2b:ed:78:ea:9e:b5:05:34:eb:5e:96:6a:1b:
     f1:76:41:5b:7a:aa:13:e0:b1:92:4a:2b:f7:2d:f0:76:e1:c0:
     da:50:94:50:50:6d:5e:a8:bd:03:9e:d4:67:58:a4:3e:22:1b:
     e6:f0:31:d4:30:56:ff:03:29:59:94:db:49:8c:16:23:b3:7a:
     3e:46:ce:9d:35:37:09:e5:4a:b2:8a:dd:fb:ef:fd:e8:3c:24:
     38:d1:69:65:b4:05:43:7b:c4:15:c9:df:7c:2f:cc:59:a0:d8:
     f4:53:09:aa:1e:2f:5c:7c:48:f8:86:67:1d:11:15:59:80:cf:
     55:21:0b:b9:30:62:e8:e3:72:fc:4e:55:55:d7:ff:b1:49:95:
     c9:5b:28:ae:56:89:e4:13:ee:71:ac:f3:a8:12:ce:93:34:f0:
     4f:99:11:82:e2:56:66:83:97:fd:ae:a7:a9:95:1a:85:ec:47:
     d4:1d:90:3b:d5:18:10:6a:05:cf:91:65:de:4d:8b:b6:b2:59:
     a2:a6:9e:f2:c3:ac:cb:33:9c:51:a0:53:2a:3a:75:83:dd:1f:
     3d:18:d1:00:45:34:c4:73:5c:74:d5:b8:f8:71:d1:83:22:bf:
     66:b6:db:6b:c6:4d:38:55:7b:b4:09:42:c5:1f:7a:21:9f:f9:
     ff:98:ce:e7:68:0f:48:6f:39:7b:fd:7b:fc:2f:e5:43:ba:f2:
     20:d8:6f:b4:cc:c4:26:d8:26:c5:b6:2e:17:d5:2b:f2:af:e7:
     a8:e3:90:e9:02:8e:5b:fe:46:f9:1c:88:89:a2:fb:0a:ec:25:
     48:97:97:b7:e0:31:be:81:3e:34:f7:94:76:1e:fa:63:76:f8:
     f9:51:e6:88:87:53:39:a5:83:ad:30:f2:f3:b5:bf:7c:d2:9c:
     da:66:a6:38:81:f5:22:8e:65:a9:a0:03:25:98:19:26:ec:2c:
     6d:43:b5:3c:4f:20:de:c6:cb:1d:7a:44:79:57:36:62:fa:22:
     03:9c:62:ce:10:39:11:2b:a9:ca:7c:1f:a2:f2:06:7b:44:83:
     0e:d3:76:65:b2:8a:94:d9:bb:30:32:e8:f7:87:dc:62:4d:d9:
     0b:98:70:95:32:8a:17:3a:bc:55:64:44:7d:9c:02:cd:b4:5d:
     61:93:af:e1:c6:75:84:44:88:ca:9b:1a:fa:09:56:b2:a5:5c:
     01:32:32:02:48:f6:a0:41:d5:8e:75:26:8b:06:ae:c7:ff:21:
     1c:05:4d:a0:32:3c:99:26
keyankay picture keyankay on 26 Sep 2017

I also checked in 3.2.7, the problem exists. I installed etcd 3.1.10 and i do not see the issue. I sense this is an etcd bug

How did you generate certs? Please provide us with concise reproducible steps.

gyuho picture gyuho on 26 Sep 2017

I generated the certificates (in two methods). Second method is at the end of this comment

First Method:
Step by step is mentioned here

https://jamielinux.com/docs/openssl-certificate-authority/appendix/root-configuration-file.html

However please find the concise steps here:

First you need to download these two files
https://jamielinux.com/docs/openssl-certificate-authority/appendix/root-configuration-file.html
https://jamielinux.com/docs/openssl-certificate-authority/appendix/intermediate-configuration-file.html

and follow these steps

Root directory
mkdir /root/ca
cd /root/ca
mkdir certs crl newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial

Create root key
cd /root/ca
openssl genrsa -aes256 -out private/ca.key.pem 4096
chmod 400 private/ca.key.pem
cd /root/ca
openssl req -config openssl.cnf \
-key private/ca.key.pem \
-new -x509 -days 7300 -sha256 -extensions v3_ca \
-out certs/ca.cert.pem

Create intermediate pair
mkdir /root/ca/intermediate
cd /root/ca/intermediate
mkdir certs crl csr newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
echo 1000 > /root/ca/intermediate/crlnumber

Create intermediate key

cd /root/ca
openssl genrsa -aes256 \
-out intermediate/private/intermediate.key.pem 4096
chmod 400 intermediate/private/intermediate.key.pem

cd /root/ca
openssl req -config intermediate/openssl.cnf -new -sha256 \
-key intermediate/private/intermediate.key.pem \
-out intermediate/csr/intermediate.csr.pem

Create the chain file
cat intermediate/certs/intermediate.cert.pem \
certs/ca.cert.pem > intermediate/certs/ca-chain.cert.pem
chmod 444 intermediate/certs/ca-chain.cert.pem

Prepare server and client certificate
Server

cd /root/ca
openssl genrsa -aes256 \
-out intermediate/private/www.example.com.key.pem 2048
chmod 400 intermediate/private/www.example.com.key.pem

Create server certificate
cd /root/ca
openssl req -config intermediate/openssl.cnf \
-key intermediate/private/www.example.com.key.pem \
-new -sha256 -out intermediate/csr/www.example.com.csr.pem

Intermediate CA to sign the certificate
cd /root/ca
openssl ca -config intermediate/openssl.cnf \
-extensions server_cert -days 375 -notext -md sha256 \
-in intermediate/csr/www.example.com.csr.pem \
-out intermediate/certs/www.example.com.cert.pem
chmod 444 intermediate/certs/www.example.com.cert.pem

Verify server certificate
openssl verify -CAfile intermediate/certs/ca-chain.cert.pem \
intermediate/certs/www.example.com.cert.pem

Create client certificate

openssl genrsa -aes256 -out intermediate/private/client.key.pem 2048
chmod 400 intermediate/private/client.key.pem
cd /root/ca
openssl req -config intermediate/openssl.cnf \
-key intermediate/private/client.key.pem \
-new -sha256 -out intermediate/csr/client.csr.pem

Intermediate CA to sign the certificate
cd /root/ca
openssl ca -config intermediate/openssl.cnf \
-extensions usr_cert-days 375 -notext -md sha256 \
-in intermediate/csr/client.csr.pem \
-out intermediate/certs/client.cert.pem
chmod 444 intermediate/certs/client.cert.pem

Verify server certificate
openssl verify -CAfile intermediate/certs/ca-chain.cert.pem \
intermediate/certs/www.example.com.cert.pem

Verify client certificate
openssl verify -CAfile intermediate/certs/ca-chain.cert.pem \
intermediate/certs/client.cert.pem

Eventhough, i did not see any startup issues in 3.1.10, my request still did not go through in this version also.

With 3.1.10 I get this error from etcd

[root@vm-188 cfssl]# curl -v --noproxy '*' --cacert /root/etcd/server_certs/ca-chain.cert.pem --cert /root/etcd/client_certs/client.cert.pem --key /root/etcd/client_certs/client.key.pem -L https://localhost:2379/v2/keys/foo -XPUT -d value=bar -v
About to connect() to localhost port 2379 (#0)
Trying ::1...
Connected to localhost (::1) port 2379 (#0)
Initializing NSS with certpath: sql:/etc/pki/nssdb
CAfile: /root/etcd/server_certs/ca-chain.cert.pem
CApath: none
Server certificate:
subject: CN=ServerKarthik,O=Alice Ltd,ST=England,C=GB
start date: Sep 25 08:28:06 2017 GMT
expire date: Oct 05 08:28:06 2018 GMT
common name: ServerKarthik
issuer: CN=IntermediateKarthik,O=Alice Ltd,ST=England,C=GB
NSS error -12276 (SSL_ERROR_BAD_CERT_DOMAIN)

Second Method:
Alternative step for generating certificate for the same issue

CA certificate:
openssl genrsa -out MyRootCA.key 2048
openssl req -x509 -new -nodes -key MyRootCA.key -sha256 -days 1024 -out MyRootCA.pem

Client CSR:
openssl genrsa -out MyClient1.key 2048
openssl req -new -key MyClient1.key -out MyClient1.csr

CA Client Signing:

openssl x509 -req -in MyClient1.csr -CA MyRootCA.pem -CAkey MyRootCA.key -CAcreateserial -out MyClient1.pem -days 1024 -sha256

Server CSR:
openssl genrsa -out MyServer1.key 2048
openssl req -new -key MyServer1.key -out MyServer1.csr

CA Server signing

openssl x509 -req -in MyServer1.csr -CA MyRootCA.pem -CAkey MyRootCA.key -CAcreateserial -out MyServer1.pem -days 1024 -sha256

If you provide me an email id, i can mail the certificates

keyankay picture keyankay on 27 Sep 2017

Unable to communicate securely with peer: requested domain name does not match the server's certificate.

On your second logs, I don't see any X509v3 Subject Alternative Name:?

gyuho picture gyuho on 29 Sep 2017

I tried again with CA certificate with X509V3 Subject Alternative Name using version etcd-v3.1.10 (where there is no error observed in etcd side).

Certificates generated as mentioned in
https://coreos.com/os/docs/latest/generate-self-signed-certificates.html

in etcdv-3.2.7 i see the following error"etcdserver/api/v3rpc: Failed to dial 10.53.70.188:2379: connection error: desc = "transport: remote error: tls: bad certificate"; please retry."

Execution command
etcd --client-cert-auth --trusted-ca-file=/root/cfssl/ca.pem --cert-file=/root/cfssl/server.pem --key-file=/root/cfssl/server-key.pem --advertise-client-urls https://10.53.70.188:2379 --listen-client-urls https://10.53.70.188:2379

`Certificate:

Data:
    Version: 3 (0x2)
    Serial Number:
        44:ce:1f:cc:af:79:c7:0b:cb:f9:96:01:8a:91:20:52:3c:53:50:dc
Signature Algorithm: sha256WithRSAEncryption
    Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Validity
        Not Before: Oct 10 05:03:00 2017 GMT
        Not After : Oct  9 05:03:00 2022 GMT
    Subject: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
            Public-Key: (2048 bit)
            Modulus:
                00:cc:08:a9:0a:7b:a1:ee:44:b0:fc:c8:82:40:59:
                2d:26:88:e8:23:6c:00:aa:4a:5b:e6:c0:33:05:08:
                6d:0f:29:31:5b:7d:ca:4b:40:9f:62:70:79:54:31:
                35:c6:f0:b9:e3:b7:18:61:2f:32:de:9a:07:6f:2f:
                10:e7:eb:cc:88:1d:93:38:42:71:47:76:12:37:9b:
                24:1e:59:be:4e:3a:45:40:20:76:f3:5c:a5:48:9e:
                c2:64:e3:45:f4:34:3d:a7:a5:58:7a:91:36:ac:24:
                24:4a:0d:0a:a5:a1:87:17:0f:af:81:af:64:a3:29:
                1d:23:e7:92:d2:6d:40:8e:ac:6b:83:30:4b:3b:27:
                6d:c0:a0:c6:2d:6e:73:d9:cb:89:14:a3:9c:5e:56:
                58:45:84:a4:56:0d:cb:74:24:43:85:1d:4d:3d:73:
                6f:d5:c4:40:aa:9d:85:66:e2:50:b3:f3:6f:29:b9:
                87:8b:36:44:95:30:73:e3:5a:ca:21:8c:5d:78:02:
                93:7b:1e:78:28:e6:c0:1b:d8:11:8a:ec:0f:59:3f:
                44:8d:ea:ed:32:6f:0b:ca:70:0f:cb:08:c8:86:00:
                b3:8a:6d:7e:9c:47:f0:10:ee:6d:fa:15:12:ef:42:
                ca:57:08:5b:b6:54:d8:86:83:ae:38:e2:11:9e:2b:
                5e:c5
            Exponent: 65537 (0x10001)
    X509v3 extensions:
        X509v3 Key Usage: critical
            Certificate Sign, CRL Sign
        X509v3 Basic Constraints: critical
            CA:TRUE, pathlen:2
        X509v3 Subject Key Identifier:
            2C:56:0F:FC:AA:97:E1:C9:07:3C:BF:D6:77:40:F8:D3:BA:14:19:42
        X509v3 Authority Key Identifier:
            keyid:2C:56:0F:FC:AA:97:E1:C9:07:3C:BF:D6:77:40:F8:D3:BA:14:19:42

Signature Algorithm: sha256WithRSAEncryption
     7a:a7:62:9a:e7:6d:a8:d7:f4:f3:7f:8f:04:58:81:07:c9:3e:
     2c:cc:56:b1:e5:99:3e:9f:d9:94:22:6b:16:51:2b:d5:8c:ad:
     2d:bb:5c:27:9e:cf:bf:02:de:83:1e:97:59:93:56:a1:5f:b7:
     05:83:20:52:07:4f:b4:96:4f:a4:41:6d:d4:19:f1:62:53:49:
     7b:84:f4:8c:b2:7e:1f:2c:d4:dc:4e:22:db:ed:ef:8f:59:bf:
     e3:5b:52:30:d1:47:dd:af:20:55:e8:a6:bf:fc:5f:2d:44:b7:
     53:76:09:44:2b:09:7b:d7:84:aa:f9:f3:79:72:b9:4a:85:5f:
     4f:8e:0f:8b:4a:84:a3:e4:97:fd:c7:8f:7f:7c:12:29:ea:18:
     d0:ed:e3:d4:85:f9:c8:10:ed:db:e2:d7:3a:03:d2:2a:25:a6:
     9e:5a:01:13:cd:44:14:b7:df:29:fc:68:59:f6:0d:bb:1a:f6:
     ac:e7:74:1c:c3:47:95:9f:4e:88:75:49:7d:08:28:7c:d0:c2:
     cc:25:ce:83:16:28:56:a2:a4:7b:c1:39:37:e9:05:0f:75:be:
     52:db:c9:41:1d:24:5b:8a:a3:27:cf:88:6f:45:0a:18:89:ac:
     69:ed:7e:ab:b6:0e:6b:d9:b0:f3:00:ef:53:2a:18:bf:35:d1:
    ed:77:5a:d1`

Server.pem
Certificate:
```
Data:
Version: 3 (0x2)
Serial Number:
53:b9:d8:c1:4f:8b:66:36:46:ea:6b:13:76:ec:1f:47:3e:f6:83:c7
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
Validity
Not Before: Sep 26 04:55:00 2017 GMT
Not After : Sep 25 04:55:00 2022 GMT
Subject: C=US, ST=San Francisco, L=CA, CN=example.net
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:20:8c:19:15:0a:e6:58:e5:80:8b:3c:1f:09:05:
e4:85:d8:2b:29:49:a1:28:d6:69:fc:d0:61:99:40:
45:c5:3b:a4:a4:31:62:63:8c:87:77:43:87:4a:43:
e2:2b:40:66:b1:fa:fa:8c:7b:fd:74:bc:25:60:7e:
5f:6c:8a:44:27
ASN1 OID: prime256v1
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
57:4C:55:B4:D7:CF:7D:F6:5C:23:8D:B0:93:7F:8A:09:F0:87:05:91
X509v3 Authority Key Identifier:
keyid:21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23

        X509v3 Subject Alternative Name:
            DNS:www.example.net, IP Address:10.53.70.188
Signature Algorithm: sha256WithRSAEncryption
     81:a6:59:9c:ea:ee:c8:56:6f:c4:7a:aa:80:85:f4:71:f0:a0:
     ac:dd:1f:0c:95:57:f7:be:b2:2f:e8:08:74:f1:aa:2a:47:59:
     c4:ff:15:c2:3b:84:f2:26:48:51:4f:d3:f8:c1:46:28:c3:72:
     23:87:2c:bf:2c:2e:2e:53:d7:86:e9:2b:28:98:6c:01:ac:0b:
     9f:e5:86:55:47:87:fe:4a:82:55:23:36:ac:7a:9f:f0:76:7f:
     10:1e:92:01:a5:29:63:18:c6:af:1d:f9:b1:be:8c:32:87:7f:
     45:72:44:0e:c8:d5:a1:1f:23:71:4e:cf:ec:39:92:fb:da:44:
     34:b9:d2:dd:f8:75:68:ce:d8:f4:13:63:62:3c:e1:48:dc:34:
     81:cf:fb:90:04:13:50:a7:dc:5a:cf:e7:da:70:b5:05:f8:d7:
     44:98:19:4f:5d:a4:f7:3e:7d:1b:fc:b3:59:60:e0:07:6b:06:
     54:78:31:18:60:c0:92:7e:68:88:47:62:3d:eb:e8:ff:d8:13:
     82:53:59:84:dc:59:bb:fb:c3:6e:d9:14:d1:e1:8e:d9:03:2e:
     28:bd:23:09:09:88:ec:df:a5:6d:26:b4:a3:fc:96:d2:4b:05:
     cc:b0:d1:6e:fb:1e:b0:f7:3b:73:14:5e:11:49:e7:48:77:da:
     bd:b1:d3:71

Client certificate

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
61:7f:67:88:1a:31:72:b1:87:b9:81:40:5f:d9:6c:09:83:bb:43:5d
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=San Francisco, L=CA, O=My Company Name, OU=Org Unit 2, CN=My own CA
Validity
Not Before: Oct 10 05:01:00 2017 GMT
Not After : Oct 9 05:01:00 2022 GMT
Subject: C=US, ST=San Francisco, L=CA, CN=client
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:b3:1b:d9:92:37:2e:e9:20:c2:45:32:ba:94:fd:
23:7d:88:a1:6e:00:f9:d8:82:20:9e:c7:34:a0:04:
a0:f5:bc:3a:5c:71:1f:db:54:98:9d:71:64:48:43:
01:39:54:b3:d3:c4:7f:9e:c4:85:6e:9f:43:86:01:
86:79:bc:93:0f
ASN1 OID: prime256v1
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Client Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
24:B8:3A:68:6D:E2:94:10:C4:81:FE:6F:2D:41:5C:EA:5F:13:39:25
X509v3 Authority Key Identifier:
keyid:21:D5:6A:6D:79:74:83:4F:25:6B:01:22:F3:DA:AB:09:4A:BF:0A:23

        X509v3 Subject Alternative Name:
            IP Address:10.53.70.188
Signature Algorithm: sha256WithRSAEncryption
     34:7c:fd:a6:aa:82:56:6e:52:28:a9:49:c8:8b:d7:28:17:be:
     2f:82:39:db:e3:65:90:e4:e8:fe:40:59:1e:7d:9f:dc:d4:9f:
     15:c7:bc:07:da:f7:d8:1b:30:88:08:f8:5c:b9:6f:ad:86:e7:
     0b:26:b9:ea:cd:99:59:a1:3e:c5:42:b2:8b:c6:4e:b6:e1:b8:
     24:25:47:4b:a7:17:03:5e:4f:25:96:3b:bc:b6:ba:b1:25:51:
     20:1c:7a:2f:db:98:24:1a:ec:22:e0:73:07:ce:7d:52:85:85:
     93:c6:a1:b3:17:7a:07:61:ab:d3:97:32:a5:06:14:e1:c0:fc:
     02:3e:03:87:e3:21:57:d7:01:3a:b9:1a:46:8d:99:9d:9a:b3:
     23:0c:71:7b:ba:ee:e2:bc:d1:41:23:f0:3d:7c:65:58:2d:2c:
     ff:fd:48:c9:77:3e:5b:0d:b2:00:1d:88:53:44:9b:d7:a3:c2:
     b6:f2:ca:b5:0d:dd:10:b1:17:7f:34:67:17:8c:a2:04:0b:b7:
     41:4d:b1:17:4e:69:c9:cf:34:35:ec:6e:9d:4a:db:13:2b:2b:
     c0:d4:6f:a1:87:07:98:56:c0:37:14:a6:aa:06:a9:e3:7c:e8:
     77:3c:25:6d:b9:d8:e2:3f:66:f7:84:39:2a:d3:09:5f:29:e3:
     2a:94:d4:a3

```

I get the error
curl -v --noproxy '*' --cacert /root/cfssl/ca.pem --cert /root/cfssl/client.pem --key /root/cfssl//client.key.pem -L https://10.53.70.188:2379/v2/keys/foo -XPUT -d value=bar -v

  • About to connect() to 10.53.70.188 port 2379 (#0)
  • Trying 10.53.70.188...
  • Connected to 10.53.70.188 (10.53.70.188) port 2379 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • CAfile: /root/cfssl/ca.pem
    CApath: none
  • unable to load client key: -8178 (SEC_ERROR_BAD_KEY)
  • NSS error -8178 (SEC_ERROR_BAD_KEY)
  • Peer's public key is invalid.
  • Closing connection 0
    curl: (58) unable to load client key: -8178 (SEC_ERROR_BAD_KEY)
keyankay picture keyankay on 10 Oct 2017

Sorry, couldn't have time to reproduce. In the meantime, could you also try http://play.etcd.io/install with latest etcd release? It explains cfssl the same way as etcd tests TLS.

gyuho picture gyuho on 10 Oct 2017

I've been able to reproduce this on 3.2.5 and 3.2.7. Doesn't seem to be occurring on a 3.1.9 cluster. The "bad certificate" error only comes up when etcd starts up, but the cluster is reporting itself as healthy (and works without any other noticeable issues) otherwise.

In our case, we're using etcd for Openshift Origin. We used the steps to generate the certs directly from their documentation (see step 7): https://docs.openshift.org/3.6/admin_guide/backup_restore.html#backup-restore-adding-etcd-hosts

rezie picture rezie on 2 Nov 2017
👍2

@rezie
Did you try with both client and server certificate ?

keyankay picture keyankay on 3 Nov 2017

@keyankay Do you mean whether I generated both the client and server sets? If so, then yes; the steps I followed are all in the documentation that I linked.

rezie picture rezie on 3 Nov 2017

In case somebody else comes across this issue, I was able to reproduce in 3.2 (3.2.11) build by using client certs with an empty SAN section. By adding the server IP address into the SAN, I was able to get connections to re-establish.

mnewswanger picture mnewswanger on 18 Feb 2018

I'm experiencing the same issue with official docker image 3.2.16 despite the cerificates are completely fine (used terraform tls provider to generate the complete chain).

Here is the output of docker logs:

$ docker logs etcd
2018-02-22 21:33:55.922579 I | pkg/flags: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=https://etcd-1.internal:2379
2018-02-22 21:33:55.922665 I | pkg/flags: recognized and used environment variable ETCD_CA_FILE=/etc/etcd/certs/etcd_ca.pem
2018-02-22 21:33:55.922673 I | pkg/flags: recognized and used environment variable ETCD_CERT_FILE=/etc/etcd/certs/etcd_server.pem
2018-02-22 21:33:55.922683 I | pkg/flags: recognized and used environment variable ETCD_CLIENT_CERT_AUTH=true
2018-02-22 21:33:55.922700 I | pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/etcd-data
2018-02-22 21:33:55.922726 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=https://etcd-1.internal:2380
2018-02-22 21:33:55.922733 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=etcd-1.internal=https://etcd-1.internal:2380
2018-02-22 21:33:55.922789 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=new
2018-02-22 21:33:55.922802 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=etcd-k8-cluster
2018-02-22 21:33:55.922809 I | pkg/flags: recognized and used environment variable ETCD_KEY_FILE=/etc/etcd/certs/etcd_server_key.pem
2018-02-22 21:33:55.922818 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
2018-02-22 21:33:55.922829 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
2018-02-22 21:33:55.922848 I | pkg/flags: recognized and used environment variable ETCD_NAME=etcd-1.internal
2018-02-22 21:33:55.922859 I | pkg/flags: recognized and used environment variable ETCD_PEER_CA_FILE=/etc/etcd/certs/etcd_ca.pem
2018-02-22 21:33:55.922866 I | pkg/flags: recognized and used environment variable ETCD_PEER_CERT_FILE=/etc/etcd/certs/etcd_peer.pem
2018-02-22 21:33:55.922873 I | pkg/flags: recognized and used environment variable ETCD_PEER_CLIENT_CERT_AUTH=true
2018-02-22 21:33:55.922882 I | pkg/flags: recognized and used environment variable ETCD_PEER_KEY_FILE=/etc/etcd/certs/etcd_peer_key.pem
2018-02-22 21:33:55.922891 I | pkg/flags: recognized and used environment variable ETCD_PEER_TRUSTED_CA_FILE=/etc/etcd/certs/etcd_ca.pem
2018-02-22 21:33:55.922898 I | pkg/flags: recognized and used environment variable ETCD_PROXY=off
2018-02-22 21:33:55.922935 I | pkg/flags: recognized and used environment variable ETCD_TRUSTED_CA_FILE=/etc/etcd/certs/etcd_ca.pem
2018-02-22 21:33:55.923002 I | etcdmain: etcd Version: 3.2.16
2018-02-22 21:33:55.923023 I | etcdmain: Git SHA: 121edf046
2018-02-22 21:33:55.923041 I | etcdmain: Go Version: go1.8.5
2018-02-22 21:33:55.923046 I | etcdmain: Go OS/Arch: linux/amd64
2018-02-22 21:33:55.923051 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2018-02-22 21:33:55.923130 I | embed: peerTLS: cert = /etc/etcd/certs/etcd_peer.pem, key = /etc/etcd/certs/etcd_peer_key.pem, ca = /etc/etcd/certs/etcd_ca.pem, trusted-ca = /etc/etcd/certs/etcd_ca.pem, client-cert-auth = true
2018-02-22 21:33:55.924260 I | embed: listening for peers on https://0.0.0.0:2380
2018-02-22 21:33:55.924316 I | embed: listening for client requests on 0.0.0.0:2379
2018-02-22 21:33:55.935526 I | pkg/netutil: resolving etcd-1.internal:2380 to 10.0.2.24:2380
2018-02-22 21:33:55.936757 I | pkg/netutil: resolving etcd-1.internal:2380 to 10.0.2.24:2380
2018-02-22 21:33:55.936811 I | etcdserver: name = etcd-1.internal
2018-02-22 21:33:55.936818 I | etcdserver: data dir = /etcd-data
2018-02-22 21:33:55.936825 I | etcdserver: member dir = /etcd-data/member
2018-02-22 21:33:55.936832 I | etcdserver: heartbeat = 100ms
2018-02-22 21:33:55.936838 I | etcdserver: election = 1000ms
2018-02-22 21:33:55.936843 I | etcdserver: snapshot count = 100000
2018-02-22 21:33:55.936877 I | etcdserver: advertise client URLs = https://etcd-1.internal:2379
2018-02-22 21:33:55.936886 I | etcdserver: initial advertise peer URLs = https://etcd-1.internal:2380
2018-02-22 21:33:55.936901 I | etcdserver: initial cluster = etcd-1.internal=https://etcd-1.internal:2380
2018-02-22 21:33:55.943476 I | etcdserver: starting member dc051765c3f0f8ee in cluster fdb09331e905ba38
2018-02-22 21:33:55.943506 I | raft: dc051765c3f0f8ee became follower at term 0
2018-02-22 21:33:55.943516 I | raft: newRaft dc051765c3f0f8ee [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2018-02-22 21:33:55.943522 I | raft: dc051765c3f0f8ee became follower at term 1
2018-02-22 21:33:55.956019 W | auth: simple token is not cryptographically signed
2018-02-22 21:33:55.965842 I | etcdserver: starting server... [version: 3.2.16, cluster version: to_be_decided]
2018-02-22 21:33:55.966966 I | embed: ClientTLS: cert = /etc/etcd/certs/etcd_server.pem, key = /etc/etcd/certs/etcd_server_key.pem, ca = /etc/etcd/certs/etcd_ca.pem, trusted-ca = /etc/etcd/certs/etcd_ca.pem, client-cert-auth = true
2018-02-22 21:33:55.967382 I | etcdserver/membership: added member dc051765c3f0f8ee [https://etcd-1.internal:2380] to cluster fdb09331e905ba38
2018-02-22 21:33:56.451204 I | raft: dc051765c3f0f8ee is starting a new election at term 1
2018-02-22 21:33:56.451538 I | raft: dc051765c3f0f8ee became candidate at term 2
2018-02-22 21:33:56.451894 I | raft: dc051765c3f0f8ee received MsgVoteResp from dc051765c3f0f8ee at term 2
2018-02-22 21:33:56.452056 I | raft: dc051765c3f0f8ee became leader at term 2
2018-02-22 21:33:56.452173 I | raft: raft.node: dc051765c3f0f8ee elected leader dc051765c3f0f8ee at term 2
2018-02-22 21:33:56.453644 I | etcdserver: setting up the initial cluster version to 3.2
2018-02-22 21:33:56.453803 I | etcdserver: published {Name:etcd-1.internal ClientURLs:[https://etcd-1.internal:2379]} to cluster fdb09331e905ba38
2018-02-22 21:33:56.454507 I | embed: ready to serve client requests
2018-02-22 21:33:56.455495 I | embed: serving client requests on [::]:2379
2018-02-22 21:33:56.469146 N | etcdserver/membership: set the initial cluster version to 3.2
2018-02-22 21:33:56.477496 I | etcdserver/api: enabled capabilities for version 3.2
WARNING: 2018/02/22 21:33:56 Failed to dial 0.0.0.0:2379: connection error: desc = "transport: authentication handshake failed: remote error: tls: bad certificate"; please retry.

The single node cluster appears to be healthy though:

# curl -v --cacert ./etcd_ca.pem --cert ./etcd_client.pem --key ./etc_client_key.pem http://etcd-1.internal:2379/health | jq
*   Trying 10.0.2.24...
* Connected to etcd-1.internal (10.0.2.24) port 2379 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: ./etcd_ca.pem
  CApath: none
* NSS: client certificate from file
*       subject: CN=etcd client,O=etcd
*       start date: Feb 22 21:29:49 2018 GMT
*       expire date: Feb 20 21:29:49 2028 GMT
*       common name: etcd client
*       issuer: CN=Trusted Root CA,OU=N/A,O=Root CA,postalCode=N/A,STREET=N/A,ST=London,C=UK
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=etcd server,O=etcd
*       start date: Feb 22 21:29:49 2018 GMT
*       expire date: Feb 20 21:29:49 2028 GMT
*       common name: etcd server
*       issuer: CN=Trusted Root CA,OU=N/A,O=Root CA,postalCode=N/A,STREET=N/A,ST=London,C=UK
> GET /health HTTP/1.1
> User-Agent: curl/7.29.0
> Host: etcd-1.internal:2379
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 22 Feb 2018 21:55:02 GMT
< Content-Length: 18
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host etcd-1.internal left intact
{
  "health": "true"
}

I'm sure that there is nothing wrong with certificates as I use similar chains in other applications and they work just fine.

Constantin07 picture Constantin07 on 22 Feb 2018
👍1

Seems like you don't specify SAN field in your certs?

gyuho picture gyuho on 22 Feb 2018

@gyuho in fact I do and specify both DNS names and IP (for simplicity I'm testing now one node):

resource "tls_cert_request" "etcd_server_cert_req" {
  key_algorithm   = "${tls_private_key.etcd_server_priv_key.algorithm}"
  private_key_pem = "${tls_private_key.etcd_server_priv_key.private_key_pem}"

  subject {
    common_name  = "etcd server"
    organization = "etcd"
  }

  dns_names = [
    "etcd-1.internal",
    "etcd-2.internal",
    "etcd-3.internal",
    "etcd.internal",
    "localhost",
  ]

  ip_addresses = [
    "127.0.0.1",
    "10.0.2.24",
  ]
}

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            45:f6:1b:85:6d:11:af:fc:5f:42:38:6d:eb:ab:fc:eb
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=UK, ST=London/street=N/A/postalCode=N/A, O=Root CA, OU=N/A, CN=Trusted Root CA
        Validity
            Not Before: Feb 22 21:29:49 2018 GMT
            Not After : Feb 20 21:29:49 2028 GMT
        Subject: O=etcd, CN=etcd server
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:af:d8:54:d1:dc:ca:b2:a4:b9:69:3c:c1:8d:ab:
                    ee:f3:36:00:c7:12:d5:b6:99:8f:f6:11:c3:2d:0a:
                    d0:8d:44:8f:f1:4f:aa:f8:58:65:ae:07:c3:3a:c1:
                    ca:2a:e8:31:da:9d:91:42:5a:30:19:f2:e3:b1:db:
                    46:1d:68:3b:41:13:c9:69:74:94:5f:fd:3e:19:ed:
                    22:96:39:f6:62:4b:38:c5:7b:d9:70:ef:33:2d:a0:
                    58:5f:d3:cb:43:8a:9f:7f:f5:ed:93:20:39:1a:b6:
                    ee:7c:ba:79:56:b5:1c:cd:b8:8b:d3:c7:82:a4:cf:
                    ab:60:c3:1c:de:f7:a2:ee:d5:dc:df:95:79:5e:b8:
                    e9:d1:42:40:79:2f:74:4e:22:13:77:d2:47:65:da:
                    48:59:72:9f:30:b5:f8:16:d3:1b:45:b6:ff:50:2d:
                    9e:60:54:4a:71:4e:f6:d0:b7:24:99:43:3a:44:65:
                    1d:58:92:59:2f:c2:bc:9a:5a:ea:d9:e1:ad:71:ae:
                    4c:ed:d3:b1:d3:a9:dc:10:55:e7:0b:90:4f:bf:19:
                    9a:63:32:c8:86:96:04:1f:75:33:89:5d:a9:14:83:
                    d3:7a:cd:ee:a1:38:32:cd:02:e8:36:b4:21:27:e5:
                    1f:ed:b8:5b:dc:5c:43:49:3b:24:a7:5b:a9:4b:7d:
                    06:e3
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Data Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                keyid:4B:42:4B:D1:3B:20:97:1B:02:51:46:1E:0B:E5:C9:B0:A7:7B:2A:14

            X509v3 Subject Alternative Name:
                DNS:etcd-1.internal, DNS:etcd-2.internal, DNS:etcd-3.internal, DNS:etcd.internal, DNS:localhost, IP Address:127.0.0.1, IP Address:10.0.2.24
...
Constantin07 picture Constantin07 on 22 Feb 2018

If I switch to the latest version 3.3.1 I get this:

...
2018-02-22 22:31:40.267065 I | embed: ready to serve client requests
2018-02-22 22:31:40.280177 I | embed: serving client requests on [::]:2379
2018-02-22 22:31:40.281609 N | etcdserver/membership: updated the cluster version from 3.2 to 3.3
2018-02-22 22:31:40.281974 I | etcdserver/api: enabled capabilities for version 3.3
2018-02-22 22:31:40.305192 I | embed: rejected connection from "127.0.0.1:40050" (error "tls: failed to verify client's certificate: x509: certificate specifies an incompatible key usage", ServerName "")
WARNING: 2018/02/22 22:31:40 Failed to dial 0.0.0.0:2379: connection error: desc = "transport: authentication handshake failed: remote error: tls: bad certificate"; please retry.

My peer certificate has no any ServerName defined in it.

Constantin07 picture Constantin07 on 22 Feb 2018

I am hitting the same issue with etcd 3.2.14

2018-04-05 04:38:28.175132 I | embed: ready to serve client requests 2018-04-05 04:38:28.175362 I | embed: serving client requests on 0.0.0.0:4001 WARNING: 2018/04/05 04:38:28 Failed to dial 0.0.0.0:4001: connection error: desc = "transport: authentication handshake failed: remote error: tls: bad certificate"; please retry. Error during start: open /var/vcap/sys/run/etcd/etcd.pid: no such file or directory 2018-04-05 04:38:57.754615 C | etcdserver: failed to purge wal file open /var/vcap/store/etcd/member/wal: no such file or directory

lgayatri picture lgayatri on 6 Apr 2018

I think the cause is missing or unclear documentation on what the server/peer & client certificates need to have in CN & SAN in order for it to work.

Constantin07 picture Constantin07 on 6 Apr 2018
👍5

Met the same issue on 3.3.3 cluster, etcdctl works fine, but not curl. I could do health check through curl, but put key request will return error "transport: authentication handshake failed: remote error: tls: bad certificate"

lynic picture lynic on 13 Apr 2018

My solution was to hit https://:2379 instead of https://127.0.0.1:2379 as I had previously. This made me think that the problem was the server certificate not including 12.0.0.1 in the SAN, but a repeat test proved 127.0.0.1 in the SAN did not solve the problem

geosword picture geosword on 23 Jan 2019
👍1 
X509v3 Extended Key Usage:
           TLS Web Server Authentication

A cert with this X509v3 usage can not be used for client auth is must also have include TLS Web Client Authentication. This is a server cert only.

hexfusion picture hexfusion on 23 Jan 2019

I think I am also running into this on the current version of etcd, with some changes.

CA pem

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            27:97:15:40:c5:11:d3:dd:ba:d9:37:58:af:3e:12:8f:49:6a:51:a4
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = Oregon, L = Portland, O = Kubernetes, OU = CA, CN = Kubernetes
        Validity
            Not Before: Oct 26 20:28:00 2020 GMT
            Not After : Oct 25 20:28:00 2025 GMT
        Subject: C = US, ST = Oregon, L = Portland, O = Kubernetes, OU = CA, CN = Kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:b6:a7:81:b9:f5:09:a4:35:f1:93:28:98:f0:2a:
                    6d:fd:a7:16:68:27:65:ac:9c:d6:c9:5d:0a:eb:f9:
                    b9:7a:9e:d0:d6:96:c0:46:8a:de:24:37:08:6f:95:
                    9e:13:77:ef:a1:c3:6c:66:df:ce:21:f1:d3:72:a5:
                    e4:40:51:2e:64:5c:e3:5f:29:a5:d5:e7:fe:32:08:
                    0c:09:93:8e:32:c8:4d:77:75:1e:61:66:c5:cc:30:
                    a1:43:36:c7:e7:e0:4d:3e:2c:ea:1b:71:b4:37:69:
                    ff:00:02:4a:ff:79:d6:03:db:37:69:7d:3d:b7:de:
                    4a:9f:df:df:a4:93:3a:66:85:3d:b1:b6:50:68:1e:
                    95:6d:95:18:1c:4d:a8:67:86:7e:31:b5:aa:d1:aa:
                    66:d3:5b:cf:16:45:18:4b:f3:60:74:b6:fe:f7:48:
                    0a:1b:50:1b:a5:82:84:ba:d4:a0:61:57:59:70:20:
                    e1:b0:9a:c2:0b:05:ee:20:27:d5:32:40:7d:63:52:
                    89:3f:0a:73:29:d0:2d:ef:9d:4c:26:de:ef:22:91:
                    75:11:cb:f0:84:04:4f:ca:72:5f:f4:56:c1:ca:88:
                    bc:72:c2:a0:3d:b8:30:52:a8:38:04:26:fd:e5:e8:
                    3c:93:0a:1f:e8:5b:8b:7d:c6:e3:6b:2b:8c:99:28:
                    1a:1b
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier:
                BA:F3:EC:B6:5B:F1:87:A4:2B:F9:C6:B7:EA:79:D0:92:FA:88:23:36
    Signature Algorithm: sha256WithRSAEncryption
         49:3e:3c:da:63:2b:6d:03:76:93:cc:16:35:1c:d6:6c:23:1c:
         23:c5:29:35:e4:e2:2b:57:97:e5:a1:e2:32:c4:44:10:b2:af:
         fd:00:50:16:0c:72:6d:39:74:00:06:a2:a6:14:30:1c:56:8b:
         61:f3:33:ac:6c:d8:b7:34:f8:c2:cd:dc:4b:83:9d:6b:58:d3:
         07:7f:54:04:f9:1f:4a:3f:8e:e4:8e:0d:ba:56:04:47:34:ae:
         ac:e3:43:26:4c:2d:a9:32:68:f7:27:b8:5e:ac:70:24:96:0e:
         99:c2:bf:8c:3d:88:2a:ad:2f:54:2b:f0:01:18:c3:9d:ca:e5:
         ea:1c:8e:4f:39:26:17:f5:d9:8e:22:02:ba:c8:e7:36:75:03:
         66:c7:72:fa:a1:09:bb:49:3f:dd:66:c5:f0:bf:8d:b5:59:b4:
         6e:d9:f8:4e:4b:a4:f3:57:f5:c3:6f:9b:4e:73:e5:13:08:f7:
         40:04:34:d3:d9:cb:1c:b4:40:66:35:c6:c1:3a:26:db:f1:fa:
         6a:f4:f2:e1:ab:22:dc:e3:67:30:80:21:dd:a9:01:a7:f8:27:
         3d:6d:87:37:24:a5:a7:1d:87:4b:85:12:49:96:d7:7d:5c:46:
         7d:bc:c1:37:bf:d9:4f:30:89:5f:c8:71:d1:26:4a:9f:78:f0:
         54:ac:3b:7b

server pem

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1e:25:88:e1:51:63:fa:05:2d:3a:95:a2:61:65:0c:8f:35:8d:bf:88
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = Oregon, L = Portland, O = Kubernetes, OU = CA, CN = Kubernetes
        Validity
            Not Before: Oct 26 21:01:00 2020 GMT
            Not After : Oct 26 21:01:00 2021 GMT
        Subject: C = US, ST = Oregon, L = Portland, O = Kubernetes, OU = Kubernetes The Hard Way, CN = kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:b6:e0:11:f2:ac:e5:93:f1:80:de:e1:ba:a6:8f:
                    63:38:21:99:18:d6:12:ba:a2:c8:cf:c5:4c:0e:89:
                    d3:68:ef:b1:84:8b:29:a9:1d:ca:c3:72:58:95:23:
                    28:22:98:b7:49:1d:8d:2e:22:fa:69:6b:41:fc:ea:
                    8f:7e:b5:96:aa:3b:df:4b:bf:4d:5b:8f:50:98:4b:
                    ff:47:d4:90:db:e1:af:d8:6a:6a:a1:96:a8:7f:b5:
                    53:fd:05:2c:b6:1e:86:1a:86:e9:86:e2:9e:cd:fb:
                    1d:6b:34:50:b0:89:cb:7d:d9:34:2f:3c:20:a4:f6:
                    4f:ff:cf:cf:81:a1:df:96:3b:2d:df:fd:99:02:bb:
                    4b:1b:15:6e:37:7f:fb:60:8e:83:9e:d2:77:fa:1a:
                    55:1d:7c:d6:6f:26:bc:fa:57:47:d0:55:6c:bc:03:
                    90:aa:dc:d9:f8:73:2f:31:4a:f1:bd:32:f2:b5:71:
                    1b:02:d3:94:0f:d8:0a:31:f7:53:92:12:24:b2:b2:
                    79:8f:b3:8e:04:16:6e:a0:a3:04:da:e1:d1:16:c0:
                    0c:1d:49:9d:2f:34:19:72:e2:2c:bb:f7:e2:00:da:
                    1d:15:3e:92:b7:20:6e:11:4e:f6:57:18:19:91:5f:
                    bf:8a:2f:fb:b3:f5:66:65:e7:27:a7:63:a9:b3:29:
                    59:7d
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                7B:F5:0C:57:00:77:95:94:1D:3C:DD:D7:F0:81:A3:C2:F0:DE:7E:FB
            X509v3 Authority Key Identifier:
                keyid:BA:F3:EC:B6:5B:F1:87:A4:2B:F9:C6:B7:EA:79:D0:92:FA:88:23:36

            X509v3 Subject Alternative Name:
                DNS:k8s-master-1.int.globius.org, DNS:k8s-master-2.int.org, DNS:k8s-worker-1.int.org, DNS:k8s-worker-2.int.org, DNS:k8s-worker-3.int.org, DNS:k8s-worker-4.int.org, DNS:hapvin100.int.org, DNS:localhost, DNS:kubernetes.default, IP Address:10.32.0.1, IP Address:10.249.60.1, IP Address:10.44.0.2, IP Address:10.249.60.2, IP Address:10.249.60.3, IP Address:10.44.0.3, IP Address:10.44.0.4, IP Address:10.200.0.52, IP Address:127.0.0.1
    Signature Algorithm: sha256WithRSAEncryption
         6e:80:4e:af:5b:cb:e0:8f:b5:d3:ac:49:f7:27:5e:cd:9b:b4:
         c3:fa:86:e6:e3:8d:06:51:ea:2b:74:15:6e:fe:0a:04:9f:e9:
         40:51:c7:da:9e:9b:d9:77:f4:e5:0d:5c:09:7f:77:39:2a:69:
         0c:bc:a7:42:39:16:78:1e:bc:fd:bd:4d:60:ae:05:75:30:e6:
         78:0c:23:93:35:a3:ff:cb:3a:e7:ee:a0:6f:43:8e:ae:88:3f:
         3c:f6:a6:95:74:3e:c4:89:62:aa:08:ec:9e:cb:4c:0a:f0:a5:
         2d:a0:4f:60:ea:fa:d9:db:df:80:46:22:a2:0c:78:1a:61:bd:
         71:31:89:df:fa:8e:65:91:1f:ea:ed:6b:f0:fb:5b:7f:38:c9:
         f5:1f:cb:04:1c:9e:14:c9:97:0b:57:5b:03:1b:49:d0:07:39:
         f8:ef:62:e4:e8:fe:91:23:14:3c:2f:9a:7d:b9:21:19:84:52:
         d7:1e:41:4f:cc:b9:51:3c:5d:95:53:9d:42:cb:32:7a:f6:c2:
         42:e1:6f:98:16:4f:3f:6d:5f:00:9f:d9:a5:93:ed:66:b5:83:
         9f:71:96:b8:d9:cb:c1:d1:86:16:48:b4:ad:64:73:c2:0d:d6:
         ff:ba:90:1c:26:95:02:25:80:ef:04:2a:0b:9f:3b:62:6d:3d:
         ea:86:6e:83

Of interest a curl connection times out with no server hello received the etcd journalctl log is saying that the CA is unknown even when the ca.pem is passed through via system unit.

curl -v --noproxy '*' --cacert ca.pem --cert kubernetes.pem --key kubernetes-key.pem -L https://localhost:2379/v2/keys/foo -XPUT -d value=bar -v
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 2379 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: ca.pem
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* Operation timed out after 300513 milliseconds with 0 out of 0 bytes received
* stopped the pause stream!
* Closing connection 0
curl: (28) Operation timed out after 300513 milliseconds with 0 out of 0 bytes received
arcreigh picture arcreigh on 27 Oct 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

primeroz picture primeroz  路  3Comments
atinsood picture atinsood  路  4Comments
hnlq715 picture hnlq715  路  3Comments
zhousoft picture zhousoft  路  3Comments
cheyang picture cheyang  路  3Comments