etcd v3.1rc.1 etcdctl times out with API=3 but is OK with API=2

[michael-px]

I setup a single node with SSL/TLS using cfssl. The cluster came up and I set following environment variables:

ETCDCTL_CA_FILE=/etc/ssl/etcd/ca.pem
ETCDCTL_KEY_FILE=/etc/ssl/etcd/etcdv3ssl.key
ETCDCTL_API=3
ETCDCTL_CERT_FILE=/etc/ssl/etcd/etcdv3ssl.pem

if I type "etcdctl member list", it says:

"Error: context deadline exceeded" after about a 3s pause.

But if I type "ETCDCTL_API=2 etcdctl member list", it works:

ETCDCTL_API=2 etcdctl member list
34b2b20e19938f29: name=etcdv3 peerURLs=https://10.0.0.254:2380 clientURLs=https://52.43.85.31:2379 isLeader=true

journalctl says the cluster is up and running fine.

[gyuho]

Since you have ETCDCTL_API=3 as environment variable, etcdctl member list defaults to v3 member list command. And our current v3 etcdctl commands do not parse environment variable. So you might want to explicitly pass certs flags to etcdctl member list command.

[michael-px]

Well, that’s good to know.

I tried it and got a different error:

etcdctl --cacert=/etc/ssl/etcd/ca.pem --cert=/etc/ssl/etcd/etcdv3ssl.pem --key=/etc/ssl/etcd/etcdv3ssl.key member list
Error: grpc: timed out when dialing

Apparently, 127.0.0.1:2379 (which isn't enabled on this system) needs to be advertised. If I --endpoints=:2379, it works!

etcdctl --cacert=/etc/ssl/etcd/ca.pem --cert=/etc/ssl/etcd/etcdv3ssl.pem --key=/etc/ssl/etcd/etcdv3ssl.key --endpoints=xx.xx.xx.xx:2379 member list
34b2b20e19938f29, started, etcdv3, https://10.0.0.254:2380, https://xx.xx.xx.xx:2379

What's the ETA on using the ETCDCTL environment variables?

[gyuho]

@michael-px Sorry I was wrong.

It is supported, but with different flag strings.

See https://github.com/coreos/etcd/issues/6645.

Basically you need

ETCDCTL_CACERT=/tmp/ca.pem
ETCDCTL_CERT=/tmp/cert.pem
ETCDCTL_KEY=/tmp/key.pem

since we have changed the flag strings for v3.

[michael-px]

I defined these variables with the correct keys and I still get

Error: grpc: timed out when dialing

So, this doesn't seem to work.

[gyuho]

@michael-px Could you create a separate issue with etcd logs and a way to reproduce?
Works fine on our side.

[michael-px]

OK, should I title it something like

API=3 ignores environment variables or something like that?

—

Predicting is tough. Especially about the future.--Yogi Berra

We sometimes make too many wrong mistakes.--Yogi Berra

You can observe a lot by watching.--Yogi Berra

The future ain’t what it used to be.--Yogi Berra

Nobody goes there anymore. It's to crowded--Yogi Berra

On Thu, Dec 1, 2016 at 5:09 PM, Gyu-Ho Lee [e-mail] wrote:

@michael-px https://github.com/michael-px Could you create a separate
issue with etcd logs and a way to reproduce?
Works fine on our side.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/coreos/etcd/issues/6915#issuecomment-264344886, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ARbCQYyzPf3DrAsemnOQ6DgkqIwL1-0Zks5rD2-sgaJpZM4LA0aU
.

[gyuho]

I think it's a separate issue?

If that doesn't work, it could be TLS configuration issue.
Either way, we need etcd server logs with client errors.

Thanks.

[michael-px]

OK, I see the problem with TLS and without TLS, so it most likely is a
separate issue;

[whereisaaron]

Error: context deadline exceeded seems like the world's most unhelpful way to say TLS certificate authentication didn't work 🤕

[ivanyinusa]

I got the same issue. "Error: context deadline exceeded"