Etcd: Support JWT in etcdctl

[paulcaskey]

We need some kind of basic implementation of JWT in etcdctl, even if it is somewhat manual to start. For instance, the user could generate a token with 'curl' and then make that accessible to etcdctl through an environment variable.

Right now, once auth is enabled with JWT on the server, etcdctl becomes useless.

[paulcaskey]

... unless that manual method already works, and I just don't know how to use it!

[hexfusion]

... unless that manual method already works, and I just don't know how to use it!

Have you tried it? This should work fine please let me know if you run into any issues.

• Setup JWT tokens on the server side [1]
• Enable auth [2]
• Use etcdctl to authenticate [3]

[1] https://etcd.io/docs/v3.3.12/op-guide/configuration/#auth-token
[2] https://etcd.io/docs/v3.3.12/op-guide/authentication/#enabling-authentication
[3] https://etcd.io/docs/v3.3.12/op-guide/authentication/#using-etcdctl-to-authenticate

[paulcaskey]

Thanks so much @hexfusion . I am so happy this is easy and I'm just over-thinking it. Yes I did try it, and I've already done [1] and [2]. Please hand me just one more small flashlight, and I should have a path out of this darkness. :)

$ export ETCDCTL_API=3
$ etcdctl role list
$ etcdctl user list
$ etcdctl user add root
Password of root:
Type password of root again for confirmation:
User root created
$ etcdctl user list
root
$ etcdctl auth enable
Authentication Enabled
$ export ETCDCTL_USER=root:xyzzy
$ etcdctl user list
Error: etcdserver: invalid auth token

Am I supposed to manually get a token via 'curl' and paste that instead of .... the password?? Don't I also need something for etcdctl to know what key/cert I used with etcd --auth-token jwt,pub-key=.....?

$ export ETCDCTL_USER=root:eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZXZpc2lvbiI6NCwidXNlcm5hbWUiOiJyb290In0.QUWm4kTknGAQriLWMRgvnK-p8OJ3eX022lTnFAbVZJWSCgvSQN1THW1nZlTun8fII3w5B6hUuHODyKZigsiN9uxIGcGQo59YB8BKDBClZv5fCXPGdf8BGzD1kQMDGd4gBxGtd6gB63qAf78RAtgxxxzAoiPqByTK2D1ARTpsUXLVdCkR8SlnH_a9ZZp0hQSQNHWmIu1MOiqtOPMZxIrjG7_JBMb5Use-2ijtQFopVy31pWXbRQHF0INV58qMZ4dAD9mrZfXMYBDIoj0d0qKY1qqMOpXVKww77Yq50bL6vu4y12bs6jT2QA0vz5UGYKsrQCcrLNCcI3zTCM5wbv_s2Q
$ etcdctl user list
Error: etcdserver: authentication failed, invalid user ID or password
$

I really feel dense. I'm clearly missing something obvious. Help??

[paulcaskey]

@hexfusion note that if you leave ETCDCTL_API=2 then of course it works, because it's ignoring JWT since etcd v3 API v2 knows nothing about JWT and _I think_ etcd will just ignore --auth-token jwt in that case. I need API 3 auth to work, with JWT, with etcdctl. That's the task at hand. Thank you and please reply if that clarifies! Especially if I'm still wrong! I'm dying to sort this out. The examples you pointed showed me in your 3 URLs make zero mention of JWT so I'm just still completely stumped.

Thanks,
Paul

[paulcaskey]

Fixed!! The problem is that I have a whole cluster of 3 nodes defined in my ECTDCTL_ENDPOINTS, not just one host. That works great for everything else I have done. But each cluster node assigns a different JWT token and etcdctl is doing something random like getting the token from one node then sending the actual GET request to another node. If I force etcdctl to only use ANY ONE of the 3 nodes, it works.

Now I just have to consider this in my deployment, so that all clients only ever try to talk to a single server, not with some list of available servers. I'll verify this in my client API too. Now I know what to look for.