Patroni: Heads-up: Etcd3 will require significant changes

[jberkus]

Etcd3 will require significant changes for compatibility:

1. protobufs and base-64 encoded keys mean that we can't use the same keying;
2. elimination of tree structures for a flat keyspace means that we have to change how we name the keys

Possibly python drivers will handle this for us, but maybe not.

https://coreos.com/blog/etcd3-a-new-etcd.html

[drnic]

Do you suppose that upgrading to etcd3 might be a patroni 2.0 level change?

On Thu, Jul 7, 2016 at 5:49 AM +1000, "Josh Berkus" [email protected] wrote:

Etcd3 will require significant changes for compatibility:

protobufs and base-64 encoded keys mean that we can't use the same keying;
elimination of tree structures for a flat keyspace means that we have to change how we name the keys

Possibly python drivers will handle this for us, but maybe not.

https://coreos.com/blog/etcd3-a-new-etcd.html

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[jberkus]

Well, it doesn't affect folks who are using Zookeeper or Consul, and we don't intend to change _how_ we use Etcd, just the protocol. So no.

[CyberDem0n]

Etcd3 will require significant changes for compatibility:
1) protobufs and base-64 encoded keys mean that we can't use the same keying;

Actually it's not that bad, I've already played around with grpcio and grpcio-tools and even got some working python code which is able to fetch key-values from etcdv3 using protobuf.

2) elimination of tree structures for a flat keyspace means that we have to change how we name the keys

It is not a problem at all, because it is still possible to get entire subtree (which is not a subtree anymore) with using single call:

ETCDCTL_API=3 etcdctl --endpoints=127.0.0.1:2379 get --prefix /service/

It is just utilizing Range method with the key="/service/" and range_end="/service0" arguments.

Possibly python drivers will handle this for us, but maybe not.

Yeah, but unfortunately there is nothing available at the moment... The only what we could do right now - generate client interfaces (python code) from the .proto service definition files and include these python files into Patroni package, though I don't really like such approach.

I don't think that implementing v3 support is really urgent task, because support of /v2 protocol in Etcd3 is still there and it should not be removed in foreseeable future.

[drnic]

Ahh good point re /v2 endpoint continuing. I was starry eyed/concerned about advertised /v3 that I forgot all our calls go to /v2.

[davissp14]

https://coreos.com/blog/toward-etcd-v3-in-container-linux.html

etcd v2 scheduled for removal from Container Linux in February, 2018

Python driver with v3 support: https://github.com/kragniz/python-etcd3

With the official EOL date set, seems like the move to etcdv3 should be made sooner than later.

[CyberDem0n]

I think this statement is about etcd binaries, not about protocol.
etcd-v3 binaries are still offering /v2 endpoint, so nothing will break.

P.S. I would be happy to implement v3 support in Patroni, but the only python module supporting v3 is still pretty raw...

[davissp14]

I think this statement is about etcd binaries, not about protocol. etcd-v3 binaries are still offering /v2 endpoint, so nothing will break.

This is true, even though I think that was a questionable decision on their part. Running two very different data-stores under a single binary is nothing short of awkward, IMO. There's also no way to spin up Etcd without the v3 data-store enabled, however you are able to disable the v2 data-store, thankfully. The v2 API / data-store is also riddled with issues and I think the only reason they are keeping it around in v3 is because gRPC is still very new to people and driver support isn't where it needs to be.

P.S. I would be happy to implement v3 support in Patroni, but the only python module supporting v3 is still pretty raw...

What kind of work needs to be done with the python module in order to get it where it needs to be?

[CyberDem0n]

What kind of work needs to be done with the python module in order to get it where it needs to be?

Mostly it would be covered by this PR: https://github.com/kragniz/python-etcd3/pull/106

But there are a lot of other stuff missing, just go through list of issues: https://github.com/kragniz/python-etcd3/issues

Different timeouts also needs to be tested carefully (connect, read, watch, etc...), because they are very important.

I've even spend some time on implementing some features for python-etcd3 last year, but since than didn't had time for further work.

[jberkus]

Due to our switch to using the Kubernetes API, this issue is no longer relevant.

[djmaze]

@jberkus Does that mean non-Kubernetes deployments will not be supported in the future?

[alexeyklyukin]

@djmaze we have no plans to abandon any of the currently supported DCSs, be it Etcd, Consul, Zookeeper or Kubernetes.

[djmaze]

Thanks for clarifying.

[jberkus]

If someone has specific breakage with etcd3, they should report that as a technical issue.

[bradnicholson]

I run it with v3 over the v2 API and have had no problems with it.

[bradnicholson]

Mentioned on slack, but will raise this here as well. etc v3.4 is going to set the v2 API off by default:

https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_4.md#make-etcd---enable-v2false-default

At best, we should probably document this as Patroni will not work with the etcd 3.4 defaults.

I suspect this is the first step with Etcd deprecating the v2 API though.