Etcd: ClusterID is not unique for different etcd clusters

[nikkictl]

Documentation states that the ClusterID _should_ be different for different etcd clusters.

// ClusterID holds the cluster-level ID reported by the server.  This
// should be different for different etcd clusters.
ClusterID string `json:"-"`

This is not the case, especially for embedded etcd clusters where we have found that the ClusterID is from from unique. https://github.com/etcd-io/etcd/pull/1286 states that the ClusterID is generated by hash(sorted(MemberIDs), discoveryURL). Confirmed by that logic and from testing multiple embedded etcd clusters, a single node embedded etcd instance with a discoveryURL of the localhost will yield a ClusterID of 3b0efc7b379f89be across multiple different clusters.

This lack of uniqueness has been problematic for our use of external and embedded etcd clusters. It would be beneficial to have a _unique_ ClusterID. Additionally, documentation should be updated if that is not the case.

[palourde]

For reference, the comment on the NewMember function mentions that time is used to generate the member IDs (which are used later to generate a cluster ID):

https://github.com/etcd-io/etcd/blob/d6280f9ea54849e5364545ca34bdac0a58317569/etcdserver/api/membership/member.go#L54-L55

That being said, it looks like the time passed when generating member IDs from the URLs map is always nil:

https://github.com/etcd-io/etcd/blob/d6280f9ea54849e5364545ca34bdac0a58317569/etcdserver/api/membership/cluster.go#L78

Therefore, even if you wipe etcd's data-dir, the member ID & cluster ID will always remain the same if you pass the same configuration.

[xiang90]

@palourde thanks for answering this question!

[nikkictl]

Hi @xiang90 I'm wondering why this issue was closed? @palourde comment just reiterates my point about the ClusterID _not_ being unique. How could we get the functionality or documentation addressed?

[xiang90]

@nikkiki yes. that is a id, not unique id. you need to generate a unique token and supply through initial-cluster-token flag.

[nikkictl]

@xiang90 the documentation states that the ClusterID "should be different for different etcd clusters" (see description above). If that is only the case when a unique initial-cluster-token is supplied, then shouldn't the documentation be updated to reflect that?

[jingyih]

@nikkiki In your use case, can you try to use unique "--initial-cluster-token" for each cluster? When bootstrapping a new cluster, cluster ID is determined (indirectly) by peer URLs, member names, initial cluster token.

[jingyih]

That being said, it looks like the time passed when generating member IDs from the URLs map is always nil:

Just for the reference. This is correct only during bootstrapping. If adding member to an existing cluster, member ID's generation will take timestamp.

[jingyih]

The linked document is v2. The officially supported etcd versions are 3.1, 3.2 and 3.3. Please refer to v3 doc: https://godoc.org/github.com/etcd-io/etcd/clientv3

[nikkictl]

@jingyih I understand that using a unique initial-cluster-token for each cluster would also assign a unique ClusterID across those similarly configured clusters. However, that value is only used when bootstrapping a new cluster:

The command line parameters starting with --initial-cluster will be ignored on subsequent runs of etcd.

Therefore, there is no way to rectify ClusterID uniqueness on _existing_ clusters.

The only change I am advocating for is to make the documentation a little more clear on this. The following description of a ClusterID makes no mention of initial-cluster-token, yet still claims the ID _should_ be unique for different clusters.

// ClusterID holds the cluster-level ID reported by the server.  This
// should be different for different etcd clusters.
ClusterID string `json:"-"`

[jingyih]

Thanks for your suggestion. We probably won't update v2 doc, as it is deprecated. If anyone found similar issue in v3 doc, we should update it.

As for rectify cluster ID uniqueness, if downtime is acceptable, you can create new clusters with existing data file.