Consul: Document a procedure for renaming DCs

[slackpad]

Carry over from here - https://groups.google.com/d/msgid/consul-tool/b80a40ef-6b02-47d2-9bee-3014135ade22%40googlegroups.com?utm_medium=email&utm_source=footer. We should document some steps to do this.

[jeffbyrnes]

Is there any way to change the datacenter name & _not_ lose what’s stored in Consul’s K/V store? Specifically, I have Vault using Consul as its backing store, and I need to change the datacenter name.

[danlsgiga]

Same here... tried changing the datacenter name and completely lost access to the cluster. I have couple ACLs and even the master token was not working at all. Had to revert the name back to regain access.

[gvenka008c]

@slackpad @here I got into the same issue. Renamed the datacenter name and restarted the service. But lost access to the cluster. What would be the best steps we can follow to rename the datacenter name without affecting the running cluster?

[c4milo]

@slackpad, any update on this issue? Is https://www.consul.io/docs/guides/autopilot.html#upgrade-migrations close to what we will have to do in order to not lose the whole cluster?

[guidoiaquinti]

What about providing an abstraction layer that can be used to rename the datacenter name at runtime? I might be oversimplifying but to change the datacenter name from A to B without downtime we could do something like:

1) execute consul datacenter --alias A B
2) change is propagated to the servers in the RAFT ring and the command is returned. From now on, all catalog/KV queries with dc=A are transparently renamed on the fly to dc=B
3) modify the agent configuration on all the clients to register to cluster B + service restart. All the clients are now registered using the new name
4) modify the agent configuration on all the servers to register as cluster B. All the nodes in the cluster are now registered using the new name
5) we verify that there aren't anymore queries to dc=A
6) we drop the alias

[joewilliams]

@guidoiaquinti's procedure looks awesome and would work great for our use case as well.

[i0rek]

This is nothing that we have time to actively work on. There is however a similar mechanism to switch domains with alt-domain, just throwing it out there. If you want to provide a PR I would be happy to look at it and give feedback.