Calico: Implement route reflector clustering in Calico v3.0

[caseydavenport]

We lost this feature in the move to v3.0 since it would have required writing to etcdv3 directly, which isn't a supported API.

To add this back, we need to re-think the configuration model (perhaps adding something configurable with calicoctl).

There are other UX considerations to make as well as part of moving the RR component to GA.

For now, if you NEED route reflector clustering, you can continue to use Calico v2.6 or use Calico v3.0 and implement a RR cluster by hand using BIRD or quagga or most other BGP stacks.

Expected Behavior

calico/routereflector should support RR clustering in Calico v3.0.

Current Behavior

It does not.

Possible Solution

• Decide on a configuration model
• Implement it.

[caseydavenport]

I think the desirable configuration model here is that calico/node can double as a RR node if configured to do so, eliminating calico/routereflector altogether.

Then, you can either:

• Run a cluster where compute nodes can act as RRs, OR
• Run calico/node on dedicated, non-schedulable nodes as a standalone RR.

The main benefit is a unified configuration model via calicoctl (no more etcdctl).

I've put together a demo of this, which if we think works then I'd be happy to continue with. Note that currently there are no concrete plans to merge this, it's just been a hobby project of mine in order to figure out the right config model.

It adds a spec.BGP.routeReflectorClusterID field to the Node object which enables that node as a RR when peering with other non RR nodes.

This should be flexible enough to automate with a controller, and also do more advanced things like set up multiple clusters in various RR configurations.

PRs are here:

• https://github.com/projectcalico/node/pull/5
• https://github.com/projectcalico/confd/pull/116
• https://github.com/projectcalico/libcalico-go/pull/861

[caseydavenport]

PRs have gone into master to support this, will be released in Calico v3.3