Rke: Allow more options for load balancing controlplane nodes externally and internally

[superseb]

Previous issues/discussion in https://github.com/rancher/rke/issues/705 and https://github.com/rancher/rke/pull/1348

Options to implement:

• Load balancing to controlplane nodes externally
• Load balancing to controlplane nodes internally (from cluster nodes to kube-apiserver on controlplane nodes), this will disable nginx-proxy and use the address specified to connect to load balanced controlplane nodes.
• Change kube-apiserver listen port

Affected logic/config:

• Additional names in certificates so it accepts request using that name (authentication.sans)
• Generated kubeconfig for cluster so it uses the load balanced address and not a single controlplane node
• Generated kubecfg files on nodes pointing to controlplane nodes/nginx-proxy
• Connectivity check to kube-api so it uses the load balances address and not a single controlplane node

loadbalancer:
  // External name to be used in generated kubeconfig and in certificates for kube-apiserver
  kubeapi_external_fqdn: kubeapi.yourdomain.com
  // Optional, listen port for kubeapi_external_fqdn which is configured at the external load balancer
  kubeapi_external_port: 8443
  // Optional, disables nginx-proxy on the nodes and uses this in the node's kubecfg
  kubeapi_internal_fqdn: kubeapi-internal.yourdomain.com
  // Optional, uses this port to connect to kubeapi
  kubeapi_internal_port: 9443

While we are at it, we might as well make kube-apiserver port configurable so everything can be adjusted network wise. This is currently hardcoded to 6443. For this we can either use --secure-port or add another key under kube-api (listen_port). Cons of the first are tying it a specific parameter, cons of the other is adding another key. We probably need the new key, which we can then also use for the kubecfg/nginx-proxy when we dont use any load balancing.

services:
  kube-api:
    listen_port: 7443

When this design is accepted, I'll put in the steps needed to make this work so it can be worked on externally or internally.

[mchucklee]

Looking forward to this feature, tks a lot

[markrevill]

Hi guys,

I am currently building out an RKE cluster (to host Rancher) on Azure VMs behind an external Load balancer and running the client via a bastion host.

Would love to know when if/when this feature request is likely to be merged as it would add a lot of flexibility to the outputted kube_config_cluster.yml

Cheers,
Mark.

[fredleger]

Any chance to see this implemented soon ? Look like it's a must have when building an rke cluster in private networks

[nightmareze1]

+1

[stale[bot]]

This issue/PR has been automatically marked as stale because it has not had activity (commit/comment/label) for 60 days. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[immanuelfodor]

Unstale

[aledegano]

Thank you for the design!
This is exactly what we need, I hope this discussion can move on.

[carloscarnero]

This feature would be really nice, especially for on-premises folks. I have managed to get load balancing working by leveraging Nginx in TCP mode and the SANS feature; however, everytime the kube config file changes (via an rke operation) I have to go and modify the API server address.

[immanuelfodor]

Same here. I'm using Kube Karp for this purpose (https://github.com/immanuelfodor/kube-karp) and as I'm version tracking the RKE and kube config files, I can git add -p the changes after RKE up, do not select the modified API server URL hunk, then git checkout -- kubeconfig to restore the original version. It would be great to get rid of the longer workaround, and just git add everything :)