Rke: Check the required open ports between nodes at the beginning of RKE deployment

[galal-hussein]

Currently the following ports should be open for rke deployment to be fully successful:

• from rke host to other nodes: we only need the ssh port 22 and 6443 to communicate for control plane nodes
• from k8s hosts to other k8s hosts: 2379,2380 (etcd), 10251 (Scheduler), 10252 (Controller), 10250 (kubelet), 10256 (kubeproxy), 6443 (kubeapi), 8472 (flannel vxlan)

rke should check for these ports before starting deployment.

[guangbochen]

@galal-hussein I think we should have separate port range check for node(s) with different role(s).

etcd, controlplane Node(s):

Protocol | Direction | Port Range | Purpose
-- | -- | -- | --
TCP | Inbound | 22 | ssh server
TCP | Inbound | 6443* | kubernetes API server
TCP | Inbound | 2379-2380 | etcd server client API
TCP | Inbound | 10250 | kubelet API
TCP | Inbound | 10251 | scheduler
TCP | Inbound | 10252 | controller
TCP | Inbound | 10256 | kubeproxy
TCP | Inbound | 8472 | flannel vxlan

Worker Node(s):

Protocol | Direction | Port Range | Purpose
-- | -- | -- | --
TCP | Inbound | 22 | ssh server
TCP | Inbound | 10250 | Kubelet API
TCP | Inbound | 10256 | kubeproxy
TCP | Inbound | 30000-32767 | NodePort Services**
TCP | Inbound | 8472 | flannel vxlan

BTW, do we have Read-only Kubelet API enabled?

Protocol | Direction | Port Range | Purpose
-- | -- | -- | --
TCP | Inbound | 10255 | Read-only Kubelet API

K8s Reference: https://kubernetes.io/docs/setup/independent/install-kubeadm/#check-required-ports

[moelsayed]

238 adds port checking for Kubernetes services and etcd, network plugins need a different set of ports and will be added in another PR.

[galal-hussein]

rke version v0.1.1-rc1

I was able to verify that port checking between hosts works okay:

INFO[0034] [network] Deploying port listener containers 
INFO[0035] [network] Successfully started [rke-cp-port-listener] container on host [35.177.75.136] 
INFO[0035] [network] Successfully started [rke-cp-port-listener] container on host [35.177.241.114] 
INFO[0035] [network] Successfully started [rke-worker-port-listener] container on host [35.177.75.136] 
INFO[0035] [network] Successfully started [rke-worker-port-listener] container on host [35.177.241.114] 
INFO[0035] [network] Port listener containers deployed successfully 
INFO[0035] [network] Running etcd <-> etcd port checks  
INFO[0036] [network] Successfully started [rke-port-checker] container on host [35.177.99.152] 
INFO[0036] [network] Successfully started [rke-port-checker] container on host [35.177.81.181] 
INFO[0037] [network] Running all -> etcd port checks    
INFO[0038] [network] Successfully started [rke-port-checker] container on host [35.177.241.114] 
INFO[0038] [network] Successfully started [rke-port-checker] container on host [35.177.75.136] 
INFO[0039] [network] Successfully started [rke-port-checker] container on host [35.177.241.114] 
INFO[0039] [network] Successfully started [rke-port-checker] container on host [35.177.75.136] 
INFO[0040] [network] Running control plane -> etcd port checks 
INFO[0040] [network] Successfully started [rke-port-checker] container on host [35.177.75.136] 
INFO[0040] [network] Successfully started [rke-port-checker] container on host [35.177.241.114] 
INFO[0040] [network] Running workers -> control plane port checks 
INFO[0041] [network] Successfully started [rke-port-checker] container on host [35.177.75.136] 
INFO[0041] [network] Successfully started [rke-port-checker] container on host [35.177.241.114] 
INFO[0041] [network] Checking KubeAPI port Control Plane hosts 
INFO[0041] [network] Removing port listener containers  
INFO[0042] [remove/rke-etcd-port-listener] Successfully removed container on host [35.177.99.152] 
INFO[0042] [remove/rke-etcd-port-listener] Successfully removed container on host [35.177.81.181] 
INFO[0042] [remove/rke-cp-port-listener] Successfully removed container on host [35.177.75.136] 
INFO[0042] [remove/rke-cp-port-listener] Successfully removed container on host [35.177.241.114] 
INFO[0042] [remove/rke-worker-port-listener] Successfully removed container on host [35.177.75.136] 
INFO[0042] [remove/rke-worker-port-listener] Successfully removed container on host [35.177.241.114] 
INFO[0042] [network] Port listener containers removed successfully 

[ahoehma]

Hello guys ... regarding to the needed ports etc. I try to install a cluster via rke in aws and I followed the old tutorial at https://rancher.com/blog/2018/2018-05-14-rke-on-aws/ ... and out of the box I got an error like this

FATA[0017] [[network] Host [xxxxx.eu-west-1.compute.amazonaws.com] is not able to connect to the fo
llowing ports: [yyyyyy.eu-west-1.compute.amazonaws.com:10250, wwwww.eu-west-1.compute.ama
zonaws.com:10250, zzzzz.eu-west-1.compute.amazonaws.com:10250]. Please check network policies and f
irewall rules]

What can I do? Is there a simple howto or whatever to define all these required ports (in aws security-group?) between the nodes?

kind regards :)
andreas