Rke: how to add windows worker to cluster?

[rgl]

RKE version: 0.3.0
Docker version: (docker version,docker info preferred) 19.03.2
Operating system and kernel: (cat /etc/os-release, uname -r preferred) Windows 2019 1809
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO) VM
cluster.yml file:

cluster_name: example
kubernetes_version: v1.16.1-rancher1-1
ssh_key_path: /vagrant/shared/ssh/rke_rsa
services:
  kube-api:
    service_cluster_ip_range: 10.53.0.0/16
  kube-controller:
    cluster_cidr: 10.52.0.0/16
    service_cluster_ip_range: 10.53.0.0/16
  kubelet:
    cluster_domain: cluster.domain
    cluster_dns_server: 10.53.0.10
network:
  plugin: flannel
  options:
    flannel_iface: eth1
    flannel_backend_type: host-gw
nodes:
  - hostname_override: rke1
    address: rke1.rancher.test
    internal_address: 10.1.0.5
    user: vagrant
    role:
      - controlplane
      - etcd
      - worker
  # windows worker node
  - hostname_override: rkew1
    address: rkew1.rancher.test
    internal_address: 10.1.0.40
    user: vagrant
    role:
      - worker

Question

I'm launching a vagrant environment with a rke managed kubernetes; the cluster has a single node controlplane (ubuntu) and I'm trying to add a windows worker node.

I'm adding the windows worker node with the normal rke up --config cluster.yaml. But it fails with:

level=warning msg="Failed to set up SSH tunneling for host [rkew1.rancher.test]: Can't retrieve Docker Info: error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed)"
level=warning msg="Removing host [rkew1.rancher.test] from node lists"

The full output is at the end of this comment.

How do I overcome this problem?

How do I customize flannel on windows? Like, I would like to set the flannel interface to be "Ethernet 3".

Vagrant environment

My vagrant environment is at https://github.com/rgl/rancher-ubuntu-vagrant/tree/feature-windows-worker-node and the part that tries do add the windows node is at https://github.com/rgl/rancher-ubuntu-vagrant/blob/feature-windows-worker-node/provision-rke.ps1.

The windows machine has docker and ssh installed.

Output

level=info msg="Running RKE version: v0.3.0"
level=info msg="Initiating Kubernetes cluster"
level=info msg="[certificates] Generating admin certificates and kubeconfig"
level=info msg="Successfully Deployed state file at [./cluster.rkestate]"
level=info msg="Building Kubernetes cluster"
level=info msg="[dialer] Setup tunnel for host [rkew1.rancher.test]"
level=info msg="[dialer] Setup tunnel for host [rke1.rancher.test]"
level=warning msg="Failed to set up SSH tunneling for host [rkew1.rancher.test]: Can't retrieve Docker Info: error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed)"
level=warning msg="Removing host [rkew1.rancher.test] from node lists"
level=info msg="[network] No hosts added existing cluster, skipping port check"
level=info msg="[certificates] Deploying kubernetes certificates to Cluster nodes"
level=info msg="Checking if container [cert-deployer] is running on host [rke1.rancher.test], try #1"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [cert-deployer] on host [rke1.rancher.test], try #1"
level=info msg="Checking if container [cert-deployer] is running on host [rke1.rancher.test], try #1"
level=info msg="Checking if container [cert-deployer] is running on host [rke1.rancher.test], try #1"
level=info msg="Removing container [cert-deployer] on host [rke1.rancher.test], try #1"
level=info msg="[reconcile] Rebuilding and updating local kube config"
level=info msg="Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yaml]"
level=info msg="[reconcile] host [rke1.rancher.test] is active master on the cluster"
level=info msg="[certificates] Successfully deployed kubernetes certificates to Cluster nodes"
level=info msg="[reconcile] Reconciling cluster state"
level=info msg="[reconcile] Check etcd hosts to be deleted"
level=info msg="[reconcile] Check etcd hosts to be added"
level=info msg="[reconcile] Rebuilding and updating local kube config"
level=info msg="Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yaml]"
level=info msg="[reconcile] host [rke1.rancher.test] is active master on the cluster"
level=info msg="[reconcile] Reconciled cluster state successfully"
level=info msg="Pre-pulling kubernetes images"
level=info msg="Checking if image [rancher/hyperkube:v1.16.1-rancher1] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/hyperkube:v1.16.1-rancher1] exists on host [rke1.rancher.test]"
level=info msg="Kubernetes images pulled successfully"
level=info msg="[etcd] Building up etcd plane.."
level=info msg="[etcd] Running rolling snapshot container [etcd-snapshot-once] on host [rke1.rancher.test]"
level=info msg="Removing container [etcd-rolling-snapshots] on host [rke1.rancher.test], try #1"
level=info msg="[remove/etcd-rolling-snapshots] Successfully removed container on host [rke1.rancher.test]"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [etcd-rolling-snapshots] on host [rke1.rancher.test], try #1"
level=info msg="[etcd] Successfully started [etcd-rolling-snapshots] container on host [rke1.rancher.test]"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-bundle-cert] on host [rke1.rancher.test], try #1"
level=info msg="[certificates] Successfully started [rke-bundle-cert] container on host [rke1.rancher.test]"
level=info msg="Waiting for [rke-bundle-cert] container to exit on host [rke1.rancher.test]"
level=info msg="Container [rke-bundle-cert] is still running on host [rke1.rancher.test]"
level=info msg="Waiting for [rke-bundle-cert] container to exit on host [rke1.rancher.test]"
level=info msg="[certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [rke1.rancher.test]"
level=info msg="Removing container [rke-bundle-cert] on host [rke1.rancher.test], try #1"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[etcd] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[etcd] Successfully started etcd plane.. Checking etcd cluster health"
level=info msg="[controlplane] Building up Controller Plane.."
level=info msg="Checking if container [service-sidekick] is running on host [rke1.rancher.test], try #1"
level=info msg="[sidekick] Sidekick container already created on host [rke1.rancher.test]"
level=info msg="[healthcheck] Start Healthcheck on service [kube-apiserver] on host [rke1.rancher.test]"
level=info msg="[healthcheck] service [kube-apiserver] on host [rke1.rancher.test] is healthy"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[healthcheck] Start Healthcheck on service [kube-controller-manager] on host [rke1.rancher.test]"
level=info msg="[healthcheck] service [kube-controller-manager] on host [rke1.rancher.test] is healthy"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[healthcheck] Start Healthcheck on service [kube-scheduler] on host [rke1.rancher.test]"
level=info msg="[healthcheck] service [kube-scheduler] on host [rke1.rancher.test] is healthy"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[controlplane] Successfully started Controller Plane.."
level=info msg="[authz] Creating rke-job-deployer ServiceAccount"
level=info msg="[authz] rke-job-deployer ServiceAccount created successfully"
level=info msg="[authz] Creating system:node ClusterRoleBinding"
level=info msg="[authz] system:node ClusterRoleBinding created successfully"
level=info msg="[authz] Creating kube-apiserver proxy ClusterRole and ClusterRoleBinding"
level=info msg="[authz] kube-apiserver proxy ClusterRole and ClusterRoleBinding created successfully"
level=info msg="Successfully Deployed state file at [./cluster.rkestate]"
level=info msg="[state] Saving full cluster state to Kubernetes"
level=info msg="[state] Successfully Saved full cluster state to Kubernetes ConfigMap: cluster-state"
level=info msg="[worker] Building up Worker Plane.."
level=info msg="Checking if container [service-sidekick] is running on host [rke1.rancher.test], try #1"
level=info msg="[sidekick] Sidekick container already created on host [rke1.rancher.test]"
level=info msg="[healthcheck] Start Healthcheck on service [kubelet] on host [rke1.rancher.test]"
level=info msg="[healthcheck] service [kubelet] on host [rke1.rancher.test] is healthy"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[worker] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[healthcheck] Start Healthcheck on service [kube-proxy] on host [rke1.rancher.test]"
level=info msg="[healthcheck] service [kube-proxy] on host [rke1.rancher.test] is healthy"
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[worker] Successfully started [rke-log-linker] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-linker] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-linker] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[worker] Successfully started Worker Plane.."
level=info msg="Checking if image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test], try #1"
level=info msg="Image [rancher/rke-tools:v0.1.50] exists on host [rke1.rancher.test]"
level=info msg="Starting container [rke-log-cleaner] on host [rke1.rancher.test], try #1"
level=info msg="[cleanup] Successfully started [rke-log-cleaner] container on host [rke1.rancher.test]"
level=info msg="Removing container [rke-log-cleaner] on host [rke1.rancher.test], try #1"
level=info msg="[remove/rke-log-cleaner] Successfully removed container on host [rke1.rancher.test]"
level=info msg="[sync] Syncing nodes Labels and Taints"
level=info msg="[sync] Successfully synced nodes Labels and Taints"
level=info msg="[network] Setting up network plugin: flannel"
level=info msg="[addons] Saving ConfigMap for addon rke-network-plugin to Kubernetes"
level=info msg="[addons] Successfully saved ConfigMap for addon rke-network-plugin to Kubernetes"
level=info msg="[addons] Executing deploy job rke-network-plugin"
level=info msg="[addons] Setting up coredns"
level=info msg="[addons] Saving ConfigMap for addon rke-coredns-addon to Kubernetes"
level=info msg="[addons] Successfully saved ConfigMap for addon rke-coredns-addon to Kubernetes"
level=info msg="[addons] Executing deploy job rke-coredns-addon"
level=info msg="[addons] CoreDNS deployed successfully.."
level=info msg="[dns] DNS provider coredns deployed successfully"
level=info msg="[addons] Setting up Metrics Server"
level=info msg="[addons] Saving ConfigMap for addon rke-metrics-addon to Kubernetes"
level=info msg="[addons] Successfully saved ConfigMap for addon rke-metrics-addon to Kubernetes"
level=info msg="[addons] Executing deploy job rke-metrics-addon"
level=info msg="[addons] Metrics Server deployed successfully"
level=info msg="[ingress] Setting up nginx ingress controller"
level=info msg="[addons] Saving ConfigMap for addon rke-ingress-controller to Kubernetes"
level=info msg="[addons] Successfully saved ConfigMap for addon rke-ingress-controller to Kubernetes"
level=info msg="[addons] Executing deploy job rke-ingress-controller"
level=info msg="[ingress] ingress controller nginx deployed successfully"
level=info msg="[addons] Setting up user addons"
level=info msg="[addons] no user addons defined"
level=fatal msg="Provisioning incomplete, host(s) [rkew1.rancher.test] skipped because they could not be contacted"

[deniseschannon]

Currently, windows is only supported in Rancher server platform and not with RKE CLI

[AMoghrabi]

Hi @deniseschannon. Your last message was a while ago and I'm trying to figure out if windows is supported through the RKE CLI. I'd appreciate your help, thanks.

[lippertmarkus]

Hi @deniseschannon. Any news on Windows Worker Support with RKE?