K3s: windows support

[perrywang]

there are pretty much customers are using windows at edge side device with installation of labview.

[ibuildthecloud]

This one is tricky. I personally don't know much about Kubernetes on Windows, but Rancher/RKE works on windows so I can find out what needs to be done here. If there's demand, I'd love to do this.

[perrywang]

on windows in general it is using a vm hyperv or vbox to run kubenetes like minikube doing. But minikube is really not production ready and there is a lot of issue and no any customization capability.

[bencompton]

Looking at Microsoft's instructions for joining a Windows node to a cluster, Kubernetes releases have binaries for running Windows worker nodes (kubelet.exe, kube-proxy.exe), but not the control plane. I see there's also an official Windows binary for Flannel, which Microsoft's PS scripts leverage.

I would think a K3s Windows build with only the agent components would be possible. At this point, it looks like Containerd support in Windows is still only in alpha and the official Windows build is still a WIP. Not sure what other dependencies in K3s would be problematic for an agent-only Windows build.

[mortezaalizadeh]

@ibuildthecloud

Hey guys, I just checked the Containerd support for Windows and sounds like it's no longer in alpha state. I am using K3S two of my side projects and supporting Windows on the edge is one of my highest priorities in both projects.

We also don't really need to run the control plane on the edge and Agent support on the Window should be more than enough. I do not know enough about the implementation details of the K3S and K8S, but I am more than happy to help to get the Windows support done.

Is there any progress on this ticket since the last comment posted on 11 April?

[KnicKnic]

my fork can start and join a windows master node. It does appear to have done a few things right, example my kubelet lists container images on my node (I am using docker desktop community). However I don't have networking setup, so the node stays unschedulable.

I would love for someone with more understanding of network configuration to take it further. And give instructions on how to setup networking. @ibuildthecloud do you know anyone that could help here?

cmd I used to get it to "somewhat" startup

.\k3s.exe server -d c:\tmp\k3s --flannel-backend none --docker  --kube-proxy-arg "proxy-mode=userspace" --disable-network-policy

Branch I was using for this experiment. https://github.com/KnicKnic/k3s/tree/k3s_run_windows_node

There are few errors (these related to cni / networking)

W0223 00:59:39.172343   20244 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
E0223 00:59:39.937342   20244 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

And this random one for TLS, have no clue why a cert would be wrong.

2020-02-23 00:54:37.591274 I | http: TLS handshake error from 127.0.0.1:2111: remote error: tls: bad certificate
2020-02-23 00:54:37.602363 I | http: TLS handshake error from 127.0.0.1:2114: remote error: tls: bad certificate

[KnicKnic]

@thxCode @alena1108 @cloudnautique @aiwantaozi pinging some other people in rancher that seemed to have worked on windows integration (for rancher) to see if they could help with setting up windows networking. See my previous comment https://github.com/rancher/k3s/issues/114#issuecomment-590047668 .

[teyc]

@KnicKnic I just bookmarked these:

https://github.com/microsoft/SDN/blob/master/Kubernetes/flannel/register-svc.ps1 a bunch of script for establishing a SDN on Windows.

In particular https://github.com/microsoft/SDN/search?q=cniconfig&unscoped_q=cniconfig Powershell scripts that generate cni configuration.

This is old How To Guide: On-premise Kubernetes for Windows using Flannel (Host-Gateway)

Troubleshooting Kubernetes Networking on Windows Part 1

Also https://github.com/microsoft/SDN/issues/170#issuecomment-548574978

[jiridanek]

Even though I can't run k3s agent on Windows right now, it should be possible to connect a k8s node on Windows to a k3s control node on Linux, yes?

[brandond]

@jiridanek vanilla k8s kubelets cannot connect to k3s servers, as they are missing the k3s-specific bootstrap code.