Kind: Errors running kind on windows 7 using docker toolbox

[aojea]

I was trying to use kind on Windows 7 with Docker-toolbox and I've found the following issues that I think makes this combination hard or very difficult to support:

1. Docker is not mapping ports to localhost, so you have to locate your docker VM IP to access the port forwarded

$ docker-machine ip
192.168.99.100

1. kind uses privileged containers and remount directories, this screws up the docker-toolbox VM and you can't use docker commands that fails with the following error

docker: Error response from daemon: cgroups: cannot found cgroup mount destination: unknown.

A workaround is resetting the VirtualBox VM

1. kind fails to create the kubedm config, seems that's related to the way that go handles windows directories,

Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.13.3) 🖼
 ✓ Preparing nodes 📦
 ✗ Creating kubeadm config 📜
Error: failed to create cluster: failed to generate kubeadm config content: got file '\', but 'C:\\' must be a directory to be a root

however, is easy to fix with the following patch

1. Once fixed 3 kind fails with the following error .... :

Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.13.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Creating kubeadm config 📜
 ✗ Starting control-plane 🕹️
Error: failed to create cluster: failed to init node with kubeadm: exit status 1

and then I gave up :)

[neolit123]

however, is easy to fix with the following patch

this seems like a viable fix to me. thank you.
the rest looks like things we need to document.

/kind documentation
/priority important-longterm

[BenTheElder]

So FWIW I use docker desktop and haven't had issues. It's been a while since I ran on windows which might explain the patching regression.

[aojea]

It seems that different docker implementations on the OSs are another factor to take into account, docker desktop vs docker toolbox is totally different.

I've run kind in my mac with docker desktop and worked (not tested multinode or special configs) , although I'm not confident that's going to work with docker toolbox in Mac.

I think that having a supportability table like this in the docs will help the users

| Operating System | Docker Desktop | Docker Toolbox | Docker |
| ------------- | ------------- | ------------- | ------------- |
| Windows | OK* | X | X |
| Mac | OK* | X | X |
| Linux | X | X | OK |

[BenTheElder]

perhaps slightly more clear:

| Operating System | Docker Desktop | Docker Toolbox | Docker-CE |
| ------------- | ------------- | ------------- | ------------- |
| Windows | ✔️ | ? | N/A |
| Mac | ✔️ | ? | N/A |
| Linux | N/A | N/A | ✔️ |

[BenTheElder]

I'm not a fan of how nearly 50% of that table is non-existent combinations though...

[aojea]

yep, the table doesn't add too much value, it's more related to Docker OS compatibility

[aojea]

Docker toolbox is considered legacy,
let's close it and reopen if necessary, I hit this because it was the only laptop I have with windows

[reza]

kind defaults to 127.0.0.1 for host API server address and a random ephemeral port, so you receive error if 'docker machine' is on a different address or the port is not forwarded properly:

~$ kubectl cluster-info --context kind-kind

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server 127.0.0.1:57099 was refused - did you specify the right host or port?

To make kind compatible with older machines that use docker toolbox and run docker-machine on 192.168.99.100 you need to override default server settings, create a kind config file and use network wide cluster settings as below:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: 192.168.99.100

Another workaround is to make kind use a fixed port number and use port forwarding on your host machine or hypervisor (e.g. VirtualBox) to your docker-machine.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerPort: 6443

In the example config below I have used both settings:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: 192.168.99.100
  apiServerPort: 6443

~$ docker-machine  ip
192.168.99.100
~$ vi kind-config.yml 
~$ kind create cluster --config=kind-config.yml
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.16.3) 🖼 
 ✓ Preparing nodes 📦 
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
~$ docker ps 
CONTAINER ID        IMAGE                                                                                          COMMAND                  CREATED             STATUS              PORTS                           NAMES
749cda06cbcb        kindest/node:v1.16.3@sha256:70ce6ce09bee5c34ab14aec2b84d6edb260473a60638b1b095470a3a0f95ebec   "/usr/local/bin/en..."   17 minutes ago      Up 2 minutes        192.168.99.100:6443->6443/tcp   kind-control-plane
~$ kubectl cluster-info --context kind-kind
Kubernetes master is running at https://192.168.99.100:6443
KubeDNS is running at https://192.168.99.100:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
~$