Rke: Host bindmount for /usr/libexec/kubernetes/kubelet-plugins fails on CoreOS

[ericlanderson]

Container Linux by CoreOS has a read-only /usr filesystem. This causes the bind mount for /usr/libexec/kubernetes/kubelet-plugins to fail with the following error:

FATA[0111] [workerPlane] Failed to bring up Worker Plane: Failed to start [kubelet] container on host [IP Redacted]: Error response from daemon: error while creating mount source path '/usr/libexec/kubernetes/kubelet-plugins': mkdir /usr/libexec/kubernetes: read-only file system
Is there the possibility for a cluster config option to support an alternate host path to support Flexvolume plugins?

Similar to https://github.com/kubernetes/kubeadm/issues/476.

RKE version:

v0.0.12-dev

Docker version: (docker version,docker info preferred)

1.12 and 17.09.0-ce

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

OS: Container Linux by CoreOS stable (1576.5.0)
Kernel: 4.14.11-coreos

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)

Bare-metal

Steps to Reproduce:

rke up

Results:
FATA[0111] [workerPlane] Failed to bring up Worker Plane: Failed to start [kubelet] container on host [IP Redacted]: Error response from daemon: error while creating mount source path '/usr/libexec/kubernetes/kubelet-plugins': mkdir /usr/libexec/kubernetes: read-only file system

[ericlanderson]

Ideally, this could be a kubelet property specifying the host path to use for the plugins bind mount where /usr/libexec/kubernetes/kubelet-plugins is the default if none is provided:

kubelet: image: rancher/k8s:v1.8.3-rancher2 cluster_domain: cluster.local cluster_dns_server: 10.233.0.3 infra_container_image: gcr.io/google_containers/pause-amd64:3.0 host_plugins_path: /opt/kubernetes/kubelet-plugins

[rukor]

Please has this been released yet? I am unable to use this with coreos at the moment

[moelsayed]

@rukor This should be in v0.1.1-rc2. Please let us know how it goes.

[ericlanderson]

I have tested this on Container Linux 1576.4.0 and 1632.2.1 and this does indeed work fine.

[rukor]

Thank you. I’ll give it a spin

---

Roland Ukor

On 7 Feb 2018, at 1:49 am, ericlanderson [e-mail] wrote:

I have tested this on Container Linux 1576.4.0 and 1632.2.1 and this does indeed work fine.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[austinnichols101]

Confirming working on 1632.2.1. Because 1632.2.1 uses docker 17.09.1-ce, I had build a new rke binary so that I can override the supported docker version (ref: https://github.com/rancher/rke/issues/168)

• clone the RKE repo and make against the master branch
• add ignore_docker_version: true to the top of my cluster.yml file.
• follow the normal documented procedure using the new rke binary

Rancher 2.0 + RKE all running on this single host. Nice work all!

[moelsayed]

Tested on latest master on CoreOS 1632.2.1with ignore_docker_version: true. RKE completed successfully with a healthy cluster.