Minikube: none: minikube should be able to be run by a non-root user (use sudo when necessary)

[sourabh1031]

If this is a bug report, please include:

I tried starting minikube with the following command
minikube start --vm-driver=none

after downloading minikube v0.34.1 from this command
curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.34.1/minikube-linux-amd64 && chmod +x minikube && sudo cp minikube /usr/local/bin/ && rm minikube

I got the output that minikube crashed

I am on ubuntu 18.04LTS machine

[hpandeycodeit]

@sourabh1031 can you provide the full output/error that you got when it crashed?

[sourabh1031]

minikube start --vm-driver=none
😄 minikube v0.34.1 on linux (amd64)
🤹 Configuring local host environment ...

⚠️ The 'none' driver provides limited isolation and may reduce system security and reliability.
⚠️ For more information, see:
👉 https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md

💡 Tip: Use 'minikube start -p ' to create a new cluster, or 'minikube delete' to delete this one.
🔄 Restarting existing none VM for "minikube" ...
⌛ Waiting for SSH access ...
📶 "minikube" IP address is 192.168.43.56
🐳 Configuring Docker as the container runtime ...
[sudo] password for sourabh:
✨ Preparing Kubernetes environment ...
💣 Failed to update cluster: downloading binaries: copy: error removing file /usr/bin/kubelet: remove /usr/bin/kubelet: permission denied

😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉 https://github.com/kubernetes/minikube/issues/new

[sourabh1031]

when i ran command using sudo i got this
[sudo] password for sourabh:
😄 minikube v0.34.1 on linux (amd64)
🤹 Configuring local host environment ...

⚠️ The 'none' driver provides limited isolation and may reduce system security and reliability.
⚠️ For more information, see:
👉 https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md

⚠️ kubectl and minikube configuration will be stored in /home/sourabh
⚠️ To use kubectl or minikube commands as your own user, you may
⚠️ need to relocate them. For example, to overwrite your own settings:

▪ sudo mv /home/sourabh/.kube /home/sourabh/.minikube $HOME
▪ sudo chown -R $USER /home/sourabh/.kube /home/sourabh/.minikube

💡 This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
💡 Tip: Use 'minikube start -p ' to create a new cluster, or 'minikube delete' to delete this one.
🔄 Restarting existing none VM for "minikube" ...
⌛ Waiting for SSH access ...
📶 "minikube" IP address is 192.168.43.56
🐳 Configuring Docker as the container runtime ...
✨ Preparing Kubernetes environment ...
🚜 Pulling images required by Kubernetes v1.13.3 ...
🔄 Relaunching Kubernetes v1.13.3 using kubeadm ...
💣 Error restarting cluster: running cmd: sudo kubeadm init phase certs all --config /var/lib/kubeadm.yaml: running command: sudo kubeadm init phase certs all --config /var/lib/kubeadm.yaml: exit status 1

😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉 https://github.com/kubernetes/minikube/issues/new

[hpandeycodeit]

@sourabh1031 can you try by setting CHANGE_MINIKUBE_NONE_USER=true or
```
sudo mv /home/sourabh/.kube /home/sourabh/.minikube $HOME

sudo chown -R $USER /home/sourabh/.kube /home/sourabh/.minikube

[evanspiatt]

Resolved #4349 by running as:
sudo /usr/local/bin/minikube start -p Dan1 --vm-driver=none

From curl install command minikube was installed in /usr/local/bin but root does not have that in PATH.

[alisianoi]

Hi, this is referenced by

https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md

Since this issue was closed, it gives the impression that the none driver could be run as non-root. However, I believe root is still required for it.

[orodbhen]

@alisianoi may be correct. Although both of these posts claim to have run as non-root:

• http://www.ethernetresearch.com/kubernetes/kubernetes-how-to-install-minikube-in-a-vm/
• https://medium.com/@nieldw/running-minikube-with-vm-driver-none-47de91eab84c

After following the instructions there (which are the same printed by the minikube installer), I still get:

[user@host ~]$ kubectl version
Error in configuration: 
* unable to read client-cert /root/.minikube/client.crt for minikube due to open /root/.minikube/client.crt: permission denied
* unable to read client-key /root/.minikube/client.key for minikube due to open /root/.minikube/client.key: permission denied
* unable to read certificate-authority /root/.minikube/ca.crt for minikube due to open /root/.minikube/ca.crt: permission denied

kubectl refused, it seems, to look in my user home for the configuration files.

[tstromberg]

This issue was unintentionally closed.

[afbjorklund]

I think we should make this the default (run none with sudo), and fix it as a part of #3718

[tstromberg]

I believe the only remaining work is to add sudo support to the Copy and Remove calls in exec_runner.go Anyone up for it?

[concaf]

@alisianoi may be correct. Although both of these posts claim to have run as non-root:

• http://www.ethernetresearch.com/kubernetes/kubernetes-how-to-install-minikube-in-a-vm/
• https://medium.com/@nieldw/running-minikube-with-vm-driver-none-47de91eab84c

After following the instructions there (which are the same printed by the minikube installer), I still get:

[user@host ~]$ kubectl version
Error in configuration: 
* unable to read client-cert /root/.minikube/client.crt for minikube due to open /root/.minikube/client.crt: permission denied
* unable to read client-key /root/.minikube/client.key for minikube due to open /root/.minikube/client.key: permission denied
* unable to read certificate-authority /root/.minikube/ca.crt for minikube due to open /root/.minikube/ca.crt: permission denied

kubectl refused, it seems, to look in my user home for the configuration files.

@orodbhen were you able to work around this error? I am getting this as well, and I need to change values in my ~/.kube/config to get this working every time I start a new cluster.

[vnzongzna]

@tstromberg Can I take this issue?

[tstromberg]

@vaibhavk - Yes, we would absolutely love the help!

This issue is currently targeted to be resolved for v1.4.0 (Sep 16th), but I think it's OK if it ships later. Just mention here if you think the issue will need more time and I'll move it to the v1.5.0 candidates instead.

Thank you!

[tstromberg]

Any update?

[tstromberg]

@vaibhavk - Anything we can help with on this issue?

[medyagh]

@vaibhavk we haven't heard any updates. I unassigned this issue.

this issue is free for anyone to take ! I would be happy to review the PR for it.

[adarshaJha]

same error, and i can't see any resolution of the problem yet. I would request community to please update on this.

[meandmymind]

@medyagh @adarshaJha @tstromberg
As was pointed out by @containscafeine, if you just move .kube and .minikube folders to your user home and change permissions, it does not change .kube/config, that has following lines:

apiVersion: v2
clusters:
- cluster:
    **certificate-authority: /root/.minikube/ca.crt**
    server: https://192.168.37.129:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    **client-certificate: /root/.minikube/client.crt**
    **client-key: /root/.minikube/client.key**

So changing /root part to /home/{your_user} fixes issues. I guess we can do it automatically but I don't know minikube codebase. At least we can put this in doc on site.

[elgcdju]

You cannot start minikube with --vm-driver=none as a non-root user. Thus you cannot move .kube and .minikube directories out of root homedir.

[concaf]

@meandmymind instead of changing paths, you can also run kubectl config view --flatten and save the result.

[meandmymind]

@elgcdju If I get you right, you say that we can't do it programmaticaly if minikube start is called from non-root user?

As a workaround, we can leave sudo minikube start ... part, but internally move .kube and .minikube to a user folder. So after launching cluster with sudo we don't need to use it like that.

[medyagh]

This issue is still open and I would be happy to review any PR that makes none run without sudo.

[belvederef]

The user VKR provided the full list of steps to make it work without sudo in this StackOverflow answer: https://stackoverflow.com/a/58557265/5663986

[jaxi]

I might have missed something here but

CHANGE_MINIKUBE_NONE_USER=true sudo -E minikube start --vm-driver=none

seems to work straight away?

maybe we don't even need CHANGE_MINIKUBE_NONE_USER as long as --vm-driver=none?

Edit: IMO DX wise it's more straight forward to chown .minikube under SUDO_USER as long as SUDO_USER exists - https://github.com/kubernetes/minikube/blob/c9ec946b6463f2e79b1a23f0d05f004840448449/pkg/util/utils.go#L71-L72

[tstromberg]

This is still on our radar.

I encourage that users of --driver=none consider using the newer --driver=docker, which does not require root access.

That said, we still plan to support --driver=none for users with use cases that are incompatible with --driver=docker, though it's unclear what those use cases are.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[AlekSi]

That said, we still plan to support --driver=none for users with use cases that are incompatible with --driver=docker, though it's unclear what those use cases are.

We encountered a problem with minikube and Docker driver on GitHub Actions CI – there is some networking or firewall problem (?), and several pods can't connect to each other. It works with none driver. Please keep it supported :)

And yeah, "automatic" sudo would be nice.

[medyagh]

@AlekSi for running minikube in githbhub action we have a tutorial here
https://minikube.sigs.k8s.io/docs/tutorials/setup_minikube_in_github_actions/

have you tried that ?

[medyagh]

meanwhile this feature is till available for anyone who wants to pick it up ! i would happily review a PR for this

[claflico]

Want to chime in on our use case. We're trying to move towards K8s from Rancher 1.6 Cattle/Traefik 1.7 running in VMware & Active Directory DNS. Management wants to transition from developers running docker-compose on their Macbooks to standalone K8s running on CentOS VMs in the VMware datacenter. Http microservices, databases, etc need to still be accessible remotely via each developer's wildcard DNS url. Would like for K8s API/dashboard to be remotely accessible as well.

Started off with minikube using none driver and couldn't solve the non-root permissions issue. Tried using the docker driver and couldn't figure out how to have K8s use the host IP address instead of a 172.x.x.x address.

Moved on to kubeadm and have K8s standalone working without root access but can't find/figure out an ingress-conroller that works on port 80. Would like to stick with Traefik if possible

Per management, cloud solutions and services are not an allowed option.

[claflico]

Not sure what's changed from 1.11.0 and 12.1 but tonight when I nuked the kubeadm cluster and setup minikube again it mostly worked:

curl -Ss -L https://github.com/kubernetes/minikube/releases/download/v1.12.1/minikube-linux-amd64 > minikube
chmod +x minikube
sudo mv minikube /usr/local/bin/
sudo -i env CHANGE_MINIKUBE_NONE_USER=true MINIKUBE_HOME=$HOME KUBECONFIG=$HOME/.kube/config /usr/local/bin/minikube start --driver=none

kubectl cluster-info Kubernetes master is running at https://10.xx.xx.xx:8443 KubeDNS is running at https://10.xx.xx.xx:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

That hadn't worked before because it was trying to read the certs in the /root/ directory.

However, now when I try to enable the dashboard I am getting access denied:

minikube addons enable dashboard enable failed: run callbacks: running callbacks: [create: open /etc/kubernetes/addons/dashboard-ns.yaml: permission denied]

To enable the dashboard I have to run
sudo -i env CHANGE_MINIKUBE_NONE_USER=true MINIKUBE_HOME=$HOME KUBECONFIG=$HOME/.kube/config /usr/local/bin/minikube addons enable dashboard

[AlekSi]

@AlekSi for running minikube in githbhub action we have a tutorial here
https://minikube.sigs.k8s.io/docs/tutorials/setup_minikube_in_github_actions/
have you tried that ?

Of course. My issue is much harder than covered by that guide :) For example, it can't be reproduced locally.

I left this comment to remove lifecycle/stale label. That did not help.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[AlekSi]

/remove-lifecycle rotten

[Git-Jiro]

Hi!
I want to start tackling this issue and got stuck pretty soon:
If you run "minikube" as a regular user, how do you "sudo" when trying to 'copy' a file out of 'pkg/minikube/assets/assets.go' ?

I am stuck at 'pkg/minikube/bootstrapper/bsutil/files.go' line 56

The only idea i have at the moment would be to do it like this:
"Copy" the file into a temporary directory and then "sudo cp" from there.
Is there a better way to do this?

[tstromberg]

@Git-Jiro - That's the best idea I have as well, unfortunately. I would start with that approach.

[Git-Jiro]

@tstromberg related question, since some automated tests are failing:
How should I detect when to use "sudo" and when to keep using the native golang functions so I do not break the non Linux use cases?