Microk8s: Is there a way to use host docker like Minikube `eval $(minikube docker-env -u)`

[melissapalmer]

Is there a way to switch microk8s to use the docker host for images?
I have read https://github.com/ubuntu/microk8s/issues/287 and https://github.com/ubuntu/microk8s/blob/master/docs/registry.md

and cannot see how to do this. Minikube's eval command is simple easy way to switch I am looking for something similar.

Running 'docker -H unix:///var/snap/microk8s/current/docker.sock ps' does not seem to have any effect for me.

Please run microk8s.inspect and attach the generated tarball to this issue.

$ microk8s.inspect
Inspecting services
  Service snap.microk8s.daemon-docker is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Service snap.microk8s.daemon-etcd is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system info
  Copy network configuration to the final report tarball
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Inspect kubernetes cluster

Building the report tarball
  Report tarball is at /var/snap/microk8s/412/inspection-report-20190306_230601.tar.gz

We appreciate your feedback. Thank you for using microk8s.

[waquidvp]

So you have a separate docker on your host machine and want to use the images there in your microk8s?

[melissapalmer]

Yes. I’m working locally on my own pc. Have Docker installed and microk8s. I’ve already built images with Docker using
‘docker build ....’ when using microk8s with helm it does not find this image. I then have to run ‘microk8s.docker build ....’ and build image again to microk8s registry.

I just want to build image once and use both via standard docker and within microk8s

[ktsakalozos]

Hi @melissapalmer

Let me first say that having your own dockerd next to MicroK8s is not a recommended setup as the two docker daemons fight over container rights. This will be addressed in the v1.14 release where MicroK8s will switch to using containerd. If you experience issues like, containers not starting and/or stopping check apparmor for errors and consider switching to the v1.14 MicroK8s with:

sudo snap install microk8s --classic --channel=1.14/beta

Regarding docker build, when you do a docker build docker builds and caches the image. If you build your image with docker only your local dockerd knows about the image you just built. If you build your image with microk8s.docker only MicroK8s' dockerd knows about the image.

My suggestion is to microk8s.enable registry and then after building your image tag it and push it to the registry used by MicroK8s:

docker tag my-image localhost:32000/my-image
docker push localhost:32000/my-image

Of course this means that in the kubernetes yaml manifests you have to specify where to get the image from (image: localhost:32000/my-image) as shown in https://github.com/ubuntu/microk8s/blob/master/docs/registry.md

To your question if you are able to have MicroK8s use a local dockerd the answer is yes, but this is a path we have not explored. If you _have_ to do this you will need to stop the systemd service snap.microk8s.daemon-docker, configure kubelet to use your local dockerd by editing /var/snap/microk8s/current/args/kubelet and finally restart snap.microk8s.daemon-kubelet.

If you give me a specific script you are using that fails I might be able to offer more concrete assistance.

Thanks

[waquidvp]

As https://github.com/ubuntu/microk8s/issues/287 mentioned, especially with the move to containerd now happening, your best option to stay future proof would be to build locally and use the registry. Eventually microk8s.docker won't be available anymore to run the microk8s.docker build command.

[melissapalmer]

Thank You, @ktsakalozos I have managed to sort out using your recommendation of tagging an image and the pushing it to the registry used by MicroK8s:

The following commands are what I used, and am happy with this it allows me to only build the docker image once.

microk8s.enable registry
docker build -t org.myorg/myimage .
docker tag org.myorg/myimage localhost:32000/org.myorg/myimage
docker push localhost:32000/org.myorg/myimage
helm install --name mychart --set image.repository=localhost:32000/org.myorg/myimage  helm-chart/mychart

[nandanrao]

I'm curious if there is any interest to support another build process? Previous I had a development process that was building using the microk8s dockerd (via microk8s.docker), which was fairly compatible with other developers on our team using minikube.

Building and pushing to the local repo is fine in theory, but it is hard to make compatible in a development process minikube. I realize this is not the main use case, but I wanted to throw it out there as a possibility and would be curious if others have the wish expressed in this issue.