Microk8s: What are the minimal hardware requirements to run microk8s

[leandrodvd]

Hi, I could not find this info in the documentation. Is there a minimal hardware requirement to tun microk8s ? I'm trying to find what is the tiniest IoT device that would be able to run it. Kubernetes stand alone takes ~400MB in my last try and that's just too much for tiny devices with <521MB of memory. I'm wondering how microk8s can reduce the amount of resources taken (mem, storage, cpu).

[xordspar0]

I'd like to know this as well. I just tried microk8s on the smallest Digital Ocean droplet (1 Xeon core, 512MB RAM). That was too small. The system load was at 20 and the memory filled up within minutes. microk8s.inspect took more than 45 minutes to run before I gave up and killed it.

[tvansteenburgh]

@xordspar0 Curious which (if any) of the addons you had enabled?

[xordspar0]

No addons, just plain old stable v1.13.2 out of the box.

[ktsakalozos]

On a t2.nano (1 core 0.5GB mem) from AWS MicroK8s will not start out of the box. Seems the API server consumes too much memory:

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                                                                                                                                    
 1725 root      20   0  445572 208888      0 S  8.5 42.5   1:00.53 kube-apiserver                                                                                                                                                                             
 1724 root      20   0  580628  28844      0 S  4.6  5.9   0:32.69 dockerd                                                                                                                                                                                    
 1722 root      20   0  204408  13624      0 R  8.5  2.8   0:47.61 kube-controller                                                                                                                                                                            
 2190 root      20   0  395160  13544      0 S  4.3  2.8   0:23.93 docker-containe                                                                                                                                                                            
  961 root      20   0  639056  11496      0 S  0.7  2.3   0:06.23 snapd                                                                                                                                                                                      
 1713 root      20   0  746472  10828      0 S  8.5  2.2   0:43.24 kubelet                                                                                                                                                                                    
  723 root      20   0  170868  10356   2504 S  0.0  2.1   0:00.08 networkd-dispat                                                                                                                                                                            
 1726 root      20   0 10.036g   9172      0 S  8.9  1.9   0:45.30 etcd                                                                                                                                                                                       
 1730 root      20   0  139328   7000      0 S  1.3  1.4   0:18.12 kube-scheduler                                                                                                                                                                             
 1727 root      20   0  136924   5984      0 S  2.0  1.2   0:17.92 kube-proxy                                                                                                                                                                                 

Here is what you could try. have a look at the admission plugins https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#how-do-i-turn-off-an-admission-controller and disable the ones you do not need. To do so you will need to edit /var/snap/microk8s/current/args/kube-apiserver and introduce the --disable-admission-plugins="plugins-to-disable" argument. This should address the memory pressure but the machine was painfully slow and i did not proceed any further. If you come up with configuration suggestions we would gladly consider them for future releases.

[CalvinHartwell]

Hi @leandrodvd,

Is this for a commercial application/deployment? If so, we can introduce you to our IoT team if you wish. Please email me at calvin.[email protected].

I can confirm that Ubuntu 18.04.02 runs on Raspberry Pi with Microk8s. I'm putting together a blog post for this soon which will be on the Canonical website. I've tested this with the RPI Model 3 B.

I can confirm that it runs, though there is only around 380mb of free memory it seems, so you can only run a few containers.

Images here: https://imgur.com/a/7ct4Uvo

The ubuntu image can be found here, it is relatively new: http://cdimage.ubuntu.com/ubuntu-core/18/stable/current/.

Cheers,

• Calvin

[leandrodvd]

Hi @CalvinHartwell I asked more to understand the limits. Some time ago I worked no something to run docker containers in IoT devices and we ended up discarding kubernetes as it would consume too much resources on tiny devices. I was just wondering how microk8s could be better in that scenario. But I'm no longer involved on it so this is really more a curiosity. Look forward to your blog post about it.

[xordspar0]

I just tried microk8s on a server with 1 GB of RAM and it works quite well. microk8s.inspect completes in under a minute, and there is plenty of memory available on the system. 20 minutes in, and it looks great. So it looks like the minimum is somewhere between 512 MB and 1 GB memory.

top - 21:04:09 up 34 min,  2 users,  load average: 0.02, 0.10, 0.23                              
Tasks: 110 total,   2 running,  75 sleeping,   0 stopped,   0 zombie                             
%Cpu(s):  4.1 us,  1.4 sy,  0.0 ni, 93.8 id,  0.3 wa,  0.0 hi,  0.3 si,  0.0 st
MiB Mem :  985.504 total,   62.203 free,  497.512 used,  425.789 buff/cache
MiB Swap:    0.000 total,    0.000 free,    0.000 used.  342.871 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                       
 3486 root      20   0  833.5m  47.5m  18.9m S  1.3  4.8   0:37.33 kubelet
 3520 root      20   0  502.3m 199.1m  22.2m S  1.3 20.2   0:56.81 kube-apiserver
 3539 root      20   0  200.3m  47.5m  12.9m S  1.3  4.8   0:42.06 kube-controller
 3494 root      20   0  136.1m  11.2m   3.4m S  0.7  1.1   0:16.99 kube-scheduler
 3507 root      20   0 10.049g  28.3m   6.4m S  0.7  2.9   0:17.03 etcd
 3555 root      20   0  378.4m  35.8m   2.9m S  0.3  3.6   0:07.80 dockerd
 4006 root      20   0  323.3m  15.4m   1.8m S  0.3  1.6   0:03.46 docker-containe
16437 jordan    20   0   39.5m   3.6m   3.0m R  0.3  0.4   0:00.02 top

[CalvinHartwell]

@leandrodvd the blog post went out, but I noticed the formatting needs to be fixed: https://blog.ubuntu.com/2019/02/27/single-node-kubernetes-on-raspberry-pi-microk8s-ubuntu. I will get that done in the next 24 hours.

I hope this helpls!

[stefankorun]

@leandrodvd the blog post went out, but I noticed the formatting needs to be fixed: https://blog.ubuntu.com/2019/02/27/single-node-kubernetes-on-raspberry-pi-microk8s-ubuntu. I will get that done in the next 24 hours.

I hope this helpls!

Thanks, great post but I have trouble installing microk8s on Raspberry Pi 2 running Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-1036-raspi2 armv7l) probably since there is no arm64 version of available.

Running:
sudo snap install microk8s --classic

Results in:

error: snap "microk8s" is not available on stable for this architecture (armhf) but exists on other architectures (amd64, arm64, ppc64el).

Is there any workaround?

[ktsakalozos]

The first models of rpi2 came with armv7. Later they switched to arm64 (same as the rpi3). For now the only option you have is to follow the instructions at https://github.com/ubuntu/microk8s#building-from-source to build a snap package from source. I do not know if all of the add-ons have armv7 images but the Kubernetes cluster should work.

[stefankorun]

I've tried the snap build process but failed multiple times (mostly over strange network errors) and since its really slow on RPi 2 I cannot give you any feedback immediately.

I'll try again in the next couple of days and report back if any progress is made.

[Swivelgames]

Unfortunately the original question has yet to be addressed: What are the minimum memory, storage, and cpu requirements for a microk8s instance?

[xordspar0]

In my experience it runs well with 1 CPU and 1 GB memory with enough extra room for a couple small pods. The snap itself takes up 216 MB of storage, so a few GB should be fine.

[harindaka]

I ran microk8s on a AWS Lightsail VPS with 1 GB RAM and 1 Core with Ubuntu 18.04. Microk8s itself runs fine but whenever I try to deploy via kubectl apply, the api server becomes unresponsive and the command times out. i.e. "Error from server (Timeout): error when creating "Kubernetes/dev-ingress.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (post ingresses.extensions)"

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.