K3s: Certify Ubuntu 20.04 on arm64

[ShylajaDevadiga]

Scope of this issue is to validate and certify Ubuntu 20.04 on Arm64 (including Jetson nano)

Ubuntu 20.04 on amd64 is validated here
https://github.com/rancher/k3s/issues/1712 on K3s v1.18.3+k3s1

[brandond]

Just got a Pi4 4GB with Ubuntu 20.04 arm64 added to my cluster as an agent last night! Working great so far. Had to add amd64 nodeSelectors to a few pods (looking at you OpenEBS) that don't have multi-platform images yet.

I have an 8GB coming soon as well, I might reserve that for use as a single-node cluster to see how sqlite and etcd work on sd cards.

[sesteel]

I replicated the failure on a Pi4 4GB found in https://github.com/rancher/k3s/issues/1712
Again, plain install with no env vars or args.

root@s2:~# k3s kubectl get node
Error from server (ServiceUnavailable): the server is currently unable to handle the request

[brandond]

Do you have any logs to go along with that error @sesteel ? I've been rotating my 4gb and 8gb through various configurations as server and agent and haven't found any issues yet other than just image availability.

I did modify the kernel args to disable apparmor and enable memory cgroup. Other than that they bog-standard installs.

[sesteel]

@brandond, Thanks for the response; I corrected my issue. When I built it, I entered the cgroup settings in the wrong location. On Ubuntu 20.04, they are to be appended in /boot/firmware/cmdline.txt.

[davwilliams]

@brandond, also have a cluster of 8GB Pi4 B but having the same message everyone else has reported. Not sure why I'm having a different experience that you are. Any thoughts on where to look?

root@rpi-k3s-mgr-01:/boot/firmware# kubectl get node
Error from server (ServiceUnavailable): the server is currently unable to handle the request

For what it's worth:

root@rpi-k3s-mgr-01:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"

root@rpi-k3s-mgr-01:/# uname -a
Linux rpi-k3s-mgr-01 5.4.0-1012-raspi #12-Ubuntu SMP Wed May 27 04:08:35 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

root@rpi-k3s-mgr-01:/$ k3s -v
k3s version v1.18.4+k3s1 (97b7a0e9)

root@rpi-k3s-mgr-01:~# crictl version
FATA[2020-06-28T21:25:51.982814356Z] failed to connect: failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded

root@rpi-k3s-mgr-01:~# ctr version
Client:
  Version:  v1.3.3-k3s2
  Revision:

ctr: failed to dial "/run/k3s/containerd/containerd.sock": context deadline exceeded

[davwilliams]

@brandond, @sesteel: All resolved now. I managed to also append the following cgroup info in the wrong place:

cgroup_memory=1 cgroup_enable=memory

Would be great if the log message referenced the correct location for cmdline.txt. It currently says:

"failed to find memory cgroup, you may need to add \"cgroup_memory=1 cgroup_enable=memory\" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)"

Either way, all good now. If there are any other recommended but undocumented tweaks, could you post them here?

[billimek]

FWIW, I've been running ubuntu 20.04 pi4 based nodes in a mixed-architecture k3s cluster for a while now without issue whatsoever. Using the following tweaks to the ubuntu arm64 image makes it pretty easy to install and operate.

The big one is the cgroup changes to cmdline.txt like you say.

[ShylajaDevadiga]

Installation of k3s version v1.18.8+k3s1 is successful.
Validated on:
Jetson nano, Ubuntu20.04 arm64
Raspberry pi, Ubuntu 20.04 arm64 with the update made to cmdline.txt

cat /boot/firmware/cmdline.txt
cgroup_memory=1 cgroup_enable=memory