Rke: Arm support

[de13]

Popularity of kubeadm comes (partially) from it's support of RPi, because running K8s on 4 Hypriot RPi's or more is an excellent training playground (far better than minikube). I'm sure that I can change images to point to Google Hyperkube, but why not providing k8s:v1.8.3-rancher2 for armhf architectures?

[ibuildthecloud]

We really just need to compile ARM images and make the image multi-arch (manifest list). Beyond compilation/packaging I don't think there is much issue with supporting ARM. Now having said that I don't think we will support anything but ARM64. We'll see. I plan on looking a ARM support in a bit as I want Rancher 2.0 (the management product) to run on ARM also.

[jmreicha]

Looks like this one got buried. Has there been any more talk or plans for adding ARM support?

Would it be possible to just replace system images with arm64 versions and disable the others?

[ags131]

I've actually been playing with this on my Pi3's running HypriotOS. So far, its been very successful, I replaced all the system images with the direct gcr.io equivalents, built rke-tools from source for arm, and just ran rke from my desktop. All worked fine once I got the right images.

As @ibuildthecloud mentioned, compiling all the images as arm and building the multi-arch manifests should be all thats needed. From what I can tell, other than hypercube, most of the images are currently just republishes of the gcr.io and quay.io images.

The only issues I ran into were a bug in Docker and another issue where Rancher 2 doesn't provide a UI option to specify agent image. Using the View in API editor, I was able to set desiredAgentImage to my image and it worked fine after. (Maybe there needs to be an Advanced setting for that) I suspect this will not survive my next Rancher server upgrade though.

I've written a short guide covering all the steps needed for assembling all this manually right now. https://gist.github.com/ags131/7bdde11c932ef7a54f44c6decbfd88b8

[Yannovitch]

Can I +1 this issue so that it gets "un-buried" ;)?

[antmanler]

have a rancher ran on raspberry pi rancherpi/rancher-arm64 inspired by your wonderful @ags131 gist thank u :)

[ekristen]

@ibuildthecloud any chance we can get your wizard skills to knock this out quick for the masses?

[ekristen]

I'm highly motivated to get this added, just need a little bit of nudge in the right direction.

aarch64 would also need to be supported by the docker manifest as that's identical to arm64 but most modern chips (pine64, rock64) identify themselves as aarch64 instead of arm64 although it's the same.

AArch64 is the 64-bit state introduced in the Armv8-A architecture. The 32-bit state which is
backwards compatible with Armv7-A and previous 32-bit Arm architectures is referred to as
AArch32. Therefore the GNU triplet for the 64-bit ISA is aarch64. The Linux kernel community
chose to call their port of the kernel to this architecture arm64 rather than aarch64, so that's
where some of the arm64 usage comes from.

As far as I know the Apple backend for aarch64 was called arm64 whereas the LLVM community-
developed backend was called aarch64 (as it is the canonical name for the 64-bit ISA) and later
the two were merged and the backend now is called aarch64.

So aarch64 and arm64 refer to the same thing.

https://unix.stackexchange.com/questions/461179/what-is-the-difference-between-different-implemetation-of-arm64-aarch64-for-linu

[tardich]

Any news on eventually supporting mixed platforms (like controlplane on armhf and worker nodes on amd64 on a single rke deployment?

[PhilsLab]

Still waiting for this to happen, and as @ibuildthecloud said, the effort is not too big.
Any news?

[tbrtje]

Has there been an update to this situation?
Running rke on a dual-arch cluster fails, as most of the docker images are amd64 only.

[optimuspaul]

👍 also having issues, dual-arch cluster is a priority for me... I guess rancher isn't going to be my solution 😢

[ekristen]

At this time the rancher agent has been upgraded to multi arch. However for cross architecture you’ll have to build the cluster and import it on your own.

For example you could build a k3s cluster that’s cross arch (at least theoretically, I haven’t tested) and I then import it into rancher2 and it’ll take over management and upgrades of k8s or k3s in this case.