Minikube: Request: Support ppc64le

[edmondsw]

I want to run minikube on ppc64le, but I found that there are no minikube-linux-ppc64le artifacts built today. I'd specifically like to see v1.2.0 and newer work with a command like this:

curl -Lo /usr/local/bin/minikube https://storage.googleapis.com/minikube/releases/v1.2.0/minikube-linux-ppc64le

Having minikube for ppc64le can be very useful for multi-arch CI testing of a number of projects (e.g. see https://github.com/bsycorp/kind), for instance.

I found this existing issue which was closed due to lack of a specific architecture request, so I'm specifically asking for ppc64le here: https://github.com/kubernetes/minikube/issues/3890

But I also found that there's another issue essentially asking for arm64 support: https://github.com/kubernetes/minikube/issues/6280

And I'm aware of some folks that will probably be asking for support for s390x very shortly as well.

The primary issue may be how to test on non-amd64 architectures. Fortunately, Travis CI now has workers for all of these architectures. See https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z . So minikube could presumably use Travis to test on all of those architectures. It could even replace the current cross-compilation logic with native builds using Travis, but that wouldn't necessarily be required.

[afbjorklund]

Cross-compiling for all Kubernetes architectures can probably be done.

We also need to update the Docker builds to build for all architectures

[afbjorklund]

Platforms:
amd64
arm (32-bit)
arm64
ppc64le
s390x

https://github.com/kubernetes/community/blob/master/contributors/design-proposals/multi-platform.md

[afbjorklund]

We have KVM support for amd64 and arm64*. Not sure which hypervisors are supported on the other architectures, so probably only native (none) support.

* No ISO for ARM, though.

[afbjorklund]

@edmondsw : can you describe the use case a little ? Like how Kubernetes would be used on the PPC etc. Think we already have S/390, but no official binaries... (but also interested in uses on _that_ mainframe)

[edmondsw]

ppc64le supports both KVM and PowerVM hypervisors. There are several hypervisors supported on s390x as well, one of which is KVM.

The use case I'm currently looking at would be to use in https://github.com/bsycorp/kind so you could check that out to see how it's using it.

[afbjorklund]

Not sure what is actually needed here, beyond extending the cross arch list from 2 to 5 architectures ? But if we add multi-arch for v1.8.0, we might as well add all k8s arch... (for minikube binary, and provisioner)

[afbjorklund]

Seems like there are two kinds of kind ? Not the same as https://github.com/kubernetes-sigs/kind

From the description linked, this particular KIND looks somewhat similar to what KIC is doing:

kind, like docker:dind but aims to make a full Kubernetes cluster available to test against in your CI pipeline. It is fast to startup and totally ephemeral, so you get a clean start for each CI run.

• https://github.com/bsycorp/kind#why

• https://github.com/bsycorp/kind#how-does-kind-work

tl;dr; Building on docker-in-docker it uses minikube and kubeadm to bootstrap and pre-configure a cluster at build time that works at runtime.

Building minikube for the ppc64le GOARCH is straight-forward enough, but it seems unlikely that this particular setup is going to be used by many ? However, running in CI is precisely what none driver is intended for so I don't really have a problem with that. It would be nice to collaborate on the particular use case of making it simple to run kubernetes in a CI system, though. But that sounds like another issue...

---

And I _still_ don't know where one would find an IBM POWER machine. They look kinda bulky :-)

https://en.wikipedia.org/wiki/Ppc64

https://en.wikipedia.org/wiki/POWER8

https://en.wikipedia.org/wiki/POWER9

The _Zaius_ looks rather nice, though.

https://cloud.google.com/blog/products/gcp/introducing-zaius-google-and-rackspaces-open-server-running-ibm-power9

https://blog.rackspace.com/first-look-zaius-server-platform-google-rackspace-collaboration

https://www.nextplatform.com/2018/03/26/google-and-its-hyperscale-peers-add-power-to-the-server-fleet/

[edmondsw]

Thanks for pointing out that https://github.com/kubernetes-sigs/kind is a thing. Looking into whether we can use that instead, since it already supports ppc64le.

[afbjorklund]

Binaries for ppc64le for minikube and storage-provisioner have been added for 1.8.0
Please feedback on how the new release is working on the IBM POWER architecture

We will need to see how we can run regression tests on these mainframe architectures,
but for now I'm not sure if Travis will add anything (we should do some integration tests)

[afbjorklund]

Now available here:
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-ppc64le

[afbjorklund]

Discussion moved to #6798