Charts: [stable/prometheus-operator] Add arm64 architecture support

[cablespaghetti]

As already reported in these previously closed issues #19341 and #22644, a number of the containers used in this Helm Chart are not multi-arch and so the chart can only be installed on amd64-based systems. It looks like the prometheus-operator project is now building multi-arch images from v0.39.0, so this is now a very achievable goal.

Is your feature request related to a problem? Please describe.
I would like to install this on clusters using arm64 such as AWS Graviton2-based instances or a Raspberry Pi cluster.

Describe the solution you'd like
Multi-arch is now becoming very common in Docker images. So we should work with the maintainers of the images used in this chart to add support for arm64, or think about changing images if necessary.

Describe alternatives you've considered
This repository exists with alternative images https://github.com/uocxp/prometheus-operator-chart-arm, but I would prefer to not have to reconfigure the Helm chart to work on arm64 as multi-arch support is now commonplace in Docker registries.

Additional context
Images which are a problem in the current chart version (9.2.1 at the time of writing):

• quay.io/coreos/prometheus-operator:v0.38.1 - this should be as simple as upgrading to v0.39.0 or newer
• quay.io/coreos/prometheus-config-reloader:v0.38.1 - same as above
• squareup/ghostunnel:v1.5.2 - existing bug https://github.com/ghostunnel/ghostunnel/issues/260
• quay.io/coreos/kube-state-metrics:v1.9.7 - existing bug https://github.com/kubernetes/kube-state-metrics/issues/1037
• kiwigrid/k8s-sidecar:0.1.151 - existing bug https://github.com/kiwigrid/k8s-sidecar/issues/48

[Johannestegner]

If it haven't changed recently, the quay registry is sadly still running an older registry version which does not accept multi-arch manifests in single pushes, making building multi-arch quite annoying (although not impossible). But seeing this is a coreos repository and quay is a redhat project, this might be possible to take care of ;)

Other than that, it seems like a few of the images are already prepared for this, and it is really a good idea to make it available for ARM64!

[cablespaghetti]

Looks like kube-state-metrics is well on the way to being done. So just the two in Docker Hub actually need work. 🙂

[cablespaghetti]

Made a PR against ghostunnel which hopefully should be merged soon. Will take a look at the kiwigrid/k8s-sidecar next :)

https://github.com/ghostunnel/ghostunnel/pull/313

[cablespaghetti]

And a PR against k8s-sidecar: https://github.com/kiwigrid/k8s-sidecar/pull/86

[cablespaghetti]

Looks like the kube-state-metrics changes have been merged and so we're just waiting for them to cut a release.
ghostunnel/ghostunnel changes have been merged and expected to cut a release in the next few days.
Waiting on a reply to the k8s-sidecar PR.

[cablespaghetti]

Current state of things:

• Still waiting for a kube-state-metrics release
• ghostunnel v1.5.3 is released and now has multiarch :tada:
• My pr for kiwigrid/k8s-sidecar has been merged so there should be a tag available for that soon too. :)

When all this is in place I'll make a PR.

[scottrigby]

🚧 This chart is now deprecated.

Further development has moved to prometheus-community/helm-charts. The chart has been renamed kube-prometheus-stack to more clearly reflect that it installs the kube-prometheus project stack, within which Prometheus Operator is only one component.

If applicable, please re-open this issue there. Thanks!