Flux: helm operator: support external helm repos

[davidkarlsen]

today flux/helm-operator is based on checking in the charts into the same git-repo as the yaml files declaring FluxHelmRelease resources.

It would be nice if instead the configuration could point at helm repositories (like for instance https://kubernetes-charts.storage.googleapis.com) - that way the helm chart repo can be centralized and reused.

Note to other people having the same problem: you can circumvent this by creating a subchart(https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/subcharts_and_globals.md) and refer the external repo through requirements.yaml and point to the external repo - but this is not optimal as it requires some scaffolding and also messes with the naming

[konfiot]

Would be great if implemented !

[ahmadiq]

Hi @davidkarlsen can you confirm if you've tried this successfully with subcharts? I tried it but I still had to check-in the downloaded chart into my repo.
ref: https://github.com/weaveworks/flux/issues/1278#issuecomment-413980355

[davidkarlsen]

@ahmadiq I have not tried it with subcharts and I'd guess it would break / you'd need to hack it

[davidkarlsen]

@ahmadiq I'll correct myself - yes it works with subcharts. The way I do it is with a hacky shellscript to suck down the charts and add them to my git-repo so that flux is happy:

cat init.sh 
#!/bin/bash

set -e
helm repo update

CHARTS="stable/prometheus stable/heapster stable/grafana stable/kubewatch stable/cert-manager stable/nginx-ingress stable/kubewatch stable/kubernetes-dashboard int128/kubernetes-dashboard-proxy kubernetes-charts-incubator/elasticsearch stable/fluent-bit stable/openebs stable/weave-scope stable/spinnaker appscode-stable/kubed stable/external-dns stable/kibana stable/hubot kubernetes-charts-incubator/elasticsearch-curator stable/elasticsearch-exporter stable/anchore-engine stable/cerebro stable/kube-ops-view stable/metrics-server stable/minio stable/datadog stable/elastalert stable/auditbeat stable/prometheus-blackbox-exporter stable/nfs-client-provisioner"

#monocular/monocular

for chart in $CHARTS; do
  helm fetch $chart --untar --devel
done

helm fetch rook-beta/rook-ceph --untar --version v0.8.1

git add . && git commit -m"update charts"

that way charts are complete and checked in - including subcharts.
The anchore one for instance uses subcharts.

[dbluxo]

+1

[aaron-trout]

For me, this is the number one missing feature at the moment.

[aaron-trout]

Related thoughts: Was thinking about using a "charts" repo like https://github.com/kubernetes/charts for our own charts. This can work but would need to run two copies of the flux helm operator, one pointing at the official charts repo, and one pointing at our custom charts repo for our own applications.

Could also be a valid use case to point at multiple chart repos, possibly a separate GH issue for that?

[stefanprodan]

The external Helm repos support has been implemented in this PR #1382

[aaron-trout]

@stefanprodan

The external Helm repos support has been implemented in this PR #1382

Oooh, thanks! I will take a look!

[davidkarlsen]

That one seems merged - but still the issue is open? Is this usable - or are other parts required?

[dbluxo]

@stefanprodan

The external Helm repos support has been implemented in this PR #1382

Does this mean I can reference dependencies in requirements.yaml pointing to our own chartmuseum repo? And a helm dependency update will be executed before installing?

[hiddeco]

@dbluxo @davidkarlsen

Those assumptions are true with version 0.5.x. For details on how to upgrade, see: https://github.com/weaveworks/flux/blob/master/site/helm-upgrading-to-beta.md

I think the issue is still open because we do not support full authentication for registries yet in a easy way. But @squaremo should be able to tell you more.

[squaremo]

I think the issue is still open because we do not support full authentication for registries yet in a easy way.

To use external repos that need authentication, you'll need to mount a repositories.yaml file listing all the repos you want to use, along with their credentials. The easiest way is to start with a fresh $HELM_HOME and do helm repo add, then grab $HELM_HOME/repository/repositories.yaml -- but be aware, this file needs a bit of editing, before being put in a Secret to be mounted. Details are at

https://github.com/weaveworks/flux/blob/master/site/helm-integration.md#authentication-for-helm-repos

I'm not sure how well our flux chart supports this at present, and anyway, it's a bit of a workaround. I've opened #1567 for implementing chartPullSecret, which lets you attach a secret with credentials to each HelmRelease.