Helmfile: Use values/secrets from within a remote chart

[witten]

My use case: I'm using Helm charts hosted on a private package server not only as a means of packaging up and distributing Kubernetes templates, but also for distributing associated per-environment values and secrets. This means that our helm-secrets files are included in the charts themselves. The main benefit of an approach like this versus keeping the secrets separately is that we don't need a source code checkout (or more realistically, n source code checkouts, one per chart) to deploy. Instead, we can just use the remote chart on the chart server (effectively a build artifact) and deploy it directly.

Where Helmfile comes in is that I'd like to declare several Helm charts to deploy in a helmfile.yaml's releases:, and have Helmfile able to pull the relevant values and secrets out of remote charts during deployment (helmfile sync). Under the hood, this might work by first helm fetching each remote chart so that the values and secrets files can be accessed.

Also, there's the question of how to refer to those values/secrets. As far as I know, values: and secrets: currently are interpreted as ~absolute paths or~ paths relative to the current directory. What we'd need for this feature request to work is a path relative to the root of the remote chart.

First, is something like this possible today? Perhaps with exec or some other mechanism? If not, does this make sense as a feature request?

[witten]

Well, here's a total hack that appears to work for this use case:

releases:
  - name: my-chart
    namespace: my-namespace
    chart: my-charts/my-chart
    set:
      - name: throwaway
        value: {{ exec "helm" (list "fetch" "my-charts/my-chart" "--untar") }}
    values:
      - my-chart/helm_vars/values.yaml
      - my-chart/helm_vars/beta/values.yaml
    secrets:
      - my-chart/helm_vars/secrets.yaml
      - my-chart/helm_vars/beta/secrets.yaml

Values and secrets from the downloaded chart appear to get consumed properly. It'd be nice if there was a better way to accomplish this. Even if it's just a better place to run exec.

[mumoshu]

@witten Hey! Thanks for the feature request. Looks like you have a very interesting use-case 👍

We're planning to implement hooks #295. With that perhaps you could write helmfile.yaml like:

releases:
- name: my-chart
  namespace: my-namespace
  chart: my-charts/my-chart
  # ...
  hooks:
    preChartLoad: # register as many commands as you want to this hook
    - command: {{` helm fetch {{ .Chart }} --untar`}}

Would it work for you?

[witten]

Yup, that would totally work. And a post hook of some sort would also allow cleanup of the untarred chart directory as well. The only downside I could see is that it could get pretty duplicative if I've got fifty releases and I need to declare the same hooks for each one.

I think the only question for you is whether you'd rather enable this use case via a hook, or whether it's the sort of thing that would be better supported as a first-class Helmfile feature. (Assuming you want to enable this use case at all.) Either way works for me.

[2ZZ]

+1 this would be very useful

[mumoshu]

And a post hook of some sort would also allow cleanup of the untarred chart directory as well

Makes sense! How about postInstall?

The only downside I could see is that it could get pretty duplicative if I've got fifty releases and I need to declare the same hooks for each one.

Yeah, I think so too. You can probably achieve that with helmfile.yaml templates, or maybe kind of plugins #325.

the sort of thing that would be better supported as a first-class Helmfile feature

I'm not yet sure what would be the best way! It may make sense to give it an out-of-box support if several users use it. But hooks, that already has other use-case, seems to support this specific feature well. I'd just begin with hooks then!

[mumoshu]

@2ZZ Thanks for the feedback! Would you mind sharing your exact use-case(s)?

[2ZZ]

@mumoshu our use case is the same, we want to store multiple environments values files within a packaged helm chart. This way we can easily version control the config in the same repository as the helm chart itself.

[witten]

Makes sense! How about postInstall?

That'd work.

In terms of use cases, I'm actually surprised that storing values/secrets inside of a Helm chart isn't more common. I honestly don't know how most organizations distribute and deploy values/secrets with Helm/Helmfile if they're not using charts for that. Clone a copy of the source for every deploy? Mostly I'm just musing rhetorically here.

[mumoshu]

Clone a copy of the source for every deploy?

Currently this, to avoid the need of maintaining an internal helm chart repository.

But yeah, having something like S3 to store charts bundled with secrets and values seems to make cloning unnecessary and results in faster deployments. Is that your intention?

[witten]

Yes, and performance aside, just because of the logistics of it. I currently have dozens of separate repositories, each one with a separate chart (+ values + secrets). If I want to deploy all of these charts via a single helmfile.yaml without the use of a chart repository, then I'd have to clone dozens of repositories at deploy time to make all of this work.

The chart repository I happen to be using is ChartMuseum.

[mumoshu]

@witten You can use prepare and cleanup hooks added in #349. You may also be interested in #195, too. It is about what if helmfile's out-of-box feature package feature enables you to package up helmfile.yaml along with local and fetched remote charts. Of course your values.yaml(.gotmpl) files would be packaged as well, into a single tgz archive.