Helmfile: Referencing environment values from within helmfile.yaml

[mumoshu]

267 supports referencing .Environment.Name from within helmfile.yaml, but .Environment.Values. The latter can be referenced from within values file templates only as of today.

Looking into use-cases like @Stono's https://github.com/roboll/helmfile/issues/243#issuecomment-418386220, I can't stop imagining how helmfile could reduce envvar references, replacing with helmfile's env values.

For example, how about exploiting {{` STR `}} expressions to nest templates?

The below example adds template: under each item in releases, so that the yaml string value for template is considered gotmpl that is rendered after .Environment.Values are loaded.

releases:
- template: |{{`
    name: sauron-web
    namespace: sauron-web
    chart: aws/platform-helm-at-service
    version: 0.1.{{ .Environment.Values.goDependencyLabelPlatform" }}
    values:
    - environments/values.yaml
    - environments/{{ .Environment.Name }}/values.yaml
    secrets:
    - environments/{{ .Environment.Name }}/secrets.yaml
    set:
    - name: deployment.containers[0].image.tag
      value: {{ requiredEnv "GO_PIPELINE_COUNTER" }}
`}}

You many prefer accepting the release template without introducing the new key template::

releases:
- |{{`
  name: sauron-web
  namespace: sauron-web
  chart: aws/platform-helm-at-service
  version: 0.1.{{ .Environment.Values.goDependencyLabelPlatform" }}
  values:
  - environments/values.yaml
  - environments/{{ .Environment.Name }}/values.yaml
  secrets:
  - environments/{{ .Environment.Name }}/secrets.yaml
  set:
  - name: deployment.containers[0].image.tag
    value: {{ requiredEnv "GO_PIPELINE_COUNTER" }}
`}}

Or maybe just allow templating per value?
In the below example, only the value for version is templated.

releases:
- name: sauron-web
  namespace: sauron-web
  chart: aws/platform-helm-at-service
  version: 0.1.{{` {{.Environment.Values.goDependencyLabelPlatform}} `}}
  values:
  - environments/values.yaml
  - environments/{{ .Environment.Name }}/values.yaml
  secrets:
  - environments/{{ .Environment.Name }}/secrets.yaml
  set:
  - name: deployment.containers[0].image.tag
    value: {{ requiredEnv "GO_PIPELINE_COUNTER" }}

[sstarcher]

One option would be to have 2 different {{ }} template enclosures. {{ }} for the first pass that must make it a valid YAML and [[ ]] for templating after loading the environment.

Overall, I don't see a strong need for this feature and people would be better off putting the values/sets in a separate yaml file instead of inline.

[mumoshu]

@sstarcher Thanks for the comments!

We'd probably need to tackle slightly similar problem in #295 cc/ @elemental-lf
In #295, we'd need to defer templating, but with different parameters like release, chart, revision, and other contextual information.

The important difference here is that [[ ]] is templated after {{ }}, but before hook commands. We can't use same symbols to start a template expression in hooks. So a helmfile.yaml would look like this:

environments:
  prod:
    values:
     - prodvalues.yaml

{{ if eq .Environment.Name "prod" }}
releases:
- name: myapp
   chart: mychart
   version: 0.1.0-[[ .Environment.Values.platform ]]
   hooks:
     preChartLoad:
      - command: {{` ./chartify -e {{ .Environment.Name }} {{ .Chart }} `}}
{{ end }}

Wow, we now have three variants of templates... 😭

[sstarcher]

sounds painful

[mumoshu]

yeah..

[mumoshu]

A little, possible simplification would be to use {{ }} instead of [[ ]].

Then, it becomes a vanilla golang template without need of double-rendering:

environments:
  prod:
    values:
     - prodvalues.yaml

{{ if eq .Environment.Name "prod" }}
releases:
- name: myapp
   chart: mychart
   version: 0.1.0-{{` {{ .Environment.Values.platform }} `}}
   hooks:
     chartify:
       on: ["preChartLoad"]
       command: {{` ./chartify -e {{ .Environment.Name }} {{ .Chart }} `}}
{{ end }}

Edit 1:

Let's try stretching the above example to make the release optional per not the environment name, but an environment value.

Suppose we add support the option to mark chart as not installed #197:

environments:
  prod:
    values:
     - prodvalues.yaml

releases:
- name: myapp
  installed: {{` {{ .Environment.Values.myapp.installed }} `}}
  chart: mychart
  version: 0.1.0-{{` {{ .Environment.Values.platform }} `}}
  hooks:
    chartify:
      on: ["preChartLoad"]
      command: {{` ./chartify -e {{ .Environment.Name }} {{ .Chart }} `}}

Seems to be working so far, isn't it? 🤔

[sstarcher]

ya, having users double escape like that would function.

[mumoshu]

I even wonder if we can make templating of helmfile.yaml optional e.g. only helmfile.yaml.gotmpl whose file ext is .gotmpl is templated before parsed as yaml.

So that a vanilla helmfile.yaml without templating would look like the below, that has no need of double-escaping, while it is still possible to use templating per value basis:

environments:
  prod:
    values:
     - prodvalues.yaml

releases:
- name: myapp
  installed: "{{ .Environment.Values.myapp.installed }}"
  chart: mychart
  version: "0.1.0-{{ .Environment.Values.platform }}"
  hooks:
    chartify:
      on: ["preChartLoad"]
      command: "./chartify -e {{ .Environment.Name }} {{ .Chart }}"

I guess we had discussed about that before #98?

Edit: Some more context

[davidovich]

I’m a bit confused. How can

releases:
- name: myapp
  installed: "{{ .Environment.Values.myapp.installed }}"

Work if you haven’t settled a way to reference an environment? This issue (special template syntax to postpone rendering) or #308. ?

Why would you need to postpone the hook? If the env is available, should it just be used directly so that the rendered result contains the hook instruction?

[mumoshu]

Work if you haven’t settled a way to reference an environment?

Similarly to https://github.com/roboll/helmfile/issues/273#issuecomment-419786262, installed: "{{ .Environment.Values.myapp.installed }}" in a vanilla helmfile.yaml is executed AFTER helmfile.yaml is loaded, but before any helmfile command is ecxecuted.

Moreover, in a helmfile.yaml.gotmpl, installed: {{ .Environment.Values.myapp.installed }} would do the job.

Why would you need to postpone the hook? If the env is available, should it just be used directly so that the rendered result contains the hook instruction?

That's because the hook command requires the release name, chart, version, and other information that differs per release. Probably https://github.com/roboll/helmfile/issues/295#issuecomment-419618342 provides you some context about it!

[elemental-lf]

Environments introduce another level of indirection which reminds me of https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering. We should be careful here, so that the complexity stays in check, and I'm not sure this is moving into the right direction currently.
I think the problems we're having are a symptom of the fact that we're trying to mix input and output data in the same file for more or less the same process. We should go back to just being able to specify an environment name on the command line. The user can then code the different values for each environment into the helmfile template itself. For example
like:

releases:
  - name: my-release
    values:
      - ../values/global/my-release.yaml
      - ../values/{{ .Environment.Name }}/my-release.yaml

or:

releases:
  - name: my-release
    values:
      - ../values/global/my-release.yaml
{{ if (eq .Environment.Name "production" }}
      - ../values/prod/my-release.yaml
    wait: true
{{ end }}

or:

# Only install this in the production environment
# (Why do we need the installed switch anyway when we can do this?)
{{ if (eq .Environment.Name "production" }}
releases:
  - name: my-release
    values:
      - ../values/prod/my-release.yaml
    wait: true
{{ end }}

The value files could even be templates on top of that already. With this scheme it is clear again what is input (the environment name on the command line) and what is output (the rendered helmfile). This will lead to some duplication and won't stay completely DRY. But being DRY isn't an end in itself. In our case it would add complexity to the implementation and it would also make it hard for the user to understand. This is all IMO of course.

[mumoshu]

@elemental-lf I basically agree with you.

One thing I'd like to add to your thought is that we still need to escape template expression for chart hooks.

I want to avoid it, because it looks very unintuitive. And such escaping can be avoided only by making templating of helmfile.yaml optional, as far as I can think.

So, for basic use-cases like I and you imagine, I'd suggest making helmfile.yaml execute template value-wise, not entire file.

But how do we toggle installation of my-release according to the selected environment name? That's basically installed: "{{ .Environment.Values.myapp.installed }}". No entire file templating. Do it per value. Add more config keys with support for templating of values if necessary. Then, there's no more confusion of mixing inputs and outputs as you pointed out!

However, I do agree with the power of templating entire helmfile.yaml with access to environment values. So keep it as the last resort, enabling it only when the file ext is ,yaml.gotmpl.

Perhaps you and I usually use vanilla helmfile.yaml's with per-value templating support. @davidovich and I will use helmfile.yaml.gotmpl with entire-file templating support, as the last resort. I can understand both use-cases. There seem to be NO only-one solution to me.

WDYT on this idea?

[davidovich]

I haven't seen so much use cases yet, so I am still blind to what the new direction would allow. When I set out to do double render, I had only the environments: section in mind, which allows natural usage lower in the helmfile. Now what I understand is that there would be special keys (installed:) that helmfile would treat specially ?

Because when I read this:

installed: "{{ .Environment.Values.myapp.installed }}", I feel (from what I gather to this date) is that there should be a corresponding

myapp:
   installed: true

in the prodvalues.yaml file.

But what about this (as mentioned by @elemental-lf):

{{ if (eq .Environment.myapp.shouldBeInstalled true) }}
...
{{ end }}

Of course all this works if you have a self-referential system as I said before (double-render or special template syntax). I prefer the double render solution at the moment, but only for populating the environments: dictionary. I don't know if people agree, but I like the generalization of allowing free-form access to the things I declare in prodvalues.yaml file. I am not sure this should propagate to the non-header parts of the file, if you know what I mean by header part (right now this would be the environments:) section.

I am not sure why you would want to segregate the helmfile.yaml and helmfile.yaml.gotmpl.

More reading on the issues, I get the why the deferred execution of the hooks is needed. I don't see that double rendering is in opposition to that, on the contrary, but I feel (repeat myself :-)) that it should only be applied to the environments: section.

Taking from https://github.com/roboll/helmfile/issues/295#issuecomment-419870215:

releases:
- ...
  hooks:
    preChartLoad: # register as many commands as you want to this hook
    - command: {{` ./chartify -e {{ .Environment.Name }} {{ .Chart }} `}}

I also prefer this form.

[mumoshu]

@davidovich Thanks! I think I understand all your points.

I am not sure why you would want to segregate the helmfile.yaml and helmfile.yaml.gotmpl

And it's just that I'm unsure all the edge-cases due to mixing of inputs/outputs! There's really no header thing in yaml or gotmpl. It looks like we're emulating it by double-rendering. Not saying it won't work(it does! good job), but I don't yet have enough confidence or theory to believe that it works robust. Also, that's what we already do for values.yaml and values.yaml.gotmpl, which may justify it for consistency reason.

In case my reason sounds not enough(I think so) - Probably I'd just proceed with double-rendering enabled by default and no way to disable it.

[davidovich]

Probably I'd just proceed with double-rendering enabled by default and no way to disable it.

Note that the prerender is supposed to be failsafe. In any error condition it will return at least the environment Name (as that is known in advance), as the current implementation does. If there was an error, it writes it as logger.Info().

If you still think it is a good idea, I will implement the final suggestions you added at #308.

[mumoshu]

@davidovich Thanks a lot for awesome work. Let's try!

Again, I understand the usefulness of the feature. So I'm fine as long as it won't break things that worked before, except the improvement to use missingkey=error more 😉

Regarding my suggestion, yeah, I appreciate it if you could address that.

[sstarcher]

@elemental-lf It seems to me that you are missing the power of the environments section. Your examples show just using the environment name to pass in a static values file.

What the environments allow you to do is collect a common set of values that can be reused and templated in more complex ways.

Say I had a environment dev and I had a AWS certificate I wanted to pass to different services.

cert: blah

• Services 1

service:
  annotations:
    cert-annotation: "{{ .Environment.Values.cert}}"

• Service 2

a:
  service:
    annotations:
      cert-annotation: "{{ .Environment.Values.cert}}"

b:
  service:
    annotations:
      cert-annotation: "{{ .Environment.Values.cert}}"

Prior to environments existing I would of had to hard code that value into 3 different places.

[elemental-lf]

@sstarcher, I'm not actually missing the power, I suggest deliberately removing it again/not implementing it. So essentially I'm suggesting to:

• Remove the environments section completely (including the ability to specify values there!)
• Keep the CLI option which sets .Environment.Name
• Render the complete helmfile (which gives us the installed feature via a simple Go template if)
• Hooks would still need escaping

So if you'd want to set different values depending an the environment (name) you'd have to code this explicitly and if needed repeatedly in your helmfile. Thus deliberately ignoring the DRY principle to reduce complexity in the implementation and in the user experience. But I don't feel too strongly about this and you're making a valid of point! Maybe I'm too pessimistic about this and all this plays out well indeed :)

[sstarcher]

Got ya, so you are on the side of not supporting that feature at all. I'm very strongly against the removal of the capability to have shared values. It causes to much of a issue when you have the same single value repeated 100 times in a large setup of microservices. Instead I can now tell a person to use a single variable rendering.

I'm 100% fine with insisting that .Environment.Values can not be used in the helmfile.yaml. Which would resolve this complexity.

[davidovich]

@sstarcher Hi, if you have time, could you pass my PR #308 through your setup and see if you like it?

I'm still not sure which complexity we are talking about. The user knowledge that it is normally not possible to use a self-reference in a golang template? or the complexity of the double-pass (or hypothetical syntax escaping) implementation of this issue? Because the more I play with it, it seems very natural to use the values directly in the "lower" parts (after environments:) sections of helmfile.

I would like to have more test cases so we can be more confident on the safety of double-render, but I fail to see right now the failures it could bring.

[sstarcher]

@davidovich without much testing my primary concern is the double-pass parsing and any edge cases. I'll try and give it a test against my current setup to see if it works.