Helmfile: [Feature] common labels for releases

[mlushpenko]

Hi @mumoshu, I've figured a proper structure to apply changes across all clusters and would like to suggest one feature.

I defined helmcharts as follows:

templates:
  autoscaler: &autoscaler
    name: cluster-autoscaler
    namespace: kube-system
    chart: stable/cluster-autoscaler
    version: "7.2.0"
    needs:
    - kube-system/kube2iam
    values: # paths are relative to the helmfile that defines releases
    - ../helm-values/common/{{ .Release.Name }}.yaml
    - ../helm-values/{{ .Release.Labels.customer }}/{{ .Release.Name }}.yaml

And then in helmafile.yaml:

{{ readFile "templates/common.yaml" }}

helmDefaults:
  kubeContext: example

releases:
- <<: *blackbox
- <<: *autoscaler
  labels:
    customer: example
- <<: *kube2iam
  labels:
    customer: example

labels are used in templating to search for proper file name for values. When number or releases per customer grows, I would have to add this block many times:

labels:
    customer: example

Would you be open to defining common labels for releases, I guess something like this, so they would be automatically added to all releases? There is a similar feature in kustomize

helmDefaults:
  commonLabels:
    label1: value
    label2: value

[mumoshu]

@mlushpenko Sounds greeat! As it isn't strictly a helm arg, I believe it worth a top-level key so that the usage looks like:

commonLabels:
  label1: value
  label2: value

releases:
- ...

WDYT?

[mlushpenko]

Yes, @mumoshu, awesome, it's indeed not a helm argument, so this looks great.

[voron]

@mlushpenko did you tried to create nested template with customer-specific common values ?

templates:
  _autoscaler: &_autoscaler
    name: cluster-autoscaler
    namespace: kube-system
    chart: stable/cluster-autoscaler
    version: "7.2.0"
    needs:
    - kube-system/kube2iam
    values: # paths are relative to the helmfile that defines releases
    - ../helm-values/common/{{ .Release.Name }}.yaml
    - ../helm-values/{{ .Release.Labels.customer }}/{{ .Release.Name }}.yaml

  autoscaler: &autoscaler
    values: # paths are relative to the helmfile that defines releases
    - ../helm-values/common/{{ .Release.Name }}.yaml
    - ../helm-values/commonLabel1value/{{ .Release.Name }}.yaml
    - ../helm-values/commonLabel2value/{{ .Release.Name }}.yaml

It requires single templates file however ...

[mlushpenko]

@voron I am not sure I understand your example. I guess you meant to create something like this:

templates:
  _autoscaler: &_autoscaler
    name: cluster-autoscaler
    namespace: kube-system
    chart: stable/cluster-autoscaler
    version: "7.2.0"
    needs:
    - kube-system/kube2iam

  autoscalerA: &customerA_autoscaler
    <<: *_autoscaler
    values:
    - ../helm-values/common/{{ .Release.Name }}.yaml
    - ../helm-values/customerA/{{ .Release.Name }}.yaml

  autoscalerB: &customerB_autoscaler
    <<: *_autoscaler
    values:
    - ../helm-values/common/{{ .Release.Name }}.yaml
    - ../helm-values/customerB/{{ .Release.Name }}.yaml

But even if I interpreted your idea correctly, that's not what I would like because this setup would import templates for all customers if I use readFile function and wouldn't be very DRY in general.

[mlushpenko]

hi, @mumoshu, is this a big feature to implement? any chance I can expect it in the near future or is it far down the list of priorities?

[mumoshu]

@mlushpenko I can't promise anything as I'm working on Helmfile for my spare time(as I don't have any sponsor) but it should be easy/done shortly if it's just about passing some map object to commonLabels in kustomization.yaml :)

[mlushpenko]

Hi, @mumoshu, we rely more and more on helmfile in production, so I would like to give a try adding this feature if you don't have time. I have quite basic Golang skills, but if you could point me in the right direction, would be happy to implement it if I can.

[mumoshu]

@mlushpenko Will try give you a brief intro towards it!

Anyway - is forking each helm chart and modify helm templates to allow injecting labels to every resource via values.yaml is a viable option for you? I think that's the easiest way to achieve your requirement.

[mlushpenko]

This feature is not about having labels for kubernetes resources, it's about having a clean helmfile without much duplication. .Release.Labels. is helmfile specific feature to manage selection of releases, like Kubernetes selectors but for helmfile releases, right? Remember we talked about that labels should be a sparate field, not part of helmDefaults as it's not realted to the helm itself, but to helmfile templating and selection of releases

[mumoshu]

@mlushpenko Ah sorry about that! I reread your original description and now understood. What confused me was that kustomize commonLabels is for labeling resources, where this one is indeed for releases.

Re: implementation, you should copy the release labels(r.Labels), and merge HelmState.HelmDefaults.Labels to the copy and use that for the selector match target here: https://github.com/roboll/helmfile/blob/1e260e4a5ecd57a4c84cdbc98f569a52b7735bb2/pkg/state/state.go#L1447-L1455

HelmState is available in the caller here: https://github.com/roboll/helmfile/blob/1e260e4a5ecd57a4c84cdbc98f569a52b7735bb2/pkg/state/state.go#L1429

Please don't forget adding the definition of labels to HelmSpec: https://github.com/roboll/helmfile/blob/1e260e4a5ecd57a4c84cdbc98f569a52b7735bb2/pkg/state/state.go#L55

[mlushpenko]

Thanks a lot, hopefully, I'll get some time this week to give it a try.