Helm-operator: Don't require values.yaml key in secret, pull in values directly

[chainlink]

The feature
Having to encode the entire contents of the values.yaml file a a string in a k8s secret or config map makes it difficult to work with or change. It would be better if you could just specify the secret or config map and the helm operator pulls all the keys instead.

Before

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  values.yaml: YWRtaW4=

After

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  username: YWRtaW4=
  password: MWYyZDFlMmU2N2Rm

Nested values can be handled with dot notation

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  db.username: YWRtaW4=
  db.password: MWYyZDFlMmU2N2Rm

New user story
Precondition: A helm chart with values stored in a configmap or secret

1. Store values directly in the configmap or secret as above
2. When the helm release is created, these values are merged with any values specified directly in the helm release.

It looks like the load happens here: https://github.com/fluxcd/helm-operator/blob/801c64c5fde1e0aa228e5012f2f9514311bff6a9/pkg/release/release.go#L388
I'm happy to look into creating a PR if this looks like a change you would support.

[chainlink]

Any thoughts here?

[zlangbert]

I would love to see this as well. For projects that dynamically create secrets based on some external source, it's usually very difficult if not impossible to embed all values under one key.

[josdotso]

(maybe) Try stringData instead https://kubernetes.io/docs/concepts/configuration/secret/#creating-a-secret-manually

[chainlink]

@josdotso can you explain? That seems to still involve embedding in values.yaml

[jantoebes]

I'm happy to look into creating a PR if this looks like a change you would support.

@chainlink I really think this would make sense 👍 . See the comments in #184 of the limitations we encounter at the moment with secretRefFile

[jeff-minard-ck]

Just fyi, it can be a bit easier to manage with:

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
stringData:
  values.yaml: |-
    username: blah
    password: blah

_edit_: derp, I see josdotso already recommended this...

[sgandon]

here is another use case,
We are using the operator to deploy charts in a multitenancy cluster so each namespace should have a secret to access a private registry. I was hoping to be able to copy the secret from a known namespace using the secretKeyRef but this is impossible.
This feature would be really helpfull for such use case.

[mrobinson513]

I'd like to contribute another use case:

We make use of Kubernetes External Secrets for retrieving values from a service such as AWS System Manager (Parameter Store). Occasionally there are values shared with apps on platforms outside of our K8S cluster, such as database passwords, API keys, that sort of thing.

In order to use secretKeyRef as documented, we have to store a secret parameter as a YAML formatted key/value pair, or a bunch of them. Another workaround has been to clone the 3rd party charts into our own ecosystem and create a modify the templates to create the external secret, then call it accordingly.

We'd greatly like to move away from either of these models. If anyone has experience with using Kubernetes External Secrets with secretKeyRef I'd be delighted to get your insights and/or clever hack-arounds 😅