Charts: [stable/prometheus-operator] good way to override default alerts

[thomasboni]

Is your feature request related to a problem? Please describe.
I need to modify default alerts.
For example some alerts are declared in file templates/prometheus/rules/kubernetes-system.yaml. There are two alerts KubeAPILatencyHigh one with 1s limit, another with 4s limit. I need to modify these alerts but I didn't find a proper way to do it (without forking the entire chart...)

Describe the solution you'd like
Possibility to override default alerts (and maybe any kind of default data) from values.

[firaxis]

Hi, as a fast way, you can try to disable some set of default rules, e.g. set defaultRules.rules.kubernetesSystem=false
and deploy rules from https://github.com/helm/charts/blob/master/stable/prometheus-operator/templates/prometheus/rules/kubernetes-system.yaml separately with modified values.
It's a bit easier than forking a whole chart.

[thomasboni]

Hello @firaxis , it's indeed a better way than forking the whole chart, I will use it as a workaround. Thanks !

But IMHO the possibility to override default alerts from values.yaml file is a cleaner way to do it and will be really useful for many users.

[vsliouniaev]

The chart syncs alerts from upstream projects, namely etcd and coreos/kube-prometheus. Since these can arbitrarily change, supporting templating all aspects of these in the values.yaml file would make the file gigantic and difficult to maintain and script.

In our case, we have a separate helm chart that allows deployment of alerts through a gitops process, where multiple people can contribute. I think that this chart is the wrong place to expand to support more complex use-cases such as this

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.