Charts: How to add additional prometheus rules?

Created on 19 Mar 2019  路  4Comments  路  Source: helm/charts

Is this a request for help?:
Yes

Version of Helm and Kubernetes:
kubernetes client version: v1.12.3
kubernetes server version: v1.12.3
helm client version: v2.12.2+g7d2b0c7

Which chart:
stable/prometheus-operator:4.3.3

What happened:
Error while upgrading with newer values:

Command returned with exit code: . And error message: Error: UPGRADE FAILED:
 failed to create resource: PrometheusRule.monitoring.coreos.com "prometheus-operator-"
 is invalid: metadata.name: Invalid value: "prometheus-operator-": a DNS-1123 subdomain
 must consist of lower case alphanumeric characters, '-' or '.', and must start and end with
 an alphanumeric character
 (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

ERROR: Job failed: command terminated with exit code 1

What you expected to happen:
I expect the additionalPrometheusRules to be created correctly without an error.

How to reproduce it (as minimally and precisely as possible):
Use this in your values.yaml:

additionalPrometheusRules:
  - groups:
    - name: elastic.rules
      rules:
      - alert: ElasticsearchDiskWarning
        annotations:
        description: Elastic Search disk full Warning
        summary: Elastic Search disk full Warning
        expr: 100 - es_fs_total_free_bytes{pod=~".*data.*"} / es_fs_total_total_bytes{pod=~".*data.*"}  * 100 > 80
        for: 1m
        labels:
          severity: warning
    - name: onesearch.rules
      rules:
      - alert: OneSearchClusterWarn
        annotations:
        description: The cluster {{ $labels.namespace }}.{{ $labels.cluster }} is yellow. Please check!
        summary: OneSearch cluster warning
        expr: es_cluster_status == 1
        for: 1m
        labels:
          severity: warn

Most helpful comment

The issue at the chart level is that every one of these prometheusrule objects needs to get its own name when it is provisioned, that's why the extra - name: is needed

All 4 comments

Your format for the value of additionalPrometheusRules doesn't look right to me:
https://github.com/helm/charts/blob/master/stable/prometheus-operator/values.yaml#L47

Yes, you are right. I was looking at the documentation here: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#prometheusrulespec. It only says groups there. Thanks!

The issue at the chart level is that every one of these prometheusrule objects needs to get its own name when it is provisioned, that's why the extra - name: is needed

Hi Guys,

I am adding the following configuration that I am passing to prometheus-operator chart:

additionalPrometheusRules:
    - name: example
        groups:
        - name: mygroup
            rules:
            - record: ContainerRestarted
            expr: count(sum by (pod)(delta(kube_pod_container_status_restarts_total[15m]) > 0))
            labels:
                severity: warning
            annotations:
                summary: "Container {{ $labels.container }} in Pod {{$labels.namespace}}/{{$labels.pod}} restarting"

When accessing Prometheus UI and searching for the alert it doesn't exist, also I am not able to find this alert in the configuration files physically when going into the Prometheus container.
The Prometheus configuration is being loaded successfully - there are no errors.
Any ideas?

Was this page helpful?
0 / 5 - 0 ratings