Kube-prometheus: Grafana overwrite every dashboard on deploy

[jalberto]

I am trying to use grafana helm deployed by kube-prometheus with a DB to back grafana state. In each deploy every dashboard is overwritten.

I understand the intention of this grafana deploy is to have a stateless service with config and dahbaord pragmatically deployed but this can cause lost of data in particular with WIP dashboards or very complex ones.

My suggestion is: Create a "kube-prometheus" folder in grafana, and keep in it every kube-prometheus related dashboard, and in each deploy only overwrite this folder (or other defined folders)

[brancz]

My suggestion is: Create a "kube-prometheus" folder in grafana, and keep in it every kube-prometheus related dashboard, and in each deploy only overwrite this folder (or other defined folders)

I think this is rather something you need to discuss with the Grafana folks, as I think that's just not how the provisioning feature of Grafana works.

[jalberto]

Actually you can set a folder and a organization in the provider:

http://docs.grafana.org/administration/provisioning/#dashboards

[brancz]

Right I meant how should Grafana know to only delete/overwrite those dashboards instead of all. If the folder solves things I'm more than happy with going for that. If I recall correctly @metalmatze wanted to organize the dashboards in folders anyways.

[jalberto]

I can think of a series of folders:

• foo (kube-prometheus)
• bar (kube-prometheus)

So on dos it's mention those folder are overwritten each time.

Another option is to use tags, with similar approach

[brancz]

I'm not exactly sure I understand, but if you can find a way that works for all users I'm happy with accepting that.

[jalberto]

So the idea will be dashbaord being provided by grafana provisioner like this:

apiVersion: 1

providers:
- name: 'kube-prometheus'
  orgId: 1
  folder: 'kube-prometheus_dont-touch'
  type: file
  disableDeletion: false
  updateIntervalSeconds: 3
  options:
    path: /var/lib/grafana/dashboards

That way only dashboards in that folder will be overwritten

[brancz]

If that works I’m more than happy with that approach.

[kskewes]

Be great to be able to specify Grafana dashboard folder when specifying dashboard to load in the following block somewhere/somehow.

  grafanaDashboards+:: {
    'my-dashboard.json': (import 'example-grafana-dashboard.json'),
  },

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity in last 60d. Thank you for your contributions.

[paulfantom]

Closing due to staleness. Please reopen if you think this is still relevant.