Gardener: Seed Grafana failed to load Extensions dashboard

[timuthy]

How to categorize this issue?

/area monitoring
/kind bug
/priority normal

What happened:
Seed Grafana failed to load "Extensions" dashboard (see screenshot). I was able to reproduce it on multiple seed clusters. It worked occasionally when I reloaded the page several times.

How to reproduce it (as minimally and precisely as possible):

1. Open Seed Grafana via grafana Ingress in garden namespace.
2. Click on Extensions dashboard.

/cc @wyb1

Environment:

• Gardener version: v1.12.3

[wyb1]

Seems to be related to the datasource. Whenever the datasource is loki, the CPU and Memory usage panels are broken.

Whenever the datasource is changed to prometheus and the label is adjusted then the CPU and memory panels work but the logs from loki no longer exist.

@Kristian-ZH any suggestions on how to get loki working with the Prometheus labels?

[rfranzke]

ping @Kristian-ZH @vlvasilev

[Kristian-ZH]

Hmm i checked and can not see any issues at the moment. We had an issue before, where the Loki was constantly scaled-up/down because of the misconfigured hvpa. This issue was fixed and currently the dashboards are looking fine.
Can you please recheck if everything is working as expected @timuthy @wyb1 ?

[timuthy]

I don't experience the issue at the moment, either. Thanks for checking @Kristian-ZH.
/close