Kube-prometheus: Need pod labels in kube-state-metrics

[kmurthyms]

What did you do?

I have a similar issue as mentioned in https://github.com/kubernetes/kube-state-metrics/issues/536 and
https://github.com/prometheus-operator/prometheus-operator/issues/3460
I need all the pod labels to be present in the metrics , so that based on the label i could have the alert routing configured

We have the standrad set of kubenertes-state-metrics rules deployed and i don't want to modify them to add join mertrics . since this is a standard set of rules that we pull from community(https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) and modifying this would add maintenance overhead every time we update.

so is there a way to get all the pod labels without modifying the rules ?

• Prometheus Operator version:

prometheus-operator:
    Image:      quay.io/coreos/prometheus-operator:v0.38.1

`Insert image tag or Git SHA here`
<!-- Try: kubectl -n monitoring describe deployment prometheus-operator -->

• Kubernetes version information:

  kubectl version
  
  Client Version: v1.19.3
  Server Version: v1.15.12-eks-31566f

[lilic]

Hello 👋 when you mean pod labels, do you mean the kube_pod_labels metrics or labels on other pod metrics?

[kmurthyms]

Hello , I mean the labels on the pod
What i am trying to achieve is to have generic implementation for an alertmanager route for my kubernetes workloads
Example: I have a label on every pod/service/rs "owner=" , with which i want have an alert manager route option to route alerts based on team_name . with kube-state-metrics , when the KubePodCrashing alert fires up , i want to fire notification to the corresponding team based on this label , to achieve this i need the pod/service labels. the alert Rules are standard rules that we borrow from community and i don't want to modify these to rules to add join metrics on kube_pod_labels or kube_service_labels (https://github.com/kubernetes/kube-state-metrics/issues/536#issuecomment-420203467). is there any way in which i could pass a label extracted from kube_<workload>_labels to other metrics ?

[paulfantom]

Seems like something that could be done with kube-state-metrics 2.0 and applied as a jsonnet patch/addon to the project. Since this is a pretty frequent request I am adding a feature label.

[kmurthyms]

@paulfantom , would like to know is this feature included in kube-state-metrics 2.0 ? i explored the docs and couldn't figure out a way, can you let me know if i can achieve this ?