Kube-state-metrics: Improve (!?) documentation for enabling VerticalPodAutoscaler

[DazWilkin]

What happened:

It took me more time than I would have liked to understand how to enable VerticalPodAutoscaler metrics.

The documentation hints at the solution but would benefit from an additional explanation for running kube-state-metrics with --resource=${RESOURCES},verticalpodautoscalers.

What you expected to happen:

A more detailed explanation of what's needed and how to enable this with e.g. Prometheus Operator.

Perhaps back-filling issues with VPA and the Prometheus Operator to provide guidance on this too.

It's very helpful just confusing to resolve.

How to reproduce it (as minimally and precisely as possible):

I'm running MicroK8s 1.20 and microk8s enable prometheus and have VPA installed:

kubectl get crd/verticalpodautoscalers.autoscaling.k8s.io \
--output=jsonpath="{.spec.versions[*].name}"
v1beta1 v1beta2 v1

After cloning this repo and deploying ./examples/standard, I realized that the Prometheus Operator bundles kube-state-metrics but then realized that it does not enable verticalpodautoscalers (by default).

kubectl logs deployment/kube-state-metrics \
--namespace=monitoring \
--container=kube-state-metrics

...
Active resources:
  certificatesigningrequests,configmaps,cronjobs,
  daemonsets,deployments,endpoints,horizontalpodautoscalers,
  ingresses,jobs,leases,limitranges,mutatingwebhookconfigurations,
  namespaces,networkpolicies,nodes,persistentvolumeclaims,
  persistentvolumes,poddisruptionbudgets,pods,replicasets,
  replicationcontrollers,resourcequotas,secrets,services,
  statefulsets,storageclasses,validatingwebhookconfigurations,
  volumeattachments
...

and:

kubectl get deployment/kube-state-metrics \
--namespace=monitoring \
--output=jsonpath="{.spec.template.spec.containers[?(@.name=='kube-state-metrics')]}" \
| jq .

Yields:

{
  "args": [
    "--host=127.0.0.1",
    "--port=8081",
    "--telemetry-host=127.0.0.1",
    "--telemetry-port=8082",
  ],
  "image": ".../kube-state-metrics:v1.9.7",
  "imagePullPolicy": "IfNotPresent",
  "name": "kube-state-metrics",
  "resources": {},
  "terminationMessagePath": "/dev/termination-log",
  "terminationMessagePolicy": "File"
}

I don't know how to configure the metrics correctlypermanently but I managed to tweak it:

RESOURCES="certificatesigningrequests,configmaps,cronjobs,"\
"daemonsets,deployments,endpoints,horizontalpodautoscalers,"\
"ingresses,jobs,leases,limitranges,mutatingwebhookconfigurations,"\
"namespaces,networkpolicies,nodes,persistentvolumeclaims,"\
"persistentvolumes,poddisruptionbudgets,pods,replicasets,"\
"replicationcontrollers,resourcequotas,secrets,services,"\
"statefulsets,storageclasses,validatingwebhookconfigurations,"\
"verticalpodautoscalers,volumeattachments"

PATCH="[{"\
"'op': 'add',"\
"'path':'/spec/template/spec/containers/0/args/-',"\
"'value':\"--resources=${RESOURCES}\"}]"


kubectl patch deployment/kube-state-metrics \
--namespace=monitoring \
--type=json \
--patch="${PATCH}" 

Anything else we need to know?:

Hopefully others who struggle with this will find this issue.

I blogged about the experience here

Environment:

• kube-state-metrics version:
• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• Other info:

[lilic]

Hello 👋 would be great to contribute this to our docs! Feel free to open a PR to help others, folks tend to not look at issues as much. :)

[lilic]

As for

A more detailed explanation of what's needed and how to enable this with e.g. Prometheus Operator.

Maybe it would better fit into the https://github.com/prometheus-operator/kube-prometheus/ docs?