Cert-manager: Add Prometheus metrics endpoint

[munnerz]

We should expose a metrics endpoint in cert-manager that can be scraped by monitoring software such as Prometheus.

This is an overall tracking issue that'll probably be broken down into a number of smaller issues through the implementation of this.

We'll need to take special consideration to avoid storing all the metric data in memory, as that'd cause an instance restart to lose information.

It may still be valuable to expose metrics for just the current instance however, as that could be a useful graph to see 😄

/cc @ahmetb
/area monitoring
/kind feature

[munnerz]

73 has added the beginnings of a prometheus endpoint. It currently just exposes the standard prometheus metrics so will need extending to add custom metrics.

[retest-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[munnerz]

/remove-lifecycle stale

[grrywlsn]

Is the idea to have a metric for each certificate? It'd be great to have a way of finding out a total of certificates, and a total number of certificates that are having errors being issued/renewed.

[whereisaaron]

A note to remember when adding the metrics to the helm chart to add a 'headless' Service (clusterIP: none) for metrics discovery. And it should allow both custom annotations and custom labels (old pre-CRD prometheus used scrape annotations, prometheus-operator uses label selectors from the ServiceMonitor CRD).

You see people add metrics as an extra port on the application's Service load balancer. But load balancing your metrics collection usually makes no sense, since you want the metrics from all replicas/endpoints, not just a random one.

[jurgenweber]

it would be super awesome to get this rls'd.....

[primeroz]

really looking forward to this, right now i only manage to monitor expiration date from external with blackbox exporter but this would be a great addition