Micrometer: Multiplied value of metric when using CompositeMeterRegistry

[rafal-dudek]

Hi,

In our application we have 2 registries: PrometheusMeterRegistry and JmxMeterRegistry.
There is created automatically CompositeMeterRegistry.

Some metrics are doubled e.g. "logback.events" (io.micrometer.core.instrument.binder.logging.LogbackMetrics) and "jvm.gc.pause" (io.micrometer.core.instrument.binder.jvm.JvmGcMetrics).

It looks like MeterBinder is binded to all 3 MeterRegistries -> counters in PrometheusMeterRegistry and JmxMeterRegistry are incremented twice.

It is similar to issue: https://github.com/micrometer-metrics/micrometer/issues/546

We are using spring-boot-actuator-autoconfigure:2.1.3.RELEASE, micrometer-registry-prometheus:1.1.3 and micrometer-registry-jmx:1.1.3.
demo.zip

[izeye]

@rafal-dudek Thanks for the report! I didn't look into JvmGcMetrics yet but confirmed LogbackMetrics has an issue with CompositeMeterRegistry. I created #1283 to try to resolve it.

For JvmGcMetrics I suspect there's the same problem if any. I'll look into it.

[izeye]

This is a general issue with Counter as all MeterBinders are bound to all MeterRegistrys including CompositeMeterRegistry in Spring Boot.

With a quick search the following MeterBinders seem to be affected:

• HystrixMetricsBinder
• JvmGcMetrics
• Log4j2Metrics
• LogbackMetrics

I can fix them like #1283 but I think we need a better fix as MeterBinder can be defined outside of this repo although I don't know how at the moment.

[izeye]

I created https://github.com/spring-projects/spring-boot/pull/16221 to try resolve this on Spring Boot side. If the PR got merged, we can close #1283.

[shakuzen]

This is a general issue with Counter as all MeterBinders are bound to all MeterRegistrys including CompositeMeterRegistry in Spring Boot.

For completeness: it is not an issue with all Counters. It is an issue when a listener is registered to update metrics in response to something happening. When two listeners for the same thing are configured (e.g. same MeterBinder that configures a listener is bound to two registries), the metric is updated twice for the same event - in the case of a counter, it is incremented twice.

[Imaskar]

I also have this problem.
Is the spring-side fix working well? I see there are doubts about it. If it works, we would need to backport it to micrometer-spring-legacy in this repository also. Is it somehow done automatically from spring boot upstream, or must be done manually?

[shakuzen]

Resolved by #1392