Occasionally these types of errors are logged:
i.m.s.StackdriverMeterRegistry - failed to send metrics to stackdriver: INVALID_ARGUMENT: Field timeSeries[13].points[0].distributionValue had an invalid value: Distribution bucket_counts(26) has a negative count.
I tracked it down to this line in the StackdriverMeterRegistry class:
// add the "+infinity" bucket, which does NOT have a corresponding bucket boundary
bucketCounts.add(snapshot.count() - truncatedSum.get());
Sometimes the snapshot.count() method returns 0, while bucketCounts may contain positive values resulting in a negative bucket size at the end of the bucketCounts list. I think this is a bug in the code creating the snapshots, but locally I made a workaround to ensure non-negative values:
bucketCounts.add(Math.max(snapshot.count(), truncatedSum.get()) - truncatedSum.get());
This is a side effect of #998.
Most helpful comment
This is a side effect of #998.