Apm-agent-dotnet: No metrics from GcMetricsProvider

[vhatsura]

Describe the bug

After update agent to the 1.3.0 version, no GC metrics are reported from application.
From logs perspective we have the following log messages related to GcMetricsProvider:

{"MetricsCollector"} Failed reading "GcMetricsProvider" 4 times: no valid samples

{"MetricsCollector"} Failed reading "GcMetricsProvider" 5 consecutively - the agent won't try reading 
 GcMetricsProvider" anymore

To Reproduce
Steps to reproduce the behavior:

1. Default configuration
2. Then call '....'
3. Then do '....'
4. See error

Expected behavior

GC metrics should be collected and reported to the APM server.

It seems that the reason for the issue in filtering mechanism of invalid metrics values. If service isn't allocated a lot of memory the GC won't be triggered for a lot of time and in this case, GCMetricsProvider will not report any values and MetricsCollector after 5 tries disable provider.
As a possible solution, IMetricsProvider response contract needs to be extended to figure out does metric collection has occurred or no. Or, we need to think about retrying mechanism after a certain time in disable mode.

[nicolasesprit]

Same error for others metrics :

• {MetricsCollector} Failed reading total system CPU time 4 times: no valid samples
• {MetricsCollector} Failed reading total and free memory 4 times: no valid samples
• {MetricsCollector} Failed reading GcMetricsProvider 4 times: no valid samples

Nuget 1.3.0
.netcore 3.1
MacOS

[gregkalapos]

Thanks for opening this.

As a possible solution, IMetricsProvider response contract needs to be extended to figure out does metric collection has occurred or no.

I agree on this, this sounds like a good option. I plan to implement this,

@nicolasesprit

Total CPU and free/total memory are not supported on macOS - you get those values on Windows and Linux, but not on macOS, so those are probably not related to the bug described here.

We had some discussion for collecting those values for macOS - that's also stated in the docs. There was some code in a PR, but it was unreliable, so we removed that - it's hard to get those on macOS. I assume no one really uses macOS in prod, and for development these values are more like nice to have and can be collected easily with other tools.

But happy to take feedback - we can reiterate this part and maybe make it work for macOS - but that should be outside this issue; I suggest we focus on the part that makes IMetricsProvider know about whether a metric was potentially already collected and solve the GC metrics issue with that.

[nicolasesprit]

@gregkalapos Thanks for your reply and precisions 👍

Indeed, I do not use macOS in prod ! I'm on MacBook Pro for development purpose only and i have tested ELK Stack via Docker. So this specific case is absolutely not a priority or important. We both agree 😄

This issue therefore remains focused on GcMetricsProvider.

[gregkalapos]

Solves in #745.