Opentelemetry-dotnet: Implement Observer instrument and corresponding aggregation.

[cijothomas]

We have an instrument named Observer already, but it does not have any 'callback' mechanisms implemented. This issue is to track implementing an Observer as per the specs.

https://github.com/open-telemetry/oteps/blob/master/text/0072-metric-observer.md

[cijothomas]

Rough usage will look like below

// create an observer. This could be alternately named RegisterInt64Observer as well.
// Unlike other instruments, this returns nothing, as there is no sync call to observe a metric.
// values are observed via callbacks.
meter.CreateInt64Observer("memoryWorkingSet", MemorySizeObserver);

// This method gets called during Collect() phase on Metric SDK.
// Values are aggregated using LastValue aggregator, which simply keeps the last value observed
private void MemorySizeObserver(Int64ObserverMetric observerMetric)
        {
            var labels2 = new List<KeyValuePair<string, string>>();
            labels2.Add(new KeyValuePair<string, string>("dim1", "value1"));

            var mem = Process.GetCurrentProcess().WorkingSet64;
            observerMetric.Observe(mem, labels2);
        }

[MikeGoldsmith]

Looks good. I think Create is more accurate as we're creating the bound instrument, where I think register would take the meter as a parameter.

[SergeyKanzhelev]

So shorter version of passing callback will be something like this:

meter.CreateInt64Observer("memoryWorkingSet", (observerMetric) => {
            var labels2 = new List<KeyValuePair<string, string>>();
            labels2.Add(new KeyValuePair<string, string>("dim1", "value1"));

            var mem = Process.GetCurrentProcess().WorkingSet64;
            observerMetric.Observe(mem, labels2);
        });       

When implementing this for things like EventCounters or such - will this allow to observer multiple observations? How the interface will look like?

[cijothomas]

@SergeyKanzhelev For observing multiple metric (i.e separate instruments) in one go, there is a different API. Its Meter.RecordBatch, which is not implemented in .NET yet.
https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/api-metrics-user.md#recordbatch-calling-convention

On the other hand if reporting CPU_Usage per core (uses single instrument, but different dimensions as coreNumber is a dimension), then we do something like this

meter.CreateInt64Observer("cpuUsageByCore", (observerMetric) => {
            var labelsCore1 = new List<KeyValuePair<string, string>>();
            labelsCore1.Add(new KeyValuePair<string, string>("core", "1"));
            var cpu = Process.GetCurrentProcess().GetCpuByCore(core:1);
            observerMetric.Observe(cpu, labelsCore1);

            var labelsCore2 = new List<KeyValuePair<string, string>>();
            labelsCore1.Add(new KeyValuePair<string, string>("core", "2"));
            var cpu = Process.GetCurrentProcess().GetCpuByCore(core:2);
            observerMetric.Observe(cpu, labelsCore2);
        });