Cadvisor: container_memory_usage_bytes prometheus metrics help doesn't specify what memory we are measuring

[octete]

As per https://github.com/google/cadvisor/issues/975, two more stats were added: RSS + cache, as container_memory_usage_bytes, which seems to come from cgroup stats includes resident memory, caches and swap.

Could we please modify the definition of it in metrics/prometheus.go so that it specify what the differences of these metrics are? (specifically as compared to container_memory_rss and container_memory_cache)

            }, {
                name:      "container_memory_usage_bytes",
                help:      "Current memory usage in bytes.",
                valueType: prometheus.GaugeValue,
                getValues: func(s *info.ContainerStats) metricValues {
                    return metricValues{{value: float64(s.Memory.Usage)}}
                },

as it is not very clear from the helper message?

If you prefer, I can send a PR, but I thought that maybe as this is just a simple help message change, it could be quicker this way.

If you think this is fine and you want the PR, please, let me know and I will send you one :-)

Thanks

[dashpole]

Thanks for pointing this out.

[dashpole]

Can you make sure #1759 is what you were looking for?

[ylfforme]

Thanks for pointing this out.

can I understand like this？ ”container_memory_usage_bytes” is rss + cache + swap
“container_memory_rss” is just rss not include cache and swap

[jstangroome]

@ylfforme

container_memory_usage_bytes == container_memory_rss + container_memory_cache + container_memory_swap + kernel memory.

But kernel memory is not exposed as a metric yet, see #2138

[thomasf1234]

@jstangroome
Might be my lack of understanding but do you how container_memory_mapped_file integrates with this?

The docs appear to say that the rss field under memory.stat is not the true RSS for the cgroup (https://github.com/torvalds/linux/blob/v4.15/Documentation/cgroup-v1/memory.txt#L524-L530).

Does this imply the following relationship?

container_memory_usage_bytes == container_memory_rss + container_memory_mapped_file + container_memory_cache + container_memory_kernel.

Note: I removed container_memory_swap because as far as I can see in the cAdvisor source code, this is using opencontainer/runc to pull the value from memory.usage_in_bytes (not memory.memswp.usage_in_bytes) (https://github.com/google/cadvisor/blob/master/container/libcontainer/handler.go#L35 + https://github.com/opencontainers/runc/blob/51f2a861da7fb1d4f6f8370a60258c6f315956ef/libcontainer/cgroups/fs/memory.go#L161)

Although upon analysis of the corresponding metric values, the following relationship appears correct just not sure the role of mapped_file (perhaps a subset of cache?):

container_memory_usage_bytes == container_memory_rss + container_memory_cache + container_memory_kernel

[dongwangdw]

@ylfforme

container_memory_usage_bytes == container_memory_rss + container_memory_cache + container_memory_swap + kernel memory.

But kernel memory is not exposed as a metric yet, see #2138

Why this metrics includes so much, it has a different result as kubectl top.