Linkerd2: linkerd command to get control plane metrics

[adleong]

The linkerd metrics command allows fetching the proxy metrics from any injected pod (including pods in the control plane) but there is not a corresponding way to get metrics directly from the control plane containers, such as the destination controller.

This can be done manually like so:

kubectl -n linkerd port-forward deploy/linkerd-controller 9996 &
curl localhost:9996/metrics

but it would be nice to have a single linkered subcommand for this.

[dr0pdb]

@grampelberg Hi, can I take this as my first issue?

[dr0pdb]

@adleong @grampelberg Hi,
I went through the source code of the metrics command of the CLI and I have a doubt regarding this issue.

The documentation of the command suggests that we can get the metrics from the linkerd-controller pod using

linkerd metrics -n linkerd $(
    kubectl --namespace linkerd get pod \
      --selector linkerd.io/control-plane-component=controller \
      --output name
  )

How is this different from the subcommand this issue suggests to implement?

[grampelberg]

@srv-twry different metrics endpoints. linkerd metrics gets the proxy prometheus metrics but does not currently get the controller metrics. Each controller process also exports its own prometheus metrics and should be part of the linkerd metrics output.

[grampelberg]

This shouldn't be lumped into the metrics command and instead should be its own. As the metrics command can hit both control plane and data plane proxies, this should be scoped just to control plane data. My proposal:

linkerd diagnostics

This would dump the metrics for the entire control plane's containers at once (not the proxies) with a comment before each section that explains where the metrics are from:

#
# POD linkerd-controller-7549c4db6-227n6 (x of y)
# CONTAINER public-api (x of y)
#

[mayankshah1607]

Is this issue still being worked on? I'd be happy to take up this one :) @adleong @srv-twry @grampelberg

[dr0pdb]

@mayankshah1607 Feel free to take this up. You can also go through my PR to get some pointers.