Aks-engine: Expose calico metrics

[javierprovecho]

Describe the request

Expose calico metrics so they can be scraped by Prometheus.

Explain why AKS Engine needs it

Allowing metrics from some components like network (calico) or storage (etcd, see #637) to be scraped by Prometheus would benefit observability.

Describe the solution you'd like

Add a new property under kubernetesConfig called networkPolicyConfig which follows map[string]string where key is and env var name and value is env var value.

Depending on current and future network policy drivers, this map may reference env vars (like calico) or flags.

Describe alternatives you've considered

-

Additional context

-

[welcome[bot]]

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it.

[palmerabollo]

Perhaps the FELIX_PROMETHEUSMETRICSENABLED var could be set to "true" at https://github.com/Azure/aks-engine/blob/master/parts/k8s/containeraddons/kubernetesmasteraddons-calico-daemonset.yaml#L553 by default to enable the metrics in the short term.

[song-jiang]

@javierprovecho You may create a GlobalFelixConfiguration to enable Prometheus right after your cluster has been created. Please refer to https://docs.projectcalico.org/v3.8/reference/felix/configuration

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[javierprovecho]

Thanks @song-jiang, we created the following object after cluster bootstrap:

apiVersion: crd.projectcalico.org/v1
kind: FelixConfiguration
metadata:
  name: default
spec:
  prometheusMetricsEnabled: true