Aks-engine: Kubelet server should not use a self signed certificate

[PascalVA]

Describe the bug
By default, when deploying a new cluster, the kubelet on each node generates a self-signed certificate and uses it for the kubelet server. The kubelet certificates should be generated from the CA that issues the kube-apiserver certificates (or another CA if preferred). The --kubelet-certificate-authority flag should then be set on the apiserver flag to make sure connections are properly verified.

The user can choose to manage their own kubelet certificates to get around this, but it would be better if the default deployment included this.

Steps To Reproduce
Deploy an aks-engine cluster without specifying the --tls-cert-file and --tls-private-key-file for kubelet.

Expected behavior
Kubelet should not use a self-signed certificate because this prohibits the kube-apiserver to validate the certificate (using the --kubelet-certificate-authority flag).

AKS Engine version
0.46.3

Kubernetes version
1.17.0

[PascalVA]

When this is fixed, --kubelet-insecure-tls can also be removed from the config of metrics-server

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[hannesg]

This issue is very annoying because e.g. Datadog doesn't work out of the box. Please reconsider this bug.

[kiss-less]

@hannesg we are facing exactly the same problem with Datadog, but our AKS cluster with self-signed kubelet certificates is already up and running. Could you please clarify how did you resolve the problem? Is it mandatory to re-deploy the cluster with the kubelet-certificate-authority flag, or it can be changed on the fly somehow? Thanks in advance!