Aad-pod-identity: Node Managed Identity (NMI) daemonset are not getting scheduled in a multi node pool cluster.

[gowrishankar0013]

Node Managed Identity (NMI) daemonset are not getting scheduled in a multi node pool cluster.
Create a Kubernetes Cluster with a single node pool and node count of 5. Later we added another node pool with node count of 1 and spot node pool with node count of 1.
It is observed that NMI daemonset got created only for the first node pool, that is 5 nmi pods got created. For the other 2 node pools it failed to create.

Steps To Reproduce

1. Created a Kubernetes Cluster with a single node pool and node count of 5.
2. Add another node pool with a node count of 1.
3. Add another spot node pool with node count of 1.
4. helm install aad-pod-identity aad-pod-identity/aad-pod-identity
5. It is observed that 5 daemonset nmi got created on the first node pool(default) as the node count was 5.

Expected behavior
NMI daemonset were supposed to get created on other two nodes. But it failed to create.

AAD Pod Identity version
1.6.3

Kubernetes version
v1.17.11

Additional context
@aramase @dhananjaypatkar

[aramase]

@gowrishankar0013 If the new node pools are tainted, then you need to add the tolerations for that taint in the NMI daemonset. PTAL at this configuration table for setting tolerations using helm - https://github.com/Azure/aad-pod-identity/tree/master/charts/aad-pod-identity#configuration

[gowrishankar0013]

Adding the tolerations on NMI daemonset resolved the issue.

[aramase]

Thank you for confirming. Closing the issue as it's now resolved. Please feel free to open a new issue if you've any questions.