Dapr: Dapr sidecar not injected on subsequent pod creations

[ryan-stinson-elembio]

/area operator

What version of Dapr?

"starting Dapr Runtime -- version 0.7.1 -- commit 16838e8"

Expected Behavior

When deploying a dapr enabled app, the sidecar is injected into my app pod correctly and if the pod is deleted, evicted or fails in some way then the pod is rescheduled and the dapr sidecar is injected into the newly scheduled pod.

Actual Behavior

The dapr sidecar is not injected into the rescheduled pod.

Steps to Reproduce the Problem

1. Deploy dapr enabled app to kubernetes.
2. Ensure the pod is running with dapr sidecar.
3. Delete the pod.

Expected: Pod is rescheduled and dapr sidecar is injected.
Actual: Pod is rescheduled but dapr sidecar is not injected.

[ryan-stinson-elembio]

Actually, it seems that the sidecar wasn't injected due to a certificate expiry that showed in the logs:

time="2020-05-27T23:32:00.385447Z" level=info msg="ready to write response ..." instance=dapr-sidecar-injector-7bc488df76-7b9tq scope=dapr.injector type=log ver=0.7.1 2020/05/29 01:55:25 http: TLS handshake error from 10.42.0.1:52692: remote error: tls: bad certificate 2020/05/29 03:28:16 http: TLS handshake error from 10.42.0.1:55667: remote error: tls: bad certificate 2020/05/29 03:29:58 http: TLS handshake error from 10.42.0.1:1371: remote error: tls: bad certificate

[PeterGerrard]

From our testing it looks like the helm chart causes certain values to change, but not others.

My best guess is the cabundle in the webhook updates every helm release. But the tls key and cert used to host the server in the injector are still the old ones as they are read on server start, and never updated

[PeterGerrard]

We've worked around it by running kubectl rollout restart deployment dapr-sidecar-injector after helm upgrades

[artursouza]

Milestone 0.10 will have multiple instances for Dapr controllers.

[wcs1only]

Solved via docs