Gardener: Seed deletion leaves workload in garden namespace

[timebertt]

What happened:
If you delete a Seed object, the garden namespace and some workload in it will not be removed from the cluster.

After Seed deletion:

$ k -n garden get pod
NAME                                         READY   STATUS              RESTARTS   AGE
aggregate-prometheus-0                       0/2     ContainerCreating   0          1s
daily-curator-1574143500-q4ffd               0/1     Completed           0          95m
elasticsearch-logging-0                      2/2     Running             0          18h
fluent-bit-j5wkl                             1/1     Running             0          18h
fluent-bit-xsflg                             1/1     Running             0          18h
fluentd-es-0                                 1/1     Running             2          18h
gardener-resource-manager-847485d776-8rffj   1/1     Running             0          10s
grafana-6cdcc5cf68-nnmt9                     1/1     Running             0          18h
hourly-curator-1574146800-ht4jl              0/1     Completed           0          40m
hvpa-controller-6c5fccf85-v8pkb              1/1     Running             0          18h
kibana-logging-74f4bf4474-8n4t5              2/2     Running             0          18h
prometheus-0                                 0/2     ContainerCreating   0          1s
reserve-excess-capacity-5cf46749-488cg       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-75f72       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-cnndr       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-gqznh       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-hqbwl       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-khr92       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-lm8l7       1/1     Running             0          18h
reserve-excess-capacity-5cf46749-twr2f       1/1     Running             0          18h
vpa-exporter-5c9cfddf47-wvpq8                1/1     Running             0          10s
vpa-recommender-7f9fbbfdbb-pb4t9             1/1     Running             0          18h
vpa-updater-84d885687-dstz8                  1/1     Running             0          18h

What you expected to happen:
Gardener should clean up all the added resources after the Seed was deregistered.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Gardener version:
• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• Others:

[timebertt]

I guess, we should check this issue after the gardenlet has been merged (#1601 )

/status on-hold

[rfranzke]

Should be fixed by switching to using the central GRM to deploy the seed bootstrapping chart.

[rfranzke]

/remove lifecycle/stale

[timebertt]

We agreed on not implementing this via a ManagedResource for the whole seed-bootstrap chart and rather via proper individual component implementations including a WaitCleanup function like in this example.

/duplicate #2754

[timebertt]

/close