Gardener: Clean up empty projects after a period of inactivity
What would you like to be added:
Deletion of empty projects after an extended period of inactivity.
Why is this needed:
On our Canary landscape I see a number of projects that are empty and have no signs of activity for the last 3 weeks (at least). Furthermore, some of these projects occupy apparently "desired" names (something short like test, 1, project or similar), resulting in an increasing amount of projects with various suffixes added. Some of these projects, too, seem abandoned.
Ideally, however, we'd motivate the project creators to choose more descript project-names to avoid getting in each other's ways - and to delete them once they are not needed anymore.
AndreasBurger
All 13 comments
How would we get to know whether the projects were unused/inactive for an „extended period“?
rfranzke
on 3 Mar 2020
Just spitballing: Whenever Gardener performs any operation on a project (shoot creation, reconciliation, basically _most anything_) annotate the projects' namespace with a timestamp. Periodically check for namespaces/projects that haven't been touched in the last 90 days and delete them.
Unfortunately, I'm not versed enough with either K8s or Gardener to offer a better suggestion. I'm not even sure the one I made makes sense.
AndreasBurger
on 4 Mar 2020
As a project in Gardener doesn't make sense without using Shoots or Plants, what do you think about storing last timestamp when all shoots/plants have been deleted? If in the next ~90 days no new shoot/plant was created then we could garbage-collect the project? Maybe we can even add a warning condition to the project that warns users that/when it's gonna be cleaned up?
/cc @gardener/gardener-maintainers
rfranzke
on 15 Apr 2020
I think it makes sense.
We could enhance the project controller to annotate a project as "to-be-deleted" in case there are no more shoots/plants (probably store the timestamp in an annotation or maybe even in the status).
If we want, we can add a condition, as you proposed, to warn the user, when it's going to be cleaned up.
In the normal project reconciliation the controller can then check, if the defined period has passed and deleted the project if so.
timebertt
on 16 Apr 2020
Yep, sounds good, thanks Tim. The condition could also be shown in the Dashboard. Do you think we need the "to-be-deleted" annotation or can this be combined with only the condition?
rfranzke
on 16 Apr 2020
Hmm, I also thought about, if we could combine the two. But I think it makes sense to store the timestamp of the deletion of the last shoot/plant as a reference point for the technical calculations.
If for example we would want to prolong this "grace period" in a new release because we figured that it's too short (and apply it also to projects, that are already marked as "to-be-deleted"), then we would have to calculate back to the deletion time of the last shoot and then calculate when the project shall be deleted.
Still, we could add the time when the project will be deleted to a user-facing condition message, which most probably should be shown in the dashboard, but will not be used by the controller.
timebertt
on 16 Apr 2020
What we should also consider is the case when new projects are created for preparation. Let's suppose an operator creates some projects for customers or a training session and the projects themselves stay inactive for a few days. While it makes sense to deleted them after some time of inactivity, we probably shouldn't show a warning for self destruction right at the beginning.
timuthy
on 16 Apr 2020
For such projects we can probably go ahead and immediately write the deletion time in the status condition. The dashboard can decide when it starts to make the alert/warning visible to the end-user (maybe only after half of the time is elapsed). WDYT?
cc @gardener/dashboard-maintainers
rfranzke
on 16 Apr 2020
How about the scenario when an empty project is used to store the cloud credentials and they are referenced with secret bindings from another project?
In this case the empty project will not contain any shoot or plant anytime, just secrets and deleting the project will delete the secrets which possibly will affect the existing shoot clusters in the other project.
vpnachev
on 29 Apr 2020
Have you seen that in the wild @vpnachev? However, it is a possibility that such projects exist. Then again, if the secret is referred to by another non-orphaned project we could count that as still in use, because there is a shoot that (indirectly) refers to this project's secret and we shouldn't/even can't (?) delete it anyway. In addition, if the user can reset the project condition, it would be OK. This way, the user can extend the project lifetime.
In the end, that's all we are after: dropping projects that were dropped by their owners.
Cosmetic comment: I wouldn't call the annotation to-be-deleted, but rather orphaned, stale, or something and describe in the docs and the Dashboard, that if orphaned for too long, we will take action/delete the project.
vlerenc
on 2 May 2020
Yes, I have seen it.
Personally I have also advised several peoples to use this approach when they want only very few people to manage the cloud credentials but a lot more to use them to create shoot clusters.
vpnachev
on 2 May 2020
Then again, if the secret is referred to by another non-orphaned project we could count that as still in use
Sounds like an excellent plan.
vpnachev
on 2 May 2020
/assign
rfranzke
on 9 Jun 2020
Related issues
rfranzke
·
3Comments
rfranzke
·
3Comments
hoeltcl
·
3Comments
hendrikKahl
·
3Comments
mvladev
·
3Comments
Most helpful comment
What we should also consider is the case when new projects are created for preparation. Let's suppose an operator creates some projects for customers or a training session and the projects themselves stay inactive for a few days. While it makes sense to deleted them after some time of inactivity, we probably shouldn't show a warning for self destruction right at the beginning.