Test-infra: Allow disabling CI for WIP PRs
What would you like to be added:
A way to disable CI for WIP PRs.
Why is this needed:
I've noticed in several WIP PRs (including my own) that there is a lot of tests being run without any use to it. Not only do they spam notifications but they also waste resources.
Additionally it would make reviewing and discussing a lot easier as one doesn't have to scroll over a huge amount of CI comments.
To fix this for one of my current PRs i tried to set the not-ok-to-test label, which didn't work.
Edit: Just to clarify, I'm not asking for CI to be generally disabled for WIP PRs, but to allow disabling trough a label in cases where author /participants see fit.
kwiesmueller
All 8 comments
kwiesmueller
on 11 Apr 2020
I've noticed in several WIP PRs (including my own) that there is a lot of tests being run without any use to it. Not only do they spam notifications but they also waste resources.
IMHO if there is no use to it is debatable. WIP doesn't necessarily mean "is broken right now" it could also just mean "its not done but the part that is done does work".
alvaroaleman
on 11 Apr 2020
I don't claim there is never a use for testing WIP PRs. Just that the possibility to disable CI would be helpful for some PRs.
As an example, a current PR of mine is pinning a dependency to a fork of mine until the respective PR there merges.
The CI fails with every push and sends a load of comments that are of no use.
The PR is in a state where the implementation itself is in progress and being discussed, but between the actual discussion there is failing tests that nobody will care about for some more time.
Adding a label that disables tests just like the ok-to-test thing works, would be a way to solve this.
kwiesmueller
on 11 Apr 2020
Alvaro how about not auto-triggering tests for draft PRs? Can still trigger with a /test all comment
fejta
on 22 Apr 2020
Alvaro how about not auto-triggering tests for draft PRs? Can still trigger with a /test all comment
Doing this for draft PRs SGTM
alvaroaleman
on 22 Apr 2020
@kwiesmueller the code for creating jobs from github events is in prow/plugins/trigger so you'd have to extend that to not create jobs on push event if the PR is a draft PR.
alvaroaleman
on 22 Apr 2020
I'll look at it, thanks!
kwiesmueller
on 22 Apr 2020
@alvaroaleman @fejta thanks for reviewing.
Do you want to announce this change in Kubernetes-dev or somewhere else?
I suppose it could be useful for more people to know about it.
How is the usual flow for changes like this.
kwiesmueller
on 9 May 2020
Related issues
Aisuko
路
3Comments
benmoss
路
3Comments
MrHohn
路
4Comments
lavalamp
路
3Comments
xiangpengzhao
路
3Comments
Most helpful comment
I don't claim there is never a use for testing WIP PRs. Just that the possibility to disable CI would be helpful for some PRs.
As an example, a current PR of mine is pinning a dependency to a fork of mine until the respective PR there merges.
The CI fails with every push and sends a load of comments that are of no use.
The PR is in a state where the implementation itself is in progress and being discussed, but between the actual discussion there is failing tests that nobody will care about for some more time.
Adding a label that disables tests just like the ok-to-test thing works, would be a way to solve this.