Test-infra: Add /hold-for command and functionality to prow

[RiRa12621]

What would you like to be added:
/hold-for command and functionality

Why is this needed:
This should allow to make depending pull requests, which is a functionality missing currently but also from GitHub itself.

An example implementation would roughly work the following:

• PR 1 is opened
• PR 2 is opened which is for example logically related to PR1
• Reviewer adds /hold-for 1 to PR2
• PR2 gets hold label added by prow
• PR 1 is merged
• PR 2 get automatically unhold by prow

[jewzaam]

If /hold is also applied the hold label shouldn't be removed. New label "do-not-merge/hold-for" ?

[cblecker]

Yeah, this would probably be a new label. It's a reasonable request, but haven't thought out the design.. would be intensive on tokens to do polling.

@stevekuznetsov @alvaroaleman @cjwagner @fejta thoughts?

[RiRa12621]

Presumably using the events api periodically should make this a bit easier on the tokens than constantly polling all referenced PRs
However I do think that it's a pretty big addition and requires a proper design doc.
I will try to come up with a doc until EOW

[alvaroaleman]

This requires us to check on all merges if a PR depending on the merged PR exists. I don't think that's doable wrt token usage. If anything, the dependant PR would also need a label and the info what other PR it blocks so we can unblock when it merges.

Plugins are currently always event driven and do conceptionally not support polling. You could probably get this functionality with a periodic that finds such PRs and checks if the dependant PRs merged, if yes, remove the label.

I am not super excited at the idea of having cross-pr state. Do you have so many depending PRs that you need something like this rather than using the occasional /hold?

[RiRa12621]

I think it's not necessarily the amount of PRs but rather how long they're hanging around in the hold state.
The process to get a PR merged can something take an increased amount of time and unblocking the depending PRs automatically would make sure whoever was working on them gets an update and can continue their work.

[sftim]

BTW, the ideal implementation of this will also depend on cross-repository holds (eg documentation waiting on a cherry-pick to the upcoming release).

Could we add the command and the label but hold off on the automation? Simply tracking the holds manually in a consistent way has value for SIG Docs and especially for the docs release team.

[sftim]

do-not-merge/hold-for

Maybe do-not-merge/hold-for-dependencies? We'd only add this once per PR no matter how many dependencies are awaited.

[RiRa12621]

I don't have time to get this done in the near future, hence closing so it's not hanging around.
I guess if anyone needs it and has capacity to work on it, they can reopen

[cblecker]

Another use case for something like this: the docs team has PRs in k/website that they'd like to hold until the main code PR in k/k merges

[irvifa]

Maybe do-not-merge/hold-for-dependencies? We'd only add this once per PR no matter how many dependencies are awaited.

For now I think we still require adjustment in prow if we want to this since we're introducing new label and logic.
I have one concern here if let's say one PR B is known to be merged after PR A and then there's turned out that PR be is should be merged after PR C, then how we will handle this since the label do-not-merge/hold-for-dependencies only applied once? This applied if one k/website PR turned out to be dependent on multiple k/k PR.

I can see the changes will go to prow/plugins and prow/cmd. Introduce watching PR in given issues label, possible problem will be introduced for cross organization though I think, but I'm not sure.