Test-infra: opting out of fetja-bot?

[bacongobbler]

Hey, over at kubernetes/helm we'd like to opt out of fetja-bot from marking issues as stale. Any way we can disable this bot for the project? Thanks!

[stevekuznetsov]

/cc @fejta @cjwagner @BenTheElder @spiffxp

@bacongobbler there has been a lot of conversation around this recently, specifically also with the Helm repo. As a super quick refresher of how I think much of that has gone:

• the steering committee has put emphasis on making sure that there is a unified process model, where the interaction with bots and expectations of contributors are clear consistent between all k8s repos
• in general there is a large backlog of stale issues that serve no purpose
• the current mechanism has a process for making an issue as frozen so it does not become stale again
• for new issues and PRs, it will take 90+30+30=150 days or half of a year (!!!!) to be closed _if nobody ever interacts with them in that time_

I think the general push-back to not having the stale bot run at all is -- especially in the lightning-fast ecosystem of k8s, half of a year is an enormous lifetime for an issue or PR and the vast majority of issues open for this long (where nobody interacts with them!) are not relevant any longer. The small fraction of issues that are untouched for half of a year and still relevant can be marked as such with /lifecycle frozen and that is a small price to pay for the valuable feature of stale issue closing in general.

I don't mean to say that more conversation is not necessary or that anything has been finally decided, just wanted to seed this issue with some of the comments that have been posted previously.

[bacongobbler]

@stevekuznetsov thank you for seeding the conversation with some background context on previous discussions. I really appreciate that. :)

in general there is a large backlog of stale issues that serve no purpose

...and the vast majority of issues open for this long (where nobody interacts with them!) are not relevant any longer.

With all due respect - for Helm in particular - I strongly disagree that these stale issues serve no purpose. A large amount of our older issues are either proposals or feature requests, all of which are fantastic features for Helm. It's usually the case that nobody's had the time to implement them, or we're waiting for the Helm Dev Summit (starting tomorrow) to discuss whether we want to consider them for Helm 3. For the core maintainers, we now have to sift through hundreds of tickets just to ensure that these enhancement proposals won't be closed. That doesn't bode for a great experience as a core maintainer of the project.

it will take 90+30+30=150 days or half of a year (!!!!) to be closed if nobody ever interacts with them in that time

It's probably a bug, but we've discovered that the stale issue bot will continue its lifecycle even after a user has commented on a stale or rotten issue, as if that user never commented. An example can be found in this ticket here.

The small fraction of issues that are untouched for half of a year and still relevant can be marked as such with /lifecycle frozen and that is a small price to pay for the valuable feature of stale issue closing in general.

Perhaps for kubernetes, sure. But for Helm, we have about 70 open feature requests and 84 open proposals, all of which have discussions lasting several months. This is a significant chunk of our ticket queue (>40% of currently open issues). For us, this isn't a small fraction of issues; it's a large majority of them. :)

In conclusion, fejta-bot has probably been the single-most controversial "feature" that has been turned on for the Helm project. There were no announcements, no visibility or no option to toggle the bot on or off, and if you gauge community feedback via reactions to fejta-bot's comments, you can clearly tell that the Helm community doesn't appreciate it either. I'm not wild about having someone not associated with the project (read: not part of the core maintainer list) turning "features" on and off for us. The model should be that projects opt in for best practices (barring legal things like the CLA bot), not top-down enforcement from the org that we have to fight to opt out of after the fact.

I am kindly asking for us to have the option to opt out of features turned on by test-infra, including the stale issue bot. If there is a way I can make that happen, please let me know and I will be more than happy to donate my time to work on a fix.

[stevekuznetsov]

@bacongobbler thanks for sharing your thoughts. It definitely seems like the current implementation (and that bug) are making this a frustrating feature for your repo contributors. In addition to this issue, I would suggest anyone interested in this to jump into the sig-contributor-experience weekly meeting Tomorrow, Wednesday the 21st at 9:30AM PST. More information here. This topic will be covered there.

[BenTheElder]

I just want to add that I've read through this and I'm holding for the discussion tomorrow but happy to PR bringing things in line with whatever consensus is reached.

[stevekuznetsov]

It has been brought to my attention that the Helm summit is starting tomorrow morning and therefore many of the core contributors on that repo specifically will not be able to attend the meeting. I've suggested sending thoughts via e-mail to the contributor experience mailing list or this thread so as to be able to share them anyway. Shame about the timing of things.

[spiffxp]

I'm at helm summit and will be happy to discuss in person. I agree that this was rolled out without consensus from all those it would affect. In general I would prefer to see us follow a model of proposals floated through sig-testing and sig-contributor-experience prior to implementation, and kubernetes-dev at least prior to rollout.

The commenter jobs can be configured to support more granularity in their queries than they do. So it's not hard to have them exclude certain repos or certain labels.

That said, consistency would greatly help with (documenting, explaining, enforcing) the contributor experience across the kubernetes project. Perhaps different kinds of issues have different lifecycles, and 90d is maybe too aggressive a cadence for issues related to roadmap and planning.
If you knew ahead of time which kinds of issues you wanted to keep open, you could triage them as such. Today that means /lifecycle frozen, and the pain you're feeling is the process debt of not having explicitly spelled out that decision with a label.

Opting out of prow plugins is less easy on our end, simply because prow's config doesn't support blacklisting, only whitelisting at the org and repo level. We prefer to turn on at the org level because most of them respond to user actions rather than preemptively act. For example I don't think it's causing helm any pain that folks could /meow all over your issues. It might even be helpful that folks can /assign and /close. But that's not always the case... I don't know, for example, whether PR's getting auto-labeled with size/S-size/XXL is causing you pain in that it conflicts with helm's manual size/small-size/large policy. And we explicitly haven't enabled approval because that requires changes to the OWNERS files.

tl;dr I like configurable opt-in as a model, but I also like consistency as a maintainer, I'm interested in how we can get feedback on process changes and roll them out more responsibly

[spiffxp]

What I got out of discussing in person with @bacongobbler was:

• we're fine leaving things as they are
• but please please please have a process that includes consent for rolling things out in the future
• especially in cases where automation acts preemptively rather than in response to user action

What do folks think about the general policy?

• proposals floated through sig-testing and sig-contributor-experience prior to implementation, and kubernetes-dev prior to rollout
• use lazy consensus for rollout decisions
• lean on contributor experience's "roadshow" to educate on the full scope of automation we offer

[bacongobbler]

I fully agree with the summary. :) However,

but please please please have a process that includes consent for rolling things out in the future

Yes for consent, but I want to express that I'm not asking for explicit sign-off from every org member. Just a contributor experience's roadshow would be more than enough. A "Hey! We're interested in rolling X out across the org. Here's why" with a short question period before the rollout would be +:100:. :)

[mithrav]

Agree with the points both of you are making. Let's make it a policy to float suggestions to the Sig before making changes and use lazy consensus with a week's delay unless otherwise specified.
Should we document a how this Sig works document for posterity?

[stevekuznetsov]

I believe that something really similar to that flow is codified in the Contributor Experience charter -- maybe @parispittman can shed some light on that

[parispittman]

I just added the change communication flow piece to the contribex charter -

In cases of impacts across a large number of areas and/or project wide, we will:

• Lazy consensus with a time box of at least 72 hours to the following mailing lists with the GitHub issue link including the subject [NOTICE]: foobar to the following mailing lists:
  
  
  
  • kubernetes-dev@
  
  
  • sig-leads@
  
  
  • kubernetes-wg-contribex@
  
  
• Announce at weekly Kubernetes Community Meeting on Thursdays

[bacongobbler]

Thanks Paris. For those who need a link (like me), the contribex charter is here: https://github.com/kubernetes/community/blob/master/sig-contributor-experience/charter.md

I think we can consider this closed. Thanks everyone!