Pip: Possibly useful Github Apps

[pradyunsg]

I was a little bored and found these bots:

Reminders for pings to an issue: https://github.com/probot/reminders
Automatically closing stale issues/PRs: https://github.com/probot/stale
Close on no response: https://github.com/probot/no-response

@pypa/pip-committers Do you think these would be useful to add to this repository (or to @pypa-bot)?

[pradyunsg]

@pypa/pip-committers a gentle ping on this.

Would appreciate inputs on this.

[pfmoore]

I'm not particularly keen on these. I don't really mind the reminder bot, but I get emails for all pip issues, and if people start using reminders, that could significantly increase my email load (the reminder isn't just a private message, it's an @ ping that will go to all subscribers as well as the mentioned person).

Automatic closing of issues seems a bit impersonal and gives the wrong message to me. I spent a while trying to better articulate my dislike of auto-closing, but I can't, so I'm just going to leave the comment as it stands.

[pradyunsg]

I'm not completely sure about these myself tbh.

I just thought maybe these would be useful in some way that I may not have thought of.

[pfmoore]

On automatic closing, it occurred to me that #988, #1668 and #3164 would all have probably been auto-closed by now if we had the bots. And that would be wrong, so how would we configure a bot in such a way that it wouldn't do that? I'm not sure that's possible... (Adding regular pings to keep the issues open is more admin work than the bots would save).

[pradyunsg]

That would be some sort of label.

Thinking about this now, the auroclosing things work better for projects that have a dedicated team for responding on the tracker and stuff like that. Basically, I don't think it fits our work flow.

For reminders, I don't think that adds any value to me personally (my mail + todo list work for me).

I'll close this issue, since there's basically nothing actionable on this now.

[dstufft]

FWIW I think the no-response bot is a good one. Historically we're bad at closing issues (even ones we know should be closed) and we're even worse at going back and looking for issues where we asked for more information and then never got it. I don't like the closing of "stale" issues, but if we've asked you for more information and you've not provided it, then I don't think it's worth keeping the issue around or having a human devote more time to it.

[pradyunsg]

Reopening.

• no-response bot makes sense -- falls in line with our "awaiting response" label.
• Will look for a bot that locks issues closed for more than a time t.

[pradyunsg]

https://probot.github.io/apps/lock/ - second point
https://probot.github.io/apps/triage-new-issues/ - I've been doing this lately and this would be helpful

[pradyunsg]

So, I'm proposing that we start using 3 bots - no-response, lock and triage-new-issues.

[dstufft]

One thing that isn't clear to me from the description of the lock bot, is whether it only applies to closed issues or not. If it doesn't, then we obviously don't want to add that-- but if it does then it sounds fine.

I guess one question is if we should use a hosted version of these bots, or if I should spin up a Heroku app to serve probot in.

[pradyunsg]

I guess one question is if we should use a hosted version of these bots, or if I should spin up a Heroku app to serve probot in.

I think a hosted version of the bots is probably easier and less work. If they don't work out for us, we'll switch to Heroku?

Further, if we do a(n?) Heroku app, I do think we should have more than one person having access to them -- lower bus factor.

[dstufft]

My only concern with the hosted bots is how what permissions they require. Are they pretty limited in terms of scope, or do they get more access then they should?

[pradyunsg]

I guess one question is if we should use a hosted version of these bots, or if I should spin up a Heroku app to serve probot in.

I think a hosted version of the bots is probably easier and less work. If they don't work out for us, we can always switch to Heroku.

Further, if we do a(n?) Heroku app, I do think we should have more than one person having access to them -- lower bus factor.

[pradyunsg]

@pypa/pip-committers Anything that's blocker to this? I'm happy to make a PR to add configuration for these.

Let's use the hosted versions of lock, no-response and triage-new-issues?

[xavfernandez]

lock, no-response and triage-new-issues bots seem like good ideas :+1:

[pradyunsg]

lock, no-response and triage-new-issues bots seem like good ideas 👍

Let's do this?

[pfmoore]

Virtualenv has apparently started using the stale bot. It doesn't seem to have caused much of an outcry, but I have been getting floods of notifications from it as it clears out a huge backlog, and while that was going on, I'm pretty sure I missed any genuine traffic that may have come through in the flood.

Just a data point, my opinion remains as above. (Not sure if "no-response" and the stale bot are the same thing).

[xavfernandez]

Let's try this :)

[pradyunsg]

no-response is automation for the "awaiting response" label that we've been using - it closes issues what didn't receive a response after one of the maintainers tagged it as "awaiting response".

It's less drastic than stale, since it only closes issues that are labeled as that and haven't received a response on some time.

[pradyunsg]

Merged configuration for lock, no-response and triage-new-issues bot.

[pradyunsg]

Installed the lock bot.

[pradyunsg]

Installed the no-response bot.

[pradyunsg]

Installed the triage-new-issues bot.

[pradyunsg]

Closing this issue now, since we've mostly done what we needed to here.

Let's see how this goes! :D

[ssbarnea]

Locking issues appears annoying to me and seems to force people to create duplicate tickets, something that's against good practices. What is the real benefit of locking a closed ticket? -- I fail to identify one.

Even closing stale tickets is problematic, especially if is done solely based on last-update time. There are valid bugs which are valid but where nobody spared time to fix them. It would be perfectly normal to have them open for long period of time, even years, until someone finds the energy to fix it or the situation changes in such way that the bug fix is no longer needed.

Closing a valid bug report sends a clear message: this will not be fixed, usually translating to "maintainer does not want to have this fixed".

To be clear regarding bots: I do not find them always useless. For example I find them really useful for reminding people to do something (like traging, or providing extra information). A smart bot can prove to be an useful secretary, but a dummy one can create more damage or noise if we are not careful.

I guess that is more of a tunning game where we play with different settings and adapt based on the results/feedback.

[anthrotype]

Locking issues appears annoying to me

FWIW, i really hate it and ended up un-watching the pip repository.

[pfmoore]

Locking issues appears annoying to me and seems to force people to create duplicate tickets, something that's against good practices. What is the real benefit of locking a closed ticket? -- I fail to identify one.

We get a steady trickle of people adding comments to closed tickets saying "I have the same problem" when their problem is actually different. Getting these as proper new tickets would be a benefit for us. Conversely, I'm not sure what value there is in people commenting on a closed ticket that's been silent for months. What do they expect to achieve? So what's the harm in locking?

FWIW, i really hate it and ended up un-watching the pip repository.

If you mean the current flood of "locking this thread" notifications as the bot catches up, I agree this is a major PITA. But it's a start-up cost, and should (hopefully) be over soon - @pradyunsg how is the bot configured? Could it be set to do a big catch-up run to get this over and done with, rather than getting batches of a hundred or so every couple of days?

But I don't see why locking issues per se would be annoying - other than when you actually want to comment on an issue and find it's locked, how does it affect you at all? And if there are issues you wanted to comment on but couldn't because the bot locked them, please provide the specifics, because maybe we need to tune the bot or even reopen an issue.

[pradyunsg]

@pradyunsg how is the bot configured?

We're using the hosted bot so it'll run in batches since it'll reach API rate limits when running. I don't think we can do better than that with configuration.

The most we can do is write a separate script to do it that uses much more optimized API requests (if that's even possible) but I've not investigated that yet. I expect that we'd probably still hit the API limit with that gaining almost similar results as with the botcurrently. That said, if someone wants to investigate that, I welcome that. Otherwise, I'm going to consider this a known transition pain due to the bot.

FWIW, it's definitely a PITA for me too but it won't last more than a week. If it is going to (someone needs to do the math), it's probably worth investigating a manual lock-issues sweep.

[pfmoore]

FWIW, it's definitely a PITA for me too but it won't last more than a week. If it is going to (someone needs to do the math), it's probably worth investigating a manual lock-issues sweep.

Thanks for the explanation, and no, I don't think it's worth worrying about. I hadn't thought about rate limiting, and that seems like a perfectly good reason for "batching" things the way we are.

[astrojuanlu]

So what's the harm in locking?

Apparently, I was subscribed to ~50 pip issues or pull requests and now I'm getting all these emails from threads that were already silent. Was there _a lot_ of people commenting in closed issues and pull requests in a way that it was severely distracting? If so, I guess locking was a good move, otherwise, the harm is the noise itself. See also #6563. I would have preferred to not see any comment, or just the bot unsubscribing me from the thread in question before.

Anyway, it's already done.

[xavfernandez]

FYI I merged #6564 to prevent further spam.
(I've already received ~1400 lock mails and we have 5800 closed issues :roll_eyes: )

[pradyunsg]

Would it be worth having a script go through and write a message on the closed issues if they don't have a comment?

[xavfernandez]

Would it be worth having a script go through and write a message on the closed issues if they don't have a comment?

I'm personally not convinced that the comment is necessary on locked issues, https://github.com/pypa/pip/issues/4255#event-2383986216 does provide a link explaining the working of the bot
(vs https://github.com/pypa/pip/issues/569#event-2373453750 with the comment).

And the only actionable possibility for a user is to open a new issue since they won't be able to comment on the closed one.

(And I'm also not convinced that we should re-enable it after the catching-up of the bot... maybe I'll change my mind when users starts creating issues asking why such issue has been locked)

[pfmoore]

Would it be worth having a script go through and write a message on the closed issues if they don't have a comment?

I don't think it's worth it - and anyway, wouldn't the messages generated by such a script generate just as much notification spam?