Pytest: add a ci bot that checks for news fragment existence or a trivial marker

[RonnyPfannschmidt]

to remind us of change-log grooming we should have a bit that does this (like pip)

keyword: changelog

[hugovk]

As far as I can see, pip uses a self-hosted bot called Browntruck to do this.

• https://github.com/pypa/browntruck

However, in https://github.com/pypa/pip/issues/6475, @webknjaz said of https://github.com/sanitizers/chronographer-github-app:

Basically, it's a rewrite of a part of the pypa/browntruck which does news fragments checks on PRs with GitHub Apps concept that @pradyunsg asked me to explore a while back.
...
Currently, it's targeting pip repo but it'll be possible to reuse it across the whole org and also extend that for the broader community which would be able to consume it (but this is out of scope of this issue, of course).

@webknjaz, would chronographer-github-app do what Ronny is after, or is there another bot/check/tool/GitHub Action you could recommend? Thanks!

[blueyed]

We have the TODO / checklist in the PR template already, but it should be made clearer that the text needs to be edited (https://github.com/pytest-dev/pytest/pull/6104).

[RonnyPfannschmidt]

i like the proposed bot

[webknjaz]

Oh, I missed this mention completely! You can try installing https://github.com/apps/psf-chronographer (it's live) and see how it goes. Essentially it only checks added file paths in the diff, no deep analysis. It's enough in 99% cases. It reports the status back via Checks API so you can enforce the check on the branch protection level if needed.

Oh, and it knows where to expect the fragment files by reading pyproject.toml.

The logic is here: https://github.com/sanitizers/chronographer-github-app/blob/master/chronographer/event_handlers.py.

P.S. The original idea was to whitelist projects that can use this app when it's deployed under the PSF infra but currently there's no such limitation and we're just battle-testing it in aiohttp and tox orgs.

You may also copy-paste this config https://github.com/aio-libs/aiohttp/blob/3cfb2a6/.github/config.yml#L1-L6 but it's optional.

[RonnyPfannschmidt]

@webknjaz on the ml @nedbat note that the app metadata and the github organisation linked to it are currently not very transparent - is there something to be done to better document those organisations ans official parts of psf infrastructure (like linking to the repository you mentioned, and making some details better known and stated on psf related websites as well)

[RonnyPfannschmidt]

@webknjaz also is there a way to disable it on repos that don't use towncrier, even if the org has it installed globally?

[webknjaz]

@RonnyPfannschmidt I had to spend some time looking for that ML. I assume you're talking about this message: https://mail.python.org/pipermail/pytest-dev/2019-December/004882.html. Please link such things in the future :)

@webknjaz on the ml @nedbat note that the app metadata and the github organisation linked to it are currently not very transparent - is there something to be done to better document those organisations ans official parts of psf infrastructure (like linking to the repository you mentioned, and making some details better known and stated on psf related websites as well)

So I feel like I might have misled you here.

When @pradyunsg asked me to look into rewriting that browntruck bot and I had something ready, I've found Ernest on PyCon to ask if we could host it under PSF. He was okay with the idea but suggested that I deploy it under a free account and then transfer the ownership later. Meanwhile, I've decided to test it on more projects so it was no problem installing the App there because the maintainers know me.

We discussed, with Ernest, having a whitelist of projects allowed to use it but I didn't implement it. I've got some other feedback from Pradyun and others. I've partially addressed that feedback but didn't fix everything. I'm collecting some of the ideas @ https://github.com/sanitizers/chronographer-github-app/issues but I think that there may be more in the logs of personal conversations.

To summarize:

• it's still going to be transferred to PSF at some point (I guess I could even do it right now) but it got a bit stuck over time
• pyba is mine, I didn't want to create it under sanitizers because it contains random stuff that might be confusing
• chronographer-github-app (src) is autodeployed to my Heroku account from master and is bound to psf-chronographer GitHub App entity on GH side
• I'm afraid there's no docs ATM but that config that I linked earlier pretty much covers all of it
• if for some reason you want to deploy your own instance (heroku + gh app) — it's quite easy to do; I've found out that the free account is mostly enough except for that annoying sleep when it may miss a few events from GH
• if there's something that needs to be adjusted, most of the time it's easy to do with a PR or by asking me

@webknjaz also is there a way to disable it on repos that don't use towncrier, even if the org has it installed globally?

Not at the moment. But it could be added. The easiest way is to just install it limiting to the target repo(s) and when somebody wants to adopt it — just add that repo from GitHub's UI.

If we want to add a config option for disabling it, we'd have to (1) figure out how that option would look like and (2) commit a config with that into every repo in the org which may be PITA.

[RonnyPfannschmidt]

@webknjaz im sorry for not linking, and thank you for putting out more details

I believe there is a need for some kind of official outline plus some more docs

I'm happy to help with that

[webknjaz]

Are you thinking of something like RTD? I'll gladly accept any help :)

N.B. I'm on PTO right now, in Chile. So I may reply with odd delays. I'll be back in your timezone in ~10 days. If I somehow miss messages, please ping me again.

[RonnyPfannschmidt]

@webknjaz in that case lets coordinate when you are back

[webknjaz]

Hey, I'm back. So do you have any specific further thoughts?

[RonnyPfannschmidt]

I caught a bad cold and will pick things up next year

[webknjaz]

Oh, too bad. Get well!

[pradyunsg]

Happy new decade folks! Gentle nudge to everyone. :)

[RonnyPfannschmidt]

thanks for the ping, i haven't had any specific thoughts, i think i mainly want to see documentations, organisations and repositories line up in a comprehensible way so that we can start using it in a documented discoverable fashion

[webknjaz]

You offered to help earlier. What exactly did you want to contribute?