Warehouse: Implement Changelog in Version History

[dstufft]

The new design has an affordance for to inline the change log for a particular version. We'll want to figure out how to extract this from projects, or failing that show nothing for the change log.

[dstufft]

This is probably going to need to happen after the launch. We don't currently have any way to get this information and it'll take some thinking to figure out how to do that. I tried to remove the change log data from the current design, but the CSS breaks when I do.

[nlhkabu]

Is it is hard requirement that we show a changelog on launch? If not, can we just remove the entire page for now?

[dstufft]

I'd like to at least show the list of versions so people can navigate to older versions. If it's hard to adjust the design so that we can get a list without the change log portion of the list we can maybe punt on it (though I think we'll need to adjust the design anyways because the changelog feature itself is probably a bit more effort than I'm going to have time in the near term since we need to figure out how to actually let people communicate their changelog to Warehouse).

[nlhkabu]

OK, I can come up with something - no problem :D

[carsonyl]

@audreyr's cookiecutter-pypackage template has a setup.py that sets the description as the concatenation of readme and history. I don't know how common this pattern is, but I thought I'd mention it in case it's relevant to the design here.

[nlhkabu]

I have opened a new issue to design a slimmed down version of the current design (see #1311). I will leave this issue open to track the implementation of the 'full' version, which will need to happen post launch.

For reference, this is the original design spec:

[kavdev]

I just came across changelogs.net. Seems pretty good at grabbing changelogs from repositories. If the history feature of warehouse is not already implemented, it might be worth a chat with the developer to bring in the functionality.

That being said, I'd much prefer to use a PEP style changelog convention that warehouse and other services could rely on.

[brainwane]

I'm guessing that code is somewhere in https://github.com/pyupio/pyup ?

[kavdev]

@brainwane: I don't believe so; that seems to be a command line tool for their main service.

[idlesign]

Another option: https://allmychanges.com
Webhooks seem to be available. CLI and API: https://github.com/svetlyak40wt/allmychanges

[jayfk]

@brainwane the code for the parser is part of the service at pyup.io and not open sourced.

Finding and parsing changelogs is relatively error prone and requires a lot of manual work from time to time. On top of that, a lot of the bigger frameworks and libraries have their changelogs in all kind of different places. This leads to a lot of repetitive spaghetti code and edge cases. Basically a unmaintainable mess.

If someone is really interested in maintaining this behemoth, I'd love to rip this out of pyup.io and open source the code.

Having said that, all the changelogs are freely available: https://pyup.io/static/changelogs.json
Maybe this could be used to fill up the holes until we find a better solution?

[kavdev]

^ it seems a PEP for changelog format/placement would make all of our lives better

[jayfk]

I've ripped the finder/parser out of pyup.io and open sourced it: https://github.com/pyupio/changelogs

I wasn't able to write a lot of documentation for it, but the code has a lot of inline comments so it should be relatively readable.

[jayfk]

To make it a bit clearer why something like this is going to be hard to maintain, take a look at the get_head function.

Changelogs are a mess :).

[di]

@jayfk Thanks for open-sourcing that! I agree, it seems incredibly hard to maintain (so thanks also for creating it, because it really makes pyup.io nice!)

Similar to what @kavdev said above, I'm wondering if it makes sense for Warehouse to support only a very specific change log format (e.g., something like http://keepachangelog.com) and fall back on just showing the plaintext change log if it doesn't parse.

[kavdev]

@di: perfect suggestion on using http://keepachangelog.com. I vote yes on that; maybe add a note that the changelog page follows that convention.

@olivierlacan, how do you feel about writing a PEP? It seems like your website already has most of the information needed.

~I think I'll just open up a repo for the pep next week or so to get the ball rolling.~ lol

[jayfk]

I've opened an issue on the keepachangelog repo with the idea to build an open source changelog index: https://github.com/olivierlacan/keep-a-changelog/issues/154

My end goal would be to provide an API that package indexes like PyPi/warehouse can use to get a changelog for every given package.

[jayfk]

I noticed that the package detail page now contains a history as part of the description 👏 https://pypi.org/project/changelogs/

I checked the commits, but couldn't find anything related to it. Where is this coming from?

[dstufft]

That project must be manually adding it to their long_description.

[kavdev]

@jayfk https://github.com/pyupio/changelogs/blob/master/setup.py#L32 :)

[brainwane]

In order for us to actually replace PyPI, I think we do need to give users some systematic way to find package changelogs. I'm fine with saying that the specific way we implement that is not specifically by implementing any of the approaches previously mentioned in this issue, so maybe this issue moves to a later milestone and we create another issue that can go into the Launch milestone. :)

[dstufft]

So legacy PyPI doesn't currently have that feature (other than by baking your changelog right into your description, which also works on Warehouse). So I guess the question is whether we think some dedicated mechanism feature is an important enough new feature to block the transition from legacy to PyPI or not.

Personally, I think we've gone a decade without it, so going a bit more without it wouldn't hurt, but others may disagree!

[brainwane]

So the limited Warehouse development funding is specifically to get to the point where we can redirect pypi.python.org to pypi.org so the site is more sustainable and reliable. That's the basis for the current development roadmap and so I've come to agree with Donald -- since this feature isn't something that the legacy site has, it doesn't need to block the transition, so I've moved it to a future milestone.