Crystal: [RFC] Improved CHANGELOG workflow

[straight-shoota]

Some of the arguments are taken from keepachangelog.com. This is generally a good resource for changelog practices.

Add changelog updates in place with PR
I propose to add new changelog entries directly with the pull request that introduces them to a Unreleased section at the top of CHANGELOG.md (see keepachangelog.com#effort)
The current practice of updating the changelog only when there is a new release is error-prone. If it is directly included, everyone reviewing the PR can directly review the changelog message and make suggestions for improvements. If it's done later, it is more likely that important aspects might be lost. Or entire entries are missed out and require someone else to double check everything (see #4854).
Approving core members can decide if a PR demands an entry in the changelog and if so, check whether it is included and properly implemented.
This way it is easy to preview what (breaking) changes an upcoming release will bring.

I don't see any benefit for updating the changelog in bulk at release, but I'd like to be entertained.

Restrict types of changes
Changelog entries should be restricted to the following set of types:

• Added for new features.
• Changed for changes in existing functionality.
• Deprecated for soon-to-be removed features.
• Removed for now removed features.
• Fixed for any bug fixes.
• Security in case of vulnerabilities.

The changelog currently fits mostly, but there are some few entries that don't adhere to this (like More iconv fixes for FreeBSD in 0.19.4).

[RX14]

I think that if we included every pr as a line in the changelog it'd be unwieldy. Doing the changelog as one operation at the end makes it much much easier to collect similar changes and comment on general efforts inside the stdlib. Viewing a change log as a linear description of simple changes is actually incorrect: that function is already performed by the git commit log. Change logs should be readable and fairly concise and be grouped by function whereas the commit log should be chronological and verbose.

[straight-shoota]

I'm not suggesting every PR should have a changelog entry, just that noteworthy changes are immediately added to the changelog. This could also be for example an addition to an existing entry - and be changed again in a subsequent PR. This procedure certainly won't prevent rewriting the changelog on release (or somewhere before) if it seems fit. But you don't have to start from scratch looking through all merges since the last release making the release process much easier. This way it's much less likely to mess something up or miss a change entirely.
And imho it makes sense for a changelog to include all changes included in the current code version, not only those that are already released.

[RX14]

Its fairly easy to find all the applicable PRs when writing the change log since we add all PRs to the "next" release. If we're going to to rewrite the change log anyway then why bother adding the extra burden to pr authors to add a line to the change log.

[ysbaddaden]

Sure, we can dig for information in commits, but a commit history can be hard to search, with noteworthy features hidden in noise, especially in such a large project. Regularly updating the changelog with noteworthy changes, especially for new features, deprecations, breaking changes and long awaited fixes, is really nicer. No need to dig for information: just look at the changelog!

Pushing the burden to the developer may be leveraged a bit, since maintainers can usually edit a pull request, thus edit the changelog before merge, squash or rebase.

Bonus: if the changelog is directly updated from a pull request, it's commit history will link directly to the curated list of commits and pull requests that affected it. Sweet.

[Sija]

I agree with @ysbaddaden here, having up to date changelog, containing list of note-worthy changes is a welcomed thing to have in every project.

Another thing to note IMO is that changelog-worthy changes might come directly from commits, not from PRs alone, so we're back to digging through history, again.

[RX14]

Sure, we can dig for information in commits, but a commit history can be hard to search, with noteworthy features hidden in noise, especially in such a large project.

This is exactly my point. By encouraging people to make changes to the change log at or around the same time they make their commit messages, the change log will naturally end up being a close reflection of the commit log. And a commit log, which is ungrouped and uncurated is a very bad change log.

[RX14]

Another thing to note IMO is that changelog-worthy changes might come directly from commits, not from PRs alone.

Nobody can commit to master without making a PR any more so this isn't a problem.

[asterite]

The times I prepared the Changelog it took me maybe 15~30 minutes to traverse the commit history, pick the interesting stuff and put it in a text file. I think it's simpler that way. It also means when someone sends a PR they don't have to worry about the Changelog.

[jhass]

We tried having changelog entries added by its PR for quite a while at diaspora. We actually got rid of it since it caused merge conflicts for almost all open PRs after any is merged.

Also I think it's wrong to put the burden of deciding whether something needs a changelog entry on the contributer, it's the maintainers job.

One trick is to amend the changelog entry into the commit as you merge. But that requires doing it locally everytime (even if it's very easy with github/hub) and actually in a rebase based merge strategy also won't show the PR as merged on Github.

[straight-shoota]

@jhass great to hear about some hands-on experience! I didn't figure it would have such an impact on the merge workflow.
While I still think it would be logical to follow the proposed procedure, I guess it makes no sense to hold this up and discuss semantic pros and cons when there is no easily accessible technical solution anyway.

But maybe there are other ideas to improve some of the mentioned issues in a different way?

• a section of Unreleased changes allows to see what (breaking) changes are to be expected in the upcoming release. This would be beneficial for library developers to release a working version of their library when a new Crystal version is release without having to regularly build a custom crystal version from master (this would probably not be as important when we have nightly builds).
• Note-worthy changes might be missed (like it was the case with 0.23.0 release). Currently, there are quite a few recently merged PRs which are not in the Next milestone. This tends to be forgotten.
• Changelog entries are written by the maintainer that publishes the release and who might not be familiar with the implications of each PR that might have been discussed weeks (or months) ago. People who are involved in a PR would have to manually look through the changelog (after the release!) to spot possible mistakes or add missing information.
• At the very least, the code at each release should include the current changelog (it was missing from 0.23.1)

Maybe it would be possible to update the changelog in an Unreleased section between releases (for example biweekly). Perhaps ping the included PRs to invite subscibers to double check. I know this adds some additional work and it's probably faster to do everything in one pece when there is a release. But this wouldn't need to be done by a core member.

[bcardiff]

I think is good to have a changelog branch to allow contributions to the changelog before releasing. It can be rebased onto master (or even receive merge from it) to signal up to which point was already revised.

That way is a process that can happen en a "weekly" basis and changes can be done incrementally.

I personally done that to keep track of where things were left.

Before releasing the changes can be squashed / grabbed and put in to master in to a single commit and the whole history of the changelog entry can be discarded (and reincarnate).

Another alternative is to flag issues / pr as "changelog:add", etc. and use that for the changelog but is more limited and solves only one part of the problem.

[akzhan]

By the way, a lot of troubles with CHANGELOG merging can be solved by commiting this file:

echo CHANGELOG.md merge=union > .gitattributes
git add .gitattributes