Mocha: discuss use of conventional-changelog, semantic-release, commitizen

[boneskull]

Releasing is a manual--and often painful--process. Especially generating the changelog. When we go a long time between releases, it takes me forever to get the changelog updated.

If we used a combination of the tools mentioned, we could likely release more quickly. The drawback being that commit messages would have to be in a very particular format.

[boneskull]

cc @mochajs/core

[ScottFreeCode]

I like the idea of making releases more automatic. I guess that's obvious though. I've long since learned that in business, "process improvement" means "this should be automated, but we're following rules manually instead of (or until we've finished) automating it."

On the other hand, I might be an oddball on this point, but dislike the idea of involving the commit log in high-level actions:

• Putting commit info in the changelog is redundant at best and at worst violates separation of concerns when it comes to writing good commit messages: a properly tagged release should enable viewing commit history from a given release to another anyway, whereas a changelog needs to provide a brief, user-relevant, high-level view instead of explanations of individual commits. I realize a lot of major projects (some bigger than Mocha will ever be!) require commit messages to be formatted for re-use as changelog info anyway, but I consider this one of those things where a widespread quasi-standard practice is better than haphazardly winging it but not actually the best practice it could be.
• Things such as semver changes are, in my opinion, better off automatically detected than relying on the commit messages to accurately reflect them. For instance: based on prior tests, based on dependents' tests, and/or based on API/documentation diffs* (Got all of these from the top of the readme for Semantic-Release, so we're actually on the right track here in any case.)

On the other hand, PRs strike me as a good place to start looking for info to re-present in a changelog, since they're a high-level counterpart to commits and (unless you make everything a PR but still push through changes in a completely atomized fashion) there are fewer of them. We could consider turning on GitHub branch protection to require changes to go through PRs rather than direct commits, as a start. Then, the most manual approach would be reviewing the PRs since the last release, while the most naive automated approach could be generating a changelog from PR messages rather than commit messages. Ideally, what I'd like to see is generating a draft changelog with links to the PRs so that the developer can review the PRs themselves and tidy the generated changelog up if need be but such manual intervention is opt-in, and then (to address the case where even many small, atomized changes are going in through the PR system) have a way to prioritize and/or filter PRs as low-level or high-level either through flags in the PR messages or through some info (probably also from the PR messages) as to how they fit into a hierarchy (so if change A is made from PRs X, Y and Z you could just grab A and ignore its parts, for instance). I don't know if there's tooling out there for that sort of thing yet or not, however.

That being said, I realize I'm in a bit of a minority position on this, however reasonable I think my position is, so as a practical matter using well-maintained tools to meet most developers' expectations has a lot going for it. That and, as I alluded to above, even if I don't think the widespread practice is actually the best it could be, it's still better than manual/ad-hoc processes. So anything's an improvement at this point, and while I wanted to make my case for something a little different, I'll defer to team preference and/or available tooling.

*(Actually, JSDoc in particular is another thing where a widespread better-than-nothing practice really could be improved upon. Why are people writing documentation in comments a la C++ and Java when in JavaScript you could provide it in a JSONesque object to automatically provide programmatic access everywhere the code itself is useable? With JSDoc, I have to run a tool to extract this stuff; with the kind of documentation I envision, I could look up descriptions and metadata on an API from the same consoles and REPLs from which I can fiddle with the API directly -- which makes it way easier to learn an API through such hands-on fiddling -- or I could write a JS program that processes it any way I want without needing a JSDoc-parsing lib to read the file. ...Yeah, I should probably make tooling to help people use my preferred form of documentation rather than ramble about it on unrelated GitHub issues.)

[dasilvacontin]

Scott, there's something a bit similar (afaik) to what you mention in your last point if you want to check it out: http://node-machine.org/

Sent from my iPhone

On 24 Sep 2016, at 21:02, Scott Santucci [email protected] wrote:

I like the idea of making releases more automatic. I guess that's obvious though. I've long since learned that in business, "process improvement" means "this should be automated, but we're following rules manually instead of automating it."

On the other hand, I might be an oddball on this point, but dislike the idea of involving the commit log in high-level actions:

Putting commit info in the changelog is redundant at best and at worst violates separation of concerns when it comes to writing good commit messages: a properly tagged release should enable viewing commit history from a given release to another anyway, whereas a changelog needs to provide a brief, user-relevant, high-level view instead of explanations of individual commits. I realize a lot of major projects (some bigger than Mocha will ever be!) require commit messages to be formatted for re-use as changelog info anyway, but I consider this one of those things where a widespread quasi-standard practice is better than haphazardly winging it but not actually the best practice it could be.
Things such as semver changes are, in my opinion, better off automatically detected than relying on the commit messages to accurately reflect them. For instance: based on prior tests, based on dependents' tests, and/or based on API/documentation diffs* (Got all of these from the top of the readme for Semantic-Release, so we're actually on the right track here in any case.)
On the other hand, PRs strike me as a good place to start looking for info to re-present in a changelog, since they're a high-level counterpart to commits and (unless you make everything a PR but still push through changes in a completely atomized fashion) there are fewer of them. We could consider turning on GitHub branch protection to require changes to go through PRs rather than direct commits, as a start. Then, the most manual approach would be reviewing the PRs since the last release, while the most naive automated approach could be generating a changelog from PR messages rather than commit messages. Ideally, what I'd like to see is generating a draft changelog with links to the PRs so that the developer can review the PRs themselves and tidy the generated changelog up if need be but such manual intervention is opt-in, and then (to address the case where even many small, atomized changes are going in through the PR system) have a way to prioritize and/or filter PRs as low-level or high-level either through flags in the PR messages or through some info (probably also from the PR messages) as to how they fit into a hierarchy (so if change A is made from PRs X, Y and Z you could just grab A and ignore its parts, for instance). I don't know if there's tooling out there for that sort of thing yet or not, however.

That being said, I realize I'm in a bit of a minority position on this, however reasonable I think my position is, so as a practical matter using well-maintained tools to meet most developers' expectations has a lot going for it. That and, as I alluded to above, even if I don't think the widespread practice is actually the best it could be, it's still better than manual/ad-hoc processes. So anything's an improvement at this point, and while I wanted to make my case for something a little different, I'll defer to team preference and/or available tooling.

*(Actually, JSDoc in particular is another thing where a widespread better-than-nothing practice really could be improved upon. Why are people writing documentation in comments a la C++ and Java when in JavaScript you could provide it in a JSONesque object to automatically provide programmatic access everywhere the code itself is useable? With JSDoc, I have to run a tool to extract this stuff; with the kind of documentation I envision, I could look up descriptions and metadata on an API from the same consoles and REPLs from which I can fiddle with the API directly -- which makes it way easier to learn an API through such hands-on fiddling -- or I could write a JS program that processes it any way I want without needing a JSDoc-parsing lib to read the file. ...Yeah, I should probably make tooling to help people use my preferred form of documentation rather than ramble about it on unrelated GitHub issues.)

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[ScottFreeCode]

Interesting; I will have to dig into that sometime. (Thanks!)

[boneskull]

Scott, there's something a bit similar (afaik) to what you mention in your last point if you want to check it out: http://node-machine.org/

wow.

[ScottFreeCode]

I know, right? I was only thinking that the same type of info that goes in doc comments should be written as JSON and assigned to a __doc property (or something like that) on the function object.

[ScottFreeCode]

A couple other tools I've found that are similar to the ones I mentioned above re. detecting/handling semver changes:
https://github.com/bahmutov/dont-break
https://github.com/bahmutov/next-update

[dasilvacontin]

Okay, back to this. We need this.

I agree with @ScottFreeCode that _every single commit_ shouldn't go in the CHANGELOG. What I usually do (and done for the release I'm crafting), is to only include merge commits + commits that were directly committed to master.

Even just listing all the PR titles + pr number + contributors (same for the direct commits to master) in the same way I'd do for the CHANGELOG is a big help already...

[dasilvacontin]

e.g. in combination with gh-sauce, or something similar.

[ScottFreeCode]

We could also protect the master branch (and any branches used to collect semver-major changes for a future release) so that changes have to go through PRs. Then we're not seeing a changelog of individual commits, instead looking at the slightly higher-level PRs/merges, but the changelog can't miss any of the updates either.

[dasilvacontin]

There's a certain beauty in committing directly to master that we would lose though...

[stale[bot]]

I am a bot that watches issues for inactivity.
This issue hasn't had any recent activity, and I'm labeling it stale. In 14 days, if there are no further comments or activity, I will close this issue.
Thanks for contributing to Mocha!

[stale[bot]]

I am a bot that watches issues for inactivity.
This issue hasn't had any recent activity, and I'm labeling it stale. In 14 days, if there are no further comments or activity, I will close this issue.
Thanks for contributing to Mocha!