Freecodecamp: Release engineering improvements

[ojongerius]

Hey there,

I'm trying to do some dev work and keeping my work up to date with staging, but often I find myself getting frustrated because "things are broken" and I'm not sure if this is due to my local setup or due to HEAD being in a bad state.

I'm not sure how things are deployed at the moment, but it would be awesome to know what was released, and when.

Some ideas, not mutually exclusive:

• Announce deployments, per env in a Gitter channel, just a "Bleep, I'm a bot and I just deployed SOME_COMMIT_HASH to SOME_ENVIRONMENT" would be really helpful.
• Tag when deploying (by deployment script from CI). Possible, but I'm not a fan.
• Expose which commit has been deployed to what environment on a status page.
• Have "proper" releases, describing which issues went in. This can be automated. I'm a fan of this. Though I'd still like to be able to see when it went out.

[ojongerius]

@QuincyLarson / @BerkeleyFalse / @raisedadead / @Bouncey something that has been discussed before? Also, is "mass" at-mentioning like this "done" or considered spammy and evil? Is there a team I should at-mention instead?

[Bouncey]

@freeCodeCamp/moderators how we mention the team @ojongerius. But @'ing individuals is fine.

We used to announce releases to production in the main feeCodeCamp Gitter room, in the announcement was a brief summary of commits released by PR title. Notable contributions got special mentions (bug fix, new feature, first-time contribution). We didn't stop doing it for any reason, we just haven't had a production release outside of hit-fixes in a long time.

I would be a fan of doing this again. Releases to beta could be announced in the Contributors Gitter room. Production releases could have this also with the addition of a tweet from @freecodecamp.

I like your releases idea, would this be similar to a Change log?

[Bouncey]

Also @ojongerius, what would you consider as broken?

[ojongerius]

Also @ojongerius, what would you consider as broken?

I had a PEBCAK issue, which I was suspecting. Not knowing a tag that I could trust felt frustrating.

I like your releases idea, would this be similar to a Change log?

I was referring to Github releases, if you are not familiar with them, they are a Github concept, and can contain meta data such as release notes, commit hash and build artefacts. Releases have URLs which is practical to link to for projects that want to link to their releases. See https://help.github.com/articles/about-releases/ for an example.

A release can be created programatically, see https://developer.github.com/v3/repos/releases/#create-a-release , and release notes could contain what has changed since the previous release / tag.

I'd starting with tagging, and autogenerating a changelog. Releases can then be generated based on those tags, and possibly the changelog.

The release bot or person could broadcast a link to the release when they push a release out to Gitter and or Twitter.

Generating changelogs

Generation is generally done based on commits, but can be done based on PRs, issues and tags too.

The approach of generating changelogs based on merged PRs, tags and issues is interesting. It would be much less verbose and fits well into the current workflow. github-changelog-generator looks pretty mature (4k+ stars, last commit 21 dags ago), they do not support generating releases yet but do express the desire to support that too.

Many tools commits and tags. As freecodecamp already enforces conventional commit messages, it would not be too hard to get set up. A CHANGELOG.md generated with conventional-changelog-cl looked fairly decent.

[erictleung]

@ojongerius @Bouncey the topic of GitHub releases has been discussed before and @BerkeleyTrue has decided against it, essentially because freeCodeCamp's code base doesn't follow a traditional software development cycle. But things may have changed since then.

See https://github.com/freeCodeCamp/freeCodeCamp/issues/13605 for the discussion on GitHub releases.

A change log may be helpful and easier now with those change log tools. From what I've seen in the past, @BerkeleyTrue's PRs that update the production site lists the notable changes associated with it.

[raisedadead]

@ojongerius @erictleung thanks a lot for discussing this.

Yes, we have had debated on the idea in the past.

Honestly, the contributions to this repo have never followed a traditional Software Engineering processes, and best practices in a strict sense. We could enforce that, by adding rule checks like we do for commits, etc. But we really count on a majority of the contribution towards the curriculum which believe it or not, sees the highest no of contributions, and most often than not from contributors who have done this once.

That's a good thing, because it shows that we have a very very wide diaspora of people contributing.

And all the more reason we should standardise things like change logs, and releases.
But, that brings up another challenge, the releases in that sense would mean not much except a chore.

I hate to admit, but yeah we have wanted to avoid that, for the sake of being able to be productive in the small ops team of ours.

BUT, I do understand you frustrations @ojongerius ! I can clearly see that when a bad commit breaks you local setup, its frustrating because you may or may not be aware of what went wrong.

That's why I propose to rather get the docker setup working, ready and even publish images to Docker Hub in future.

I don't think having GitHub releases will help much. Just to give an hint:

1. It only gives you a .zip of all the code at a point of time (tag).
2. npm dependencies are not bundled.
3. We actually don't have any binary artefact that we build for deployment.
4. From an ops perspective are not focused on distributing software, but being able to run the platform for the community, which is the current production.

We would absolutely love any contribution that helps improve the DX in terms of automated setup (container) and documentation.

I guess, when we release beta, we will also start having Changelog too.

[ojongerius]

@erictleung: thanks for linking that issue! That's very useful.

I guess, when we release beta, we will also start having Changelog too.

@raisedadead Cool, is that a fact? Has that decision been made? Is there an issue that I can track?

This end user should assume that if the master branch has been updated then their version is out of date.

@BerkeleyTrue or someone merged a breaking change. What confidence can a user have in HEAD if it's unknown what has been deployed, when?

Why Docker alone does not solve the problem

Docker compose will make setting up of the freeCodeCamp environment faster, but it will not add any guarantees about if HEAD works or not.
A Docker image only adds confidence in it working if the image has a tag, which effectively is the same a release (in the generic sense of a release, not the Github concept of a release). If we tag a Docker image we might as well create a git tag.

Why Github releases are helpful even though we do not need all of it's features

Because they awesome for communication: You can link to the release, the release notes support Markdown so are easy on the eye. This ads value when sharing those releases on Gitter and potentially Twitter.
Not that a release strictly does not have to include a changelog. There is value in adding them an d changelogs are automated I think it would add value. But is going through all issues / commit PRs manually is prohibitive, we could still create a release with an executive summary.

Why tags and or releases do make sense for a website instead of traditional software

If a dev wants to verify that she is building / testing against something stable, having a tag that you know is in production is really really helpful.
The cost involved for freeCodecamp to tag on deployment is minimal. The technical implementation of automated tagging on deployment is trivial.

Why are releases better than descriptive PRs

There are many more merged PRs than there are releases.

[ojongerius]

Since raising this issue I've learnt that, at this moment, the release frequency is very low. I'm happy to close this of for now. When Beta gets out, releases should happen more frequently and we can revisit this.

Thanks you for your contributions in providing some history and context of the current state of affairs!