Parse-server: ♻️ Fixed Release Cycles

[mtrezza]

New Feature / Enhancement Checklist

• [x] I am not disclosing a vulnerability.
• [x] I am not just asking a question.
• [x] I have searched through existing issues.

TLDR

The goal is to have predictable, frequent, immediate, up-to-date releases of Parse Server with a high degree of automation by mid / end of 2021.

Current Limitation

Parse Server has no fixed release cycles.

Feature / Enhancement Description

Introduce a fixed release cycle which helps developers, contributors and repo managers to better plan.

This requires the introduction of a versioning system. Ideally, we go with semver:

• Breaking change, significant feature additions, UI redesign be a major increment.
• Feature additions be a minor increment.
• Bug fixes be a patch increment.

A simple release cycle, for example:

• A major release every 6 months (to make breaking changes that have been announced in at least the previous major release).
• A minor release every month (to release new features, improvements)
• A patch release every month (may be merged with the minor release) or immediately if the fix is at least a bug of severity S2

I expect that in the process we can automate some of the release tasks, so that it becomes less work to actually do a release and we can drive down the times between minor and patch releases.

Long Term Support (LTS)

LTS means that certain versions do not receive breaking changes but only bug fixes or minor compatibility changes that are not breaking. Once a version goes into LTS, a developer can update to newer patched versions without having to worry about significant or breaking changes, while knowing that the LTS version keeps receiving the latest security fixes. It basically decouples feature changes from security fixes.

Maintaining multiple LTS versions significantly increases resource requirements and may therefore require more extensive considerations. To follow a step-by-step approach it makes sense to keep LTS out of scope for this discussion. Since LTS depends on release cycles, I suggest to first finish this issue with a satisfying level of automation and based on that experience, open the LTS discussion with more insight at a later stage.

Related issues:

• https://github.com/parse-community/parse-server/issues/7198
• Feature Spotlight obstacle

---

Roadmap

• [x] Introduce bug severity levels to classify severe bug fix for patch release https://github.com/parse-community/parse-server/issues/6984
• [x] Add changelog entry with every PR to reduce release effort https://github.com/parse-community/parse-server/pull/7051
• [ ] Introduce versioning system https://github.com/parse-community/parse-server/pull/7294
• [x] Introduce centralized deprecation mechanism https://github.com/parse-community/parse-server/issues/7302
• [x] Introduce deprecation policy https://github.com/parse-community/parse-server/pull/7199
• [ ] Introduce new branch system
• [ ] Automate release process to reduce effort to 5 mins + 5 mins review
• [ ] Introduce release cycle policy
• [ ] Coordinate with contributors of open PRs to merge before 5.0.0
• [ ] Release 5.0.0 as "fresh start" (mid/end 2021)
• [ ] Advertise new release cycle to community to get more deployments onto the release train
• [ ] Experiment with Long-Term-Support (2022/23)

[sadortun]

Hi guys!

As a "user" We are using Parse Server on many different projects, I think there is a few important things that should be taken care of. As mentioned previously, update fatigue is something to consider.

When making your decision on a standardized release cycle, I have a few things :

---

1) LTS releases / branches are very important

Whatever the new version is, it's super important to know if this major/minor release will be supported and for how long. I would never update a project to the next major if I am not sure for how long it will be supported with patches.

Current v4.5.x should be a LTS. Users who don't want to receive breaking changes should be able to continue on this and continue to receive bugfixes and security fixes for 1+ year.

There should be at lease one LTS per Major. For at least the current and last Major.

---

2) Release schedule

• Having at least patch release per month is a must. If your release process is not too consuming, maybe every 2-3 weeks :)

• Minor releases every 2-4 months.

• For major releases, I'm not sure if it make sense to have a release schedule since this project is based in community contribution. That said, it would be really great in the readme to have a section about the Upcoming Major detailing a roadmap, expected deprecations and expected release.

Have a nice day!

[mtrezza]

Here is a pro / con list as basis for discussion:

Phased Deprecation

Pros:

• Solves pain point that has been mentioned repeatedly in community feedback
• Decouples breaking changes from new features, improvements & bug fixes
• Developers are more likely to upgrade due to gradual breaking changes (only in major versions)
• Forces us to think more modular in PRs to turn on/off segments of code, which is the essence of Parse Server anyway
• Refreshes code more frequently, as code blocks are more likely to be written from ground up instead of old code being hacked / modified and dragged along
• We have experience with optional features (phasing-in instead of out), so we should be able to estimate the risk of additional effort quite accurately. See experimental features like directAccess, singleSchemaCache, PagesRouter. For example, for the PagesRouter I already added a TODO list to the PR for what needs to be changed to move it out of experimental status, so it will easy to pick this up again in a few months. There is not much difference in work volume, had I done it immediately.

Cons:

• Splits a PR into 2 PRs: a PR for the deprecation warning and optional change, another to remove the deprecated code.
• If the second PR are not done by the original contributor(s), it could take more effort for someone unfamiliar with the changes to do the PR.

Versioning System (semver)

Pros:

• Developers can properly use range operators for parse server repo dependency
• Developers more likely to upgrade throughout a major version due to separation of breaking changes (major version) from new features, improvements (minor version) and bug fixes (patch version). Think Node.js or MongoDB, we are currently upgrading our CI test environments almost effortlessly.
• Parse Server complies with npm versioning best practice, which we expected from other dependencies but ironically don't contribute to the JS ecosystem ourselves

Cons:

• Effort to create policy document

Release Cycles

Pros:

• Increased adoption rate of changes because developers can use tools like npm outdated to be made aware that there is a new release; now these tools falsely report that they are on the latest and greatest version of Parse Server, which cannot be true when the last release was 4 months ago and there are many improvements queued in the master branch
• Implementing release automation should zero out or even save us time in the long run (even with shorter release cycles) than manually doing releases once in a while
• Historically, we are releasing every 2-4 months, so releasing every month would not be significantly different, even without more automation. The current manual releasing should not take more than 5-10 mins. Update version number in package, clean up the changelog (which should already be clean and complete) and get review approval.

Cons:

• Effort to set up and refine release automation

General Note

Most important for all of the above is that they mitigate upgrade fatigue, because developers don't have to worry doing a patch or minor version upgrade. Which means developers are more likely to upgrade to newer versions, ergo we get more feedback more quickly for more recent versions of Parse Server, which speeds up

• the development cycle
• identification and fixing of bugs
• maturation of experimental features

We currently block our own gears. We have 2-4 months between releases, presumably the majority of developers is not deploying the master branch, and if, most likely not in production, that creates a feedback lag of presumably half a year(!).

[sadortun]

How much time does it take you to make a release ?

[mtrezza]

@davimacedo How long did it usually take to make a Parse Server release?

I think the key is to maintain it constantly and require less ad-hoc effort on release. We are already doing that for Parse Server because the changelog is now always up-to-date as it is a requirement since we switched to the new PR template half a year ago. There should be virtually no effort anymore other than a superficial changelog review.

[davimacedo]

It is not that much work. Let's say 30 min. Now that we are adding the entries to the changelog in each PR, it should be faster. It is more time consuming when we have relevant changes in the dependencies and we need to release everything bottom up. I'd love to migrate the project to a monorepo with all js projects.

[mtrezza]

My expectation is that this should not take longer than 5 minutes (+5 min effort for one reviewer) even without any more sophisticated automation. A release should be exclusively a meta review, without having to review or touch any actual code.

@davimacedo If it's fine with you, I'll do the next release with your guidance, this or next week and we'll document any bottlenecks and see what we have to address to get to the 5 mins. And we'll do the versioning already according to semver to finally get this ball rolling.

[davimacedo]

Sure. We can do that.

[mtrezza]

Here is a practical example of how to do a breaking change going forward.

[sadortun]

@mtrezza in this case, what can be done would be to imement the PR now, but add a config parameter that would default to false

If the parameters remind undefined you display a deprecation warning to the logs until v5.0.0, after that you change the default behaviour.

Since youre also considering refactoring the server config, I would recommend implementing support for the new structure in current v4.x with a mention that previous structure will be deprecated later.

Deprecations are just fine if:

• We know in advance something might change
• We don't loose time working on a feature that's going to be deprecated
• We don't have to implent 200 deprecations at the same time. ( Ex: put them gradually, so we know in advance what work need to be done even before looking at upgrading to v5.0 or looking at a CHANGELOG.

For example, WordPress is dooing an awesome job at this!

[mtrezza]

@sadortun How does this differ from the suggestion made in https://github.com/parse-community/parse-server/issues/7292#issuecomment-803989620?

The current plan is to constantly allow the addition of deprecation warnings, then a release has to happen to make these warnings visible to everyone who upgrades. Then these warnings are displayed during one full major release cycle, I'm currently estimating 6 months or 12 months. Developers have (half) a year time to adapt their deployments. Then, on the next major release, the breaking changes are actually coming into effect. That guarantees that deprecation warnings are displayed during at least one major release.

We know in advance something might change

Good point, there is a PR in the works to centralize deprecation warnings. A contributor can easily see which parts of the code are deprecated by looking into the deprecation definitions.

[mtrezza]

For future discussion about release automation, we may use something like https://github.com/semantic-release/semantic-release. That would also define the branch structure to use. Still requires to compare the different auto-release approaches available.

[sadortun]

You guys might also want to have a look at GitHub Projects. You can have projects per Repo, and for the entire Parse organization if you need to organize/plan things for the future.

It can also give a clear view on what features will be included in the upcoming releases, and the status of theses releases.

https://github.com/features/project-management/

[mtrezza]

@sadortun Thanks for suggesting; we can definitely look into that once we have our releases structured. Maybe you have an idea about how we can use the kanban in a beneficial way?

[sadortun]

Kanban = 🚀

While we're at it, maybe a Discord server too ? :)

[mtrezza]

Feel free to join the Community Forum 🙂

[mtrezza]

After further looking into various release automators, semantic-release seems to be exactly what we need.

It offers a versatile, forward-looking branch structure that is especially useful to better manage breaking changes with pre-release versions. That means we can work on future branches that include breaking changes while still patching existing releases, all in a managed context.

I will go ahead and set up an example repo where we can get a feeling for how this could work for Parse Server. @davimacedo I will look out for monorepo support along the way.

[mtrezza]

@L3K0V

because you mention the Parse Server 5 and the big step forward. Is there something similar for the dashboard like Parse Server Dashboard 3?

If you mean release automation, eventually (ideally) yes. After Parse Server, the automated release process will likely be rolled out to other repos. No time frame for Parse Dashboard yet though.

[Moumouls]

@mtrezza you are right https://www.npmjs.com/package/semantic-release is what we need. I use it extensively on my project.
Also to make it work correctly, since sementic release can scan commits to increment version correctly, i can suggest to add Conventional Commit System.

We can setup automatic commit lint before commit hook with:

• https://www.npmjs.com/package/@commitlint/cli
• https://www.npmjs.com/package/@commitlint/config-conventional
  -https://www.npmjs.com/package/husky

Commit lint file example:

#commitlint.config.js
module.exports = {extends: ['@commitlint/config-conventional']}

In husky folder

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

yarn commitlint --edit 

[mtrezza]

Yes, we have a playground repo in which we'll refine the setup before applying it to Parse Server.

[Moumouls]

then it will force contributors to commit something like fix: undefined file or test(auth adatper): add spy

[Moumouls]

here also example of release rc that i use

{
  "branches": [
    {
      "name": "rc",
      "prerelease": true
    },
    "release-validated"
  ],
  "plugins": [
    "@semantic-release/commit-analyzer",
    "@semantic-release/release-notes-generator",
    "@semantic-release/github"
  ]
}

A special note about the github integration, it's nice because it add a tag released on xxxx on included PR, but also comments, so 2 points here:

• Cool because contributors will be notified when their PRs are included into a release
• Bad because reviewers/participants will also be spammed (and yes if you reviewed 40 PRs, you will receive 40 emails ahahha, if you have github email notification activated)

[Moumouls]

May be out of context:

I can also suggest to add automatic scope labeling on PRs via Github action Labeler

name: "Labeler"

on: pull_request

jobs:
  triage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/labeler@main
        with:
          repo-token: "${{ secrets.GITHUB_TOKEN }}"

Then we can have some label based on what is touched on the PR like:

• scope/graphql
• scope/adapters
• scope/db
• scope/auth
  etc...

[mtrezza]

@Moumouls Thanks, that is all planned (not to that granularity though), see the README in https://github.com/parse-community/release-automation-playground

[Moumouls]

thanks @mtrezza As usual it seems I'm a bit late 🙂

[mtrezza]

Not at all, good to know you have some experience in this, I will tag you when related issues come up. We will likely discuss these things in the playground repo discussions. Closed to keep discussion around specific issues.

[Moumouls]

okay @mtrezza i started a discussion on https://github.com/parse-community/release-automation-playground/discussions/1