Stacks.js: Package versions are out of sync

[yknl]

The release versions for the packages in the mono repo are out of sync. This could lead to bugs as there are internal dependencies. When updating and publishing a new package version, all of the packages should receive a version bump.

[agraebe]

@marcosc90 @reedrosenbluth could you chime in and see what the internal dep issues might be?

[reedrosenbluth]

I'm pretty sure this is how Lerna's fixed versioning is supposed to work. On publish, Lerna only bumps a packages version if it has been changed, or if another package in the monorepo that it depends on has changed. I believe the only case where everything gets bumped simultaneously is if there is a major version bump.

I've found the Lerna readme to be somewhat confusing, but the conversation in this issue is somewhat clarifying:

https://github.com/lerna/lerna/issues/1611

[yknl]

You can do it both ways with lerna. However, it is much less confusing for both users and maintainers if the versions are kept in sync. That way it's easy to tell at a glance if you are at the latest version and whether the versions are compatible with each other. It's as simple as running lerna version v1.3.5 from the repo when you create a release.

[reedrosenbluth]

Are you sure that Lerna's fixed mode is supposed to work the way you are describing? The documentation for the version command reads:

Bump version of packages changed since the last release

Additionally, in the issue I posted above, a maintainer writes:

To be clear, the default "fixed" versioning scheme does not publish all packages for non-major bump unless all packages (or their local dependencies) have changes since the previous release (annotated git tag).

If I'm not misunderstanding things, the purpose of fixed mode isn't to bump the version of and publish each package every time a change is made, but it's to maintain a single version that each package gets bumped to when it does require publishing, so that all of the packages stays relatively close, and that they are always on the same _major_ version.

Am I missing something?

[yknl]

My point is that it makes life harder for both the users and maintainers when version numbers are all over the place.

[marcosc90]

AFAIK the only way to bump all packages even if it's not needed is by using --force-publish

lerna publish patch --force-publish

Will bump all packages to the same version.

[yknl]

You should be able to do this:

lerna version v1.2.6
lerna publish from-git

That's what I used before to bump and publish all packages.

[agraebe]

@reedrosenbluth could you follow up with the latest conclusion? thanks!

[reedrosenbluth]

Closing this issue for now as I think we should continue to use Lerna in its default fixed mode, rather than using --force-publish to keep all package versions in sync. I think it's best to only publish new versions of packages if they have been updated.

[yknl]

Is there any reason other than it being the default mode? It really creates confusion for developers using the packages when the versions are all different. For example, it's difficult for a developer to know whether @stacks/[email protected] is compatible with @stacks/[email protected]. Other maintainers and users of this repo should chime in as well. cc @hstove @aulneau

[kyranjamie]

I'd also advocate for aligned package versions. While it creates extra versions bumps with no changes, there's no cost to this.

Predictability: Ken's point. Inter-package incompatibilities will be introduced. Aligned packages remove any uncertainty.

Ease of install/update: I want to be able to run yarn add @stacks/{transaction,network,keychain}@1.3.38 to target a specific version.

There's a precedent for doing this: Angular, Babel, Storybook etc.

[reedrosenbluth]

If a change has been introduced to a stacks.js package that is breaking, or introduces a inter-package compatibility issue, it justifies a major version bump right? When using lerna to release a new major version, default behavior is to bump the major version of _all_ of the packages in the monorepo, keeping their major versions in sync.

Shouldn't this be enough for developers to avoid compatibility issues? They simply need to make sure they are using stacks.js packages with the same major version.

@kyranjamie as far as I can tell, babel manages their packages versions in this way: https://github.com/babel/babel/tree/main/packages.

[marcosc90]

I agree with @reedrosenbluth

I don't think a version should be increased if there's no change at all to that package, it would be confusing for users only using a single package IMO. Bumping the version for all packages should only happen when a major version is released, and this scenario is already happening.

[aulneau]

I don't think a version should be increased if there's no change at all to that package, it would be confusing for users only using a single package IMO. Bumping the version for all packages should only happen when a major version is released, and this scenario is already happening.

I am in this camp, although I can see both sides.

I personally prefer to only update versions if there are changes in the package, or updates to peer deps.

[CharlieC3]

I don't regularly contribute to this project, but want to offer my 2¢. Personally I'm inclined to agree with @aulneau and @marcosc90 and think the versions should _not_ be incremented all together. If there's confusion around compatibility between package versions, in my experience that usually hints at a different root problem. Aligning all of the versions to increment together regardless of changes made feels more like a band-aid.

Instead, dependency versions can be made more specific in each library's package.json to include lower _and_ upper version bounds for their dependencies.

For example, we can get as specific as we want here:

{
  "dependencies": {
    "foo": "1.0.0 - 2.9999.9999",
    "bar": ">=1.0.2 <2.1.2",
    "baz": ">1.0.2 <=2.3.4",
    "boo": "2.0.1",
    "qux": "<1.0.0 || >=2.3.1 <2.4.5 || >=2.5.2 <3.0.0",
    "til": "~1.2",
    "elf": "~1.2.3",
    "two": "2.x",
    "thr": "3.3.x",
  }
}

If we don't specify upper version boundaries for internal dependencies, then those projects may be prone to incompatibility bugs. Including those upper version boundaries would help avoid this, but the downside would be slightly more overhead when updating versions. But from my perspective that's an inevitability I would expect in a sophisticated ecosystem.

However, it is much less confusing for both users and maintainers if the versions are kept in sync. That way it's easy to tell at a glance if you are at the latest version and whether the versions are compatible with each other.

I agree it would be easier for users to understand if they're using the latest package available, but I don't agree that if all the versions line up it implies they're all compatible with each other. If an incompatible change were made to one package while unknowingly breaking compatibility with a dependent package, they would still all be incremented together when released. I feel it would be even more confusing if all versions incremented together, and find one of those packages is incompatible with its version-aligned dependents.

[yknl]

If a change has been introduced to a stacks.js package that is breaking, or introduces a inter-package compatibility issue, it justifies a major version bump right?

I don't think that is true. In my experience working on this monorepo, there are many instances where a small change in one of the packages requires a dependent package to also be updated. However, no breaking change has occurred from the user's perspective. Is that deserving of a major version bump?

Shouldn't this be enough for developers to avoid compatibility issues? They simply need to make sure they are using stacks.js packages with the same major version.

As a developer, I don't want to spend time checking individual @ stacks package releases to make sure they're at the right version. It makes life much easier if I see that the latest version for 1 package is v1.3.5, I just bump every package to that version. This makes a difference for developers of apps with high security requirements like a wallet. All dependency versions are usually locked since bumping might trigger an audit.

If an incompatible change were made to one package while unknowingly breaking compatibility with a dependent package, they would still all be incremented together when released. I feel it would be even more confusing if all versions incremented together, and find one of those packages is incompatible with its version-aligned dependents.

The idea is that if the versions are in sync they are assumed to be designed and tested compatible. Of course you can't guarantee that just like you can't guarantee there aren't any bugs. But in most cases, this makes it easier for developers to debug.

[diwakergupta]

Hi all, thanks for your feedback. However I worry that we're now venturing into bikeshed territory.

In the interest of everyone's time, let me try to get to some closure here. I don't expect agreement on all the points, but that's the beauty of open-source and permissive licensing -- folks can make different choices if they wish to:

• I think we're all aligned that version numbers should convey meaningful information. If there are breaking changes, versioning should indicate that. To Ken's point, it would be great if version numbers could also indicate if packages within a monorepo are compatible with each other.
• IMO none of this is incompatible with the idea that changes in one package should _not_ automatically require version bumps in other packages. For instance, if package A goes from 1.2.1 to 1.2.2 and this causes package B 1.2.1 to break, then per SemVer, package A should actually have been version 2.0.0, and have a similar cascading effect on all downstream dependencies. All that to say, a convention that sub-packages in a mono-repo that share the same major and minor version (but might have differing patch versions) are guaranteed to be mutually compatible seems entirely reasonable to me.
• Finally, I don't think how/where a package happens to reside should dictate it's versioning. For instance, tomorrow it may make sense for BNS library to live outside stacks.js; we should strive to have a versioning scheme that stay consistent through such transitions.

I'm going to close out this issue. @yknl if you feel strongly about this, let's open a discussion item instead (I'm enabling discussions on this repo as well) and solicit feedback from the broader community.