Three.js: Increasing Three.js's popularity via reducing API changes with semantic versioning?

[bhouston]

So since I started contributing to Three.js back in November, I've seen a number of breaking API changes occur such as the change of function names in the Math library (multiplyMatrix -> applyMatrix4, addVectors -> add, etc.) These are all improvements but they break code compatibility with pre-existing applications that are using Three.js.

I think that as Three.js gets more popular we need to almost promise that the existing interface of the library won't change between minor version increments, rather only functions are added and bugs fixed. The interface only changes between major version number increments. This way if I am using say version 1.56, I can upgrade without any worries to 1.57. But if I tried upgrading to 2.57 from 1.56, I need to assume that I will have to update my code in my application.

I think that introducing this will make it easier for other people to build on top of Three.js more reliably as right now basically you can not upgrade Three.js without something probably breaking.

I would suggest adopting major version numbers to signify when interfaces have a breaking change. And minor version numbers for just adding new features and fixing bugs.

I'd suggest that we come to an agreement as to when we will introduce the first major veresion number and work to solidify the interfaces for that and then hold them fixed for as long as possible after that. Major libraries tend not to increment their major version numbers more than once a year and sometimes they can go for years without breaking compatibility.

So there is this great document that outlines the "theory" of semantic versioning, which is basically what I am advocating but in a simplified form:

http://semver.org/

It may be possible to hide a lot of Three.js's functionality in closures and if it is in closures we can change it as needed without breaking the public API.

[mrdoob]

Well, the last few revisions have been specially breaking as we have changed core stuff but that has been pretty exceptional.

I totally understand the benefits of such approach, but I don't think we're ready for that yet. The API still needs some design work and managing different versions would get in the way (time/motivational-wise).

[zz85]

i think of three.js version numbers like the infinite versioning except that api breaks, yes..

the incentives why users would want to upgrade would be to get the latest and "good-est" from the library. its still some sort of pain though, thats why its always a good idea to keep a copy of three build used for a particular three.js project you've worked on.

the migrate wiki helps, but its not perfect. years ago, i follow the development of lilypond, which introduces breaking changes with their improved syntax. what they have is a format converter, which takes files with old format and convert it version by version to the latest.

implementing this might potentially help, but with its challenges too. somewhere you might need to annotate or specify the current threejs build, and it probably needs to parse js and html. but it might be useful as a script or service (along the likes of online js minifiers or compilers)

[evillemez]

Even if you use Semver (definitely a good idea), technically anything below version 1.0.0 means the API is by definition unstable, and can change any time. There are already tagged releases, so you could start by translating something like r56 to 0.5.6. API changes would increment the middle number, bug fixes/patches would increment the last number. So, you could end up with 0.12.18, if you need to.

Eventually, once the community seems to think it's stable, you can start serious discussion around a 1.0.0 tag. Until then, you can still use semver and break the API whenever you want. At least this way it would be clear when the API changed, and it wouldn't really be much extra effort. Simply tag a release when the project is in a working state.

[bhouston]

The semver.org proposal is not about adding management overhead of multiple releases, but is simply managing communications with the end users.

If @mrdoob does think the library is pre-1.0, then putting a release version of 0.XX.YY will communicate that. And incrementing the minor version number (XX) for every breaking change will help communicate that there are breaking changes while the release number will continue to increment for every release (YY).

Adopting a versioning strategy pre 1.0 will also help users trust and upgrade their use of three.js in a post 1.0 world.

Almost every other incredibly popular library uses a similar type of versioning system.

[bhouston]

@evillemez, I think I just wrote what you wrote in exactly the same language. Oops.

[bhouston]

@zz85 I think that format converters are acceptable for data files, but I never seem anything that modified source code used in a serious fashion.

[WestLangley]

I expect that every release so far has broken the API, so we have been effectively doing what is proposed here.

We can make the next release 0.57.00, and everyone can be happy.

[zz85]

maybe, but there's this thing called meta-programming which is using code to modify code.

I never seem anything that modified source code used in a serious fashion.

I'm not a fan of semantic versioning here - it feels too traditional software engineering and feels like there's too many numbers to remember.

Apart from having an API that changes, I wouldn't know if I'd consider three.js alpha or pre-1.0, I think it's been stable enough to be used in production, it feels funny to be reseted to a sub-zero version.

On other hand, I'm actually like the way the current versioning is done, its incremental and simple. Even Firefox has taken the cue from Chrome and accelerated its version numbers, I think we would be fine even with "integer" version numbers (in fact, I remember there were discussions within the Firefox project to eliminate its version numbers).

[mrdoob]

We can make the next release 0.57.00, and everyone can be happy.

I think 0.0.57 would be even more accurate ;)

The side effect of this change is that this will create a lot of branches. People will expect bug fixes on old versions and that means branches and making maintaining harder (thus nulling my energies for actual development).

[bhouston]

@mrdoob, I will think about this and look at some other projects and how they do it. I really do not want to put any additional load on you, I actually don't know how you do as much as you already do, and you do it so well, it is phenomenal.

@zz85 I think comparing to Firefox is a bit misleading as it isn't a library, rather it is a consumer facing version number in that case. Firefox's JavaScript library and other related tools (WebGL, Canvas, etc.), is actually very tightly controlled and they work hard never to break backwards compatibility and they do have version numbers. We don't want to go the committee route though, it only for fundamental standards. I think Three.js is more comparable to incredibly popular libraries like jQuery and nodejs and they do have version numbers, although they have team running them with a company or foundation backing them.

[evillemez]

@bhouston @zz85

I agree that Firefox isn't the best example for the same reason - it's an end-user app. Three is a library that other applications are increasingly depending on. A huge part of maintenance is being able to manage dependencies as painlessly as possible, and lots of packages managers already recognize the semver format.

[zz85]

Gmail had been for a longest time in beta, erms.. it changed the way how beta was seen and my point just being that we don't always have to follow what others do.

If we look at Facebook API, which I used since it started, it has always been constantly revolving, so good and bad - old code breaks but on Facebook side its probably better for them to depreciate old API.

Understand where you guys are coming from... and here's my favorite phrase from Juliet / Shakespeare

"What's in a name? That which we call a rose
By any other name would smell as sweet."

So I'll leave names in @mrdoob's hands, as long as we are still in love three.js ;)

[bhouston]

@zz85. The Facebook comparison is in my opinion somewhat problematic. People don't use Facebook because of its API, people use its API because facebook has a billion users. Facebook's API is horrible but because people need to make money via Facebook we all put up with it. Here are great articles about Facebook's horrible API practices:

http://techcrunch.com/2011/08/11/facebook-wins-worst-api-in-developer-survey/
http://blog.nik.me/post/30400168363/dont-like-facebooks-api-wait-five-minutes
http://www.sethcall.com/blog/2010/09/30/facebook-api-does-not-care/
http://techblog.bozho.net/?p=377
http://blog.zuupy.com/3-reasons-that-we-are-moving-away-from-facebo

[mrdoob]

@mrdoob, I will think about this and look at some other projects and how they do it. I really do not want to put any additional load on you, I actually don't know how you do as much as you already do, and you do it so well, it is phenomenal.

Thanks! :_)

[PaulTondeur]

I can't understand why this issue is waved off and the project won't stick to very useful conventions. Semver is something that works really nicely and this project could easily make use of this.

At this moment I'm helping a friend out transforming his project into something that uses both bower and grunt. It is a pain in the butt because three.js doesn't support semver and we can not download the version we need using bower. Installing through bower should be easy, which is now clearly not. If semver is not used, this project would be better off unregistering for bower as it is not compatible with its specifications.

Well, enough with the rant. :-) I would like to offer my help and make a pull request so that this project will be compatible with bower. I even would like to help setting up a nice grunt script for compilation, but also tagging on github. This makes it very easy to maintain for you. I also think bower support would be useful for many.

But before I spend my precious time on it I would like to know if you are willing to accept such pull request.

[bhouston]

I adopted a partial semver for npm three publishing here: https://npmjs.org/package/three My npm publishing script was accepted into master.

[PaulTondeur]

I know and have seen this. But NPM is for node dependencies. Bower is for frontend dependencies. As three.js will run on the frontend and has nothing to do with node I don't think it is a real solution to the matter.

It would be the same as making a ruby gem out of it, or download it by using composer.

As much as I appreciate the initiative, I don't think this is the way it has to be solved. It's a workaround and too much of a hassle. It should just work like it does for all other popular javascript libraries out there.

[bhouston]

I'm not saying that npm is a solution, but rather the way that I created a semver for three.js. Basically even though npm isn't officially supported by ThreeJS I made a solution that allows us (the creators of http://clara.io) to use ThreeJS via NPM quite effectively. I am suggesting you do the same for a parallel Bower deployment. :-) I'm not saying use NPM.

[PaulTondeur]

Ah right! Yes that might be a workaround and a last resort solution, but it will be a big hassle to set this up. While the solution should just be plain easy. Thanks for the idea though :-)

[SpainTrain]

:+1: for semver

I think I understand @mrdoob 's hesitation, but consider also that library users would rather focus on development than dependency management. All you need is awareness of the type of change(s) you are releasing and a release script that does all the _grunt_ work for you (e.g., https://github.com/vojtajina/grunt-bump) :wink:

[PaulTondeur]

Exactly, grunt-bump would be a great way to automate this and take the pain away. I also understand that it will be more relevant to focus on core features instead of dependency management.

I don't know how the build is currently made and put alive. But I certainly believe that automation for this would be the way to go. Grunt-bump would be part of that.

My offer stands to set this properly up and I even would like to write up how releases with grunt for this should be made. But before I dedicate some of my valuable time to this I need to know that this will be accepted if I deliver something that is ready to be used and implemented.

[mrdoob]

I don't know how the build is currently made and put alive.

Have a look at utils/build.

[PaulTondeur]

Thanks, just had a look at this and it seems doable to make releases with grunt instead and automate pushing & tagging on GitHub. As soon as I have time I'll create a fork and ultimately make a pull request. To be continued.

[bhouston]

@PaulTondeur: This is great. I do not know grunt, but if you can add publishing to the Three NPM repository using the script I contributed, that would be excellent. I can turn over the Three NPM to MrDoob or whatever needs to be done to allow an automated script to publish to the Three NPM repository.

BTW if you are automating things, one could also run the unit tests run as well to ensure correctness before publishing.

[PaulTondeur]

I've create a pull request which builds using grunt and uses semver. Once this pull request is merged, lets look into automatically publishing to the NPM repo too. I've never done that thus far, but it should be trivial to add.

[SpainTrain]

@paultondeur yes, npm exposes it's cli commands via API. So just include
the npm package itself as a devDependencies and create a grunt task that
requires npm then calls npm.commands.publish()

On Sunday, November 3, 2013, PaulTondeur wrote:

I've create a pull request which builds using grunt and uses semver. Once
this pull request is merged, lets look into automatically publishing to the
NPM repo too. I've never done that thus far, but it should be trivial to
add.

—
Reply to this email directly or view it on GitHubhttps://github.com/mrdoob/three.js/issues/3102#issuecomment-27648003
.

-- Sent from mobile device

[trusktr]

Being informed of breaking changes or backward compatible changes via semver would be great!

[ghost]

I'd suggest that on NPM the next release should be 75.0.0 instead of 0.75.0, as capturing the meaning of semver is more important than the number itself, IMO. Then, non-breaking additions to the API would lead to 75.1.0, 75.2.0, etc, and simple patches would increment the last number, as in 75.2.1.

Or, it can even just start at 1.0.0 and go from there. I think having semver is ultimately more practical than keeping the first number small, if that's what you're worried about, but I think starting at 75.0.0 captures the essence and history of previous versions of three.js while benefiting from what semver offers.

[mrdoob]

Everyone always talk about the benefits of semver but no one talks about the complications it brings. People suggesting using it tend not to be maintainers themselves.

On paper semver is beautiful, in reality it makes the maintaining job much harder.

[ghost]

People suggesting using it tend not to be maintainers themselves.

I use semver in everything I maintain.

Everyone always talk about the benefits of semver but no one talks about the complications it brings.

The benefits for the library's end users far outweigh the downsides for the library developers, IMO.

it makes the maintaining job much harder

Not really, because you're already writing detailed change logs, which are most of the work. Using semver is merely adding a very short summary of those change logs.

Cheers! :]

[ghost]

Plus, platch bumps don't need a change log really. You could also just skip patch bumps.

[ashconnell]

As great as semver is, it's almost not needed at the moment as each release always has breaking changes. The grand sum of effort to implement would result in being in the exact same place as we are now

[mrdoob]

People suggesting using it tend not to be maintainers themselves.

I use semver in everything I maintain.

Oh! Which projects are those?

Everyone always talk about the benefits of semver but no one talks about the complications it brings.

The benefits for the library's end users far outweigh the downsides for the library developers, IMO.

What are the downsides from your point of view?

it makes the maintaining job much harder

Not really, because you're already writing detailed change logs, which are most of the work. Using semver is merely adding a very short summary of those change logs.

Yeah, now multiply that by 3 and add the time of backporting fixes, and adding yet more items in the change log.

[ghost]

Oh! Which projects are those?

Not any as big as Three.js. I don't know why infamous has hundreds of downloads. It's definitely not ready yet, nor is there any easy-to-find examples or documentation. Thanks for the inspiration. ;)

What are the downsides from your point of view?

Having to keep track of whether changes are inert packages that don't change the public API (patch), changes that add to the API (minor), or changes that break existing API (major). That's probably it for me. I have bash shortcuts to do the actual version bump, push, and publish.

Yeah, now multiply that by 3 and add the time of backporting fixes, and adding yet more items in the change log.

Well, you don't have to do that. You can simply bump the major version if you want, if that's easier for you (I would be totally fine with that): 75.0.0, 76.0.0, 77.0.0, etc. Here I'm simply suggesting the use of semver, but I'm not suggesting how the code base should be developed (f.e. keep backwards compat, or not). Then you'll be conveying the same messages as you do currently with versions like r75, r76, r77, etc, but in a fashion adopted by the community. In my case, infamous, which is not entirely ready for production, is already at version 3.x.x, and will probably be much higher before it has docs, examples, and recommendation for production. I just find doing versions like that more meaningful based on what semver versions mean. I don't see the sentimental value some people have for 1.0.0 meaning "rock-solid for production".

[ghost]

TLDR: By looking at a semver version, I can tell whether I should update or not with a conventionally defined amount of confidence, without necessarily having to read a change log. Patch bump? Probably update to fix a bug, code should still work. Minor bump? I can update, and I know there's new stuff if I care to go read the changelog. Major? I probably don't need to update the current project unless I really want the breaking change, and I can possibly update when I move to a new project instead, as this could possibly be the most time-consuming type of update. patches, less time; major, most time. Those clues are helpful.

[mrdoob]

Having to keep track of whether changes are inert packages that don't change the public API (patch), changes that add to the API (minor), or changes that break existing API (major).

Those are problems that we don't have at the moment. If we want to keep up the development we should find ways of reducing problems, not ways of having more problems.

Right now, making sure everything woks in all the GPUs out there is hard enough.

I don't see the sentimental value some people have for 1.0.0 meaning "rock-solid for production".

Philosophical talks about release numbers? That doesn't sound very appealing...

[ghost]

Yeah, you're right!! I'm unsubscribing before I get pulled back in. Use any versioning system you want. ;)

[cronvel]

I have to respectfully disagree, because I don't see why SemVer would add more "problems" or "work".

SemVer is not about maintaining and fixing older/deprecated releases branch, it is JUST about giving your releases meaningful numbers.

It also helps with registry like NPM and dependency management, especially because upgrading a dependency is not always done by a human.

How does it cost: taking few minutes to think if the new release contains improvement and bugfix (increase patch number), contains new features but is backward-compatible (minor number), or contains backward incompatible changes (major number).

SemVer (by itself) never forced anyone to do complex branching. You can continue using a linear workflow (e.g. tag master with the appropriate numbers once in a while).

[mrdoob]

It also helps with registry like NPM and dependency management, especially because upgrading a dependency is not always done by a human.

npm package is already released as 0.75.0.
https://github.com/mrdoob/three.js/blob/master/package.json#L3

How does it cost: taking few minutes to think if the new release contains improvement and bugfix (increase patch number), contains new features but is backward-compatible (minor number), or contains backward incompatible changes (major number).

If that's how it works... It should be 75.0.0 to avoid disappointments.

[trusktr]

Okay, okay, I'm back one last time.

If that's how it works... It should be 75.0.0 to avoid disappointments.

True, to avoid breakage in automatically updated apps as @cronvel alluded to,but even more importantly, so that users updating three.js automatically can enjoy the latest non-breaking changes without having to update package.json manually!

Note, by default, the npm install --save command will write a version to package.json in the form ^x.x.x using the ^ symbol, which effectively means "update to a new version only if the new version is backwards compatible". This is the semver contract. Automatic updates should, in theory, not break an app based on the limitations imposed by the ^ symbol.

Now, since three.js is using 0.X.0 format for all releases currently, then, an automatic update won't break an app because of the leading 0. prefix. In fact, nothing will happen at all, hence no breakage. But, the downside is that if the change from 0.75.0 to 0.76.0 is non-breaking and backwards compatible, for example, then an automatic update will fail to update the user's version of three.js to that awesome new version.

I also wanted to add that @bhouston's title of this issue is misleading. This issue is not about

Three.js's popularity

or

reducing API changes

whatsoever.

The issue is simply about numbering as @cronvel mentioned, to help end users have the best package management experience (as far as "un-appealing" theories in that field go today).

A more appropriate title would be "[idea] Use semantic versioning to help people avoid broken apps when automatically updating NPM packages."

Three.js' popularity stems from what the library itself does, not from it's versioning, not from it's lack of API changes. If the API must change to make the library more powerful, more performant, easier to use, etc, then definitely, please change it (with the proper number).

---

---

This topic is boring compared to writing 3D engines and graphics. Nonetheless, the topic is important, not just boring.

_Unsubscribed again, as what was on my mind is free, and I want to focus on other more interesting things!_ :smile:

[bhouston]

I think we are unlikely to revisit this in the near term. I'm going to focus on making ThreeJS better in terms of features. I think this is an issue that is pretty easy to work around, at least we find it to be in our extremely deep usage of ThreeJS.

[tbriley]

just happened across this issue. my $.02: in my experience coming from games development, with 3rd party engines generally you lock in the version you're using on a project early on and don't upgrade unless a release includes a feature or bug fix that specifically solves a problem you're facing. am slowly realizing the web is a very different place in this regard, but still the current release strategy makes total sense to this user.