Node: Changing Definition of "Locked" for Stability Index in API Docs

[MylesBorins]

As discussed in last weeks @nodejs/ctc meeting

Currently our api docs have a stability index with fairly conservative definition of Locked

Stability: 3 - Locked
Only bug fixes, security fixes, and performance improvements will be accepted.
Please do not suggest API changes in this area; they will be refused.

This has been fairly useful when discussing changes to APIs that are potentially breaking, such as any change to the module system. With changes coming to the ecma262 standard on a yearly basis now, we find ourselves in a slightly different situation then when this was originally drafted. As the language changes we may be required to change locked APIs, such as assert or module, in order to accommodate the changes to the language.

It can be argued that this is already covered by bug fixes, but that is somewhat ambiguous, and it would be better to be explicit about it

A suggested alternative text

Stability: 3 - Locked
Only bug fixes, security fixes, and performance improvements will be accepted.
Please do not suggest API changes in this area; they will be refused. Locked APIs
are subject to change to support new language features that are added to the ecma262
specification.

Thoughts?

[targos]

Looks good to me, with s/ecma262/ECMA-262/.

[Trott]

I have mixed feelings.

I think the last sentence seems to contradict the first two sentences (with an emphasis on _seems_).

I also wonder if the new sentence is more directed at ourselves than a typical contributor. Like, instead of "It can be argued that this is already covered by bug fixes", I would say that is in fact the plain reading of the text. I think the whole "will reject attempts to accommodate actual language features" was a result of us not adhering to that text and instead trying to use Locked as this strange we-get-to-have-it-both-ways "no, we're not deprecating this, but it's not for you and you shouldn't use it" statement.

Maybe the actual Locked text can be left alone in all its wonderful brevity (and in fact, if anything, it could be made more brief by simply removing the second sentence as it is redundant) and we can instead add this additional information as explanatory text that isn't part of the ends-up-in-a-bannerish-box-in-the-docs text:

Stability: 3 - Locked
Only bug fixes, security fixes, and performance improvements will be accepted.



Changes to support new language features that are added to the ECMA-262
specification are considered bug fixes and are therefore permitted in Locked
APIs.

EDIT: That would be in document.md in the section that lists/explains the stability indexes.

[bnoordhuis]

I like Rich's suggestion.

[jasnell]

I'm not that comfortable classifying all changes to support new language features as bug fixes. Some of them are flat out API changes. For instance, if TC-39 adds a new method to Uint8Array, then that becomes a new API feature of Buffer. It's not a "fix". I would word it differently, then:

Changes to support new language features that are added to the ECMA-262
specification are handled independently of this stability index and are permitteed
in Locked APIs.

[Trott]

@jasnell suggested:

Changes to support new language features that are added to the ECMA-262
specification are handled independently of this stability index and are permitteed
in Locked APIs.

Remove "handled independently of this stability index" (does not impart any useful information, just raises questions and muddies the otherwise clear meaning) and you've got something good, I think:

Changes to support new language features that are added to the ECMA-262
specification are permitted in Locked APIs.

[rvagg]

+1 although I'd go with "may be considered" rather than "are permitted"

[ChALkeR]

Not sure about this (the part that limits the allowed changes to ecma262-induced).
As this looks to me, the proposed wording does not cover changes in #10282.

It also doesn't seem to cover other changes that landed to «Locked» modules not so long ago, e.g.:

• #3374, #668 — error message changes in assert and/or timers,
• #639 ­— deepStrictEqual in assert,
• #4362 — timers: Fail early when callback is not a function,
• #5950, #5689, #1812, #1363 — module loading changes in modules.

_Note: the list above is outdated, there are probably newer ones._

[Trott]

Not sure about this (the part that limits the allowed changes to ecma262-induced).

I don't think there's any language that limits anything to ecma262-related stuff, or at least that's not my intention. It's supposed to explicitly allow ecma262-related stuff, but not necessarily limit it to just that.

Perhaps that's an argument for leaving it out entirely and just going with:

Stability: 3 - Locked
Only bug fixes, security fixes, and performance improvements will be accepted.

We decided at the CTC meeting that things like "make assert.deepStrictEqual() work as expected with Maps and Sets" could be considered a bug fix (on a case-by-case basis) but we don't necessarily have to get into that level of detail in our docs about the Stability Index.

[ChALkeR]

@Trott

We decided at the CTC meeting that things like "make assert.deepStrictEqual() work as expected with Maps and Sets" could be considered a bug fix (on a case-by-case basis)

What I am talking about is that assert.deepStreetEqual landed as a brand new feature while assert module had already been «locked» for some time. (That is one example, but there are more of major/minor changes to those modules in the git log).

That is either a bug in the process of accepting patches to those three modules (modules, timers, assert) or a bug in the documentation of that process. If the former, then no such pull requests should have been accepted at the first place, and shouldn't be accepted in the future. If the latter, the stability index documentation needs a more severe change that covers the way how we have been actually dealing with accepting patches to those modules in a clear way.

Now the actual process breaks users expectations, as it significantly differs from what is documented — we can't keep saying that those modules accept only bugfixes and keep accepting semver-major and semver-minor level changes there, we have to choose something.

[ChALkeR]

I personally think that with #7964 in place, we could move away from the «Locked» concept entirely (re-label those three modules as Stable).

And that would match with how we have been actually dealing with patches to those modules.

[Trott]

That is either a bug in the process of accepting patches to those three modules (modules, timers, assert) or a bug in the documentation of that process.

IMO, it's the former.

and keep accepting semver-major and semver-minor level changes there

I don't think that's the plan.

I personally think that with #7964 in place, we could move away from the «Locked» concept entirely (re-label those three modules as Stable).

Works for me if I'm wrong and we do plan on accepting semver-minor and semver-major on currently Locked APIs.

[gibfahn]

That is either a bug in the process of accepting patches to those three modules (modules, timers, assert) or a bug in the documentation of that process.

IMO, it's the former.

@Trott so you're of the opinion that things like assert.deepStrictEqual() shouldn't have been added?

---

Given the definition of <Stable>:

The API has proven satisfactory. Compatibility with the npm ecosystem
is a high priority, and will not be broken unless absolutely necessary.

I'm not sure what <Locked> really adds, are we saying we won't break compatibility even if absolutely necessary? Or only if really really absolutely necessary?

Or are we saying that we won't add new APIs either? If so what is the rationale behind that? Not breaking things seems somewhat orthogonal to adding new APIs.

_EDIT:_ It seems Locked hasn't really changed since it was added in https://github.com/nodejs/node/commit/d71f9944cc5bf485287442707bf31f0deb8b252b (Feb 2012)

---

I think the whole "will reject attempts to accommodate actual language features" was a result of us not adhering to that text and instead trying to use Locked as this strange we-get-to-have-it-both-ways "no, we're not deprecating this, but it's not for you and you shouldn't use it" statement.

@Trott Does locked mean kinda-deprecated? I just thought it meant even-more-stable.

[Trott]

@Trott so you're of the opinion that things like assert.deepStrictEqual() shouldn't have been added?

I'm under the impression Locked is supposed to preclude new methods, so yes, that is my opinion.

@Trott Does locked mean kinda-deprecated? I just thought it meant even-more-stable.

For assert, it's been used as kinda-deprecated. I think your even-more-stable-than-Stable interpretation would be accurate for how it is used on module. ¯\_(ツ)_/¯ I think if we wanted to treat assert as even-more-stable-than-Stable, that would work for me. And we've already moved in that direction by removing the not intended to be used as a general purpose assertion library text in the doc. :ship: ⛵️ : <- My attempt to use emoji to say "that ship has sailed".

I'm all for simplifying. If the consensus is that Stable is all we really need and we should get rid of Locked, I'm +1 on that. The fewer categories in the stability index, the better. Or, at least, that's true if we're violating the stability index with some frequency, which sure seems to be the case based on what @ChALkeR has posted.

[sam-github]

I like the idea of less stability levels.

In the distant past, the intention of experimental was, I believe, to say that the feature could change in the next release (at that time, a minor jump from 0.x to 0.x+1). But that would imply that semver-major changes happen to experimental, but AFAICT, we treated the "experimental" APIs with as much care as the "stable" APIs, because apps depended on them.

It looks like some attempt was made to retroactively define "experimental" as meaning "you have to turn it on with a command line option, and we really do feel free to change this in the next major, you've been warned", see https://nodejs.org/api/documentation.html#documentation_stability_index, which seems more protective.

But these experimental APIs uses don't match that definition:

• https://nodejs.org/api/all.html#os_os_platform, is this even an API? What does experimental mean for it? I think Android should be linked to our supported platform list/policy, not to the "experimental" stability index.
• https://nodejs.org/api/all.html#documentation_json_output, not hidden behind a CLI flag, or even an API per-se, and we've had it since 0.6, should just have the stability removed and the docs left otherwise alone (its not an API).
• https://nodejs.org/api/all.html#url_the_whatwg_url_api is not hidden behind a CLI flag, it should be marked stable, AFAICT, or should have been hidden behind a CLI flag if it was supposed to be experimental

The fourth current use is https://nodejs.org/api/all.html#debugger_v8_inspector_integration_for_node_js, which is actually hidden behind a flag, so fits definition of experimental.

There are no other uses, leaving it pretty close to droppable.

Locked was intended to discourage feature requests. I don't know if it has done that, and it is only used for timers, assert, and module. It might be better to rename them to stable, and just add a note in the docs about why we aren't interested in feature additions, or even a node FAQ or pre-canned response, which allows us to add any API that we, the people writing nodejs unit tests, would find useful without getting yelled at, but still reject requests to make it better for non-node core unit testing.

[sam-github]

Also, note that two of the experimentals are tracking upstream deps over which we have no control, and that we will follow whatever they do (v8 debug protocol and WhatWG URL spec).

Also, the inspect behaviour is a flag, so whether that counts as being "behind a flag" (as the experimental stability requires) or not is debatable.

I think we should delete both experimental and locked, in case my conclusion wasn't clear.

[Trott]

As a first step, I'm going to see if there's general support for making assert Stable instead of Locked.

If that seems to work for people, we can talk about timers and module as well and see about perhaps getting rid of Locked altogether.

But if we don't have the will to do it for assert, then that's a non-starter and we're stuck with Locked, I think.

[joyeecheung]

@sam-github I am not sure about the WHATWG URL implementation being becoming stable, it's not behind flags but do we mean to keep it exposed as require('url').URL and require('url').URLSearchParams? @nodejs/url

For general usage it's pretty spec-complaint at this point AFAICT, minus some edge cases, but then browsers are not 100% spec-complaint either, so it doesn't need a flag to discourage broader usage IMO.

[sam-github]

@joyeecheung I don't have any comment on the stability of the WhatWG spec, maybe they do allow breaking changes.

However, url.URL is described as Experimental in the docs, see https://nodejs.org/api/url.html#url_the_whatwg_url_api, but the definition of Experimental from https://nodejs.org/api/documentation.html#documentation_stability_index is

This feature is subject to change, and is gated by a command line flag. It may change or be removed in future versions.

By that definition, url.URL _cannot_ be regarded as "Experimental". The other options are Locked, Deprecated, and Stable. I don't think its Locked, and its definitely not Deprecated, so it must be Stable. Or the definition of "Experimental" is wrong, take your pick!

[joyeecheung]

Sorry, I was a little bit vague, I mean the URL implementation is sort of between "Experimental" and "Stable". The API is not controlled by Node.js and it doesn't fit the description of "Stable"(not "proven satisfactory" and still accepts "nice-to-have" but not "absolutely necessary" edge-case breaking changes), it is experimental at this point AFAICT(perf and spec compliance still has work to do) but in practice it's not behind a flag. I think it's about whether we should let the stability index reflect our practices or the other way around.

[joyeecheung]

Also I think URL still fits in the description of Experimental if the "gated by command line flag" part is loosen up a little? And this won't be a question if we want to leave the experimental status in Node 8? (Not very sure about the timeline though, maybe other people from @nodejs/url has more clue than I do)

[sam-github]

Except that historically, introducing an API means it cannot be changed without breaking code, and people scream when their code breaks, the fact that having said we reserve that right doesn't make them feel better.

The putting it behind a flag means that it is actually impossible for a node module to depend on this feature unless the ultimate consumer of that module specifically passes a flag to node when they invoke it, which is a fairly strong indicator of willingness to live on the edge. It means both the author of the js code depending on the experimental feature, and the consumer of the js code explicitly indicate their willingness to use experimental code. Otherwise, its easy for someone to accidentally use an API and not notice its experimental, and its almost impossible for a consumer of a module to know that happened.

In the absence of the passing of a command line flag, I don't think we should be breaking our APIs - everything should be treated as 'stable', and we should go through a deprecation/warning cycle if we want to introduce a breaking change.

[gibfahn]

So we can:

1. Put URL behind a command line flag.
2. Treat it as a stable API
3. Change our stability index to introduce a new category for URL.

Is there a downside to 1.? Presumably it should be a bugfix rather than a semver-major.

[ChALkeR]

@gibfahn No, removing url.URL or putting it behind a flag now looks semver-major to me.

As for the stability indexes — looking at what we have now and how we used that, I suppose that we need only Deprecated (for soft/hard-deprecated things), Experimental (for flagged things), and Stable (for the rest). I'm not even completely sure if we should keep calling that «Stability index», as everything which is not marked as deprecated or experimental should be treated as Stable. It doesn't hurt, though =).

[ChALkeR]

@sam-github

https://nodejs.org/api/all.html#os_os_platform, is this even an API? What does experimental mean for it? I think Android should be linked to our supported platform list/policy, not to the "experimental" stability index.

It doesn't look linked to the «Experimental» stability index in the docs or directly related to it. Platform support is another question from the API stability indexes.

https://nodejs.org/api/all.html#documentation_json_output, not hidden behind a CLI flag, or even an API per-se, and we've had it since 0.6, should just have the stability removed and the docs left otherwise alone (its not an API).

Perhaps deserves a separate issue.

https://nodejs.org/api/all.html#url_the_whatwg_url_api is not hidden behind a CLI flag, it should be marked stable, AFAICT, or should have been hidden behind a CLI flag if it was supposed to be experimental

Filed an issue: https://github.com/nodejs/node/issues/11362

[Trott]

assert is now Stable and not Locked.

Someone want to step up to open a PR and make the case for timers?

[mhdawson]

I'm + 1 for removing locked stability level.

[jasnell]

url.URL is still experimental. Once it goes supported it should be semver-minor. There is absolutely no reason to put it behind a flag and I would definitely be -1 on any proposal to do so.

+1 on removing the Locked status

[gibfahn]

@jasnell I think we either need to put url.URL behind a flag or change the definition of Experimental to not require that the feature be "gated by a command line flag". Otherwise the definition of Experimental just doesn't apply to url.URL.

Stability: 1 - Experimental 
This feature is subject to change, and is gated by a command line flag. 
It may change or be removed in future versions.

There is absolutely no reason to put it behind a flag

@sam-github gave some reasonable-sounding reasons for putting it behind a flag in https://github.com/nodejs/node/issues/11200#issuecomment-279521320, could you counter those points?

[sam-github]

To be clear, NOW I think it should just be moved to Stability: 2 - Stable. It doesn't fit the definition of an Experimental API.

When it was first introduced, it should have been behind a flag until its quality was such that we were happy that npm modules could depend on it (it looks like it is that quality now).

[Trott]

PR to unlock module and remove Locked from the stability index: https://github.com/nodejs/node/pull/11661

[Trott]

Locked is removed in 51cea054a276c6425e26b25219e67b37207dee3f. Closing.