Ecma262: Extending "Forbidden Extensions"

[littledan]

This nice idea from @bakkot may deserve further investigation in TC39. What do you all think?

I think the "forbidden extensions" thing only really makes sense in the historical context of browsers experimenting with language extensions without consulting with each other or the broader community (e.g.). Since they don't seem to be inclined to do that sort of thing anymore (thankfully), I don't think we necessarily need to worry about people shipping a syntax before we have consensus on it.

Personally I'd be happiest if we could add a "everything not permitted is forbidden" part to the spec, but that might be hard to get through & would require some careful wording to allow browsers to ship stage 3 proposals (since the current spec makes no reference to the proposal process at all).

I'd be happiest with the strictest practical definition of what JS is, and I see nothing impractical about @bakkot 's proposal. This proposal could enable us to share more tests between implementations through test262, for one benefit.

[bakkot]

FWIW, test262-parser-tests already maintains a list (slightly outdated, but updating it is on my todo list) of illegal program texts, where "illegal" means "not currently permitted by the specification" rather than "explicitly not permitted by the specification". Part of the reason it's split out as a separate project is because of @leobalter's very valid concern that it is difficult to figure out which illegal program texts to include, and how to organize them; that project takes a more scattershot approach of "we'll include any test anyone thought worth writing, and we're not going to worry too much about coverage or duplication".

[erights]

Ignoring extensions, JavaScript has many useful inabilities; most obviously memory safety. However, I see nothing in the current spec that would forbid an implementation adding a non-config, non-write data property with builtin peek and poke methods, for directly accessing raw memory addresses. However, any theorem about what JS programs cannot do relies on such inabilities. As long as such extensions are not forbidden, such theorems are not sound.

I solicit help from formalists to help answer the following question:

What seem to be the intended inabilities that should enable useful theorems to proceed soundly? How should we specify them?

[littledan]

@erights What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

[erights]

@littledan Anything that gets us started, makes progress, would be great!

[allenwb]

What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262? Or host/environment specific features such as JSX?

In practice, such a probation would be ignored and that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

The current "forbidden extensions" were all carefully selected to address very specific concerns. By being limited and targeted they convey that TC39 is serious about the importance of keeping the specific identified behaviors out of the language. Expending it to say something like "no new syntax" means that TC39 is neither serious or realistic when it talks about restricting extensions and makes it more likely that the actual important restriction could be ignored.

[zenparsing]

The current "forbidden extensions" were all carefully selected to address very specific concerns.

Also note that the specific concerns addressed by each restriction vary. The ":" restriction is interesting, as I think it can be interpreted two ways. On one hand, it may indicate the desire to reserve syntactic space for type annotations in JS. On the other hand, it may indicate the desire to reserve that space for compile-to-JS supersets.

@allenwb Do you remember what motivated the ":" restriction?

[allenwb]

Do you remember what motivated the ":" restriction?

type annotations

[bakkot]

@allenwb

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262?

per the OP: this "would require some careful wording to allow browsers to ship stage 3 proposals". I expect such wording could be written. We of course would not try to prevent implementations shipping stage 3 proposals which had not yet made it into the spec.

In practice, the such a probation would be ignored

Can you say more about why you think so? I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

[leobalter]

I believe that generally forbidding syntax extensions is weak and too loose. The examples I'm aware so far can be specifically annotated and described. The prevented extensions for : on type annotations and the one proposed for dynamic imports are a good signal we can do a better job specifying what we want to secure for the web compat in the future. We can do the same work for other parts / proposals.

[allenwb]

Can you say more about why you think so?

The 70 year history of programming languages and how they evolve. Including the relative lack of success of previous attempts by various languages to restrict extensions. The times when it has worked (to some degree) have generally been situations where there were IP based contractual restrictions that limited extensions.

Standards are followed as long as they are useful to a business or user community. As soon as a potential competitive advantage or environmental requirement conflicts with a standard the standard goes out the window.

I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

Ah... Babel; JSX; Node, non-standard top level semantics ...

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Power can be lost if it isn't used carefully.

[ljharb]

Which implementations shipped runtime support for jsx? Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

[devsnek]

The only thing I can think of is V8's natives syntax feature, which allows embedders and V8 itself to inline calls to V8 internals from JS. (const target = %JSProxyGetTarget(someProxy);)

[bathos]

@ljharb I think the line gets fuzzy sometimes:

require('@babel/register');

[ljharb]

@bathos sure but that’s runtime stuff, not shipped by the implementation, nor syntax. If anything that’s an argument for forbidding any extensions, because it can always be enabled by a runtime mechanism.

[allenwb]

Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

Of course Babel is an implementation of ECMA-262 and historically one that has tended to have lots of deviations and extensions to the specification.

There is (intentionally) nothing in ECAM-262 that says what technologies or techniques must be used to implement the specified language. Source to source translation (regardless of the target language) is a perfectly valid implementation technique.

[bakkot]

Babel does not by default include any plugins which extend the spec, and plugins can and do change the syntax and semantics of things which are covered by the spec, rather than merely adding non-forbidden extensions. I don't think babel would have a problem with this, at least any more than it has a problem with the current spec - it is already the case the current spec does not carve out nearly enough space in what extensions it permits to fit all of babel's plugins. (I am technically a member of the babel team, but am not speaking on behalf of that team.)

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Sure, but the question is whether this is actually unrealistic. To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax. If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

[allenwb]

To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax.

JSX?

If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

I expect change, in everything. Expecting that a status quo will continue is unrealistic.

[zenparsing]

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

[bakkot]

JSX?

To my knowledge most consumers of JSX also make use of :-style type annotations, which are currently explicitly forbidden by the spec. I don't think this has created problems for anyone.

I don't think those two things are meaningfully different. Either the spec should forbid both, and people wishing to use build tools to add non-standard syntax which compiles to spec-compliant JS can continue to ignore the part of spec forbidding such extensions just as they do today, or the spec should permit both.

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

Yeah; I think this depends on whether we think browsers will be sufficiently motivated to ship non-standard non-proposed syntax that they're willing to ignore that authority. Personally, I don't think they will. If the committee thinks otherwise, certainly we should not pursue this.

[ljharb]

Browsers and all major engines are on the committee - if the committee thinks that forbidding syntax extensions is tenable, how is that not an explicit indicator that the spec indeed does have the authority to do so, granted by the implementations themselves?

[rwaldron]

If ECMA-262 had disallowed all extensions of the language's syntax...

• SpiderMonkey might've never created the never-in-the-spec experimental features found in "JS 1.7" and "JS 1.8" (I think those were a good thing)
• Nashorn might not have have cherry-picked some of those SpiderMonkey experiments https://wiki.openjdk.java.net/display/Nashorn/Nashorn+extensions
• JSI might not have implemented a type annotation syntax... (lololol this predates the Forbidden Extension) https://jsish.org/fossil/jsi/doc/tip/www/functions.wiki

...I'm sure there are more, but you get the idea.

---

I am the person that originally suggested that the class private fields/methods proposal add a new item to the "Forbidden Extensions" list that would forbid the "shorthand" form as a syntactic extension, that could later be "relaxed" (ie. removed) when/if a new proposal came along to re-introduce a "shorthand" form for accessing private fields/methods.

[rwaldron]

@zenparsing

re:

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

Presumably, you're referring to this:

The current "forbidden extensions" were all carefully selected to address very specific concerns. By being limited and targeted they convey that TC39 is serious about the importance of keeping the specific identified behaviors out of the language. Expending it to say something like "no new syntax" means that TC39 is neither serious or realistic when it talks about restricting extensions and makes it more likely that the actual important restriction could be ignored.

In the context of _my_ original suggestion, described above, I disagree with your interpretation, and I believe my strategy fits well within @allenwb's description of the use and intent of Forbidden Extensions.

The proposal put forth in this issue's original post: "everything not permitted is forbidden", is bad for JavaScript.

[bakkot]

@rwaldron, we also wouldn't have to have Annex B.3.3, which I personally think would alone outweigh the benefits of all those extensions by far.

[littledan]

Banning syntax extensions doesn't mean no one should or would extend the syntax. It just means that the extended language would no longer be standards-complaint JavaScript.

JSX and TypeScript both frame themselves as JavaScript extensions, but one is a forbidden extension and the other is permitted. I don't think this distinction is important for their users, though.

For me, one useful thing about this proposal will be being able to share tests for syntax errors between JS implementations. Another useful thing will be to be saying more precisely what it means to be standards-compliant, in particular that extensions need to be standardized to be JavaScript.

[anba]

Banning syntax extensions doesn't mean no one should or would extend the syntax. It just means that the extended language would no longer be standards-complaint JavaScript.

All browsers implement assignments to function calls as runtime errors, whereas the spec requires this to be a parser error. So, does that mean we need to label the JavaScript version implemented in browsers as non-standards-complaint JavaScript?

[littledan]

@anba I don't think that particular deviation from the specification is permitted as a section 16 "language extension", as the specification is explicit about the early ReferenceError in that case (search for the string "Reference Error" to see the occurrences). The specification is very specific:

An implementation must report as an early error any occurrence of a condition that is listed in a “Static Semantics: Early Errors” subclause of this specification.

I'd suggest that we address these sorts of web reality/spec mismatches via pull requests in this repository, rather than by expanding what sorts of language extensions are permitted.

[leobalter]

It just means that the extended language would no longer be standards-complaint JavaScript.

This is a legal concern for any implementation landing Stage 3 or 4 features or changes related to PRs that are not yet on any official release of EcmaScript.

The staging process is possible because we are allowed to keep implementations as legit standards complaint JS while we land features not yet released on new editions.

I urge this is a concern to have legal consulting within the implementations before we say it's ok.

[ljharb]

How is it a legal concern? Are there any engines that have a legal requirement in them to be compliant with the ecmascript standard?

[leobalter]

any new feature before a standard official release is an extension. If we ban extensions all the implementations might not be considered standards complaint anymore, unless they only implement features after the spec releases.

Also, not implementing a syntax feature of any released spec - by any reason - might also be considered an extension. That means, how do we allow not fully implemented engines to be spec complaint? Are all the major engines really ok with that?

[bakkot]

As it says in the original post, doing this at all would require some careful wording to allow browsers to ship stage 3 proposals.

That means, how do we allow not fully implemented engines to be spec complaint?

Such engines are already not spec compliant. The spec does not permit leaving out parts of the language (except Annex B).

[littledan]

What are the next steps on this thread? Does the committee have much appetite for pursuing this path that @bakkot suggested?