Ecma262: [Discussion] Are keywords (ab)used too much in upcoming proposals?

[dmitriid]

I would like to open a discussion on whether upcoming proposals in TC39 read too much into the existence of new.target and super.* and (in my opinion) abuse the language with extensions and additions that could potentially be solved in a different manner.

I'm copying some of the discussion over from es-discuss for greater visibility and as a possibility for other people to voice their views (not just those subscribed to the mailing list).

My original concern is with:

• import being extended to become import() and then having import.meta
• function.sent

The summary is as follows:

In Javascript, we now have:

• keywords that are just keywords, really (typeof, case, break, etc.)
• keywords that are just keywords, but don’t even exist in a language. They are reserved for future use in various contexts: always reserved, only in strict mode, only in module code etc. (enum, public, private, await etc.). May never be used and may possibly be removed, as some keywords have been (int, byte, char etc.)
• literals that are basically keywords (null, true, false)
• non-keywords that are for all intents and purposes keywords (eval, arguments)
• keywords that look like objects (because they have additional properties) which are not objects (new with new.target)
• keywords that look like functions (because they are invoked like functions and return values like functions) which are not functions (import and import())
• keywords that look like objects and functions but are neither (import and import() and import.meta)

It gets even worse. Because “metaproperties” are not just attached to keywords. They are attached to keywords which have fundamentally different semantics in the language:

• new is an operator, it gets new.target
• A function is a callable object, and it gets a function.sent
• import is … I don’t know what import is. It gets transformed into a separate, made-just-for-import CallExpression and then it gets an import.meta on top of that (as a hardcoded “metaproperty”).

The ECMAScript spec lays down no guidelines for when "metaproperties" should be used and for when keywords can or should be extended with additional behaviour. My personal feeling that this is used to great detriment to the language, as vastly different and often highly context-dependent "things" are "just added" to the language. From a developer experience point of view, I believe, this makes the language increasingly more complex, unstructured, unpredictable, and chaotic.

[isiahmeadows]

@dmitriid I'll start from the top:

I would like to open a discussion on whether upcoming proposals in TC39 read too much into the existence of new.target and super.* and (in my opinion) abuse the language with extensions and additions that could potentially be solved in a different manner.

I'm copying some of the discussion over from es-discuss for greater visibility and as a possibility for other people to voice their views (not just those subscribed to the mailing list).

I do agree that there is reason to have concern with the way meta properties are being pretty liberally considered at times. new.target does feel a little weird from a language design perspective, but it does generally fit - it's literally the target of a new call.

My original concern is with:

• import being extended to become import() and then having import.meta

First, plenty of other things have been considered for import(...) and import.meta both.

Second, I'd say this is closer to super(...) (long reserved for that purpose) and new.target.

• super(...) performs a context-dependent action, just like import(...).

  • The context of each is derived from the lexical scope
  • super(...)'s scope is derived from that of the enclosing this
  • import(...)'s scope is derived from that of the enclosing module

• new.target gets context-dependent metadata, just like import.meta.

  • The context of each is derived from the lexical scope and the current execution context
  • new.target's scope is derived from that of the enclosing this
  • import.meta's scope is derived from that of the enclosing module

It would've been nice if super was somehow bound to this (like this@super or something), so it would be as clean as import.meta.

Oh, and fun fact: super(...) is actually an expression, not a statement, and it returns the super call's result (as in, the resolved this). In addition, it's available in nested arrow functions. So in theory, super could be implemented as a callable proxy of the inherited prototype, like this, just exposed as a special keyword. The only reason it can't be spec'd that way is because super itself, just the keyword, isn't an expression.

So if you want to know of an object that isn't, super(...) is about as close as you're going to get. Not even import compares to this one.

• function.sent

I do agree that name is probably among the problematic ones, and IMHO it should probably be yield.sent or similar instead (just filed a bug about it). It really looks wrong for that usage.

My bigger issue really boils down to this: why didn't they address that potential issue when they designed generators, rather than a year later.

In Javascript, we now have:

• keywords that are just keywords, really (typeof, case, break, etc.)
• keywords that are just keywords, but don’t even exist in a language. They are reserved for future use in various contexts: always reserved, only in strict mode, only in module code etc. (enum, public, private, await etc.). May never be used and may possibly be removed, as some keywords have been (int, byte, char etc.)

I'd like to correct a couple of your examples:

1. await has since been used for async functions.
2. public and private were reserved for controlling method privacy, but I see these as likely to be dropped, considering their original intended future use case uses a sigil instead.

• literals that are basically keywords (null, true, false)
• non-keywords that are for all intents and purposes keywords (eval, arguments)

I presume the main reason they weren't properly reserved was probably because @BrendanEich (please correct me if I'm wrong) didn't have enough time to consider and address that in his design.

• keywords that look like objects (because they have additional properties) which are not objects (new with new.target)

Not really. Unlike import() vs import.meta and especially super() vs super.foo,

• keywords that look like functions (because they are invoked like functions and return values like functions) which are not functions (import and import())
• keywords that look like objects and functions but are neither (import and import() and import.meta)

See my previous notes on super(...).

It gets even worse. Because “metaproperties” are not just attached to keywords. They are attached to keywords which have fundamentally different semantics in the language:

• new is an operator, it gets new.target

Arguably, this is the ideal distinction - operators aren't properties, so there is literally zero risk for confusion here.

• A function is a callable object, and it gets a function.sent

function.sent is unique to generators, not just any callable object. See my previous notes on it, for my personal objections, but they're purely on naming and lack of foresight, not the choice of a meta property at all.

• import is … I don’t know what import is. It gets transformed into a separate, made-just-for-import CallExpression and then it gets an import.meta on top of that (as a hardcoded “metaproperty”).

Consider it this way:

• import ... from "foo" statically imports values from "foo". It's a top-level statement to reflect its static nature.
• import(mod) dynamically imports values from mod. It's a call-like expression to reflect its dynamic nature.
• import.meta contains the import metadata for this module. It's a meta property to reflect its property-like nature.

The ECMAScript spec lays down no guidelines for when "metaproperties" should be used and for when keywords can or should be extended with additional behaviour. My personal feeling that this is used to great detriment to the language, as vastly different and often highly context-dependent "things" are "just added" to the language.

In general, here's the patterns I've observed with the choice of what is used:

• If it's something static like var, export, etc., it's almost always a highly declarative construct.
  
  
  
  • Exceptions: break, continue
  
  
• If it's something control-oriented like yield or await, it's almost always space-separated.
  
  
  
  • Exceptions: static control flow like if/else and for
  
  
• If it's something action-oriented like super(...) or import(...), it's almost always call-like.
  
  
  
  • Exceptions: template tags, typeof
  
  
• If it's something metadata-oriented, it's almost always a meta property.
  
  
  
  • Exceptions: arguments
  
  

From a developer experience point of view, I believe, this makes the language increasingly more complex, unstructured, unpredictable, and chaotic.

In my experience, it's been adding much needed structure and rigor.

1. Consider how many constructor patterns there were pre-ES6, and the fact new existed without a way to declaratively make a constructor.
2. Have you tried writing something like takeWhile for Java-like iterators? It's trivial using ES6 generators and for ... of.
3. Promises were almost a no-brainer for the spec - there was already the Promises/A+ spec (which ES6 promises were strongly derived from), and most Promises/A+-compliant promise libraries were also almost-compliant ES6 promise polyfills.

Also, here's a rundown of all the existing Stage 4 proposals with significant added syntax:

• The exponentiation operator does not use any keywords.
• The trailing function comma proposal does not use any keywords.
• Async/await required await to become a contextual keyword in order to avoid ambiguity.
• The template literal revision was mostly a design-related bug fix, and does not use any keywords.

And stage 1-3 with significant added syntax (ignoring RegExp stuff):

• Stage 3: Rest/spread properties does not use any keywords.
• Stage 3: Async iteration allows async and generator function syntax to intersect.
• Stage 3: import(...) has already been discussed above.
• Stage 3: Public/private class fields uses a sigil (#), not a keyword.
• Stage 2: function.sent has already been discussed above.
• Stage 2: Decorators do not use any keywords.
• Stage 2: import.meta has already been discussed above.
• Stage 1: Generator arrow functions allows arrow functions and generators to intersect.
• Stage 1: Optional chaining does not use any keywords.
• Stage 1: do expressions have bodies.

[andreineculau]

@isiahmeadows first of all, thanks :clap: for the comprehensive write! It is way easier to digest this, no matter if the reader agrees or not with the argumentation.

I personally need to digest this in peace and quiet, but there are three things that I actually miss before starting the digestion.

---

Alternatives through globals

Something that @dmitriid left out of this github issue, but exists in the es-discuss thread that triggered this issue, is the suggested alternative to his complaints (isn't that constructive criticism? :) ).

That suggestion goes about introducing new globals, in the line of how Map, Set, Symbol, Reflect and Proxy came up. He mentions those globals could tackle introspection at specific levels i.e. Module, or in the mindset of "introduce as few things", at a general level i.e. System which can then have reference introspection at specific levels. Those are irrelevant details - the core is "introduce a new global for introspection".

My question on this topic is whether you or somebody else listening in has any idea of whether this/similar was suggested and why was it rejected?

Not keywords, but still new syntax

While new.target, function.sent and import.meta are not keywords, they are still additional syntax since new, function and import are not objects, but keywords. That is why now parsers need to be aware of such things - random example from babylon https://github.com/babel/babylon/pull/402/files#diff-94eebbd7c72a61803345c949037690d8 .

The only detail that makes them not keywords, is that the dot that follows carries meaning (separator) allowing a parser to detect the keyword e.g. new then look forward if a dot follows (maybe after some whitespace) and decide whether the code want new or new.target.

From 30.000 ft though, new.target could be easily added to the keywords list since it behaves, smells and quacks like a keyword. "new.foo" doesn't make sense and throws. Put differently, if we do call and treat new.target as a keyword (i.e. the dot carries no meaning; treat it as if it's "newtarget" or "new_target"), parsers couldn't care less. We could even have "new.meta.target" but if "new.meta" isn't an object with a "target" property, then "new.meta.target" is also a ~keyword.

This is also why you cannot have this.super (you gave the this@super alternative) - the dot carries semantics, but this.super cannot be turned into a standalone "keyword" (i.e. super as a metaproperty of this).

What stands out is that a whole new world opens up which adds syntax. For instance, a proposal could add metaproperties to a simplified for loop syntax e.g. for collection console.log(\${for.key}=${for.value}`). Another proposal could add a counter to thewhileloopwhile (true) console.log(while.counter). Or we could promote metaproperties to operators, and create even more operators e.g.let a = 5 /.mod 2` (modulo) Etc. This is the world that we opened. And all of these are additional syntax. An old parser just cannot parse. The language will grow, the parsers will grow.

What I'm missing on this "Not keywords, but still new syntax" is to hear agreement or disagreement. Because keywords or not (see my silly example with an operator) is simply irrelevant detail in my view. What I prioritize is simplicity at cognitive and parser level, sometimes linearly connected but not always.

Precedence

Correct me if I'm wrong but the first in this league was new.target, and followed by function.sent. The rest is history - I'm personally interested in seeing how meta-properties came to be, not in how a new meta-properties is aligned with another one. The inflection point is at the first or second occurrence (in case the first was an involuntary mistake). If the discussions around those are not substantial, then we have no induction so to speak.

All I could find about new.target are these notes https://github.com/rwaldron/tc39-notes/blob/master/es6/2015-01/jan-27.md#44-subclass-instantiation-reformation-status-and-open-issues where @allenwb says at some point:

Sees options in the future to add things like function.callee
Maybe it could be class.target but new.target is more accurate. These are called MetaProperties in the spec.

Before that there's nothing I can find, the previous suggestion being noted by @wycats :

Early strawman was arguments.constructor.

Same month, but a bit earlier there a reference here https://github.com/tc39/ecma262/blob/master/workingdocs/ES6-super-construct%3Dproposal.md also talking about new.target .

What follows is a document from February 2015 advocating for more metaproperties, describing the problem domain and the opportunity (the new world mentioned above) https://github.com/allenwb/ESideas/blob/master/ES7MetaProps.md#the-problem-and-opportunity . That was good to read, as I think anyone agrees with the problem domain that go like "can't do X", but it still doesn't clarify the addition of meta properties in general, and of the first one "new.target".

The discussion on the function.sent property (just a few months later) doesn't clarify that much either:

• proposal from May 2015 https://github.com/allenwb/ESideas/blob/master/Generator%20metaproperty.md
  
  
  
  • notes from Nov 2015 complain that there isn't much feedback https://github.com/tc39/tc39-notes/blob/master/es7/2015-11/nov-17.md#functionsent
  
  

Can someone clear the clouds and point to other online resources - proposal, notes? Otherwise, if they want and can, input from the driver of metaproperties on how they came to be would definitely go a long way (I'm guessing you @allenwb drove this ?!).

[jmdyck]

new.target first appeared (tentatively) in draft 31 of ES6 (on 2015-01-15). See https://esdiscuss.org/topic/a-new-es6-draft-is-available#content-14 and following.

[dmitriid]

What annoys and scares me the most is that the discussion starts of as valid:

Assume a developer who has never seen this new.target construct before. They will first think that this is an invalid expression, as new is an operator. Then, upon seeing this code execute, the natural question is "What is new? Is it an identifier injected into Environment Records created by [[Call]] and [[Construct]]? Does this identifier resolve to an object (so that the MemberExpression would make sense)?"]

And it is immediately dismissed with

In general, ES6 has new syntax, so this is a "learn it and use it" bump, one of many.

And most discussions regarding new syntax, especially the extension of keywords, is usually immediately brushed off with the same argument

I also love this proposal:

For the long term, I'd like to see a new identifier injected into function scopes which exposes the Lexical Environment/Environment Record internals. Then we can use __scope__.new.target or Reflect.isNewed(__scope__) (or isConstructed, which may make more sense seeing as there will be Reflect.construct). Of course, this __scope__ binding should only be injected in the Environment Record if the binding does not exist yet after registering the function body's declarations, for back-compat reasons. And obviously, __scope is just a placeholder name for this suggestion, I don't really mind how it will be called.

Note how __scope__(or whatever name it may be) solves both new.target, and function.sent, and import.meta

[ljharb]

It would not be feasible to "inject" a new non-syntactical identifier into function scopes, because no matter what identifier was chosen, it would break code that was relying on that identifier being a global variable. Only syntax can be used in this manner.

[andreineculau]

@ljharb I want to clarify in my head what you wrote with an example.

You're saying that "before" you could have some code like:

let __scope__ = {new: {target: 123}};
let fun = function() {
  console.log(__scope__.new.target); // forget that i'm not in a constructor for a second
}

but "after" this could lead to a different value being logged, that is the actual new.target ?

You say feasible, not possible. So what's the inconvenience with detecting whether __scope__ is used or not in the function?

[bakkot]

@andreineculau

So what's the inconvenience with detecting whether __scope__ is used or not in the function?

eval, for example.

But even then, how would detecting it help? How would you know if this was old code intending to refer to the global, or new code intending to refer to the magic identifier being introduced?

[andreineculau]

eval I saw it coming the second I pressed Comment, but.. eval...

How would you know if this was old code intending to refer to the global, or new code intending to refer to the magic identifier being introduced?

Do you need to know exactly that? You don't know that even with the new Map, Set, Symbol, Proxy.

What I'm thinking is that at runtime, you can see if __scope__ is defined in the local/closure/global scope. Return that if defined, return the new definition if not.

Same problems as with the new globals i.e. anyone can window.Symbol = function(){} 💣 💥 , except __scope__ would be localized i.e. traverse the scopes, if undefined, return a localized value. The proposal could come with immediately marking __scope__ as a future keyword, and giving warning whenever it is defined.

---

PS: do not do window.Symbol = function(){} on a github page, before you press Comment. It will frighten you.

[ljharb]

@andreineculau when adding a new global, someone relying on their own global of that name will end up shadowing the new global, so their code won't likely break. It's only a concern if someone is detecting the presence of the new global, and behaving differently based on that - and that's exactly why global can't be added under that name.

Regardless, anything that's context-sensitive has to be syntax.

[bakkot]

Do you need to know exactly that? You don't know that even with the new Map, Set, Symbol, Proxy.

Those are on the global object, not in every function scope. As such, there's no concern with those about the new name shadowing something; they're at the top of the scope chain. This would not be the case if we started injecting identifiers into function scopes.

What I'm thinking is that at runtime, you can see if __scope__ is defined in the local/closure/global scope.

With this proposal, would you be able to close over it? If not, it really is a keyword, not just an identifier; if so, you could never use it in an inner function, because it would always be defined in the outer one.

---

Anyway, separately, I have to say I can't really imagine how you could find "inject a new identifier into some function scopes depending on, at runtime, whether those functions already have visibility of an identifier of that name" to be a more reasonable design than a new keyword or metaproperty.

[dmitriid]

This is a circular argument. It contains the same arguments over and over again.

We can't blindly add global objects

Yes, we can. See, Reflect, Proxy, Symbol, temporal proposal. As one comment in temporal issues stated, "Each major browser release adds 4-5 new global objects".

This will break user code

See new global objects above.

Objects/globals/whatnot cannot have access to local context

Yes, they can. It all boils down to how you define them in grammar, and how you describe their behaviour. new.target is basically a hardcoded value in the current spec. import.meta is hardcoded in the proposal. Their behaviour is explicitly specced out for their particular purpose.

They cannot be too global. They cannot be too context-sensitive. Or other arguments between these two extremes

Yes, they can. super.* is context-dependent. function.sent is extremely context-dependent. import's behaviour changes completely depending on the scope it's in. await is only defined for "module scope" (if I'm not mistaken).

This will make the parser more complex. This will make the VM more complex. This will make property access more complex.

Handling of super requires changes to parser/VM: this cannot appear before super call etc.

Handling of metaproperties requires multiple specific changes to both the parser and the VM: function.sent can only be encountered in generator functions. import is top-level only, import() is kinda anywhere else, import.meta has many other specific behaviours associated with it (the algorithm for import.meta is a whole screen of instructions).

This will break expectations from code. This will make magical auto-variables. This will make it harder to reason about code. etc.

.<metaproperty> is applied to very different parts of the language: operators, objects, functions, separate entities only recently introduced into the language. If anything, the whole "BigInt" proposal could be solved with "metaproperties": 1 +.big 2 and 14 **.big 29 (or, better still, we could allow prefix and postfix operators in JS: +.pre 1 2 3 4 and 2 5 6 7 *.post).

import.meta is just as automagical as __scope__.context.module. However, the user of the language can predict and expect __scope__ (or Introspect) to contain just that: scope info or introspection API.

And that's just metaproperties. The whole "let's change import into import()" is another can of worms altogether.

[andreineculau]

@dmitriid please stay focused.

Objects/globals/whatnot cannot have access to local context

Yes, they can. It all boils down to how you define them in grammar, and how you describe their behaviour. new.target is basically a hardcoded value in the current spec. import.meta is hardcoded in the proposal. Their behaviour is explicitly specced out for their particular purpose.

You're conflating keywords/metaproperties into "Objects/globals/whatnot".
And it was already opinionated that syntax may be the only way to have context-sensitive information... Thus everything that you wrote from there on is just ready to get a "Yeah, syntax is context-sensitive, correct. We just told you that. Your point is?"

[andreineculau]

@ljharb

It's only a concern if someone is detecting the presence of the new global, and behaving differently based on that

So you're saying that someone might have in their code if (window.__scope__)... ?
I do not see how this is different from introducing window.Symbol ? Couldn't someone have had if (window.Symbol)... before Symbol was defined?
Can I kindly ask you to shed some light?

and that's exactly why global can't be added under that name.

Care to clarify? Which global? Under what name? I simplify do not follow. Thanks.

Regardless, anything that's context-sensitive has to be syntax.

I digested a bit, and there's at least Error which is context-sensitive.
So to play along, if I'm allowed to change a bit the pattern of __scope__, what's off with

class Foo {
  constructor() {
    let scope = new Scope();
    if (!scope.new.target) {
      throw new Error('Please use new Foo().');
    }
  }
}

Thanks a lot for your time.

[ljharb]

Yes, someone could have. And if they had, Symbol probably couldn't have been introduced without breaking sites.

For global, I'm referring to https://github.com/tc39/proposal-global

Error isn't context-sensitive; new Error is via error stacks which are not in the spec and it's been highly problematic for security reasons that that's the case for a long time. See https://github.com/tc39/proposal-error-stacks for an attempt to begin to specify stacks.

[andreineculau]

re: global - now I'm with you, though I tripped once again reading Further research has determined that using global will not break existing code., when in fact this is the relevant issue. So to rephrase, without skipping any relevant context:

and that's exactly why global can't be added under ~that name~ window.global. It might go as System.global, granted we don't find problems with System.

But now that I'm with you, we can agree at least that globals can be introduced (yes, iff they don't break the web). Maybe we were never in disagreement, but that's how I've read your and Kevin's replies. Maybe my bad.

---

Anyways, why do you say Error is not context-sensitive? Or how come you consider an error's stack (not spec-ed, but maybe spec-ed after your proposal) is not context-sensitive? Context as in literal context. When a stack gets created it gets information about the parent function name, location, etc. It also gets runtime information - the stack frames.

So on which level does new Error() get different to this invented new Scope() that would carry different information based on who is the caller?

---

Off-topic: System.global, System.getStack, System.WeakRef --- so there's a convergence towards a global System, but is there a proposal/decision overall towards System ?

[andreineculau]

@bakkot

Anyway, separately, I have to say I can't really imagine how you could find "inject a new identifier into some function scopes depending on, at runtime, whether those functions already have visibility of an identifier of that name" to be a more reasonable design than a new keyword or metaproperty.

Those are two very separate things imho. One is design, one is implementation. The design part is that we want to have __scope__ in the language. The implementation part is the because , it cannot be a keyword, thus . Example: design was adding fat arrows syntax, implementation was that parsers had to now distinguish arrow function parameter list and comma separated expression list. Similarly, I could say that I can't really imagine how [...] yet here we are.

if so, you could never use it in an inner function, because it would always be defined in the outer one.

Correct. The constraint is not to break current code. So if someone would window.__scope__ = undefined; then nobody would get to do introspection. Same applies to all the globals, so I must be missing the point you want to make into a counter-argument.

[ljharb]

@andreineculau
A new global "__scope__" would not break the following code, but a new function-injected var "__scope__" would:

function foo() {
  var __scope__ = false;
  function bar() {
    if (__scope__) {
      return 3;
    }
    return 4;
  }
  return bar();
}

[andreineculau]

@ljharb this was an answer to which of my questions? I can only confirm that's correct, but I don't see where I said otherwise.

We're playing with the idea that since __scope__ cannot be a keyword (TC39 doesn't want to introduce keywords, though I'm telling you that new.target still quacks like a keyword), then it can be a global that returns contextualized information. Error returns stacks. Date returns system time. __scope__ or Scope or new Scope() or System.Scope or new System.Scope() would return introspection information.

[bterlson]

This is not the repository for discussing proposals. If you have feedback on a particular proposal, use that proposal's repository. More general feedback should stay in es-discuss.