Webcomponents: How to define APIs only for custom element authors

[tkent-google]

The following issues need APIs which should be used by custom element authors and should not be used by custom element users.

• Need callback for form submit data (#187)
• Custom pseudo-classes for host elements via shadow roots (:state) (#738)

At the March F2F we came up with one idea. However using ShadowRoot for such APIs looks very weird to me and I'd like to discuss it again.

Candidates:

• ShadowRoot
  Pros: Usually only custom element implementation knows the instance of ShadowRoot which is used to implement the custom element. It's difficult for custom element users to get a ShadowRoot instance created by a custom element implementation [1]
  Cons: ShadowRoot is an interface for tree-encapsulation. Adding features unrelated to tree-encapsulation looks like a design defect. We should not make ShadowRoot a kitchen sink.
  Cons: Not all custom element implementations need ShadowRoot. For example, a checkbox custom element won't need ShadowRoot. Creating unnecessary ShadowRoot for such APIs is not reasonable.
  Cons: If a custom element implementation uses no ShadowRoot, a custom element user can call element.attachShadow() to get a ShadowRoot instance, and thus get access to these private APIs.

• new Something(customElement)
  It throws if users try to create it for the same element twice. It throws if the constructor is called with non-custom elements. The interface Something is called as ElementStates in #738, and called as HTMLElementPrimitives in #187.
  Pros: ShadowRoot won't have unrelated APIs.
  Pros: Usually only custom element implementation knows the instance of Something. It's difficult for custom element users to get a Something instance created by a custom element implementation [1]
  Cons: If a custom element implementation uses no Something, a custom element user can call new Something(element) successfully to get a Something instance.

• Deliver a Something instance by a custom element callback
  See https://github.com/w3c/webcomponents/issues/187#issuecomment-388740230 for more details.
  Pros: ShadowRoot won't have unrelated APIs.
  Pros: Custom element users can not create a Something instance unlike other two candidates.
  Pros: It's difficult for custom element users to get a Something instance delivered to a custom element [1]
  Cons: UA implementation would need larger code for a new callback, compared to the other two candidates.

IMO, the second one or the third one is much better than the first one.

@annevk @domenic @rniwa @trusktr What do you think?

[1] If a custom element implementation stores a ShadowRoot / Something instance to this.foo_, a custom element user can get the instance by accessing yourElement.foo_. There are some techniques to avoid such casual access.

[caridy]

I have a similar realization the other day when polyfilling shadowRoot.ariaLabel and co., which can fall into the same category. shadowRoot becoming a kitchen sink will be unfortunately, but considering that WC APIs are very low level, it might be ok.

[annevk]

I don't quite understand why delivering Something via a callback is more expensive than new Something(). (A way to avoid the "cons" of new Something() is perhaps to allow declaration ahead of time that the custom element won't use it, or indeed, require explicit opt-in that it's going to be used.)

(Another "cons" of ShadowRoot is that it's not limited to custom elements.)

[alice]

I'm writing up the properties @caridy mentioned.

The design problem we're trying to solve there is:

• We need a way to set non-reflected versions of the ARIA IDL attributes which will change dynamically
  
  
  
  • e.g. ariaChecked, which represents the current "checked" state
  
  
• These versions should be shadowed by the reflected version
  
  
  
  • e.g. if I set ariaChecked on the host element, as well as the shadow root, the version on the host element should override whatever was set on the shadow root - but then if I delete the ariaChecked property on the host, it should revert to using the version on the shadow root
  
  

ShadowRoot really seems like a good fit for this problem in some ways, since it's analogous to using :host in a <style> element within the shadow root - but unlike style, the properties on the host element can only be set directly on the host element (i.e. can not be defined anywhere but directly on the affected element), so there's no logical place to put the :host equivalent.

It also means that you can encapsulate semantic information about the host element without needing to register a custom element.

However, I can imagine asking the question "why can't I set shadowed versions of any element property on the shadow root?" - which definitely sets us on a road to a kitchen sink scenario.

Also, if your custom element doesn't otherwise need a shadow root, it seems like a shame to need to attach one for this purpose.

We could theoretically create a new type of object to hold semantic properties, analogous to a constructable stylesheet, but unlike constructable stylesheets there's no precedent for the type of object this would need to be - it's really just a simple, small map of properties.

Does anyone have any ideas for alternatives to using ShadowRoot here?

[annevk]

@alice @tkent-google's post has a number of alternatives, no? We'd need a pick a better name than Something, but otherwise one of those approaches would work I think.

[alice]

@annevk I should have addressed those better, sorry!

Here are my concerns with the those approaches (I count only two, really - the first alternative is just to use ShadowRoot):

• Either way, we'll have to create a new type which is simply an empty object with AccessibilityRole and AriaAttributes mixed in, which is kind of awkward.
• The "con" noted for the new Something(element) option seems like a deal breaker to me (that anyone can construct one for an element which doesn't already have one) - it's the opposite of encapsulation, which is what we're aiming for. (Presumably constructing a Something for a built-in element would be forbidden). Do authors need to construct a new Something defensively whenever they create a custom element, for all possible values of Something (if many APIs use this strategy)?
• The lifecycle callback option is probably the one I would be least uncomfortable with, but I think it implies having a single object named something like ElementConfiguration which can grow to hold all of these various objects (ElementStates, HTMLElementPrimitives, etc.) which is passed in in a single lifecycle event. Otherwise, it seems like either each new API has to create a new lifecycle event (?) or else the createdCallback ends up with a mess of arguments.

[trusktr]

but considering that WC APIs are very low level, it might be ok.

That may be an overstatement. Web devs want to make components, so if they move off React or Vue (or etc) to Web Components, they will use ShadowDOM. It's not very low level, it's a normal part of organizing web UI. Any serious web developer should be expected to organize code with components of nested trees; it's standard.

I'm in favor not to use it as a kitchen sink for component parts. Rather, ShadowDOM is it self _a_ component part, and should keep it's concerns specific. Other parts can have other jobs.

---

@tkent-google About new Something (element), what's that idea? Is that a suggestion for an entity-component system?

[alice]

@annevk and @tkent-google I just raised https://github.com/WICG/aom/issues/127 to try and avoid further rabbit-holing here on this specific issue, but I'd appreciate your thoughts over there if you have time!

[alice]

One benefit of the createdCallback (presumably?) option is that, if we do combine all possible settings into something like ElementConfiguration, the API gives you an opportunity to learn by exploration of that object.

With new MyThing(element), authors would have to learn about each MyThing separately (although obviously good docs can help with this).

Also, if new MyThings keep being added, custom elements which were written before they existed run the risk of having MyThings being defined by the page instead. This may not be a concern, but it seems awkward to me.

[trusktr]

@tkent-google About new Something (element), what's that idea?

Oops, nevermind, it was based on the idea of new ElementStates(this) which you linked to. Sorry for the noise.

Deliver a Something instance by a custom element callback

I think that's my favorite idea.

I added an example using that idea here: https://github.com/w3c/webcomponents/issues/738#issuecomment-412694539

To avoid enabling all features (and avoid runtime cost) maybe they can be opt-in? f.e.,

import Privates from './privates-helper'
const _ = new Privates

class MyEl extends HTMLElement {
  useFeatures(features) {
    _(this).cssStates = features.use('cssstates')
    // or
    _(this).cssStates = features.cssStates()
  }

  connectedCallback() {
    // blink every second
    setInterval(() => {
      _(this).cssStates.has('enabled') ?
        _(this).cssStates.remove('enabled') :
        _(this).cssStates.add('enabled')
    }, 1000)
  }
}

We could achieve this example with CSS, but this example is similar to how a div element doesn't add a hover class for you, but instead a :hover state, so it doesn't interfere with the div user's class list.

[hayatoito]

@tkent-google,

Deliver a Something instance by a custom element callback

It looks this idea is the most favored. Something might become yet another kitchen-sink, but I think that would be much better than using ShadowRoot here.

Could you have a chance to make the idea more concrete one or prototype?
Several new features are waiting for this idea, I guess.

[tkent-google]

Many people are favor of the third one. Let's proceed with the third one if no one has a strong opinion against it.

We need to define the followings:

Interface name

HTMLElementPrimitives (drop HTML?), ElementSemantics, etc.
Please reply your ideas.

Callback

First I thought we could add a Something argument to connectedCallback, but I found that custom element implementations needed to call a protected API before connecting to a document tree in Form Participation API. So we should introduce new callback.

My current proposal is createdCallback, which is called just after upgrade, maybe before attributeChangedCallback.

[tkent-google]

@annevk

I don't quite understand why delivering Something via a callback is more expensive than new Something().

I guess adding a new custom element callback requires larger code in UA than allowing new Something(element) just once. We already have four callbacks, and adding another would not be a big issue.

[annevk]

Another name idea: ElementInternals (after internal slots). Without HTML seems reasonable as we'd reuse this if we ever added custom elements in other namespaces. createdCallback also seems reasonable, though a bit unfortunate we then have both that and a constructor.

[caridy]

ElementInternals is definitely better than ElementSemantics, +1 on that.

[alice]

Would we put all of the properties on what I've called ElementSemantics on ElementInternals, or would we have a semantics object hanging off ElementInternals like I proposed (I referred to ElementInternals as ElementConfiguration as a placeholder in that doc, and in my comments above)?

[caridy]

@alice I think a flatten structure is just fine. HTMLElement itself is mostly flatten anyways.

[trusktr]

Extending from option 3, could the following help with performance?

Option 4: static list of features.

What if elements could specify specific features to use in a static prop, so the engine doesn't unnecessarily have to pass every single feature into a callback even if an element won't use each feature?

F.e.

class Foo extends HTMLElement {
  static get observedAttributes() { ... }
  static get features() { return [ ... ] }
}

Then the features could do what they want: specify new callbacks, or provide certain instance props, etc.

Either it would be up to the feature how it is exposed, or perhaps they all get injected into a standard callback like option 3's createdCallback (though I think a different name would be better because we already have constructor).

As for the static list of features, if they were strings,

  static get features() { return [ 'builtin-feature', 'user-feature' ] }

it would be easiest, but with the problem of name clashing (suppose we let anyone provide features, not built-in features):

  static get features() { return [ BuiltinFeature, UserFeature ] }

References would avoid name clashing, and not require a registry:

But then, this seems like mixins!

Option 5: class-factory mixins

Could mixins do the trick?

class Foo extends BuiltinFeature( UserFeature( HTMLElement ) ) {

As a real-world example, SkateJS is a web component library whose features are consumable as mixins, making them easy to mix and match.

[tkent-google]

ElementInternals sounds good. If no one objects it, let's adopt it.

@alice @caridy I also suppose flatten structure. ElementInternals interface has attributes/operations for various features such as accessibility, form controls, ...

@trusktr I'm not sure I understand what you wrote correctly. Probably such feature list isn't necessary because we assume a single ElementInternals instance handles all features for the associated element?

Re: callback name
createdCallback represents the timing when it is called, and doesn't represent the purpose. We may give more descriptive name like elementInternalsCreated receiveElementInternals.

[trusktr]

@tkent-google

because we assume a single ElementInternals instance handles all features for the associated element?

That's what I was trying to avoid with the idea of a list or mixins, because otherwise it means ElementInternals will include all possible features even if the features are not going to be used.

---

In Option 5 above, class-factory mixins provide a way to opt-in to using specific features (and avoid resource waste by creating the feature instances only when needed):

import UserFeature from 'npm-package'
const {BuiltinFeature1, BuiltinFeature2} = customElements.elementFeatures

class MyEl extends BuiltinFeature1( BuiltinFeature2( UserFeature( HTMLElement ) ) ) {
  // ...
}

customElements.define('my-el', MyEl)

In that sample, the class will have only the features it has specified.

---

If the list gets long, then

const Features = BuiltinFeature1( BuiltinFeature2( UserFeature( HTMLElement ) ) )

class MyEl extends Features {
  // ...
}

[trusktr]

@alice An example of option 5 for element semantics would be like

// ...
const {withSemantics} = customElements.elementFeatures

class MyEl extends withSemantics( HTMLElement ) {
  // ...
  receiveSemantics( semantics ) {
    const privates = elementPrivates.get(this);
    privates.semantics = semantics;
  }
}

customElements.define('my-el', MyEl)

where in this example, the receiveSemantics callback will be called by the withSemantics mixin implementation.

The withSemantics implementation might call the receiveSemantics method in its constructor, for example.

The implementation of withSemantics might look like the following (except it may not be JavaScript):

window.customElements.elementFeatures.withSemantics = function withSemantics(Base) {
  return class WithSemantics extends Base {
    constructor() {
      super()
      if (typeof this.receiveSemantics === 'function') {
        const semantics = ___; // create the semantics for `this` element
        this.receiveSemantics( semantics )
      }
    }
  }
}

---

It could also be placed on window.elementFeatures, but that creates a new window global. Seems like customElements would be a good fit for this.

[trusktr]

Another example could be observing children. A mixin could make this an opt-in feature:

const {withChildren} = customElements.elementFeatures

class MyEl extends withChildren( HTMLElement ) {

  // This callback is provided by the `withChildren` mixin.
  childrenChangedCallback() {

    // Work with children here, don't worry about
    // if children exist in `connectedCallback`.

    // Not only will this fire whenever children change, but
    // also once after `connectedCallback` has been called.

    // Maybe children here are guaranteed to be already
    // upgraded if they are custom elements?

  }

}

[tkent-google]

@trusktr, with your ideas, can we realize APIs which are available only for custom element authors?

[trusktr]

can we realize APIs which are available only for custom element authors

Are you wondering if we can allow the mixins to be used only with Element classes? If so, then yeah we can:

The mixin could check the base class to make sure it is an Element class, f.e., pseudo code of the native code in JS:

function withAwesomeness(Base = HTMLElement) {
  if (!extendsFrom(Base, Element)) throw new TypeError('mixin can only be used on classes that extend from Element')

  // continue, return class extends Base...
}

Or did you mean something else?

[caridy]

@trusktr this issue is about creating an API that can be used only by the custom element author, where consumers of the element can't access, influence or observe it. From that point of view, the question from @tkent-google is whether or not your mental model is considering that the primary driver? Seems to me that mixins are the wrong abstraction is you want to hide things from the consumer of the component.

[calebdwilliams]

If that’s the point of this issue, why not just make them private members on the base element classes? If this proposal waits on that to be approved and proceed? No need to reinvent the wheel.

[caridy]

@calebdwilliams where to store the reference of the instance generated by something like new ElementInternals(customElement) is not the issue. You can use a weakmap, a symbol, a private field, etc. This is analog to what this.attachShadow({ mode: 'closed' }) does today where you're responsible for storing the internal reference to the shadowRoot reference if you want to keep it around. This issue is about how and when to create an instance of ElementInternals, who can create it, and what are the semantics of it.

[calebdwilliams]

@caridy, I understand that, my thought was really to adopt that as a means of constructing those internals similar to the way a shadow root is attached. Something like this.#attachElementInternal('name'). I believe the extending class should have access to that, right?

[caridy]

No @calebdwilliams, that's not how our private fields proposal works.

[mkay581]

@caridy can you please identify exactly why @calebdwilliams' suggestion wouldnt work? because I honestly was thinking the same thing. I have been following the private members proposal, but probably not as closely as I should, so I'm sure you may be right. It would just be helpful to explain. Thanks!

[caridy]

Private fields from the private fields proposal are specified as hard privates:

It means that private fields are purely internal: no JS code outside of a class can detect or affect the existence, name, or value of any private field of instances of said class without directly inspecting the class's source, unless the class chooses to reveal them. (This includes subclasses and superclasses.)

More info here: https://github.com/tc39/proposal-class-fields/blob/master/PRIVATE_SYNTAX_FAQ.md#what-do-you-mean-by-encapsulation--hard-private

[tkent-google]

I asked a TAG review for Form Participation API including the third idea of this thread. When the review finish, let's follow the review feedbacks.

[caridy]

@tkent-google I'm still fuzzy about the createdCallback and its semantics. Does it have different semantics as the constructor? Can an attribute be added there? Does any user-land code executes between the constructor and the createdCallback? I suspect all answer will be "No".

If my assumptions are correct about the internals, they are never accessible from outside, and they will probably never dictate any state or any public API of the component. I believe we can just rely on the connectedCallback. Adding the internals argument to the connectedCallback should be fine since it is just relaxing an existing API that doesn't have any argument.

[annevk]

@caridy I think connectedCallback is a little weird. E.g., if I do const ce = new CustomElement(), I wouldn't want to have to do treeItem.append(ce) before being able to setup various aspects of its behavior.

I'm also not sure all your assumptions are necessarily correct. E.g., if we allow setting ARIA roles and states through it, I'd expect those to be observable if we also add an API for computed ARIA role or some such.

[caridy]

@annevk yeah, I thought about that, the analog case I found was getComputedStyle, which is really useful only after insertion. The main reason being the fact that the semantics that you want for you component sometimes depend on the structure of the DOM, e.g.: are you focusabled?. Computed AOM could probably be the same.

[annevk]

Even if that would always hold (I don't think it does, the proposed computed ARIA role API would work for elements without being connected), it's not clear to me why that means it should be bound to insertion. Presumably if you get removed and then inserted again none of the state would change? Given that, it makes more sense to only hand out the state object once, at creation-time.

[tkent-google]

Does it have different semantics as the constructor?
Can an attribute be added there?

That's a good point. We don't allow DOM mutation in a custom element constructor (See step 6.1.4-7 of create an element. If we kick the callback inside create an element steps, we should prevent the callback from DOM mutation in order to avoid breaking createElement() semantics.

[caridy]

it's not clear to me why that means it should be bound to insertion.

@annevk agreed. Maybe something more analog to attachShadow() without the public getter is more suitable then. Something that you can optionally invoke during construction, e.g.:

class Foo extends HTMLElement {
    constructor() {
        super();
        this._internals = this.createInternals();
    }
}

With the same semantics as attachShadow(), which means it can only be called once, and in the future we can add arguments to that API.

It just feel to me that the createCallback() is a misstep, from many angles, including the pedagogical one, how to teach people that this callback is kinda special?

If we kick the callback inside create an element steps, we should prevent the callback from DOM mutation in order to avoid breaking createElement() semantics.

@tkent-google yes, that will be weird, that callback is not really a callback from the point of view of the author, and it doesn't follow the other callbacks semantics either.

[annevk]

So that is the new Something() proposal from above, with a slightly different API. It seems we could address the concerns about such an API (raised by @alice above) by a) limiting it to being invoked during construction only, so others cannot create one for you and b) making it a "mixin target" similar to the other proposals so you don't need a bunch of Somethings, just the one. It'd still be somethingInstance.ariaXXX etc.

I think I wouldn't mind circling back to that (either API shape), given that additional callbacks do have some unwelcome complexities.

[caridy]

@annevk that's very interesting. I think I like that better, "restricting access to such thing during the construction phase", being completely optional, and mixin safe.

I still think that new Something(this) is very weird, I prefer this.something() to produce the stateful obj that represents the internals, and that can be accessed at any level in the prototype chain during construction only.

[rniwa]

The problem with this.something() is that anyone can call it. Since the point of this API is to provide new API surface only to custom element implementation, ideally, we wouldn't expose it to everyone else.

[annevk]

@rniwa if we only allow calling it during construction and throw otherwise, that's not really a problem, as constructors of normal elements don't run JavaScript that could invoke it.

[rniwa]

@annevk yeah but exposing a method on Element which can't be invoked isn't great either since it pollutes the element's prototype.

[annevk]

Given the number of members we haven't been super concerned about that thus far, but yes. I think it's either that or an awkward possibly-hard-to-design second JavaScript invocation during construction (as a callback). Unless there's some way to expose an object in the constructor somehow?

[domenic]

I really don't think speccing or designing createdCallback would be that bad. Maybe @tkent-google can speak to the implementation? At this point that seems like the best path to me.

[annevk]

If we invoked the callback directly after step 6.1.3 of https://dom.spec.whatwg.org/#concept-create-element we'd maintain the invariant checks and not run JavaScript at a new unexpected point. It doesn't have "CEReaction timing", but that's fine.

[domenic]

That wouldn't run it when you do new XElement() directly. I think it needs to be in the the [HTMLConstructor] algorithm. I agree this wouldn't be a CE reaction.

[rniwa]

new Something(this) or some static method on Something which throws when called outside the constructor would be probably fine.

[rniwa]

I'd rather not have a new callback which is different from custom element reactions. Invoking JS from C++ is slower than invoking C++ functions from JS for various reasons, and this will be yet another JS object we need to [[Get]] and store per every custom element interface.

[caridy]

Anther side effect of the new callback: the possibility to bypass internals defined during the inheritance. This might work fine for other callbacks, but for the definition of the internals, guaranteeing that the super class is defining the right internals seems to be very important, and the subclass should not have too much control other than overriding internals after the fact.

On the other hand, Something constructor is guaranteed to be untouchable by the subclass, while this.something() provides a little bit more flexibility, in case the subclass want to redefine that particular method for more control. It seems the best of both worlds.

@domenic the callback remind me a lot to the init() method in the original Realm API, which was called by the Realm constructor algo, and many folks pushed back on that one for very similar reasons that those described here.

[domenic]

I was warming up to the something-that-throws-outside-the-constructor idea (my bikeshed was window.customElements.createInternals(this)) but I can't figure out how to make it work. We have no ability to run code after element construction in the new XElement() case. There the only code we run is inside [HTMLConstructor], i.e. the super() call. So, assuming we use the super() call to flip the flag saying "you can now use the thing", we have no way to later flip a flag that says "can no longer use the thing".

In other words, if we want new XElement() to continue to work, I can't see how we'd be able to turn on access to the thing only inside constructors.

[rniwa]

Hm... I guess another option is make HTMLElement's constructor take an argument. So you do:

class MyElement extends HTMLElement {
    constructor() {
        const myInternal = new ElementInternal;
        super(myInternal);
    }
}

That's pretty funky. It would mean that ElementInternal (I don't think it's a good name but for the purpose of discussing the semantics, I'd use this name) would have a pre-attachment / pre-construction state too. I guess element.attachElementInternal() which would throw on a second invocation would be fine... that would be analogous to how shadow DOM API would work. Custom elements that don't use it can either attach it and never use it, or we could make this an opt-in feature where you'd have to declare it in a static field like we did for observedAttributes.

[tkent-google]

I talked to JavaScript-binding experts in the team, and our conclusion was it's very difficult to detect whether an operation is inside new MyElement() or not.
Also, I noticed it's difficult to define createdCallback behavior in new MyElement() case.

I think the second approach (new ElementInternals(this), this.attachInternals(), customElements.createInternals(this), or something) is acceptable though it's impossible to prevent non-ce-authors from using it perfectly. We may clear can-create-internals-flag in create-an-element path and upgrade-an-element path, and may clear it when the 'custom'-state element is connected.

[tkent-google]

or we could make this an opt-in feature where you'd have to declare it in a static field like we did for observedAttributes.

Opt-in sounds a good idea. Adding a flag to ElementDefinitionOptions would be an easy way.

customElements.define('my-element', MyElement, {needsElementInternals: true});
// ==> ElementInternals creation is allowed.

customElements.define('my-element', MyElement);
// ==> ElementInternals creation is disallowed.

[caridy]

if we want new XElement() to continue to work, I can't see how we'd be able to turn on access to the thing only inside constructors.

I think I'm fine with that. Let's not bend backward for this feature, and keep the new XElement() intact.

to prevent non-ce-authors from using it perfectly

I agree with @tkent-google, it should be fine, just like it is fine for attachShadow. It you claim it during the construction process, no one else can claim it after that.

Opt-in sounds a good idea. Adding a flag to ElementDefinitionOptions would be an easy way.

@tkent-google no, it is not. In many cases, an author produces a component, and the consumer of that component is responsible for registering, and decide what tagName to use. This model works pretty well for us today, and having to have a coordination between the registerer and the author is going to be pretty bad. The static field as the opt-in described by @rniwa is more realistic since it is described by the author.

I guess element.attachElementInternal() which would throw on a second invocation would be fine

@rniwa I think that breaks any possibility for multiple classes in your hierarchy to tap into the internals. It is not a deal breaker IMO, but will require some coordination by the subclass to wrap the call to attachElementInternal is it needs access to the internals, but I do agree that this is probably our best option so far.

[annevk]

@caridy don't you need such coordination anyway given styling proposals and such that are underway?

[domenic]

Anther side effect of the new callback: the possibility to bypass internals defined during the inheritance.

This is true with any of the options on the table. E.g. you say that the Something constructor is guaranteed to be untouched, but that's not true; the subclass does not need to call it (they can use Reflect.construct to directly call HTMLElement, and bypass the superclass constructor). In the end the author of the custom element class has ultimate control.

Hm... I guess another option is make HTMLElement's constructor take an argument.

Interesting. This seems workable to me, although a bit weird for developers. Especially because standing advice for JS class usage is "don't do anything before calling super()".

Basically ElementInternal is useless until it's passed to a [HTMLConstructor] via super(), then it becomes useful. That's not too complicated.

I think this works, although there may be Reflect.construct shenanigans that could mess it up.

I guess element.attachElementInternal() which would throw on a second invocation would be fine... that would be analogous to how shadow DOM API would work. Custom elements that don't use it can either attach it and never use it, or we could make this an opt-in feature where you'd have to declare it in a static field like we did for observedAttributes.

The problem there is you can attach it to arbitrary elements you don't own. Even if we only allow it on custom elements, that seems unfortunate.

Also, I noticed it's difficult to define createdCallback behavior in new MyElement() case.

I think you just define it to happen at the bottom of the [HTMLConstructor] steps. (So it happens when you call super(), before the rest of the author-defined constructor code.)

Opt-in sounds a good idea. Adding a flag to ElementDefinitionOptions would be an easy way.

So I guess the idea here is that this reduces the bad case to when custom element authors opt-in with needsElementInternals: true, but fail to do this.attachInternals() (or whatever) before calling any author code. That helps a bit.

---

Overall I think we have three proposals that each have OK tradeoffs:

• Pass ElementInternals to super(). Pro: encapsulation seems perfect. Con: a weird pattern for developers to learn.
• createdCallback(), invoked by super(). Pro: encapsulation seems perfect. Con: potentially two JS callbacks if developers use both constructor and createdCallback().
• needsElementInternals + this.attachInternals() or something. Pro: doesn't have other options' cons. Con: encapsulation is imperfect if custom element author is not careful.

I personally think the createdCallback() version is simplest, especially because I'd anticipate authors only using either createdCallback() or the constructor, not both. But all of these seem livable.

I think it would be good to spend pre-TPAC time keeping our minds open and fleshing out the list of alternatives, either by adding more or expanding on the pro-cons. We've had a lot of good brainstorming over the last week that feels productive. We can then narrow the field at TPAC.

[rniwa]

@domenic : for createdCallback, I'd add another con that invoking the subclass' method while executing super() in that subclass' constructor is a very strange programming pattern.

class MyElement {
    constructor() {
        super(); // This calls createdCallback
        this._myStateObject = ~;
    }
    createdCallback(elementInternal) {
        // Can't see this. _myStateObject.
    }
}

We have to do this for the same reason that we can't tell the end of the constructor. An alternative is to enqueue a custom element reaction after the constructor had run but then it would mean that the constructor won't have access to ElementInternal even though that's the most natural place to set the default tab index, etc...

[tkent-google]

Some code examples and comments.

• Pass ElementInternals to super()

class MyElement extends HTMLElement {
    constructor() {
        const myInternals = new ElementInternals();
        // myInternals is inactive.  All operations on myInternals throw exceptions.
        super(myInternals);
        // myInternals is active.
    }
}

• createdCallback(), invoked by super()

Custom element users can steal ElementInternals instance by inheritance.

Code by a custom element author:

class MyElement extends HTMLElement {
  createdCallback(internals) {
    this.internals_ = internals;
  }
}

Code by a custom element user:

class MyElement2 extends MyElement {
  createdCallback(internals) {
    // This is called instead of MyElement's one.
    super.createdCallback(intrenals);
  }
}

• Opt-in + this.attachInternals()

class MyElement extends HTMLElement {
  // static getter is better than define()'s option?
  static get needsElementInternals() { return true; }

  constructor() {
    super();
    this.internals_ = this.attachInternals();
    // attachInternals() throws if needsElementInternals doesn't exist or returns false.
  }
}

[annevk]

During the F2F the point was made for the third option that static is likely better than define() due to it working for inheritance.

[annevk]

I made the point that maybe getInternals() is a better name, since if you subclass a builtin it'll have some non-default state stored there already.

[rniwa]

Rough consensus at TPAC F2F: We'll tentatively use attachInternals with static field indicating that the element internals should exist for a given custom element (so that it would work for subclasses) where calling it twice will throw but @domenic will consult with TC39 to see if there is a generic protected field mechanism that we should be using here.

[caridy]

Excellent, options 3 sounds promising, and should work fine when providing abstractions on top of HTMLElement, which is what we do mostly.

[domenic]

TC39 cross-check posted at https://github.com/tc39/ecma262/issues/1341; stay tuned.

[littledan]

The approach you settled on here sounds fine for now, if you need to get something in soon, but if you can wait a little bit for decorators, then we might have an option which is a little more ergonomic: https://github.com/tc39/ecma262/issues/1341#issuecomment-435487021

[domenic]

So, reporting back from the TC39 thread in a bit more detail. A few takeaways:

• There may be an elegant future solution with decorators. However, decorators are a "stage 2" feature and move very slowly through the standards process; indeed at stage 2 it's unclear whether they will be implemented at all. So we should proceed with an interim solution, and in some years maybe decorators will be available and we can consider that. We should not take on a dependency.
• At the same time, we may want to contemplate whether there are features of our interim solution that we could tweak to make it more upgradeable to a decorator future.
• TC39 provided some useful feedback on our existing solution at https://github.com/tc39/ecma262/issues/1341#issuecomment-435492456 which I think we should take into account. In particular it seems like subclassing does not work well with our existing solution, so the needsInternals placement on the class instead of as an option to customElements.define() is a bit questionable now.
• Again about subclassing, there was the insight that basically if you want subclassing to work, you're going to lose the protection of disallowing arbitrary consumers from attaching ElementInternals. This insight emerged in https://github.com/tc39/ecma262/issues/1341#issuecomment-435519499.

Taking all this advice in to account, my proposed solution is that we go with attachInternals() but no needsInternals. Thus, attachInternals() becomes similar to the (closed) shadow DOM model:

• It's somewhat "uncoupled" from custom elements. The main coupling is that we restrict it to only be callable on custom element instances, just like attachShadow({ mode: 'closed' }) is only callable on custom elements + a safelist of elements.
• If a custom element doesn't call attachShadow({ mode: 'closed' })/attachInternals() in its constructor, then any consumer of that element can call the method and start messing with the element's shadow tree/internals. Oh well.
• If a custom element does call attachShadow({ mode: 'closed' })/attachInternals() in its constructor, then subclasses are screwed. (I.e., they need to coordinate with their superclass if they want access to the shadow root/internals.)

This also has the benefit of only adding one API, so that in some glorious decorators future, we can explain the decorator as sugar over one old API, instead of two, and we don't have to worry about interactions (e.g. what if someone uses the decorator + needsInternals?).

What do folks think?

[caridy]

@domenic I'm fine with that. Just having a precedent on attachShadow will definitely help to explain this new API. As a component author, if you care about it, you can just call it during construction, if you extend a class that cares about it, well, you have to coordinate it, just like attachShadow.

[littledan]

About the timing of decorators: my current plan is to propose it for Stage 3 in January 2019. If this is successful, I hope we could get to the point of decorators shipping in browsers by late-2019/mid-2020 (given that multiple browsers have been expressing support for the decorators proposal). If this schedule would get interfere with your timeline, and if it's OK if there isn't any strong protection guarantee, Domenic's solution sounds fine to me.

[rniwa]

I don't think we'd be okay with attachInternal being callable without the knowledge of the custom element itself. @saambarati is following up on that thread but it doesn't seem like a good idea to implement any interim solution until there is a general consensus / agreement as to how a protected state is being done in TC39.

[annevk]

Requiring needsInternals might still be a good way of allowing component developers to opt into this explicitly and not require all existing components to retroactively start calling attachInternals() early enough to avoid being bamboozled. That'd avoid @rniwa's concern as well.

[rniwa]

Right, requiring needsInternals would address my concern.

[littledan]

needsInternals sounds like a sound solution if it's passed as an option to customElements.define, but I guess it leaves two issues, that it's strange to have to mention this in two places, and the difficulty of working with subclasses. And if you make your class needsInternals accidentally, this opens you up to anyone calling attachInternals.

Here's a slightly different possibility, as an analogue of this decorator-based solution . At least it solves the last issue (which might not be so serious).

// In the browser
customElements.getAttachInternals = function(klass) {
  if (klass is already registered as a custom element) throw new TypeError;
  return function(element) {
    // Possibly generalize this check to make working with subclasses better
    if (element's custom element definition's constructor !== klass) throw new TypeError;
    return element.[[Internals]];
  };
}

// Usage example
class MyElement extends HTMLElement {
  #internals = attachInternals(this);
}
let attachInternals = customElements.getAttachInternals(MyElement);
customElements.define('my-el', MyElement);

By requiring that getAttachInternals be called before customElements.define, the system ensures that this call is collaborating with the code which defines the element (if custom element classes don't sit around un-defined for long). You could later make getAttachInternalsbe overloaded as a decorator, to save a little bit of code, but no need to block on that.

[domenic]

@rniwa @annevk can you say why you are OK with attachShadow({ mode: 'closed' }) being called without the knowledge of the custom element itself, but not OK with the same for attachInternals()?

The reasons I think needsInternals is subpar are:

• It adds more surface area to obsolete in a decorator-friendly future. In that future (and I don't share @littledan's optimism on the timeline, but it may indeed happen eventually), we could straightforwardly explain @HTMLElement.internals as sugar on top of attachInternals(). But explaining its interactions with a two-API feature is harder. E.g. what happens if you use @HTMLElement.internals + needsInternals together, or a base class uses one but not the other.
• To make it tamper-proof we have to move it to customElements.define(), or do a custom prototype walk that checks own properties until it hits a built-in element prototype. The former option conflicts with @caridy's desire to separate class definition from registration, and the latter option is really weird and unprecedented in both JS and web specs. On the other hand, if we don't make it tamper-proof, it doesn't have any real benefits. Which leads me to just omitting it.

I think there is consensus/agreement on how protected state is done in TC39; it's not built in to the language, but decorators provide a powerful enough mechanism to enable patterns like protected, friend, .NET-style internal, etc.

[annevk]

@domenic well, existing components are aware of attachShadow(), but not attachInternals(). (And observing such a static at define() time would be consistent with observedAttributes et al, no?)

[domenic]

existing components are aware of attachShadow(), but not attachInternals()

So this is just a staging problem? I.e. needsInternals is an API to work around the fact that we didn't introduce all web components APIs into the world at the same time? That seems like an unfortunate way of designing APIs...

And observing such a static at define() time would be consistent with observedAttributes et al, no?

I'm not sure what this is in reference to.

[domenic]

Some further discussion in IRC: https://freenode.logbot.info/whatwg/20181108#c1817142

[caridy]

I could not agree more with @domenic, you're trying to make this tamper-proof, but there are so many ways to go around it (e.g.: get the constructor, change proto, etc.) that I don't think it is worth it. I don't think that's the goal of the web-component APIs. Instead, you can provide those low-level primitives that people can use, and bend, and do crazy stuff with it, and that is the way-of-the-web.

[rniwa]

I think it was a mistake that attachShadow was allowed on a custom element without an opt-in. The fact anyone can add a shadow root on a custom element is highly undesirable. The existence of this mistake shouldn't encourage us to make more mistakes.

[annevk]

Another way to slice this might be to offer a way to opt-out of encapsulated features, a static unneeded taking ["shadow", "internals"], which would make attachShadow() and getInternals() (or attachInternals() if there's some kind of cascade going on) throw, similar to how they throw for the html element.

(I don't think anyone is trying to make this secure/tamper-proof as that is indeed not possible; this is about the encapsulation boundary.)

[domenic]

I really like @annevk's idea. A way of preventing unexpected uses without having to say "give me an X, even though I will do nothing with it" makes a lot of sense. It leads to more readable code and seems easier to optimize.

[tkent-google]

@rniwa , what do you think about the opt-out idea?

My opinion is that either of opt-in, opt-out, nothing is acceptable. I'd like to proceed this anyway :-)

[rniwa]

Opt-out seems good especially since it would also work for shadow root but unneeded doesn't seem like a good name though. Maybe disable? We probably need to come up with a slightly longer name since it would be a static class variable, and we'd want to be mindful of existing custom elements which may or may not have such a static class variable.

[annevk]

Paint color I'd be okayish with: disableEncapsulation = ["shadow", "internals"]. (I also considered disableFeatures and disableFunctionality, which I'm meh on, but could live with too. unneeded also seems fine though and probably unique enough given it's a little odd.)

[rniwa]

I don't think disableEncapsulation makes much sense because we're making attachShadow throw. disabling encapsulation on surface sounds like something you'd do to expose internals to the outside.

disableFeatures is a lot better name IMO although I think we can do better. We should probably name it disabledFeatures since this isn't really a method. Or maybe unusedFeatures?

[tkent-google]

+1 todisabledFeatures.

[trusktr]

Hey you all, but this attachInternals and disabledFeatures business implies there is a set of features that is pre-defined by the browser vendors, and does not let library authors define them. Furthermore, something like a disabledFeatures blacklist would not make sense with an unknown set of features provided by library authors.

Mixins are better in the sense that you can get them from anywhere (as builtins, or from libraries) and they are all opt-in.

Can we try to keep the API space symmetric between builtin and userland?

---

On another note,

protected-like methods from a mixin

Here's an idea on how to expose a feature from a mixin to a CE author, while letting the CE author decide to (or not to) expose the API to public space.

Suppose the mixin accepts a key. Then inside the module or closure where the CE class is defined, we can create the mixin:

const key = Symbol() // key can be any reference, like WeakMap keys.
const WithSauce = makeWithSauce(key)

then extend from it (assume the mixin extends HTMLElement by default):

const key = Symbol() // key can be any reference, like WeakMap keys.
const WithSauce = makeWithSauce(key)
export default class SaucyEl extends WithSauce() {}

Note we do not export the key.

Now to use a method createSauce provided by the mixin, we can pass the key in:

const key = Symbol() // key can be any reference, like WeakMap keys.
const WithSauce = makeWithSauce(key)
export default class SaucyEl extends WithSauce() {
  connectedCallback() {
    this.createSauce(key) // it works!
  }
}

Now suppose a user of the element tries to call the method on an instance of the element. The user's code runs outside the module scope where SaucyEl was defined, so

import SaucyEl from 'saucy-lib'
customElements.define('saucy-el', SaucyEl)
const saucyEl = document.querySelector('saucy-el')

saucyEl.createSauce() // throws an Error

const key = Symbol()
saucyEl.createSauce(key) // throws an Error

To expose the feature for public use, the library author just has to expose the key, or provide a public proxy method that doesn't require a key.

This technique is "protected-like", because a subclass (SaucyEl) is able to use the method from a parent class (the WithSauce mixin). But it is also private-like because the SaucyEl subclass has the key, but subclasses of SaucyEl don't, so the method is sort of private for subclasses of SaucyEl.

[tkent-google]

@trusktr thank you for the idea.

then extend from it (assume the mixin extends HTMLElement by default):

The mixin needs to support not only HTMLElement but also any classes extending HTMLElement as a base class. I have two concerns about it.

• Hard to standardize it because WebIDL is not capable to define such mixin.
• Implementability is questionable. Usually web platform features are implemented at outside of a JavaScript engine. I'm not sure if all UAs can provide such dynamic mixins with their JavaScript engines. Actually I can't imagine how to implement it in Blink with V8 though I'm a Chrome engineer.

[domenic]

I don't think we should be contemplating such unusual patterns for the web platform; we should stick with normal class hierarchies.

[trusktr]

@tkent-google

I can't imagine how to implement it in Blink with V8 though I'm a Chrome engineer.

Isn't there a feature of Chrome where new features can be implemented in JavaScript before they are (if ever) converted to C/C++? I've seen the debugger pause on such builtin features before. And, I'm just guessing, but if they can be implemented in JS, could the same mechanics be used on the native side?

Okay, assuming implementation details work out fine, is there anything bad about the pattern itself?

Why can't WebIDL define such a sort of feature? Is it because it is language specific (JS has this ability)?

In that sense, then aren't classes also language-specific and thus we shouldn't use them (f.e. Custom Elements)?

[trusktr]

we should stick with normal class hierarchies.

What constitutes a normal hierarchy?

Can't we say that after we've mixed a mixin class into a new class definition, that the new definition is now a normal hierarchy? It works just like other hierarchies, plus we have things like Symbol.hasInstace to make instanceof work fine, etc.

By the way, I'm not absolutely married to the idea, but so far I like it. I'm curious to know why it would be a bad pattern compared to others.

[trusktr]

Hell you all, I just want to reiterate that I feel passing options like needsInternal or default styles (re: https://github.com/w3c/webcomponents/issues/468) to customElements.define seems out of place because:

• authors define classes.
• authors can delegate the use of customElements.define to end users so that end users can name the elements as they wish.
  
  
  
  • it wouldn't make sense to require all end users to know the specifics of the class in order to pass those specifics into the customElements.define call.
  
  
  • This is what class is for! Let the author define the class specifics on the class, and let the end user use the element without worrying about the class specifics.
  
  

For this reason I think that mechanisms like @decorators, static props, and Mixins(). are better. _The author can decide what APIs a class needs, what styles it has, what behaviors it has, etc._

The end user just needs to define the element, and use it.

So in my opinion, authors of the custom elements should use class infrastructure to its fullest rather than putting parts of the class into customElements.define calls.

Author:

import styled from 'lib1'
import withRender from 'lib2'
import html from 'lib3'
import attribute from 'lib5'
// or const {attribute} = CustomElements // builtin
const [One, Two, Three] = window.ElementFeatures // builtin

@styled({
  foo: {
    color: 'skyblue'
  }
})
class AwesomeElement extends withRender(HTMLElement) {
  static elementInternals = [One, Two, Three]
  static userAgentStyle = new window.CSSStyleSheet( ... )

  @attribute('material-color', Color)
  materialColor = new Color('red')

  render() {
    return html`<div class="${this.classes.foo}"></div>`
  }
}

End user:

import AwesomeElement from 'awesome-element'
customElements.define('awesome-el', AwesomeElement)
document.body.innerHTML = `<awesome-el material-color="pink"></awesome-el>`

But then again, the author could abstract it away.

Author:

// adding onto the same file:
export const define = name => {
  customElements.define(name, AwesomeElement, {
    userAgentStyle: new window.CSSStyleSheet( ... ),
    elementInternals: [One, Two, Three]
  })
}

End user:

import define from 'awesome-element'
define('awesome-el')
document.body.innerHTML = `<awesome-el></awesome-el>`

I guess abstracting works, but now its an extra step.

If I'm already using class to define my class, why not just use class to define everything about my class? 😃

[trusktr]

The only reason why customElements.define even needs to accept a name is presumably to allow the end user to customize this important aspect of an element. Otherwise, all we need is

class MyEl extends HTMLElement {
  static get elementName() { return "my-el" }
}

customElements.define(MyEl)

So unless it is critical for an end user to change something, then I feel it doesn't need to go into customElements.define because we already have class.

[trusktr]

But then again, isn't an end user able to change stuff by extending from a class?

F.e., a class can easily be extended by an end user and a static elementName value overridden. It just seems like if we're using class, maybe we should be sticking class specifics inside the class definition rather than passing class specifics to an API outside of the class definition.

class MyEl extends HTMLElement {
  static get elementName() { return "my-el" }
}

class NewEl extends MyEl {
  static get elementName() { return "new-el" }
}

customElements.define(MyEl) // defines <my-el> element
customElements.define(NewEl) // defines <new-el> element

What would be the downside of having class-specifics inside the class definitions instead passing the specifics to customElements.define?

I can see passing the name into customElements.define makes it easy to do a simple re-name without having to write an additional class ... extends ... { static elementName = '...' }, but for other stuff like styles and API features, I think the class definition is there for that!

[tkent-google]

As for ElementInternals, we made consensus around TPAC 2018. Though I haven't received any feedbacks from TAG, a specification PR was already approved, and we'll merge it with a PR for form-associated custom element.

[justinfagnani]

Does anyone have an ElementInternals explainer?

[tkent-google]

@justinfagnani AFAIK, Form Participation API Explained is the only document other than the HTML specification.

[annevk]

In due course we should probably add an introduction section to the HTML Standard.

[frank-dspeed]

Maybe we should simply point out that customElements are not a magic thing without the define stuff and all that they are

//CustomElement based on HTMLUnknownElement
const myCustomElement = document.createElement('my-element')
//Then apply some lhooks or what ever to it
myCustomElement.connectedCallback = ()=> {
  this.innerHTML = '<p>Iam a Complex Application </p>'
}
document.body.append(myCustomElement)

// We can also use existing HTML Elements
const MYCustomElment = document.getElementById('my-element')
MYCustomElment.connectedCallback = () => {
  this.innerHTML = '<p>Iam a Complex Application </p>'
}
document.body.append(myCustomElement)

Now the leaved out question is how does get connectedCallback get called ? you can do it manual or via mutationObserve and register it then you can also attach more behavior or bindings its a customElement so your free

as long as you understand a CustomElement is a modifyed version of a HTMLElement Instance.