Aframe: How to deal with Shadow DOM?

[trusktr]

I'm wondering, have you tried transcluding (composing) scenes together by using ShadowDOM and the <content> (or the new <slot> in WebKit) element? If so, did it work, and how did you handle those cases?

Because, for example, I'm supposing that you are relying on attached/detachedCallback in order to construct an equivalent behind-the-scenes scene graph in Three.js. I am doing the same thing (constructing a virtual scene graph from custom elements, but in my case the graph is not made with Three.js) over at http://infamous.io.

The problem that I face is that when I try to transclude things into shadow DOM that everything just falls apart. It seems like the root elements (in my case, <motor-scene>, in your case <a-scene>) may need to keep it's own shadow DOM and have absolutely no nested shadow DOM, and that the user-land HTML, which may or may not contain nested shadow DOMs, needs to be able to work completely independently of the behind-the-scene tree structure, and that we'll need a way to traverse the user-land nested shadow DOMs in order to gleen the structure that we need to create (for example, to render with Three.js, etc), and then we need to construct that actual structure in our shadow DOM (if we intend to have elements in the dev tool for inspectability).

In your case, you are considering only WebGL. My case is more tricky: rendering to DOM (CSS 3D) is a required ability, and that DOM will be mixed with WebGL like what Famous was making (but has abandoned) over at http://famous.org.

TLDR: I am wondering if you guys have thought about transclusion in webcomponents (shadow DOM) and how you're dealing (or will deal) with it.

To get more of a sense of my problem, I also asked a similar question over at w3c/webcomponents.

[dmarcos]

AFRAME only uses the custom elements API. We decided to go that route because the API spec is mature and it can be easily polyfilled. We want AFRAME to reach the largest possible browser base without much performance hit and minimum bloat in the code base. We keep en eye on the shadow DOM but we have not played with it in the context of AFRAME but we are defintively interested. @trusktr how does your approach work across different browsers with no shadow dom implementation?

[trusktr]

@dmarcos

AFRAME only uses the custom elements API
how does your approach work across different browsers with no shadow dom implementation?

Same here, it works fine when we ignore shadow DOM (f.e. in browsers that don't have that).

My custom elements also do not use Shadow DOM, but the concern I express here is a little different: What if users want to use their own shadow DOM then they transclude our custom elements?

For example, suppose an end user writes this:

<!-- This is "light DOM" -->
<div id="scene-container">
    <a-sphere ...></a-sphere>
</div>

and then that user procedes to add a shadow DOM to the scene-container:

const el = document.querySelector('#scene-container')
const shadowRoot = el.createShadowRoot()
shadowRoot.innerHTML = `
    <!-- This is "shadow DOM" -->
    <a-scene>
        <content selector="a-sphere">
        </content>
    </a-scene>
`

This isn't a very good example, but shows the problem. What will happen is that the <a-sphere> element will be transcluded (placed) into the inert <content> element of the shadow DOM. The result is

  <!-- shadow DOM -->
    <a-scene>
        <content selector="a-sphere">
            <a-sphere ...></a-sphere>
        </content>
    </a-scene>

which results in the behind-the-scenes "flat DOM" (the thing the browser actually renders):

  <!-- flat DOM -->
    <a-scene>
            <a-sphere ...></a-sphere>
    </a-scene>

That's what I'm wondering about, if you guys have considered what would happen when a user writes HTML like that.

[trusktr]

@dmarcos Here's a better example. Suppose I want to make a layout component (using Custom Elements) that will accept AFrame elements and transclude them into the layout, where the layout's root element is an <a-scene>. Following is basic layout component (this better shows why one would use Shadow DOM as have above, in the context of a custom element component). Imagine that this simple component's goal is to use A-Frame in order to build a UI that has a header, footer, sidebar, and content area:

// awesome-layout.js
import 'aframe' // registers all the <a-*> elements

export default
document.registerElement('awesome-layout', class AwesomeLayout extends HTMLElement {
    createdCallback() {

        // give the component a shadow root, so we can transclude things into our component (similar to React).
        this.shadowRoot = this.createShadowRoot()

        // this is the content of our shadow root (similar to the render() method of a React component).
        this.shadowRoot.innerHTML = `
            <div class="scene-container">
                <a-scene>
                    <a-entity position="...">
                        <content selector="[header]">
                        </content>
                    </a-entity>
                    <a-entity position="...">
                        <content selector="[sidebar]">
                        </content>
                    </a-entity>
                    <a-entity position="...">
                        <content selector="[content]">
                        </content>
                    </a-entity>
                    <a-entity position="...">
                        <content selector="[footer]">
                        </content>
                    </a-entity>
                </a-scene>
            </div>
        `
    }
})

Now, this component can be imported and then used in a new component (we'll use React in this example) that will define the pieces that go in the header, sidebar, content, and footer areas of the layout:

import './awesome-layout' // registers the <awesome-layout> element
import React from 'react'
import ReactDOM from 'react-dom'

// render <awesome-layout> into the body of the page:
ReactDOM.render(
    <awesome-layout>
        <a-image header ...></a-image>
        <a-plane footer ...></a-plane>
        <a-plane sidebar ...></a-plane>
        <a-plane content ...></a-plane>
    </awesome-layout>,
    document.body
)

This is a contrived example, but you can get the gist of the concept: we've rendered an <awesome-layout> element, and in it we have placed four items that will get transcluded (placed) into the ShadowDOM of the awesome-layout. As you will be able to guess, the a-plane and a-image elements will be moved into the corresponding a-entity elements of the awesome-layout component. Keep in mind that when calling parentNode on one of the transcluded elements after is has been transcluded will result in a reference to the <content> element, not the <a-entity> elements. So it seems that for a ShadowDOM-compatible solution to exist, the fact that parentNode may be a <content> element and not a <a-*> element has to be taken into consideration (same with the upcoming <slot> elements).

We can also note that it makes sense for the a-plane or a-image elements to be transcluded into the a-entity elements, but it may not make so much sense for them to be transcluded into any other type of elements.

So, that's basically the idea I'm working with, how to handle transclusion so that the custom elements still work and so that a behind-the-scenes scene graph can be constructed properly (in your case with Three.js, in my case with a custom implementation).

Cheers! :]

[trusktr]

(Also note, in the current state of things, <content> elements from ShadowDOM v0 are being replaced by <slot> elements in ShadowDOM v1, and the <content> element selector doesn't seem to work in Chrome, but you can reproduce the issue by not using a selector and having only a single <content> element, in which case all the element outside of the shadow root will get transcluded into the single <content> element.)

There's also the issue of end users possibly appending new elements into the awesome-layout, in which case I believe some extra logic is currently needed in order to place those into the ShadowDOM. I believe this might be fixed with slots in v1 though.

[trusktr]

@dmarcos

@trusktr how does your approach work across different browsers with no shadow dom implementation?

I'm planning to use polyfills for now, but I think by the time my library is ready (when it has it's first docs up and demos) then by that time all browsers might already support ShadowDOM natively.

On another note, I believe I have a solution to the problem (which your shadow-dom-using users will eventually run into), but I haven't implemented it yet. The solution is not too difficult of a change to make with regards to my library, which I'll do soon...

[trusktr]

Wait, nevermind, that solution only covers cases where children are invalid, but if a <motor-node> child is attached to somewhere other than a <motor-node> or <motor-scene> then there's no way for the child to know the problem when distributed into a closed Shadow tree.

There has to be some other thing I can do to complete the solution, as parent-to-child observation only solve half of the problem when the parent is indeed one of my elements.

[ngokevin]

Don't think shadow DOM is necessary (at least for core). The template component or using three.js scene graph to represent as the "shadow" scene graph are other options

[trusktr]

@dmarcos @ngokevin Here's two jsfiddles that illustrate the problem.

This first example is basic, and works: https://jsfiddle.net/trusktr/1ecfha5h

In this second example, we are using ShadowDOM in a very simple way, and the example fails to work: https://jsfiddle.net/trusktr/1ecfha5h/1

I believe it's important for the A-Frame elements to work with ShadowDOM because ShadowDOM is (currently) an important part of the future of components in the web.

I'm beginning to tackle this in my own lib; I'll report back on how I solve it.

[ngokevin]

Actually not too worried about shadow DOM. I don't think it solves many issues in the 3D/VR space where the actual HTML ends up pretty hidden, and we have a lot of components/tools to abstract/render subtrees (whether in DOM or 3D scenegraph). Even in Web Component land (which is not too adopted itself), shadow DOM isn't well adopted. But yeah, if you can find an easy fix that doesn't require much maintenance, cool!

[trusktr]

Hey guys, the way I solve ShadowDOM in infamous is actually super easy. I am using MutationObserver to detect changes in children. Then, f.e., if a child is added to a node, and the parent also has a shadow root, then in my JavaScript scene graph I simply add the child to the parent as normal (just like you do) but then I also put something like a isDistributed flag to true on the child if it matches the selector (v0) or slot name (v1) in the shadow root.

In parents with shadow roots, I add a shadowChildren collection to them which contains the top level elements in the shadow root. It is easy to update this list using a MutationObserver on the parent's active shadow root (in v1 ShadowDOM there can only be one shadow root).

It is just some book keeping.

When traversing the scene graph for rendering, you normally traverse to children. However, you simply traverse to shadowChildren if you see that a parent node has a root.

With some event tracking on the slot elements, you can then keep track of where a node got distributed too. So when you get to a node with a slot, you will have book-kept which nodes are distributedChildren for example.

What you end up with (using my method) is the same tree with parents and children which mirrors what you see in the "light DOM", and then you also have these extra collections representing the composed tree.

When creating world transforms, you would do those based on the composed tree that you traverse.

We want AFRAME to reach the largest possible browser base without much performance hit and minimum bloat in the code base. We keep en eye on the shadow DOM but we have not played with it in the context of AFRAME but we are defintively interested. @trusktr how does your approach work across different browsers with no shadow dom implementation?

In browsers with no ShadowDOM, not to worry. People don't have to use ShadowDOM in those browsers, or they can supply their own polyfills if they do.

The thing is, you wouldn't be forcing people to use shadow Dom, you would only be compatible with it for users that have those browsers, otherwise A-Frame would continue to work just like before. It entirely an opt-in feature for users that have compatible browsers.

If you search for "shadow root" or similar at http://GitHub.com/trusktr/infamous you will probably find my implementation of the book keeping.

I recently just added WebGL, which is currently only rendering from the "light tree", not looking at the shadow Dom stuff yet.

When I get more time, I can explain it better by pointing to code references and examples once my WebGL is using the ShadowDOM book-kept stuff...

[trusktr]

It might be tempting to write components using aframe-react because the component model is perhaps nicer and cleaner than with shadow DOM, but writing with shadow has the result that new HTML components can be used everywhere other than with React.

I'm still pondering on how to do this as cleanly as possible with my Elements while still providing an imperative API that would work if the custom elements are disabled.

That's one thing I've been working towards is allowing an option to disable DOM so that no DOM is outputted of using the imperative API, rendering straight to WebGL with no DOM overhead). I'm thinking of how to decouple the SahdowDOM features from the imperative features in a way that the imperative stuff will work fine without shadow DOM. This will make the API flexible for use in many ways, even outside DOM.

[donmccurdy]

One important note that I don't think has been mentioned in this thread — A-Frame does not, by default, write attribute values to the DOM. If an entity's position changes on every frame, A-Frame will only add the position attribute to the DOM once, and after that all updates are virtualized. More information in docs for the debug component.

All that to say, A-Frame's entities are already a sort of virtual DOM — if you can measure noticeable DOM overhead (after initial scene setup) I would be surprised.

Being able to run A-Frame in DOM-less environments, e.g. serverside, is not built in. I would be curious how that goes for you, if that's something you're planning to try.

[trusktr]

@donmccurdy

A-Frame does not, by default, write attribute values to the DOM. If an entity's position changes on every frame, A-Frame will only add the position attribute to the DOM once, and after that all updates are virtualized.

I saw that in aframe-react, it passes non-string values to setAttribute. This is what I've been planning to do in infamous too. 👍

Being able to run A-Frame in DOM-less environments, e.g. serverside, is not built in. I would be curious how that goes for you, if that's something you're planning to try.

I'm planning to do SSR stuff, but more like outputting HTML from the server, then the HTML becomes live on the client, so that will work. If I actually want to _run_ it on the server, I'd probably use Chrome headless or something, so it'll still work.

---

Anywho, those points aren't necessarily related to ShadowDOM, in the sense that they don't prevent A-Frame from being ShadowDOM-compatible. Because, for example, if I add a shadow root to a parent a- element, then the children will be distributed somewhere into the shadow tree. A-Frame could have MutationObservers and slotchange event listeners that detect the distribution of nodes and A-Frame can then update the virtual scene graph (the "sort of virtual DOM" you mentioned) based on those changes, regardless of what setAttribute does.