Knockout: Virtual-dom?

[unsafecode]

There has been a lot of discussion lately about virtual-dom (https://github.com/Matt-Esch/virtual-dom/wiki, http://futurice.com/blog/reactive-mvc-and-the-virtual-dom/), so I was wondering: will it be possible to leverage the approach in Knockout? Will it require a serious refactor of DOM-related code, or can we design a separated module?

Also, would it be actually beneficial for performance? It will be interesting to set up a comparison, since most benchmarks in the wild do not measure Knockout againt virtual-dom.

[brianmhunt]

Thanks @unsafecode - interesting projects!

Knockout has had virtual elements of a sort for an aeon(2011).

Are there any benefits of this new virtual-dom over the virtual-elements already in KO? I feel like virtual-dom is on a different wavelength (i.e. it'd be a pretty big refactor to KO), so to speak, but it would be interesting to know if there were any benefits that could be brought over.

Cheers

[mariusGundersen]

virtual-DOM is not the same as virtual-elements.

I've had some problems recently that I believe a virtual DOM could help out with. Lets say you have an observableArray of objects that is populated with the result of a JSON request to the server. The JSON returned would be an array of objects, and the request can be done many times, but it returns almost the exact same result each time. The observableArray is bound to the DOM to create a navigation menu of <a href> elements, so you can use tab to keyboard navigate through the menu. When you click on an item in the menu, the XHR request is made, JSON is fetched from the server and the observableArray is updated. At this point the element that had focus will be replaced, since it is not the same object it was before, and the browser will reset the tab index, since the element that had tab index doesn't exist anymore. Chrome will reset the tab index to the start of the document, Firefox will reset it to the parent element.

When the observableArray is updated the foreach binding will discard any DOM element bound to an entry that doesn't exist anymore. It could look at the entries in the list and realize that the new entries are pretty much exactly the same as the ones it had before, and reuse the exist DOM it has. This is what a virtual DOM can do.

Here is a simple example that shows the issue: http://jsbin.com/suhanupefo/4

[brigand]

Knockout starts with the browser's dom, and modifies it with JavaScript. "Virtual dom" starts as plain objects in JavaScript, and changes are 'flushed' to the browser dom, often using a diffing strategy for potential performance gain.

It would require a major rewrite, most likely break compatibility with many binding helpers, and have questionable benefit.

[brianmhunt]

Many thanks for the explanations and opinions.

It may be easier to update the observableArray and related foreach bindings to respect changes with minimal DOM changes via the arrayChange event, per #1584

[brigand]

I would close this in favor of #1584... foreach is the largest area of concern. If anyone's interested, using a transform of knockout templates you could generate React.js code for use as a virtual dom rendering engine for KO. No changes need to be made to KO to accomplish this, but you need to rewrite builtin binding handlers in a React compatible way. This is much easier in KO than something like angular due to the rigidity of templates.

It'd be an interesting experiment to see if there's noticeable difference in performance. I don't have the time to do this, but I'd be happy to optimize the engine after it's working.

[brianmhunt]

Thanks @brigand -- @unsafecode are you cool with closing this in favour of #1584, since that (foreach loops) is really the key benefit we would seem to be aiming for here and it doesn't require a relatively massive rewriting? I might be wrong though. :) Are there other benefits to the virtual DOM that make it worth investigating further and keeping this issue open?

[mariusGundersen]

Just a follow up on the fact that a virtual dom isn't faster than clever code or using plain knockout: http://react-benchmark.mariusgundersen.net/

[matthew-dean]

Are there other benefits to the virtual DOM that make it worth investigating further and keeping this issue open?

Yes, there are probably bunches, but the basic idea is that you're never writing directly to the DOM when a model changes. You send DOM updates to a queue, and a view manager diffs the DOM and decides the fastest method of insertion / updating. Sometimes it's changing the attributes of an element or replacing that node, sometimes it's an innerHTML operation, and, as others have said, it waits to actually make those changes until an animation frame is available, to limit jankiness and repaints.

However, one of the OTHER things the view manager in React can do is insert based on visibility state. That is, say your model represents a really long list, like a scoreboard. The scores keep updating the model, and React keeps updating the virtual DOM. But the scores are in a long scroll list. Therefore, you tell the view manager to only insert the DOM nodes for where they would be visible in the scroll list.

So, in that way, the Virtual DOM represents the elements on the page, but it's actually a kind of superset of the DOM. The actual DOM is minimized to the fewest number of elements needed for rendering at any given time.

Incidentally, Ractive.js uses a declarative syntax like Knockout, and manages updates with a virtual dom for very slick performance. http://www.ractivejs.org/ - However, Ractive doesn't have as easy-to-use two-way databinding, IMO, which is why I've just started using Knockout for one of my projects even though I normally use Ractive.

[matthew-dean]

@brigand

If anyone's interested, using a transform of knockout templates you could generate React.js code for use as a virtual dom rendering engine for KO.

I think that's a non-starter. The two platforms have different requirements, and you would then need to insert both frameworks into your page, no? I can't see anyone wanting to do that, because any performance gained by virtual DOM you've subsequently killed (at least in part) by having two rendering engines.

[matthew-dean]

More about virtual DOM from the writer of the React virtual component a virtual DOM library (https://github.com/Matt-Esch/virtual-dom) is here: http://stackoverflow.com/questions/21109361/why-is-reacts-concept-of-virtual-dom-said-to-be-more-performant-than-dirty-mode

[brianmhunt]

I just wrapped my head around the virtual dom, and it's wickedly cool.

I think we'd need a whole other abstracted operational-layer in-between the binding handlers and the DOM, but there's definitely some space for this to be explored. I don't know a middle layer can totally abstract it away, but I think it's definitely worth a shot if the performance benefits are there.

[PaulHuizer]

Isnt angular 2.0 much more performant than angular 1.x? Could something be learnt from that?

[chrisknoll]

I like the idea of the virdual dom, and i'm wondering if databindings to actual HTML nodes is much different than binding to virtual ones.The idea behind virtual nodes would be that they behave the same as physical nodes, yes? I haven't read up enough on React as to how you start with an html page and it gets munched into virtual nodes, but at some level, my sense of it is that the react virtualizing of the dom nodes is separate from knockout's physical binding to nodes that shoudln't they just co-exist in such a way that they don't even need to know about each other?

Maybe this can be contained in the handling of creating nodes beneath a template or component binding such that we just carve out a little of the virtual-dom-ness of things in those specific contexts?

Or maybe that's what the talk about the if: binding optimization is talking about?

[brianmhunt]

@chrisknoll Yeah, I think you hit the nail on the head. The strength of ko strikes me as the markup-oriented approach, which is a lower barrier to entry and easier to lay out (IMO) than React's virtual nodes, KO is loosely connected to the DOM via views through bindings, and the simplified logic is visible in the markup. The strength of React strikes me as the speed, compartmentalization and reusability, Javascript-first logical structure with a tight connection between DOM and data.

I see no reason why React could not co-exist with Knockout, other than developers having to know and maintain two frameworks (which is not to minimize that reason, only that it's the only one I can think of).

[matthew-dean]

I think you're comparing the wrong thing. React uses virtual DOM AND it has a vastly different concept of views. Virtual DOM =/= React's use of virtual DOM, which is almost a necessity because of it's extreme abstraction away from HTML.

A better parallel if you want to see virtual DOM architecture that is markup-oriented / markup-based is to look at Ractive.js, a library whose performance is quite good, and the library is tiny compared to the behemoth that is React. (Seriously, comparing anything to React is a fool's errand. It's a whole different paradigm from most frameworks.)

Of course, not sure if Ractive is a hairy topic since it was basically created to be a more performant take on Knockout with simpler syntax... But maybe even more worth looking into for that reason?

Another one is http://riotjs.com/. Looks cool but I've never used it so can't speak for how good it is.

[brianmhunt]

Good thoughts, @matthew-dean - Cheers :beers:

[alexswan10k]

If anyone is interested I did bash out a primitive sandbox/prototype to test out feasibility of react/knockout integration:

https://github.com/Metal10k/knockout-react-sample

This lets you bridge between the knockout and react worlds with relative ease.

To put this into context, my organization have a rather large application driven by knockout components, and we are experiencing serious performance issues specifically with large numbers of nested components (and foreach of course). I have tried to simulate these situations to some extent using the provided layout.

Unfortunately initial numbers do not look so great, so as it was suspected, there is a certain overhead with bringing in a second framework, and plumbing both paradigms together. Maybe someone can push this further though.

[k-ode]

@Metal10k Cool stuff! About the overhead of brining in React - you could try using a smaller dependency, like virtual-dom or snabbdom. We use snabbdom to bring virtual dom to knockout components at work, and so far it works great for us.

[alexswan10k]

This sounds really interesting. Have you got any kind of examples of how you might go about this? I am not really sure where to start if i'm honest. Would you databind to a virtual node?

[brianmhunt]

An aside: @Metal10k - If foreach performance is the bottleneck, you may be interested in knockout-fast-foreach

[k-ode]

Yeah, sort of. We use a binding handler to initialize the virtual dom, and a wrapping component view model to handle when data should be updated.

ko.bindingHandlers.virtualDom = {
    init(element, valueAccessor) {
        const { Component, params } = ko.unwrap(valueAccessor());

        const div = document.createElement('div');
        element.appendChild(div);

        this.vdom = Component({ element: div, params });

        ko.computed(() => {
            // Options here is a deferred pureComputed that has dependencies on 
            // all the data that we want to pass down to the patch method.
            const options = ko.unwrap(valueAccessor());
            this.vdom.update(options.params);
        });

        return { controlsDescendantBindings: true };
    }
}

import snabbdom from 'snabbdom/snabbdom';

const patch = snabbdom.init([
    require('snabbdom/modules/class'),          // makes it easy to toggle classes
    require('snabbdom/modules/props'),          // for setting properties on DOM elements
    require('snabbdom/modules/style'),          // handles styling on elements with support for animations
    require('snabbdom/modules/eventlisteners'), // attaches event listeners
    require('snabbdom/modules/attributes')
]);

class VirtualDom {
    constructor({ element, params = {} }) {
        this.element = element;
        this.params = params;
     }

    update(params) {
        this.params = params ? params : this.params;
        this.element = patch(this.element, this.render(this.params));
    }
}

class TestComponent extends VirtualDom {
    constructor({ element, params } = {}) {
        super({ element, params });
    }

    someMehtodThatHasToCallUpdateManually() {
       this.update();
    }

    render(params) {
        return <div>{params.title}</div>;
    }
}

The components view model could then look like the following.

class ViewModel {
    constructor(params) {
        this.title = ko.observable('test');
        this.virtualDomOptions = ko.pureComputed(() => {
            return {
                name: TestComponent,
                params: {
                    title: this.title
                }
            };
        }).extend({ deferred: true });
    }
}

// template
`<div data-bind="virtualDom: virtualDomOptions"></div>`

This will re-render anytime the data changes, but also leaves the options to call update manually (useful if you are using some private state, for instance).

Using deferred updates usually means re-rendering is really efficient, but you could go even further and use immutable data and look into things like Snabbdoms thunk function.

[alexswan10k]

@brianmhunt We are already actually using fast-foreach and it has definitely improved the situation, so thanks for that :)

@kimgronqvist This is indeed pretty cool so thanks for sharing. Unfortunately however from what i can tell you are using a FRP pattern inside of a component, which is most likely fine for what you are doing, but unfortunately we have a large number of existing highly composable components that use the conventional MVVM pattern, and ideally these need to just work as is.

What I was hoping for was a way to literally ko.applyBindings({}, virtualTree) and then use this virtual tree to update the real tree at throttled intervals perhaps (using rxjs or similar to perhaps mediate the process)? This way a whole subtree could be wrapped in a 'bridge' component (similar to how I have bridged between react and knockout in my sample), and thus could be updated 'offline'.

Having briefly played around with both api's for virtual-dom and snabbdom, neither appear comprehensive enough to fool knockout. There is then also the problem of getting back any changes. I guess this was perhaps a long shot.

I am reasonably confident from profiling that highly nesting templates is causing the most trouble. As I understand it each component effectively rips out all of the child nodes ($componentTemplateNodes), only to be reinserted with the template binding to the next layer. This happens over and over for each level of the hierarchy, and thus causes a huge amount of addNode and removeNode calls (especially in loops).

My markup looks structurally something like this (obviously in the real world ko-template is replaced with real components that do something useful)

                  <ko-template>
                        <ko-template>
                            <ko-list params="length: 12">
                                <ko-template>
                                    <ko-template>
                                        <ko-template>
                                            <ko-component></ko-component>
                                        </ko-template>
                                    </ko-template>
                                </ko-template>
                            </ko-list>

                        </ko-template>
                    </ko-template>

[alexswan10k]

I wanted to briefly just update this thread as I have had some success with binding to a virtual dom via another react tool called react-faux-dom
https://github.com/Olical/react-faux-dom.

React-faux-dom is a virtual dom with an api surface which (unlike most others I have seen yet) can substitute the browser's. This is currently used by some to allow using d3 in a more pure functional manner, and also removing the requirement that d3 manages the underlying DOM nodes directly. This opens the door to server side rendering, etc.

The basic idea here is that react-faux-dom creates a tree of virtual nodes. These are then bound to via ko.dataBind({}, fauxDomNode). Knockout then manipulates these nodes, and at some indeterminate time in the future, these changes are then propagated back to the real UI via reactDOM.render(fauxDomNode.toReact(), document.getElementById('react'));.

I appreciate this sample is incredibly simple (binding a text) and I have had to fudge in a number of missing api's to faux-dom, but I think this proves the principle. React-faux-dom can most likely be expanded to cover the needed knockout api's. React is then also used to solve the dom diffing aspect of the problem. The only problem left untouched is determining when to propagate changes back to the UI.

https://github.com/Metal10k/knockout-react-sample/tree/react-faux-dom-test

[IanYates]

🎉 @Metal10k - that looks pretty neat