Svelte: Benchmarking

[Rich-Harris]

There are lots of changes we could make to the way that client-side code is generated — using innerHTML for example, or concatenating text nodes, or creating update functions like this...

function getUpdater ( node, attr, value ) {
  node[attr] = value;
  return function ( newValue ) {
    if ( newValue !== value ) {
      node[attr] = value;
      value = newValue;
    }
  };
}

function renderSomeFragment ( root, component ) {
  var p = createElement( 'p' );
  var update_p_class = getUpdater( p, 'class', root.foo );

  return {
    mount: function ( target, anchor ) {
      // ...
    },

    update: function ( changed, root ) {
      if ( 'foo' in changed ) update_p_class( root.foo );
    },

    teardown: function ( destroy ) {
      // ...
    }
  };
}

...rather than the current approach of doing lots of book-keeping inside renderSomeFragment.

All of these ideas, and dozens more, are likely to have trade-offs between performance generated code size. In order to make educated decisions we need a decent way to track performance across browsers between different versions, and between the last version and a particular branch.

• The tests should be based on non-contrived real world applications as far as possible
• Automation would be nice but is of secondary importance, I think
• We would ideally track performance, memory usage, generated code size and JS parse time (and compile time? not important, but certainly no harm in tracking it). Anything else?

So, given all that... where do we start? Never been terribly structured about this sort of thing, so am open to any and all ideas.

[cristinecula]

The Angular team has developed benchpress, a tool for running macro benchmarks on real application code and it's not limited to the Angular framework.

I have not used it personally, but it looks like a good starting point.

[Rich-Harris]

I was having a poke at benchpress over the weekend. I like the idea, but it's hard to use and lacks documentation. It also only works in Chrome and iOS Safari (Firefox support is coming).

So automation might have to wait for another time. Instead I've put together a separate project called svelte-bench, which runs various tests across different versions and produces reports like this:

There's still lots to do, but it's something to iterate on.

[Swatinem]

Nice work!

I remember I have mentioned helpers like your example getUpdater already in a previous PR. I like the idea.

I would have also liked to move things like if and each completely into helpers but the way you handle nested scopes, (like passing the each index as a separate argument) prohibits that for now.

Concatenating text nodes is already tracked in #14 and is basically also needed for generating hydration code when you want to use SSR properly.

[tbillington]

In reply @Rich-Harris about performance vs code size, do you think it'd be possible to have a compile time switch which will go between them?

I don't know it if will make the implementation overly complex (or impossible).