Choo: improve perf

[katopz]

Hi,

I'm testing choo against others and really wish it perform great but it turn out to be really slow there.
so I think I should ask you first to ensure that I didn't miss something which mess the results.

here's the code https://github.com/rabbots/todomvc-perf

Thanks

[yoshuawuyts]

hahaha, lol - that's super interesting! - this has not been tuned for perf _at all_ so ummm yeah we'll look into it (pretty sure I know where some of the easy gains lie haha). Do you by any chance have a source file available? - https://github.com/rabbots/todomvc-perf/blob/master/todomvc/choo/dist/index.js seems to just be a dist. Thanks! :sparkles:

[yoshuawuyts]

Also I'm pretttttty sure it's because reloads don't have an upper bound; so every mutation to the state results in a full re-render - which is fine in real world scenarios (how often do you click on a thing, or type a key) but shows up horribly in benches

Oh and every re-render on the same view currently does a full router traversal which isn't _super_ slow, but slow enough to impact micro benches like these.

Ahhh well haha, we'll fix this after v3 as this doesn't quite have any real-world implications at the moment :')

[katopz]

Nice to here that, Please keep me post when you done 😃
BTW, I build from your todomvc example without touching anything there https://github.com/shuhei/todomvc-choo

Just build and cp over there should did the trick.

Thanks for your great work. I really hope that choo is cute, small and fast at the same time ⚡

[yoshuawuyts]

Also: I've got #99 issues and I guess perf is one

_drumroll_

[yoshuawuyts]

@katopz haha I'm pretty sure once this is fixed we'll be comparable to react's speed - last I checked virtual-dom was about 2x as fast as React; with morphdom being about the same speed.

I've done some instrumentation on the call stack before and things generally look good; so I think we should start with the low hanging fruit before diving into smoothing out tiny fixes. Some known low hanging fruit would be:

• [x] batch updates to DOM (#81)
• [x] memoize last route in router (https://github.com/yoshuawuyts/sheet-router/issues/17)

Currently a click from the DOM generally only has the overhead of accessing an object (reducer), calling the synchronous function in it, accessing the router and then creating the DOM tree. So I reckon once this is fixed we'd be mostly limited by what morphdom can do

Also I'm renaming this issue to "improve perf" so we can keep track of perf stuff here - really appreciate you opening this issue! :grin:

edit: lil extra note for anyone concerned with "OH NO, CHOO IS SLOW RIGHT NOW" - the fact that mutation changes aren't capped at 60fps means that above 60 mutations a second we're not batching updates. Given that that's the exact test that's being run, we obv come out last. If your application doesn't trigger 60 actions a second I doubt there's any real impact; if anything: choo might actually be faster than others _because_ we're not using any sort of raf - which yeah, we should probably do - but this is def something to reflect on when looking at these particular benchmarks. Tl;Dr: we're probably about react-speed perf wise for any real world scenarios right now

[fyrkant]

Updated the todomvc to choo 3.0.1 and compiled with yo-yoify, cut the time in about half!

[yoshuawuyts]

oooh, seems like that confirms the suspicions I had. Out of curiosity, would you mind running the raf branch? - I reckon this might slash it further :sparkles:

[fyrkant]

HOLY MOLY!!

[yoshuawuyts]

lol

[toddself]

💯💯💯💯💯💯💯‼️

On Jul 6, 2016, at 06:43, Yoshua Wuyts [email protected] wrote:

lol

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[fyrkant]

Hmm, seems like I was a bit over-eager, doesn't seem to work, just todomvc-perf that got fooled or something... Investigating.

[yoshuawuyts]

hahaha, and looks like I was over eager to post it on twitter - ah well, it's not like scoring high or low on this bench is reflective of real world scenarios anyway haha :sparkles:

[fyrkant]

Sorry!! 🙇

[katopz]

Watch out for event trigger there, you should see 100 items adding on screen while it render. Do select checkbox to 2 items there for more easier investigation. 😁

[localvoid]

Out of curiosity, would you mind running the raf branch? - I reckon this might slash it further

It will be even better to use batching with setTimeout(update, 10000) instead of rAF, it will significantly improve numbers in benchmarks. Cheating in benchmarks by preventing work that everyone else is doing won't make it faster.

[yoshuawuyts]

@localvoid hah, are you being salty? - don't be, these benchmarks are meaningless anyway. If they're off, we'll get them fixed - no worries (':

[fyrkant]

Got the raf-branch version working again (by adding prev to the _frame call here) and it's still as fast! But yeah, still meaningless, I guess! 😄

[yoshuawuyts]

oooops - hahaha yeah so I published nanoraf v2 yest which requires a prevarg - ahhh welll :sparkles: Heaps thanks for benching this :grin:

[kristoferjoseph]

Seems important to break down what the learnings are from these perf tests.

[yoshuawuyts]

@kristoferjoseph haha; there are few I reckon - if I had to sum up why choo might be fast I reckon it's because:

1. function calls are quite cheap, especially if you declare them once and then call them over and over
2. allocating new objects is kinda expensive, try and keep new arrays / objects to a minimum
3. if / else statements are cheap; using them is cool

And that's it I probably? Does that answer your question?

edit: if the question is about the perf tests themselves: I guess that they're kinda redundant as when anything hits above 60fps it's like fine - there's so many more factors that are wayyy more important:

• how big is the API? How much new stuff must be learned?
• how fast is the application overhead? how many bytes are we sending over the wire?
• how well is data isolated? how many things do I have to remember while working on the app? can teams work on separate features in parallel?
• do I need to learn a new language?
• how good is the documentation?
• what's the tooling looking like? How many steps does it take to get started?
• are the people in the community helpful? Do they make me feel welcome and comfortable?

Feel that kinda stuff is a lot more important than some chart that says: "hey, we can render 60 fps more efficiently than you can" - in the end it's all smooth and nobody can tell; other factors matter more.

[kristoferjoseph]

Less of a question, more of a request to summarize the learnings from making the todomvc perf app and what the results actually show.

If I were to form this as an actual question I would say:

Did adding raf actually speed things up or just make the test harness reflect reality better?

I know what I think, but would be cool to break it down for people that are just wooed by colorful graphs. 😉

[yoshuawuyts]

Did adding raf actually speed things up or just make the test harness reflect reality better?

In addition to the edit I posted on my prev post, I think the addition of raf might make a difference in some real world scenarios. This is a different stance than I had before.

I realized that when creating higher level effects, it might very well be the case that multiple models are called. If each of those models changes a bunch of state, then mutations might cause a spike in performance, resulting in yank / lag while navigating. By flushing state changes to the DOM on a set interval, we prevent any outliers from messing up performance - ever.

So yeah, it definitely helps for the benchmarks - and managing bursts of updates in real world scenarios too. It's important to keep in mind what this benchmark tests though: which is _"running a bunch of mutations of the state"_.

Hope I'm making any sense at all here, haha :sparkles:

[kristoferjoseph]

That was exactly what I hoped you'd say. The other comments could be misinterpreted, but the facts about perf gains from batching updates I feel are important for people to see.
Thanks for clearing that up and also thanks to @mw222rs for adding the perf tests!

[katopz]

No worry guys, VueJS already discuss about this pref and even remove the results from their page. But the things is VueJS pref test is on par with other not totally far behind.

The thing is if you don't look at the graph but just what you saw when it rendering.
Can you tell the different there? If yes, something really wrong with your frame work tho.
If no, do try again on mobile and answer that question again. And that's UX user maybe see :)

And this is not just one pref/bench I'll run, There will be more to come I just want to make sure you guys are in good shape before I begin the other test. :)
https://github.com/rabbots/benchmark.js
https://github.com/rabbots/js-repaint-perfs
https://github.com/rabbots/lighthouse

Do tell me if have more to add, maybe some pref you trust. I'm open all ears for it. :grinning:

Cheers

[yoshuawuyts]

@katopz oh cool! - yeah if you run the raf branch it should be good to go; I reckon if there's a lot of deeply nested paths then the router might become slow-ish (never super slow, trie structures are cool), but besides that I think raf should be good enough for now :sparkles:

edit: yeah, and like you say: all these perf benchmarks are not super meaningful - if anything it will let us know if there's got some extreme case where we fail and should pay attention to

[yoshuawuyts]

Hmmm, might also be interesting to add a choo version to this one - https://github.com/krausest/js-framework-benchmark :sparkles:

• http://www.stefankrause.net/wp/?p=301

[forresto]

More on Elm's benchmarks... not using rAF: http://elm-lang.org/blog/blazing-fast-html-round-two

[yoshuawuyts]

@forresto lol what - seems they really care about their benchmarks - very interesting stuff tho

[Ryuno-Ki]

Did adding raf actually speed things up or just make the test harness reflect reality better?

If I remember correctly, callbacks of setTimeout are queued when the active tab is left and “caught up” once the user navigates back to it - which means, you'll see pretty bad animations (the browser tries to quickly execute all the missed callbacks).

requestAnimationFrame (a.k.a. raf) would prevent that.