React: ReactDOM 15.6.1 vs 16.0.0-rc2 benchmark difference

[mobinni]

Do you want to request a feature or report a bug?
Hey there,

So at my work we got into the discussion of what the fastest render cycle would be. Namely, if functional components that render functional components as children would be faster than a class-based component that renders through private function members of the class.

We set out to prove it in a very simple and naïve way. We used Benchmark.js to run ReactDOM.render against components that render the same output to different (existing) dom-nodes.

What is the current behavior?

We tested it against React 15.6.1 and 16.0.0-rc2 and the results for both tests against all 3 cases were relatively close in terms of ops/sec. However between React 15 and 16 the amount of ops/sec varied so heavily, I wanted to know if the approach taken was the problem or if it shows a flaw in ReactDOM v16.

I'm not suggesting there is anything wrong, however the results varied so widely, I thought I'd share it here, since maybe someone would have more accurate numbers, or a better way of testing it that would actually highlight the perf.

Dev build
ReactDOM 16 results:

render functional component tree x 8,756 ops/sec ±7.80% (49 runs sampled)
render class based component tree x 13,924 ops/sec ±4.44% (50 runs sampled)
render class that renders functional components x 12,921 ops/sec ±7.06% (21 runs sampled)
Fastest is render class based component tree, render class that renders functional components```

ReacDOM 15.6.1 results:

render functional component tree x 33,815 ops/sec ±4.34% (23 runs sampled)
render class based component tree x 100,684 ops/sec ±4.01% (43 runs sampled)
render class that renders functional components x 109,085 ops/sec ±2.48% (56 runs sampled)
Fastest is render class based component tree

Production build
ReactDOM 16 results:

render functional component tree x 299,810 ops/sec ±8.57% (49 runs sampled)
render class based component tree x 83,106 ops/sec ±8.96% (41 runs sampled)
render class that renders functional components x 88,360 ops/sec ±4.81% (54 runs sampled)
Fastest is render functional component tree

ReactDOM 15.6.1 results:

render functional component tree x 200,604 ops/sec ±5.43% (50 runs sampled)
render class based component tree x 400,852 ops/sec ±4.37% (34 runs sampled)
render class that renders functional components x 413,435 ops/sec ±1.98% (55 runs sampled)
Fastest is render class that renders functional components,render class based component tree

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem via https://jsfiddle.net or similar (template: https://jsfiddle.net/ebsrpraL/).

https://github.com/mobinni/react-perf-dom

What is the expected behavior?

yarn build
yarn serve
go to localhost:9000
open console, output should show up after it runs the tests

Which versions of React, and which browser / OS are affected by this issue? Did this work in previous versions of React?

15.6.1/16.0.0-rc2
Chrome 60
node 7.8.0

[aweary]

Hey @mobinni, it looks like you're running these benchmarks with the development build of React using webpack-dev-server with the eval-source-map devtool option, which isn't meant for use in production. If you're running benchmarks you should make absolutely sure that you're using production builds, as the development builds contain extra checks and validation that will skew your results.

I recommend looking over these resources for more information:

https://facebook.github.io/react/docs/optimizing-performance.html
https://webpack.js.org/guides/production/
https://webpack.js.org/configuration/devtool/

[mobinni]

That makes so much sense, let me re-run these!

[mobinni]

@aweary So I ran everything again against a production build and the results were WAY better, however it seems that reactDOM render is still slower than in 15.6.1, which could just be how it's supposed to be. These results do make much more sense.

Production build
ReactDOM 16 results:

render functional component tree x 299,810 ops/sec ±8.57% (49 runs sampled)
render class based component tree x 83,106 ops/sec ±8.96% (41 runs sampled)
render class that renders functional components x 88,360 ops/sec ±4.81% (54 runs sampled)
Fastest is render functional component tree

ReactDOM 15.6.1 results:

render functional component tree x 200,604 ops/sec ±5.43% (50 runs sampled)
render class based component tree x 400,852 ops/sec ±4.37% (34 runs sampled)
render class that renders functional components x 413,435 ops/sec ±1.98% (55 runs sampled)
Fastest is render class that renders functional components,render class based component tree

[gaearon]

Can you please publish on GitHub so these links don't expire or accidentally get deleted when we come back to check them?

[mobinni]

@gaearon Sorry, I had auto-upload on it was easier to share that link initially. It's in a repo now: https://github.com/mobinni/react-perf-dom

[mobinni]

@gaearon @aweary I ran the tests against a different machine with node 8 and I had react 16 beat out 15.6.1. I don't know what the issue was on my other machine. I'm sorry for wasting your time.

[gaearon]

Hmm. My understanding was that you ran these tests in the browser. Why would Node version matter?

[mobinni]

@gaearon Yeah the Node version really doesn't matter, I made it sound more important than it is. I reran the tests on my work computer again which runs Chrome 60.0.3112.101 and it still has the low render sample output. However my other Macbook Pro 2015 (i5/16gb) which runs Chrome 61 had no issues with render speed. I'm going to bump the version now to 61 and see what happens.

Update output:

render functional component tree x 329,995 ops/sec ±1.99% (53 runs sampled)
render class based component tree x 517,531 ops/sec ±2.82% (51 runs sampled)
render class that renders functional components x 530,799 ops/sec ±1.74% (56 runs sampled)
Fastest is render class that renders functional components,render class based component tree

Maybe important, my chrome had an update pending and had been running for a long time without being restarted, memory clog or some process wearing down Chrome could have also contributed to those buggy numbers.

[gaearon]

I’d like to advise you that microbenchmarks like this are not a good way to decide whether to use functional components or classes (if that’s what you’re trying to do). They’re pretty useless and don’t represent how real apps behave in practice.

That said they can catch some silly regressions so it would be good to know if we’re accidentally doing something slow by mistake.

[mobinni]

Thanks for the advice! I do know that these tests do not represent any 'real' rendering data, it was a naive test setup pretty quickly. I don't think it was a React problem at the end of the day, but a browser problem on my end. In any case, I'm sorry for wasting your time.

[gaearon]

It’s also worth checking if you ran some of the tests with DevTools open, or with any browser extensions active. They’ll influence the results.

Again, this is not representative of real performance, but I consistently get better results with 16:

render functional component tree x 115,932 ops/sec ±4.10% (55 runs sampled)
render class based component tree x 255,407 ops/sec ±4.46% (58 runs sampled)
render class that renders functional components x 252,045 ops/sec ±5.47% (56 runs sampled)
Fastest is render class based component tree,render class that renders functional components

16

render functional component tree x 204,931 ops/sec ±2.21% (59 runs sampled)
render class based component tree x 339,215 ops/sec ±2.98% (58 runs sampled)
render class that renders functional components x 326,880 ops/sec ±4.51% (56 runs sampled)
Fastest is render class based component tree,render class that renders functional components

[mobinni]

That makes sense, thanks for checking it out. Case closed I guess 😃