Next.js: Investigate performance difference with Nuxt.js on synthetic benchmark

Created on 28 Oct 2017 · 9Comments · Source: vercel/next.js

Follow up from this conversation:
https://twitter.com/nuxt_js/status/924309056247992322

This was comparing bench folder in both packages: https://twitter.com/Atinux/status/924323037322891264.

Allegedly the difference in performance is ~10x on this synthetic benchmark.

However we determined that vanilla React vs Vue only differ by about ~1.2x on the same benchmark:
https://twitter.com/dan_abramov/status/924338662212501504

I would very much appreciate if somebody could dig into what’s causing such a significant difference between Next.js and Nuxt.js since apparently it’s not due to React and Vue differences.

Thanks!

Source

gaearon

👍1

Most helpful comment

Okay, here's what's happening.

This 10x slowness simply because of our benchmark app doesn't expose NODE_ENV properly. Here's the fix we did.

"So, this does not affect any production Next.js at all. Next.js is fast as it should be."

I did some benchmarks on my machine using ab.
(different benchmark apps may give different results)

React vs Vue

React: 60 req/sec
Vue: 115 req/sec (1.9x faster)

Next.js vs Nuxt.js

Next.js: 40 req/sec
Nuxt.js: 67 req/sec (1.7 faster)

The ab results are here: https://imgur.com/a/tTQWS

So clearly, Nuxt.js is not 10x faster. But thank you @Atinux for initiating this. It helps us to fix our benchmark app.
Of course thanks for @gaearon for helping us out with the plain React vs Vue benchmark app.

In addition to the fix I mentioned, we also made this change to make sure something like this won't happen again.

arunoda on 29 Oct 2017

👍20 🎉1

All 9 comments

One observation right off the bat is that your version of the stateless-big benchmark uses renderToNodeStream:

app.get("/", (req, res) => {
  res.write("<!DOCTYPE html><html><body>");
  res.write("<div id='content'>");
  const stream = renderToNodeStream(React.createElement(App));
  stream.pipe(res, { end: false });
  stream.on('end', () => {
    res.write("</div></body></html>");
    res.end();
  });
});

whereas currently we use renderToStaticMarkup. However, I don't think that can be a sufficient explanation, because the faster Vue benchmark submitted by @Atinux uses a similar API as ours:

app.get('/', (req, res) => {
  renderer.renderToString(new Vue(App))
  .then((html) => {
    res.write('<!DOCTYPE html><html><body>')
    res.write('<div id="content">')
    res.write(html)
    res.write('</div></body></html>')
    res.end()
  })
  .catch((err) => {
    res.send(err)
  })
});

Note: adding streaming support to Next.js is something that's _already planned_. That said, I'd love to fully understand the bottlenecks here first.

rauchg on 28 Oct 2017

Actually I tried on @gaearon react/ version to use renderToStaticMarkup and also renderToString and I got the same results (around 34 req/sec).

I don't think the streams are really important (Nuxt.js does not use streams for example, see https://ssr.vuejs.org/en/streaming.html, at the bottom).

I believe the performance of Nuxt is due to the runInNewContext = false option we set, that avoid Vue to create a new virtual machine for each request and loose a lot of time. I don't know on React side if they use it and if there is a possibility to disable it?

Atinux on 28 Oct 2017

👍2

Okay, here's what's happening.

This 10x slowness simply because of our benchmark app doesn't expose NODE_ENV properly. Here's the fix we did.

"So, this does not affect any production Next.js at all. Next.js is fast as it should be."

I did some benchmarks on my machine using ab.
(different benchmark apps may give different results)

React vs Vue

React: 60 req/sec
Vue: 115 req/sec (1.9x faster)

Next.js vs Nuxt.js

Next.js: 40 req/sec
Nuxt.js: 67 req/sec (1.7 faster)

The ab results are here: https://imgur.com/a/tTQWS

In addition to the fix I mentioned, we also made this change to make sure something like this won't happen again.

arunoda on 29 Oct 2017

👍20 🎉1

Awesome. Thanks for getting to the bottom of this!

Any ideas why I'm not seeing as big a difference on Vue vs React case with rest-bench? Could machine make a difference? Which Node were you using?

gaearon on 29 Oct 2017

Great leg work.

reneruiz on 29 Oct 2017

@gaearon It could be the machine. And may be with the work load an so on.
I don't think any of these are scientific tests though.

Basically the point is, it's not 10X faster or slower.

arunoda on 29 Oct 2017

I think we can close this now.

arunoda on 29 Oct 2017

It did surprise me that the original tweet also pointed out a comparison with the screenshot in our blog post. If you read our blog post carefully, we used ab to compare one Next.js version another:

We didn't publish what hardware was used, which means the original Nuxt.js tweet is also not very meaningful as a comparison.

Personally, I use (and encourage everyone at ▲ZEIT as well) pretty underpowered hardware. It's something important to keep in mind when comparing your notes with someone else's.

rauchg on 29 Oct 2017

👍11

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.