Next.js: [RFC] Telemetry

[timneutkens]

Goals

• More visibility into Next.js feature usage
• Better prioritization
• Collecting more feedback from users

Background

Next.js was released almost 3 years ago, and the framework has grown considerably in these 3 years, from new features to better defaults for all users.

The way that we've been approaching this improvement process has been very much a manual one.

ZEIT has a few large Next.js applications. For example, zeit.co, zeit.co/docs, and nextjs.org. We've been dogfooding Next.js internally at ZEIT and improved Next.js based on our experiences.

On top of that we actively engage with the community to gather feedback. Chances are that you have talked to me before to provide feedback on how you're using Next.js at your company.

For example, if you use a custom server, if you have custom webpack configuration, and more. This feedback is extremely valuable in steering feature development for Next.js.

However, there is a problem with this approach, which is that we can only collect feedback from a subset of users. This subset may have different needs and use-cases than you.

For this reason, we're currently exploring a more automated approach to collecting these points of feedback so that we can improve Next.js even more in the near future.

One example of this would be importing CSS files, which is non-standard, but quite a chunk of users seem to be using this, either through next-css (or Sass/Less), or a custom configuration. If we know what percentage of users are using that specific approach we can prioritize improving it.

Furthermore, this will allow us to verify if improvements made to the framework are improving the baseline of all applications.

Proposal

We're planning to add anonymized telemetry to the Next.js CLI. We'll provide an optional opt-out through an environment variable or CLI command.

When enabled the CLI will collect telemetry at development and build time, when telemetry is collected we'll show a message explaining that telemetry is being collected with a link to documentation.

When telemetry is collected it'll be possible to add a flag to the CLI to see the exact values being sent. These values, as said before, will be completely anonymized.

Initially we're planning to have telemetry for:

• Next.js version
  
  
  
  • This metric will give us better insights into what version of Next.js users are using. This is important because it'll help us measure: (1) if users prefer a quicker or slower release cadence, (2) if we should start releasing Nightly builds instead of just Canaries, and (3) how often we should introduce changes, measured by how many users have migrated.
  
  
• Subcommand invocations duration
  
  
  
  • Usage of dev, build, and export
  
  
• Next.js plugin usage (@zeit/next-css, @zeit/next-preact, @next/mdx, et al)
  
  
  
  • We want to know what Next.js plugins are being used so that we can properly prioritize the maintenance order of these plugins. Furthermore, we can also use this to include commonly utilized plugins directly in Next.js' core (for example CSS imports).
  
  
• CSS-in-JS solution usage
  
  
  
  • Which solution is being used, being the built-in styled-jsx or styled-components, emotion, etc.
  
  

In the future, we may want to track the following:

• Custom configuration
  
  
  
  • This metric would help us understand what users are configuring, for example if you have a custom webpack or babel configuration and similar.
  
  
• Time to bootup development environment
  
  
  
  • This will help us understand how long it takes before users are able to access http://localhost:3000 after running next dev
  
  
• next CLI vs custom server usage
  
  
  
  • Next.js 9 introduced improvements that make a custom server not needed in many cases. This way we're able to see if more improvements can be made to make Next.js easier to use and faster, as most custom servers include express or another server framework
  
  
• Cold page compilation times
  
  
  
  • How long it takes to compile when there is no cached version of the page available
  
  
• Incremental page compilation times
  
  
  
  • How long it takes to compile when a page is cached and built before (making edits)
  
  
• Size of build artifacts (bundles, initial chunks)
  
  
  
  • This allows us to verify improvements in chunking logic and bundle size optimizations
  
  

Once the initial implementation has landed we can investigate exposing the results to the Next.js community through public dashboards.

There's quite a few open source projects that take a similar approach to this RFC, most notably:

• Homebrew
• VSCode
• Gatsby

This RFC was authored by @timer and me.

[kachkaev]

Are you planning to share collected stats with the community or will the data be available only to Zeit employees?

[timneutkens]

It's mentioned in the spec:

Once the initial implementation has landed we can investigate exposing the results to the Next.js community through public dashboards.

[erikras]

How about including errors when they originate inside the Next stack? Or anonymizing the non-Next stack? Things like too many ReferenceError: req is not defined in getInitialProps() might indicate that people don't understand that req is only available during SSR, for example. 🤔

[timneutkens]

@erikras eventually we might add that in, however it's slightly tricky to determine coming from the next stack vs not, as many errors also happen client-side for example, but we can definitely investigate this 👍

[ForsakenHarmony]

an optional opt-out

please make it opt-in and or ask on first use

[timneutkens]

please make it opt-in and or ask on first use

Making it opt-in would mean that you get into the same situation that I outlined in the "background" section, where a very small percentage of users outline the usage, which means that prioritization would be partially flawed. But as outlined in the RFC we'll show a message with link to the docs on CLI bootup so that you can decide still.

[ForsakenHarmony]

The message will be shown every time though?

[stormy251]

I am all for this approach, I think that collecting data will better situate you, and your team to better account for the global audience and not just the largest or most vocal! As always, keep up the great work!