Next.js: Explicit API to pass data to rendered page from a custom server

[sarneeh]

Feature request

Is your feature request related to a problem? Please describe.

TL;DR: It is very confusing that there's no explicit API to pass data from the custom server to the rendered page.

I've found that there's an example for passing data explicitly to the rendered page via a custom server. However, IMHO the provided example is just a filthy hack. I was looking for a solution for that on the documentation of next.js, but, for sure, I would not think that using a query property, which as documentation states is a: query string section of URL parsed as an object. It works - OK. But I suppose it's just working by accident and not by purpose. If it would be on purpose I think there would be some documentation about it. And I certainly would not want to rely on solutions that work by accident on my production applications.

Describe the solution you'd like

• explicit API to pass data from a custom server to the rendered page which will be available via getInitialProps.
• documentation of Next programmatic instance (i.e. getRequestHandler, render, handleRequest functions)

Describe alternatives you've considered

IMO when it comes to server-side rendering on a custom server, passing data from the server to the rendered page (template) is the most natural thing to have. I was given a few examples but all of them just feel like a hack:

• doing it inversely: get data explicitly from the server while rendering server-side
• pass data via the query parameter (which I was describing above)
• make a request to call yourself (call the server which actually renders the page)

Additional context

Note that I'm only talking about the server-side render.

P.S. Still an awesome piece of software! 🎉 🎉 💃 😄

[timneutkens]

This API is intentionally not provided as it makes your rendering dependent on the custom server, the custom server should only implement routing / caching. Providing data is up to getInitialProps, in which you can get data from anywhere. It leads to unexpected behavior when using next/link (client page transitions). Tbh I feel like that example should be removed as it shows an anti-pattern.

[sarneeh]

@timneutkens Okay! Then I guess the documentation needs a few extra insights, as it's was not clear to me how should I use the custom server functionality. And while talking with people around I notice that it's not only me. Actually, many people use the custom server feature to create an API (data provider) for the Next app. But as the purpose of Next is getting more clear to me - I guess that the API (from which Next will fetch data from) should be developed completely separate. Am I right? 😄

[timneutkens]

I guess that the API (from which Next will fetch data from) should be developed completely separate.

If you care about performance, definitely. The larger problem at hand is that React SSR is synchronous, so if you also build an API on the same process it will lead to slowdowns.

Also, building the API separately allows you to scale your frontend / API separately. And introduces a nice separation of concerns, eg if your API crashes your frontend can still render an error page.

[timneutkens]

It's also the case that a page has a data dependency to render, using getInitialProps it's immediately clear where the data is coming from and in what way it's being fetched / read.

[sarneeh]

@timneutkens Thank you very much for the explanation. Makes total sense now 👍

[timneutkens]

:pray: Happy to explain!

[aequasi]

The only issue i've found with this, is how are you supposed to implement caching using something like redis?

You can't load redis in the browser, so currently, you are forced to pass stuff through the ctx objects

[sarneeh]

@aequasi But why would you want to load redis in the browser? Here you have an in-memory caching example for the SSR pages. And when it comes to the data you need to render them - cache it on the API server side.

[sergiodxa]

@aequasi you should not import Redis inside getInitialProps, that's where the custom server comes in, you import it there and cache the rendered HTML by Next and in the next request you send the same HTML to the user from the Redis cache.

Your React app is never aware is being cached, it only cares of fetching data from an API inside getInitialProps and rendering using such data as props.

[aequasi]

I don't want to, but I do want to use redis on the server side. I also don't necessarily (in my case) want to cache the entire HTML, just specific things.

So, I'm using the server to provide the data, and cache it there, which according to @timneutkens isn't the way you're supposed to do that.

[sergiodxa]

If you want to cache the data and not the HTML then a better place to do this is your API providing the data, that way multiple requests to the same API will resolver faster and will cause the getInitialProps method to complete faster and start rendering right away.

[sarneeh]

@timneutkens provided some very good reasons for not using your custom next server to provide data. And believe me - it's also a lot easier to develop and think about when you separate those concerns.

[aequasi]

Caching before hitting the API will cause the getInitialProps to resolve even faster, since there will be no network calls.

[sergiodxa]

Well you can do an in-memory cache and set it outside your component, that _may_ cause the cache to be reused between requests, but I have to try to be 100% sure it's going to work that way. You could use an LRU cache as in https://github.com/zeit/next.js/tree/canary/examples/ssr-caching.

[sergiodxa]

Note your client-side code will have its own cache in getInitialProps, so having a cache in the API will still beneficial to improve client-side navigation with next/link.

[andrewmclagan]

@sergiodxa I disagree. Really the aim is to not make a network request at all.

e.g.

Imagine a use-case where a page is rendered from API calls to a headless CMS. Fetching the menu items on each load request.

Caching these requests on the Next server with a TTL of 2-3 hours increases performance massively.

The alternate to add caching to the headless CMS API - is great and should be done also - but does not save useless network HTTP stack overheads.

[andrewmclagan]

Most apps that are performant have a "hydrate" endpoint that sends initial data that is basically static down the wire. Such as menu items, categories, enumerables... etc. Rarely changing static data.

useful for initiating web app (nextjs) and mobile app clients

[iMerica]

Where do I make database calls then?

Here?

  // pages/cool-page.js
  static async getInitialProps (props) {
    const data = getThing(props.query)
  }

or Here?

  // server.js
  server.get('/:slug/', async (req, res) => {
    const data = getThing(req.params.slug)
    return app.render(req, res, '/profile', {data: data})
  })

(Please don't say "use ajax". If I wanted to use ajax, I would build a SPA and render everything in the client side)

[revskill10]

@iMerica iMe

return app.render(req, res, '/profile', {data: data})

the {data:data} is query string which is not suitable to render large data. So the first option is where you make the call to get data.

[iMerica]

Excellent. Thanks for the information 👍.

[iMerica]

Update
I can't build my project when I do database fetching inside my page component's getInitialProps(). It introduces a whole range of build errors. The only thing that has worked for me thus far is doing the query inside the express JS handler and passing data to the component using the query parameter of the app.render() method (which should be named context, or payload IMO).

Question
Is no one actually doing DB queries in projects that use Next.js? For performance reasons (and clean architecture reasons), I want to query my database server side in the same request/response cycle and render everything on the server (a basic thing people have been doing since the 90s, not some kind of edge case).

[sarneeh]

@iMerica

For performance reasons (and clean architecture reasons)

Actually for performance and clean architecture reasons you should separate your backend from your frontend.

Read the thread from the start, especially this comment.

[iMerica]

Actually for performance and clean architecture reasons you should separate your backend from your frontend.

In an ordinary client side SPA this separation of concerns is valid. But Next.js' unique value proposition is server-side rendering (AKA render your frontend in your backend). We're in a server, let's do normal server-side things.

[sarneeh]

But Next.js' unique value proposition is server-side rendering.

Exactly.

(AKA render your frontend in your backend)

Nope. Server-side rendering means rendering client-side apps on the server.

Please, check out Tim's answer. Next.js probably was just not designed for your use case.

[iMerica]

Nope. Server-side rendering means rendering client-side apps on the server.

We are saying the exact same thing. "Frontend" is a generic term for the user interface, the view/presentation/template/client (choose your synonym). These terms are all interchangeable in the grand scheme of things.

Next.js probably was just not designed for your use case.

As I mentioned above, I have successfully setup Next.js to render my react components populated with data directly from my from database/cache (using the express handler strategy)

Everything is now working and I'm currently running performance tests (so far; so good). Once my project matures, I'll post a batteries included Next.js SSR-only fork with different trade-offs.

[sarneeh]

We are saying the exact same thing. "Frontend" is a generic term for the user interface, the view/presentation/template/client (choose your synonym). These terms are all interchangeable in the grand scheme of things.

You're right. What I mean is that Next.js focuses on the rendering part. Notice that Next.js does not encourage or mention anywhere, that you can build your whole backend + frontend app with it. It just renders a SPA on the server and gives a lot of improvements out of the box like code splitting etc.

As I mentioned above, I have successfully setup Next.js to render my react components populated with data directly from my from database/cache (using the express handler strategy)

Everything is now working and I'm currently running performance tests (so far; so good). Once my project matures, I'll post a batteries included Next.js SSR-only fork with different trade-offs.

As I already stated in my initial question - this works, but it's not intended to be used that way. The query object should hold request query data, not anything else. It was designed that way.

Also notice, that your solution won't work on page switches - after initial render, Next.js app is just a SPA, it doesn't invoke your server anymore.

[Timer]

@iMerica you're probably looking for getServerProps as defined in this RFC: https://github.com/zeit/next.js/issues/9524.