Serverless-next.js: Handle future getStaticProps / getStaticPaths / getServerProps features of Next JS

[Ziltosh]

Hello,

I would like to use the future combo: getStaticProps / getStaticPaths / getServerProps for SSG/SSR generation (see here: https://github.com/zeit/next.js/issues/9524 )

Currently, the functions don't seem to work properly on serverless-next.

On Zeit Now, it's ok, all the pages are generated at build time, everything is saved in .next/serverless/pages when run locally with next build and on _next/data and _next/static folder when online.

With this plugin, the deployment is good (slow) but I don't see any pages in the _next folder of the S3 container.

If I visit the site, the getStaticProps method seems to be called each time the page loads.

It makes a GET request to the cache file but can't find it.

It would be really nice to have that added. Zeit has limitations that AWS does not have.

[nodabladam]

I too just built a static marketing site and was sad to find getStaticProps / getStaticPaths not loading quickly. Is there any workaround...do we need to be manually pushing some missing files to s3?

[romainquellec]

Same here.

[MasterEric]

As of 2 days ago, static site generation is now part of the official Next.js release.

Changes to support these should be implemented to keep full compatibility with newly released projects.

[oseibonsu]

@danielcondemarin if you can outline how you would like this implemented, my company can work on a pull request for this.

[danielcondemarin]

I have been looking into what's involved to support the SSG Improvements. Here are my findings so far:

1. getStaticProps

Supporting this should be easy, we just need to upload the JSON file with the static props generated by next at build time to S3, under a new data folder, e.g. _next/data/{BUILD_ID}/todos/new/new.json .

Then this file is requested by next client side and it should be available to be fetch-ed from S3.

1. getStaticPaths

This is a tricky one because of the Fallback option. Generating on demand static pages will need to be done at Lambda@Edge, whenever a requested dynamic route is not pre-rendered, a new page should be generated and saved onto S3 (as described in https://github.com/zeit/next.js/issues/9524). For this we're going to call renderReqToHTML on the serverless page and save the result onto S3. Similar to what next-server does here

1. getServerSideProps

The work here needed is to ensure client side transitions still work. From the RFC:

Next.js will also automatically expose an API endpoint that returns the result of calling getServerSideProps. This is being used for client-side routing.

---

@danielcondemarin if you can outline how you would like this implemented, my company can work on a pull request for this.

Thanks @oseibonsu for the offer. I'll look into it further to see _exactly_ what needs doing and will let you know 👍

[SarKurd]

I would like Serveless-next.js to move away from Serverless components before supporting any Nextjs features, Serverless components are most likely abandoned, there are PRs everywhere nobody reviews.

[oseibonsu]

@danielcondemarin I'm doing some testing. It looks like the '.next/prerender-manifest.json' has the prerendered path is there. Right now I'm just adding the list to the sortedPagesManifest. That works for those paths explicitly listed in getStaticPaths. After the pages are rendered using renderReqToHTML, would the manifest.json in the lambda need to be rewritten each time? Or do you want to go down a different path?

[danielcondemarin]

I would like Serveless-next.js to move away from Serverless components before supporting any Nextjs features, Serverless components are most likely abandoned, there are PRs everywhere nobody reviews.

The work done to support the SSG improvements is not specific to serverless components, and I intend to make it as reusable as possible so it can be used for the CDK project I'm working on https://github.com/danielcondemarin/serverless-next.js/tree/aws-cdk-support.

More generally, the direction I intend to take with the project is to have a _reusable_ set of packages in the monorepo that can be used for serverless components, aws cdk and any other platforms we'd like to use for hosting serverless next.js

[oseibonsu]

@danielcondemarin This works for basic support of static paths. Once you decide on the caching strategy for pages that were not prerendered I can work something up for that. https://github.com/oseibonsu/serverless-next.js/commit/6fcfa60e4defe98ac8b059e7bcb87fc73cbd1e70

[danielcondemarin]

@danielcondemarin I'm doing some testing. It looks like the '.next/prerender-manifest.json' has the prerendered path is there. Right now I'm just adding the list to the sortedPagesManifest. That works for those paths explicitly listed in getStaticPaths. After the pages are rendered using renderReqToHTML, would the manifest.json in the lambda need to be rewritten each time? Or do you want to go down a different path?

Great stuff, a few comments:

• Rather than checking for _next/data at lambda@edge I think we could add another CloudFront cache behaviour for _next/data that forwards traffic to S3 and completely bypasses lambda@edge. It will save cost and complexity.

• Rewriting the manifest at the edge won't work because if a new Lambda is spun up it won't contain those changes. I need to think more about this but ideally we don't need to keep the manifest in sync for this to work, as in, just make sure those pages are rendered and uploaded to S3 whenever there is a request and if is the first request for that route, serve the fallback.

[oseibonsu]

@danielcondemarin Makes sense. I need to learn more about how to set up the CloudFront config with serverless components.

[janus-reith]

@danielcondemarin @oseibonsu There also is a new but not yet documented property revalidate in getStaticProps which if you specify an amount of seconds uses stale-while-revalidate pattern to rebuild pages with fresh content.
I'm not sure how far this affects your considerations regarding caching, but it would be great if this could be incorporated as for me it solved a long outstanding question on how to handle things best.

[janus-reith]

I think there could be multiple ways to approach this, this seems the easiest:

Make sure each function creates a static page in S3 and serve that over Cloudfront. Then for the fallback behaviour, use a function as a Second origin, which creates the static page and makes sure it is stored in S3 and returns the fallback page(there is a fallback property in the useRouter hook to detect if currently in fallback mode)

Initially I though we could simply skip S3, as Cloudfront could also just cache the result from the Lambda, which would also make using revalidate easier(Cloudfront would just call the function again after the expiry time of the s-maxage header nextjs sends). But the catch here is, we dont want to await creating the page, instead return the fallback page before rendering the actual page might be finished. (Altough that could maybe also be an additional option for some who actually want that "blocking" behaviour).

For me this raises another question: Should the function return the fallback page, or could we even save some ms of response time by having a generic static fallback page in S3 for each page that supports fallback mode, and have the function call for the page creation with the specific slug happening client side? So something like /someroute/dynamicvalue would always return /someroute/[thestaticfallback] if requested from S3. Result would be the same, the function creates the page in S3, so subsequent request directly get the page from S3. Might be Interesting to see how nextjs is doing this internally when running next start.

Another concern would be revalidate. As described above, I think there is the need to also store the fallback created pages on S3, as else it would be hard to distinguish when a page already exists, and when it is requested the first time and needs the fallback loading view.
We wouldn't want to show a fallback loading page again in case a page is past revalidate time, but return the existing page, and somehow trigger recreation in the background. An easy way could be to also have this happening on the client, but I think there could be a cleaner solution. I wonder if Cloudfront allows for some kind of side effects, to return the page it has, and on the same time detect that that it is past the s-maxage time and trigger the page recreation.

Also, I'm still wondering if there might be an easier approach where my assumptions regarding the fallback created pages needing to be present in S3 are wrong and we therefore could have easier revalidate support, and still return the isFallback loading page the first time the page is requested. While on a fallback loading page, we would always await the page being present on the server, so this could also be used as the trigger of the function for the actual creation, which is passed though cloudfront which then has the page in cache.

[Timer]

Preview Mode also needs to be supported:
https://nextjs.org/docs/advanced-features/preview-mode

[danielcondemarin]

Hi folks, sorry for the slow response on this issue. I've picked this up over other things and now opened a more formal RFC where we can discuss the implementation https://github.com/danielcondemarin/serverless-next.js/issues/355.

[dphang]

I think most features have been implemented except the revalidate. We will track in another issue.