Serverless-next.js: Feature: Improved page and static resources Routing

[danielcondemarin]

Motivation

Currently there are no routing capabilities for static assets. For example, you can't have /robots.txt, /browserconfig.xml or maybe a service worker /sw.js.
Also, although the custom routing for pages works and is flexible it can be quite verbose to declare:

custom:
  serverless-nextjs:
    nextConfigDir: ./
    pageConfig:
      post:
        events:
          - http:
              path: post/{slug}
              request:
                parameters:
                  paths:
                    slug: true 

That is just for a custom post page route: post/{slug}.

Proposal

1 - Upload static/ directory contents to S3 assets bucket

The concept of a static folder is familiar already to next developers. Is an easy way to get static content available to the application. The plugin already manages an S3 bucket where the next build assets are uploaded, so the static/ dir contents will also be uploaded to the same bucket.

2 - Add support for a new routes plugin configuration array:

# serverless.yml
custom:
  serverless-nextjs:
    nextConfigDir: ./
    routes:
       -
          # post page routing
          # https://123.execute-api.us-east-1.amazonaws.com/dev/post/123
          src: pages/post.js
          path: post/{slug}
          request:
            parameters:
              paths:
                slug: true
       -
          # https://123.execute-api.us-east-1.amazonaws.com/dev/robots.txt
          src: static/robots.txt
          path: /robots.txt
       -
          # https://123.execute-api.us-east-1.amazonaws.com/dev/browserconfig.xml
          src: static/browserconfig.xml
          path: /browserconfig.xml

3 - Make api gateway act as a proxy to the S3 bucket for any static route requests such as /robots.txt, /browserconfig.xml etc.

Do note that you can only setup static routes for files inside the static/ directory.

Related: https://github.com/danielcondemarin/serverless-nextjs-plugin/issues/29, https://github.com/danielcondemarin/serverless-nextjs-plugin/issues/14

[ulysset]

That looks very practical, can't wait to try it out!

So by default, it will upload/sync everything that is in /static dir to the Bucket? Like if I put an image (e.g. logo.png) in the /static dir it will be automatically available at this uri : https://123.execute-api.us-east-1.amazonaws.com/dev/logo.png ?
Or do I need to specify the src and path for every single asset from the /static dir?

[danielcondemarin]

So by default, it will upload/sync everything that is in /static dir to the Bucket?

Yes.

Like if I put an image (e.g. logo.png) in the /static dir it will be automatically available at this uri : https://123.execute-api.us-east-1.amazonaws.com/dev/logo.png

No. However, you would be able to access the image via the bucket url:

const config = {
  target: "serverless",
  assetPrefix: "https://s3.amazonaws.com/mybucket"
};

https://s3.amazonaws.com/mybucket/static/logo.png

The static routes are for resources that you need hosted in the main URL, such as /robots.txt, etc.

Having said that, I could setup API GW to also proxy any requests to /static to the bucket, e.g.

https://123.execute-api.us-east-1.amazonaws.com/dev/static/logo.png
-> proxies to ->
https://s3.amazonaws.com/mybucket/static/logo.png

However, I'm not sure that this encourages best practice. If you can already access the resource via S3, why go through the API GW which incurs in extra cost etc.?

[ulysset]

Yes you are right it's better like you said, I was mixing in my head the bucket URL and the API GW URL...
Can't wait to try it out 😉

[barrysteyn]

This is great @danielcondemarin and is exactly what I need.

[danielcondemarin]

FYI - just merged static directory upload support in https://github.com/danielcondemarin/serverless-nextjs-plugin/pull/54. Next will be looking at the routing (step 2 from the list above).

[aheimlich]

FYI, zeit/next.js#7213 landed recently, which impacts Step 2 and Step 3

[Vadorequest]

Indeed, /public seems like the way to go, especially now that it's officially supported by Next.js
It may even make step 2-3 unneeded? @danielcondemarin FYI

Haven't tried it out, but looks like it should handle gracefully/automagically everything in /public. Not sure if it will be compatible with serverless though, but it should if it's handled by Next.js itself.

With a bit of luck, the only thing left could be the automated upload of static files.

[danielcondemarin]

@aheimlich @Vadorequest AFAIK https://github.com/zeit/next.js/pull/7213 only works for next target: server. For serverless deployments, hosting and routing of any assets have to be done outside of next. In the case of this plugin S3. I could in the future adopt this convention of a /public folder, but I don't think it adds any value outside the proposal above.

[Vadorequest]

Ah, indeed. I missed that, that wasn't explicit. Too bad, but you're right, your proposal fits the need.

I wonder if the s3 bucket is created by default or must be explicitly listed, that's not mentioned in 1)
I also wonder if a default routing shouldn't be considered, basically any file in /static could be automatically served in 2), what do you think? Seems odd to add static files that shouldn't be, and I'd assume all files in static folder are meant to be served, so why not make that a default?

[danielcondemarin]

I wonder if the s3 bucket is created by default or must be explicitly listed, that's not mentioned in 1)

The bucket is not created by default. It can be provided either via next.config assetPrefix field or the plugin config value assetsBucketName as documented here.

I also wonder if a default routing shouldn't be considered, basically any file in /static could be automatically served in 2), what do you think?

I think that will encourage using the api gateway to route static assets that could be requested directly from the S3 bucket url. You would be paying extra for the API GW data transfer for little value. However, in cases where you do need the static resource routed in the main app domain (such as /robots.txt, maybe a service worker etc.) then serving the static asset through the api gw is necessary.

[Vadorequest]

I'm not sure that's a bad thing though. It makes getting started and test things easier (better DX), as long as the feature behaviour is properly documented (warning about cost, how/why it should be used) then I don't see the point for not automating it. In the end, it's the developer's choice.

What matters is to educate the choice, but I don't see how making it harder to implement something so trivial would really be of any benefit for developers. I'm all for automation, simplicity, and proper documentation/guidelines/advices on how to do things, rather than making things harder in order to avoid misusage.

[danielcondemarin]

I'm not sure that's a bad thing though. It makes getting started and test things easier (better DX), as long as the feature behaviour is properly documented (warning about cost, how/why it should be used) then I don't see the point for not automating it. In the end, it's the developer's choice.

What matters is to educate the choice, but I don't see how making it harder to implement something so trivial would really be of any benefit for developers. I'm all for automation, simplicity, and proper documentation/guidelines/advices on how to do things, rather than making things harder in order to avoid misusage.

I'm still not convinced api gateway is designed for this. Sounds more like a job for S3 (better yet with CloudFront). Having said that api gateway has binary support and behind the scenes comes with its own cloud front distribution so it might be fine to do this. In any case, the work in this PR is more focused in simply supporting root hosted resources such as /robots.txt etc. and if anything does most of the work to possibly support something like default /static routing.

[danielcondemarin]

FYI - https://github.com/zeit/next.js/pull/7220 adds native support for dynamic routing client side which is perfect for serverless usage. I'll create a separate feature request for this 🎉

[danielcondemarin]

Pleased to say this is now released in https://github.com/danielcondemarin/serverless-nextjs-plugin/releases/tag/v1.8.0 🎈 🎉 . I've also updated the examples/ using the new API. Please try it out and feedback if something doesn't work, thanks 🙏