Redwood: Feature Request: Support Deployments to AWS Lambda

[Jaikant]

In continuation to the forum discussion. Would love to have direct deployments of redwood apps to Lambda and S3.

What about something like:

yarn rw deploy aws \ Deploy to lambda and S3
yarn rw deploy azure
yarn rw deploy gcp

Or we could do a generic yarn rw deploy which picks the provider config from the redwood.toml

Opinions?
On deploying to AWS lambda, we would get an api endpoint. This api endpoint would be needed within the web part in apollo client for the graphql queries.

To provide the deployed endpoint to the web part - we could save it into an environment/config variable which is read by the web part (the current implementation picks the config from redwood.toml and then uses the webpack define plugin).

The build step would need the api to be built first, before the web, so the endpoint is known.

Deploying the API AWS Lambda

I deployed the redwood api, to Lambda successfully. Here are my rough notes, to deploy the api side to AWS lambda. It is only a few packaging changes as the zip file generated by @netlify/zip-it-and-ship-it is not directly deployable to AWS Lambda.

Go to api/prisma and update schema.prisma with the binary target to be rhel-openssl-1.0.x.

generator client {
  provider      = "prisma-client-js"
  binaryTargets = ["native", "rhel-openssl-1.0.x"]
}

Install @netlify/zip-it-ship-it, serverless & serverless-dotenv-plugin

 yarn workspace api add @netlify/zip-it-and-ship-it --dev
 yarn workspace api add serverless --dev
 yarn workspace api add serverless-dotenv-plugin --dev

Build the API

 yarn rw build api

* Package it for lambda, using zip-it-ship-it*

mkdir zipball
yarn zip-it-and-ship-it api/dist/functions/ zipball/

Changes to zip file generated by zip-it-ship-it zip-it-ship-it doesn't package the binary target and schema.prisma file, could be related to this issue

// Unzip zipball/graphql.zip and move graphQL.js to function directory. So, the import paths for
// graphql.js corrected by

 mkdir dist/functions
 mv graphql.js dist/functions/

// zip-it-ship-it adds the root directory instead of node-modules, 
// lets extract the node-modules from it. copy the node-modules from `Users` into zipball
// On a macbook pro, it is as below. My app name is chat-support
  mv Users/jai/redwood/chat-support/node_modules .

// copy the binary target file into node modules, zip-it-ship-it skips the binary target 
// as well as the schema.prisma file
    cp ../node_modules/@prisma/client/runtime/query-engine-rhel-openssl-1.0.x ./node_modules/@prisma/client/runtime
  cp ../node_modules/@prisma/client/schema.prisma ./node_modules/@prisma/client/

// Lets clean up the directory, before zipping it
rm -rf Users/
// remove the older zip file, graphql.zip
rm graphql.zip

//zip the current directory, which has dist and node-modules
 zip -r graphql.zip .

Using serverless to deploy, the config serverless.yml, which is at the root of the project.

service: jai-redwood-1
plugins:
  - serverless-dotenv-plugin

provider:
  name: aws
  runtime: nodejs12.x
  region: ap-south-1

package:
  artifact: zipball/graphql.zip

functions:
  graphql:
    description: redwood on aws lambda
    memorySize: 1024
    timeout: 30
    handler: dist/functions/graphql.handler
    events:
      - http:
          path: graphql
          method: GET
          cors: true
      - http:
          path: graphql
          method: POST
          cors: true
      - http:
          path: playground
          method: ANY
          cors: true

resources:
  Resources:

Deploy to lambda by:

  yarn serverless deploy -v

To make netlify zip-it-and-ship-it work for redwood, we could write a wrapper over it or we could raise an issue with @netlify/zip-it-and-ship-it. We could also use something like serverless-webpack to zip it.

Having a similar deployment to Azure && Google Cloud, could help with an optimal design if any takers?

[benmccann]

I think something like yarn rw deploy with config file would be more configurable than yarn rw deploy aws. Just aws seems to decide too much when I imagine there are a number of different configurations users could want. E.g. you may want static files on Netlify, lambdas on AWS, and database migration on RDS. Or you may want static files on S3, lambdas on AWS, and database migration on Aurora Serverless. Just saying aws might be a bit too generic in this case. We could help users get started with default configs or cookbook documentation.

[mojombo]

The core team has talked about this and we'd like to prioritize getting a 2nd deploy target in place, and build it out using a plugin pattern that will allow people to contribute or write their own deploy strategies. We can ship with a handful of built-in strategies that have helpful defaults and allow partial or total override with a config file. We could use netlify or aws as strategy names and have the default be netlify in redwood.toml. It wouldn't need any config file. If you change that to a different strategy, then you could specify a config file in redwood.toml.

[Jaikant]

@benmccann you are right. We have the web, api and db pieces and each could choose a different provider. As @mojombo mentioned having a default configuration in redwood.toml and the ability to include other configurations within it would bring in the flexiblity you are talking about.
We could have someone configure a deployment with static files in netlify, lambdas in AWS and database on RDS and call it 'anyname'. A yarn rw deploy anyname would then deploy it to the specified targets.

[jeliasson]

I'm not sure which direction this will take (The Serverless Framework vs maintaining deployment providers in Redwood) but here is some Azure specific on the topic.

Azures equivalent to AWS's Lambda and S3 is Azure Functions and Azure Storage. If the deployment would utilize Git deployment to the fullest, would there be any need for a storage account/bucket?

References

• NodeJS specifications: azure/azure-functions/functions-reference-node
• Typescript definition for Function worker: Azure/azure-functions-nodejs-worker
• Azure Storage nodejs package: Azure/azure-storage-node

[hemildesai]

I tried this with the latest version of zip-it-and-ship-it for AWS Lambda and it seems to have worked out of the box. I did not have to make any changes to the zip folder and I used handler: graphql.handler in serverless.yml. Looks like the binaries and schema are now included in node_modules/.prisma/client/ in the bundled function.

[peterp]

@hemildesai That's amazing, would you mind sharing the steps that you took to make that work? It would be really awesome to have a deployment strategy for aws lambdas.

[hemildesai]

I just followed all the steps mentioned in the description except for

Changes to zip file generated by zip-it-ship-it :zip-it-ship-it doesn't package the binary target and schema.prisma file, could be related to this issue

And then changed the one line in serverless.yml from handler: dist/functions/graphql.handler to handler: graphql.handler since graphql.js now lives in the root of the zipball.

[peterp]

I've just chatted with @hemildesai and we've come up with some concrete steps for adding AWS deployments using the serverless framework.

Add a deployment generator (1st PR)

• [x] yarn rw generate deploy aws-serverless
• [x] Check if serverless is installed. They recommend that it's a global installation. If it isn't raise an error and ask the user to install it.
• [x] Install zip-it-and-ship-it in ./package.json and run yarn install
• [x] Generate a serverless.yml file that references the functions available in api/src/functions
• [x] Let the user know that they need credentials for AWS, provide them with additional information via the help.
• [x] Update the generator deployment docs to give information on how to run the command and how to get the AWS credentials. (Link to AWS's docs.)

Introduce a deploy command (2nd PR)

• [x] yarn rw deploy api aws-serverless
• [x] runs yarn build api, built files are in ./api/dist
• [x] runs zip it and ship it, placing the files in ./api/dist/
• [x] runs serverless deploy

[peterp]

Closed by #976 and #1012