Middy: Postgres connection pool example

[DavidWells]

Here is an example of middy middleware connecting to postgres by @dschep

const pg = require('pg-promise')();
const {getParameter} = require('./ssmParameterStore');

const getDb = () => getParameter('db_password')
  .then((password) => pg({
    host: process.env.DB_HOST,
    port: Number(process.env.DB_PORT),
    database: process.env.DB_DATABASE,
    user: process.env.DB_USER,
    password,
  }));

const dbMiddleware = () => ({
  before: (handler) => getDb().then((db) => {
    handler.context.db = db; 
  }),
  after: (handler) => {
    handler.context.db.$pool.end();
  },
  onError: (handler) => {
    handler.context.db.$pool.end();
    throw handler.error;
  },
})


module.exports = {getDb, dbMiddleware};

We should start collecting examples in the wild and housing them somewhere =)

props to @alexdebrie for the find

[DavidWells]

Is everyone open to doing something like this https://github.com/serverless/plugins#community-contributed-plugins for middleware we find?

The markdown table is automatically generated via npm run docs via the data in https://github.com/serverless/plugins/blob/master/plugins.json

I can set this up and do a PR

[lmammino]

That would be awesome @DavidWells :)

[NPCRUS]

What is the purpose of ending the connection after handler, since lambda freeze the closure, so you can reuse existing connection.

I founded a lot of solutions, where connection is setup outside of handler and handler reuses it on every lambda call. And in middleware you can just check existing connection - is it timed out or not?

Best regards

[dschep]

@NPCRUS that's correct. However the lambda was timing out. This could probably be rewritten to take advantage of the doNotWaitForEmptyEventLoop middleware to avoid having to close the pool. I just haven't gotten around to it as I'm not using this in a performance sensitive environment.

Also, @DavidWells, you probably wanna refactor it to pull the creds from process.env or something so there isn't a dependency on my (not included) SSM parameter store wrapper :wink:

Edit: and I love the idea of the examples "library" I'll probably add the same to lambda_decorators

[DavidWells]

@dschep do you want to publish the code/repo for this? I'm missing './ssmParameterStore' file =)

Need something to kick off the list.

Happy to help you setup the same on https://github.com/dschep/lambda-decorators

I'm using a package I wrote to take the pain out of managing docs https://github.com/davidwells/markdown-magic

[NPCRUS]

Does anyone interesting in such middleware but with knex usage. Working on it right now for my own purposes. We can add it to prebuild middlewares as well if you want guys.

[dschep]

Haha, sure @DavidWells, it just requires that you've set a $STAGE and $SERVICE_NAME env var.

const AWS = require('aws-sdk');

const parameterCache = new Map();

/**
 * Get secrets from process.env and decrypt them with KMS
 * @param {String} parameterName - the name of the environment variable to fetch & decrypt
 * @param {Boolean} decrypt - automatically decrypt secure strings
 * @returns {Promise} a promise resolving to the decrypted environment variable
 */
function getParameter(parameterName, decrypt = true) {
  const ssm = new AWS.SSM(
    {region: process.env.AWS_REGION && process.env.AWS_REGION !== 'undefined' ? process.env.AWS_REGION : 'us-east-1'});

  if (parameterCache.has(parameterName)) {
    return Promise.resolve(parameterCache.get(parameterName));
  }
  return ssm.getParameter({
    Name: `/${process.env.SERVICE_NAME}/${process.env.STAGE}/${parameterName}`,
    WithDecryption: decrypt,
  }).promise()
    .then(({Parameter: {Value}}) => {
      parameterCache.set(parameterName, Value);
      return Value;
    });
}

module.exports = {getParameter};

Re: setting it up on lambda_decorators, I'll probably do that using the existing sphinx docs process I've got (tho that's currently 100% auto-doc based from the lambda_decorators.py source itself).

[dschep]

Tho, random thought: if you're making a library of middleware examples, the above getParameter stuff could be another middleware that adds ssm params to the context object.

[lmammino]

@DavidWells, we could create a repo under the middyjs org for this purpose. What do you all think?

[dschep]

Rewrote the SSM stuff as a middleware this evening:
https://github.com/dschep/JUMPStart/blob/bdf99edaabd78599edf0897b7b4d6d28fe15f7c2/handler.js#L9-L21

[dschep]

Btw, The original PG middleware has a bug we figured out today (we wen't actually returning any values yet, just doing UPDATEs). The before&after need to accept & call next.

[willfarrell]

FYI there is a fairly new middleware, db-manager, that might be worth checking out. It uses knex, there is a PR to add in support to rdsSigners https://github.com/middyjs/middy/pull/448