Middy: Version 2

[willfarrell]

See /docs/CHAGNELOG.md & /docs/UPGRADE.md on release/2.x for what has changed.

TODO / Ideas

• init commit: https://github.com/middyjs/middy/pull/587
• [x] security and performance best practice by default
• [x] Node v14, maybe v12, drop v10 (will use conditional chaining)
• [x] async/await core/middlewares (This will break existing middlewares) #356
• [x] AWS SDK v3 #584
  
  
  
  • [x] Update paginated requests to use generators
  
  
• [x] Unify middleware options
  
  
  
  • [x] context manipulation
  
  
  • [x] FIPS 140-2 support - https://aws.amazon.com/compliance/fips/
  
  
  • [x] role assume (sts)
  
  
  • [x] caching
  
  
  • [x] enforse TLS connections https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/enforcing-tls.html
  
  
• [x] allow ssm to pull > 10 keys (if possible)
• [x] update avj to support draft-2019-09
• [x] Review and Update error handling
• [x] optimize dependency inclusion

If anyone has comments, suggestions, or must-have features. Please post them here.

Documentation

• [x] update options for each middleware
• [x] detailed upgrade guide with why
• [x] detailed changelog
• [x] typescript section
• [x] example showing chaining between middleware that use internal
• [x] Examples showing middleware ordering (apigw, s3, sqs, sns, edge)
• [x] Profiler documentation (includes clinicjs)
• [x] bundling details - provide .yarnclean for middy and deps w/ ref to https://github.com/93v/starve
• [x] serverless config recommendations
• [x] lambda env for activating experimental flags - https://nodejs.org/api/cli.html#cli_node_options_options
• [x] update logo to svg
  
  
  
  • [x] AppoloClient common pattern https://github.com/middyjs/middy/issues/527
  
  
  • [x] final proof read
  
  
  • [x] update deprecation notice on older versions
  
  
  • [x] updated code of conduct to v2
  
  

Missing / Blocked

• [x] Node.js v14 support on AWS Lambda Expected 2021-04-30
  
  
  
  • will transpile down to v12, not a blocker
  
  
• [x] TypeScript Definitions
• [x] Profile running code & micro optimmize
  
  
  
  • [x] validator: 60ms setup time (draft-2019), 50ms (draft-07) ajv-validator/ajv#1098
  
  
• [x] publish with cjs & mjs https://github.com/gfmio/typescript-esm-cjs-hybrid-example
• [x] ability to mock out middy with nock https://github.com/matt-sanders/middy-nock-error
• [x] License check
• [x] update website? no
• [x] code review (core,util, ssm) by @lmammino
• [x] util stay in core or move to own package?
  
  
  
  • [x] update code and docs to reflect decision
  
  
  • [x] resolve chicken & egg problem with util in middleware
  
  
• [x] merge to main branch

Testing

• [x] 100% test coverage
• [x] FIPS 140-2 endpoints @willfarrell
• [x] RDS @willfarrell
• [x] Secrets Manager #151 #228
• [x] SQS #471
• [x] SSM @willfarrell
• [x] STS #570
• [x] webpack
• [x] rollup

Dependencies

• [x] aws-sdk/clients/rds missing promise() pattern https://github.com/aws/aws-sdk-js/issues/3595
• [x] ajv
  
  
  
  • [x] ajv: optimize initialization https://github.com/ajv-validator/ajv/issues/1098
  
  
  • [x] ajv-errors with i18n support https://github.com/ajv-validator/ajv-errors/issues/81
  
  
  • [x] json-schema-traverse: https://github.com/epoberezkin/json-schema-traverse/pull/34 https://github.com/epoberezkin/json-schema-traverse/pull/8
  
  
• [x] busyboy
  
  
  
  • [x] dicer: https://github.com/mscdex/dicer/pull/21
  
  
• [x] http-errors
  
  
  
  • [x] setprototypeof: https://github.com/wesleytodd/setprototypeof/pull/17
  
  
• [x] json-mask: https://github.com/nemtsov/json-mask/pull/125
• [x] qs: https://github.com/ljharb/qs/pull/389

Observations

• core npm size 38.4kb -> 12.8kb
• cold start times decrease by up to 20%

[baileytincher]

Can we rename the generics in the core declarations file? https://github.com/willfarrell/middy/blob/c5464931349a81bad412c52588a1245acb88b39e/packages/core/index.d.ts#L28

Instead of T, R, C we can go to something more descriptive like TEvent, TContext, TCallback?

Would this break backwards compatibility with anything? (I don't think it would)

[willfarrell]

v2 is going to be a breaking change. I think this is a great opportunity to make the names more descriptive.

[willfarrell]

After some reflection and discussion with others, I've decided to roll back to using aws-sdk v2 over v3. v3 just isn't as mature as it should be as flagged by the issues above. I'm confident AWS will address these open issues, but there is no guarantee they will in the near term.

[willfarrell]

The profiler has been removed as a requirement for v2 alpha. Opening up for discussion on the best approach https://github.com/middyjs/middy/issues/599

[prisis]

Hey :), do you know when the first v2 alpha will be released?

[willfarrell]

Thanks for the question @prisis. It's mostly there right now. @lmammino and I are have a discussion scheduled to resolve some of the outstanding issues, and potentially breaking changes to the current branch. I may release the first alpha without TS support (the biggest missing piece), just to get it out there and start IRL testing. I'll have a better idea next week, but could be as early as the end of the month. I will post here when a published alpha is ready for other the download and test.

[willfarrell]

After some reflection and discussion with @lmammino. ESM support will be remove from this version. This will decrease the barrier to upgrade and reduce the complexity of packaging. We will consider adding this back in based on community feedback and broader adoption around esm.

[prisis]

Did you try developit/microbundle for bundling all the packages?

[willfarrell]

I'm not familiar with this one specifically. But am with others. This approach makes more sense when you expect the package to be included in the browser and you want every flavour or packaging to fix your needs. On nodejs, providing multiple can over complicate the documentation and over all developer experience.

By only supporting commonjs both const middy = require('@middy/core') and import middy from '@middy/core' work based on the context they're imported into. No file paths necessary.

This also keeps maintenance cost low.

[ghostd]

Hi,

I'm just discovering this project (which seems awesome). Do you plan to write some documentation for beginners? Something like 'how to run your lambda on your local machine?' or 'how to write a simple test with middy?'

Cheers

[willfarrell]

@ghostd We're always looking to improve out documentation for those just getting started. However we can't document everything. Keeping the focused on Middy ensures we're not wasting valuable resources where others are already covering well.

how to run your lambda on your local machine?: Checkout serverless-offline
how to write a simple test with middy?: Checkout any of the test files in this project. v1 uses jest, v2 uses ava + sinon

[prisis]

@ghostd 
how to run your lambda on your local machine?: install npmjs.com/package/express, https://www.npmjs.com/package/cors and npmjs.com/package/lambda-local

create a lambda-server.js file and then run it with node ./lambda-server.js

const path = require("path")
const express = require("express")
const lambdaLocal = require("lambda-local")
const cors = require("cors")

const app = express()

app.use(express.text())
app.use(cors())

app.use("/{your url}", async (req, res) => {
    const result = await lambdaLocal.execute({
        lambdaPath: path.join(__dirname, "dist", "index.js"),
        lambdaHandler: "handler",
        envfile: path.join(__dirname, ".env"),
        event: {
            headers: req.headers, // Pass on request headers
            body: req.body, // Pass on request body
            queryStringParameters: req.query,
        },
        timeoutMs: 5000,
    })

    // Respond to HTTP request
    res.status(result.statusCode).set(result.headers).end(result.body)
})

app.listen(3000, () => console.log("listening on port: 3000"))

[ghostd]

@prisis Nice snippet, thanks!

[willfarrell]

Alpha has been released. Now on npm.

[nfantone]

@willfarrell Good stuff! Anywhere we could gather some release notes on what was added/changed apart from the checklist on this issue?

[willfarrell]

You bet. See /docs/CHAGNELOG.md for high level & /docs/UPGRADE.md for the granular details on release/2.x for what has changed.

[willfarrell]

TypeScript definitions will be merged soon. Expect a beta release in the coming days!

[HassanMahmud]

Would you consider V2 production ready?

[willfarrell]

Yes, myself and others been using it in production since the first alpha release in January. We're planning when the stable release will happen soon, we hope to lock down a date next week.