Serverless-express: Using mock request/response instead of proxy

[dougmoscrop]

Possible outcomes:

• merging this project with https://github.com/dougmoscrop/serverless-http
• replacing the use of proxy in this project with mocked or stubbed request/response objects
• stay the course -- having two approaches is valuable!

Considerations:

• maintainability
• ❌performance -> not worth it, performance difference is not significant enough

Maintainability

This can get pretty subjective. Not going to fill this in at this time.

Performance

This seems reasonably objective! Here's some testing I did:

• using loadtest to call a deployed Lambda function after warmup: https://github.com/awslabs/aws-serverless-express/pull/116#issuecomment-382191862
• running the "lambda code" entirely locally (faked invocation of handler): https://github.com/awslabs/aws-serverless-express/pull/116#issuecomment-382197301

percentiles of ms for a complete request (res.json({ hello: 'world!' }) express app)

50 0.16584000000000287
90 0.2303940000000182
95 0.8473280000000045
99 9.287000000000006
100 9.287000000000006

--aws-serverless-express--
50 1.2586509999999862
90 1.9880190000000084
95 2.5715459999999837
99 15.208489999999983
100 15.208489999999983

You can see:

• serverless-http seems quite faster than aws-serverless-express when running locally
• performance varies so much in deployed Lambda that neither approach seems of any consequence (~2ms doesn't matter when each request can vary by ~400 ms of jitter)
• there's some kind of weird slow start locally that I'd like to investigate

[brettstack]

On maintainability there are two concerns:

1. Code maintainability
2. Mock gaps/edge cases

I'm mostly concerned with gaps in mocking. How many issues have been discovered in the mocking that had to be fixed? The primary reason I initially went with actual requests over a socket instead of mocks is because of these unknowns.

[dougmoscrop]

Right on - so, I think it's been pretty good but not flawless. There were some hiccups with local (the serverless local plugin) and I don't know if you had those as well though. Other issues were things like discovering who's expecting address to be a function and some collisions with object property names but I've tried to ensure that things are tested. I haven't felt "pain" from maintaining this library if that's worth anything :P

[dougmoscrop]

The inline numbers above are comparing the actual performance of the libraries. The linked comment contains the real-world performance, in Lambda; but it is so variable that implementation does not matter. Even if one implementation added 20ms of latency over the other, Lambda is going to add hundreds of ms of latency jitter.

[brettstack]

Thanks for the comparison @dougmoscrop. I'm going to stick with current implementation for now until we have a strong case for changing.

[brett-vendia]

Hey @dougmoscrop, I'm going to go ahead and add in the mock request/response approach in v4. I was able to take your Request and Response classes and replace the local server pretty easily. I'll add an attribution link to the top of these files 🙂

Going forward, Vendia plans on adding multi-cloud support. v4 was refactored to allow for multiple event sources (way back in May 2019 but never released), so I'm hoping it'll mostly be a matter of detecting the incoming payload and mapping to the correct event source mapper (which is really just a request/response mapper) regardless of which cloud service it's coming from.

Email me at [email protected]. I'd love to get on a Zoom