Serverless-express: Use callback instead of context.succeed

[lopezjurip]

This is not compatible with https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html neither Serverless (https://github.com/serverless/serverless/issues/2379)

[brettstack]

During tests, I found that using context.succeed was more performant than using callback. This is because callback waits for the event loop to be empty, which doesn't happen with Express server always listening. There is the context.callbackWaitsForEmptyEventLoop option documented here http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html, however I have not tested/profiled using this.

[dinvlad]

A bit off-topic, but is context.succeed API deprecated, or can we still rely on it for the foreseeable future? If it is, then are there any breaking changes that we should expect for aws-serverless-express in this regard?

[brettstack]

This will be supported for the foreseeable future. Even if it did go away,
the change on your end will be as simple as passing through the callback
to the proxy.

On Thu, May 11, 2017, 3:29 PM Denis Loginov [e-mail]
wrote:

A bit off-topic, but is context.succeed API deprecated, or can we still
rely on it for the foreseeable future? If it is, then are there any
breaking changes that we should expect for aws-serverless-express in this
regard?

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/awslabs/aws-serverless-express/issues/32#issuecomment-300934355,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABy6l8uUuKbcWFSBo-IlfKKm56tUSqrmks5r44vTgaJpZM4LNW3X
.

[chrisfinch]

Hi all,

Wondering if there has been any movement on this? I currently have a usecase where I would like to run an aws-serverless-express application under a single serverless lambda function. It seems that the lack of callback creates an incompatibility between the two.

Any advice would be helpful - I'd like to avoid having to split out the two applications if possible.

[geovanisouza92]

@chrisfinch You could provide a fake context, it works nice.

[mgmarino]

Hi all,

a few things:

• Are we sure this will be around for a long time? AWS has it in their documentation here that requests to continue using node v0.10 must come before June 30, 2017, perhaps this also means these functions will be removed soon thereafter for node lambda functions version > 4.x.

• We have a use case where we need to wait for the run-loop to clear. Our work around (not sure if this is applicable to your use case, @chrisfinch?) is:

export function handler(lambdaEvent: any, context: lambda.Context, callback: Callback) {
    try {
        const server = awsServerlessExpress.createServer(app);
        context.succeed = (response) => {
            server.close();
            callback(null, response);
        };
        return awsServerlessExpress.proxy(server, lambdaEvent, context);
    } catch (error) {
        // ...
    }

where we actively stop the server so that the lambda function doesn't hang after the other entries in the run loop have resolved. I haven't seen any significant performance difference between doing this and letting the server "run", but YMMV.

• It would be really nice to not have to start a http server inside of a lambda, would it not be possible to call express directly with request/response objects (e.g. app.handle(req, res);)? I understand this has other complexities, but I think it would make the behavior inside of the lambda better (e.g. no long-running code that needs to be paused). Perhaps something for another issue.

[chrisfinch]

@geovanisouza92 @mgmarino Thankyou both - very helpful - I'll give the fake context a try.

[chrisfinch]

@geovanisouza92 @mgmarino Perhaps you could help me further if possible. I seem to be missing a part of the puzzle or my understanding is lacking.

I'm getting ERR_CONTENT_DECODING_FAILED in my browser which according to the commented code is related to binary mime types not being handled correctly by serverless. I'm trying to run it locally at the moment with serverless-offline.

I'm also getting a lot of the following warnings from serverless which may or may not be related:

WARNING: Attempting to listen on socket /tmp/server0.sock, but it is already in use. This is likely as a result of a previous invocation error or timeout. Check the logs for the invocation(s) immediately prior to this for root cause, and consider increasing the timeout and/or cpu/memory allocation if this is purely as a result of a timeout. aws-serverless-express will restart the Node.js server listening on a new port and continue with this request.

Here's an excerpt from my serverless.yml

  express:
    handler: server/express/express.handler
    timeout: 300
    events:
    - http:
        path: /express/{proxy+}
        method: ANY

And here's the handler code using aws-express-serverless

'use strict'
const awsServerlessExpress = require('aws-serverless-express')
const app = require('./app')

// NOTE: If you get ERR_CONTENT_DECODING_FAILED in your browser, this is likely
// due to a compressed response (e.g. gzip) which has not been handled correctly
// by aws-serverless-express and/or API Gateway. Add the necessary MIME types to
// binaryMimeTypes below, then redeploy (`npm run package-deploy`)
const binaryMimeTypes = [
    'application/javascript',
    'application/json',
    'application/octet-stream',
    'application/xml',
    'font/eot',
    'font/opentype',
    'font/otf',
    'image/jpeg',
    'image/png',
    'image/svg+xml',
    'text/comma-separated-values',
    'text/css',
    'text/html',
    'text/javascript',
    'text/plain',
    'text/text',
    'text/xml'
]

exports.handler = (lambdaEvent, context, callback) => {
    try {
        const server = awsServerlessExpress.createServer(app, null, binaryMimeTypes);
        context.succeed = (response) => {
            server.close();
            console.log(response);
            callback(null, response);
        };
        return awsServerlessExpress.proxy(server, lambdaEvent, context);
    } catch (error) {
        // ...
    }
}

Could either of you give me any pointers? Many thanks in advance!

[dinvlad]

The sock warning is likely because you start a new server on every request. You should only do that once outside of your Lambda, so it will be 'warm' on the next request, as suggested earlier by @brettstack

Binary support currently requires quite a few tweaks, both on the server and the client side (incl. you have to send an Accept header that starts with one of your binary types). It may be worth for you to enable */* to simplify things. That is not specific to aws-serverless-express I'm afraid.

[mgmarino]

The workaround with stopping the stopping/starting the server was necessary for our use case because we needed the event loop to be cleared. @chrisfinch if you don't need this, you can move the server instantiation out, but if you do this you will need to set:

context.callbackWaitsForEmptyEventLoop = false;

at least for running in lambda.

[brettstack]

I assume you're getting ERR_CONTENT_DECODING_FAILED locally because serverless-offline doesn't convert Base64 to binary for you. Try removing all the binary mime types (at least for local).

[brettstack]

If anyone wants to provide benchmarks with callback and context.callbackWaitsForEmptyEventLoop I'd be happy to move over to that.

[brettstack]

@chrisfinch can you elaborate

I currently have a usecase where I would like to run an aws-serverless-express application under a single serverless lambda function. It seems that the lack of callback creates an incompatibility between the two.

[brettstack]

As mentioned by @dinvlad I strongly advise against awsServerlessExpress.createServer inside your handler for performance reasons.

[brettstack]

Curious what all these use-cases are for waiting for event loop to clear. Shouldn't your Express route wait for everything necessary to finish before returning?

[mgmarino]

Our use case is essentially queuing and sending events to other services (logs, database updates, etc.) and allowing other processing to still continue. It's true that we could have serialized all of these and explicitly waited for everything to end, but we found it simpler to essentially "fire and forget" as we were processing, and then wait (if necessary) for the event loop to clear.

So, regarding the point about performance, would there be any improvement if one could do away with the http server completely? It certainly seems possible to call e.g. app.handle or app with assembled request and response objects directly, or is there something I'm missing?

[geovanisouza92]

@mgmarino I'd tried just call app.handle, but in my case, I needed the full req object to make it work. It could be possible to just create one from scratch, populating it from event, but I didn't pursued it.

[chrisfinch]

@brettstack @dinvlad I managed to fix the ERR_CONTENT_DECODING_FAILED by removing the binary types as mentioned but I'm still getting the sock warning.

I've tried starting the server outside the handler function but it seems to have no effect - can you think of any other possible causes?

Thanks again.

[chrisfinch]

@chrisfinch can you elaborate

@brettstack sure. I've recently come on to a new project where there is a pre-existing codebase running on Lambda/Serverless. In this project multiple lambda functions are being used as routes that render html pages as well as many that are simple API routes.

Due to project requirements (server-rendered react components etc..) it's necessary to replace these html routes with express, hence the experimentation with aws-serverless-express.

As the existing code works well in serverless there is some resistance to porting everything to express - hence the attempt at integration.

[dinvlad]

@chrisfinch if it's working well without Express, why would you want to convert back? AFAIK aws-serverless-express is more of a compatibility layer for existing Express apps, not the other way around. Even @brettstack recommended to me to code straight into Lambda to avoid performance overhead and for better support of the microservices architecture. Vanilla Lambda can render HTML body just as well. If you need to render templates, feel free to use a templating library without the overhead of Express..

[brettstack]

There are valid reasons to move to Express. Perhaps the biggest one is simplicity of managing a single Lambda function and simple API Gateway Proxy configuration. If you benefit from microservices, then Express probably isn't for you. But the majority of applications don't need the benefits and overhead of Microservices, and would benefit from the simplicity offered by Express.

[chrisfinch]

@brettstack @dinvlad Thankyou both - I think I have a good handle on the pros and cons of splitting in to Lambda methods vs proxying to an express app.

On another note: I'm still getting this error:

WARNING: Attempting to listen on socket /tmp/server0.sock, but it is already in use. This is likely as a result of a previous invocation error or timeout. Check the logs for the invocation(s) immediately prior to this for root cause, and consider increasing the timeout and/or cpu/memory allocation if this is purely as a result of a timeout. aws-serverless-express will restart the Node.js server listening on a new port and continue with this request.

Here's my Lambda code;

const server = awsServerlessExpress.createServer(app, null, binaryMimeTypes);

module.exports = {
    handler: (lambdaEvent, context, callback) => {
        try {
        context.callbackWaitsForEmptyEventLoop = false;
        context.succeed = (response) => {
                callback(null, response);
            };
            return awsServerlessExpress.proxy(server, lambdaEvent, context);
        } catch (error) {
            console.error('=====> Error: ', error);
        }
    }
};

As you can see I'm starting the server outside of the handler as recommended by @brettstack but it still produces the warning.

Thankyou again all for you help - it's really appreciated - I understand everyone is busy with their day jobs!

[marjanSterjev]

Hi, not reading the whole event loop and freezing the container is not an option for many use cases. For example, Facebook Messenger Botly Framework or Microsoft BotBuilder. They return HTTP response, but they also emit additional events that will trigger after the response has been sent.

[MaiKaY]

@chrisfinch i got the same warning as you. Did you find a solution on that?

[dinvlad]

IMHO in cases like these it is better to use whichever tool makes less overhead, both performance and maintenance-wise. So if freezing is a problem, just use vanilla Lambdas unless there are portability considerations..

[marjanSterjev]

Maybe this is the way to go:

https://www.npmjs.com/package/lambda-restify

This library doesn't spawn local container http server, so there is continuous event pump into the event loop.
The library is work in progress, but with a couple of tweaks it works as expected.
BR

[chrisfinch]

@MaiKaY unfortunately we never found a good solution and in the end extracted our express application on to a more traditional server architecture 😢

[brettstack]

To be clear: this is not a library error. It is a warning saying something went wrong in your application layer (unhandled error or timeout on your part). It is annoying that the warning repeats itself, but at the same time, the more common the error, the more it becomes obvious in the logs.

This seems to be the most asked question for this library so I think I'll just remove it.

[MaiKaY]

Indeed it was an application issue. I am using firebase-admin in my lambda which makes troubles because of the opened database connections.

I fixed it with using goOnline and goOffline in my index.handler:

import AwsServerlessExpress from 'aws-serverless-express';
import Firebase from 'firebase-admin';
import App from './src/app';

const server = AwsServerlessExpress.createServer(App);

exports.handler = (event, context, callback) => {
    Firebase.database().goOnline();
    try {
        context.callbackWaitsForEmptyEventLoop = false;
        context.succeed = (response) => {
            Firebase.database().goOffline();
            callback(null, response);
        };
        return AwsServerlessExpress.proxy(server, event, context);
    } catch (e) {
        console.info('handler error: ', e);
    }
};

[benswinburne]

@marjanSterjev did you ever find a solution to this with the botframework? I'm struggling to get it to respond with this package and as well as packages such as botbuilder-aws-lambda too.

[autumnli]

hi~
I want to know is there a plan to support such enhancement soon? (Thank you @mgmarino! I use your workaround for now.)
We still have some scenarios that needs to wait for the eventloop to clear. Since lambda provide the flexibility with the property callbackWaitsForEmptyEventLoop, I think it'd be great if we can have such option with this awesome project!

[brettstack]

For v4 I plan on allowing you to specify the resolve method - context.succeed, callback, or async (default)

[vibhav107]

Hi, Is there any update on it?
Actually I'm using 'aws-serverless-express' package and I'm calling sql query and close the database connection after 30 seconds ideal time, I'm getting Error: Quit inactivity timeout.

But if I create a new lambda function without using this express package and use callback(null, response); working perfect, but How do I call callback(null, response) when use 'aws-serverless-express' package?

[brettstack]

This is now available in v3.3.3. Implemented here https://github.com/awslabs/aws-serverless-express/pull/173

[schuberttobias]

Thank you for creating aws-serverless-express! It allowed me to quickly set up an Express project in AWS.
Unfortunatley, the implemented Lambda function occasionally timed out.

It then took quite some effort to fix the timeouts, because I first had to undertake a lot of research to understand the architecture of aws-serverless-express. The required information was spread out in several places (e.g. Github issues and "develop" branch).

It might help future users of aws-serverless-express, if these information were added in a central location!
I am not submitting a pull request, because I am unsure where the following information is best documented. However, feel free to copy-paste my explanations to the right place.

Proposed changes:

1. Add documentation:
   aws-serverless-express makes use of Lambda container reuse. It does so by only creating the http server socket during the first invocation of a Lambda container. Subsequent invocations that reuse the container will use the same server socket.
   Therefore, there are 2 possible ways to use aws-serverless-express:

   1. Create the server outside of the function handler, it will be used in subsequent invocations, if the container is reused.
   2. Create the server inside the function handler and call server.close() at the end. If server.close() is not called, the Lambda container will contain a lot of old, unclosed server sockets in /tmp. Although the socket name contains random characters, a collision might occur prohibiting new sockets from being created. (like it was the case before the socket name was randomized, see https://github.com/awslabs/aws-serverless-express/issues/5)

2. Add documentation:
   If resolveMode 'CALLBACK' is used, it is neccessary to also context.callbackWaitsForEmptyEventLoop = false. Otherwise, the Lambda function just times out.

3. Add documentation:
   If resolveMode is not explicitly set, context.succeed() is called after the request has been handled.

4. If the lambda handler has branches that do not call awsServerlessExpress.proxy(server, event, context), additional consideration has to be shown: If these branches call callback() but aws-serverless-express is used with resolutionMode CONTEXT_SUCCEED, said branches will timeout. The timeout occurs, because there are still open http server sockets from previous invocations in the same Lambda container. These open http server sockets will result in the Node.js event loop never being empty and subsequently the function never terminates.
   There are two approaches to fix this in user code:

   1. Use resolutionMode CALLBACK with context.callbackWaitsForEmptyEventLoop = false
   2. Call context.succeed() in these other branches, instead of callback(). This will always lead to function termination, regardless of the Node.js event loop

5. Update the documentation in master branch:
   New users of aws-serverless-express are likely to look for information in the master branch (and not in the develop branch), therefore it might make sense to add the info in the master branch.

Thanks!

[schuberttobias]

@brettstack As this issue is already closed, I am unsure whether notifications were sent out for my last comment. I'd appreciate a short info on whether you saw it. Thanks!