Serverless-express: "Possible EventEmitter memory leak detected" warning coming from AWS execution environment

[janaz]

Hello,

Today I noticed the following message in cloudwatch logs. It appears about 15 times a day.

ERROR   (node:8) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 beforeExit listeners added. Use emitter.setMaxListeners() to increase limit

My code (including all node_modules) doesn't listen on the beforeExit event.

My lambda function uses aws-serverless-express and generates traffic of 918k invocations / day.

I have another lambda function that has similar traffic profile, but it's not responding to API Gateway requests and is not using aws-serverless-express. I haven't seen this error logged there.

According to node documentation https://nodejs.org/docs/latest-v10.x/api/process.html#process_event_beforeexit

The 'beforeExit' event is emitted when Node.js empties its event loop and has no additional work to schedule. Normally, the Node.js process will exit when there is no work scheduled, but a listener registered on the 'beforeExit' event can make asynchronous calls, and thereby cause the Node.js process to continue.
The listener callback function is invoked with the value of process.exitCode passed as the only argument.
The 'beforeExit' event is not emitted for conditions causing explicit termination, such as calling process.exit() or uncaught exceptions.
The 'beforeExit' should not be used as an alternative to the 'exit' event unless the intention is to schedule additional work.

It seems like the AWS controlled code that is invoking the user provided handler functions listens on the beforeExit event. For some reason when aws-serverless-express is used the listener function is attached more than once to process. Is there anything that can be done in this library to mitigate from the issue?

[brettstack]

Thanks for the report. Some things that first come to mind:

1. Are you calling .listen() in your Express layer?
2. Are you initializing aws-serverless-express outside of the handler on each invocation?
3. Which runtime and resolution mode are you using (default is context.succeed)?

[janaz]

Hi Brett,

My answers are below

1. Are you calling .listen() in your Express layer?

No.

1. Are you initializing aws-serverless-express outside of the handler on each invocation?

Yes. (I recently reported an issue with insufficient file descriptors that was caused by initializing the server inside the handler. Now it's definitely outside of it. I just double checked to be sure)

1. Which runtime and resolution mode are you using (default is context.succeed)?

Runtime Node10.x.
aws-serverless-express v3.3.5
I don't specify the resolution mode, so it's using the default.

The code looks like this (it's TypeScript):

import awsServerlessExpress = require('aws-serverless-express');
import * as app from '../web';

const server = awsServerlessExpress.createServer(app.default);

export const handler = (event: any, context: any) => {
  return awsServerlessExpress.proxy(server, event, context);
};

[piyush-banknovo]

I am facing the same issue. Is it being resolved?

[ryross]

Also seeing this issue :|

[misschak]

+1

[RobinRockz]

+1

[nsutcliffe-speedline]

+1

[gareth-rylance-aurora]

Switching to the Promise based resolver stopped the error for me.

const server: Server = createServer(setUpApp(express()));

export const handler = async (event: APIGatewayEvent, context: Context) => {
    logger.info('Starting Express proxy');
    const response = await proxy(server, event, context, 'PROMISE').promise;
    logger.debug(`Sending Response Express proxy ${response}`);

    return response;
};

or

const server: Server = createServer(setUpApp(express()));

export const handler = async (event: APIGatewayEvent, context: Context) => {
    return proxy(server, event, context, 'PROMISE').promise;
};

[brett-vendia]

Fixed in v4

[jp928]

@brett-vendia I am still facing this with "@vendia/serverless-express": "^4.3.8",

I am exactly following the nestjs example

[jp928]

After cache the server problem solved.