Serverless-offline: Loading native module causes Error: Module did not self-register. After first call

[glani]

Bug Report

Current Behavior

I have experienced weird behavior.

Simple lambda that uses native module causes error after the first call

Sample Code

• file: serverless.yml

service: my-service

plugins:
  - serverless-plugin-typescript
  - serverless-offline
  - serverless-aws-documentation
  - serverless-domain-manager
  - serverless-stack-output

functions:
  processMedia:
    handler: src/process-media.processMedia
    description: Returns the result.

    events:
      - http:
          path: /process-media
          method: post
          cors: true
          documentation:
            summary: Returns the result.
            tags:
              - Media
            description: Returns the result of the API.
            methodResponses:
              - statusCode: '200'
                description: Returned when the operation is completed successfully.

• file: processMedia.ts

import 'reflect-metadata';
import { Handler, Context, Callback, APIGatewayEvent } from 'aws-lambda';

import * as cv from 'opencv4nodejs';
const processMedia: Handler = async (event: APIGatewayEvent, context: Context, callback: Callback) => {
    callback(undefined, {
        statusCode: 200,
        body: JSON.stringify({opencv: {major: cv.version.major, minor: cv.version.minor}})
    });
};
export { processMedia }

Expected behavior/code

Environment

• serverless version:

serverless --version
Framework Core: 1.61.3
Plugin: 3.2.7
SDK: 2.3.0
Components Core: 1.1.2
Components CLI: 1.4.0

• serverless-offline version: [v5.10.1]
• node.js version: [v10.17.x]
• OS: [Ubuntu 18.04]

Additional context/Screenshots

First call provides:

{
    "opencv": {
        "major": 3,
        "minor": 4
    }
}

However the second call causes:

Serverless: POST /process-media (λ: processMedia)
Serverless: Error while loading processMedia
Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:807:18)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/thirdparty/opencv4nodejs/lib/opencv4nodejs.js:20:14)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/src/process-media.ts:8:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.createHandler (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/serverless-offline/src/functionHelper.js:215:15)
    at handler (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/serverless-offline/src/ApiGateway.js:485:40)
    at module.exports.internals.Manager.execute (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/@hapi/hapi/lib/toolkit.js:41:33)
    at Object.internals.handler (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/@hapi/hapi/lib/handler.js:46:48)
    at exports.execute (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/@hapi/hapi/lib/handler.js:31:36)
    at Request._lifecycle (/home/denis/sources/krakozyabra2/krakozyabraio-serverless/node_modules/@hapi/hapi/lib/request.js:312:68)
    at process._tickCallback (internal/process/next_tick.js:68:7)

/home/denis/sources/krakozyabra2/krakozyabraio-serverless/thirdparty/opencv4nodejs/lib/opencv4nodejs.js:20:14

it is a fork from github of this library (opencv4nodejs)
where require hardcoded to the absolute path provided from environment variable

Minimized example by parcel works perfectly on aws lambda with the same native library.

I am working on the fork of having opencv4nodejs statically linked with opencv.

[gfpacheco]

This is not a question, I'm having the same issue.

I use canvas package and it works on the first time I start the server, but after saving a file and the server restarts automatically I start getting the Module did not self-register error.

@glani were you able to make it work? It really sucks having to manually restart the server every time I change a file

[glani]

@gfpacheco No. As I remember I decided not to use serverless because of this issue. And was doing several experiments by using different approach.

[mhilgefort]

I'm currently experiencing the exact same issue as you @gfpacheco
Did you by chance manage to find a solution for this?

[gfpacheco]

@mhilgefort nop, gotta restart the server every time

[mhilgefort]

@gfpacheco That's a bummer. Thank's a lot for your answer tho

[starsprung]

As a workaround to this issue, I found using allowCache let me use native modules. The offending code appears to be:

https://github.com/dherault/serverless-offline/blob/1454da7dcff5f48da9f19135cb18590b22e29cc2/src/lambda/handler-runner/in-process-runner/InProcessRunner.js#L102

Clearing the modules this way doesn't unload the native library code that has been loaded into Node, so it attempts to load multiple instances and fails.