Sharp: Using Sharp on AWS Lambda Nodejs (8.10) - Unable to import module 'handler'

[JayAdra]

Hey there!

I know there's been a lot of AWS Lambda issues posted in the past, but having read them all and tried everything I can think of, I still can't get Sharp to run in my Lambda.

Here's the error I get:

Unable to import module 'handler': Error
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/libs/imagiser.js:4:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)

Now for some context, I have received this error in the past when trying to create some Lambda@Edge functions (using Nodejs 6.10), and the issue was the usual: I was uploading the Windows DLLs from my local Sharp installation, rather than compiling a Linux version and uploading that instead.

I fixed that issue and had Sharp working on my Lambda@Edge function, by setting up an EC2, compiling Sharp with Nodejs 6.10, downloading the node_modules folder and replacing my local files before deploying. This worked great and was done with the following script in my package.json:

rm -rf node_modules/sharp && copyfiles --up 1 _compiled/sharp/** node_modules

So now when I've attempted to use Sharp in my other Lambda functions (which use Nodejs 8.10), I figured all I'd need to do, is the same process, but compile Sharp using Nodejs 8.10.

Unfortunately this hasn't worked, and I've tried all sorts of variations, versions etc. and none have worked.

At one point I got a different error:

module initialization error: Error
was compiled against a different Node.js version using
NODE_MODULE_VERSION 48. This version of Node.js requires
NODE_MODULE_VERSION 57. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).
at Object.Module._extensions..node (module.js:681:18)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/node_modules/sharp/lib/constructor.js:10:15)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)

But I suspect this is just when I was using the Nodejs 6.10 compiled files. Every variation since has given me the initial error I posted above.

I guess my question is, is there any reason anyone can see why this wouldn't be working? Or any specific steps needed to get Sharp working on Nodejs 8.10? Or maybe some other compatibility issue with this specific version? Or with Lambda itself?

I'm short on time to get this working, so any suggestions at all are greatly appreciated!

Thanks!
Jay.

[lovell]

Hello, are you sure the Unable to import module 'handler' message is coming from sharp? It suggests a Lambda (or Webpack, or serverless?) config problem.

https://github.com/search?q=%22Unable+to+import+module+%27handler%27%22&type=Issues

[lukeclewlow]

I have a potentially related problem working in 8.10, have followed the instructions in the docs to build for Lambda from a windows setup.

rm -rf node_modules/sharp
docker run -v "$PWD":/var/task lambci/lambda:build-nodejs8.10 npm install

Which works great. You are then able to test it runs in that environment by running
docker run --rm -v "$PWD":/var/task lambci/lambda:nodejs8.10 handler.myHandler

Which shows that sharp has been required just fine. But when deploying to Lambda, it no longer loads with the error:

module initialization error: Error
at Object.Module._extensions..node (module.js:681:18)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/node_modules/sharp/lib/constructor.js:10:15)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)

[JayAdra]

@lovell Yes I believe so. Please see below comment:

Unable to import module 'handler' usually occurs due to missing dependencies. The error is misleading in this case, but you can think of it as Lambda (the Node.js runtime) trying to import your handler. If there are any syntax errors, missing modules or similar, Unable to import module 'handler' gets thrown.

I'm fairly sure I received this same error initially when using Sharp on my Lambda@Edge functions, and I was able to resolve it by bundling a precompiled Sharp from Amazon Linux.

I will keep trying to debug it and make sure it's not something else causing the issue, just in case. But any other suggestions in the meantime are appreciated!

[JayAdra]

As an update - I thought I found the issue in my own custom "imagiser" class (a wrong capitilisation on my required lib), but fixing that has taken me back to this familiar, but sharp-specific error:

Unable to import module 'handler': Error
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/node_modules/sharp/lib/constructor.js:7:16)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)

I looked at:

at Object.<anonymous> (/var/task/node_modules/sharp/lib/constructor.js:7:16)

and it has:

const semver = require('semver');

Is this potentially not being included somehow?

[JayAdra]

Final update! It's fixed!

So I'm not sure why or what's different about it, but when I set up my deployment to go through a Bitbucket Pipeline (using the node:8.11 image), it ran npm install, which downloaded the linux x64 libs and deployed them with my function and is now working.

I don't know how this differs from my installation from my Linux EC2, but there must be some discrepancy there. I downloaded the deployed zip folder to check the files, but can't see any obvious differences. I suppose it doesn't matter now, but moreso just curious as to the difference between these compilations.

Thanks for your help and keep up the good work on this awesome library!

[lukeclewlow]

@JayAdra any chance of uploading the working compilation?

[hamletrp]

Right, good question, would you kindly share ?

Cheers,
Hamlet

[artboard-studio]

Running

rm -rf node_modules/sharp
docker run -v "$PWD":/var/task lambci/lambda:build-nodejs8.10 npm install

solved the problem