Sharp: aws lambda install Please help me

[jaman99]

i use lambda layers sharp npm file zip than upload on lambda layers not its not working .. ; "errorMessage": "Cannot find module 'sharp'",

npm install --arch=x64 --platform=linux --target=8.10.0 sharp

[lovell]

Hi, there's not a lot to go on here. Does it work without the use of layers?

I know Lambda layers have very specific file and directory naming requirements, so make sure all of those are correct and that the filesystem inheritance between layers is working as expected, perhaps with another module/dependency.

https://sharp.pixelplumbing.com/en/stable/install/#aws-lambda

[exonis]

Hi, yes it works using layers, here is how I did it as a mac user with the serverless awsnodejs-ecma-script template which uses webpack to bundle files.

1- In your local lambda directory create your layer folder, I called it "sharpLayer"
2- Get the libvips release for linux x64 (same as aws lambda environment):
One of the ways to get it is to run this the following command in a temp folder:
npm install --no-cache --arch=x64 --platform=linux --target=8.10.0 sharp
It's going to create a folder named "false" which contains the libvips build release files.
Find and copy libvips-cpp.so.42, libvips-cpp.so.42.9.5, libvips.so.42 and libvips.so.42.9.5 under sharpLayer/lib.

Having a /lib folder is very important, this is where aws lambda is going to look for the shared library at runtime.

3- Get the sharp modules:
Clone the official aws-lambda docker into the folder of your choice on your machine:
git clone https://github.com/lambci/docker-lambda.git
Then cd into it and run:
docker run --rm -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp
It is going to download and install sharp using the lambda environment.
Copy the newly created node_modules folder and copy it into sharpLayer

Your folder structure should now look like this:
sharpLayer/
--lib/
----The 4 files from libvips
--node_modules/
----The sharp install and its dependencies generated using docker`

4- Configure serverless:
Add those lines at the end of your serverless.yml file:
layers:
--sharp:
----path: sharpLayer

5- Configure webpack (for webpack users only):
Because sharp is in a layer, you don't want to let serverless bundle it with webpack when deploying.

To achieve this you need to replace all your import sharp from 'sharp' to const sharp = require('sharp')

Then tell webpack that sharp is an external dependency in your webpack.config.js like so:

module.exports = {
--entry: ...,
--target: ...,
--module: {...},
--output: {...},
--externals: {
---- 'sharp': '../../opt/node_modules/sharp'
--}
};

We also tell webpack to rewrite require('sharp') to require('../../opt/node_modules/sharp') which is where aws lambda expects to find it at runtime.

6- Deploy and add the layer to your lambda:
Run sls deploy from your lambda function directory
Your layer should be around 33MB and once deployed serverless will display its ARN:
layers:
--sharp: arn:aws:lambda:us-east-1:17XXXXXXXX77:layer:sharp:1

In your aws console go to your Lambda function and add the layer using its ARN.

Note that the last digit of the ARN represents the layer version number(:1) and will be incremented after each deployment so you'll need to respecify the correct layer each time.

Et voila! The entire library is working perfectly for me on lambda.
Good luck

[jlarmstrongiv]

Just a few more details to the excellent instructions above.

Folder structure:

sharpLayer
- serverless.yml
- sharpLayer
  - nodejs
    - node_modules
  - lib
    - libvips.so.42.9.5
    - libvips-cpp.so.42.9.5

A few other notes. On mac, the npm install command fails unless I have run npm init or created a package.json beforehand.

The other two versions of libvips are no longer installed, and they are not needed in the final package.

Using the new folder structure, according to this post, will allow you to use const sharp = require('sharp') without digging through opts directory; it worked for me.

Here is a sample serverless.yml:

service:
  name: sharp-layer

provider:
  name: aws
  runtime: nodejs10.x

layers:
  sharp:
    path: sharpLayer
    name: ${self:provider.stage}-sharp
    description: [email protected]
    compatibleRuntimes:
      - nodejs10.x
    licenseInfo: Apache License 2.0
    allowedAccounts:
      - '*'
    retain: false

See serverless docs for more options.

[nazari-dev]

---- 'sharp': '../../opt/node_modules/sharp'

Should be '../../opt/sharpLayer/node_modules/sharp'

Btw, Thanks for sharing the approach.

[mcroker]

I really struggled with the because I was also using serverless-webpack which re-installs all packages, and I couldn't find the right combination of config for package.json, webpack.config.json and serverless.yml...

In the end the work-around I went with was to run the serverless deploy command from within docker.

docker run -v $PWD:/var/task -v $HOME/.aws:/root/.aws lambci/lambda:build-nodejs12.x ./node_modules/.bin/serverless --stage dev --aws-profile myProfile deploy

I may come back and try and get the layer working later - but this got me past the problem.

[abdulmoizlakhani]

If you are a windows user then on Step#3, you may need to use:
docker run --rm -v "%cd%":/var/task lambci/lambda:build-nodejs10.x npm install sharp

Replaced: ${PWD} with "%cd%"

I hope it may help those who are getting the following kind of error.
_docker: Error response from daemon: create ${PWD}: "${PWD}" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path._

Cheers