Serverless-python-requirements: When using slim, .so files are being broken

[ysgit]

Issue was mentioned on thread for another issue - https://github.com/UnitedIncome/serverless-python-requirements/issues/216#issuecomment-419037475

Trying to deploy python lambda app using circleci. Build and deploy is successful but then when lambda runs we get the following error

Runtime.ImportModuleError: Unable to import module 'src.aws_lambda.handlers': /tmp/sls-py-req/PIL/_imaging.cpython-37m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned

Plugin configuration:

custom:
  pythonRequirements:
    fileName: requirements-prod.txt
    dockerizePip: true
    zip: true
    slim: true
    useStaticCache: true
    useDownloadCache: true
    cacheLocation: './.serverless-python-cache'

Same serverless.yml configuration works when I run it from my own mac OS computer.

On circleci I've tried using dockerizePip: true with the circleci/classic:201808-01 machine and using dockerizePip: non-linux using the circleci/python:3.7-node docker image. In the latter case the whole build takes place in the circleci docker container - originally I thought this was the cause of the issue.

I don't think I can stop using slim because the file not meet the aws lambda requirements.

[palmerabollo]

Same here. I'm using camelot-py==0.7.2 that uses opencv-python-headless==4.1.0.25 and this is the error I see:

{
  "errorMessage": "Unable to import module 'handler': /opt/python/cv2/cv2.cpython-37m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned",
  "errorType": "Runtime.ImportModuleError"
}

This is my pythonRequirements setup just in case it helps:

  pythonRequirements:
    dockerizePip: non-linux
    slim: true
    zip: true
    layer:
      name: ${self:service}-python-requirements
      description: Python requirements lambda layer
      compatibleRuntimes:
        - python3.7
      licenseInfo: GPLv3
      allowedAccounts:
        - '*'

The error persists even using the flag strip: false in the pythonRequirements section (see also #345):

...
Serverless: Docker Image: lambci/lambda:build-python3.7
Serverless: Running docker run --rm -v /Users/demo/.serverless/requirements\:/var/task\:z -u 0 lambci/lambda\:build-python3.7 /bin/sh -c 'python3.7 -m pip install -t /var/task/ -r /var/task/requirements.txt && find /var/task -name \\*.so -exec strip \\{\\} \\;'...
Serverless: Zipping required Python packages...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Removing Python requirements helper...
Serverless: Packaging Python Requirements Lambda Layer...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...

[vekerdyb]

I'm seeing the same issue when adding Pillow as a dependency. Setting slim: false succeeds as a workaround, but I need to slim the lambda package.

[pranavbadami]

I'm seeing the same issue as well with opencv-python in an individually packaged function.

[pranavbadami]

After reading this issue, I realized that my serverless-python-requirements version was 4.3.0 which doesn't contain the PR #326. When I followed the workaround mentioned in the issue (basically pinning my serverless-python-requirements to a commit in master more recent than PR #326 ), I was able to solve the ELF command error which was occurring for me with opencv-python-headless.

[mirstan]

Could we get a new release that includes this fix, please?

[tomonodes]

I had this problem persist through trying both slim: False and strip: false.

The error occurred after adding the serverless-basic-authentication plugin. I have both opencv-contrib-python==4.1.1.26 and Pillow==6.1.0 in my dependencies that are zipped.

Error:

Unable to import module 'handler': /tmp/sls-py-req/cv2/cv2.cpython-36m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned

What fixed it for me was to set slim: False do sls remove, remove the cache in Library/Caches/serverless-python-requirements/... together with .requirements.zip and run sls deploy.

[boxabirds]

Harumf I'm having the same problem but sadly still no luck. Interestingly I don't see a serverless-python-requirements dir in Library/Caches/ at any stage.

Seems to me a binary incompatibility issue.

[freekwiekmeijer]

A note for anyone running into this error: ELF load command address/offset not properly aligned.

It may not have to do with your serverless packaging at all.

I spent a bunch of time figuring out our error and it was related to a version incompatibility between numpy 1.8.0 and scipy 1.5.4. The error showed as: _fblas.cpython-37m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned

When I downgraded scipy to 1.4.1, the problem was solved.