Aws-cdk: CDK tool does not include installed external packages in virtualenvs when deploying

[c0debreaker]

Reproduction Steps

from glom import glom

$ cdk deploy

Error Log

I have a python code that imports glom. I used poetry add glom to install it. Lamda function couldn't find it. I tried pip install glom, Lambda function still encountered an error.

The error is

[ERROR] Runtime.ImportModuleError: Unable to import module 'auto_tag': No module named 'glom'

Environment

• **CLI Version : 1.18.0 (build bc924bc)
• **Framework Version: Python 3.7
• **OS : MAC OSX
• **Language : Python

Other

By following this solution, I got it to work. However, my project now contains external library that has to be committed to the git project. It looks messy.
https://stackoverflow.com/questions/58855739/how-to-install-external-modules-in-a-python-lambda-function-created-by-aws-cdk

I got it to work by running pip install glom --target lambda where lambda is the directory configured in this block

        eventHandler = _lambda.Function(
            self,
            'resourceTagger',
            runtime = _lambda.Runtime.PYTHON_3_7,
            code = _lambda.Code.asset('lambda'), <------- THIS
            handler = 'auto_tag.handler'
        )

But look how messy it is now - https://i.imgur.com/I5YMwtR.png

Are there other ways to do it something that we don't have to install it in the path specified in _lambda.Code.asset?

---

This is :bug: Bug Report

[asfourco]

One method we at Campo dealt with this issue was to create a layer that contained the necessary libraries, and then referred to it within the stack definition of the lambda function.

For example:

# Stack.py
class MyStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

       sendgrid_layer = _lambda.LayerVersion.from_layer_version_attributes(
            self,
            "sendgrid_api_layer",
            layer_version_arn="arn:aws:lambda:_aws-region_:_aws-account-id_:layer:sendgrid:1",
            compatible_runtimes=[
                _lambda.Runtime.PYTHON_3_6,
                _lambda.Runtime.PYTHON_3_7
            ]
        )

        sync_lambda = _lambda.Function(
            self,
            "LambdaFunctionName",
            runtime=_lambda.Runtime.PYTHON_3_7,
            code=_lambda.Code.from_asset("src"),
            handler="lambda-function.handler",
            layers=[sendgrid_layer]
        )

To create a layer, you can apply the same method as you did to install the libraries locally:

cd $(mktemp -d)
python3 -m venv .env
source .env/bin/activate

pip install sendgrid -t python
# Add proper file permissions
# N.B. not sure if this is necessary
chmod -R 755 .

# Create the deployment package
zip -r /tmp/layer.zip python

# Upload to your aws lambda
aws lambda publish-layer-version \
    --layer-name  layer-name \
        --description "Simple description of layer" \
    --compatible-runtimes python3.6 python3.7 \
    --zip-file fileb:///tmp/layer.zip
# The output of this command will also display the layerVersionArn, which 
# you will use in the definition of the layer within your lambda stack.
# You will also need the version number of the layer for the next command

# Don't forget to add permissions to the layer, get the layer version number from
# the previous command or from
aws lambda list-layers

# Command to add layer permissions
aws lambda add-layer-version-permission \
    --layer-name layer-name \
    --version-number version_number \
         --action "lambda:GetLayerVersion" \
         --principal "*" \
         --organization-id _your_organisation_id_\
         --statement-id _give_this_a_name_

You can find a curated list of AWS layers at: https://github.com/mthenw/awesome-layers

References that helped me create this structure:

• https://aws.amazon.com/premiumsupport/knowledge-center/build-python-lambda-deployment-package/
• https://aws.amazon.com/premiumsupport/knowledge-center/lambda-deployment-package-nodejs/
• https://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html#nodejs-package-dependencies

[c0debreaker]

Thank you so much! I'm going to try this soon. I'll let you know.

[c0debreaker]

Can't CDK do this? This way we don't have to execute aws lambda cli.

[asfourco]

Can't CDK do this? This way we don't have to execute aws lambda cli.

By "this", I assume you mean the layer deployment into AWS? If so, I haven't found a way to do this yet. Let me know if you find a way to do so.

[c0debreaker]

Your layers worked like a charm! Thanks a lot! :)

[kgunny]

I found this forum trying to solve this exact problem. I was able to create a layer directly from within the CDK code. Layer created, and referenced by lambda function, but still i get the error when i test the function from the AWS console. Where you able to run tests from console?

Create local directory (same as for function code) for layer code, then run 'pip install fabric --target ./fn_layer_train'

`

Lambda layer for access to python libaries

    fn_layer_train = lam.LayerVersion(self, "fn_layer_train",
                                      layer_version_name="fn_layer_train",
                                      code=lam.Code.asset("./fn_layer_train"),
                                      compatible_runtimes=[lam.Runtime.PYTHON_3_7]
                                      )

    # Lambda fn to start training job on EC2 instance
    fn_start_train = lam.Function(self, "fn_start_train",
                                  runtime=lam.Runtime.PYTHON_3_7,
                                  handler="fn_start_train.main",
                                  code=lam.Code.asset("./fn_start_train"),
                                  environment={"EC2_INSTANCE_ID": ec2_instance_id},
                                  layers=[fn_layer_train]
                                  )

`

[asfourco]

Can't CDK do this? This way we don't have to execute aws lambda cli.

By "this", I assume you mean the layer deployment into AWS? If so, I haven't found a way to do this yet. Let me know if you find a way to do so.

I found out to build the layer through AWS CDK and deploy it, I created a github gist to demonstrate how.

[asfourco]

@kgunny There is a way to test your lambda function locally, it requires AWS SAM installed

# from within your project
$ cdk synth > template.yml
# you'll need the function name that cdk assigned to your lambda function
# e.g., fn_start_trainEO123Q2
$ sam local invoke --no-event --debug  _cdk_assigned_function_name_ 

N.B. I've had to modify the template to point to the correct asset in cdk.out in the past. Not sure if it's still the case.

Here's the mock.sh script I created to run within my cdk python app project for local testing:

#!/bin/bash

# run cdk synth once to look for the function name and put it here
functionName=" GET FROM THE TEMPLATE"

cdk synth | gsed -r 's/^(\s*aws:asset:path:\s*)(asset\w*)/\1cdk.out\/\2/' > template.yml 
sam local invoke --no-event --debug  $functionName 

[kgunny]

Ok sorry it was my issue, it was due to python 3.7 expecting the files in the layer to be in a certain path.

For python 3.7 you must package your libraries in /python/lib/python3.7/site-packages for the layer so they end up in /opt/python/lib/python3.7/site-packages .

Also to run on AWS these need to be Linux built versions of the libraries.