Aws-toolkit-jetbrains: Unable to import module 'app': No module named 'requests' - Hello World example

[theoribeiro]

Describe the bug
Trying to invoke the AWS Sam Helloworld example using the plugin fails.

/Users/theoribeiro/anaconda3/bin/sam local invoke --template "/private/var/folders/sr/dx8vj9sd4gqcm35m8ygw7wsm0000gp/T/[Local] app.lambda_handler-template3196.yaml" --event "/private/var/folders/sr/dx8vj9sd4gqcm35m8ygw7wsm0000gp/T/[Local] app.lambda_handler-event1667.json" --debug-port 49597 --debugger-path /Applications/PyCharm.app/Contents/helpers/pydev --debug-args "-u /tmp/lambci_debug_files/pydevd.py --multiprocess --port 49597 --file" 2019-01-15 16:22:07 Found credentials in environment variables. 2019-01-15 16:22:08 Invoking app.lambda_handler (python3.6) 2019-01-15 16:22:08 Decompressing /var/folders/sr/dx8vj9sd4gqcm35m8ygw7wsm0000gp/T/6421406990658360484.zip

Fetching lambci/lambda:python3.6 Docker container image...... 2019-01-15 16:22:09 Mounting /private/var/folders/sr/dx8vj9sd4gqcm35m8ygw7wsm0000gp/T/tmpbj35fu9a as /var/task:ro inside runtime container Connected to pydev debugger (build 183.5153.39) START RequestId: c195948c-81dc-44de-a3d6-c9f6ddcca32c Version: $LATEST

Unable to import module 'app': No module named 'requests' END RequestId: c195948c-81dc-44de-a3d6-c9f6ddcca32c {"errorMessage": "Unable to import module 'app'"} REPORT RequestId: c195948c-81dc-44de-a3d6-c9f6ddcca32c Duration: 15 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 27 MB

If I unzip the package file in the temp folder I can see that indeed the requests package was not installed correctly in the package.

To reproduce

1. Start a new AWS SAM Project
2. Click on Lambda icon next to function handler
3. Click on Run or Debug
4. Error shows up in Console

Expected behavior
Function should be invoked after compiling the requirements correctly.

Your Environment

• OS: Mac OS X - Mojave
• JetBrains' Product: PyCharm Professional
• JetBrains' Product Version: 2018.3
• Toolkit Version: v1.1
• SAM CLI Version: 0.10.0
• JVM/Python Version: Anaconda 3.6

[abrooksv]

Did you install requests into the virtual env that your IDE is using for the project?

[theoribeiro]

@abrooksv I did not, given that sam cli documentation says that after running sam build, sam local invoke should work by using the package built in the docker container. This works perfectly in my environment without having to create a virtual environment (as it should because I want the dependencies to be built for the lambda container architecture, not my own computer's).

If AWS Toolkit uses sam cli to invoke the functions, shouldn't it work exactly the same as if I were on the terminal?

[abrooksv]

The AWS Toolkit was built in parallel to sam build, but due to time constraints and limitations at the time we could not use sam build for the run/debug steps, only the deploy serverless application system. We are looking into and tracking the work required to migrate off of our own custom python and java packagers and using sam build instead.

Once that happens, the toolkit will match the terminal.

Blocking issue for Python: https://github.com/awslabs/aws-sam-cli/issues/930

[abrooksv]

Our feature is #483

[rli]

Closing this so we can track in central issue #483

[ajbufort]

Note that I am getting the same type of error on Windows 10, and I HAVE installed the required package, boto3, for my sample function:

So what's the workaround for someone in my position?

[jsiesquen]

Hello @ajbufort I have this same problem also, your know like resolve it?

I novice and I trying execute on "hello world" lambda function...

Have this:
Package Version
arrow 0.14.2
astroid 2.2.0
atomicwrites 1.3.0
attrs 19.1.0
aws-lambda-builders 0.3.0
aws-sam-cli 0.17.0
aws-sam-translator 1.10.0
awscli 1.16.160
awsprocesscreds 0.0.1
backcall 0.1.0
binaryornot 0.4.4
boto3 1.9.162
botocore 1.12.150
certifi 2019.3.9
chardet 3.0.4
chevron 0.13.1
click 6.7
colorama 0.3.9
cookiecutter 1.6.0
dateparser 0.7.1
decorator 4.4.0
docker 4.0.1

[smdelacruz]

I have the same issue. How can we resolve this ?

[zhangzhx]

Hi @ajbufort, are you seeing this error from the SAM CLI? Can you show me your template.yaml file? It seems your project structure is not set correctly to me.

[josh-padnick]

I think the issue here is that the error message is wrong. I found I was able to resolve this when I removed the following from my Python Lambda function:

import requests

[rmzelnick]

I'm having the same problem when I try running,

C:\Users\rzelnick\aws\vendor-disbursements-lambda>sam local invoke -t templates/dev-template.yaml --debug -e events/event.json
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
local invoke command is called
No Parameters detected in the template
1 resources found in the template
Found Serverless function with name='VendorDisbursementsFunction' and CodeUri='../vendor_disbursements/'
Found one Lambda function with name 'VendorDisbursementsFunction'
Invoking app.lambda_handler (python3.7)
Environment variables overrides data is standard format
Loading AWS credentials from session with profile 'None'
2019-10-22 10:39:26 Found credentials in shared credentials file: ~/.aws/credentials
Resolving code path. Cwd=C:\Users\rzelnick\aws\vendor-disbursements-lambda\templates, CodeUri=../vendor_disbursements/
Resolved absolute path to code is C:\Users\rzelnick\aws\vendor-disbursements-lambda\vendor_disbursements
Code C:\Users\rzelnick\aws\vendor-disbursements-lambda\vendor_disbursements is not a zip/jar file
Skipping building an image since no layers were defined

Fetching lambci/lambda:python3.7 Docker container image......
Mounting C:\Users\rzelnick\aws\vendor-disbursements-lambda\vendor_disbursements as /var/task:ro,delegated inside runtime container
Starting a timer for 3 seconds for function 'VendorDisbursementsFunction'
START RequestId: 593a0fb1-0ede-18e7-76f2-1016b0ff1955 Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'app': No module named 'requests'
END RequestId: 593a0fb1-0ede-18e7-76f2-1016b0ff1955
REPORT RequestId: 593a0fb1-0ede-18e7-76f2-1016b0ff1955     Duration: 3.27 ms       Billed Duration: 100 ms Memory Size: 128 MB     Max Memory Used: 22 MB  
{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Unable to import module 'app': No module named 'requests'"
}
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam local invoke', 'duration': 2426, 'exitReason': 'success', 'exitCode': 0, 'requestId': '63bdbf44-04a5-49b0-bbb2-948da334e22a', 'installationId': 'ee29ec27-4de3-4958-84f4-6871709ed809', 'sessionId': 'cdbaaf7e-d8d6-420d-b5f8-fb68ff09c773', 'executionEnvironment': 'CLI', 'pyversion': '3.6.7', 'samcliVersion': '0.21.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

Here's my build

```C:\Users\rzelnick\aws\vendor-disbursements-lambda>sam build -u -t templatesdev-template.yaml --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
Starting Build inside a container
No Parameters detected in the template
1 resources found in the template
Found Serverless function with name='VendorDisbursementsFunction' and CodeUri='../vendor_disbursements/'
Building resource 'VendorDisbursementsFunction'

Fetching lambci/lambda:build-python3.7 Docker container image......
Mounting C:\Users\rzelnick\aws\vendor-disbursements-lambda\vendor_disbursements as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
Full dependency closure: {requests==2.22.0(wheel), chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), jsondiff==1.2.0(sdist), urllib3==1.25.6(wheel)}
initial compatible: {requests==2.22.0(wheel), chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), urllib3==1.25.6(wheel)}
initial incompatible: {jsondiff==1.2.0(sdist)}
Downloading missing wheels: {jsondiff==1.2.0(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch jsondiff==1.2.0
compatible wheels after second download pass: {requests==2.22.0(wheel), chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), urllib3==1.25.6(wheel)}
Build missing wheels from sdists (C compiling True): {jsondiff==1.2.0(sdist)}
calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/jsondiff-1.2.0.tar.gz
compatible after building wheels (no C compiling): {requests==2.22.0(wheel), chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), jsondiff==1.2.0(wheel), urllib3==1.25.6(wheel)}
Build missing wheels from sdists (C compiling False): set()
compatible after building wheels (C compiling): {requests==2.22.0(wheel), chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), jsondiff==1.2.0(wheel), urllib3==1.25.6(wheel)}
Final compatible: {chardet==3.0.4(wheel), certifi==2019.9.11(wheel), idna==2.8(wheel), requests==2.22.0(wheel), urllib3==1.25.6(wheel), jsondiff==1.2.0(wheel)}
Final incompatible: set()
Final missing wheels: set()
PythonPipBuilder:ResolveDependencies succeeded
Running PythonPipBuilder:CopySource
PythonPipBuilder:CopySource succeeded
Build inside container returned response {"jsonrpc": "2.0", "id": 1, "result": {"artifacts_dir": "/tmp/samcli/artifacts"}}
Build inside container was successful. Copying artifacts from container to host
Copying from container: /tmp/samcli/artifacts/. -> C:\Users\rzelnick\aws\vendor-disbursements-lambda.aws-sam\build\VendorDisbursementsFunction
Build inside container succeeded

Build Succeeded

Built Artifacts : .aws-sam\build
Built Template : .aws-sam\buildtemplate.yaml

Commands you can use next

[] Invoke Function: sam local invoke
[] Package: sam package --s3-bucket

Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 6025, 'exitReason': 'success', 'exitCode': 0, 'requestId': '0811107e-d01d-4cc7-b6d6-52bd6947b983', 'installationId': 'ee29ec27-4de3-4958-84f4-6871709ed809', 'sessionId': '5a17b749-48d5-483c-8626-398652ac5de9', 'executionEnvironment': 'CLI', 'pyversion': '3.6.7', 'samcliVersion': '0.21.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

And here's my template,

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
vendor_disbursements

Sample SAM Template for vendor_disbursements

More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst

Globals:
Function:
Timeout: 3
Environment:
Variables:
vendor_disbursements_rest_endpoint: REMOVED
s3bucket: REMOVED
key_folder: REMOVED
slack_webhook: REMOVED

Resources:
VendorDisbursementsFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: vendor_disbursements
Description: Verifies the integrity of drools RESTful call responses
Role: REMOVED
CodeUri: ../vendor_disbursements/
Handler: app.lambda_handler
Runtime: python3.7
VpcConfig:
SecurityGroupIds: [sg-1586ac62,sg-366ec147,sg-1169c660,sg-15efb868]
SubnetIds: [subnet-de62ed85]
Events:
VendorDisbursementsSQS:
Type: SQS
Properties:
Queue: REMOVED
BatchSize: 10
Enabled: true

Outputs:
VendorDisbursementsFunction:
Description: "Vendor Disbursements Lambda Function ARN"
Value: !GetAtt VendorDisbursementsFunction.Arn
```

[tonywang806]

I have the same problem too!

It was failed in WSL ,
But,It's successful when I run in PowerShell...

[michaelschem]

Is it possible to use a requirements.txt file to let the docker know what requirements to install?

[michaelschem]

Is it possible to use a requirements.txt file to let the docker know what requirements to install?

Woopse, I guess it is possible, but I had my requirements.txt in the wrong place.

I had my requirements.txt in the root

/
--requirements.txt
--hello_world
----app.py

Not in the application folder as it's supposed to be.

/
--hello_world
----app.py
----requirements.txt