Aws-sam-cli: sam build fails for python3.7 functions with pip==19.3 installed

[t2b]

Description

with aws-sam-cli==0.22.0 and pip==19.3 installed sam build fails:

Building resource 'TestFunction'
Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'module' object is not callabl

Steps to reproduce

template.yml

Transform: AWS::Serverless-2016-10-31
Resources:
  TestFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: app.handler
      Runtime: python3.7

requirements.txt

commands to reproduce

pip install pip==19.3 aws-sam-cli==0.22.0
sam build

Observed result

command output with --debug flag set.

Using SAM Template at /x/x/x/x/template.yml
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
1 resources found in the template
Found Serverless function with name='TestFunction' and CodeUri='.'
Building resource 'TestFunction'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /home/ttb/axsemantics/code/eddaplayground/requirements.txt --dest /tmp/tmp4wnow8ph
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    requirements_path=self.manifest_path
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 143, in build_dependencies
    requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 209, in build_site_packages
    scratch_directory, requirements_filepath)
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 234, in _download_dependencies
    requirements_filename, directory)
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 330, in _download_all_dependencies
    self._pip.download_all_dependencies(requirements_filename, directory)
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 632, in download_all_dependencies
    raise PackageDownloadError(error)
aws_lambda_builders.workflows.python_pip.packager.PackageDownloadError: Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'module' object is not callable


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 248, in run
    action.execute()
  File "/home/ttb/.local/share/virtualenvs/eddaplayground-9KTS0LqI/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'module' object is not callable


Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 347, 'exitReason': 'UserException', 'exitCode': 1, 'requestId': 'b01b8130-a7a5-4176-8301-0e0b5a021a34', 'installationId': '8cd66ae9-04d8-4819-94c9-8b15f5cf52b7', 'sessionId': 'd530a4e3-c3aa-4bf3-a830-01f00e264479', 'executionEnvironment': 'CLI', 'pyversion': '3.7.4', 'samcliVersion': '0.22.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'module' object is not callable

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: fedora 30
2. sam --version: 0.22.0

[joshmesilanecmd]

I'm also experiencing exactly the same issue for my customers.

Same pip and sam versions as well.

[Konchannel]

I had the same error with the same pip version.
It was solved by lowering the version of pip.
pip install pip==19.2.3

[joshmesilanecmd]

Can confirm lowering the pip version works.

[justinoue]

Thank you guys for having this problem. Looks like pip 19.3 was just released 14 hours ago.

For anyone using pipenv and not straight pip/requirements.txt you can do pipenv run python -m pip install pip==19.2.3

[igorborgest]

Same for Python3.6

[sriram-mv]

I think this is to do with new versions of pip supporting pyproject.toml as well as setup.py / requirements.txt model.

[justinoue]

Same for Python3.6

Yeah this happened for us on 3.6 with the ubuntu 1.0 codebuild image. We tried moving to the Ubuntu 2.0 codebuild image which forced a 3.7 update and same thing. It would seem to be entirely related to the pip 19.3 update. Kinda disappointing that either this package or the codebuild containers don't enforce stricter version control for pip.

[sriram-mv]

With the latest from develop, this issue should be gone. the fix will roll out as part of the next release.

[tomislacker]

@TheSriram What commit(s)/PR(s) are responsible for fixing this problem?

[jfuss]

@tomislacker https://github.com/awslabs/aws-lambda-builders/pull/132

[mguarascio]

Any idea when the next release will be? I'm running into this issue on a build with Docker image lambci/lambda:build-python3.7. Supposedly it's using pip 19.0.3, but I'm still hitting the issue:

22-Oct-2019 19:24:49 You are using pip version 19.0.3, however version 19.3.1 is available.

[jfuss]

This was released in v0.23.0.

Closing.