Aws-sam-cli: AWS CLI not working with SAM CLI 0.5.0

[ejhayes]

Description:

It appears that the latest version of the AWS CLI is not working with SAM CLI. I had to install AWS CLI version 1.14.46 to get it working with SAM CLI.

Steps to reproduce the issue:
When I install the latest version of awscli:

$ pip install awscli --user --upgrade
Collecting awscli
  Downloading https://files.pythonhosted.org/packages/4e/cf/0c313db4b8e3b231447d3807657db4f5e7fad26d5eaeb294b3cfa1388a6c/awscli-1.15.84-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 4.0MB/s
Requirement already satisfied, skipping upgrade: docutils>=0.10 in /Library/Python/2.7/site-packages (from awscli) (0.14)
Requirement already satisfied, skipping upgrade: rsa<=3.5.0,>=3.1.2 in /Library/Python/2.7/site-packages (from awscli) (3.4.2)
Requirement already satisfied, skipping upgrade: colorama<=0.3.9,>=0.2.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli) (0.3.7)
Collecting botocore==1.10.83 (from awscli)
  Downloading https://files.pythonhosted.org/packages/b4/fa/b0b08db2c935ae27fb0c88bad73d9a04fee95984a5a616836ae49fc8c445/botocore-1.10.83-py2.py3-none-any.whl (4.5MB)
    100% |████████████████████████████████| 4.5MB 4.0MB/s
Requirement already satisfied, skipping upgrade: PyYAML<=3.13,>=3.10 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli) (3.12)
Requirement already satisfied, skipping upgrade: s3transfer<0.2.0,>=0.1.12 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli) (0.1.13)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in /Users/****/Library/Python/2.7/lib/python/site-packages (from rsa<=3.5.0,>=3.1.2->awscli) (0.4.4)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from botocore==1.10.83->awscli) (0.9.3)
Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /Users/****/Library/Python/2.7/lib/python/site-packages (from botocore==1.10.83->awscli) (2.7.3)
Requirement already satisfied, skipping upgrade: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /Library/Python/2.7/site-packages (from s3transfer<0.2.0,>=0.1.12->awscli) (3.2.0)
Requirement already satisfied, skipping upgrade: six>=1.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore==1.10.83->awscli) (1.11.0)
boto3 1.5.13 has requirement botocore<1.9.0,>=1.8.27, but you'll have botocore 1.10.83 which is incompatible.
localstack 0.8.4 has requirement flask==0.10.1, but you'll have flask 1.0.2 which is incompatible.
localstack 0.8.4 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible.
Installing collected packages: botocore, awscli
  Found existing installation: botocore 1.8.50
    Uninstalling botocore-1.8.50:
      Successfully uninstalled botocore-1.8.50
  Found existing installation: awscli 1.14.46
    Uninstalling awscli-1.14.46:
      Successfully uninstalled awscli-1.14.46
Successfully installed awscli-1.15.84 botocore-1.10.83

The AWS CLI works just fine:

$ aws s3 ls
2015-10-26 14:07:18 mybucket

But the SAM CLI is failing:

$ sam validate
Traceback (most recent call last):
  File "/Users/****/Library/Python/2.7/bin/sam", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 3105, in <module>
    @_call_aside
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 3089, in _call_aside
    f(*args, **kwargs)
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 3118, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 580, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 593, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/pkg_resources/__init__.py", line 781, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'botocore<1.9.0,>=1.8.27' distribution was not found and is required by boto3

If I reinstall SAM CLI it starts working:

$ sudo pip install --user aws-sam-cli --upgrade
Requirement already up-to-date: aws-sam-cli in /Users/****/Library/Python/2.7/lib/python/site-packages (0.5.0)
Requirement already satisfied, skipping upgrade: six~=1.11.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.11.0)
Requirement already satisfied, skipping upgrade: click~=6.7 in /Library/Python/2.7/site-packages/click-6.7-py2.7.egg (from aws-sam-cli) (6.7)
Requirement already satisfied, skipping upgrade: enum34~=1.1.6 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.1.6)
Requirement already satisfied, skipping upgrade: Flask~=1.0.2 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.0.2)
Requirement already satisfied, skipping upgrade: boto3~=1.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.5.13)
Requirement already satisfied, skipping upgrade: PyYAML~=3.12 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (3.12)
Requirement already satisfied, skipping upgrade: cookiecutter~=1.6.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.6.0)
Requirement already satisfied, skipping upgrade: aws-sam-translator==1.6.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.6.0)
Requirement already satisfied, skipping upgrade: docker>=3.3.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (3.5.0)
Requirement already satisfied, skipping upgrade: dateparser~=0.7 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (0.7.0)
Requirement already satisfied, skipping upgrade: python-dateutil~=2.6 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (2.7.3)
Requirement already satisfied, skipping upgrade: Jinja2>=2.10 in /Users/****/Library/Python/2.7/lib/python/site-packages (from Flask~=1.0.2->aws-sam-cli) (2.10)
Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /Users/****/Library/Python/2.7/lib/python/site-packages (from Flask~=1.0.2->aws-sam-cli) (0.24)
Requirement already satisfied, skipping upgrade: Werkzeug>=0.14 in /Users/****/Library/Python/2.7/lib/python/site-packages (from Flask~=1.0.2->aws-sam-cli) (0.14.1)
Collecting botocore<1.9.0,>=1.8.27 (from boto3~=1.5->aws-sam-cli)
  Downloading https://files.pythonhosted.org/packages/5a/9f/c74178afb86983446a4aea1f308272f6e92a96dd7b6efd086eb50241b10c/botocore-1.8.50-py2.py3-none-any.whl (4.1MB)
    100% |████████████████████████████████| 4.1MB 4.7MB/s
Requirement already satisfied, skipping upgrade: s3transfer<0.2.0,>=0.1.10 in /Users/****/Library/Python/2.7/lib/python/site-packages (from boto3~=1.5->aws-sam-cli) (0.1.13)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from boto3~=1.5->aws-sam-cli) (0.9.3)
Requirement already satisfied, skipping upgrade: binaryornot>=0.2.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (0.4.4)
Requirement already satisfied, skipping upgrade: jinja2-time>=0.1.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (0.2.0)
Requirement already satisfied, skipping upgrade: poyo>=0.1.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (0.4.1)
Requirement already satisfied, skipping upgrade: future>=0.15.2 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (0.16.0)
Requirement already satisfied, skipping upgrade: whichcraft>=0.4.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (0.4.1)
Requirement already satisfied, skipping upgrade: requests>=2.18.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from cookiecutter~=1.6.0->aws-sam-cli) (2.18.4)
Requirement already satisfied, skipping upgrade: jsonschema~=2.6 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-translator==1.6.0->aws-sam-cli) (2.6.0)
Requirement already satisfied, skipping upgrade: backports.ssl-match-hostname>=3.5; python_version < "3.5" in /Users/****/Library/Python/2.7/lib/python/site-packages (from docker>=3.3.0->aws-sam-cli) (3.5.0.1)
Requirement already satisfied, skipping upgrade: ipaddress>=1.0.16; python_version < "3.3" in /Users/****/Library/Python/2.7/lib/python/site-packages (from docker>=3.3.0->aws-sam-cli) (1.0.19)
Requirement already satisfied, skipping upgrade: docker-pycreds>=0.3.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from docker>=3.3.0->aws-sam-cli) (0.3.0)
Requirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /Users/****/Library/Python/2.7/lib/python/site-packages (from docker>=3.3.0->aws-sam-cli) (0.46.0)
Requirement already satisfied, skipping upgrade: pytz in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from dateparser~=0.7->aws-sam-cli) (2013.7)
Requirement already satisfied, skipping upgrade: tzlocal in /Users/****/Library/Python/2.7/lib/python/site-packages (from dateparser~=0.7->aws-sam-cli) (1.5.1)
Requirement already satisfied, skipping upgrade: regex in /Users/****/Library/Python/2.7/lib/python/site-packages (from dateparser~=0.7->aws-sam-cli) (2018.8.17)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /Users/****/Library/Python/2.7/lib/python/site-packages (from Jinja2>=2.10->Flask~=1.0.2->aws-sam-cli) (1.0)
Requirement already satisfied, skipping upgrade: docutils>=0.10 in /Library/Python/2.7/site-packages (from botocore<1.9.0,>=1.8.27->boto3~=1.5->aws-sam-cli) (0.14)
Requirement already satisfied, skipping upgrade: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /Library/Python/2.7/site-packages (from s3transfer<0.2.0,>=0.1.10->boto3~=1.5->aws-sam-cli) (3.2.0)
Requirement already satisfied, skipping upgrade: chardet>=3.0.2 in /Users/****/Library/Python/2.7/lib/python/site-packages (from binaryornot>=0.2.0->cookiecutter~=1.6.0->aws-sam-cli) (3.0.4)
Requirement already satisfied, skipping upgrade: arrow in /Users/****/Library/Python/2.7/lib/python/site-packages (from jinja2-time>=0.1.0->cookiecutter~=1.6.0->aws-sam-cli) (0.12.1)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /Users/****/Library/Python/2.7/lib/python/site-packages (from requests>=2.18.0->cookiecutter~=1.6.0->aws-sam-cli) (2017.11.5)
Requirement already satisfied, skipping upgrade: idna<2.7,>=2.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from requests>=2.18.0->cookiecutter~=1.6.0->aws-sam-cli) (2.6)
Requirement already satisfied, skipping upgrade: urllib3<1.23,>=1.21.1 in /Users/****/Library/Python/2.7/lib/python/site-packages (from requests>=2.18.0->cookiecutter~=1.6.0->aws-sam-cli) (1.22)
Requirement already satisfied, skipping upgrade: functools32; python_version == "2.7" in /Users/****/Library/Python/2.7/lib/python/site-packages (from jsonschema~=2.6->aws-sam-translator==1.6.0->aws-sam-cli) (3.2.3.post2)
Requirement already satisfied, skipping upgrade: backports.functools_lru_cache>=1.2.1 in /Users/****/Library/Python/2.7/lib/python/site-packages (from arrow->jinja2-time>=0.1.0->cookiecutter~=1.6.0->aws-sam-cli) (1.5)
awscli 1.15.84 has requirement botocore==1.10.83, but you'll have botocore 1.8.50 which is incompatible.
Installing collected packages: botocore
  Found existing installation: botocore 1.10.83
    Uninstalling botocore-1.10.83:
      Successfully uninstalled botocore-1.10.83
Successfully installed botocore-1.8.50

SAM CLI Works:

$ sam validate
2018-08-23 12:45:18 Found credentials in shared credentials file: ~/.aws/credentials
2018-08-23 12:45:19 Starting new HTTPS connection (1): iam.amazonaws.com
/path/to/template.yaml is a valid SAM Template

AWS CLI Not Working:

$ aws s3 ls
Traceback (most recent call last):
  File "/Users/****/Library/Python/2.7/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/awscli/clidriver.py", line 19, in <module>
    from botocore.hooks import AliasedEventEmitter
ImportError: cannot import name AliasedEventEmitter

Looking at BotoCore the last version in the 1.8.x tags is 1.8.50 from Feb 23, 2018 (https://github.com/boto/botocore/commit/354cdc10f2d57225813bb7915e005677ad4c8038#diff-44a6ba70b6b43c6a26b74ec895788c49)--the version of the AWS CLI that day is 1.14.46.

Installing that version of CLI results in both AWS CLI and SAM CLI both working at the same time:

$ pip install awscli==1.14.46 --user
Collecting awscli==1.14.46
  Downloading https://files.pythonhosted.org/packages/be/40/6aacaf3eeb9e99999c428c056de639401ad32e451abe028b3a849b3dfa69/awscli-1.14.46-py2.py3-none-any.whl (1.2MB)
    100% |████████████████████████████████| 1.2MB 7.5MB/s
Requirement already satisfied: colorama<=0.3.7,>=0.2.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli==1.14.46) (0.3.7)
Requirement already satisfied: docutils>=0.10 in /Library/Python/2.7/site-packages (from awscli==1.14.46) (0.14)
Requirement already satisfied: rsa<=3.5.0,>=3.1.2 in /Library/Python/2.7/site-packages (from awscli==1.14.46) (3.4.2)
Requirement already satisfied: PyYAML<=3.12,>=3.10 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli==1.14.46) (3.12)
Requirement already satisfied: botocore==1.8.50 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli==1.14.46) (1.8.50)
Requirement already satisfied: s3transfer<0.2.0,>=0.1.12 in /Users/****/Library/Python/2.7/lib/python/site-packages (from awscli==1.14.46) (0.1.13)
Requirement already satisfied: pyasn1>=0.1.3 in /Users/****/Library/Python/2.7/lib/python/site-packages (from rsa<=3.5.0,>=3.1.2->awscli==1.14.46) (0.4.4)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from botocore==1.8.50->awscli==1.14.46) (0.9.3)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /Users/****/Library/Python/2.7/lib/python/site-packages (from botocore==1.8.50->awscli==1.14.46) (2.7.3)
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /Library/Python/2.7/site-packages (from s3transfer<0.2.0,>=0.1.12->awscli==1.14.46) (3.2.0)
Requirement already satisfied: six>=1.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.8.50->awscli==1.14.46) (1.11.0)
localstack 0.8.4 has requirement flask==0.10.1, but you'll have flask 1.0.2 which is incompatible.
localstack 0.8.4 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible.
Installing collected packages: awscli
  Found existing installation: awscli 1.15.84
    Uninstalling awscli-1.15.84:
      Successfully uninstalled awscli-1.15.84
Successfully installed awscli-1.14.46

AWS CLI Working:

$ aws s3 ls
2015-10-26 14:07:18 mybucket

SAM CLI Working:

$ sam validate
2018-08-23 12:45:53 Found credentials in shared credentials file: ~/.aws/credentials
2018-08-23 12:45:53 Starting new HTTPS connection (1): iam.amazonaws.com
/path/to/template.yaml is a valid SAM Template

Observed result:
Latest AWS CLI does not work if I install SAM CLI

Expected result:
AWS CLI should work as expected after installing SAM CLI.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
Mac High Sierra (10.13.3)

Output of sam --version:

$ sam --version
SAM CLI, version 0.5.0

$ aws --version
aws-cli/1.14.46 Python/2.7.10 Darwin/17.4.0 botocore/1.8.50

[alexgurrola]

This saved me from a lot of pain trying to debug where the latest release was broken. Thanks!

[jfuss]

@ejhayes Thanks for the report. Python dependencies strikes again!

This one is interesting. By the looks of the output you provided, it seems like pip doesn't inspect the requirements of the boto3 installed but rather checks the lowers version (in our case boto3~=1.5). So even though we have a relaxed version of boto3, boto3 defines an upper and lower bound for botocore. When installing, pip only checks that the requirements are met that are defined and not the dependencies.

This is indicated by these two lines in If I reinstall SAM CLI it starts working: section above:
Requirement already satisfied, skipping upgrade: boto3~=1.5 in /Users/****/Library/Python/2.7/lib/python/site-packages (from aws-sam-cli) (1.5.13) .... Collecting botocore<1.9.0,>=1.8.27 (from boto3~=1.5->aws-sam-cli) Downloading https://files.pythonhosted.org/packages/5a/9f/c74178afb86983446a4aea1f308272f6e92a96dd7b6efd086eb50241b10c/botocore-1.8.50-py2.py3-none-any.whl (4.1MB) 100% |████████████████████████████████| 4.1MB 4.7MB/s

We can just version bump boto3 to match (since we are behind some versions). I want to see if there is a different solution that wouldn't require an AWS CLI upgrade if SAM CLI is upgraded, but that could be the only option. The true solution here is to have a isolated way to install AWS CLI and SAM CLI on the same machine. We are investigating ways to make installs isolated for SAM CLI, so hoping things like this will become the thing of the past.

[AlJohri]

homebrew could be a good option for an isolated install on macOS

[jfuss]

@AlJohri @ejhayes @alexgurrola We are requesting for you to upgrade Boto3 (pip install [--user] --upgrade boto3) on where the AWS CLI and SAM CLI are installed.

We are actively looking at options for isolated installs which we believe will solve this completely. We don't want to dump the Boto3 version in our requirements because this constraints customers further (since we are installed globally).

I have tested locally that upgrading Boto3 will resolve the issue and leave both AWS CLI and SAM CLI working as expected.

Leaving this open for tracking on a proper solution to this.

[FabianFrank]

This bug actually reproduces independently of aws-sam-cli. For example if you installed awscli (but not boto3) a few days ago, you'll have the following versions and things are working as expected:

root@e95fa38de79f:/# pip3 list
...
awscli (1.15.78)
botocore (1.10.77)
...

root@e95fa38de79f:/# aws
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
...

Now a few days later I installed a proprietary package that requires boto3 etc in its requirements.txt:

...
boto3>=1.8.2
botocore>=1.11.2
...

After that you'll end up with an incompatible set of versions:

root@4bb9fb7646f9:/# aws
Traceback (most recent call last):
  File "/usr/local/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python3.5/dist-packages/awscli/clidriver.py", line 19, in <module>
    from botocore.hooks import AliasedEventEmitter
ImportError: cannot import name 'AliasedEventEmitter'

root@4bb9fb7646f9:/# pip3 list
...
awscli (1.15.78)
...
boto3 (1.9.0)
botocore (1.12.0)
...

It's not like pip doesn't know that something's wrong:

root@8cf767bd6a56:/# pip3 check
awscli 1.15.78 has requirement botocore==1.10.77, but you have botocore 1.12.0.

This started to make me wonder (I'm not that familiar with pip) if this is a deliberate design choice on pip's end?

The fix is to manually update awscli:

root@4bb9fb7646f9:/# pip3 install awscli --upgrade
...
Successfully installed awscli-1.16.10

root@4bb9fb7646f9:/# aws
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]

[jfuss]

We now have installers for SAM CLI. These should be used over pip to avoid these types of issues.

Closing due to installer being released.