Aws-cli: ImportError on any CLI command on Windows

[AndreLouisCaron]

Hi there,

I just ran pip install awscli in a new Python 2.7 virtual environment on Windows. It installed successfully, but any command I run gives output like this:

$ aws ecr get-login --region us-east-1
Traceback (most recent call last):
  File "...\.tox\tools\Scripts\aws.cmd", line 50, in <module>
    import awscli.clidriver
ImportError: No module named awscli.clidriver

The package in the virtual environment seems fine and I can run this command successfully (no ImportError):

$ python -c "from awscli.clidriver import main; main()" ecr get-login --region us-east-1
Unable to locate credentials. You can configure credentials by running "aws configure".

Any idea what's wrong?

I'm using this version:

$ pip freeze | grep awscli
awscli==1.11.36

[dstufft]

I suspect that this is going to be an error in aws.cmd but after looking at it I'm not entirely sure what it might be. Could you tell me what specific version of Python and virtualenv you're using here so I can try to replicate it?

[AndreLouisCaron]

Hey there! Thanks for the quick reply :-)

Here is the full Python version:

$ py -2.7
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32

Here is the virtualenv version:

$ py -2.7 -m pip freeze | grep -i virtualenv
virtualenv==15.0.3

Let me know if I can provide any extra info that could help!

[AndreLouisCaron]

Did you get a chance to try and reproduce the issue?

[kyleknap]

In my experience, I have not run into any issues like that installing the CLI with pip on Windows machines. Are testing framework for Windows use pip to install it as well. So my guess is that it may be an environment issue.

Have you taken a look at this issue: https://github.com/aws/aws-cli/issues/116? Looks like a similar problem as yours. For them looks like they did not set their PATH variable appropriately. It seems that script is not able to find the packages in the virtualenv you installed.

[AndreLouisCaron]

I have never needed to set PATH for any Python tool installed in a virtualenv since the virtualenv's activate script does it automatically for me. What should I add in PATH?

[AndreLouisCaron]

Just to be clear, I just reproduced this with the following sequence of commands:

$ py -2.7 -m virtualenv awscli-bug

$ awscli-bug\Scripts\activate

$ pip install awscli
...
Successfully installed PyYAML-3.12 awscli-1.11.39 botocore-1.5.2 colorama-0.3.7 docutils-0.13.1 futures-3.0.5 jmespath-0.9.0 pyasn1-0.1.9 python-dateutil-2.6.0 rsa-3.4.2 s3transfer-0.1.10 six-1.10.0

$ aws ecr --help
Traceback (most recent call last):
  File "F:\scratchpad\awscli-bug\awscli-bug\Scripts\aws.cmd", line 50, in <module>
    import awscli.clidriver
ImportError: No module named awscli.clidriver

[AndreLouisCaron]

OK, took a second to try to dig a bit deeper.

I copied the Python interpreter lookup from aws.cmd into a new script. Here is aws-py-lookup.cmd:

@echo OFF
for %%i in (cmd bat exe) do (
    for %%j in (python.%%i) do (
        echo "%%~$PATH:j"
    )
)

This script outputs:

$ aws-py-lookup.cmd
""
"C:\Users\acaron\.console\develop\python.bat"
"F:\scratchpad\awscli-bug\awscli-bug\Scripts\python.exe"

Looks like I have a python.bat in my PATH, which is found before the python.exe in my virtualenv.

I don't recall using that python.bat, so I deleted it and it now works fine. I guess this resolves my issue but...

Why would you match a python.bat in that loop?

[CMCDragonkai]

I'm getting as well, but I'm pretty sure I do not have a python.bat. I'm on python 3.6 on windows 10.

[arsho]

I had similar issue with Windows 10 (64 bit). Python 3.5 and Python 2.7 are installed in my PC. I was getting ImportError: No module named awscli.clidriver.

Then I added %USERPROFILE%\AppData\Roaming\Python\Python35\Scripts in environment path variable and removed Python 2.7 from the environment path variable. Now I can successfully use awscli.

I have created a step by step AWSCLI installation guide in this Github repository: https://github.com/arsho/installation/tree/master/awscli_installation.

[yi-jiayu]

Just ran into this issue as well. I have both Python 3.6 and 2.7 installed on my PC and registered in my path as python and python3 respectively, and similarly aliased their respective pips to pip and pip3. I installed the aws cli into my Python 3 installation.

However, because aws.cmd has a hardcoded search for python.{cmd,bat,exe}, it finds and uses python.exe on my system which points to Python 2 and does not have awscli installed.

Apart from adding some heuristics to locate the python executable closest to the script location, perhaps just moving the second block of for loops which looks for a file association for .py files in front may solve the problem for users who have manually set a file association or use the Python Launcher for Windows specified in PEP 397 and set their default Python distribution and fall back to the first block otherwise:

Place this block:

for /f "tokens=2 delims==" %%i in ('assoc .py') do (
    for /f "tokens=2 delims==" %%j in ('ftype %%i') do (
        for /f "tokens=1" %%k in ("%%j") do (
            call :SetPythonExe %%k
        )
    )
)

before this one:

for %%i in (cmd bat exe) do (
    for %%j in (python.%%i) do (
        call :SetPythonExe "%%~$PATH:j"
    )
)

[dstufft]

Quite honestly, I wonder why we're using aws.cmd at all instead of using the built in wrapper support in setuptools/pip.

[AndreLouisCaron]

Would you consider a patch that removes aws.cmd and replaces it with a setuptools command-line entry point?

[dstufft]

I think that is the ideal resolution to this, but I'm unsure why this ever used aws.cmd instead of an entry point. It'd probably be worthwhile to figure that out before we go around changing it incase there was a good reason for it. Unfortunately this code has been around forever and there's no comments, so there may not be a way to get that information.

I'm inclined to say we should just move to the script wrappers, but perhaps @jamesls @stealthycoin @kyleknap or @JordonPhillips have a different opinion.

[dstufft]

It'd be great if one of the affected folks could try out https://github.com/aws/aws-cli/pull/2562 and double check to see if that solves your problem.

[AndreLouisCaron]

I'll gladly try it out next week when I get back to work on the PC that was experiencing the issue.

[yi-jiayu]

Fixes it for me.

[davclark]

I'm confused. I see that there was a merged pull request that, for example, deleted aws.cmd. I tried installing awscli from pypi and also from HEAD of develop here, and aws.cmd is back and creating this problem again... sorry if I'm just being dense!

[iongion]

This issue still exists as of 09 Feb 2018, just installed all in a fresh msys2 environment and for a certain reason, it invokes the CMD file (I am calling this program from an npm script, as npm is a native app, it calls exec of NodeJS, which in turn, calls the cmd file :(