Botocore: ImportError: cannot import name 'IPV6_ADDRZ_RE'

[jaredsilver]

Edit: if you're getting this error, it's most likely due to a dependency incompatibility issue. Search your logs for "incompatible" to see if there are any such errors.

Describe the bug
My deploy process leverages aws-cli, which leverages botocore. As of botocore's most recent release, this new line causes an error to be thrown: https://github.com/boto/botocore/commit/81768347efe3806decbbc43a61d3816620ee9ce9#diff-d5dc8bb337890042943977b62e8bb2e7db71415d35bb17a41ad4bf4780fb3617R30

Are others experiencing this issue?

Traceback

Traceback (most recent call last):
  File "/usr/local/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python3.5/dist-packages/awscli/clidriver.py", line 17, in <module>
    import botocore.session
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 30, in <module>
    import botocore.credentials
  File "/usr/local/lib/python3.5/dist-packages/botocore/credentials.py", line 34, in <module>
    from botocore.config import Config
  File "/usr/local/lib/python3.5/dist-packages/botocore/config.py", line 16, in <module>
    from botocore.endpoint import DEFAULT_TIMEOUT, MAX_POOL_CONNECTIONS
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 22, in <module>
    from botocore.awsrequest import create_request_object
  File "/usr/local/lib/python3.5/dist-packages/botocore/awsrequest.py", line 25, in <module>
    import botocore.utils
  File "/usr/local/lib/python3.5/dist-packages/botocore/utils.py", line 30, in <module>
    from urllib3.util.url import IPV6_ADDRZ_RE
ImportError: cannot import name 'IPV6_ADDRZ_RE'

[nateprewitt]

Hi @jaredsilver,

IPV6_ADDRZ_RE should be available in urllib3 1.25.4 forward which is the lowest version of the library botocore supports. Could you provide some info on how you're installing/upgrading botocore and the current depedency list you have installed?

[jaredsilver]

Thanks for the response, @nateprewitt!

I'll try my best to help answer that, though in full disclosure I have minimal Python knowledge and I'm really opening this issue on behalf of aws-cli, which I simply use in my deploy process. (If someone with better Python chops is also experiencing this issue, please chime in!)

aws-cli has a bot that automatically pulls new releases of botocore, and they updated to 1.19.0 in this commit: https://github.com/aws/aws-cli/commit/cc58a4b39da7f7a9cc12e8bfb95213befcbd8374

It looks like their dependency list and process might both be explained with the files in that commit. Does this provide enough information to answer your questions, or would additional information be helpful?

Apologies again for my lack of Python knowledge, and thanks for taking a look at this!

[jaredsilver]

Looks like they're using 1.25.7? https://github.com/aws/aws-cli/blob/master/scripts/make-bundle#L29

[nateprewitt]

No worries, Python's dependency management is a bit involved compared to other languages. awscli doesn't install urllib3 directly in most cases (outside of the bundled installer). It will rely on botocore to specify necessary requirements for urllib3 which should be >= 1.25.4 for botocore 1.19.0, which the bundler also provides.

Do you know off hand how you're getting awscli into your build environment? Are you performing pip install awscli in your build script, is it being provided by your operating system image/container, or you installing from source with the bundled installer?

[jaredsilver]

Thanks for your patience! 🙏

I'm using a CircleCI orb, and their script for installing awscli can be found here: https://github.com/CircleCI-Public/aws-cli-orb/blob/master/src/commands/install.yml

It occurs to me that CircleCI's aws-ecr-orb is pinned to the awscli orb's version 1.2, which strikes me as odd.

In addition, I've dug up the following two lines that were buried somewhere deep inside the build output:

awsebcli 3.19.1 requires botocore<1.18,>=1.17, but you'll have botocore 1.19.0 which is incompatible.
botocore 1.19.0 requires urllib3<1.26,>=1.25.4; python_version != "3.4", but you'll have urllib3 1.24.3 which is incompatible.

It seems this issue is almost certainly related to CircleCI's orb packages and not because of botocore. I very much appreciate your help pointing us in the right direction! Now to figure out how to resolve it w/ CircleCI 😅

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[jaredsilver]

As a hacky fix, I did this:

pip install -Iv botocore==1.17

This will do for the time being.

[ccwillia]

We had the same error and determined it was the awscli version which caused the problem. We have now hardcoded our awscli version to 1.18.159.