Botocore: 'AWSHTTPSConnection' object has no attribute 'ssl_context'

[fsateler]

Hi,

I'm getting the above error when trying to use Route53 service. I have investigated and concluded that the fault is the copying of functions from AWSHTTPConnection to AWSHTTPSConnection:

# Now we need to set the methods we overrode from AWSHTTPConnection
# onto AWSHTTPSConnection.  This is just a shortcut to avoid
# copy/pasting the same code into AWSHTTPSConnection.
for name, function in AWSHTTPConnection.__dict__.items():
    if inspect.isfunction(function):
        setattr(AWSHTTPSConnection, name, function)

The problem is that this ends up overriding the __init__ function too, so the VerifiedHTTPSConnection constructor is never invoked. I don't know why this worked in the past, or why it has become a problem recently. I have locally patched that file to add and function.__name__ != '__init__' to the if above, and copied manually the constructor to AWSHTTPSConnection (to invoke VerifiedHTTPSConnection constructor). Things seem to work fine now.

[jamesls]

Hi, can you share a set of steps that demonstrate the issue? Also, how did you install botocore?

I'd like to repro this on our end to confirm the fix. Thanks!

[fsateler]

I am creating route53 dns record via boto3 (which uses botocore):

        client = boto3.client('route53')
        response = client.change_resource_record_sets(
                HostedZoneId=hosted_zone,
                ChangeBatch={
                    'Comment': 'Creando empresa ' + self.args.empresa,
                    'Changes': [
                        {
                            'Action': 'UPSERT',
                            'ResourceRecordSet': {
                                'Name': source,
                                'Type': 'CNAME',
                                'TTL': 300,
                                'ResourceRecords': [{'Value': target}]
                            }
                        }
                    ]
                })

Here is the trace my script generates:

Traceback (most recent call last):
  File "./scripts/create-app.py", line 169, in <module>
    creator.run()
  File "./scripts/create-app.py", line 136, in run
    self.create_dns_record()
  File "/usr/lib/python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "./scripts/create-app.py", line 84, in create_dns_record
    'ResourceRecords': [{'Value': target}]
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 310, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 586, in _make_api_call
    operation_model, request_dict)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 141, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 170, in _send_request
    success_response, exception):
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 249, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/lib/python3/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 204, in _get_response
    proxies=self.proxies, timeout=self.timeout)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 846, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 311, in connect
    if self.ssl_context is None:
AttributeError: 'AWSHTTPSConnection' object has no attribute 'ssl_context'

I use boto3 and botocore from my distribution (debian unstable).

[andyt10]

I encountered this issue with AWS cli working with S3, and I think it came down to a new version of 'requests' (2.18 vs 2.12) that was installed not working with AWS CLI. Had been using debian testing. Removed it from sources and it worked ok. Issue only popped up within last 24-48 hours.

[e0x70i]

Same issue here on Kali Rolling 2017.2. On requests 2.18.

[DarrenCook]

Same issue with aws s3. I did pip3 uninstall requests (which was 2.18, as other people have said) and now it works. (This on Mint 18.)

[joguSD]

These kinds of issues tend to show up when people use Ubuntu, and it seems like it extends to any debian based distribution. The debian package maintainers remove our vendored version of the requests library, which causes issues that we don't really have a lot of control over.

[fsateler]

These kinds of issues tend to show up when people use Ubuntu, and it seems like it extends to any debian based distribution. The debian package maintainers remove our vendored version of the requests library, which causes issues that we don't really have a lot of control over.

I disagree with this assessment. Yes, with the (quite old) version of requests bundled into botocore things currently work. This does not mean the current code is correct. Botocore is currently calling the wrong base class constructor. This worked for a while because HTTPSConnections constructor did not do much that was relevant to botocore. With newer requests versions, this is no longer the case.

Please fix AWSHTTPSConnection to call the correct base class constructor.

[joguSD]

I didn't mean to imply that this is not an issue or that we can't fix the issue, just that using the packages maintained by debian may have more issues than the versions installed via pip because the debian maintainers remove our vendored version of requests.

[fsateler]

I didn't mean to imply that this is not an issue or that we can't fix the issue,

Oh, sorry. I misunderstood what you were saying.

just that using the packages maintained by debian may have more issues than the versions installed via pip because the debian maintainers remove our vendored version of requests.

I'd say different issues, rather than more. The requests module also ships bugfixes in new releases.

[fsateler]

@jamesls

Hi, can you share a set of steps that demonstrate the issue? Also, how did you install botocore?

I think I have answered these questions. Short recap: you need to unvendor requests, at least version 2.18.1 triggers this problem.

I'd like to repro this on our end to confirm the fix. Thanks!

Of course. Anything else I can do to help move this forward?

[nicolas17]

A few days ago I bisected requests from git so I have a smaller version range: this incorrect code from botocore works with requests 2.15.1 and fails with requests 2.16.0, which is when requests stopped bundling urllib3. I couldn't figure out why unbundling urllib3 triggers the bug, but I suspect the code was previously ending up with multiple versions of urllib3 loaded at the same time(!), since requests's bundled version had a different package name.

[OlafvdSpek]

$ aws s3 ls

'AWSHTTPSConnection' object has no attribute 'ssl_context'

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=874367

What do I do to work around this?

This affects Debian & Ubuntu stable / production.

[mha6]

pip3 uninstall requests
pip3 install requests==2.12

This workaround did the trick for me.

[BenCoffeed]

So, This specific error occurred for me on Ubuntu 16.04 using Ansible's aws_s3 module. My workaround was to switch from using the apt module to install boto to using pip to install. Pip has newer versions available that seem to resolve the issue, in particular boto 2.48.0

[sonjz]

Same issue, but strange, because it was working 2 weeks ago, I made no changes and my cron started failing.

Original

apt-get install awscli ;

My workaround (keep original, but add upgrade after):

apt-get install python3-pip ;
pip install awscli --upgrade ;

[awilkins]

For me it only worked after

apt-get autoremove awscli

And installing via pip.

[aungwinthant]

I was uploading ssl certificates.pip3 uninstall requests solved for me. Thanks @DarrenCook

[mwilck]

The same thing happens on openSUSE Tumbleweed. It probably happens always if nice packager guys try to do the right thing and have botocore use the official requests module.
Any chance that this will ever get fixed such that botocore doesn't need to ship custom versions of requests and six any more?

[Hernrup]

Considering that #1466 unvenders requests and urllib without any apparent adverse effects, and that no maintainer has even bothered to comment... I would say that this issue seems to have been burried deeply.

[mwilck]

Ah, I missed that, thanks. Let's hope that maintainers pick it up soon, then.

[jjcampbell]

same thing as the above comment from aungwinthant
Thought I would add my name for those having similar problems.
I was trying to do certificate updates when the message 'object has no attribute ssl_context' - strange message!
I got around it on a Ubuntu 16 system by doing pip3 uninstall requests followed by pip3 install requests==2.12
Very odd but I did it and it worked like a champ! Thanks all -- and Google search!

[joguSD]

We have un-vendored requests/urllib3 and are now using urllib3 directly as of botocore v1.11.0.
Relevant upgrade notes.

It's important to note that the upgrade notes assumed that you were using a version of botocore that didn't already have requests/urllib3 un-vendored like some distribution maintainers have done in the past.

Also, as a note to any distro maintainers who had previously un-vendored our vendored version of requests, you'll notice that we do still import some of the old exception classes for backwards compatibility with new exceptions that we throw. This is likely irrelevant as the old vendored exceptions would have never been thrown using these packages (with unvendored requests) and the vendored requests and references to it can probably be removed entirely.

[a13xb]

Not aimed at anyone in particular, but if you stumble on this issue (just like I did today), and you are using letsencrypt Ubuntu PPA ppa:certbot/certbot on 16.04, it replaces a bunch of system libraries, including python-requests, which breaks system awscli. It still works with 16.04's own packages.

[P4Cu]

Another option to avoid this hell is to use python venv.

[bzamecnik]

On Ubuntu 16.04 awscli stop working with the 'AWSHTTPSConnection' object has no attribute 'ssl_context' message eg. with ECR login command: $(aws ecr get-login --region eu-west-1|sed 's/-e none//')

What helped us was to downgrade:

• python3-requests from 2.18.1-1+ubuntu16.04.1+certbot+1 to 2.9.1-3ubuntu0.1
• python3-requests from 1.21.1-1+ubuntu16.04.1+certbot+1 to 1.13.1-2ubuntu0.16.04.2

sudo apt install python3-requests=2.9.1-3ubuntu0.1 python3-urllib3=1.13.1-2ubuntu0.16.04.2

Care had to be taken that certbot or awscli might have been uninstalled:

sudo apt install awscli certbot python3-certbot-nginx

[colinmollenhour]

I get this same error on Ubuntu 14.04 LTS installed from PPA. I downgraded and holded the packages @bzamecnik referenced but then cannot install certbot due to dependencies not met. This is exactly why I hate Python, suffers massively from dependency hell... Unfortunately on this server the kernel is too old to run Docker. Any ideas?

[OlafvdSpek]

@colinmollenhour Upgrade kernel and/or distro to 18.04 LTS?

[colinmollenhour]

Thanks, @OlafvdSpek that would probably work but I shouldn't have to upgrade a supported OS just because of certbot...

Finally got it working by installing certbot via pip3 and had to uninstall and install old versions of requests and urllib3 and also manually install ndg_httpsclient.

[OlafvdSpek]

You shouldn't but... ;)

[krozinov]

On Ubuntu 16.04 awscli stop working with the 'AWSHTTPSConnection' object has no attribute 'ssl_context' message eg. with ECR login command: $(aws ecr get-login --region eu-west-1|sed 's/-e none//')

What helped us was to downgrade:

* `python3-requests` from `2.18.1-1+ubuntu16.04.1+certbot+1` to `2.9.1-3ubuntu0.1`

* `python3-requests` from `1.21.1-1+ubuntu16.04.1+certbot+1` to `1.13.1-2ubuntu0.16.04.2`

sudo apt install python3-requests=2.9.1-3ubuntu0.1 python3-urllib3=1.13.1-2ubuntu0.16.04.2

Care had to be taken that certbot or awscli might have been uninstalled:

sudo apt install awscli certbot python3-certbot-nginx

This worked for me as well. I had awscli installed via pip, but had to downgrade requests library as follows:

sudo apt install python3-requests=2.9.1-3ubuntu0.1 python3-urllib3=1.13.1-2ubuntu0.16.04.2

After this, awscli and certbot both worked. This is all on Ubuntu 16.04.05 LTS

You must be careful about automatic future updates (i.e. apt update && apt upgrade) because that will upgrade your downgraded packages. To prevent that use this:

apt-mark hold python3-requests
apt-mark hold python3-urllib3

Later, you can remove the hold by using apt-mark unhold <package-name>

[preeteshjain]

On EC2, installing requests 2.12 worked for me:

pip3 install requests==2.12

[Digitalroot]

Came across this in Jan of 2019. I fixed this for AWS Lightsail Ubuntu 16.04 [LAMP (PHP 7) 7.1.20-1] by applying @fsateler fix.

Note: certbot version 0.28.0-1+ubuntu16.04.1+certbot+4 no upgrade at this time.

Edit awsrequest.py around line 252
sudo nano /usr/lib/python3/dist-packages/botocore/awsrequest.py

The full code change I applied can see seen at https://github.com/boto/botocore/pull/1259/files

After making this change, renewal worked and the error "AttributeError: 'AWSHTTPSConnection' object has no attribute 'ssl_context'" went away.