Aiohttp: Unexpected SSL error (CERTIFICATE_VERIFY_FAILED)

[mpitt]

Long story short

aiohttp fails to make a request to a URL with a valid certificate.

Expected behaviour

A request to a URL presenting a valid certificate should succeed.

Actual behaviour

An SSLError exception is raised.

Steps to reproduce

async def do_request(url: str, params: dict={}):
    async with aiohttp.ClientSession() as session:
        async with session.get(url, params=params) as resp:
            print(resp.status)
            print(await resp.text())

loop.run_until_complete(do_request('https://api.cerved.com/'))

Traceback (most recent call last):
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/connector.py", line 655, in _create_direct_connection
    local_addr=self._local_addr)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 802, in create_connection
    sock, protocol_factory, ssl, server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 828, in _create_connection_transport
    yield from waiter
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/sslproto.py", line 503, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/sslproto.py", line 201, in feed_ssldata
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/connector.py", line 375, in connect
    proto = yield from self._create_connection(req)
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/connector.py", line 632, in _create_connection
    _, proto = yield from self._create_direct_connection(req)
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/connector.py", line 682, in _create_direct_connection
    (req.host, req.port, exc.strerror)) from exc
aiohttp.client_exceptions.ClientConnectorError: [Errno 1] Can not connect to api.cerved.com:443 [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "<stdin>", line 3, in do_request
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/client.py", line 626, in __aenter__
    self._resp = yield from self._coro
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/client.py", line 225, in _request
    conn = yield from self._connector.connect(req)
  File "/Users/m/.virtualenvs/stari-most/lib/python3.6/site-packages/aiohttp/connector.py", line 380, in connect
    .format(key, exc.strerror)) from exc
aiohttp.client_exceptions.ClientConnectorError: [Errno 1] Cannot connect to host api.cerved.com:443 ssl:True [Can not connect to api.cerved.com:443 [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)]]

requests does not have this problem

r = requests.get('https://api.cerved.com')
print(r.status_code)

Your environment

MacOS 10.13.3, Python 3.6.1, aiohttp 2.0.7 or 3.0.7 (confirmed on both versions)

Additional information

The certificate has been confirmed valid: https://www.ssllabs.com/ssltest/analyze.html?d=api.cerved.com

[asvetlov]

By running your snippet I see 404 Not Found response but SSL certificate checks all passed.
aiohttp does nothing special with certs but uses built-in Python support.
In turn Python is not shipped with certificates bundle but uses OS one.

requests uses certifi for root certificates source. This approach has own advantages and disadvantages. Several years ago we decided to not rely on custom certificate libraries.

But you can override aiohttp defaults by passing ssl=ssl.SSLContext() into session.get().
It's up to you how to construct SSL context and what certificates to load into.
As second option you can update certificates on your MacOS X globally (sorry, I have no idea how to do it, I'm Ubuntu user)..

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].