Httpx: SSL handshake failed on verifying the certificate

Created on 23 Sep 2019  路  19Comments  路  Source: encode/httpx

im using charlesproxy for debugging each requests. Proxy ip and cert is from charlesproxy

My code:

import httpx
client = httpx.Client(proxies={
    "http": "http://192.168.1.78:8888",
    "https": "http://192.168.1.78:8888",

})


client_cer = r'C:\Users\localhost\Documents\123.pem'
request = client.get("http://whoer.net",verify=client_cer)

output


SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000020CDCD7E470>
transport: <_SelectorSocketTransport fd=768 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000020CDCD7E470>
transport: <_SelectorSocketTransport closing fd=768 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
Traceback (most recent call last):
  File "C:/Users/localhost/PycharmProjects/TribalW_Test_Env/Basic/tests/httpx/proxy_test.py", line 11, in <module>
    request = client.get("http://whoer.net",headers=headers,verify=client_cer)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 818, in get
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 729, in request
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 760, in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 241, in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 230, in _get_response
    return await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 41, in __call__
    return await self(next_request, get_response)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 31, in __call__
    response = await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 191, in get_response
    request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 243, in send
    request=request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\connection_pool.py", line 118, in send
    connection = await self.acquire_connection(origin=request.url.origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 92, in acquire_connection
    return await self.tunnel_connection(origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 110, in tunnel_connection
    await self.tunnel_start_tls(origin, connection)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 194, in tunnel_start_tls
    timeout=timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 216, in start_tls
    timeout=timeout.connect_timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 416, in wait_for
    return fut.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 1134, in start_tls
    await waiter
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
bug proxies tls+pki
Source
picture GnubiBORED

Most helpful comment

Figured it out! PR will be incoming soon! :)

picture sethmlarson on 24 Sep 2019
❤1 🎉1

All 19 comments

Don't think I can help a lot here, but for info here's how we derive the CA bundle in config.py:

https://github.com/encode/httpx/blob/c9810a79d9c61583165e9aa237d1ac039086f3a8/httpx/config.py#L121-L134

To help debug this, can you make sure the program goes through L129 with the certificate you mentioned? I assume it does, because verify is obviously not a bool, and if it didn't find your certificate you'd get the IOError, but maybe we're don't correctly pass verify along to self.verify there.

florimondmanca picture florimondmanca on 23 Sep 2019

how i can make sure my code is going through line 129 with my client_cer variable? Im not experienced coding but if you tell me i can test it.

GnubiBORED picture GnubiBORED on 23 Sep 2019

Could you try running that script with HTTPX_DEBUG=1 set as an environment variable?
I can't see why TLS would be getting used unless the website is redirecting to HTTPS.

I'm guessing your proxy configuration generates a certificate for the HTTPS host on the fly and that cert you're passing is the cert to trust that self signed cert?

sethmlarson picture sethmlarson on 23 Sep 2019

Could you try running that script with HTTPX_DEBUG=1 set as an environment variable?
I can't see why TLS would be getting used unless the website is redirecting to HTTPS.

I'm guessing your proxy configuration generates a certificate for the HTTPS host on the fly and that cert you're passing is the cert to trust that self signed cert?

like these?

request = client.get("http://whoer.net",headers=headers,verify=client_cer,HTTPX_DEBUG=1)

TypeError: get() got an unexpected keyword argument 'HTTPX_DEBUG'

yes it generates one certificate thath is signed by charlesproxy so you can sniff all calls using charlesproxy mitm proxy and see there

GnubiBORED picture GnubiBORED on 23 Sep 2019

Like this: $ HTTPX_DEBUG=1 python run_script.py from the command line.

sethmlarson picture sethmlarson on 23 Sep 2019

i dont know if i did it good

i did in windows machine, i can send you certificate if you want or whatever.

cmd /c "set HTTPX_DEBUG=1 && python C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx\proxy_test.py"

@sethmlarson 


C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\network\client>cmd /c "set HTTPX_DEBUG=1 && python  C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx\proxy_test.py"
SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0x000001261B0329E8>
transport: <_SelectorSocketTransport fd=636 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x000001261B0329E8>
transport: <_SelectorSocketTransport closing fd=636 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
Traceback (most recent call last):
  File "C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx\proxy_test.py", line 11, in <module>
    request = client.get("http://whoer.net",headers=headers,verify=client_cer)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 818, in get
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 729, in request
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 760, in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 241, in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 230, in _get_response
    return await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 41, in __call__
    return await self(next_request, get_response)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 31, in __call__
    response = await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 191, in get_response
    request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 243, in send
    request=request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\connection_pool.py", line 118, in send
    connection = await self.acquire_connection(origin=request.url.origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 92, in acquire_connection
    return await self.tunnel_connection(origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 110, in tunnel_connection
    await self.tunnel_start_tls(origin, connection)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 194, in tunnel_start_tls
    timeout=timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 216, in start_tls
    timeout=timeout.connect_timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 416, in wait_for
    return fut.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 1134, in start_tls
    await waiter
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)

C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\network\client>
GnubiBORED picture GnubiBORED on 24 Sep 2019

Hmm, you should be seeing some debug messages. Do you have the latest version of HTTPX installed? (Try python -m pip freeze and paste the output)

Maybe try setx HTTPX_DEBUG=1 then run the python command?

sethmlarson picture sethmlarson on 24 Sep 2019
❤1

now yes
now worked good the debug and yes im using last httpx version i updated 7h ago with pip install httpx - upgrade when you tell me proxy support was shipped :)


04:23:35.687 - httpx.dispatch.proxy_http - forward_connection proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='http' host='192.168.1.78' port=8888)
04:23:35.687 - httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='http' host='192.168.1.78' port=8888)
04:23:35.688 - httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='http' host='192.168.1.78' port=8888))
04:23:35.689 - httpx.dispatch.connection - start_connect host='192.168.1.78' port=8888 timeout=TimeoutConfig(timeout=5.0)
04:23:35.690 - httpx.dispatch.connection - connected http_version='HTTP/1.1'
04:23:35.690 - httpx.dispatch.http11 - send_headers method='GET' target='http://whoer.net' headers=Headers({'host': 'whoer.net', 'user-agent': 'python-httpx/0.7.3', 'accept': '*/*', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive'})
04:23:35.691 - httpx.dispatch.http11 - receive_event event=NEED_DATA
04:23:35.774 - httpx.dispatch.http11 - receive_event event=Response(status_code=301, headers=[(b'date', b'Tue, 24 Sep 2019 02:23:36 GMT'), (b'transfer-encoding', b'chunked'), (b'cache-control', b'max-age=3600'), (b'expires', b'Tue, 24 Sep 2019 03:23:36 GMT'), (b'location', b'https://whoer.net/'), (b'vary', b'Accept-Encoding'), (b'server', b'cloudflare'), (b'cf-ray', b'51b14c1b4f9dc67f-MAD'), (b'connection', b'keep-alive')], http_version=b'1.1', reason=b'Moved Permanently')
04:23:35.776 - httpx.dispatch.proxy_http - tunnel_connection proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443)
04:23:35.778 - httpx.dispatch.connection - start_connect host='192.168.1.78' port=8888 timeout=TimeoutConfig(timeout=5.0)
04:23:35.780 - httpx.dispatch.connection - connected http_version='HTTP/1.1'
04:23:35.780 - httpx.dispatch.http11 - send_headers method='CONNECT' target='whoer.net:443' headers=Headers({'host': '192.168.1.78:8888', 'user-agent': 'python-httpx/0.7.3', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive', 'accept': '*/*'})
04:23:35.781 - httpx.dispatch.http11 - receive_event event=NEED_DATA
04:23:35.798 - httpx.dispatch.http11 - receive_event event=Response(status_code=200, headers=[], http_version=b'1.1', reason=b'Connection established')
04:23:35.799 - httpx.dispatch.proxy_http - tunnel_response proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443) response=<Response [200 OK]>
04:23:35.801 - httpx.dispatch.http11 - receive_event event=PAUSED
04:23:35.825 - httpx.dispatch.proxy_http - tunnel_start_tls proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443)
SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0x000001F3546149E8>
transport: <_SelectorSocketTransport fd=496 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x000001F3546149E8>
transport: <_SelectorSocketTransport closing fd=496 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
Traceback (most recent call last):
  File "C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx\proxy_test.py", line 11, in <module>
    request = client.get("http://whoer.net",headers=headers,verify=client_cer)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 818, in get
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 729, in request
    proxies=proxies,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 760, in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 241, in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 230, in _get_response
    return await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 41, in __call__
    return await self(next_request, get_response)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 31, in __call__
    response = await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 191, in get_response
    request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 243, in send
    request=request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\connection_pool.py", line 118, in send
    connection = await self.acquire_connection(origin=request.url.origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 92, in acquire_connection
    return await self.tunnel_connection(origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 110, in tunnel_connection
    await self.tunnel_start_tls(origin, connection)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 194, in tunnel_start_tls
    timeout=timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 216, in start_tls
    timeout=timeout.connect_timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 416, in wait_for
    return fut.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 1134, in start_tls
    await waiter
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
GnubiBORED picture GnubiBORED on 24 Sep 2019

Okay, so it's failing on the tunnel_start_tls which is good because that is what I figured what was happening. :) Now we just have to figure out why the SSLContext isn't trusting the self-signed certificate despite you configuring it that way.

I've got charles proxy downloaded now, this will be a great resource when developing for proxies in the future. :) I'll see if I can resolve it locally and then reply back here! Thanks for all the help you're giving here :bowing_man:

sethmlarson picture sethmlarson on 24 Sep 2019

Figured it out! PR will be incoming soon! :)

sethmlarson picture sethmlarson on 24 Sep 2019
❤1 🎉1

@GnubiBORED Can you confirm you're no longer seeing this issue in the latest release?

sethmlarson picture sethmlarson on 27 Sep 2019

Hi @sethmlarson,
I have not exactly the same issue, hower, when I set verify via httpx.Client it works. But when I trying to use api directly (like httpx.get), it seems verify don't propagate to proxies

MikalaiDavydzenka picture MikalaiDavydzenka on 1 Oct 2019

sorry @sethmlarson i been 1 week not joining github, i can confirm issue isnt solved, i have same problem with 0.7.4 i just updated now

pip install httpx --upgrade

check next comment for debug output.

GnubiBORED picture GnubiBORED on 2 Oct 2019 
C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx>set HTTPX_DEBUG=1

C:\Users\localhost\PycharmProjects\TribalW_Test_Env\Basic\tests\httpx>python proxy_test.py
22:50:14.296 - httpx.dispatch.proxy_http - forward_connection proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='http' host='192.168.1.78' port=8888)
22:50:14.296 - httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='http' host='192.168.1.78' port=8888)
22:50:14.298 - httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='http' host='192.168.1.78' port=8888))
22:50:14.298 - httpx.dispatch.connection - start_connect host='192.168.1.78' port=8888 timeout=TimeoutConfig(timeout=5.0)
22:50:14.299 - httpx.dispatch.connection - connected http_version='HTTP/1.1'
22:50:14.300 - httpx.dispatch.http11 - send_headers method='GET' target='http://whoer.net' headers=Headers({'host': 'whoer.net', 'user-agent': 'python-httpx/0.7.4', 'accept': '*/*', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive'})
22:50:14.301 - httpx.dispatch.http11 - receive_event event=NEED_DATA
22:50:14.332 - httpx.dispatch.http11 - receive_event event=Response(status_code=301, headers=[(b'date', b'Wed, 02 Oct 2019 20:50:15 GMT'), (b'transfer-encoding', b'chunked'), (b'cache-control', b'max-age=3600'), (b'expires', b'Wed, 02 Oct 2019 21:50:15 GMT'), (b'location', b'https://whoer.net/'), (b'vary', b'Accept-Encoding'), (b'server', b'cloudflare'), (b'cf-ray', b'51f98c30ed2ed685-MAD'), (b'connection', b'keep-alive')], http_version=b'1.1', reason=b'Moved Permanently')
22:50:14.333 - httpx.dispatch.proxy_http - tunnel_connection proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443)
22:50:14.334 - httpx.dispatch.connection - start_connect host='192.168.1.78' port=8888 timeout=TimeoutConfig(timeout=5.0)
22:50:14.335 - httpx.dispatch.connection - connected http_version='HTTP/1.1'
22:50:14.335 - httpx.dispatch.http11 - send_headers method='CONNECT' target='whoer.net:443' headers=Headers({'host': '192.168.1.78:8888', 'user-agent': 'python-httpx/0.7.4', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive', 'accept': '*/*'})
22:50:14.335 - httpx.dispatch.http11 - receive_event event=NEED_DATA
22:50:14.348 - httpx.dispatch.http11 - receive_event event=Response(status_code=200, headers=[], http_version=b'1.1', reason=b'Connection established')
22:50:14.348 - httpx.dispatch.proxy_http - tunnel_response proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443) response=<Response [200 OK]>
22:50:14.349 - httpx.dispatch.http11 - receive_event event=PAUSED
22:50:14.352 - httpx.config - load_ssl_context verify=True cert=None trust_env=None http_versions=HTTPVersionConfig(['HTTP/1.1'])
22:50:14.353 - httpx.config - load_verify_locations cafile=C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\certifi\cacert.pem
22:50:14.368 - httpx.dispatch.proxy_http - tunnel_start_tls proxy_url=URL('http://192.168.1.78:8888') origin=Origin(scheme='https' host='whoer.net' port=443)
SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000018EAE7FB3C8>
transport: <_SelectorSocketTransport fd=672 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000018EAE7FB3C8>
transport: <_SelectorSocketTransport closing fd=672 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
Traceback (most recent call last):
  File "proxy_test.py", line 11, in <module>
    request = client.get("http://whoer.net",headers=headers,verify=client_cer)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 820, in get
    trust_env=trust_env,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 735, in request
    trust_env=trust_env,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 764, in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 261, in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 255, in _get_response
    return await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 41, in __call__
    return await self(next_request, get_response)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\middleware\redirect.py", line 31, in __call__
    response = await get_response(request)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\client.py", line 216, in get_response
    request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 248, in send
    request=request, verify=verify, cert=cert, timeout=timeout
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\connection_pool.py", line 118, in send
    connection = await self.acquire_connection(origin=request.url.origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 97, in acquire_connection
    return await self.tunnel_connection(origin)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 115, in tunnel_connection
    await self.tunnel_start_tls(origin, connection)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\dispatch\proxy_http.py", line 199, in tunnel_start_tls
    timeout=timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\site-packages\httpx\concurrency\asyncio.py", line 236, in start_tls
    timeout=timeout.connect_timeout,
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 416, in wait_for
    return fut.result()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 1134, in start_tls
    await waiter
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\asyncio\sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:\Users\localhost\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)
GnubiBORED picture GnubiBORED on 2 Oct 2019

I'm facing the same issue after trying out the library, I guess it's back to requests

JosXa picture JosXa on 4 Nov 2019

@JosXa @MikalaiDavydzenka Could either of you provide some more info to help make it easier for us to replicate this issue ourselves, so we can get it resolved? Are you using charesproxy too, or something else?

tomchristie picture tomchristie on 4 Nov 2019

@tomchristie I was under a corporate network where all outgoing connections should go through a proxy. So, I don't know which proxy server was used there, and moreover, there is no way to get this information. Unfortunately, right now I don't have access to the code and network to provide more detailed examples and debugging information. If I get access again (this may happen next month), I'll post additional info here.

MikalaiDavydzenka picture MikalaiDavydzenka on 4 Nov 2019

I had a quick look at this, it seems verify is not passed into the proxy dispatchers on the client.get call but it is on Client.__init__. On initialization the Client defaults to verify=True and the proxies are setup accordingly which causes the self-signed cetificate error.

We can fix that but in the meantime a solution might be:

import httpx
client_cer = r'C:\Users\localhost\Documents\123.pem'
client = httpx.Client(proxies={
    "http": "http://192.168.1.78:8888",
    "https": "http://192.168.1.78:8888",
}, verify=client_cer)

request = client.get("http://whoer.net")

@GnubiBORED can you give that a try?

yeraydiazdiaz picture yeraydiazdiaz on 6 Nov 2019
👍1

Given @yeraydiazdiaz's comment here, and given that issuing verify=... on a per-request basis is now pending deprecation I'm going to close this off.

tomchristie picture tomchristie on 20 Dec 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

florimondmanca picture florimondmanca  路  4Comments
iwoloschin picture iwoloschin  路  4Comments
paulchubatyy picture paulchubatyy  路  4Comments
edocod1 picture edocod1  路  3Comments
StevenBorg picture StevenBorg  路  3Comments