Requests: Socks5 proxy do not resolves dns through the proxy server

[wavenator]

since requests==2.13.0
Using socks5 proxy seems to resolve domains through the client instead of the proxy server.
downgrading to requests==2.12.0 ... 2.10.0 resolves the problem.

In [1]: import requests

In [2]: requests.get(url='https://3g2upl4pq6kufc4m.onion/', proxies={'https': 'socks5://127.0.0.1:9050'})
---------------------------------------------------------------------------
gaierror                                  Traceback (most recent call last)
/usr/lib/python3.6/site-packages/socks.py in connect(self, dest_pair)
    782                 negotiate = self._proxy_negotiators[proxy_type]
--> 783                 negotiate(self, dest_addr, dest_port)
    784             except socket.error as error:

/usr/lib/python3.6/site-packages/socks.py in _negotiate_SOCKS5(self, *dest_addr)
    447         self.proxy_peername, self.proxy_sockname = self._SOCKS5_request(self,
--> 448             CONNECT, dest_addr)
    449

/usr/lib/python3.6/site-packages/socks.py in _SOCKS5_request(self, conn, cmd, dst)
    510             writer.write(b"\x05" + cmd + b"\x00")
--> 511             resolved = self._write_SOCKS5_address(dst, writer)
    512             writer.flush()

/usr/lib/python3.6/site-packages/socks.py in _write_SOCKS5_address(self, addr, file)
    562             # Resolve locally
--> 563             addresses = socket.getaddrinfo(host, port, socket.AF_UNSPEC, socket.SOCK_STREAM, socket.IPPROTO_TCP, socket.AI_ADDRCONFIG)
    564             # We can't really work out what IP is reachable, so just pick the

/usr/lib/python3.6/socket.py in getaddrinfo(host, port, family, type, proto, flags)
    742     addrlist = []
--> 743     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
    744         af, socktype, proto, canonname, sa = res

gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

GeneralProxyError                         Traceback (most recent call last)
/usr/lib/python3.6/site-packages/requests/packages/urllib3/contrib/socks.py in _new_conn(self)
     87                 timeout=self.timeout,
---> 88                 **extra_kw
     89             )

/usr/lib/python3.6/site-packages/socks.py in create_connection(dest_pair, proxy_type, proxy_addr, proxy_port, proxy_rdns, proxy_username, proxy_password, timeout, source_address, socket_options)
    212     if err:
--> 213         raise err
    214

/usr/lib/python3.6/site-packages/socks.py in create_connection(dest_pair, proxy_type, proxy_addr, proxy_port, proxy_rdns, proxy_username, proxy_password, timeout, source_address, socket_options)
    202
--> 203             sock.connect((remote_host, remote_port))
    204             return sock

/usr/lib/python3.6/site-packages/socks.py in connect(self, dest_pair)
    786                 self.close()
--> 787                 raise GeneralProxyError("Socket error", error)
    788             except ProxyError:

GeneralProxyError: Socket error: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    599                                                   body=body, headers=headers,
--> 600                                                   chunked=chunked)
    601

/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    344         try:
--> 345             self._validate_conn(conn)
    346         except (SocketTimeout, BaseSSLError) as e:

/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    843         if not getattr(conn, 'sock', None):  # AppEngine might not have  `.sock`
--> 844             conn.connect()
    845

/usr/lib/python3.6/site-packages/requests/packages/urllib3/connection.py in connect(self)
    283         # Add certificate verification
--> 284         conn = self._new_conn()
    285

/usr/lib/python3.6/site-packages/requests/packages/urllib3/contrib/socks.py in _new_conn(self)
    109                         self,
--> 110                         "Failed to establish a new connection: %s" % error
    111                     )

NewConnectionError: <requests.packages.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f99daa25978>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
/usr/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    422                     retries=self.max_retries,
--> 423                     timeout=timeout
    424                 )

/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    648             retries = retries.increment(method, url, error=e, _pool=self,
--> 649                                         _stacktrace=sys.exc_info()[2])
    650             retries.sleep()

/usr/lib/python3.6/site-packages/requests/packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    375         if new_retry.is_exhausted():
--> 376             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    377

MaxRetryError: SOCKSHTTPSConnectionPool(host='3g2upl4pq6kufc4m.onion', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f99daa25978>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-2-ad633d784dc6> in <module>()
----> 1 requests.get(url='https://3g2upl4pq6kufc4m.onion/', proxies={'https': 'socks5://127.0.0.1:9050'})

/usr/lib/python3.6/site-packages/requests/api.py in get(url, params, **kwargs)
     68
     69     kwargs.setdefault('allow_redirects', True)
---> 70     return request('get', url, params=params, **kwargs)
     71
     72

/usr/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     54     # cases, and look like a memory leak in others.
     55     with sessions.Session() as session:
---> 56         return session.request(method=method, url=url, **kwargs)
     57
     58

/usr/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    486         }
    487         send_kwargs.update(settings)
--> 488         resp = self.send(prep, **send_kwargs)
    489
    490         return resp

/usr/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    607
    608         # Send the request
--> 609         r = adapter.send(request, **kwargs)
    610
    611         # Total elapsed time of the request (approximately)

/usr/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    485                 raise ProxyError(e, request=request)
    486
--> 487             raise ConnectionError(e, request=request)
    488
    489         except ClosedPoolError as e:

ConnectionError: SOCKSHTTPSConnectionPool(host='3g2upl4pq6kufc4m.onion', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f99daa25978>: Failed to establish a new connection: [Errno -2] Name or service not known',))

after pip install -U requests[socks]==2.12.0

In [1]: import requests

In [2]: requests.get(url='https://3g2upl4pq6kufc4m.onion/', proxies={'https': 'socks5://127.0.0.1:9050'})
Out[2]: <Response [200]>

[Lukasa]

Yes, this change was intentional. This was to bring us in line with curl, which uses different schemes to decide whether to resolve via the proxy or via the client. If you use the scheme socks5h that should be sufficient to return to the prior behaviour.

[wavenator]

I agree, it should be documented

[ghost]

This change is especially dangerous for people relying on SOCKS for anonymity - by implementing it you potentially put those people at risk as any Requests-based tool would now silently leak DNS requests.

I'd recommend reverting to the previous behaviour and creating a different proxy scheme for the new behaviour - socks5nodns?

[sigmavirus24]

@Rjevski we follow standards, we don't make things up. cURL established the standard and we're following it.

[elliott-beach]

@Lukasa Hi, is there a reason this issue is still open when the PR #4229 was merged?

[nateprewitt]

Nope, looks like the documentation concerns have been addressed. Thanks for checking in @e-beach!

[emorell96]

So I understand that this applies for Socks5 protocol. But what if you use an HTTP/HTTPS proxy. Is the DNS resolved in the proxy side or client side? Thanks.