Gunicorn: Gunicorn 20.0.0 raises RecursionError

[harpener]

Hi guys,

I'm having a problem with the latest Gunicorn version 20.0.0 (eventlet workers) that does not occur with previous 19.9.0 version.

I'm using Python 3.7.3, eventlet 0.25.1 and requests 2.22.0.

When I download something using requests library, RecursionError: maximum recursion depth exceeded is raised from Python's ssl module.

Traceback (most recent call last):
  File "/test.py", line 30, in proxy
    requests.get("https://www.google.com")
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 370, in connect
    cert_reqs=resolve_cert_reqs(self.cert_reqs),
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 273, in create_urllib3_context
    context.options |= options
  File "/usr/local/lib/python3.7/ssl.py", line 507, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.7/ssl.py", line 507, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.7/ssl.py", line 507, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 472 more times]
RecursionError: maximum recursion depth exceeded

It's suspicious to me that the issue origins elsewhere, but since updating the gunicorn caused the issue for me, I wanted to let you know and ask if you can reproduce it and help me. :-)

Thanks

[jamadden]

There's an eventlet issue about this and a related gevent issue. It probably depends on when the eventlet monkey-patches are getting applied. If they're applied too late, you may get this problem.

Are you preloading your app? If so, try disabling preload and see if that solves the problem.

(Possibly related: https://github.com/benoitc/gunicorn/issues/1559)

[jamadden]

For reference, gunicorn 19.8 and 19.9 did not actually monkey-patch when using eventlet. See #1847 and #1884

[harpener]

Ah, okay. My app doesn't monkey patch either, it was working without it okay. I will investigate on my own now, thanks. :-)

[gangm]

i have the same problem, any suggestion?
my app.py:
from flask import Flask
import requests
import time
app = Flask(__name__)

@app.route('/')
def index():
print("in")
print(requests.get('https://www.baidu.com'))
return 'hello world'

boot cmd:
gunicorn --preload -k eventlet -w 1 app:app --log-level=debug

curl localhost:8000
err msg:
File "/home/code/api/app.py", line 9, in index
print(requests.get('https://www.baidu.com'))
File "/usr/lib/python3.7/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, *kwargs)
File "/usr/lib/python3.7/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, *kwargs)
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, *send_kwargs)
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 596, in send
r = adapter.send(request, *kwargs)
File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 423, in send
timeout=timeout
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/connectionpool.py", line 595, in urlopen
chunked=chunked)
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/connectionpool.py", line 352, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/connectionpool.py", line 831, in _validate_conn
conn.connect()
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/util/ssl_.py", line 291, in ssl_wrap_socket
ciphers=ciphers)
File "/usr/lib/python3.7/site-packages/requests/packages/urllib3/util/ssl_.py", line 254, in create_urllib3_context
context.options |= options
File "/usr/lib/python3.7/ssl.py", line 518, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/lib/python3.7/ssl.py", line 518, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/lib/python3.7/ssl.py", line 518, in options
super(SSLContext, SSLContext).options.__set__(self, value)
[Previous line repeated 474 more times]

[harpener]

@gangm I suppose you're not patching the standard library with eventlet either. But eventlet has a bug with Python 3.7's ssl library: https://github.com/eventlet/eventlet/issues/526
I haven't had time to investigate, but I'm considering trying either gevent or sync workers.