Incubator-superset: gunicorn==19.8.1 does not work on Python 3.7

[kinghuang]

PR #8309 recently declared Python 3.7 as supported. But, superset has a stated install_requires of gunicorn<19.9.0 (fixed at gunicorn==19.8.1 in requirements.txt).

gunicorn<19.9.0 has a module named gunicorn.workers.async that is not valid in Python 3.7 (benoitc/gunicorn#1795). This was fixed in gunicorn 19.9.0. Superset should be updated to at least gunicorn 19.9.0 for Python 3.7 compatibility.

Expected results

apache-superset 0.34.0 should run under Python 3.7.

Actual results

Running superset through gunicorn produces an error.

/ # gunicorn \
>       -w 10 \
>       -k gevent \
>       --timeout 120 \
>       -b  0.0.0.0:6666 \
>       --limit-request-line 0 \
>       --limit-request-field_size 0 \
>       --statsd-host localhost:8125 \
>       superset:app

Error: class uri 'gevent' invalid or not found: 

[Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 135, in load_class
    mod = import_module('.'.join(components))
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  File "<frozen importlib._bootstrap_external>", line 860, in get_code
  File "<frozen importlib._bootstrap_external>", line 791, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 30
    from gunicorn.workers.async import AsyncWorker
                              ^
SyntaxError: invalid syntax
]

Screenshots

N/A

How to reproduce the bug

1. On Python 3.7, use pip to install apache-superset==0.34.0
2. Use gunicorn to run superset as described on A proper WSGI HTTP Server.

Environment

(please complete the following information):

• superset version: Superset 0.34.0
• python version: Python 3.7.4
• node.js version: N/A
• npm version: N/A

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [x] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.93. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[villebro]

Uhh, thanks for pointing this out. Shows that dev envs tend to not be very representative of prod. Gunicorn 19.9.0 has a serious regression that makes it not work for SSL terminated LBs. There has been a github issue collecting cherries for a 19.9.1 since April ( https://github.com/benoitc/gunicorn/issues/2022 ), but unfortunately that release is still pending. However, there seems to be renewed momentum ( https://github.com/benoitc/gunicorn/pull/2114 ), so perhaps this will be resolved soon. If not, I propose deprecating gunicorn and moving on to some other framework.

[mulyadi]

Thanks @kinghuang and @villebro. I am facing the same issue too. Is there a work around? How do I force Superset to compile or run with older Gunicorn? Thank you.

[kinghuang]

@mulyadi I'm running it under Python 3.6.

[mistercrunch]

We recommend 3.6 as this is what we use for unit/integration tests and production

[tilgovi]

We'll work on getting 19.9.1 out ASAP.

@villebro which is the regression around SSL terminated LBs? I'd like to get that resolved, too.

[villebro]

Thanks for checking in @tilgovi! It is my understanding that the cherry that fixes the issue is https://github.com/benoitc/gunicorn/commit/e179dc25759405159e0bedbf6a0780eff2e5f043 . I haven't tested it myself, but if you check my comment in https://github.com/benoitc/gunicorn/issues/2022, running curl -v -H "X-Forwarded-Proto: https" http://url should return https:// if the regression is fixed (currently 19.9.0 returns http://). Please let me know if you need my help testing the 19.9.1 RC.

[tilgovi]

@villebro you can try out the branch at benoitc/gunicorn#2114 if you want to provide early feedback. It includes the referenced commit.

[villebro]

Sure thing @tilgovi , I'll post my results on the PR thread.