Incubator-superset: Docker-->ModuleNotFoundError: No module named 'gevent'

[hellohappy0]

I have reproduced the issue with at least the latest released version of superset.
I have checked the issue tracker for the same issue and I haven't found one similar.

Expected results

superset starts up without error.

Additional context

Steps to reproduce:

git clone https://github.com/apache/incubator-superset/
cd incubator-superset/contrib/docker
docker-compose run --rm superset ./docker-init.sh
docker-compose up

Actual results

When I run to this step, the display is as follows

docker-compose up

/usr/lib/python2.7/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.22) or chardet (2.2.1) doesn't match a supported version!
RequestsDependencyWarning)
Starting superset_postgres_1 ... done
Starting superset_redis_1 ... done
Recreating superset_superset_1 ... done
Attaching to superset_postgres_1, superset_redis_1, superset_superset_1
postgres_1 | 2019-05-16 04:51:54.489 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2019-05-16 04:51:54.489 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2019-05-16 04:51:54.502 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2019-05-16 04:51:54.519 UTC [21] LOG: database system was shut down at 2019-05-16 04:48:03 UTC
postgres_1 | 2019-05-16 04:51:54.530 UTC [1] LOG: database system is ready to accept connections
redis_1 | 1:C 16 May 04:51:54.449 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | _._
redis_1 | _.-__ ''-._ redis_1 | _.- ._. ''-._ Redis 3.2.12 (00000000/0) 64 bit
redis_1 | .-.-```. ```\/ _.,_ ''-._ redis_1 | ( ' , .-` | `, ) Running in standalone mode redis_1 | |`-._`-...-` __...-.-._|'_.-'| Port: 6379 redis_1 | |-._ ._ / _.-' | PID: 1 redis_1 |-._ -._-./ _.-' _.-'
redis_1 | |-._-._ -.__.-' _.-'_.-'| redis_1 | |-._-._ _.-'_.-' | http://redis.io redis_1 |-._ -._-.__.-'_.-' _.-'
redis_1 | |-._-._ -.__.-' _.-'_.-'| redis_1 | |-._-._ _.-'_.-' | redis_1 |-._ -._-.__.-'_.-' _.-'
redis_1 | -._-.__.-' _.-'
redis_1 | -._ _.-' redis_1 |-.__.-'
redis_1 |
redis_1 | 1:M 16 May 04:51:54.451 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 16 May 04:51:54.451 # Server started, Redis version 3.2.12
redis_1 | 1:M 16 May 04:51:54.451 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 16 May 04:51:54.451 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 16 May 04:51:54.451 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 16 May 04:51:54.451 * The server is now ready to accept connections on port 6379
superset_1 | + '[' 0 -ne 0 ']'
superset_1 | + '[' development = development ']'
superset_1 | + celery worker --app=superset.sql_lab:celery_app --pool=gevent -Ofair
superset_1 | + cd superset/assets/
superset_1 | + npm ci
superset_1 | Traceback (most recent call last):
superset_1 | File "/usr/local/bin/celery", line 10, in
superset_1 | sys.exit(main())
superset_1 | File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
superset_1 | maybe_patch_concurrency()
superset_1 | File "/usr/local/lib/python3.6/site-packages/celery/__init__.py", line 142, in maybe_patch_concurrency
superset_1 | patcher()
superset_1 | File "/usr/local/lib/python3.6/site-packages/celery/__init__.py", line 110, in _patch_gevent
superset_1 | import gevent
superset_1 | ModuleNotFoundError: No module named 'gevent'

Environment

centos7

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [x] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.86. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[hellohappy0]

gevent should be optional though, it's really only used if you happen to use gunicorn in async mode, The reason for the failure is not here. In fact, the YML of this docker is written for Ubuntu. After my test, there is no problem with using ubuntu. I hope someone can write the CentOS adapted docker file for public use in the future. It's closed because it doesn't want to burden developers.

[ali-bahjati]

The problem persists on ubuntu 18.04 too. Celery is clearly using gevent as pool
celery worker --app=superset.sql_lab:celery_app --pool=gevent -Ofair

[ali-bahjati]

You should open this issue again

[gabrielclimb]

I'm having the same problem in Mac OS

[epaolillo]

Same with Ubuntu 18.04!

[directionless]

AFAICT the issue is that the docker build doesn't include gevent, but that the entrypoint starts celery using it.

This should be fixable via any of:

• include gevent in requirements.txt
• explicitly install gevent in the Dockerfile
• Don't start celery using gevent in docker

I'd PR any of that, but I don't know which the recommended path is.

[mistercrunch]

@directionless

1. not ideal since gevent is an optional dep
2. yup, either that is fine by me or adding a requirements-extra.txt in the contrib/docker folder with an entry for gevent
3. nope

So option 2 is best. Happy to review / merge the PR