Incubator-superset: PyDruid is not actually optional

[EvaSDK]

When creating a Docker image from setup.py, pydruid does not end up being installed.
This is legitimate as it is an optional dependency as stated in setup.py but also the code in superset/connectors/druid/models.py seems to be making the effort to allow the interpreter to load the module even if druid is not installed.

Expected results

Apache Superset can be installed without PyDruid installed as hinted by setup.py

Actual results

Apache Superset fails to start.

Screenshots

Truncated superset entrypoint logs:

2019-10-22 12:18:58,192:DEBUG:asyncio:Using selector: EpollSelector
2019-10-22 12:18:58,212:DEBUG:asyncio:Using selector: SelectSelector
2019-10-22 12:18:58,341:DEBUG:asyncio:Using selector: EpollSelector
2019-10-22 12:18:58,395:DEBUG:asyncio:Using selector: EpollSelector
[2019-10-22 12:18:59 +0000] [32] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/home/superset/superset/__init__.py", line 241, in <module>
    ConnectorRegistry.register_sources(module_datasource_map)
  File "/home/superset/superset/connectors/connector_registry.py", line 37, in register_sources
    module_obj = __import__(module_name, fromlist=class_names)
  File "/home/superset/superset/connectors/druid/__init__.py", line 17, in <module>
    from . import models  # noqa
  File "/home/superset/superset/connectors/druid/models.py", line 121, in <module>
    class DruidCluster(Model, AuditMixinNullable, ImportMixin):
  File "/home/superset/superset/connectors/druid/models.py", line 173, in DruidCluster
    def get_pydruid_client(self) -> PyDruid:
NameError: name 'PyDruid' is not defined
[2019-10-22 12:18:59 +0000] [32] [INFO] Worker exiting (pid: 32)

How to reproduce the bug

1. Build a Docker image using:

RUN pip install --no-cache-dir \
    -r requirements.txt \
    celery[redis,gevent] \
    statsd==3.3.0 \
    -e .[hive,postgres]

instead of the current line in repository.

1. Try to run the image
2. See error with docker logs

Environment

• superset version: Image built from git commit 03b35b3c1121bb452a7938ed92c6bede657377dd
• python version: Python 3.6.9
• node.js version: v10.16.3
• npm version: 6.9.0

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [x] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.87. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[dpgaspar]

Hi @EvaSDK,

Thank you for reporting this!

This is already fixed on master and should be included on superset's next release 0.35.1