Incubator-superset: Pydruid requirements incorrect in setup.py

[EvaSDK]

Following issue #8425,

I am trying to setup a Docker image based on setup.py to select some extra dependencies.

Expected results

Installing superset via pip install -e .[druid] does provides a working superset installation with Druid support.

Actual results

Druid is installed but at version 0.5.2 not version 0.5.6 which is listed in /requirements-dev.txt/

Screenshots

Loaded your LOCAL configuration at [/home/superset/superset/superset_config.py]
[2019-10-22 13:49:07 +0000] [28] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/home/superset/superset/__init__.py", line 241, in <module>
    ConnectorRegistry.register_sources(module_datasource_map)
  File "/home/superset/superset/connectors/connector_registry.py", line 37, in register_sources
    module_obj = __import__(module_name, fromlist=class_names)
  File "/home/superset/superset/connectors/druid/__init__.py", line 17, in <module>
    from . import models  # noqa
  File "/home/superset/superset/connectors/druid/models.py", line 462, in <module>
    class DruidDatasource(Model, BaseDatasource):
  File "/home/superset/superset/connectors/druid/models.py", line 832, in DruidDatasource
    def get_post_agg(mconf: Dict) -> Postaggregator:
NameError: name 'Postaggregator' is not defined
[2019-10-22 13:49:07 +0000] [28] [INFO] Worker exiting (pid: 28)

How to reproduce the bug

1. Build a Docker image using:

RUN pip install --no-cache-dir \
    -r requirements.txt \
    celery[redis,gevent] \
    statsd==3.3.0 \
    -e .[druid,hive,postgres]

instead of the current line in repository.

1. Try to run the image
2. See error with docker logs

Environment

• superset version: Image built from git commit 03b35b3
• python version: Python 3.6.9
• node.js version: v10.16.3
• npm version: 6.9.0

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [x] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.88. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[willbarrett]

Hey @EvaSDK - I just tried to recreate this bug. For me pydruid is installed at 0.5.7, which is correct. Can you share the full Dockerfile (including the base image reference) and your requirements.txt file?

[dpgaspar]

Hi @EvaSDK,

Thank you for reporting this. The bug is fixed on master and will be included on superset's next release

[metaperl]

The bug is fixed on master and will be included on superset's next release

But it is only listed as a requirement in requirements-dev not requirements.txt

flask fab create-permissions fails with NameError: name 'Postaggregator' is not defined if you simply do pip install apache-superset.