Incubator-superset: Potential Circular Reference in 0.36.0

Created on 20 Apr 2020  路  3Comments  路  Source: apache/incubator-superset

models/core.py and db_engine_specs/base.py both try to import each-other.

Expected results

This is happening when startingcelery which loads sql_lab.py. This used to work with 0.35.2.

Logs

...
 File "/home/superset/venv/lib/python3.7/site-packages/superset/sql_lab.py", line 43, in <module>
    from superset.db_engine_specs import BaseEngineSpec
  File "/home/superset/venv/lib/python3.7/site-packages/superset/db_engine_specs/__init__.py", line 36, in <module>
    from superset.db_engine_specs.base import BaseEngineSpec
  File "/home/superset/venv/lib/python3.7/site-packages/superset/db_engine_specs/base.py", line 54, in <module>
    from superset.models.sql_lab import Query
  File "/home/superset/venv/lib/python3.7/site-packages/superset/models/__init__.py", line 17, in <module>
    from . import core, datasource_access_request, schedules, sql_lab, user_attributes
  File "/home/superset/venv/lib/python3.7/site-packages/superset/models/core.py", line 55, in <module>
    from superset.db_engine_specs.base import TimeGrain
ImportError: cannot import name 'TimeGrain' from 'superset.db_engine_specs.base' (/home/superset/venv/lib/python3.7/site-packages/superset/db_engine_specs/base.py)

How to reproduce the bug

Run celery flower --app=superset.sql_lab:celery_app

Environment

(please complete the following information):

  • superset version: 0.36.0
  • python version: 3.7

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

  • [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • [x] I have reproduced the issue with at least the latest released version of superset.
  • [x] I have checked the issue tracker for the same issue and I haven't found one similar.
answered docker

Most helpful comment

Use the following command instead:
celery worker --app=superset.tasks.celery_app:app -Ofair

Use the config to define what get's imported by celery, example:

class CeleryConfig(object):
    BROKER_URL = "redis://localhost:6379/0"
    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
    CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
    CELERYD_PREFETCH_MULTIPLIER = 10
    CELERY_ACKS_LATE = True

All 3 comments

Use the following command instead:
celery worker --app=superset.tasks.celery_app:app -Ofair

Use the config to define what get's imported by celery, example:

class CeleryConfig(object):
    BROKER_URL = "redis://localhost:6379/0"
    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
    CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
    CELERYD_PREFETCH_MULTIPLIER = 10
    CELERY_ACKS_LATE = True

Thanks, changing the celery app to superset.tasks.celery_app:app worked.
I was using a command line that was defined in 0.30 and was not updated since...

Was this page helpful?
0 / 5 - 0 ratings