Incubator-superset: Async queries failing in celery workers

[robdiciuccio]

When running a query in SQL Lab against a database with async queries enabled, the following error is raised by the celery worker:

Task sql_lab.get_sql_results[051acb0d-30a5-4dbe-b30e-5b16bc9d8545] raised unexpected: RuntimeError('Working outside of application context.\n\nThis typically means that you attempted to use functionality that needed\nto interface with the current application object in some way. To solve\nthis, set up an application context with app.app_context().  See the\ndocumentation for more information.',)
Traceback (most recent call last):
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/celery/app/trace.py", line 648, in __protected_call__
    return self.run(*args, **kwargs)
  File "/Users/rob/work/incubator-superset/superset/sql_lab.py", line 154, in get_sql_results
    with session_scope(not ctask.request.called_directly) as session:
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/contextlib2.py", line 79, in __enter__
    return next(self.gen)
  File "/Users/rob/work/incubator-superset/superset/sql_lab.py", line 117, in session_scope
    app.config["SQLALCHEMY_DATABASE_URI"], poolclass=NullPool
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/werkzeug/local.py", line 348, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/werkzeug/local.py", line 307, in _get_current_object
    return self.__local()
  File "/Users/rob/work/incubator-superset/venv/lib/python3.6/site-packages/flask/globals.py", line 52, in _find_app
    raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context().  See the
documentation for more information.

Expected results

Celery worker should run the query and store the results in the RESULTS_BACKEND.

Actual results

Exception raised (see above).

Screenshots

If applicable, add screenshots to help explain your problem.

How to reproduce the bug

1. Go to SQL Lab
2. Run a query against a database with Asynchronous Query Operation enabled
3. Note that the UI hangs and a result is never returned
4. Celery worker raises the above error

Environment

(please complete the following information):

• superset version: master (acf0753504095a5c0e320b049370aaf6f04e5a4f)
• python version: 3.6.8
• node.js version: v12.4.0
• npm version: 6.9.0

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [x] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Per @craig-rueda, may be related to application context changes in https://github.com/apache/incubator-superset/pull/8418

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.94. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[tcopple]

I received the same error when starting up a new environment in docker using docker-compose up. Fresh repo checkout and fresh container setup.

[t-mart]

@tcopple, I also received this error using the docker-compose up command from the installation manual page, but after leaving the command running for a few minutes of no more logs being written, the application started up.

I'm just starting with Superset, so I'm not sure if there's some deeper problem that will affect me later, but it seems ok from here.

At the very least, this is a crummy startup experience.

[craig-rueda]

Sorry to hear that, @tcopple. As soon as this is merged, you should be good to go. We are also working on some more solid docker images that will help stuff like this along. As a work around, try pulling the latest 0.35.x tag.

[tcopple]

Yea, I pulled a previous tag and things were fine. Thanks for the quick response.