Incubator-superset: Install fails on superset db upgrade step on centos

[larshelge]

Overview

When installing superset with pip, the superset db upgrade step fails.

Superset version

0.28.1

Expected results

No output.

Actual results

(venv) [centos@ip-172-16-1-141 venv]$ superset db upgrade
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade bddc498dd179 -> 4451805bbaa1, remove double percents
Traceback (most recent call last):
  File "/home/centos/venv/bin/superset", line 15, in <module>
    cli()
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/flask/cli.py", line 380, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/flask/cli.py", line 257, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/flask_migrate/cli.py", line 134, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/home/centos/venv/lib64/python3.6/site-packages/flask_migrate/__init__.py", line 95, in wrapped
    f(*args, **kwargs)
  File "/home/centos/venv/lib64/python3.6/site-packages/flask_migrate/__init__.py", line 280, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/command.py", line 276, in upgrade
    script.run_env()
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/script/base.py", line 475, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/util/pyfiles.py", line 90, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/util/compat.py", line 156, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/centos/venv/lib64/python3.6/site-packages/superset/migrations/env.py", line 100, in <module>
    run_migrations_online()
  File "/home/centos/venv/lib64/python3.6/site-packages/superset/migrations/env.py", line 93, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/runtime/environment.py", line 839, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/centos/venv/lib64/python3.6/site-packages/alembic/runtime/migration.py", line 361, in run_migrations
    step.migration_fn(**kw)
  File "/home/centos/venv/lib64/python3.6/site-packages/superset/migrations/versions/4451805bbaa1_remove_double_percents.py", line 82, in upgrade
    replace('%%', '%')
  File "/home/centos/venv/lib64/python3.6/site-packages/superset/migrations/versions/4451805bbaa1_remove_double_percents.py", line 53, in replace
    .join(Table)
  File "/home/centos/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2234, in join
    from_joinpoint=from_joinpoint,
  File "<string>", line 2, in _join
  File "/home/centos/venv/lib64/python3.6/site-packages/sqlalchemy/orm/base.py", line 220, in generate
    fn(self, *args[1:], **kw)
  File "/home/centos/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2413, in _join
    left, right, onclause, prop, create_aliases, outerjoin, full
  File "/home/centos/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2436, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
  File "/home/centos/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2567, in _join_determine_implicit_left_side
    "Can't determine which FROM clause to join "
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.

Steps to reproduce

sudo pip install virtualenv

python3.6 -m venv venv
. venv/bin/activate

sudo pip install --upgrade setuptools pip

# Install superset
pip install superset

# Fix version compatibility problem
pip uninstall pandas
pip install pandas==0.23.4

# Continune install superset
fabmanager create-admin --app superset
superset db upgrade

At the db upgrade step, the install fails with an SQL related exception.

Environment

Python was installed with yum and IUS.

Centos 7 (centos-release-7-6.1810.2.el7.centos.x86_64)
Flask 0.12.4
Python 3.6.7 (default, Dec  5 2018, 15:02:05) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
pip 19.0.3
Superset 0.28.1

[nanshouyong]

i got the same error.

[pawan92328]

Got the same error , this solved it https://github.com/apache/incubator-superset/issues/6977

[larshelge]

Thanks @pawan92328 for the comment, that worked for me too. For convenience I am posting the steps for the workaround below.

# Install superset
pip install superset

# Fix pandas version compatibility problem
pip uninstall pandas
pip install pandas==0.23.4

# Fix sqlalchemy compatibility problem
pip uninstall sqlalchemy
pip install sqlalchemy==1.2.18

# Continune install superset
fabmanager create-admin --app superset
superset db upgrade

This fix was done in the following environment:

Python 3.6.7
pip 19.0.3
Superset 0.28.1

I believe the dependencies should be made more explicit in the superset install then as there seems to be compatibility issues with the mentioned pandas and sqlalchemy libraries.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[emthegem69]

Emma Guard has sent you an email via Gmail confidential mode:Re: [apache/incubator-superset] Install fails on superset db upgrade step on centos (#6988)This message was sent on May 23, 2019 at 11:56:25 AM PDTYou can open it by clicking the link below. This link will only work for [email protected] the emailGmail confidential mode gives you more control over the messages you send. Set an expiration time, disable printing or forwarding of a message and more. Learn moreGmail: Email by GoogleUse is subject to the Google Privacy PolicyGoogle LLC, 1600 Amphitheatre Parkway, Mountain View, CA 94043, USAYou have received this message because someone sent you an email via Gmail confidential mode.

[rangerzhou]

rm -rf ~/.superset