Incubator-superset: superset db upgrade fails

[lilila]

Make sure these boxes are checked before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if any
• [x] I have reproduced the issue with at least the latest released version of superset

• [x] I have checked the issue tracker for the same issue and I haven't found one similar

  table clusters already exists

  Superset version

23.3

Expected results

Upgrade superset DB with

superset db upgrade

after running

pip install superset --upgrade

Actual results

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table clusters already exists [SQL: 'nCREATE TABLE clusters (ntcreated_on DATETIME NOT NULL, ntchanged_on DATETIME NOT NULL, ntid INTEGER NOT NULL, ntcluster_name VARCHAR(250), ntcoordinator_host VARCHAR(255), ntcoordinator_port INTEGER, ntcoordinator_endpoint VARCHAR(255), ntbroker_host VARCHAR(255), ntbroker_port INTEGER, ntbroker_endpoint VARCHAR(255), ntmetadata_last_refreshed DATETIME, ntcreated_by_fk INTEGER, ntchanged_by_fk INTEGER, ntPRIMARY KEY (id), ntUNIQUE (cluster_name), ntFOREIGN KEY(created_by_fk) REFERENCES ab_user (id), ntFOREIGN KEY(changed_by_fk) REFERENCES ab_user (id)n)nn'] (Background on this error at: http://sqlalche.me/e/e3q8)

Steps to reproduce

pip install superset --upgrade
superset db upgrade

[mistercrunch]

Table clusters already exists is a weird issue since clusters is a table that was added really early on in the project. Have you been issuing alembic related commands? Altering the alembic_version table?

[victorgil88]

I have the same problem.
In my case, the answer is: "superset" is not recognized as a command

[xrmx]

@victorgil88 then it's not the same problem, you probably have installed superset in a virtualenv that you forgot to activate

[victorgil88]

At the end, this worked for me :

pip install superset
cd \venv\Scripts
python superset db upgrade

and the rest is the same but adding "python" as the first word

I am working with windows, for Linux, instead of Scripts is bin.

[prabhatmishra33]

I am also facing the same issue on windows while running python superset db upgrade.
May i know what this command does and do we need to setup tables or some config files.

```INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade bddc498dd179 -> 4451805bbaa1, remove double percents
Traceback (most recent call last):
File "superset", line 15, in
cli()
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 722, in __call__
return self.main(args, kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesflaskcli.py", line 380, in main
return AppGroup.main(self, *args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 697, in main
rv = self.invoke(ctx)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 895, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 535, in invoke
return callback(args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickdecorators.py", line 17, in new_func
return f(get_current_context(), *args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesflaskcli.py", line 257, in decorator
return __ctx.invoke(f, args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesclickcore.py", line 535, in invoke
return callback(args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesflask_migratecli.py", line 134, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesflask_migrate__init__.py", line 95, in wrapped
f(args, *kwargs)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesflask_migrate__init__.py", line 280, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembiccommand.py", line 276, in upgrade
script.run_env()
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembicscriptbase.py", line 475, in run_env
util.load_python_file(self.dir, "env.py")
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembicutilpyfiles.py", line 90, in load_python_file
module = load_module_py(module_id, path)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembicutilcompat.py", line 156, in load_module_py
spec.loader.exec_module(module)
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessuperset/migrationsenv.py", line 100, in
run_migrations_online()
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessuperset/migrationsenv.py", line 93, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembicruntimeenvironment.py", line 839, in run_migrations
self.get_context().run_migrations(kw)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagesalembicruntimemigration.py", line 361, in run_migrations
step.migration_fn(*kw)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessupersetmigrationsversions4451805bbaa1_remove_double_percents.py", line 82, in upgrade
replace('%%', '%')
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessupersetmigrationsversions4451805bbaa1_remove_double_percents.py", line 53, in replace
.join(Table)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessqlalchemyormquery.py", line 2234, in join
from_joinpoint=from_joinpoint,
File "", line 2, in _join
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessqlalchemyormbase.py", line 220, in generate
fn(self, *args[1:], *kw)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessqlalchemyormquery.py", line 2413, in _join
left, right, onclause, prop, create_aliases, outerjoin, full
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessqlalchemyormquery.py", line 2436, in _join_left_to_right
) = self._join_determine_implicit_left_side(left, right, onclause)
File "H:POCAPACHE_SUPERSETvenvlibsite-packagessqlalchemyormquery.py", line 2567, in _join_determine_implicit_left_side
"Can't determine which FROM clause to join "
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.

(venv) (base) H:POCAPACHE_SUPERSETvenvScripts>```

[pammi22]

@prabhatmishra33 I encountered the same issue, downgrading sqlalchemy version to 1.2 fixed this for me. i hope it helps.

[mshiner]

Hey @pammi22 - fixed for me as well! Cheers for the heads up

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[pckhoi]

@lilila very late answer to your original question but hope this will be helpful to those who comes later. Your alembic_version table is probably corrupted. In my case what I did was running fabmanager create-db --app superset initially instead of superset db upgrade, which create the tables but not the alembic_version table. To fix it I figured out the latest migration and ran this in my database:

CREATE TABLE alembic_version (version_num character varying(32) PRIMARY KEY);
INSERT INTO alembic_version (version_num) VALUES ('d6ffdf31bdd4');

You'll have to replace the version_num with your own appropriate value of course.