Sentry: Upgrade to 7.1.1 fails

[gsakkis]

I'm getting the following error when trying to upgrade to 7.1.1:

 > sentry:0135_auto__chg_field_project_team
FATAL ERROR - The following SQL query failed: ALTER TABLE "sentry_project" DROP CONSTRAINT "team_id_refs_id_78750968"
The error was: cannot ALTER TABLE "sentry_project" because it has pending trigger events

Error in migration: sentry:0135_auto__chg_field_project_team
The following content types are stale and need to be deleted:

    sentry | projectcountbyminute
    sentry | groupcountbyminute

Any objects related to these content types by a foreign key will also
be deleted. Are you sure you want to delete these content types?
If you're unsure, answer 'no'.

    Type 'yes' to continue, or 'no' to cancel: no
Traceback (most recent call last):
  File "/opt/sentry/bin/sentry", line 9, in <module>
    load_entry_point('sentry==7.1.1', 'console_scripts', 'sentry')()
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry/utils/runner.py", line 396, in main
    initializer=initialize_app,
  File "/opt/sentry/local/lib/python2.7/site-packages/logan-0.5.10-py2.7.egg/logan/runner.py", line 169, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry/management/commands/upgrade.py", line 34, in handle
    verbosity=options['verbosity'],
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/management/commands/syncdb.py", line 103, in handle_noargs
    management.call_command('migrate', **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/opt/sentry/local/lib/python2.7/site-packages/raven/contrib/django/management/__init__.py", line 37, in new_execute
    return original_func(self, *args, **kwargs)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
    ignore_ghosts = ignore_ghosts,
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 256, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 331, in migrate_many
    result = self.migrate(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
    return self.run_migration(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration
    migration_function()
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
    return (lambda: direction(orm))
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry/migrations/0135_auto__chg_field_project_team.py", line 60, in forwards
    db.alter_column('sentry_project', 'team_id', self.gf('sentry.db.models.fields.FlexibleForeignKey')(to=orm['sentry.Team']))
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear
    return func(self, table, *args, **opts)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 500, in alter_column
    self.delete_foreign_key(table_name, name)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear
    return func(self, table, *args, **opts)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 798, in delete_foreign_key
    "constraint": self.quote_name(constraint_name),
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute
    cursor.execute(sql, params)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: cannot ALTER TABLE "sentry_project" because it has pending trigger events

Answering 'yes' instead of no has no effect in the error.

[pzinovkin]

Same for me.

[dcramer]

If you 'sentry upgrade' again do you hit the same issue?

Also can you confirm database and version?

[gsakkis]

Yes, rerunning sentry upgrade it hits the same issue. PostgreSQL 9.3.5

Btw I worked around it by commenting out the offending migration and running it manually on psql after the rest migrations ran.

[LeonB]

Same issue.

Commenting out db.alter_column('sentry_project', 'team_id', self.gf('sentry.db.models.fields.FlexibleForeignKey')(to=orm['sentry.Team'])) in migrations/0135_auto__chg_field_project_team.py "fixed" the issue but now I'm not use what part of the migration I'm missing...

[meverone]

This should solve the problem (But this isn't the best solution!)

if not db.dry_run:
    try:
        db.start_transaction()
        self.fix_missing_teams(orm)
        db.commit_transaction()
    except Exception as e:
        import traceback; traceback.print_exc()
        raise

more info in doc: http://south.readthedocs.org/en/latest/migrationstructure.html#transactions

[dcramer]

This isn't going to be great. It's sort of an issue with how we create the default project/team, and how we attempt to fix an existing one.

I've pushed b18eb83821f3a1461358922c44bf56733ec09304 and 1925bf46ab35d72562334d98386cebc6810b5fba which should help address this.

[dcramer]

I've released 7.1.2 which includes the above changes

[dcramer]

I'm not convinced this is resolved for existing installations yet. I'm going to dig into the transaction stuff (per @meverone's example), but I need to get a reproducible local environment setup first.

As a quick fix you can:

update sentry_project set team_id = (select id from sentry_team limit 1) where team_id is null;

[camilonova]

I just have installed 7.1.4 and the issue still exists.

Running the quick fix solves the issue.

[camilonova]

@dcramer how do you alter the schemas?

# You should not change this setting after your database has been created
# unless you have altered all schemas first
SENTRY_USE_BIG_INTS = True

I can take down sentry and run the required SQL but I can't find it.

[dcramer]

@camilonova what we did was dump the database schema => run a regexp on the file to change all (id|*_id) columns to BIGINT => load the database schema => replicate to new database

So we did it with minimal downtime, but you could do the above and just instead of replicate have a "load the data" step.

[camilonova]

Thanks, will try it.

[dmr]

"update sentry_project set team_id = (select id from sentry_team limit 1) where team_id is null;" fixed the problem for me, thanks!

[favadi]

This error is still present with sentry==7.4.1 when upgrade from sentry==6.4.4. I wonder if there is a proper fix yet?

[georgewhewell]

I also got this error- @meverone's fix worked. Perhaps this could be merged.

[WoLpH]

I can confirm that @dcramer his sql query fixed it for me. After that I could upgrade without issues

[WoLpH]

As a reference for future people, the following query should fix it:

update sentry_project set team_id = (select id from sentry_team limit 1) where team_id is null;

[dcramer]

If anyone sees this in 8 we'll re-open this, but I know we made some changes to improve various things.

[sfermigier]

Did happen while migrating from 6 to 8.

Fixed using update sentry_project set team_id = (select id from sentry_team limit 1) where team_id is null;. Thanks @WoLpH