Mastodon: Index name 'index_statuses_20180106' on table 'statuses' already exists

[otih]

Installing mastadon fresh the db init fails.

part of the docker-compose.yml
image: tootsuite/mastodon:latest

Versions

Docker version 1.13.1, build 092cba3
Linux xxx 4.13.0-45-generic #50-Ubuntu SMP Wed May 30 08:23:18 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 17.10

command:
docker-compose run --rm web bundle exec rake db:migrate

output:

...
Caused by:                       
ArgumentError: Index name 'index_statuses_20180106' on table 'statuses' already exists                                              
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:1169:in `add_index_options'
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql/schema_statements.rb:465:in `add_index'
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:871:in `block in method_missing'          
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:840:in `block in say_with_time'           
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:840:in `say_with_time'                    
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:860:in `method_missing'                   
/mastodon/vendor/bundle/ruby/2.4.0/gems/strong_migrations-0.2.2/lib/strong_migrations/migration.rb:75:in `method_missing'           
/mastodon/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb:8:in `block in change'
/mastodon/vendor/bundle/ruby/2.4.0/gems/strong_migrations-0.2.2/lib/strong_migrations/migration.rb:6:in `safety_assured'            
/mastodon/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb:7:in `change'        
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:814:in `exec_migration'                   
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'      
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:797:in `block in migrate'                 
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:796:in `migrate'                          
/mastodon/vendor/bundle/ruby/2.4.0/gems/strong_migrations-0.2.2/lib/strong_migrations/migration.rb:13:in `migrate'                  
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/railtie.rb:37:in `block in migrate'                                   
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/strategy.rb:70:in `wrap'                                              
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy.rb:201:in `strategy'                                                  
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/railtie.rb:37:in `migrate'                                            
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:977:in `migrate'                          
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1345:in `ddl_transaction'                 
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'   
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1262:in `each'                            
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1262:in `migrate_without_lock'            
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1210:in `block in migrate'                
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1363:in `with_advisory_lock'              
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1210:in `migrate'                         
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/railtie.rb:37:in `block in migrate'                                   
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/strategy.rb:70:in `wrap'                                              
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy.rb:201:in `strategy'                                                  
/mastodon/vendor/bundle/ruby/2.4.0/gems/chewy-5.0.0/lib/chewy/railtie.rb:37:in `migrate'                                            
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1036:in `up'                              
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/migration.rb:1011:in `migrate'                         
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/tasks/database_tasks.rb:172:in `migrate'               
/mastodon/vendor/bundle/ruby/2.4.0/gems/strong_migrations-0.2.2/lib/strong_migrations/database_tasks.rb:4:in `migrate'              
/mastodon/vendor/bundle/ruby/2.4.0/gems/activerecord-5.2.0/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/mastodon/vendor/bundle/ruby/2.4.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'                                               
/mastodon/bin/bundle:3:in `load' 
/mastodon/bin/bundle:3:in `<main>'                                
Tasks: TOP => db:migrate         
(See full trace by running task with --trace)

[otih]

as mentioned within #8000 db:migrate works with v2.4.2

[ProgVal]

I have the same issue here, without docker. Going back to v2.4.2 does not fix db:migrate.

[ProgVal]

Here is the problematic migration:

Migrating to RevertIndexChangeOnStatusesForApiV1AccountsAccountIdStatuses (20180514140000)
== 20180514140000 RevertIndexChangeOnStatusesForApiV1AccountsAccountIdStatuses: migrating 
-- index_exists?(:statuses, {:name=>"index_statuses_20180106"})
   -> 0.0058s
-- add_index(:statuses, [:account_id, :id, :visibility, :updated_at], {:order=>{:id=>:desc}, :algorithm=>:concurrently, :name=>:index_statuses_20180106})
rails aborted!
StandardError: An error has occurred, all later migrations canceled:

Index name 'index_statuses_20180106' on table 'statuses' already exists

[ProgVal]

Workaround:

diff --git a/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb b/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb
index 4a8761fec..a6c908f20 100644
--- a/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb
+++ b/db/migrate/20180514140000_revert_index_change_on_statuses_for_api_v1_accounts_account_id_statuses.rb
@@ -5,7 +5,7 @@ class RevertIndexChangeOnStatusesForApiV1AccountsAccountIdStatuses < ActiveRecor

   def change
     safety_assured do
-      add_index :statuses, [:account_id, :id, :visibility, :updated_at], order: { id: :desc }, algorithm: :concurrently, name: :index_statuses_20180106 unless index_exists?(:statuses, name: "index_statuses_20180106")
+      #add_index :statuses, [:account_id, :id, :visibility, :updated_at], order: { id: :desc }, algorithm: :concurrently, name: :index_statuses_20180106 unless index_exists?(:statuses, name: "index_statuses_20180106")
     end

     # These index may not exists (see migration 20180514130000)

[Gargron]

@akihikodaki Any idea why that line of code fails?

[nolanlawson]

Cherry-picking the commit from #8026 appears to fix this for me (migrating from 2.3.3 to 2.4.3).

git cherry-pick 7a686082370ad6d1c7a7d0ad331c22bf3e1fbede

[rawkode]

@Gargron This should have been picked into 2.4.4, as right now nobody can upgrade?

[JohnTitor]

I cannot migrate the database (on v2.4.4).
Also, I do not know if that is the reason, but when I try to register my user, I fail the registration like #8000 .

[mbilokonsky]

This is the second time I've tried to update my instance and had the whole thing fail due to this issue. Why is this happening?

How do you launch multiple production versions with a broken migration that leaves admins unable to start their instances? What the hell? I know I'm not the only one having this problem. And once the problem has been triggered the DB is in a bad state and I have to restore from a backup. I get that this is FOSS and no guarantees and etc but how am I supposed to use this tool if I can't trust the code?

[nolanlawson]

For the record, there are three commits you need to cherry-pick, at least to upgrade from 2.3.3 to 2.4.3 or 2.4.4: https://github.com/tootsuite/mastodon/issues/8007#issuecomment-413388963

And yes, if you don't do this, then your database will be in a weird state (_update: maybe not, see Gargron's comment_) and you'll need to restore from a backup. I agree this is a less-than-ideal situation.

@gargron Can we get a 2.4.5 release with those commits cherry-picked? It may prevent some admin from botching their instance.

[Gargron]

@mbilokonsky

• No the database is not in a broken state. Failing migrations do nothing because everything in PostgreSQL is in transactions that can be rolled back on failure. This error means "the migration couldn't be run" not "the migration has ruined your database"
• This particular issue arose from one of the contributors modifying a migration after its initial release, which meant that most people upgraded when it was working fine; now again the issue is fixed in up-to-date code but it has not been backported into 2.4.4

Do git cherry-pick 7a686082370ad6d1c7a7d0ad331c22bf3e1fbede and then run db:migrate again and it should go through fine this time.

[nolanlawson]

Sorry for spreading FUD, my bad. When I had a failed upgrade on malfunctioning.technology I couldn't figure out how to resolve it except to restore from backup.

[rawkode]

I'm concerned that 2.4.4 was released without the fix, and that 2.4.5 still doesn't exist.

For those of us that deploy with Docker, cherrying picking isn't an option. You've now got 2 tagged Docker releases that do not work.

[mbilokonsky]

So, respectfully, the first time I ran this, after this migration failed my database was in a state where even rolling back the git commit gave me db errors. I even tried, then, to cherry-pick this commit and rerun. It continued to fail. So I tried to manually make the same change against an older git version, and it still failed.

It's possible I guess that I did something else wonky, but I ultimately had to restore from a backup.

I know mistakes happen and running a huge project like this is challenging. But a lot of us who run instances rely on the tagged releases working as-is, because we don't necessarily have the time to spend scouring documentation and issues and trying to figure out what to do. It's frustrating that a release went out without these issues being taken into account. It would be really nice if you could just merge the cherry-picked commits that you and @nolanlawson are referring to and launch it as 2.4.5.

[mbilokonsky]

(and to be clear I'm deploying via docker, using pre-built images, if that makes a difference here.)

[Gargron]

@rawkode Pushed out 2.4.5: https://github.com/tootsuite/mastodon/releases/tag/v2.4.5

[mbilokonsky]

Thanks so much, Eugen!

On Fri, Aug 24, 2018 at 3:37 PM Eugen Rochko [e-mail]
wrote:

@rawkode https://github.com/rawkode Pushed out 2.4.5:
https://github.com/tootsuite/mastodon/releases/tag/v2.4.5

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tootsuite/mastodon/issues/8001#issuecomment-415861661,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAk1hVcd7ccz2oxx8WbL1ypDi6TonALhks5uUFYAgaJpZM4VLtSu
.

[rawkode]

Thank you, @Gargron. Appreciate the quick turnaround :+1: