Openfoodnetwork: Database migrations to v2 fail in one go

[mkllnk]

Description

I tried to run the database migrations on French production data and it failed. Just re-running it works. The issue is some cached knowledge within ActiveRecord.

Expected Behavior

Migrations run without errors.

Actual Behaviour

One migration fails:

==  AddUniquenessOfOrderIdToSpreeShipments: migrating =========================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

missing attribute: lock_version
/home/maikel/code/openfoodnetwork/db/migrate/20180910155506_add_uniqueness_of_order_id_to_spree_shipments.rb:26:in `destroy_all_but_latest_shipments'
/home/maikel/code/openfoodnetwork/db/migrate/20180910155506_add_uniqueness_of_order_id_to_spree_shipments.rb:4:in `change'
/home/maikel/.rbenv/versions/2.1.5/bin/bundle:23:in `load'
/home/maikel/.rbenv/versions/2.1.5/bin/bundle:23:in `<main>'

Caused by:
ActiveModel::MissingAttributeError: missing attribute: lock_version
/home/maikel/code/openfoodnetwork/db/migrate/20180910155506_add_uniqueness_of_order_id_to_spree_shipments.rb:26:in `destroy_all_but_latest_shipments'
/home/maikel/code/openfoodnetwork/db/migrate/20180910155506_add_uniqueness_of_order_id_to_spree_shipments.rb:4:in `change'
/home/maikel/.rbenv/versions/2.1.5/bin/bundle:23:in `load'
/home/maikel/.rbenv/versions/2.1.5/bin/bundle:23:in `<main>'

Steps to Reproduce

1. Copy the French production database. There may be a better way but I haven't investigated that yet. I tried to load sample data and created an order but the migrations worked on that data.
2. git checkout master
3. rake db:migrate

Context

I was working on #3978.

Severity

bug-s3: a feature is broken but there is a workaround

The workaround is to run the migration twice. But we don't want to deal with a workaround like this when deploying v2. It extends the downtime and could be confusing in case there are other errors. And the admin will panic at first.

Your Environment

• Version used: migrating from v1.31.0 to master (723499332a4c43011bbddab1d3fc6bc75be57ba0)

Possible Fix

[Matt-Yorkley]

And the admin will panic

Yeah, lets try to avoid that :sweat_smile:

[Matt-Yorkley]

Wow, these migrations take an incredibly long time with the French data... it's taken an hour and a half!

It's failing with missing attribute: lock_version at the call to shipments_to_delete.destroy_all, and I'm pretty sure this is caused by the deletion of related objects with dependent: destroy, namely Spree::InventoryUnit, which has it's lock_version attribute removed in a migration a few steps before this one.

Apparently there's some caching which is reset at each call to db:migrate, but not between each migration. I think we can do something like this to fix the failing migration:

def change
   Spree::InventoryUnit.connection.schema_cache.clear!
   Spree::InventoryUnit.reset_column_information
...

[Matt-Yorkley]

@mkllnk the migrations for AddShippingRatesToShipments and MigrateTaxCategoriesToLineItems take around 40 minutes each using French data. I've got 16G of memory and a quad-core i5. Is it worth trying to make them more efficient so they don't take as long? I think we might end up with ~2 hours of downtime just to run all the migrations...

[sauloperez]

I could help on that. I dealt with slow data migrations quite a lot in the past. There is not much we can do for DDL migrations though.

[Matt-Yorkley]

I could help on that. I dealt with slow data migrations quite a lot in the past.

Das ist sehr gut, herr Pau!

[mkllnk]

I think there are only one or two that are really slow. It may be good to look into those. But we should also be really careful because migrations are not covered by unit tests and new migrations are not tested by the servers that have v2 already. France would be a guinea pig for those.