Django-celery-beat: InconsistentMigrationHistory when updating to 1.3.0

[timthelion]

 $ python3 manage.py showmigrations

shows

 [X] 0001_initial
 [X] 0002_auto_20161118_0346
 [X] 0003_auto_20161209_0049
 [X] 0004_auto_20170221_0000
 [X] 0005_add_solarschedule_events_choices
 [ ] 0006_auto_20180210_1226
 [X] 0006_auto_20180322_0932
 [X] 0007_auto_20180521_0826
 [X] 0008_auto_20180914_1922
 [ ] 0006_periodictask_priority

And I cannot migrate.

Seems to me that https://github.com/celery/django-celery-beat/commit/639ceb2d6eec0d9cf3771ad6025d791d622c85cc is relevant.

"django-celery-beat": {  "hashes": [  "sha256:b25bc19a589e2851361408708a2aac43524608dd35b85d34a295f90f873a75e5",  "sha256:f3be14a02280d9977757acfcc8464d0bb5a24c3ed5fbd88b60ca63bb24fce632"  ],  "index": "pypi",  "version": "==1.2.0"  },

Was in my Pipfile.lock BEFORE the migration.

"django-celery-beat": { "hashes": [ "sha256:800b67d396b12c59ca40b50a10c3ace58ca5907879237cad4e0cd404e779aa39", "sha256:81cdf4b2dfe64433ec2527ce654bc4bf3e3a3410690163205f6afcd3697e4158" ], "index": "pypi", "version": "==1.3.0" },

Is in my Pipfile.lock now.

[CsHeng]

can't migrate from fresh even.

[auvipy]

@thedrow should we delete all old migration and generate fresh new migrations?

[thedrow]

That would break our existing users.

[timthelion]

While it isn't possible to delete old migrations, I think it is somewhat safe to reset the migrations directory to before commit https://github.com/celery/django-celery-beat/commit/9059d74ef5c490a9e9c730063b2f669b4e89caaf and then automatically regenerate the migrations. I think the problem is that the commit added a NEW dependecy to an OLD migration https://github.com/celery/django-celery-beat/commit/9059d74ef5c490a9e9c730063b2f669b4e89caaf#diff-74007946e366af31fb74393276e87dc4R13 so anyone who had already applied 0006_auto_20180322_0932 ends up with an inconsistent migration history because 0006_auto_20180322_0932 is applied and its (new) dependency 0006_auto_20180210_1226 is not.

[timthelion]

Those who had isntalled fresh from 1.3 could do manage.py migrate 0005_add_solarschedule_events_choices before upgrading. Which is an extra step, but at least possible to fix ;)

[thedrow]

I'll try to get to it over the weekend.

[iamit]

Having the same problem over here, was too late to solve the puzzle, so i turned off celery beat (the scheduling system is not yet in production at this customer). By the way, i learned (also the hard way like this) to never change the migration path of history once you have published code. Django (but before also south) cannot handle this. So better to have 2 more migrations, than several migration paths. Once you are in to this, the whole migration (also for other packages) are stuck. So once you have published migration 6... (on pypi)..... never go back and make a new migration with the same number (6 in this case)..... it's pretty hard to solve it in a proper way now... because there are already people on 2 paths now.... The best way in my opinion is to tag the commit before the mess started in a clear name, so people can easy go back to this commit, and go back in migration, and than continue with the new path. And cretae some clear descirption of how people can solve this problem in your docs (include an .md with the steps).

[thedrow]

Sadly I don't have time to resolve this right now. PRs are welcome.

[auvipy]

@tinylambda could you please again check the issue?

[tinylambda]

@tinylambda could you please again check the issue?

This is a migrations issue related to Django ORM schema evolution machanism. I met the same migration issue after upgrading to 1.3.0 , I finally solve it by dropping the database (test environment), and then make migrations again .

I will check this issue and to see if I can find an elegant solution.

[tinylambda]

I reproduced the inconsistent migration history exception:

(venv) localhost:metadata_management_service Felix$ python manage.py makemigrations
No changes detected
(venv) localhost:metadata_management_service Felix$ python manage.py migrate
Traceback (most recent call last):
File "manage.py", line 16, in
execute_from_command_line(sys.argv)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(args, *cmd_options)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(args, *options)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(args, *kwargs)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 85, in handle
executor.loader.check_consistent_history(connection)
File "/Users/Felix/PycharmProjects/metadata_management_service/venv/lib/python3.7/site-packages/django/db/migrations/loader.py", line 297, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration django_celery_beat.0006_auto_20180322_0932 is applied before its dependency django_celery_beat.0006_auto_20180210_1226 on database 'default'.

[tinylambda]

I analyze the two migration file:
django_celery_beat.0006_auto_20180322_0932
django_celery_beat.0006_auto_20180210_1226

found that we can safely remove the dependency for django_celery_beat.0006_auto_20180210_1226 in django_celery_beat.0006_auto_20180322_0932:

django_celery_beat.0006_auto_20180210_1226.py
alter 3 fields of crontabschedule:

class Migration(migrations.Migration):

dependencies = [
    ('django_celery_beat', '0005_add_solarschedule_events_choices'),
]

operations = [
    migrations.AlterField(
        model_name='crontabschedule',
        name='day_of_month', #<<<<<<<<##################################
        field=models.CharField(default='*', max_length=124,
                               verbose_name='day of month'),
    ),
    migrations.AlterField(
        model_name='crontabschedule',
        name='hour', #<<<<<<<<##################################
        field=models.CharField(default='*', max_length=96,
                               verbose_name='hour'),
    ),
    migrations.AlterField(
        model_name='crontabschedule',
        name='minute', #<<<<<<<<##################################
        field=models.CharField(default='*', max_length=240,
                               verbose_name='minute'),
    ),
]

and django_celery_beat.0006_auto_20180322_0932.py alter the three fields too:

class Migration(migrations.Migration):

dependencies = [
    ('django_celery_beat', '0005_add_solarschedule_events_choices'),
    # ('django_celery_beat', '0006_auto_20180210_1226'),
]

operations = [
    migrations.AlterModelOptions(
        name='crontabschedule',
        options={
            'ordering': [
                'month_of_year', 'day_of_month',
                'day_of_week', 'hour', 'minute', 'timezone'
            ],
            'verbose_name': 'crontab',
            'verbose_name_plural': 'crontabs'
        },
    ),
    migrations.AddField(
        model_name='crontabschedule',
        name='timezone',
        field=timezone_field.fields.TimeZoneField(default='UTC'),
    ),
    migrations.AlterField(
        model_name='crontabschedule',
        name='day_of_month', #<<<<<<<<##################################
        field=models.CharField(
            default='*', max_length=124, verbose_name='day of month'
        ),
    ),
    migrations.AlterField(
        model_name='crontabschedule',
        name='hour',  #<<<<<<<<##################################
        field=models.CharField(
            default='*', max_length=96, verbose_name='hour'
        ),
    ),
    migrations.AlterField(
        model_name='crontabschedule',
        name='minute', #<<<<<<<<##################################
        field=models.CharField(
            default='*', max_length=240, verbose_name='minute'
        ),
    ),
]

the definition of the 3 fields remains the same, so I think we can safely remove the dependency for django_celery_beat.0006_auto_20180210_1226 in django_celery_beat.0006_auto_20180322_0932

any suggestions ?

[tinylambda]

After removing the dependency for django_celery_beat.0006_auto_20180210_1226 in django_celery_beat.0006_auto_20180322_0932, just execute python manage.py migrate, everything goes well。

[tinylambda]

anyone can test it ?

https://github.com/tinylambda/django-celery-beat/tree/inconsistent-migration-bugfix

[thedrow]

If this works, I'll merge and release a new version.

[tinylambda]

If this works, I'll merge and release a new version.

I think we need serval people to to test it and get feedback.

[tinylambda]

Anyone can do some tests on this solution?

https://github.com/tinylambda/django-celery-beat/tree/inconsistent-migration-bugfix

[iamit]

i'll give it a go, result soon, need to logon to my customer server to try this...

[iamit]

Yep, this solved it for me!

[BITSOLVER]

We must remember that the end user should only have to run "python manage.py migrate", the should not have to run makemigrations.

[Simage]

I can also confirm this solved it for me

[thedrow]

I just released this as 1.4.0.