Django-celery-beat: The task celery.backend_cleanup will delete my tasks added before if have the same crontab time

Created on 14 Mar 2020 · 7Comments · Source: celery/django-celery-beat

I set my task with crontab <0, 4, *, *, *>, then restart celery.
I find my task was deleted, then I debug the code and I find a snippet executed in django-celery-beat/models.py:

@classmethod
    def from_schedule(cls, schedule):
        spec = {'minute': schedule._orig_minute,
                'hour': schedule._orig_hour,
                'day_of_week': schedule._orig_day_of_week,
                'day_of_month': schedule._orig_day_of_month,
                'month_of_year': schedule._orig_month_of_year,
                'timezone': schedule.tz
                }
        try:
            return cls.objects.get(**spec)
        except cls.DoesNotExist:
            return cls(**spec)
        except MultipleObjectsReturned:
            cls.objects.filter(**spec).delete()
            return cls(**spec)

I wonder why delete the crontab if MultipleObjectsReturned error occurred even if the crontab is bound with other task?

bug

Source

linxi-1214

👍1

All 7 comments

mvikharev on 29 Jun 2020

This is a major issue indeed, just came across it recently.
Probably for historic reasons and because there is no unique constraint, schedules tables can contain duplicates.
So, schedules should not be deleted as they can be linked to other tasks, IMO the correct code should be:

@classmethod
    def from_schedule(cls, schedule):
        spec = {'minute': schedule._orig_minute,
                'hour': schedule._orig_hour,
                'day_of_week': schedule._orig_day_of_week,
                'day_of_month': schedule._orig_day_of_month,
                'month_of_year': schedule._orig_month_of_year,
                'timezone': schedule.tz
                }
        try:
            return cls.objects.get(**spec)
        except cls.DoesNotExist:
            return cls(**spec)
        except MultipleObjectsReturned:
            return cls.objects.filter(**spec).first()

I'll send a pull request ASAP.

EDIT: note that this is not specific to CrontabSchedule, the bug is for all schedule classes.