Django-celery-beat: Celery Beats sends a lot of tasks

Created on 15 Feb 2018  Â·  14Comments  Â·  Source: celery/django-celery-beat

Hello
I'm very new with celery, so maybe there are some problems with my configuration...

The problem is:
I have to run celery beat, that sends the tasks assigned
I have to run celery worker to receive the tasks and run that.
Is ok with that?

THe problem is when i run beat, sends a lot of tasks by second. So if the task is send a email, my case, that cause sends a lot of email when i need only 1.

The timezone in django and celery are the same..
I created PeriodicTask with Crontab

What could be? How to solve?

picture dpineiden

Most helpful comment

If your issue is resolved, please close.

As an aside, it is nearly always a mistake to create server-side code using anything other than pure UTC. It is just too hard to get right, hard to test, and always open to confusion. I write as the author of possibly the first serious attempt to provide a comprehensive timezone support API.

picture ShaheedHaque on 21 Feb 2018
👍2

All 14 comments

What type of schedule are you using (interval, crontab, solar) and what are the detailed timings/settings?

ShaheedHaque picture ShaheedHaque on 15 Feb 2018

Is a Crontab, at some hour selected from 24 hours avalaibles, everyday. And what do you mean with timings settings?

dpineiden picture dpineiden on 15 Feb 2018

By timings/settings, I am asking you to describe exactly what values you are putting into the CrontabSchedule object and also some log extracts showing where it "sends a lot of tasks by second".

I was unable to understand the rest of your reply, but just in case, the way to understand crontab is that it is like a "pattern match". If the various fields match, then bingo, the schedule will fire. There is a reasonable description in the docs.

ShaheedHaque picture ShaheedHaque on 15 Feb 2018

Oh!, The arguments inside de Crontab, is it?

On settings i have:

CELERY_BROKER_URL = 'amqp://localhost'
CELERY_BROKER_USER = "mpuser"
CELERY_BROKER_PASSWORD = "xxxxxx"
CELERY_BROKER_VHOST = "horaenlinea"
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'America/Santiago'
CELERY_ENABLE_UTC = True
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'
CELERY_MAXINTERVAL = 30

On the 'action':

from django_celery_beat.models import CrontabSchedule,PeriodicTask

def edit_appt_notifications(request):
    doctor = request.user.doctor_profile
    form = DoctorNotificationForm(request.POST, instance=doctor.mail_notifications)
    print("Nueva hora de email periodico...")
    print(request.POST)
    print(form)
    dr_id=request.user.id
    time=request.POST['appointments']
    h_t=time.split(':')
    print("Doctor id",dr_id)
    taskname='apps.doctors.tasks.send_daily_agenda'
    if form.is_valid():
        schedule,_=CrontabSchedule.objects.get_or_create(
            hour=str(int(h_t[0])),
            minute=str(int(h_t[1])),
            day_of_week='*',
            day_of_month='*',
            month_of_year='*',
        )
        print("Crontab created: ", _)
        ptask, created=PeriodicTask.objects.get_or_create(
            name='Enviar agenda de doctor %s' %doctor,
        )
        if created:
            print("Periodictask created")
            ptask.crontab=schedule
            ptask.task=taskname
            ptask.kwargs=json.dumps({"dr_id":dr_id})
        if not created:
            #update
            print("Periodic task not created")
            ptask.crontab=schedule
        ptask.save()

        print("Task added to list")
        form.save()
        data = {}
        return HttpResponse(json.dumps(data), content_type="application/json")
    return HttpResponseForbidden()
dpineiden picture dpineiden on 15 Feb 2018

Please understand that I need to see what these hour and minute parameters look like:

        schedule,_=CrontabSchedule.objects.get_or_create(
            hour=str(int(h_t[0])),
            minute=str(int(h_t[1])),
            day_of_week='*',
            day_of_month='*',
            month_of_year='*',
         )

Otherwise, how do you think I can understand what is going on?! And as I asked, please provide an extract from the worker log file showing the actual frequency of the tasks you are complaining about.

ShaheedHaque picture ShaheedHaque on 16 Feb 2018

Ok

Example:

h_t="12:00"

The tasks are defined like that:

@shared_task
def add(x, y):
    z = x + y
    print("Tarea suma periodica")
    print(datetime.now())
    print("La suma x+y es: %s" %z)
    return x + y

@shared_task
def send_daily_agenda(dr_id:int=0):
    #check dr_id instance
    dr_instance=Doctor.objects.filter(user__id=dr_id).first()
    if dr_instance:
        #proceed to send email
        email_daily_agenda(dr_instance)

And, to test i have other tasks every minute that's executed now many times by seconds.

The log says:

[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] {'task_id': '28699fa0-0768-4ed2-b092-0c6ecca56cda', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.666803
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '6104306f-f330-415e-aae4-f0a13a2485e0', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-2] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'c1ea6628-ac00-4681-be91-b14e89dc84de', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '28699fa0-0768-4ed2-b092-0c6ecca56cda', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,668: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,668: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] {'task_id': '7a666313-405f-44da-b7b4-35cc8a06844c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] {'task_id': 'c5f56dc4-f532-4c5e-bdb3-308c266d6755', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] Tarea suma periodica
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.669516
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] 2018-02-16 00:29:57.669541
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] La suma x+y es: 7
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] La suma x+y es: 7
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] {'task_id': 'f962bc59-7370-4bc6-ac52-f2c3ea6c4113', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-2] Tarea suma periodica
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] {'task_id': 'fc6f36cd-41f1-4d4b-be3b-40fdc7267aaa', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-2] 2018-02-16 00:29:57.671566
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.671413
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-2] La suma x+y es: 7
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-4] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '7a666313-405f-44da-b7b4-35cc8a06844c', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'c5f56dc4-f532-4c5e-bdb3-308c266d6755', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-2] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'f962bc59-7370-4bc6-ac52-f2c3ea6c4113', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'fc6f36cd-41f1-4d4b-be3b-40fdc7267aaa', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] {'task_id': '68624f8e-2ffb-4200-8e59-500ff95d4679', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] {'task_id': '49dfade4-f4bb-48dc-a00a-10eb1b3c84c2', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.677012
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] La suma x+y es: 7
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.677462
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-2] {'task_id': '14206fce-80ac-4b4d-9059-a422a8c1880f', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-2] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] {'task_id': 'e578b14c-060a-4065-8223-37bab2ce947c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] Tarea suma periodica
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] 2018-02-16 00:29:57.678470
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-2] 2018-02-16 00:29:57.678438
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-2] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-4] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '68624f8e-2ffb-4200-8e59-500ff95d4679', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
^C[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'e578b14c-060a-4065-8223-37bab2ce947c', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] {'task_id': '784bedff-8fe1-42d5-a43c-bc5b5d6e5f5c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,693: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.693007
[2018-02-16 00:29:57,680: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,693: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '49dfade4-f4bb-48dc-a00a-10eb1b3c84c2', 'kwargs': {}, 'retval': 7}
dpineiden picture dpineiden on 16 Feb 2018

Thanks. First, your CrontabSchedule setup looks fine, AFAICS, there should
be one task at 12:00 each day. Second, I see that the tasks that are being
run are sometimes running multiple times in one millisecond timestamp, and
these timestamps have nothing to do with 12:00.

This looks very similar to my issue #128. Is it possible that like me, at
some point in your development cycle, you may have accidentally created a
repeating schedule with zero time delay? If so, your message queue may have
built up as mine did.

Try deleting or disabling all your PeriodicSchedules from the Django admin
GUI. Do these tasks continue even when no database schedules are
present/enabled?

If do, and you are using Rabbitmq, my posting #128 shows how to check the
queue and clear it.

On 16 Feb 2018 03:32, "David Pineda" notifications@github.com wrote:

Ok

Example:

h_t="12:00"

The tasks are defined like that:

@shared_task
def add(x, y):
z = x + y
print("Tarea suma periodica")
print(datetime.now())
print("La suma x+y es: %s" %z)
return x + y

@shared_task
def send_daily_agenda(dr_id:int=0):
#check dr_id instance
dr_instance=Doctor.objects.filter(user__id=dr_id).first()
if dr_instance:
#proceed to send email
email_daily_agenda(dr_instance)

And, to test i have other tasks every minute that's executed now many
times by seconds.

The log says:

[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] {'task_id': '28699fa0-0768-4ed2-b092-0c6ecca56cda', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.666803
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,666: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '6104306f-f330-415e-aae4-f0a13a2485e0', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-2] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'c1ea6628-ac00-4681-be91-b14e89dc84de', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,667: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '28699fa0-0768-4ed2-b092-0c6ecca56cda', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,668: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,668: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] {'task_id': '7a666313-405f-44da-b7b4-35cc8a06844c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] {'task_id': 'c5f56dc4-f532-4c5e-bdb3-308c266d6755', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] Tarea suma periodica
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.669516
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] 2018-02-16 00:29:57.669541
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-4] La suma x+y es: 7
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-1] La suma x+y es: 7
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,669: WARNING/ForkPoolWorker-2] {'task_id': 'f962bc59-7370-4bc6-ac52-f2c3ea6c4113', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-2] Tarea suma periodica
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] {'task_id': 'fc6f36cd-41f1-4d4b-be3b-40fdc7267aaa', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,670: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-2] 2018-02-16 00:29:57.671566
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.671413
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-2] La suma x+y es: 7
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-4] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '7a666313-405f-44da-b7b4-35cc8a06844c', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,671: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'c5f56dc4-f532-4c5e-bdb3-308c266d6755', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,672: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-2] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'f962bc59-7370-4bc6-ac52-f2c3ea6c4113', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,673: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'fc6f36cd-41f1-4d4b-be3b-40fdc7267aaa', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-2] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,675: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] {'task_id': '68624f8e-2ffb-4200-8e59-500ff95d4679', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] {'task_id': '49dfade4-f4bb-48dc-a00a-10eb1b3c84c2', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.677012
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] La suma x+y es: 7
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-3] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-3] 2018-02-16 00:29:57.677462
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-2] {'task_id': '14206fce-80ac-4b4d-9059-a422a8c1880f', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,676: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-2] Tarea suma periodica
[2018-02-16 00:29:57,677: WARNING/ForkPoolWorker-3] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] {'task_id': 'e578b14c-060a-4065-8223-37bab2ce947c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] Tarea suma periodica
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] 2018-02-16 00:29:57.678470
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-2] 2018-02-16 00:29:57.678438
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-1] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-2] La suma x+y es: 7
[2018-02-16 00:29:57,678: WARNING/ForkPoolWorker-4] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '68624f8e-2ffb-4200-8e59-500ff95d4679', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-1] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-3] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-1] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
^C[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-4] debug
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-1] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': 'e578b14c-060a-4065-8223-37bab2ce947c', 'kwargs': {}, 'retval': 7}
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] {'task_id': '784bedff-8fe1-42d5-a43c-bc5b5d6e5f5c', 'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'kwargs': {}, 'args': [3, 4]}
[2018-02-16 00:29:57,692: WARNING/ForkPoolWorker-4] Tarea suma periodica
[2018-02-16 00:29:57,693: WARNING/ForkPoolWorker-4] 2018-02-16 00:29:57.693007
[2018-02-16 00:29:57,680: WARNING/ForkPoolWorker-2] debug
[2018-02-16 00:29:57,679: WARNING/ForkPoolWorker-3] <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>
[2018-02-16 00:29:57,693: WARNING/ForkPoolWorker-3] {'task': <@task: apps.doctors.tasks.add of horaenlinea at 0x7f7857789208>, 'state': 'SUCCESS', 'args': [3, 4], 'task_id': '49dfade4-f4bb-48dc-a00a-10eb1b3c84c2', 'kwargs': {}, 'retval': 7}

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/celery/django-celery-beat/issues/129#issuecomment-366135575,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEp7KXpEkFMgiPbm5qJsBCW1L6GAZwONks5tVPawgaJpZM4SGPln
.

ShaheedHaque picture ShaheedHaque on 16 Feb 2018

Hello.

Indeed there are no 12:00 taks on that log because i recently started the celery beat-worker.
I checked about tasks with zero time delay, but no. It's only what you see on the code and no more on the database than both tasks (add and send_mail)
Using rabbitmq i check the queue using the admin interface(web) and yes, when i run beat it sends a lot of messages.
And yet, at this time, the same...
Thanks :)

dpineiden picture dpineiden on 18 Feb 2018

On 18 Feb 2018 02:06, "David Pineda" notifications@github.com wrote:

Hello.

Indeed there are no 12:00 taks on that log because i recently started the
celery beat-worker.
I checked about tasks with zero time delay, but no.

Ok.

It's only what you see on the code and no more on the database than both
tasks (add and send_mail)
Using rabbitmq i check the queue using the admin interface(web) and yes,
when i run beat it sends a lot of messages.

To be clear, are you saying that there are messages queued in rabbitmq? Can
you share the numbers I highlighted in #128? Did the command to clear the
backlog help?

And yet, at this time, the same...
Thanks :)

If you are sure that you never created a schedule with zero delay, and you
have many queued items then it could be you have the problem reported in

95. Try upgrading to master (since the 1.1.1 release seems to be delayed).

I was unable to repro that problem with the latest on master.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/celery/django-celery-beat/issues/129#issuecomment-366486738,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEp7KdqHzvCmk7ydmn1Z5wWbAAQLe9mQks5tV4WCgaJpZM4SGPln
.

ShaheedHaque picture ShaheedHaque on 19 Feb 2018

1.1.1 is now released.

thedrow picture thedrow on 19 Feb 2018

I found the bug
The model Interval has some errors, so if i set a task every garminute it sends every refresh time.
Also i will upgrade the package
Thanks!

dpineiden picture dpineiden on 20 Feb 2018

well
Now is more quiet
But beat doesn't send the tasks!
Beat side:

[2018-02-20 19:53:48,111: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:53:53,114: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:53:58,119: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:03,120: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:08,123: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:13,124: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:18,129: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:23,135: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:28,140: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:33,146: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:38,151: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:43,155: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:48,157: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:53,162: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:54:58,168: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:55:03,169: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2018-02-20 19:55:08,175: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.

Worker side:

[tasks]
  . apps.doctors.tasks.add
  . apps.doctors.tasks.send_daily_agenda
  . horaenlinea.celery.debug_task
  . imagekit.cachefiles.backends._generate_file

[2018-02-20 19:48:39,041: INFO/MainProcess] Connected to amqp://agenda:**@127.0.0.1:5672/horaenlinea
[2018-02-20 19:48:39,051: INFO/MainProcess] mingle: searching for neighbors
[2018-02-20 19:48:40,067: INFO/MainProcess] mingle: all alone
[2018-02-20 19:48:40,077: WARNING/MainProcess] /home/david/.virtualenvs/horalinea/lib/python3.5/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2018-02-20 19:48:40,077: INFO/MainProcess] celery@pineiden ready.
dpineiden picture dpineiden on 20 Feb 2018

And...
Checking
WIth TImezone in UTC works fine, very well....
On both settings: Django and celery

dpineiden picture dpineiden on 21 Feb 2018
👍1

If your issue is resolved, please close.

As an aside, it is nearly always a mistake to create server-side code using anything other than pure UTC. It is just too hard to get right, hard to test, and always open to confusion. I write as the author of possibly the first serious attempt to provide a comprehensive timezone support API.

ShaheedHaque picture ShaheedHaque on 21 Feb 2018
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

cmberryau picture cmberryau  Â·  8Comments
dpineiden picture dpineiden  Â·  5Comments
linxi-1214 picture linxi-1214  Â·  7Comments
srikanth-chandaluri picture srikanth-chandaluri  Â·  3Comments
pramttl picture pramttl  Â·  10Comments