Celery: CELERY_TIMEZONE and CELERY_ENABLE_UTC have no effect

[lonemc]

I am trying to run a celery worker from console with a timezone different than the local timezone with no luck. No matter what I do, the logs of the worker are always shown with the local timezone (which is not UTC). Shouldn't the worker start by default in the UTC timezone? (assuming the CELERY_ENABLE_UTC is enabled by default).

Here's one possible cause: I vaguely remember this was working a few months ago, so might it be because now the Daylight Saving Time is active?

My real problem is that I have a Django app using the local timezone (which is not UTC), and a few celery tasks inside the Django app. I need those tasks to be run as celery workers that are using the UTC timezone.

[jackyfkc]

The directives CELERY_TIMEZONE and CELERY_ENABLE_UTC are worked for celery beat, not worker.

The logging in celery does not take into account CELERY_ENABLE_UTC, it just a simple wrapper to python logging module.

[thedrow]

CELERY_ENABLE_UTC is not enabled by default because people might use very old Django versions (below 1.4).
Can you please provide an example?

[malinoff]

@thedrow just to note, we should enable UTC by default in October, when Django 1.4 LTS support will be ended (https://www.djangoproject.com/download/)

[thedrow]

Yes we probably should do it for 3.2.0.

[thedrow]

Now that I checked the documentation I saw that CELERY_ENABLE_UTC is enabled by default so I think this bug is only related to logging.

[ask]

Closing this, as we don't have the resources to complete this task.

May be fixed in master, let's see if comes back after 4.0 release.

[mihai-carneylabs]

As a helpful note for anyone who may stumble on this discussion, I was able to run my celery workers on a different timezone (UTC) than the Django project (EST). I gave celery a different Django settings file (i.e. celery_settings), in which I just import the original settings (from settings import *) and then overwrite the TIME_ZONE parameter.

[marvelph]

It seems that this problem still remains in Celery 4.0.2.
The time stamp of the standard output log is affected by Django's TIME_ZONE parameter.

[georgepsarakis]

Issue #4006 suggests the opposite. @marvelph this may be fixed in current master.

[marco-silva0000]

I had problems with CELERY_TIMEZONE = 'Europe/Lisbon' and django TIMEZONE = 'Europe/Lisbon' using the beat scheduler. it would wrongly calculate the dates date of the next is_due.
On the pip version I had the calculation be negative and that would lunch tasks imediatly for ever, on the current master i had it calculate to 1h+repeat time.
setting both settings to 'UTC' fixes the problem.

[georgepsarakis]

We merged #4173 to master. If you can use the master branch and check if the issue remains, it would be great, thanks.

[marco-silva0000]

I tested with master, please see the latest comment here https://github.com/celery/celery/issues/4041
There is still a problem with celery timezone on the latest master, if I have both CELERY_TIMEZONE and TIMEZONE to 'Europe/Lisbon' the django celery beat schedules the next task to 1h more than the correct time

[Ken4scholars]

I have both TIME_ZONE and CELERY_TIMEZONE set to Europe/Moscow in my project but Celery beat still uses UTC. I've also tried switching the CELERY_ENABLE_UTC setting but nothing changed. My system time is Europe/Moscow and this is really frustrating

[madEng84]

Same problem here (CELERY_TIMEZONE = TIME_ZONE = Europe/Rome) with celery 4.1 and django 1.11.x

• My periodic_tasks are not working at expected time. (I haven't fixed it :/)
• When i retry a celery task, the calculated ETA is wrong (lesser than now) so the retry is instantaneous. For this problem I have calculated the eta passing it to my retried task, as you can seee here (https://github.com/celery/celery/issues/4221#issuecomment-324204504)

I'm in Work in Progress, if i find a better solution, i will write here

[MounirMesselmeni]

I have the same issue as @madEng84 Django 1.11 and Celery 4.1, tasks retry does not work with countdown, only with eta.

[expresspotato]

This is an absolute f* ing joke. All celery is supposed to do is task -> run it on time. Why the hell do these developers waste all their time adding exotic feature xyz that no one gives a sh*t about when it can't even do what's its designed to! Holy crap!

CELERY_TIMEZONE = 'US/Eastern'

Tasks run at completely the wrong time on 4.1, was okay back on 3. whatever

[thedrow]

@expresspotato If you are not happy about the work we do FOR FREE go fork celery.
You can also pay us to fix your problems.
Otherwise self-entitlement is not welcome. Consider this your last warning before I ban you.

[Jamim]

As far as I know this issue was fixed in master few months ago. I used master branch pinned to be55de622381816d087993f1c7f9afcf7f44ab33 instead of release to avoid this and it worked as expected.
So, it's a great question why is it still not released? It's incomprehensible.

[thedrow]

@Jamim We have a few more pull requests to merge before releasing. See the milestone.

[auvipy]

https://github.com/celery/celery/commit/be55de622381816d087993f1c7f9afcf7f44ab33

[zhangchmeng]

set timezone 'Asia/Shanghai'.The runtime calculates the correct time,but When retry is enabled eta don't calculate '+' behind time. like this '''ETA:[2018-04-19 11:14:53.216361+08:06] ''''.Only the calculation sheet is not used.I'm going to add the number of seconds to the given time. He works,But what's not perfect?I hope my statement is useful.

[kylebebak]

@auvipy
It's obvious this should not have been closed.

I'm running celery==4.1.0 and Django==1.11.6.

Regardless of what I set for CELERY_TIMEZONE and TIMEZONE, Celery beat uses UTC. No combination of True and False for CELERY_ENABLE_UTC and USE_TZ makes a difference.

It would be better to acknowledge this a bug than make users think it works.

[auvipy]

try 4.2rc2 and report please

[trianglesis]

I can't understand what is going on with time settings in Celery, but I fixed
base.py -> now
And now it report actual time for selected timezone 'Europe/London', see log:

[2018-06-22 13:24:37,338: ERROR/MainProcess] <=Celery App Base=> now -> now_in_utc                      2018-06-22 12:24:37.336361+00:00
[2018-06-22 13:24:37,338: INFO/MainProcess] <=Celery App Base=> now -> self.timezone                    Europe/London
[2018-06-22 13:24:37,338: ERROR/MainProcess] <=Celery App Base=> now -> now_in_utc.astimezone(self.timezone)                            2018-06-22 13:24:37.336361+01:00
[2018-06-22 13:24:37,338: ERROR/MainProcess] <=Celery Schedules=> now -> self.app.now() 2018-06-22 13:24:37.336361+01:00
[2018-06-22 13:24:37,338: ERROR/MainProcess] <=Utils Time=> remaining -> now 2018-06-22 13:24:37.338674+01:00
[2018-06-22 13:24:42,349: ERROR/MainProcess] <=Celery Schedules=> now -> self.nowfun() 2018-06-22 13:24:42.349524+01:00

But task is still show time as raw UTC - 2018-06-22 12:24:42.350384

Test change:

[2018-06-22 13:40:30,568: ERROR/MainProcess] <=Celery App Base=> timezone_func conf.timezone: Europe/Kiev
[2018-06-22 13:40:30,569: ERROR/MainProcess] <=Celery App Base=> timezone_func timezone.get_timezone(conf.timezone): Europe/Kiev

[2018-06-22 13:40:30,603: ERROR/MainProcess] <=Celery App Base=> now -> now_in_utc.astimezone(self.timezone) 1                          2018-06-22 15:40:30.600489+03:00
[2018-06-22 13:40:30,603: ERROR/MainProcess] <=Celery Schedules=> now -> self.app.now() 2018-06-22 15:40:30.600489+03:00
[2018-06-22 13:40:30,604: ERROR/MainProcess] <=Utils Time=> remaining -> now 2018-06-22 15:40:30.603985+03:00
[2018-06-22 13:40:30,604: ERROR/MainProcess] <=Celery Schedules=> now -> self.nowfun() 2018-06-22 15:40:30.604436+03:00

So seems like now settings timezone have effect, BUT tasks are still using an UTC time!

[trianglesis]

Firstly, this probably should be the datetime.now instance, not datetime.utcnow(), because why would we decide to UTC already an UTC?

def to_utc(dt):
    """Convert naive :class:`~datetime.datetime` to UTC."""
    return make_aware(dt, timezone.utc)

It now allows me to set London time everywhere:

<=Celery App Base=> timezone_func conf.timezone: Europe/London
<=Celery App Base=> timezone_func timezone.get_timezone(conf.timezone): Europe/London
<=Celery Schedules=> now -> self.nowfun() 2018-06-22 14:11:03.625825+01:00
<=Utils Time=> to_utc -> What is dt_utc: 2018-06-22 14:11:03.626616
<=Utils Time=> remaining -> now 2018-06-22 14:11:03.629900+01:00
<=Celery Schedules=> now -> self.nowfun() 2018-06-22 14:11:03.630299+01:00

<=Utils Time=> make_aware -> _localize(dt, is_dst=None) 2018-06-22 14:11:03.630514+00:00
<=Utils Time=> to_utc -> make_aware(dt_utc, timezone.utc): 2018-06-22 14:11:03.630514+00:00
<=Utils Time=> make_aware -> What is dt: 2018-06-22 14:11:03.630514

But still task recieved in older time:

Received | 2018-06-22 13:05:02.712443 UTC
-- | --
Sent | 2018-06-22 13:05:02.707066 UTC
Started | 2018-06-22 13:05:02.716835 UTC
Succeeded | 2018-06-22 13:05:03.029372 UTC
Retries | 0
Timestamp | 2018-06-22 13:05:03.029372 UTC

[georgepsarakis]

@trianglesis if possible, please open a PR with your changes and we can discuss it there.

[qburst-starsona]

Youy can try the nowfun argument in crontab for setting different timezone

https://stackoverflow.com/questions/21827290/celery-beat-different-time-zone-per-task

[liziwl]

I have the same problem that date_done always using UTC when I have set

cel_app.conf.timezone = 'Asia/Shanghai'
cel_app.conf.enable_utc = True

on v4.3.0 installed by pip and redis, so the problem is fixed or not?