Ghost: Add & use a global blog timezone

[ErisDS]

It's time :clock10: for us to finally add proper timezone support to Ghost. The original main issue for this (with a great deal of discussion) is here: https://github.com/TryGhost/Ghost/issues/1265.

Other related issues which may or may not be helpful:

• Add timezone dropdown to settings #2360
• Determine the best list of time zones for use in Ghost #2359
• An ancient PR #2387

The timezone problem:

The problem we face with timezones revolves almost entirely around published dates.

Take the following scenario:

• A user in UTC-8 (San Fran) opens their post settings menu, and sets the published at date to be 8pm on Dec 31st 2015.
• Ghost correctly stores that as 4am Jan 1st 2016 UTC.
• The blog is set to use Casper, which doesn't have localisation in the theme.
• They share their post straight away and all the readers see it marked as published in the future!

This gets more confusing when post scheduling comes into play. Take the example of a San Fran startup blog. They like to publish posts at lunchtime for their readership which is mostly also in San Fran. They hire a writer in Singapore, who is in UTC+8.

• The writer creates their post, and sets the published at date to be 12:30pm (lunchtime) on Feb 1st 2016 (a date that is in the future).
• Ghost correctly stores that as 4:30am Feb 1st 2016 UTC.
• The scheduler will publish the post at 4:30am Feb 1st 2016 UTC which lunchtime in Singapore, but for the San Fran readership, that's 8:30pm on the previous day.
• Meanwhile the post itself will output on the frontend saying it was posted at 4:30am.

There are two underlying problems here:

1) There's no way to tell Ghost what timezone to use when outputting a date in your theme (other than using localisation in the theme), or anywhere else that a date might be output.

2) When you set a date in the PSM, there's no way to know whether that's using your timezone, the server's timezone, or the blog's timezone (which doesn't exist yet). It actually uses your own timezone, but then stores UTC, which makes things work sort of correctly, but is confusing.

We're going to solve this the same way that Twitter, Tumblr, WP, Slack etc all do. Add a setting to the general settings, and use this to output all dates everywhere, including when setting the published at date.

The Timezone Setting

The timezone setting itself will take the form of a dropdown list, exactly the same as you see in Tumblr or Slack.

Tumblr:

• The setting should exist under general settings in the Ghost admin
• This setting should be a dropdown list
• The format should be (UTC<offset>) List, Of, Cities just like Tumblr
• The list should contain exactly the same entries as Tumblr
• The default should be GMT London (the closest real TZ to UTC and the middle of the list)
• The text under the dropdown should read "The local time here is currently HH:mm am/pm" - the time should reflect the TZ selected, and should ideally tick over each minute (like on Buffer)

The Post Settings Menu

In the post settings menu, the date picker needs to be updated to use the new global blog timezone instead of the user's timezone.

That means, if I'm in UTC+2, and I login to my blog which is set to GMT, when I create a new draft the PSM should show me the time in GMT, not in UTC+2. Equally if I set a post to publish at 2pm, this should be 2pm GMT not 2pm UTC+2 i.e.12pm GMT.

The Date Helper

There are actually 2 date helpers in Ghost:

• /core/server/helpers/date.js -> used in themes
• /core/client/app/helpers/gh-format-timeago.js -> used in the admin panel

Both of the Date helpers need to be updated to take the timezone into account. As a result, the date output should always be correct according to the blog's timezone.

[nuclearpengy]

this is going to be rad.

[szelpe]

Hi @AileenCGN, @ErisDS!

This is a really great improvement, as a longtime Ghost user from Hungary I'm looking forward to finally set my blog to my local timezone.

May I point to a possible shortcoming of your current implementation? I've checked the commit (7d20553651e7263ea5fde4017cbe97695d9d8a89), and if I'm not mistaken, utcOffset is used to calculate the local time, which, I understand, is incorrect.

The thing is that the UTC offset is not fixed, but affected by the daylight saving time, e.g. currently in the UK the offset is 0, but after 27th of March it will be 60.

To make things worse, the start of daylight saving time depends on the Country/region, e.g. in the US it starts on 10th March, or if I'm correct there is no daylight saving time in Egypt at all.

The best thing to do in my opinion is to use moment-timezone, which can handle this.

Here is an example:

var m = moment('2016-03-20T10:22');
console.log(m.utcOffset(-300).isDST());      // false
console.log(m.tz("US/Eastern").isDST());     // true
console.log(m.tz("Europe/Dublin").isDST());  // false

Not handling the DST properly can lead to several issues, including misleading information for the users and incorrect post scheduling.

Not to mention the sort of an edge case, when a user from the UK sets the publishedAt property to March 27th 2:30, which is kind of non-existing because at 2:00 the time is shifted to 3:00, will the post be published or not?

[ErisDS]

Hey @szelpe, this kind of constructive feedback is incredibly helpful, thanks for taking the time, much appreciated :+1:

[jloh]

Another thing to consider (if it hasn't been already) is users who chose to have the date of posting in their URL's.
Take for instance my migration to GhostPro from a server running in UTC+10 (AEST). Several of my posts ended up being back a day so the old URL now returned a 404.

ie http://example.com/2016/02/01/somepost/ -> http://example.com/2016/01/31/somepost/

So, when timezones get enabled in blogs, what is going to happen to posts that _might_ have their URL's changed in this scenario? Are they just going to return 404's? Is it going to be possible to add remaps or something?

[acburdine]

_This was going to be a quick comment and then I did some more testing (so bear with me)_

In response to @Mooash:

Part 1

On master (w/o the changes from #6421 added), I set a post's published date to yesterday at 11pm (I'm in GMT-6 timezone), and published it. With date included in my post permalinks, it showed up as /2016/03/17/post-name/).

I then added the changes from #6421 on top of master. The permalink _didn't change_; however, the shown published date inside the post-settings-menu did (it changed to today at 4am -> the time difference between GMT-6 and the default timezone of GMT - daylight savings time adjusted.). Also, when viewing the list of posts in the frontend or the actual post itself, the "date published" shows that it was published today.

I then went into settings/general and set the timezone of my blog back to GMT-6, and then all of the dates in the places returned to what they were before I added the changes from the PR.

tl;dr - The timezone PR doesn't change permalinks/urls of _published_ posts that were published before the code changes occured

Part 2:

I then tested to see if a new post with the PR changes was affected by the timezone setting. It wasn't actually. With a timezone of GMT, I set the post's published date to 2AM today. The permalink showed up as yesterday, in line with the current server timezone. This might _actually_ be an issue b/c if the server is hosted in a timezone different from that of the client, then permalinks would not reflect any timezone changes, which might be confusing.

So....while the original question won't happen, it might cause other errors....

My idea on how to fix this:

If dates are included in permalinks, then handle them based on the timezone set in settings. As a stopgap to prevent 404's on previous posts, possibly do a 301 redirect based on the server's timezone to the actual timezone.

Pros with this behavior:

• Permalinks are set relative to the timezone of the blog
• any timezone changes will be reflected in the permalinks

Cons:

• for people that don't know about the change, all of the permalinks would change _until_ they changed their timezone back to what it was before the update (unless we implemented the 304 redirection idea)
• if the timezone of the server is different than the timezone of the client than any posts before the change would 404 with date-enabled permalinks (again, unless we implemented the 304 redirection idea)

[aileen]

any timezone changes will be reflected in the permalinks

Does it make sense, to allow permalinks being changed afterwards? I think it would make more sense, to always reflect the server UTC time in the permalinks. When they are used as a link somewhere and the user changes his timezone, the link should still be the same. What do you think?

[acburdine]

I guess that does make sense, yeah. And honestly it would only make a difference in the url for just the people that use dates in the permalinks. As long as it doesn't break existing links (which it doesn't) it should be fine

[aileen]

Could you find any further issues with the permalinks? I tested it as well, and the dated permalinks always show the UTC server time. Of course, this could be confusing, if the user is not aware of that, but at least it should prevent the permalinks from breaking if a user changes his timezone. As I couldn't find any further issues with that, I think it's working fine now and there's no need for a 'fix'. Is that ok for you, @acburdine?

[acburdine]

@aileen that sounds good to me, but I'll defer to @erisds 's judgement before merging the PR :smile:

[ErisDS]

Sorry, I know this is waiting for a comment from me, I've been mulling it over. Although it's not a bug, per se, it seems to be a big missing piece of the timezones puzzle and it doesn't appear to have an easy solution.

Up until now, we haven't had a concept of timezone setting, we've always used the server, so permalinks using the server timezone was OK. Now that we have a timezone setting, it seems to me that it would be the user's expectation that their post URL would reflect their timezone setting.

E.g. if in January I write a post just after midnight to say 'Happy New Year' and I'm in Spain and have set my timezone to UTC+1, I wouldn't expect my happy new year post to have a url of 2016/12/31/happy-new-year/.

Therefore, although it's not changing permalinks, and is not a bug, it still breaks user expectations. Therefore I'd rather not merge the timezones PR in until we have a clear idea how to do this extra piece. Otherwise one will definitely go out without the other and it could get harder to reconcile the behaviour.

There are two possible ways I see of fixing this:

One is implementing 301 redirects from server time to the blog-setting time as suggested by @acburdine. The major downside to this is that it only accounts for a single timezone change.

The second is to store a reference to which timezone was set when the post was published. My first idea here is to have a published_tz field added to posts which defaults to the server timezone (so all existing posts would have it set to the server timezone). And then any future posts get whatever the timezone setting is on the blog at the time the post is published. The downside here is storing additional data which means complexity & also having to figure out what to do with imports/exports.

I think this calls for a bit of research into how these things are handled elsewhere. If anyone has any ideas or suggestions or examples of handling of timezones in other platforms, please chime in!

[aileen]

Found some discussions about dated permalinks regarding the timezones issue:

• https://code.djangoproject.com/ticket/18794
• https://github.com/jekyll/jekyll/issues/492
• http://programmers.stackexchange.com/questions/161643/which-timezone-is-used-for-permalinks-of-posts

[kirrg001]

Thanks for the pre-work @ErisDS and @acburdine. Here is my conclusion of how things can work together.

Why we need UTC at application level (https://github.com/TryGhost/Ghost/pull/6421#issuecomment-212541046)

• right now we store dates in the database based on the local server time
• Ghost(Pro) always stores UTC in the DB because our servers are configured that way
• self installed ghosts store the dates in any timezone depending on the server setup. that was a plus until now because we didn't support active timezone setting. so blogger just configured their server to the timezone they needed for their correct blog timezone
• now that we offer timezones as a setting, we should use UTC from application perspective
• a good reason is: default scheduling will not work when we don’t use UTC as default because:

CASE:

• server is configured to run in GMT+2, but active timezone is GMT-6 (so your blog runs in GMT-6 timezone)
• imagine now is 2016-01-01 2PM GMT-6
• you now want to schedule a post for 2016-01-01 5PM GMT-6 (so for 3 hours later)
• ember admin will send this date over to the backend as UTC (which is 2016-01-01 11PM)
• because the server is configured as GMT+2, the DB would store 2016-01-02 1AM
• the default scheduler would publish the post at 2016-01-02 1AM (which is in GMT-6 2016-01-01 7PM)
• but we expect the post to get published for 2016-01-01 5PM GMT-6
• so the post would get published 2 hours too late!
• this only happens if the server is not configured to UTC, so only to self installed ghosts
• but still we want to ensure that if we ship features, they should work on Ghost(Pro) and on self hosted ghosts

Because of this case we should ship the ghost node application running in UTC.
Either we force the user to change the system time (which may not work, because some people will not read the docs good enough or are not able to change it) or we force UTC in the node process. For example we can add UTC support by adding process.env.TZ = 'UTC' before any date is used. I will read into known issues later how we can avoid running into trouble.

When we change the process to run in UTC, we need to convert all existing posts in the database to be in UTC, if the local server time is not UTC. That means we need a migration script. The migration script will read the timezone of the server. Then we transform all post dates into UTC and update them. The rest will work automatically because every blog who gets this new ghost update, will have a default timezone in the settings (which is Dublin TZ). But permalinks will break afterwards. Thats a very good reason to implement the redirect option. The redirect idea is the simplest solution we can offer and it will solve a lot of edge case.

redirect broken permalink and scheduling

• every time a reader opens a permalink from the web, the server will compare the date permalink prefix (/:year/:month/:date) and if not equal, it will redirect to the one that is valid (based on the current timezone setting)
• we always find a post based on slug (or id), so the date prefix in the url actually doesn't matter
• lets do a big example

CASE:

• the new year example
• your blog timezone is GMT+1
• your server is running on Ghost(Pro) hosting (so node runs with UTC)
• you schedule a post for 2016-1-1 12:00AM
• the database stores it as 2015-12-31 11:00PM (which is UTC)
• because the permalink would be based on current timezone setting, the link would be /2016/01/01/happy-new-year if the post gets published (because transforming 11:00PM UTC back to GMT+1, would result in day 2016-01-01 instead of 2015-12-31. (thats good)
• now you change the active timezone of your blog to GMT-4
• if we don't change the old UTC date 2015-12-31 11:00PM, the post would get published 4 hours too early!!!
• thats why we need to listen on active timezone changes
• in this case the post published_at in the database needs to be changed to 2016-01-01 4AM
• the valid permalink for this post would be 2016-01-01 4AM UTC ---> 2016-01-01 12AM GMT-4 --> /2016-01-01/slug (which is correct)

Regarding to imports from wordpress blogs or existing non hosted ghost blogs:

When we support the redirect feature, importing data is not a problem at all. Because we can redirect every permalink to the correct url on the fly.
The only thing we should add is a meta attribute in the JSON file in which timezone the dates in the JSON file are stored to ensure we can convert the dates to correct UTC values.

---

@AileenCGN
Here is a list of what needs to be done to get it work:

• implement the new redirect logic for permalink in /controllers/frontend/index
• we have a util fn urlPathForPost which calculates the permalink based on a post based on server timezone, this fn needs to be extended to calculate the url based on blog timezone
• update all scheduled posts published_at fields when blog owner changes the timezone settings, this needs to trigger a reschedule event (post.js model already triggers all kinds of schedule events)

I will implement the UTC force on application level and care about the importer.
I’m happy for feedback and edge cases which cannot be handled with this strategy.