Freecodecamp: Longest Streak is inaccurate

[elMP]

I'm a new camper. Last days I did challenges every day

My longest streak should be 4, but it's 1. I think January 8 is missing.

In 7 and 9 January I did challenges at night, may be 2 or 3 AM. January 8 I did 3 challenges in the evening, about 22 PM. My timezone UTC+3.

Desktop (please complete the following information):

• Ubutnu
• Chrome and Firefox
• Version 78 and 71

[raisedadead]

Thanks please share your username / profile URL

[elMP]

Thanks please share your username / profile URL

https://www.freecodecamp.org/fcc395594e5-a5f9-4ca1-8a8a-ef3fa24286c0

[raisedadead]

Hi @ahmadabdolsaheb please investigate

[moT01]

The pr I made is really a separate issue from this I think. That one has to do with the heatmap accuracy and not the streak accuracy.

I was looking into the streak issue a little, and I think I'm onto something but haven't dug into it too deep.. I think the timestamps are being created with the users local time and saved into the db that way. And the streak is being created based on UTC time here

[ojeytonwilliams]

And the streak is being created based on UTC time here

Yeah, but that's always called by this
https://github.com/freeCodeCamp/freeCodeCamp/blob/2f39dd07d66c55d498399a6fc6cfe21e196ebd2f/api-server/server/utils/publicUserProps.js#L69

so, effectively never defaults to UTC, since if it's undefined in the initial call to getProgress it's EST for everything else. I think UTC is a better default, but regardless it should be the same everywhere.

That could explain the discrepancy, since there's an 8 hour difference between the @elMP's timezone and EST, which is enough to push the last challenge on the 7th, "2020-01-07T03:01:02", back to the 6th while leaving the next challenge on the 8th.

After more digging through the code, I can't see anywhere that the timezone actually gets set. Can someone with access to the db take a look and confirm/deny? cc @ahmadabdolsaheb

[scissorsneedfoodtoo]

@moT01 and @ojeytonwilliams, great discussion here. I've always wondered how this works and I think we're getting to the bottom of this longstanding issue.

The timestamps are stored as UTC in the database, so it may be the conversion from UTC to EST (or some other timezone) that's causing the large difference here.

@elMP, do you mind if I share some of the timestamps that are being stored in your account so we can take a closer look at this?

[moT01]

I've been bouncing this around in my head for a few days and I don't see a single perfect solution.

I see two problems...

• this streak issue, in which the streak is calculated on our server with UTC or EST time, I believe.
• the second issue is that, when viewing another persons profile - I see their data with my timezone. So the screenshot in the OP's description - I don't see it like that because his timeline and heatmap are being generated on the client side with my timezone. Here's a screenshot of what I see on his profile for the same two weeks in his screenshot...
  

Solution...
As far as I can tell, all timestamps for the completedChallenges and progressTimestamps are being saved in the DB with the users local time. So there's no way to tell what day they were actually submitted on, because we don't know what timezone they were in when they were submitted. One way to make this perfect, would be to save a timezone with each challenge submission. Then we could be sure of the exact day something is submitted, even if users are switching timezones. But it seems a little late for that. I guess we could do that moving forward and add our best guess of the timezone to the existing fields or something.

Another solution would be to save a single timezone field and use that to calculate everything. It may still be slightly off a little bit if the user has been switching timezones, but it would be a lot better. Perhaps have it as a field the user can set themselves on their profile or something? I'm not sure I like that, but it's something.

[ojeytonwilliams]

I'm 99% sure the timestamps are just the UTC when the challenge was submitted:

https://github.com/freeCodeCamp/freeCodeCamp/blob/c76978bdfe233b3842aeb5f29f9f936578894b74/api-server/server/boot/challenge.js#L218-L223

Wouldn't the easiest solution be to have a single timezone that's updated when, say, the user submits a challenge? Or via their profile, as Tom suggested. That way people who don't move timezones will always have the right streaks and the streaks and calendar should match. Obviously there will be anomalies when they do (DST for instance), but otherwise we'd need to figure out how to calculate streaks when each timestamp can have a different timezone. I doubt this would be easy.

[moT01]

I need to read closer, took two comments to realize...

The timestamps are stored as UTC in the database

That should make things easier for some solutions.

An alternative might be to just ignore the timezone completely and make everything based on UTC time. At least it would be predictable.

I feel like we should get some more opinions on this. @freeCodeCamp/dev-team, @freeCodeCamp/moderators - feel free to chime in here.

[raisedadead]

An alternative might be to just ignore the timezone completely and make everything based on UTC time. At least it would be predictable.

Yes please.

[mokrane-karar]

Hi everyone! I'm curently facing the same problem! One mounth ago I have decided to make code everyday for the next 365 days and, I was realy motivated by the longest streak option, unfortunately the current streak is wrong since the last 3 days. It should be 32 but it is 2. There is a second problem and maybe it is the cause of the first one, you can see in the second picture that the date is 13 instead of 12 February. I hope that this can help.
My profile here : https://www.freecodecamp.org/fcc8095740b-2616-4944-b217-7aeaf549fa9b

[moT01]

What do you think about getting rid of the streaks @raisedadead? I think that's where a lot of user frustration comes in. They might have a long streak going and all of a sudden it gets cut off by a timezone issue or something. I have been looking at this the last couple days, and it is proving to be complicated to get all of these aligned.

[raisedadead]

We have considered getting rid of the streak in the past. I will speak to the team internally and check their views on this.

[ojeytonwilliams]

The main problem re: the streaks is they're calculated in one timezone, but the heatmap and timeline depend on the user's timezone. Unless they all match (which they will not for most people) it's not possible to completely rule out inconsistencies.

[mokrane-karar]

In my opinion, you should consider instead to maintain this option as many
people like me rely on it. Maybe you can fix definitely this problem if you
set only one time zone(heatmap and streak) fore all campers around the
world.

On Fri, Feb 14, 2020, 3:06 PM Oliver Eyton-Williams <
[email protected]> wrote:

The main problem re: the streaks is they're calculated in one timezone,
but the heatmap and timeline depend on the user's timezone. Unless they all
match (which they will not for most people) it's not possible to completely
rule out inconsistencies.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/freeCodeCamp/freeCodeCamp/issues/38037?email_source=notifications&email_token=AORCKPDGRPRSQGPQ42VYHB3RC2QQFA5CNFSM4KE75LP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELZEI7Q#issuecomment-586302590,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AORCKPCC7WTFS2RUNCEP36TRC2QQFANCNFSM4KE75LPQ
.

[ojeytonwilliams]

If we do keep the streaks, I think we should calculate them client-side. That way the heatmap, timeline and streaks can all be calculated with the same timezone.

[moT01]

I think that's the way we should go @ojeytonwilliams - the streak seems popular.

[raisedadead]

Maybe you can fix definitely this problem if you
set only one time zone(heatmap and streak) fore all campers around the
world.

Unfortunately that's easier said than done. We have been trying to keep this consistent for years now.

If we do keep the streaks, I think we should calculate them client-side.

Yup, this was not possible until we released the new platform last nov. I think its doable, but I fear there will be corner cases regardless.

[mokrane-karar]

You can do a poll to see how popular it is but, at my point of view it seems to be pretty popular for those people like me who are doing the 100 days of code challenge and variants.

[raisedadead]

We understand that this feature is quite popular and also a source of frustration, if you go through all the closed issues on this repo.

From my experience of over 5 years with this project, I have not seen a perfect solution to this issue.

The reason being the timestamps for the challenges are centralised to the server for database consistency (a high priority). The client views (heatmaps, etc.) are by design required to be in a user facing timezone.

If this were distributed version controlled like git or something, it would have been so perfect because we would just capture the client timezone and timestamp. But we can't, because that would mean just like git the timestamp from the client (user) can be spoofed. Defeating the whole purpose of validation.

Now like Oliver proposed in the linked PR, we can compute things at client side, but the root cause still is that the captured data is inconsistent. Because of timezones.

In the past we have tried a zillion things including, having some "packing" or buffer around the computation, etc. But you still see the inconsistency.

That's the most simplest form of explanation by comparison I can possibly give.

We understand that the streak is a motivational indicator made popular by the 100daysofcode and other similar challenges, but unfortunately we have higher priorities than trying to solve one single difficult problem.

We sincerely welcome any constructive solutions in this thread, and would love to keep supporting the features.

[raisedadead]

The changes from Oliver's PR are live on the production platform.