Habitica: Record Yesterday's Activity (RYA) not obeying Custom Day Start time (CDS) when displaying Dailies after midnight?

[Alys]

_EDIT 2019-07-13:_ See this comment for some details about how to reproduce the bug.

_In the Report a Bug guild, thebat137 (dea8a9e8-c084-438a-9a14-7fe96a73fd0a) wrote:_

I had an interesting bug the other day related to custom day start settings and checking off yesterday's tasks at initial check-in.
My custom day start is 9 AM.
For various reasons, I didn't do my first check-in for my Habitica day until after midnight on the clock day.
When the modal for checking off yesterday's tasks popped up, it included a greyed-out task that actually wasn't due on the Habitica day I was doing final task check-off for, but was due on the Habitica day I was about to start.
I did not check it off, because it was greyed out (and I hadn't done it yet!), but I did check off all my other tasks.
I then received a small amount of damage (possibly one task's worth?) and did not get a perfect day buff for the "new" day.
It's possible that the damage and the loss of the perfect day were a result of a sync issue, but the only task I saw with a 0 streak afterwards was the greyed-out task that I hadn't checked off. So it seems that the check-off-yesterday's-tasks modal was respecting my clock day rather than my Habitica day.
...
looking at my tasks, the only one with a streak of zero right now is my "Organize the armoire" daily, so that must be it. It is due on Sunday, Tuesday, and Thursday. I think the incident happened on Tuesday night/Wednesday morning (i.e. my "Habitica day" was still Tuesday, but the clock day was Wednesday), so this would make sense.

_Note from Alys: I've confirmed that that Daily was the only one that had been marked by cron as incomplete for the past few days._

{
    "_id": "e1f7bcc5-ad6b-43d5-9c3a-9d1f6918e104",
    "userId": "dea8a9e8-c084-438a-9a14-7fe96a73fd0a",
    "text": "Organize the armoire",
    "updatedAt": "2017-10-06T01:26:14.172Z",
    "createdAt": "2017-02-25T21:57:07.732Z",
    "reminders": [],
    "group": {
        "approval": {
            "requested": false,
            "approved": false,
            "required": false
        },
        "assignedUsers": []
    },
    "challenge": {},
    "attribute": "con",
    "priority": 1.5,
    "value": 90.48727586946804,
    "tags": [
        "3a19d240-c08d-4c49-b220-8b71a8e7c951",
        "a0c8af16-48e1-422d-a3eb-c921d323288c"
    ],
    "notes": "(fold laundry)",
    "type": "daily",
    "checklist": [],
    "collapseChecklist": false,
    "completed": false,
    "history": [
        {
            "date": 1488206412082,
            "value": 12.354166666666666
        },
        {
            "date": 1488302521275,
            "value": 16.13888888888889
        },
        {
            "date": 1488486851842,
            "value": 16.13888888888889
        },
        {
            "date": 1488566749449,
            "value": 18.661234126550717
        },
        {
            "date": 1488653657090,
            "value": 18.661234126550717
        },
        {
            "date": 1489246338161,
            "value": 19.891255809232153
        },
        {
            "date": 1489749856559,
            "value": 20.491916953481063
        },
        {
            "date": 1489771880785,
            "value": 21.083403350000133
        },
        {
            "date": 1489852996092,
            "value": 21.665992092831868
        },
        {
            "date": 1489929029501,
            "value": 23.912925122261377
        },
        {
            "date": 1490019026629,
            "value": 24.492734784252946
        },
        {
            "date": 1490101744454,
            "value": 24.492734784252946
        },
        {
            "date": 1490190453646,
            "value": 25.072544446244514
        },
        {
            "date": 1490275683561,
            "value": 25.072544446244514
        },
        {
            "date": 1490363969550,
            "value": 26.385095895364056
        },
        {
            "date": 1490450202722,
            "value": 26.964905557355625
        },
        {
            "date": 1490552340874,
            "value": 27.544715219347193
        },
        {
            "date": 1490635187133,
            "value": 26.964905557355625
        },
        {
            "date": 1490744027879,
            "value": 26.964905557355625
        },
        {
            "date": 1490809603000,
            "value": 26.385095895364056
        },
        {
            "date": 1490887317130,
            "value": 26.385095895364056
        },
        {
            "date": 1491054492417,
            "value": 26.964905557355625
        },
        {
            "date": 1491147737149,
            "value": 27.544715219347193
        },
        {
            "date": 1491251882764,
            "value": 28.12452488133876
        },
        {
            "date": 1491334082944,
            "value": 28.12452488133876
        },
        {
            "date": 1491487647317,
            "value": 28.70433454333033
        },
        {
            "date": 1491588556856,
            "value": 29.284144205321898
        },
        {
            "date": 1491656923104,
            "value": 29.284144205321898
        },
        {
            "date": 1491749682696,
            "value": 29.284144205321898
        },
        {
            "date": 1491857892237,
            "value": 29.863953867313466
        },
        {
            "date": 1491939837389,
            "value": 29.863953867313466
        },
        {
            "date": 1492098337242,
            "value": 29.863953867313466
        },
        {
            "date": 1492733746241,
            "value": 30.443763529305034
        },
        {
            "date": 1492788728402,
            "value": 31.023573191296602
        },
        {
            "date": 1492898277520,
            "value": 31.023573191296602
        },
        {
            "date": 1493041764262,
            "value": 31.60338285328817
        },
        {
            "date": 1493129477398,
            "value": 32.18319251527974
        },
        {
            "date": 1493225252772,
            "value": 32.76300217727131
        },
        {
            "date": 1493306528240,
            "value": 32.76300217727131
        },
        {
            "date": 1493410024664,
            "value": 32.18319251527974
        },
        {
            "date": 1493487516993,
            "value": 32.18319251527974
        },
        {
            "date": 1493647794963,
            "value": 32.76300217727131
        },
        {
            "date": 1493737600887,
            "value": 32.76300217727131
        },
        {
            "date": 1493852009132,
            "value": 32.18319251527974
        },
        {
            "date": 1493929171956,
            "value": 32.18319251527974
        },
        {
            "date": 1494000821440,
            "value": 31.603382853288174
        },
        {
            "date": 1494151774835,
            "value": 31.603382853288174
        },
        {
            "date": 1494435885594,
            "value": 31.023573191296606
        },
        {
            "date": 1494508159030,
            "value": 31.603382853288174
        },
        {
            "date": 1494595862856,
            "value": 32.18319251527974
        },
        {
            "date": 1494684058834,
            "value": 32.18319251527974
        },
        {
            "date": 1494795472414,
            "value": 32.18319251527974
        },
        {
            "date": 1495015970010,
            "value": 31.603382853288174
        },
        {
            "date": 1495463801118,
            "value": 32.18319251527974
        },
        {
            "date": 1495574371928,
            "value": 32.174382522670214
        },
        {
            "date": 1495670495644,
            "value": 32.75419218466178
        },
        {
            "date": 1495817216788,
            "value": 33.33400184665335
        },
        {
            "date": 1495923111733,
            "value": 33.33400184665335
        },
        {
            "date": 1496149757012,
            "value": 33.91381150864492
        },
        {
            "date": 1496410049117,
            "value": 35.073430832628055
        },
        {
            "date": 1496519138661,
            "value": 35.653240494619624
        },
        {
            "date": 1496674468798,
            "value": 36.23305015661119
        },
        {
            "date": 1496773739384,
            "value": 36.23305015661119
        },
        {
            "date": 1496861311246,
            "value": 36.81285981860276
        },
        {
            "date": 1496927558826,
            "value": 36.81285981860276
        },
        {
            "date": 1497296038988,
            "value": 37.39266948059433
        },
        {
            "date": 1498138616625,
            "value": 37.9724791425859
        },
        {
            "date": 1498398700911,
            "value": 37.9724791425859
        },
        {
            "date": 1498399405556,
            "value": 37.9724791425859
        },
        {
            "date": 1498408241379,
            "value": 37.9724791425859
        },
        {
            "date": 1498517883667,
            "value": 38.552288804577465
        },
        {
            "date": 1498606766023,
            "value": 38.552288804577465
        },
        {
            "date": 1498661676695,
            "value": 39.13209846656903
        },
        {
            "date": 1498749254717,
            "value": 39.13209846656903
        },
        {
            "date": 1498847965379,
            "value": 39.7119081285606
        },
        {
            "date": 1498849262166,
            "value": 40.29171779055217
        },
        {
            "date": 1498956637489,
            "value": 40.29171779055217
        },
        {
            "date": 1499006923906,
            "value": 40.29171779055217
        },
        {
            "date": 1499359664773,
            "value": 40.29171779055217
        },
        {
            "date": 1499461412255,
            "value": 44.964791701248394
        },
        {
            "date": 1499529166826,
            "value": 45.54460136323996
        },
        {
            "date": 1499626047169,
            "value": 46.12441102523153
        },
        {
            "date": 1499721195875,
            "value": 46.7042206872231
        },
        {
            "date": 1499946184141,
            "value": 46.12441102523153
        },
        {
            "date": 1499964367105,
            "value": 46.12441102523153
        },
        {
            "date": 1500053463429,
            "value": 46.7042206872231
        },
        {
            "date": 1500149398398,
            "value": 47.28403034921467
        },
        {
            "date": 1500221867801,
            "value": 47.28403034921467
        },
        {
            "date": 1500337624439,
            "value": 47.863840011206236
        },
        {
            "date": 1500442884814,
            "value": 47.863840011206236
        },
        {
            "date": 1500490379648,
            "value": 48.443649673197804
        },
        {
            "date": 1500575292819,
            "value": 48.443649673197804
        },
        {
            "date": 1500754599411,
            "value": 48.443649673197804
        },
        {
            "date": 1500840179157,
            "value": 48.443649673197804
        },
        {
            "date": 1500906429561,
            "value": 49.02345933518937
        },
        {
            "date": 1500996251927,
            "value": 49.02345933518937
        },
        {
            "date": 1501079459105,
            "value": 49.60326899718094
        },
        {
            "date": 1501164373749,
            "value": 49.60326899718094
        },
        {
            "date": 1501257252889,
            "value": 50.18307865917251
        },
        {
            "date": 1501356697413,
            "value": 50.18307865917251
        },
        {
            "date": 1501540483530,
            "value": 50.76288832116408
        },
        {
            "date": 1501596146139,
            "value": 51.342697983155645
        },
        {
            "date": 1502141880831,
            "value": 51.92250764514721
        },
        {
            "date": 1502203556824,
            "value": 51.92250764514721
        },
        {
            "date": 1502307300699,
            "value": 52.50231730713878
        },
        {
            "date": 1502379757055,
            "value": 52.49350731452925
        },
        {
            "date": 1502481681793,
            "value": 53.07331697652082
        },
        {
            "date": 1502568089112,
            "value": 53.07331697652082
        },
        {
            "date": 1502847751225,
            "value": 53.07331697652082
        },
        {
            "date": 1502903907475,
            "value": 53.65312663851239
        },
        {
            "date": 1502981077624,
            "value": 54.23293630050396
        },
        {
            "date": 1503159601579,
            "value": 54.23293630050396
        },
        {
            "date": 1503239523072,
            "value": 54.812745962495526
        },
        {
            "date": 1503331611605,
            "value": 57.43918774883942
        },
        {
            "date": 1503411858956,
            "value": 57.43918774883942
        },
        {
            "date": 1503881012033,
            "value": 59.32729963695131
        },
        {
            "date": 1503929322375,
            "value": 62.2465320550323
        },
        {
            "date": 1504041505035,
            "value": 62.2465320550323
        },
        {
            "date": 1504198443436,
            "value": 64.8729738413762
        },
        {
            "date": 1504359903427,
            "value": 65.45278350336777
        },
        {
            "date": 1504362649302,
            "value": 65.45278350336777
        },
        {
            "date": 1504447327865,
            "value": 65.45278350336777
        },
        {
            "date": 1504630225617,
            "value": 66.03259316535934
        },
        {
            "date": 1504820957706,
            "value": 66.03259316535934
        },
        {
            "date": 1504880212295,
            "value": 68.50051471546281
        },
        {
            "date": 1504969301009,
            "value": 68.50051471546281
        },
        {
            "date": 1505070174947,
            "value": 71.12695650180672
        },
        {
            "date": 1505145078017,
            "value": 71.70676616379829
        },
        {
            "date": 1505246817413,
            "value": 71.70676616379829
        },
        {
            "date": 1505324264829,
            "value": 72.28657582578987
        },
        {
            "date": 1505501278273,
            "value": 72.86638548778144
        },
        {
            "date": 1505587856420,
            "value": 76.9596497364861
        },
        {
            "date": 1505740852724,
            "value": 78.847761624598
        },
        {
            "date": 1505744106173,
            "value": 79.42757128658957
        },
        {
            "date": 1505852609720,
            "value": 79.42757128658957
        },
        {
            "date": 1505928130133,
            "value": 80.00738094858114
        },
        {
            "date": 1506007475985,
            "value": 82.05401307293347
        },
        {
            "date": 1506191229726,
            "value": 82.05401307293347
        },
        {
            "date": 1506265689847,
            "value": 82.63382273492505
        },
        {
            "date": 1506378472716,
            "value": 83.21363239691662
        },
        {
            "date": 1506630422352,
            "value": 83.21363239691662
        },
        {
            "date": 1506700104174,
            "value": 87.88670630761285
        },
        {
            "date": 1506820895657,
            "value": 87.88670630761285
        },
        {
            "date": 1506869098238,
            "value": 87.88670630761285
        },
        {
            "date": 1506953777043,
            "value": 88.46651596960443
        },
        {
            "date": 1507057324384,
            "value": 89.046325631596
        },
        {
            "date": 1507177079801,
            "value": 88.45770597699492
        },
        {
            "date": 1507221067223,
            "value": 88.45770597699492
        }
    ],
    "streak": 0,
    "repeat": {
        "su": true,
        "s": false,
        "f": false,
        "th": true,
        "w": false,
        "t": true,
        "m": false
    },
    "startDate": "2017-02-25T05:00:00.000Z",
    "everyX": 1,
    "frequency": "weekly",
    "__v": 210,
    "daysOfMonth": [],
    "weeksOfMonth": [],
    "isDue": true,
    "nextDue": [
        "Sun Oct 08 2017 00:00:00 GMT-0400",
        "Tue Oct 10 2017 00:00:00 GMT-0400",
        "Thu Oct 12 2017 00:00:00 GMT-0400",
        "Sun Oct 15 2017 00:00:00 GMT-0400",
        "Tue Oct 17 2017 00:00:00 GMT-0400",
        "Thu Oct 19 2017 00:00:00 GMT-0400"
    ],
    "yesterDaily": true
}

[thebat137]

I believe I just experienced this bug again.

As I noted the last time, I have a 9 AM custom day start. I didn't do my first check-in for today ("Monday") until after midnight (i.e., real-world "Tuesday" but still Habitica "Monday"). The "record yesterday's activity" modal (i.e., finishing up "Sunday") presented me with a grayed-out task that's not normally scheduled on "Sunday", but is scheduled on "Monday" ("Collect the post"). I checked it off because I suspected something was funny and didn't want to lose my streak or my perfect day buff. I also checked off all the other tasks that the modal presented to me but somehow I took damage anyway (maybe a couple tasks' worth?). Looking at my tasks, the only ones that had their streaks set to zero ("Beat the rugs", "Organize the armoire") seem to be two that are normally scheduled for "Sunday" (and not "Monday"), but these tasks did not appear in my RYA modal, so I was unable to check them off.

Anyway, just offering this up in case this second incident might provide more data to help solve the problem.

[Alys]

Another report of this from @Aerilita (3b724a41-48d3-4048-ae27-70b258cf5a72):

"I didn't interact with Habitica on Wednesday (I'm pretty sure I closed the site before local midnight on Tuesday). My Custom Day Start is 4am. It is 2am now. So it is after midnight and Thursday by calendar, but Habitica-wise, should be Wednesday still. When I opened the site, I got the RYA screen, supposedly for Tuesday. However, there were two Wednesday dailies in it, grey. I did a hard reload, the RYA showed up exactly the same, dailies persisted, still gray. Because they didn't appear due and besides, I'm in the inn, there's no damage from it, but it's still not right. It's probably a known bug, not sure if my report helps, but just in case."

I'll be doing some testing for it on a test account over the next few days and I'll report what I find here next weekend.

_EDIT 2019-07-09: For my reference, see the email "Weird RYA pop up on the website" with my tag on it._

[MeanderingCode]

@alys When doing the shared dailies work, I discovered helpers for CDS. I don't know if they would be useful here, but I had to export them before using them client side. See:

https://github.com/HabitRPG/habitica/blob/e0977e3afb62930290bb1a0c41b05d4068135434/website/common/script/ops/scoreTask.js
https://github.com/HabitRPG/habitica/blob/e0977e3afb62930290bb1a0c41b05d4068135434/website/common/script/cron.js
https://github.com/HabitRPG/habitica/blob/e0977e3afb62930290bb1a0c41b05d4068135434/website/common/script/index.js

[Alys]

@MeanderingCode Ah thanks so much! I bet that will help.

BTW, if you're interested in taking on this issue, just let us know. :)

[MeanderingCode]

Might...Just as soon as I write more tests and the last bit on shared dailies ;)

I'll let you know if so.

[Alys]

RYA is definitely not always showing the correct Dailies when the CDS is non-zero. An easy way to test for this on your local install is to:

• create seven Dailies with names like "due Monday"
• set each them to be due on just the day specified in its name
• make sure none of them have been completed
• set all their streaks to a non-zero value (so that you can easily tell which one cron causes damage for) db.tasks.update({ userId:'12345....', type:'daily'}, {$set:{ 'streak': 9 }},{multi:true});
• set your lastCron date to two or more days in the past db.users.update({_id:'12345....'},{$set:{ 'lastCron': new Date('2019-07-01') }});
• set your CDS to a time greater than the current time (this means that it's still "yesterday" within Habitica) db.users.update({_id:'12345....'},{$set:{ 'preferences.dayStart': 10 }});

Reloading the website shows RYA with the wrong Daily appearing, and that Daily is grey. The grey colour is actually correct for that Daily; the problem is that the Daily should not appear. The Daily that should appear does not.
For example, if today is real-world Saturday and the current time is 9am, you'd set your CDS to 10am and reload the website. Because the current time is less than your CDS, it's still Friday within Habitica, so RYA SHOULD show you the Daily for Thursday. However RYA shows you the Daily for Friday, and it's grey (which is correct because that Daily was not due on Thursday in Habitica).
When you click the "Start My New Day" button on the RYA screen, cron runs, and if you examine your Dailies' streaks after that, you'll see that the Daily for Thursday had its streak reduced to zero (which is correct), and the the Daily for Friday still has a non-zero streak (also correct). I.e., cron itself is correctly determining which Dailies were due but RYA is not.

I haven't looked at the RYA code in detail but I would guess that the comment above from MeanderingCode would be a good place to start investigating.

If anyone wants to work on this and finds that this comment isn't clear enough, please ask here. :)