Habitica: Member Counts Inaccurate in Guilds and Parties

[lemoness]

Currently, there are some discrepancies with member counts for guilds and parties, often showing more members than usual.

~If you're experiencing this bug, just ignore it for now :) We are aware of it and will go through and fix the member counts once we have handled the more urgent bugs.~

_edit by Alys 2018-01- 08, taken from TheHollidayInn's comment below:_

This is the desired fix:

change the group remove, leave and join routes to recompute memberCount each time it needs to be updated (i.e., count all existing members rather than add/subtract one to the current value of memberCount)

[Alys]

~This will be fixed by a database migration, so it's not a normal code bug that contributors can work on.~
_EDIT: We've changed our intentions for fixing this. It is now a normal code fix._

[Melissarie]

We are also having this issue, but more specifically none of our party members can see an active member with the username Lunarqueen. At this point not even she can see herself in the party but she can still accept quests and participate, her ID# is eceef4dc-8009-43ad-b5e9-d7149909acbb

[Alys]

@Melissarie Your party seems to have the correct member count. I can find 80 players who are in that party and the member count is set to 80. Not being able to see some members of the party and Lunarqueen not being able to see herself in the member list isn't related to the member count. I'm guessing you can see only 30 players - that's a deliberate limitation (at the moment at least) to reduce the load on the database when displaying very large parties like yours. If you have questions about it or would like more information, on the website go to Help > Report a Bug, which will take you to our new bug report guild. This isn't strictly a bug, but I know it might feel like one to affected players, so that's a good place for us to talk about it. :)

[Melissarie]

@Alys Thanks for looking into this! Is there anyway for the 30 players shown to be active members? Most of the visible players are all long inactive and it makes it very hard to know who is in a quest when it will not show all the members. Ex. The last quest was only showing 7 players even though I know there were at least 10 people on the quest. I know it's mostly just due to how large the party is but thanks again for checking into the problem.

[paglias]

I'm supposing this has been fixed. Reopen if it hasn't

[Alys]

I've been seeing a few reports of this lately. Some could be from old glitches, but there's one report today of a recently created party, from airylef (28d8a117-dc7a-4699-a578-e8d6541a640e) in the Report a Bug guild:

"I created a party and invited my husband. He accepted, but it's saying there's 3 members. When I click on the list, it just shows us. I think some bug occurred that let him accept twice or something? Is there a way to fix this? It's showing as 3 everywhere even though there's only 2."

I confirmed the member count was wrong and fixed it. The party didn't exist on 2017-12-25 (the most recent time I captured data for group plan troubleshooting), so this is a recent bug. Party ID 459de02e-ffcc-4959-849b-2f9c78a98246

[TheHollidayInn]

The field is computed, so we should probably just change the remove, leave and join routes to recompute the member count since those endpoints probably don't get used a ton from a single user. If we see performance issues, we can always update for those scenarios.

[Alys]

I just found a guild (e5fc71ad-00e4-4081-b578-576b05b050d7) where the memberCount was zero even though it had one member (the original leader). It had been created yesterday and is now deleted due to the request of the person who created it. It was a group plan guild (so here's a link to https://github.com/HabitRPG/habitica/issues/10124 for cross referencing), but the quantity field and payment was correct (3, $9), so this was more of a memberCount bug than a payment bug.

[veeeeeee]

Changed priority because this likely touches on Group Plans.

[Alys]

It's possible that when the member count is wrong, it causes the member list to display duplicates of some members. Below are posts from the Report a Bug guild indicating that that might be happening. That's not something that would need an extra fix (the preferred fix is still to just make it impossible for the member count to be wrong) but I'm recording it here so we know about it (especially if it turns out to not be related to the memberCount bug).

@Kishmet (e8c0df54-ba47-4ab9-90a8-4200780babbd): "I am currently leader of the party... We have four members which are counted twice and also show twice within the 'partycounter' could one of you guys have a look at this as time allows?"

@alys: "I've adjusted the member count... Having members showing twice isn't something that we hear about often. Is that when you click on the 'Member List' badge to see the full list of members? Can you reload the website with your browser's refresh button and tell me if that's still happening?"

@Kishmet: "It seems like everything is fine again :) maybe some helpful explanation: this was no temporary bug (related to refreshing) but this bug was around for quite a while already (since the major party bug that happened a few days/weeks back) The members that were counted twice but changed sometimes... also the number of members counted twice changed... in the beginning there was only one member counted twice and in the end it were four (that's when I wrote it here)."

[jackieklaura]

Hey there, I had this issue in the _Hardware Hackers_ guild (59d2df2b-1c6b-416f-8fb7-deb1ef31d388) recently when I removed inactive members. Was in contact with @Alys already in the _Report a Bug_ guild. Now I tried again to remove one of the members who could not be removed (now only one at a time) through the Party & Guild Data Tool, and now I have the same issue again. Now a member count of 94, while there still are 95 that get fetched.

I already tried to set the memberCount with a POST request to https://habitica.com/api/v3/groups/59d2df2b-1c6b-416f-8fb7-deb1ef31d388 and the following request body:

{
    "memberCount": 95
}

Although I did get back a 200, nothing changed in the guild data. But ok, that was just a quick try.

But now that I tried to remove just one single user I got an error alert from the data tool saying:

BadRequest
User validation failed

After trying do delete them again, there was the same error. And the memberCount was again decremented by 1.

So while I so far understood that this is due to the currrent database design not supporting transaction, this could still in code be mitigated by first removing a member and afterward decrementing the memberCount and only if the first request was successful. Not sure if this is an easy fix. Unfortunately atm I don't have a lot of spare time to dig into the backend code.

But on track with debugging again. When I tried to remove the same member once more through a manual POST request (using _Postman_) to https://habitica.com/api/v3/groups/59d2df2b-1c6b-416f-8fb7-deb1ef31d388/removeMember/fd9cf005-fd01-45b7-a0d8-20a11f252895?message=justaplaceholderforbrevity, I get back a _400 Bad Request_ with the following response body:

{
    "success": false,
    "error": "BadRequest",
    "message": "User validation failed",
    "errors": [
        {
            "message": "invalid url",
            "path": "url",
            "value": "http://localhost/api/habitica"
        },
        {
            "message": "invalid url",
            "path": "url",
            "value": "https://localhost:3045/api/habitica"
        }
    ]
}

And again the memberCount of the guild was decremented. Just in case I also tried without the message parameter in the query. Same result.

I also tried to join the guild with a test account. The memberCount increased accordingly by one. When I removed the newly joined test member again manually (same method as above), that worked too. memberCount was again decreased by 1 and I got a _200 OK_ response with following body (with the notifications cut out here due to privacy and being irrelevant in this case):

{
    "success": true,
    "data": {},
    "notifications": [
    ],
    "userV": 67143,
    "appVersion": "4.130.2"
}

Well, that is all I could test for now. I hope this is not all just redundant information. But I thought I'll post ist, as I didn't find a corresponding description here related to my bug experience. Might also be that the failing user removal is alltogether a different bug than the thing with memberCount being off, but that's how I've experienced it now for the second time in the Hardware Hackers guild. If I can help with any more debugging that does not necessitate digging deeply into the backend code, let me know.

And if there is any way to trigger a new memberCount calculation client side, I can try to come up with something useful to merge into the Party & Guild Data tool so that people can fix it themselves.

[jackieklaura]

Ahem, and one more thing. Kind of funny. But now I just wanted to try to remove another inactive member that was not removed back then in my first attempt to batch remove around 160 members. Although, not completely sure if I included them (was on the fringe of our 1 year idle time criterium). Nevertheless, this worked, and now the other member I already tried to remove several times (described above, with the ID _fd9cf005-fd01-45b7-a0d8-20a11f252895_) also was gone and the memberCount was correct again. But that was not the case when I removed my own test account member just a few minutes before.

[Alys]

@jackieklaura Thanks for the detailed information!

From the "localhost" addresses in the response body, I think postman might have been doing something odd.

Anyway I've fixed your guild's member count again. Thanks for testing if you could do it, shame it didn't work. You're welcome to post to the Report a Bug guild the next time it needs doing.

[jackieklaura]

Hey @Alys , thanks for fixing. But when did you fix it? Already in the few minutes between my last two posts? That would explain it. Otherwise something slightly weird (or magITal) happend, as described in my last post. But yes, if I need a quick memberCount fix I'll let post in the Report a Bug guild. This here is just to provide more insight on the bug, and because I'm really curious what causes it.

And the "localhost" part does not come from postman. That is the exact response data I got from the server. Should be just the same when I do it e.g. from the CLI with curl. Only can't test it in this specific case, because the user now is already removed from the guil. But postman provides the unfiltered response body.
It looks more like that this comes from some backend part behind a reverse proxy. Also one time there is localhost without TLS and then there is localhost:3045 with TLS. Also the /api/habitica seems to be something internal as the public API has its endpoints under /api/v3.

From that I would guess that it might be either a bug in code or in the setup of the webservers, probably some proxy and rewrite rules. The weird thing is that it only happens with a few certain accounts. The only significant difference between the member that could not be removed directly and other members is, that their _auth.local.username_ property is null and also their _flags.verifiedUsername_ is false. But I don't know if among the other around 160 members I bulk removed have been others where this is the case.

This is just all the details I could gather from this bug, that might help if someone wants to debug the backend code/setup.

[Alys]

@jackcogdill I fixed the membercount just before I made my previous post (about 9 hours ago), so definitely not between your two earlier posts from 12 hours ago.

The localhost thing I can't explain.

We recently discovered a bug where you can't remove a party/guild member if they haven't validated their @Username (these are players who haven't used Habitica for many months) - that will be the null/false thing that you described. I can't find the issue for it though. Maybe it only came up in the Report a Bug guild and hasn't been logged in GitHub yet. I'll look into logging it on the weekend if I still can't find it.

[Alys]

@jackcogdill I can't actually replicate the issue with removing members where flags.verifiedUsername is false. If you notice problems with removing players again, tell us in the Report a Bug guild and we'll see if we can work out what's going wrong.

[SabreCat]

Closing this overburdened ticket and replacing with a clean version--see above!