Habitica: API removing users from guild sometimes does not send the complete optional message.

[cTheDragons]

General Info

User Id: bc10ec41-4213-4482-bbb8-558e3c3ea8d5
Opera 42.0.2393.137
Windows 10 32-bit

Description

When removing a member from a guild using the API, you have the option of sending an optional message. Sometimes the API cuts the message off. For example the message:

Thank you for helping me Test! Could you please rejoin the [Testing & Bug Squashing for Dragon Tools](https://habitica.com/#/options/groups/guilds/d9a0ec1e-352b-4697-a5d5-fb45c98fb4a3) Guild and stated that you were removed by iceberg. Could you let me know if you received this message by email (If attached to your account) and PM. Thanks again!

is cut to:

Thank you for helping me Test! Could you please rejoin the [Testing

It was originally thought it was the first 70 characters. However the message
1234567891123456789212345678931234567894123456789512345678961234567897123456789812345678991234567890
is send correctly and in full.

Possible the& is causing the issue.

Using the Party & Guild Data Tool it is shown that it is not the code use to call the API as the PM is sent using the same code and the entire message is intact.

There are no console errors when sending the message

[Alys]

I think your idea about the & causing the problem has a good chance of being correct. If so, this is likely to be an easy fix.

[TheHollidayInn]

Maybe the messages needs to be encoded then and is not?

[sriramsharma25]

Could I contribute here? Can someone guide me on how to get started? Thanks!

[aureo96]

Have you tried testing this theory whit a text similar to what you want to send but without the "&"? What comes to mind is that when sending the message the message is chopped into chuncks (for faster and easier reading) by http or some other protocol. So, if you do not keep the connection active only a chunck will be sent.
Note: Also are you converting the text to hex before sending it?

[Alys]

@aureo96 The successful test with the 70-character-long string implies it is the &. It's almost certainly that encoding is needed, as TheHollidayInn suggests. It's unlikely to be from the connection not staying alive long enough - we don't have that problem when sending much larger sets of data.

[aureo96]

Is the Message being sent as html, Json, or something else?

Sent from my iPhone

On Feb 3, 2017, at 1:44 AM, Alys <[email protected][e-mail]> wrote:

@aureo96https://github.com/aureo96 The successful test with the 70-character-long string implies it is the &. It's almost certainly that encoding is needed, as TheHollidayInn suggests. It's unlikely to be from the connection not staying alive long enough - we don't have that problem when sending much larger sets of data.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/HabitRPG/habitica/issues/8443#issuecomment-277184445, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYO-5vneF0NRGUbBjg51VDHIfpFry7F1ks5rYtrjgaJpZM4LqDkX.

[aureo96]

Also I recommend using wireShark to capture what is being sent and see the contents.

Sent from my iPhone

On Feb 3, 2017, at 1:44 AM, Alys <[email protected][e-mail]> wrote:

@aureo96https://github.com/aureo96 The successful test with the 70-character-long string implies it is the &. It's almost certainly that encoding is needed, as TheHollidayInn suggests. It's unlikely to be from the connection not staying alive long enough - we don't have that problem when sending much larger sets of data.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/HabitRPG/habitica/issues/8443#issuecomment-277184445, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYO-5vneF0NRGUbBjg51VDHIfpFry7F1ks5rYtrjgaJpZM4LqDkX.

[Alys]

@aureo96 WireShark is for analysing network problems, which is unlikely to be relevant to this issue.

@sriramsharma25 I'm sorry, we missed your post from three days ago. To get started, you'd install Habitica locally then do some testing with the API's removeMember route to see if & and perhaps other characters are not being encoded correctly, or to work out if the problem is from some other cause. Are you still interested in working on this? If you are, you'd be most welcome to since you asked first! If you're no longer able to, just let us know (aureo96 might then want to start investigating). Sorry about the delay!

[aureo96]

I have been giving it a thought and I regretfully decline the offer if the responsibility of fixing the issue falls to me. While I do want to help improve or fixed Habitica bugs I currently can't because I will be entering finals in a week. Plus, I am not sure if I have set up Habitica correctly in my computer. After finals I will be glad to help where I can.

Sent from my iPhone

On Feb 3, 2017, at 4:29 PM, Alys <[email protected][e-mail]> wrote:

@aureo96https://github.com/aureo96 WireShark is for analysing network problems, which is unlikely to be relevant to this issue.

@sriramsharma25https://github.com/sriramsharma25 I'm sorry, we missed your post from three days ago. To get started, you'd install Habitica locallyhttp://habitica.wikia.com/wiki/Setting_up_Habitica_Locally then do some testing with the API's removeMemberhttps://habitica.com/apidoc/#api-Group-RemoveGroupMember route to see if & and perhaps other characters are not being encoded correctly, or to work out if the problem is from some other cause. Are you still interested in working on this? If you are, you'd be most welcome to since you asked first! If you're no longer able to, just let us know (aureo96 might then want to start investigating). Sorry about the delay!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/HabitRPG/habitica/issues/8443#issuecomment-277380170, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYO-5pKK-f2LrYSQjIUanmRvxeUdcS61ks5rY6pIgaJpZM4LqDkX.

[johnpaulada]

Hi! I think I can work on this. 😄

[Alys]

@johnpaulada Great, thanks! Ask here if you have any questions or run into problems.

[johnpaulada]

Hi! How do I get gems to make a guild? Not entirely sure which field to modify to get gems.

[Alys]

Use the debug menu in the footer. There's an option for gems.

[johnpaulada]

Welp, I did not see that. LOL. Thanks!

[johnpaulada]

Hi again! Where can you see the message after you've been removed from the guild?

[Alys]

@johnpaulada Sorry about the delay on an answer for you! It's sent by email and by push notification, depending on your settings from Settings > Notifications on the website. That will make it difficult and maybe impossible to test fully on a local install since emails typically can't be sent from one but other admins might have more ideas about how to test.

[Caryyon]

How is this issue going? I'd like to lend a hand.

[johnpaulada]

I wasn't able to test it because the email isn't being sent. Go ahead. :)

[Mintzmyer]

Hi! Just checking in about the status of this issue
@Caryyon and @johnpaulada where are each of you at? I'd also like to lend a hand, if that's useful.

[johnpaulada]

Hi! I don't know how to send an email when I'm running a local install so I can test what's actually going on.

[Alys]

@johnpaulada I'm not sure it's possible (or at least not easily possible) to set things up so that emails can be sent from a local install. You might like to do a pull request with the code you have so far and we'll review it and see what testing might be needed.

[johnpaulada]

I'll do so when I found it. LOL. Not sure where I put it.

[joe-salomon2]

Hi guys, checking in on the status of this, I might be able to move it forward.

[Alys]

@joe-salomon This one is lower priority than the other one you claimed (I just commented on that one), so I'll leave this open for someone else to work on. Thank you anyway! If it's still unclaimed when you've submitted a PR for https://github.com/HabitRPG/habitica/issues/8643, then comment again here.

[johnpaulada]

Hi! Didn't really get the time to deal with the issue.

[Neromaru]

I'd like to help!) If this is still in progress ) #"New to git"#

[SabreCat]

@Neromaru Go for it! @johnpaulada relinquished claim on it, so the in-progress tag is yours. Let us know here or in the Aspiring Blacksmiths Guild if you need any assistance navigating Git or the code!

[Neromaru]

where can i get the code to watch through ?

[SabreCat]

@Neromaru You can browse the code right here on GitHub! https://github.com/HabitRPG/habitica lets you navigate through using GitHub's site.

For editing, you can get a step-by-step how-to on obtaining your own copy of the code and setting up your own Habitica server on the wiki here!

[Neromaru]

@SabreCat Sorry for disturbing, but can i find neede file quikly by using "#" sign ? And also now i'm installing Habitica server

[SabreCat]

@Neromaru What do you mean by finding files with #? I'm not familiar with that convention.

[Neromaru]

@SabreCat Sorry for disturbing again I mean after request, posted #8893 or etc and near some code file on git I also see "#" sign
Sorry, I'm really newbie to git but want to help.

[SabreCat]

Each issue ticket and pull request gets a unique ID. Referencing that ID with a # in front of it in a GitHub comment automatically creates links between the comment and the ticket or PR, and from that ticket or PR back to the comment.

I'm still not sure what you mean by code files with #. :\ If you go to the GitHub page for a file, like https://github.com/HabitRPG/habitica/blob/develop/website/server/libs/cron.js, and click a line of code, you'll get a link with a # on the end that you can use to link people to that specific part of the code--is that what you mean?

[Neromaru]

Thanks, u understood me correctly now i se that was my conprehansion issue of that thanks a lot.

[SabreCat]

OK, great! You can also click a code line number, hold Shift, and click another line number to highlight a block of code. That makes a link like this: https://github.com/HabitRPG/habitica/blob/develop/website/server/libs/cron.js#L19-L27

[Neromaru]

@SabreCat https://drive.google.com/open?id=0B_UxcC6DaSQQM2NTd3BrU3Q1Wmc
got this thing while installing server try to go on nd error connection occured

[SabreCat]

@Neromaru Looks like maybe MongoDB isn't running. You'll need to run mongod in your MongoDB install's bin directory before you start the Habitica server.

[Neromaru]

@SabreCat ok i get one thing, but when i delete member nothing comes to him, i mean notification, but in messages it goes clear

[SabreCat]

@Neromaru I believe this issue only comes up when making an API call to delete the member; it's not implemented on the current Web site.

[Neromaru]

@SabreCat accept me regretions, i failed , i'll keep trying but again, sorry, my knowledges aren't good enogh as i thought...

[SabreCat]

@Neromaru: OK, thanks for letting us know! Feel free to further hunt the "help welcome now" tag for something that might better fit your abilities.

[jchn-codes]

Hey guys! I'm interested in working on this issue. As far as I understand no one is on it?

[SabreCat]

@TheZeroNine: You are correct! Feel free to investigate, with our thanks!

[jchn-codes]

Is there a testcase for this route / function somewhere? I'm not familiar with the project setup yet and I might overlook something.

[SabreCat]

@TheZeroNine: Relevant tests can be found in https://github.com/HabitRPG/habitica/blob/develop/test/api/v3/integration/groups/POST-groups_id_removeMember.test.js . I believe the specific test is https://github.com/HabitRPG/habitica/blob/develop/test/api/v3/integration/groups/POST-groups_id_removeMember.test.js#L126 ; using an & in the query param https://github.com/HabitRPG/habitica/blob/develop/website/server/controllers/api-v3/groups.js#L776 should reproduce the issue.

[jchn-codes]

Thanks! That's what I was looking for ;)

[jchn-codes]

I've been trying to reproduce this bug with a test. I always got message = undefined with

await partyLeader.post(`/groups/${party._id}/removeMember/${partyMember._id}`, { message: 'custom remove message'});

it seems to me that req.query is always undefined for this route. after changing let message = req.query.message to let message = req.body.message it worked.
(https://github.com/HabitRPG/habitica/blob/develop/website/server/controllers/api-v3/groups.js#L883)

I guess this is an issue with my setup, because op seems to be able to send messages without & without issues. I've run the tests as described in the wiki and via PHPStorm 2017.2.

Any ideas / advices?

[SabreCat]

@cTheDragons, do you know what might be different between your and @TheZeroNine's setup?

[cTheDragons]

@TheZeroNine is your message:

'custom remove message'

? As stated above I think the issue is because there is an ampersand (&) in the message that is causing the issue.

Regarding environment, I'm using Javascript as per my Party & Guild Data Tool

I have just tested it with the latest public release. The following message:

Thank you for helping me Test! Could you please rejoin the Testing & Bug Squashing for Dragon Tools Guild and stated that you were removed by iceberg. Could you let me know if you received this message by email (If attached to your account) and PM. Thanks again!

is still emailed out as:

Greetings FireEupnea,

You have been removed from your guild "Testing & Bug Squashing for Dragon Tools". You can form your own guild or look for a new one under Social > Guilds.

Your former Guild Leader has provided the following message:

Thank you for helping me Test! Could you please rejoin the [Testing

Stay productive,
Leslie and the Habitica Team
Habitica

Follow Habitica on social media!
https://twitter.com/habitica
https://www.facebook.com/Habitica
http://habitica.tumblr.com

As you can see it is getting cut off at the ampersand (&)

The Ajax call I'm using is:

        urlToPost = serverUrl + serverPathGroup + '/' + groupId + serverRemoveUser + '/' + obj[uuidIndex] + '?message=' + removeMessage
        $('#posting #statusPost').text('Removing ' + obj[nameIndex] );
        if (debugShowObject) console.log(urlToPost);
        $.ajax({
            url: urlToPost,
            type: 'POST',
            dataType: 'json',
            cache: false,
            beforeSend: function(xhr){
                    xhr.setRequestHeader('x-api-user', userId);
                    xhr.setRequestHeader('x-api-key',  apiToken);
                },
            success: postRemoveUserSuccess,
            error: postRemoveUserFailure
        });

btw is there a way to link to a specific line of code in GitHub?

Thanks for your time with this! Let me know if I have misunderstood anything with this, I'm still very much a newbie at this.

[jchn-codes]

@cTheDragons I cannot send any message at all, the req.query is always undefined when I try to send messages. The message is in req.body so I think I'm doing something wrong, because every other route just works as expected (with the POST params being in req.query).

I will try your AJAX call on the weekend to see if it works on my side.

Yes, you can just append #L123 to the end of the github link to link to a specific line of code (e.g.: https://github.com/HabitRPG/habitica/blob/develop/website/server/controllers/api-v3/groups.js#L883).

[jchn-codes]

I'm still not able to reproduce the issue as mentioned in my previous comment.. sorry guys! =/

[SabreCat]

@TheZeroNine Sorry it didn't work out! Thanks for letting us know; I've opened this back up for help.

[TheHollidayInn]

There is a client side issue for this. I believe the issue is from the client as testing this from the api works.