Signal-android: identical message displayed multiple times

[sbugreport]

I have:

• [x] searched open and closed issues for duplicates
• [x] read https://github.com/WhisperSystems/Signal-Android/wiki/Submitting-useful-bug-reports

---

Bug description

When sending a Signal message (not SMS) from a heavily overloaded (WIFI) network the client attempts to send the message multiple times (expected) and eventually displays a red "!" ("Not delivered. Tap for details").
Apparently the message went through nonetheless and not just once but multiple times and is displayed at all recipients multiple times (group chat).

We observed this problem within multiple group chats and was triggered by distinct senders.
(I personally observed it in two distinct group chats, friends also confirmed this in other group chats.)

Note: the sender did _not_ press any of the "RESEND" buttons.

Steps to reproduce

• reproducing the exact network failure rates/environment is not really possible, but here are the simple steps

• sender: try to send a message
  while the 3 dots show that the client attempts to still send

• recipient: displays the same message (by the same sender) multiple times

Actual result: Message is displayed multiple time (between 6 and 15 times in our cases)
Expected result: The recipient detects that this is the identical message (via a message id, replay protection?)

Device info

Sender's Signal version: 3.25.3 and others
Recipient's Signal version: 3.25.3 and others

[flothemaker]

I can confirm such a behavior, but I also didn't know a procedure to reproduce.

In my experience the network strength/stability (not only WIFI, also mobile data) of the sender is a key factor for such a result.

[johanw666]

This sounds the same as https://github.com/SilenceIM/Silence/issues/498, which is solved in Silence in https://github.com/SilenceIM/Silence/commit/8cc6e72edac3d88b76ce2e449206751237fd8e67 .

[haffenloher]

@johanw666 that's unrelated, it's about how actual duplicate messages (i.e. with the same ciphertext, not only the same plaintext) are displayed in the inbox

[riyapenn]

Also, you had removed the portion of the issue template regarding debug logs. Please provide debug logs from both sender and a receiver. Related: https://github.com/WhisperSystems/Signal-Android/issues/5355

[JohnnyMT]

also sounds like my issue in #5891

[dreamflasher]

I can confirm this behaviour. I've reproducible observed this with bad internet connection, it looks like that messages get resent for every user in the group chat, so they end up appearing 10 times (when a group has ten participants). (Unfortunately, this pissed off a bunch of people I tried to convince of encrypted IM, so they ended up using Whatsapp -.-)
An easy fix could be to just ignore duplicate messages in the client? (eg. don't show a message if it has the same string as the previous message from that user if it was within the last 5 minutes)

[riyapenn]

@DreamFlasher Please refrain from responding with "I can confirm" "me too" "+1" if you do not have any new or actionable information. Debug logs from the sender and a receiver receiving the "duplicate" messages are the most useful.

If you really want, you can "Add your reaction" using the emoji icon in the header of the comments OR you could choose to subscribe to notifications for this issue by choosing so in the options of the right pane (when viewed in a browser).

[dreamflasher]

@riyapenn Why? Can you please refer to any WhisperSystems specific issue tracking style guide?
I am asking because everywhere else my comment would have been seen as useful and helpful.

[riyapenn]

read https://github.com/WhisperSystems/Signal-Android/wiki/Submitting-useful-bug-reports

[2-4601]

Just to remind, the original report says:

Note: the sender did _not_ press any of the "RESEND" buttons.

@DreamFlasher Did the group members in your case hit the resend button? If they did, that's a separate issue #2639.

[JohnnyMT]

Here another debug log created right after this error happened again...
https://gist.github.com/anonymous/7467d036968ebdd77dcc6b81986abc18
and I can confirm, it happened WITHOUT hitting the resend button.
Does it help if I post more of this? I get this error almost every day, so I can give you lots of logs ;-)

[haffenloher]

@JohnnyMT That log is from the sending phone, right?
Lots of RateLimitExceptions in there.

[dreamflasher]

read https://github.com/WhisperSystems/Signal-Android/wiki/Submitting-useful-bug-reports

@riyapenn I read it again (already read it previously, before filing other bugs), I couldn't find anything that the WhiserSystems open source community doesn't find reproducing bugs and providing possible solutions not helpful. Could you please specify which part of the link you provided refers to this?

@DreamFlasher Did the group members in your case hit the resend button? If they did, that's a separate issue #2639.

@2-4601 Thank you for confirming, yes that is exactly why I posted here.

[Trolldemorted]

This is a duplicate of #5579, isn't it?

@JohnnyMT I do not think so.

I am pretty sure (i have no confirmation from moxie or anyone else) the problem is:

• PushSendJobs have the retry count 5
• In case an exception during onRun() is thrown, the jobmanager decreases the retry count, and asks the task (onShouldRetry) if it should be run again (if count>0)
• IOExceptions from libsignal are converted to RetryLaterExceptions
• These RetryLaterExceptions are never handled, thus the jobmanager asks the PushTextSendJob if it should retry: if (exception instanceof RetryLaterException) return true;

So any IOException during sending will cause reencrypting and resending (for a maximum of 5 times), completely without user actions. If the server received a message despite the IOException (which obviously happens from time to time), you will send a duplicate message.

Since the problem is pretty obvious (and the logs don't contain anything useful, because failing/retrying jobs are not logged (?)), more debug logs will not contribute to solving this problem faster.

[haffenloher]

I can reliably reproduce this now. If there's an unregistered member in the group, Signal requests prekeys for that user on every outgoing group message. After a few messages in quick succession, the server's rate limiting kicks in. As @Trolldemorted noted, the jobmanager then tries to send 5 times. I think this needs to be caught in libsignal to be able to properly continue sending to the remaining group members after the RateLimitException is thrown. I'll take a look.

[Trolldemorted]

@haffenloher i completely forgot about the prekey-ratelimits, your PR looks good to me!

If your PR and the sync messages fix get merged, groups with unregistered users should no longer be unusable.

[moxie0]

@haffenloher Problem is that if you encapsulate that exception, you can't just ignore it in the send job. The purpose of those encapsulated exceptions is generally to discern which failures applied to which recipients. In the case of someone unregistering, however, no "retry" action is appropriate.

Instead we should probably encapsulate the original unregistered failure, and have the send job handle that by removing the user from the group.

[haffenloher]

@moxie0 unregistered failures are already encapsulated and ignored, there's no error message or retry action, so I thought we could handle rate limit failures like that as well. But yeah, it feels more like a workaround than a fix...

would you want to remove the user locally only? Or should a group update be sent out?
We'd need to think about how to handle the case of someone temporarily unregistering, being removed from groups, re-registering, and then thinking they're still a member while all the others do not.

[JohnnyMT]

@haffenloher jep, those debug logs are always from the sender phone (me).

@moxie0 removing a member from a group is a long time requested feature, especially for people not using Signal anymore but didn't unregister ;-)
I would really appreciate it if you can implement that.

Are you sure, the message gets repeated "only" five times in a row? I have in mind that it got repeated until I switched my phone to flight mode.
I will check this tomorrow, since I have this issue nearly every day, twice today...

One thing is still strange for me... the group has a total of 6 users, but I'm the only one experiencing this "spam bug", no other user spams the group.
This started when I got a new phone and they updated the group for me.
I also realized, that I only happens, after I successfully sent 1-3 messages after a "conversational break".
Meaning: somebody send a message for the first time of the day, I can reply with 1-3 messages, after that, it starts to fail, and doesn't stop until ich go to flight mode and wait for several minutes (just switch to flight mode and back will not work, you have to wait a few minutes) or reboot my phone.

[Trolldemorted]

@haffenloher If you send a group update without all previous members, it will be ignored by all receiving clients iirc. If you remove the user locally, all your subsequent group updates will be ignored because they do not contain the unregistered user.

@JohnnyMT you only run into rate limits when you send several messages in a given timeframe. One message is ok, 3-4 are not. If your group members do not send messages as rapidly as you do, they will not encounter this bug. Signal-Desktop is struggling with unregistered group members too: https://github.com/WhisperSystems/Signal-Desktop/issues/989

[flothemaker]

Here is another log:
https://gist.github.com/anonymous/2b75bce09c156987ec1fe7232861fe13
I was the receiver of two identical messages in a group chat.

[flothemaker]

And another one:
https://gist.github.com/anonymous/ef42b4acab19221303b03077ed215a87
I was the receiver of three identical messages in a one-on-one chat.

[Trolldemorted]

@braaccckkk the issue is already identified and there is a PR that mitigates it, we do not need further logs. But thanks for your assistance!

[haffenloher]

@Trolldemorted duplicates in a one-on-one chat are most likely a separate issue (i.e. not linked to prekey rate limiting)

[Trolldemorted]

i thought this issue was specifically for the limit-duplicates, but i am also pretty sure all other (except for the resend button) duplicate messages are caused by the same problem.

If SignalServiceMessageSender.sendMessage throws an IOException (which happens during poor network connectivity), the PushTextSendJob catches it and throws a RetryLaterException, and the jobmanager happily restarts the job.

Imo we should have changed libsignal to offer a way of resending w/o reencrypting, but we appear to have bugged moxie enough that he implemented filtering duplicates by their timestamp on the receiving end, if i interpret the commit correctly. That should hide most duplicate failures to the user, including the ones from this issue, since i believe the timestamps are not modified when a job is restarted.

[JohnnyMT]

After the latest update, I still have this issue on the sender side, but I don't have the option to resend the message:

So it shows that something went wrong, but I don't have further info/can't do anything about it.

[Trolldemorted]

@JohnnyMT sounds like a seperate issue. Was the message sent 5 times?

[rainerzufall]

I had a similar problem of identical messages displaying several times and a wrong error message. The debug log after a false error message: https://gist.github.com/anonymous/4d1b044ca039aa3f4fcb03c235838bc3

What happened:
A group with 8 participants (including me and an unregistered number). Messages have been exchanged. no Problem. First one of my messages gets received 5 times by other group members (Without displaying an error). Later I get an error message, although my message was received 5 times on the phone of the other participants. When looking at my Desktop version my messages are not received at all. (desktop log: https://gist.github.com/anonymous/f728cc07bd01aebb5ab9ec55abe5a220)
I have not tinkered with any custom ROM or Backup thing. internet connection was good all the time.
Will 7e51d61 solve this problem?

ps. I hope i filed my report in the correct issue. If not please let me know and i will move it.

Device : Fairphone FP2 (FP2)
Android : 5.1 (r4275.1_FP2_gms72_1.11.1, r4275.1_FP2_gms72_1.11.1)
Memory : 89M (4.98% free, 128M max)
Memclass: 128
OS Host : SGSGP60SLV1
App : Signal 3.27.1

[Trolldemorted]

@rainerzufall messages that hit ratelimits on unknown users are sent 5 times as described here: https://github.com/WhisperSystems/Signal-Android/issues/6003#issuecomment-271435623

messages in groups with unknown users are not sent to slave/sibling devices as described here: https://github.com/WhisperSystems/Signal-Android/issues/4859#issuecomment-221949214

7e51d61c7900e9d57c15eb6a6d595103bfdb7f79 will make the receiving Signal-Android devices hide messages it considers duplicate, all others will behave as usual.

[haffenloher]

This has been fixed server-side (see https://github.com/WhisperSystems/Signal-Android/pull/6175#issuecomment-280738291), so the issue can be closed.