Signal-desktop: Linux: Message download speed is slow

[Mushoz]

[X] I have searched open and closed issues for duplicates

I was active in issue #1990, but that one was closed. I was instructed to use #3010, but that issue was closed for new contributors (up until this point I had nothing to add, hence I am not considered a contributor for the new issue yet). When asked about it, I was instructed to open another issue, so here it is.

Bug Description

Whenever I haven't used the desktop client for a while, it needs to load thousands of messages. At ~10 messages a second, it can take minutes to be able to actually use the application. Topic #3010 has asked people to test with Signal 1.22, and I would like to share my experiences.

While it helps with large attachments on slow internet connections, without any attachments or on fast connections such as mine, I am not really seeing any improvements compared to previous versions. From what I see while it's loading, the process doesn't seem to be properly multithreaded. Suggestions for a better experience:

1. Utilize multiple threads to load new messages or apply other optimizations to speed up the process.
2. Instantly open the application itself, so I can at least send a quick message if I need to, and download the new messages in the background.
3. In addition to 2., download the messages in reverse order, so new messages will be visible the earliest, ready for me to reply to them.

Steps to Reproduce

1. Use Signal actively on my phone for a (few) week(s)
2. Start Signal desktop on my desktop

Actual Result:

Loading the new messages takes minutes.

Expected Result:

Loading the new messages should take a few seconds tops with the specs I am using, and/or it should happen in the background so I can start using the application the moment I open it.

Screenshots

N/A

Platform Info

Signal Version: 1.22

Operating System:

Arch Linux, 4.20.11 kernel

Linked Device Version:

Android 4.34.8

Link to Debug Log

https://debuglogs.org/e4cfa43d55f8ae57b040e8d5d0c6a52839f07b6f41f327df34cec332bc15adff

[scottnonnenberg-signal]

Looking at the log, I do see message processing taking longer than I expect it to:

INFO  2019-02-21T19:11:10.484Z onProgress: Message count is 10
INFO  2019-02-21T19:12:20.420Z onProgress: Message count is 530
  70 seconds for 530 messages
  ~132ms per message

INFO  2019-02-23T11:31:23.379Z onProgress: Message count is 10
INFO  2019-02-23T11:32:54.859Z onProgress: Message count is 790
  91 seconds for 790 messages
  ~115ms per message

I'd expect, based on my testing of v1.22, an average of ~70ms to decrypt each message, so about half that. I'll be doing some more detailed benchmarks on Linux soon to understand why this is.

You can help by providing detailed stats about the computer that produced these results.

[kpuljek]

After a day of using Signal at work, starting the desktop client at home can take over 40 minutes for 500-1000 messages. My CPU is intel i5-4690, I have 16gb of ram, nvidia 960gtx and ubuntu 18.10. running signal-desktop-beta

I can capture the log or anything needed once I get home today, just let me know

Edit: My log:
https://debuglogs.org/97e6df5f744d155081514eca479f2aae83656cc8712eda8db277ccfd23919bee

[scottnonnenberg-signal]

@kpuljek What kind of hard drive does that computer have?

[kpuljek]

@kpuljek What kind of hard drive does that computer have?

my OS is installed on an SSD - Samsung 850 EVO
and my home partition is on a Seagate 1TB HDD. I ran some tests on it, it checks out, and I've been playing really intensive games from it and never noticed any performance issues.

[goexle]

Ich have the same problem

[scottnonnenberg-signal]

@goexle Please tell us about the hard drive on your linux computer with slow performance.

[janvlug]

Yesterday, downloading messages took extreme long. Signal Desktop got very confused. In one group, I got more than 100 times the same message. The history of that group is now lost. I also got "Error handling incoming message" notifications in multiple conversations. Here is a debug log.

[scottnonnenberg-signal]

@janvlug: First, how long had it been since you had previously opened Signal Desktop? Second, I think it's probably worthwhile for you to enter a new bug, since this is about the errors and not performance. When you do that, the more information you can provide about the conversations where the errors happened, the better. We're looking for any kind of pattern you can find in the conversations which had errors vs. those which didn't - what's the difference in conversation history between them?

[scottnonnenberg-signal]

@janvlug Also, please reach out to [email protected]. We really should try to understand why your computer has such a hard time executing our SQL queries in a timely fashion. There are so many timeout errors in that log, it's hard to find the encryption-related errors. I'd really like to ensure that nobody else runs into what you're dealing with!

[janvlug]

@scottnonnenberg Maybe I should have reported the debug log to issue #3251. I will update that issue with more information. After having provided more info in that issue I will contact Signal support as requested.

[bennlich]

I've also noticed that it can take quite a while for my desktop client to get up to date after starting. Here's a debug log from a recent signal desktop startup: https://debuglogs.org/1a7deeb16caa339140233f779a817ca4e6c0cbefe28a1c0903ec6baa62a5d251

And my computer specs:
CPU: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
HD: SSD
RAM: 16GB
OS: Debian Stretch

Running Signal v1.23.2

Let me know if any other info would be helpful.

[webketje]

The log attached below is from starting the Signal Desktop app to first-time to responsive: this took about 17 minutes to load only 300 messages with an average completion time between 1.5 - 5 seconds for every SQL channel job signallog_27052019T22243000.log

Computer specs: see full at https://termbin.com/ci1i

• CPU: Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
• HD: Seagate SATA HDD
• RAM: 16GB
• OS: Linux Mint 19.1 Cinnamon
• Signal v1.24.1

[kb100]

This is an issue for me as well on my travel laptop, which I sometimes go a month without opening Signal on. In my opinion the fact that downloads are slow is an issue, but not the main issue. The real issue is why are the message downloads at startup not async? Download asynchronously and show me the messages that are already downloaded so there is at most an out of sync issue, but never an issue with accessing messages that I've already downloaded.

Throwing in my debug log as well:
https://debuglogs.org/8a93ed298c40206bb09bab8932bd662063c61664fde66736373aa2c9a7b293a6

CPU: Intel(R) Core(TM) i7 CPU Q 720 @ 1.60 GHz
HD: Samsung SSD 850 PRO 256GB
RAM: 6GB
OS: Debian Sid
Signal v1.24.1

[kb100]

It seems that the solution I would expect is already implemented in the way that I would expect in the very similar setup that the app is already open but disconnected from the internet. When the app reconnects, the GUI does not freeze or block everything with a "loading messages" page, it just has messages pop up one by one as it downloads them. Why not just do exactly the same thing at startup?

[nvbln]

Adding to @kb100's issue (which mine is very similar to, if not the same), this also occurs when using the desktop application on a weekly or daily basis.

[blamarche]

Myself and my friends who recently started using signal as our group chat also have this problem on Windows. It takes about 3 minutes to load the application and message history each morning. I agree with kb100, why lock out the UI when loading the messages on startup? Or how about giving an option to limit how much history is loaded on startup

[MPE2016]

I seem to have the same issue.

signallog.txt

[link2xt]

I have not looked at the code, but the problem is likely that you save each message in its own SQLite transaction. HDDs have good throughput, but also high latency, so waiting for the writing to finish takes longer than writing itself.

You should open a transaction instead, download a batch of ~50 messages, save them, close the transaction, then mark messages as received on the server so it removes them from the queue. This way you will spend 50 times less on syncing the changes to disk.

I also have HDD and it usually takes minutes to sync changes.