Signal-desktop: Eliminate loading screen, load new messages in the background

[TurkeyMan]

• [x] I have searched open and closed issues for duplicates

---

Bug description

I sleep my computer when it's off.
I often wake my computer from sleep, and then Signal plays catch-up with all the activity from my phone or other clients.
__Every single message__ received while my computer was asleep shows a pop-up notification, and makes the new-message 'ding-a-ling' noise.
Today, I have hundreds in the catch-up queue. I've been sitting here for over 5 minutes so far just hearing my computer ding and spamming notifications about messages I received the last couple of days.

Does this happen if my computer is off rather than asleep? I can imagine perhaps, since the client was launched before the messages were received, that when I wake the computer, it thinks the messages were received during this active session?

Steps to reproduce

• Run the desktop client (I'm using electron, but i think chrome works too)
• Sleep the PC
• Do activity with your phone
• Wake the PC, prepare for a torrent of activity...

Actual result: Torrent of catch-up activity; very annoying
Expected result: Signal should know my computer was off, and not give me notifications that have already been read on other devices.

Platform info

Operating System: Win10 64
Browser: Electron
Signal version: [...where is the version information on the electron client?]

[scottnonnenberg]

First, you can get the version information from the top of the debug log.

Sadly, we have a bit of a problem implementing this today. We don't know how many messages are waiting for us in the queue on the server, so we don't know a massive torrent is coming. And today we have nothing attempting to detect full application sleep.

So perhaps we can talk about the expected user experience. Would you be okay if the loading screen came back when we were able to detect that we had 20+ messages to pull down? Or would you like the UI to be changing real-time as you navigated around (unread counts going both up and down, new messages showing up unread, then being marked as read, etc.), just no notifications?

[TurkeyMan]

I reckon if I were designing the experience, I'd probably do something like; when there is data to fetch from the server, fetch _all of it_ in the background until it's fully synchronised. Then, now that all new messages are available locally, apply the UI update in one hit? Ie, all the new messages would appear in the UI at the same time.
There are 2 problems this would mitigate:

1. The contacts on the left are ordered relative to most recent interaction, and that bounces around like crazy when synchronising
2. Each new message produces a notification and a noise. If the program fully synchronises before emitting notifications, it can estimate that I don't want more than [magic number] notifications in one hit, or maybe even just emit one notification saying "many new messages".

It seems fetching from the signal server is surprisingly slow, so perhaps a piece of UI to indicate that a synchronisation is ongoing in the background would be a good hint to users that there is background activity and they should expect new messages to appear soon.

Now the reason I suggest to maintain a usable UI even though there is pending background activity, is that the user may want to author+send new messages while the background sync is ongoing. I can't imagine why that would be impossible...? Is it possible to send a message immediately even if the client hasn't yet 'caught up'? If not, then the client should obviously take my authored message(/s) and send after sync has concluded.

[skulumani]

Is it possible to send a message immediately even if the client hasn't yet 'caught up'? If not, then the client should obviously take my authored message(/s) and send after sync has concluded.

Frequently I'll try to send a message while the app is still synchronizing in the background. I will inadvertently send a message (current time) and then the previous messages from the contact will arrive on the desktop. This will create a situation where the newest sent message is displayed before the older messages from the contact.

The sent message will only have 1 check mark, signifying it's been sent and not received. Most of the time I'll notice this and be unsure if the recipient has the message (due to some logic to prevent out of order messages on the Android?) and have to resend once everything is updated.

Now my desktop and android message timelines have diverged and it's confusing for a short time until everything gets synced back up.

[deutrino]

I noted in the Chromium app that the syncing process is hidden behind the curtain when the machine has been off, but not when it has slept, as is noted above, but I wanted to clarify that bit.

Either way it's annoying, and the biggest annoyance is the glacially slow speed. I was away from my computer for over a week using Signal the whole time on my phone, and it literally took nearly an hour to sync when I turned the computer back on. I know there are protocol limitations influencing this, but it's really not something that could be explained to 99% of users when every other messaging app Just Works in the same scenario.

So if I were project manager, the absolute highest priority would be making the sync happen as quickly as possible. I don't know how to do that without leaking information - would the server volunteering "hey, there are 500 messages in the queue" be too much of a leak? - but hopefully it is possible.

As far as UI, seeing the entire song and dance and hiding it behind the curtain are both bad. The former makes the desktop client difficult to use for the duration, and the latter makes it impossible to use. I suspect there aren't any easy solutions to this, but for now, it would be somewhat less unpleasant if messages sent AFTER the date of messages being received as part of a sync, would stay on the bottom of the scroll.

[TurkeyMan]

but for now, it would be somewhat less unpleasant if messages sent AFTER the date of messages being received as part of a sync, would stay on the bottom of the scroll.

This.

The former makes the desktop client difficult to use for the duration, and the latter makes it impossible to use.

The latter makes it impossible to use under the assumption that the sync will take an hour... if the sync was 'fast' then it it's not a problem at all, or is there some other problem with the experience that I'm not thinking about?

[deutrino]

Apologies if this is duplicate info. I wanted to characterize the behavior I'm seeing in Electron a bit better. I'm a pretty heavy Signal user; between my contacts and the one messaging group I'm in, it's typical for me to exchange hundreds of messages per day. This is a major pain point for me.

I'm experiencing the problem in this ticket 100% reproducibly on Electron. It is always glacially slow to sync a message backlog when the Electron client hasn't been running while the messages were exchanged on other devices. The UX varies depending on whether the client was shut down or running but with the computer asleep. If the client was shut down, the "loading messages" interstitial appears. If the client was running on a sleeping computer, the messages are replayed, generating huge numbers of notifications and making the app essentially unusable for messaging contacts for whom there is a backlog being replayed. This has a severe negative impact to the usability of the app during the replaying, which can take tens of minutes.

STEPS TO REPRODUCE:

1. Use the Electron client and the Android client. These function as expected when both are running.
2. Shut down the Electron client, or leave it running but sleep the computer on which it is running.
3. Use the Android client normally.
4. Start the Electron client, or wake the computer on which it is running.

WHAT ACTUALLY HAPPENS:

If Electron client was shut down:

1. The "loading messages" interstitial pops up.
2. Messages load very very very slowly, VERY slowly, taking tens of minutes for an evening's worth of chatting with a couple friends. I am unsure if the number of messages the app is claiming to load is accurate - someone mentioned in a related ticket that they suspect it is not.
3. Eventually the app finishes loading the messages and functions normally.

If Electron client was running but computer was asleep:

1. The app UI appears normally (no interstitial, etc).
2. Messages load very very very slowly as described above.
3. Notifications display (Linux Mint 18) for most or all messages as they replay, which is very annoying.
4. Each message is put into the contact pane as it would be if it had just arrived in real time (i.e. not as part of a backlog). This makes the app difficult or impossible to use for messaging contacts with whom messages were exchanged when Electron was sleeping, as it is not possible to see what you've sent or to see replies from your contact once backlogged messages come in and the newer messages scroll out of view.
5. Once the messages are replayed, the app functions normally.

WHAT I EXPECT TO HAPPEN

(Please note, I know there are fairly fundamental issues behind this problem, I write these from the "it should Just Work™" user perspective.)

A. The app should sync the entire backlog of messages in a reasonable time - I would pick 30 seconds as a goal to shoot for, because I suspect that users other than just myself will start to get more and more annoyed beyond about 30 seconds.
B. If A cannot be accomplished, app should not spew messages into the panes of individual contacts when the client returns after a computer sleep, as this makes chatting with said contacts nearly impossible.
C. If A cannot be accomplished, app should not spew notifications for messages as they are synced.

Regarding "what I expect to happen," I understand there are protocol issues in the way of point A. If I understand correctly, part of this is that the desktop app doesn't have a way to know that the messages are part of a backlog, which makes points B and C quite difficult?

Even implementing point C alone would be a considerable improvement - because Electron is popping up hundreds of notifications, the aggravation is leaking outside of the app itself and negatively impacting the usability of my entire computer.

Anyway, hopefully a little more detail is helpful, let me know if there is any logging or testing I can help with. Thanks!

[scottnonnenberg]

@deutrino Thanks for the additional detail. You'll be happy to see that Electron v1.0.27 implements your option C: https://github.com/WhisperSystems/Signal-Desktop/pull/1507

[deutrino]

I just had my laptop powered down for about 2 hours.

• During that time, I exchanged approximately 75 messages while using Android (I went back and counted)
• Upon booting my laptop, the Electron client took over 17 minutes to sync
• The "n so far" message count crawled slowly up to 220 (about 75 * 3??) before the sync was complete.

https://gist.github.com/deutrino/3e771ef1f00eec7a96a9bafe6ac2caee

This is really, really bad. And not even remotely out of the ordinary for that volume of messages.

Edit: Forgot to add, the signal-desktop process with the lower pid shows constant CPU usage the entire time, usually about 20-25% of a core. Also, I am starting to wonder more about #1495, I am a really heavy user so maybe that plays a part.

[scottnonnenberg]

@deutrino I took a look at your log, and there's a lot to digest.

First, there are a bunch of these errors, which makes me wonder about the health of your database. How big is your ~/.config/Signal directory?

Error: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing.
    at Driver.update (file:///opt/Signal/resources/app.asar/js/components.js:21629:44)
    at Driver.execute (file:///opt/Signal/resources/app.asar/js/components.js:21587:22)
    at ExecutionQueue.execute (file:///opt/Signal/resources/app.asar/js/components.js:21941:29)
    at next (file:///opt/Signal/resources/app.asar/js/components.js:22021:34)
    at child.sync (file:///opt/Signal/resources/app.asar/js/components.js:22027:13)
    at child.sync (file:///opt/Signal/resources/app.asar/js/components.js:20052:28)
    at child.save (file:///opt/Signal/resources/app.asar/js/components.js:20259:18)
    at file:///opt/Signal/resources/app.asar/js/signal_protocol_store.js:800:36
    at SignalProtocolStore.addUnprocessed (file:///opt/Signal/resources/app.asar/js/signal_protocol_store.js:798:20)
    at Object.add (file:///opt/Signal/resources/app.asar/js/libtextsecure.js:37148:48)

Those are all in the session right before the slow load you're talking about, but they do help set the context. Not long after the start of the slow startup in question, there was 30-second period where nothing was logged:

INFO  2017-10-26T01:44:42.885Z queueing envelope +[REDACTED]108.1 1508972281775
INFO  2017-10-26T01:45:16.369Z queueing envelope +[REDACTED]935.1 1508972281775

And from then on there is general slowness, which I have to imagine is due to some sort of database difficulty. Either that, or there's generally a lot of stuff going on in the process otherwise. You can see that there are only about five log entries per second after that. Might have something to do with all the message XXX expired entries that come after that. Come to think of it, the 30s hang above could be all that work to grab all the cached envelopes (of course, it was only 11 in your case).

This is an interesting section - the message from entry is a message from someone else, and the sent message to is one synced from one of your other clients. There are twelve seconds between them, and I'm not sure why. Could be even loop saturation from all that expired message querying. It's also interesting to note that the next message expires 2017-10-26T01:53:04.848Z is logged out at 2017-10-26T01:53:14.596Z, 10 seconds after it should have fired. Seems like the whole thing is bogged down.

INFO  2017-10-26T01:53:05.770Z message from +[REDACTED]935.1 1508975673900
INFO  2017-10-26T01:53:06.036Z next message expires 2017-10-26T01:52:42.756Z
INFO  2017-10-26T01:53:06.036Z loading expired messages
INFO  2017-10-26T01:53:06.914Z message 1508377984513 expired
INFO  2017-10-26T01:53:09.947Z next message expires 2017-10-26T01:53:04.848Z
INFO  2017-10-26T01:53:09.948Z loading expired messages
INFO  2017-10-26T01:53:14.596Z next message expires 2017-10-26T01:53:04.848Z
INFO  2017-10-26T01:53:14.598Z loading expired messages
INFO  2017-10-26T01:53:17.052Z sent message to +[REDACTED]005 1508975673900 from +[REDACTED]935.1 1508975673900

I've got two competing theories: a) the database is starting to buckle under the weight of your usage, and b) You're a very heavy disappearing messages user, and that is contributing to database saturation or general event loop saturation.

Some more information on your database size, and then some CPU profiles would be useful. You can open the devtools and collect a javascript profile during a slow startup, and that might help us understand things a little bit better. I'll also take a look at our expiring message handling.

[deutrino]

I have disappearing messages set to 1 week with almost all my contacts, including the ones I message a LOT. ~/.config/Signal is 584M.

I don't have time to poke more at it right this second but here is some further system data. Also, the hard drive is glacially slow, but I looked and nothing was thrashing the disk while it was loading. I will probably have several opportunities in the next few days where my laptop will be offline while I am using Signal on Android, so hopefully I can gather more data.

CPU:       Dual core Intel Core i5-4210U (-HT-MCP-) cache: 3072 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 9578
           clock speeds: max: 2700 MHz 1: 1118 MHz 2: 1186 MHz 3: 1100 MHz
           4: 1575 MHz
Info:      Processes: 265 Uptime: 21:20 Memory: 6232.6/7887.7MB
           Init: systemd runlevel: 5 Gcc sys: 5.4.0
           Client: Shell (bash 4.3.481) inxi: 2.2.35 

[scottnonnenberg]

I just renamed this issue to reflect its current meaning - the loading screen, shown on launch of the app, should be eliminated, and messages can be loaded in the background.

[Lesik]

I find the UX solution of a different instant messenger quite elegant, simply replace the "Signal" text in the top-left with "Syncing..." or something like that.

[scottnonnenberg]

@Lesik That assumes you are able to determine what subset of your contacts have messages needing to be sync'd. The signal server presents us with a simple queue of messages, and we don't know who they are from until we pull them down.

[Lesik]

@scottnonnenberg There seems to be a misunderstanding, my proposal does not require to know what contacts have new messages. I simply suggested that instead of a full-screen loading screen, the "Signal" text in the top-left corner to be replaced with "Syncing...", as other messengers do.

Current UI:

My proposal:

[scottnonnenberg]

@Lesik Thanks for the clarification. The standard problem arises, then, of sending a message in a conversation when there are pending incoming messages - things end up out of order. Perhaps the right tradeoff is the top-level notification you mention, then a popup if you try to send a message when that's showing. You can navigate and compose messages all you want...

[breznak]

Thanks Scott for moving the discussion here.
You guys here are solving the problem to eliminate the loading screen and making the UI more responsive.

My question is on implementation level, why does it take that long to download & decipher the few hundreds (thousands) new messages on the sync? It could be what..a few MB, that's a matter of a minute max on any modern connection and hardware. So what is causing the slow-down we observe? Also, are the messages locally cached? To me it seems I download them all over each time Signal is restared.

Other than that, as I understand it now, all the messages from any of the "linked devices" are kept on the OWS servers (hopefully end-to-end encrypted :) ) and waiting for the sync to the remaining linked device. Then they are deleted? What happens if the linked device never connects anymore, is there a EOL time? Is the server side infrastructure open-source as well? It would be nice to have an option in the app to "wipe all of my data from the servers"...

[scottnonnenberg]

@breznak - Remember, it's not just download. It's download, decrypt, and processing of each message. Including potentially large attachments, which then also need to be decrypted. Of course, that's not a complete explanation, because we could still do better.

Here's the server source code: https://github.com/WhisperSystems/Signal-Server

Regarding, 'wiping your data.' We don't believe it to be a high risk, because the messages are encrypted. Only from/to/timestamp is known to us. Most of the time there aren't many messages there anyway, because clients generally stay up to date, and there's a limit to the messages we save. But if you're concerned about it, you might consider unlinking a desktop client that you don't really use anymore.

[deutrino]

Even given all that work, it's still really really REALLY slow. To the point of, as discussed, tens of minutes or worse for a client that missed a couple hundred messages.

What is the fundamental cause of that - a round trip for every message? The more people adopt the desktop app, the more this pain point is going to get noticed and become a black mark when people decide whether or not to recommend the app.

[scottnonnenberg]

@deutrino Please provide logs when things take a long time. It's very help for us to see what happens on machines other than our own.

Digging a bit deeper, I think some of the undue delay is because of IndexedDB churn - we save to the database before we decrypt, then again, after we decrypt, then finally after we've fully processed the message. But I think our biggest opportunity for improvement here has to do with how we schedule all this work - for protocol reasons we need to decrypt in strict incoming order, but there are other things we can do outside that ordering - downloading attachments, and processing the messages on the application side. Once we break those up into separate queues, I think the whole process will go a lot faster.

Note: I'm not far from locking this conversation, because I think we've covered all the bases. I really don't want to get into bunch of repeated "why is it so bad?" questions. Feel free to add emoji responses, of course - that helps us understand how important things are to y'all.

[deutrino]

Okay. I did provide logs back in November but will try to provide more as, having taken a glance back at that part of the thread, it looks like there was a bunch of unhelpful noise in those logs.

As you know I'm a heavy user with a longstanding desktop install. I'll start from that baseline and then later if you need me to reinstall etc, I'm happy to do that - I'm traveling so turnaround time might be a bit slow but I have a major interest in helping to improve this any way I can. I can gather logs when waking my laptop from sleep with a running client, and/or when starting the client after it's been offline a while, let me know if you have any preference.

I'd be happy to move this to a separate issue, whatever works for you. I have a major interest in seeing improvements to this sync process and would have helped more already, it's just the usual herding of cats problem for me :)

[breznak]

Here is my debug log:
https://gist.github.com/breznak/d995577b87f8229899caaf5de290e0f1

Things I noticed:

• I think all the messages are being downloaded/processed again?
• my .config/Signal/ is 800MB (~1000 messages, lot of attachments)
• it would be iteresting to do I/O, CPU profiling during the sync process

[breznak]

Also, I was concerned to find the link in the log:
https://whispersystems-textsecure-attachments.s3-accelerate.amazonaws.com/3399142333737380479?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20180103T223203Z&X-Amz-SignedHeaders=content-type;host&X-Amz-Expires=3600&X-Amz-Credential=AKIAJHWS3AOTJTASHBDA/20180103/us-east-1/s3/aws4_request&X-Amz-Signature=e1e8f3ac9782447d120430b2747fbe4cf067262a33302ae9cc60ee02a6e29b97
It containes credentials, but I was not able to download it with wget, so I hope it's safe...

[scottnonnenberg]

@breznak What looks like downloading messages 'again' is likely all the communication between your devices - your mobile phone tells your desktop install that a given message has been read. Anyway, thanks for the log.

Regarding the attachments, the links expire over time, and the actual file contents are encrypted. But you're right, we could probably obscure some of those querystring arguments to make it even harder to get access to those attachments.

[deutrino]

Okay, I kept Signal Desktop offline for a while while using Android normally, got some observations and logs. This is with Signal Desktop 1.1.0 (electron) on Linux Mint 18.3 (Cinnamon), Signal Android 4.14.10. The sync performance has increased somewhat in that it only took about 18 minutes to sync somewhere around 400-450 messages, including with image and audio attachments.

This test was conducted on a largely idle laptop, freshly rebooted, plenty of free RAM and nothing else hammering the disk.

https://gist.github.com/deutrino/22af29459318990b226976b0be6bfe17

• Upon loading the program, it takes a few minutes to get from "Loading..." to "n so far..."
• Once it's at "n so far..." (near the start of minute 16:25Z) it settles in to ~20% CPU on one process, and ~1-5% CPU each on two processes (including the one with the gpu-render switches)
• The disk is not completely thrashed but there is a lot of disk access with few pauses lasting more than a fraction of a second
• As I noted before, the app is claiming to sync far more messages than were exchanged while it was offline. The "n so far" count got to about 1030, but a manual count of all messages exchanged on my Android client was 400-450 (I counted quickly while scrolling).

[scottnonnenberg]

@deutrino Thanks for the additional logs. It's interesting that you note 20% CPU usage during load, which means we can definitely schedule our work better. Again, the count to ~1000 shows the number of 'envelopes' we processed from the server, only some of which will represent messages from other users.

[deutrino]

It really seems to be a heck of a lot of disk access for what the app is actually doing, too. Dunno if that's something that could be sped up. As always, if you'd like me to do any specific testing, please ask :)

[Paradoxeuh]

Naïve question here:
Do you need to load message N before loading message N+1 ?

I'm asking that because it seems easier to load last 10 messages during the startup rather than the whole pile.

[scottnonnenberg]

@Paradoxeuh It has to do with the encryption mechanisms in play. A prior message from a contact may have set up a totally new session with you, meaning that no subsequent messages can be decrypted without processing that setup message.

[deutrino]

I was looking through #1842, which seems to have some overlap with this one, and just wanted to come back and say that while these problems are not completely gone, they do seem to be noticeably better since the last handful of updates (I just updated to 1.9.0). :)

[majkinetor]

and why that cannot be done given Signal's encryption scheme

This is total deal breaker and should be made to work NO MATTER the problems with design. Everyone I know that uses Signal wants to stop using it because of this or something related to this. Waiting for minutes for social input/outputs defies the purpose of the tool - its faster to even use email.

Perhaps signal should have service that does this in the background without compromising security, which isn't electron bloated and could be as simple as it gets for just syncing.

[sakudo]

I completely agree with @majkinetor . If you want to have some future of the Desktop Client, you should try to eliminate the waiting time upon startup.

• Can the messages that have been previously synchronized be cached or something like that? - So the desktop client only synchronizes the latest messages.
• If this is not possible, it would desirable to have a background feature in the client: the client would run in the background and as long it is not closed it would keep up with the synchronization. The window should kind of disappear into the system tray (or however you call this place on a windows 10 machine... I mean: program running but with hidden window).

Unfortunately, I cannot support with technical programming skills, but I only can contribute stupid consumer remarks... nevertheless, I think it is important to improve this aspect. - Keep Signal going! Wonderful work in general and thanks so much!

[majkinetor]

@sakudo

The window should kind of disappear into the system tray (or however you call this place on a windows 10 machine... I mean: program running but with hidden window).

This is already possible via shel script for example on Windows

start %LOCALAPPDATA%\Programs\signal-desktop\Signal.exe --use-tray-icon --start-in-tray

This can't resolve the problem tho - signal often crashes, computers restart unattended, and is unusable if multiple users exist on OS having signal etc.

[scottnonnenberg-signal]

@majkinetor It seems like you're experiencing a few unrelated bugs: "signal often crashes" and "unusable if multiple users exist on OS having signal." Please enter a new bug for each, providing additional details. Thanks!

[majkinetor]

That is irrelevant really. I can live with crash here and there and nowdays its MUCH lower then few months back. Other is not a bug but scenario.

This loading problem however...

[mkatychev]

as a short-term alternative, can we have an option to only load the last _n_ messages from every user? I think this will make startup much quicker specially if it is on by default.

[ChrislyBear-GH]

Signal Desktop isn't usable at the moment:

I have my phone in the vicinity. I get a message. I open Signal Desktop. "Loading... xx so far". Crap! I clos it again, get up and look at my actual phone.

Signal Desktop is useless.

[kapsiR]

@Chrisly3ear I delete all my data in signal desktop and reconnect it every few days.
What I don't understand is, if I delete my messages manually within the conversation, signal desktop loads them in the background anyway?!

[Powersource]

I sometimes end up sending messages to friends using e.g. wire since it loads a lot faster.

[scottnonnenberg-signal]

@mkatychev We cannot load only a few messages as you suggest because, to properly decrypt all messages, we need to pull them all down, and process them in order.

@Chrisly3ear The only way we can help you out is if you provide your debug logs.

@kapsiR When you delete or read a message, you're only reading the locally-downloaded copy. What happens when you read a message, however, another message is sent to all of your linked clients telling them that you've read this message. So desktop needs to download two messages for every single message you receive and then read on your mobile device.

[sakudo]

Unfortunately, I have not sufficient knowledge to help, but surely, the concerns here can somehow be justified: Why cannot Signal Desktop be coded such a way that a quick start up is possible? Would it not be possible to save the status of the program once it is synchronized? A long start up time at the first time would be acceptable, but not every time you start it.

Probably, one would need to accept a security compromise and save the status in a somehow unencrypted state (or encrypted, but additionally keys would be saved along side too).

You could even tell the user about it or offer him a choice at first start up: Would you like to have a) short start up times but stored messages locally unencrypted or b) long start up times and full encryption.

In many cases, the local machine is trusted anyway 100%, so a "working" Signal Desktop would bring more advantages than a very slow one.

[scottnonnenberg-signal]

@sakudo It's not about local encryption. All incoming messages Signal Desktop downloads are encrypted and must be decrypted to if we are to do anything useful with them.

Without changing the fundamental nature of the encryption protocol, there are still some things we can do:

• download attachments asynchronously (or fully on-demand)
• better parallelize the various streams of work: download, decryption, updating local state, and updating UI (the last three are done in serial)
• download messages in the background (though this comes with a set of interesting tradeoffs - what happens when you send a message before 20 more messages are downloaded in that conversation? how should things be ordered in that case?)

[sakudo]

It is just puzzling that a state that is reached after synchronization cannot be saved locally. Then at a next startup, this state is _instantly_ reproduced without waiting time. Surely, some more synchronisation has to be carried for all messages that have been missed and were sent during the Desktop was shut down or off line.

To reach a non-technical audience and to promote Signal as a messaging system as a whole, a solution has to be found somehow... I am just adopting a very consumer-like perspective. (The beloved competitor Whatsapp does not show this delay.)

[scottnonnenberg-signal]

@sakudo If you could explain what you're seeing in more detail, that would be extremely helpful. We can only help if you provide high levels of detail in what you're seeing.

Anyway, I'm reaching here, but it sounds like you're seeing Signal Desktop download a lot of messages, and then on immediate restart, it processes even more on startup. That can happen if we fail to successfully process messages the first time - we save them and retry them a second time. It would be helpful to get your logs in that case to understand what messages are being retried. Potentially there's a bug we could fix there?

[majkinetor]

download messages in the background (though this comes with a set of interesting tradeoffs - what happens when you send a message before 20 more messages are downloaded in that conversation? how should things be ordered in that case?)

How about maximal priority for the pending messages of the currently open conversation with attacments downloaded after the most important thing - chat ? Waiting for single stream to finish is still better then waiting for messages of currently irrelevant conversations.

[scottnonnenberg-signal]

@majkinetor The server has a single queue of all incoming messages, so we don't currently have the ability to do that. Certainly a good idea.

[kapsiR]

When you delete or read a message, you're only reading the locally-downloaded copy. What happens when you read a message, however, another message is sent to all of your linked clients telling them that you've read this message. So desktop needs to download two messages for every single message you receive and then read on your mobile device.

@scottnonnenberg-signal That's ok, but if I delete all my messages manually in the conversations and then start signal desktop again (close and start) it's still loading a huge amount of (deleted?) messages, even if there are no new ones. Only if I delete all data (via settings) from signal desktop and then re-link it, the startup is fast again.

[scottnonnenberg-signal]

@kapsiR As I mentioned above, it's not necessarily re-downloading when it is processing messages on startup. It could be retrying messages whose processing didn't complete successfully. This is why we need logs when you report issues, so I can take a look at the messages being retried, see if I can address the underlying problem.

[EricMC0]

Hi. Playing catchup here.

I mostly use Signal on my phone, but have played with Signal Desktop a few times. I always get that "Loading 10" .... "Loading 90".... count on startup; this doesn't ever happen on my (Android) phone.

Since it takes a prohibitively long time, I generally just pick up the phone when I want to message someone quickly.

But tonight, I tried the following:

• I open Signal Desktop
  (I wander away to something else after a minute or two whilst it is "Loading messages. 60 so far...");
• Once it is fully open, I send one message to a friend.
• I close Signal Desktop

• I open Signal Desktop again
  "Loading" sequence starts;
• again, it is loading scores of messages and taking many minutes to do so
  "~ 60 so far..."

• Once it is open, I immediately close Signal Desktop again (with no messages sent)

• I open Signal Desktop again
  "Loading" sequence starts;

• again, it is loading messages but now takes around two minutes doing so,
  stopping after "Loading messages. 40 so far..."

When it comes to a stop, nothing has changed in the list of message threads.

If I understood the comments above correctly, this is not how it is supposed to behave.
I only have two devices running Signal — and both are synced.
So what are all these supposedly unseen messages it is loading?

Signal Loading debug log.txt

[scottnonnenberg-signal]

@EricMC0 Looks like you're falling prey to contact sync spam - in your provided log there are 273 contact syncs just from what looks like about the last 24 hours. The good news is that there's a change coming in android which will limit those syncs https://github.com/signalapp/Signal-Android/commit/bf692e8da354d3bfd44fe85fb3abb3cadec5e857

[scottnonnenberg-signal]

@EricMC0 About the retrying of messages, it's not clear. I doubt those are taking the majority of the time, though. Contact syncs require an attachment download then dealing with every contact included in the package.

[Lesik]

I know my comment isn't contributing much to the issue, but given this isn't exactly a low-frequency thread I wanted to add that Signal Desktop isn't by any means _useless_. I use it every day, it's a great companion and I'm glad it exists. The message loading is a little slow, but has gotten a lot better since when this issue was opened almost a year ago. It is definitely within the realm of what is acceptable of a messenger that puts security and privacy over other priorities.

[EricMC0]

Thanks Scott and Lesik. I appreciate the insights.

Sorry to be ignorant. Could you tell me more about Contact Sync Spam?

• What is "every contact included in the package"?
— the download of someone's address book (perhaps even of mine)?
— would such contacts have come from a hack - if so, whose?

• Or are we just talking about messages sent by syncing apps or add-ons which help you share your address book with yourself on another device?

[thommierother]

Supporting statements of @sakudo and others:
I am using signal desktop on various Linux (SUSE) and Windows Systems and can compare its useability with both Whatsapp and Threema. The bootup delay time is more or less unpredictabe. Sometimes its faster, but most times its loading and loading and loading forever hundreds of messages although I dont have such a big traffic on the signal apps on my mobile devices. This is a big disadvantage of Signal compared to other messagers. If you want to use the current critical discussion about Whatsapp in the general public and make some profits from new users for signal, this "useability deseaster" of the long startup time has to be solved fast.

[OtherSystems]

I just spent 30 minutes waiting for the loading screen before I could sent a message. Followed by still getting lots of live loading of messages and contacts changing order.

I understand it's not a simple problem, but could at least the contacts and 'send a message' field be available while it's loading? For me the order of the contacts isn't super important as long as I can find them and send them a message. Ordering could perhaps be non-instant, just every ?

As one of the issues seems to be getting the order of the messages, could attachments maybe be loaded later, or on view?

Key issue for me would be having the contact list + send-message-box available.

Thanks!

[majkinetor]

There is an idea there. One of the key problems for me is that I can't send the message and do other stuff. Many times I don't care if its actually going to be sent ASAP or in 10 minutes. Long loading of messages makes the one constantly returning to signal screen to see if its done, so that I can actually send 1 or 2 messages.

I suggest to at least let the people see (cached) contacts and enter messages, knowing that they will be actually sent to the contact when message sync is done (i.e. pending messages). Then we don't need to keep returning to the Signal Desktop as we will get notified as soon as person responded to the message (and Signal could actually notify when "pending messages" are actually sent)

This is IMO still WAY better then what we have now and could be easily implemented until the proper solution is found.

[OtherSystems]

As one of the problems is sort-order (messages are sorted on received time instead of sent time) and sent messages will have a received time at the moment of sending. Perhaps a solution could be moving to sorting of 'send time'. The problem with receiving earlier sent messages which would disappear in the backlog could be solved with an 'unread messages' warning and/or a quick way to show or scroll to unread messages.

[strugee]

@OtherSystems that sounds like a separate bug. Can you file a new issue after you search for duplicates?

[Liquidmasl]

i have a suggestion that could eleviate this problem (for some).

What i do not understand, is it nescessary for the decryption of new messages to decrypt all prior messages?

If not, could you give us the option (like a tickmark in settings) in the desktop app do not load ANY older messages? so when we open the app its just empty, any new message will be loaded and it is possible to send messages immidiatly?

I use Signal mostly on my phone and just start it up on my desktop machine to send a fast screenshot or copy pasta something which i have open on my desktop.
It would be really very helpful if the app would just let me do that, instead i have to wait a considerable amount of time!

I cought myself photographing my desktop screen with my phone to send a "screenshot" to a friend. and i feel very dirty from it.

[imrekalo]

hey! just a quick report on my experience regarding this issue. using signal desktop 1.18.0 on linux and 4.29.7 on android.
been trying to work around the painfully slow start of the desktop app by having my laptop sleep/hibernate instead of turning it off/on, so signal keeps running. but i haven't noticed any improvement. today i had my laptop sleep for an hour, had 11 incoming and 7 outgoing messages, short ones and no attachments, just text. one contact. took several minutes with high cpu load to sync this one conversation, the app not responding.

[janvlug]

See also: https://github.com/signalapp/Signal-Desktop/issues/2733

[imrekalo]

it's a lot faster now for some reason, even though i had my laptop turned off.

[scottnonnenberg-signal]

@imrekalo Glad to hear you've seen an improvement. Is that a change that came along with the v1.18.0 release?

Also, in the future, please include a debug log whenever reporting performance issues (or really any issue!).

[imrekalo]

@scottnonnenberg-signal i shouldn't have counted my chickens before they were hatched...

tonight my desktop app had to load more than a few messages and it took very long to sync, with signal desktop being unresponsive and cpu running around 90%. when it was responding, i could see that it takes the conversations one after the other and updates them, the messages coming in very slowly. here is the debug log: https://pastebin.com/J7TvtBJq
didn't wanna paste the whole thing here. i also copied the last lines from before sending my laptop to hibernation. yes, i'm running 1.18.0.

please let me do if there's anything i can do, test, whatever. i'm eager to help.

[thecyberd3m0n]

is it possible to transfer messages from mobile client using LAN connection, not from servers? It would be so much faster, and could be also encrypted

If my phone is on 192.168.1.101, and PC on 192.168.1.102, why Signal Desktop must download it from servers?

[MartinX3]

A local discovery service and if it doesn't find the partner, it would speak with the server. :)
Syncthing does the same.

[Liquidmasl]

Pretty sure not the downloading part is time intensive but the decryption is

[MartinX3]

It depends on your CPU, SSD/HDD, Motherboard and Internet speed.

[uUJsd7PX]

How about syncing in the background and updating UI (adding/removing messages) as new information is received?

[aquamammal]

Still waiting for this shit to load up, after finishing this whole thread, lol

[ChrislyBear-GH]

@aquamammal

As far as I have understood it, it's because the encryption works by chaining messages together. So if you have very long message threads it takes forever to decrypt everything from the beginning of time until the present.

And if you only casually open your Signal Desktop from time to time you'll always have to wait until it takes so long you'll just grab your phone anyway.

And they'll tell you that it's not a bug but a feature.

smh

[stephanemagnenat]

I'm still having this issue with 1.25.1. The average CPU load of signal-desktop is less than 20%, and overall my computer is 65% idle. I have gigabit fiber connection to internet, and only some (30-ish, maybe 5 images) messages since last time I started signal desktop. After 5 minutes only 150 messages were loaded (it seems more messages than only the new ones), with a throughput of loading about 30 messages per minute. I of course do not know the technical details of the protocol, but it looks to me that some improvements should be possible.

[disconn3ct]

Same issue, here's more datapoints if you want.

v1.25.1 Linux, running on Chrome OS, Slate tablet (beta channel 75.0.3770.61, with keyboard) using the standard upstream Linux container and the official Linux install docs for Ubuntu/Debian. Debug log from a few minutes after startup.

As a vote in favor of dumping electron in favor of ANYTHING else, its also using a significant amount of memory just idling along. Freshly launched it is over a gig resident, and by the time it has been up a few days it starts to push everything out of ram until it is restarted (at the huge battery/time cost of the startup sync.) And that hits on my osx client as well :(

[haary]

Same with 1.25.2 on Arch Linux. Loading took 10 minutes.

[jmrodri]

Using v1.25.3 on Fedora Linux, I had to change the group to only load 1 week's worth of messages. It still takes a lot longer than say Telegram or other desktop chats. I've gotten to the point where I start it in the morning and go do other things.

[ChrislyBear-GH]

As a vote in favor of dumping electron in favor of ANYTHING else...

When you're designing an application and selecting a platform for it, if your choice falls on Electron you've made a huge mistake or missed something along the way. Electron is always the wrong choice, unless you want to build a Chrome flavoured web browser.

[MartinX3]

Yes, my experience as developer is that electron doesn't give you the performance and the resource usage you need.

[Aspire1Inspire2]

Here is the log file.
Start time is still very slow comparing to other messaging apps.

Yes, it is improved. But the normal bar of "starting IM app" is higher.
Thanks for your great work.

[hpvd]

Slow Startup not only a problem of speed -> loosing messages #3636

[LoveIsGrief]

Another user here with continuous, long, slow startups and laggy interface due to Electron. However, if I disconnect my internet, start Signal, and then reconnect, startup is magically speedy.

v1.27.4 debuglogs

Please drop electron. Qt Quick or even Java Swing or Java FX for cross-platform use would be preferable.

As for the slow loading messages, the mobile apps don't have such a slow startup, so why does the desktop app? Maybe it's possible to replicate what they're doing on desktop.