Signal-android: Signal update 4.1.0 wiped out lost erased entire message history without plain text backup or like

[seandubs]

Bug description

I opened Signal after an automatic update to 4.1.0. It asked me to re-initialize my phone number as if it were a new install. Once it opened, my message history was completely gone. I re-imported from the sms database, and it only imported everything from before I installed Signal, nearly 6 months ago. I attempted to import from a text backup to see if the update had made one, but it was not found. It appears the message history is lost.

Steps to reproduce

• Allow Signal to auto update via the play store.
• Open Signal
• It will ask you to re-initialize your phone number.
• Upon successful initialization, find all of your messages lost.

Actual result: All of my Signal message history was gone. I was able to re-import SMS messages from before installing Signal the first time, over 6 months ago.
Expected result: I would still have my SMS message history.

Device info

Device: Google Nexus 5
Android version: 0.0.0
Signal version: 6.0.1

Link to debug log

https://gist.github.com/d153a015ef8574c0cf7bf3092d40c964

[johanw666]

Strange, it should not delete your data on update. Even after unregistering and reregistering the messages should not be gone.

Because Signal does not use the internal sms database but stores its messages in its own encrypted storage it's not surprising that only the old messages were restored. And the plaintext backup is not made automatically, if you didn't trigger it manually it won't be there.

[moxie0]

Sorry this happened to you, but we have no control over the update process. It sounds like your ROM either wiped the data or you suffered some kind of disk corruption. That's not something we have any control over.

[moxie0]

Hey Randy, just so that we're on the same page:

1) We distribute an APK file to Play and to the website, which is what ends up on your device. The APK is literally a ZIP file. There is no "logic" in it, the update process is handled entirely by the OS. No code of ours runs during the update, and Signal is not even made aware of the fact that it is being updated.

2) If installation failed several times, it sounds like there was some kind of problem, but that process is not within our control. If we wanted to create a situation where you got an "insufficient storage" update failure message for a 10MB update, I can't think of how we would do it. It sounds to me like something is wrong on your device.

3) If you are prompted to re-register, that means data has been lost. The only way that Signal knows whether you're registered or not is by tracking that state with data stored in Signal's /data/data/... application directory. If that state is lost, to Signal that looks identical to a new install, which is why it prompts you to register. There is nothing that Signal could do in between the update beginning and Signal being restarted to delete that data, since Signal wasn't running or given the opportunity to run during that time.

It is entirely possible that there is some problem with Signal, but given what you've described (particularly the errors you encountered during the update process), it sounds unlikely to me. There is also nothing that I could investigate at this point, since by the time that you restarted Signal all of the data was already gone. I'm really sorry this happened to you and I wish there were something that I could do, but it's not within our control.

[moxie0]

I know you're a small group so I have to wonder how adequate and thorough your testing processes and procedures are. Do you just test on one hardware platform or one version of OS? Do you ever download the APK from the Play Store in order to verify the the filesize, checksum and also to test the APK that's being distributed to the rest of the world?

The APKs that we distribute are signed by keys that we control. If the APK were modified or corrupted in any way while in transit, the signature would fail to validate. The APKs that we build are also built deterministically, so if you wish you can pull the APK off your device (no root required) and validate that it is the exact product of the source that we publish:

https://whispersystems.org/blog/reproducible-android/

I understand how APKs are distributed to the Play Store, my intuition, my IT 'gut feeling' here is that your update, through whatever processes and for whatever reasons, somehow got corrupted, either in filesize or checksum or what I don't know. How else to explain why this ~10MB update failed to install at least 10 times with over 300MB free when I was installing much larger updates without any issue.

An APK's integrity should be cryptographically verified by the OS, so any type of corruption should prevent the update process from even beginning. Based on what you've described, it sounds like something is not well with your device state.

• It's not the lost chat history, although there is that, but it's the belief that there was genuinely an issue with the update and I get the feeling that Whisper Systems can't be bothered.*

We really have no control over the update process, but if you have any suggestions for what you feel that I should do in this case, I'm willing to listen. I'm doing tech support at 9:44pm on a Saturday, so I wouldn't exactly call this blowing you off.

[mulander]

I had a similar issue just now. Signal updated via Google Play Store resulted in a 'fresh' copy with all my current setup gone from it. I do have a clue on what might have happened though.

The phone I am running has Android 6.0 with only 8 GB of internal storage and a 32 GB SD card formatted as adoptable storage. Since my internal storage is so low I started moving apps to the SD card including Signal. Like mentioned above after the update Signal asks me to re-register but what I noticed is that the Google Play Store moved it back to the internal storage after the update.

This seems to be a common thing with this version of Android:

• http://forums.androidcentral.com/samsung-galaxy-s5/507724-my-s5-lollipop-upgrade-killed-all-my-apps-my-sd-card-how-can-i-get-them-back.html
• https://productforums.google.com/forum/#!topic/play/6-f0Jp1mYWE

Now, it's not possible to move Signal back to the SD card as it results with a 'not enough space' message for the SD card (which is false, there's about 28 GB left) - I believe it can't move because the previous signal folder/data is already present in the target copy location. It's also not possible to change the default install location to SD card as adoptable storage doesn't seem to allow that.

Now the big part, I did notice this problem with other apps like:

• Twitter
• Tusky

same situation, being moved to internal storage on update but there are apps that seem to be able to survive this. The Strava running app also updated on the same day and it remained on the SD card without loosing any data. This makes me hope that there is something that could perhaps be done to prevent this on the app side and if not at least put out a warning that this can result in data loss which is particularly harsh with Signal.

[moxie0]

@mulander Thanks for the investigation. I'm curious how you moved Signal to the sdcard to begin with? The way the Signal AndroidManfiest.xml is setup, running Signal on the sdcard should not be possible.

[mulander]

Adding screen shots on mobile, more details in a second.

[mulander]

OK, so as you can see on the screenshots the interface is not preventing me from moving the app. I can't move it again on my phone due to the 'not enough space' issue from the update.

My wife has the exact same model of the phone and is still able to move it (I guess I can move the app on her phone and provide a screenshot as proof if that's needed).

Android system info:

version: 6.0
kernel: 3.18.19

let me know if you need any specifics.

[sigenc]

@mulander Wow i wasn't able to move Signal to sd. U say 6.0. Wich phone is this?

related: https://www.reddit.com/r/signal/comments/68fjzk/save_signal_data_to_sd_card/

I heard Signal uses a lot of storage and I'm ok with that as long as it is running on a micro sd card. How can I get signal to have it's data to a micro sd card on my android phone (it is an LG G4 running android 6.0). I opened the settings and opened apps and then signal but there was no option for me to move Signal to my sd card (_my phone does let me move some other apps though_).

This is how it should behave.

[mulander]

The phone model is IRON_2 build number IRON_2_1.0.1 custom build S153MWB_TL_M_A10_V0.5_201612021000.

Link to the marketing page of the manufacturer:
http://www.myphone.pl/en/produkt/hammer-iron-2/

EDIT:
@sigenc did you have your SD card formatted as adoptable storage or a plain SD Card? In my case the card is on adoptable storage.

I found this reddit post explaining the Android storage system very helpful:
https://www.reddit.com/r/Android/comments/496sn3/lets_clear_up_the_confusion_regarding_storage_in/

[johanw666]

Forgive me if I completely misunderstand this, but I thought the whole idea behind adoptable storage was to combine the external and internal sd card as one big storage container so there can not be any move from one to the other after that.

[mulander]

@johanw666 according to the linked reddit it's still separate partitions because an SD card can fail or be removed by force. The linked thread has much more details on why it's that way and how it's exactly partitioned.

[sigenc]

@mulander It was plain sd card. Thx for the information and link