Signal-android: Helium backup is not allowing backups of Signal - no backup possible

[kyanha]

Helium Backup is not allowing backups of Signal. I assume this is because of the lack of a BackupAgent implementation.

The error message I get from Helium is:

Backup Disallowed
------------------------
Signal has disallowed backup
of application data. Please
leave a comment with the
developer of Signal and ask
them to allow application data
backup.

As far as I can tell, there currently exists no way to backup the Signal text and MMS database.

[johanw666]

This restriction was added deliberately on Android 6 to prevent sensitive date getting uploaded in a cloudservice (default on Android 6). Use Titanium Backup (requires root) for full backups, but you might have to deregister and re-register after a restore because keys can be out of sync.

That being said, it's really about time the old and removed backup function is fixed and re-added.

[kyanha]

This is Android 4.4.4.

Also, rooting a device with unknown and unaccountable binaries just screams "leave a rootkit" to me. I use Signal because I want to improve my security, not undermine it.

[fajabird]

Please correct me from wrong - but allowing adb-backups in the apk-manifest without a BackupAgent implementation did not make your data beeing uploaded to google in the past. Android 6 introduces a new feature called "auto backup for apps" which indeed does not need a backupagent anymore - but there are options to customize it via the android:fullBackupContent tag which points to an XML-file containing rules (includes and excludes). This way you could still allow adb Backups (which will not leak data to google on android < 6 because of missing backupagent) and which will not leak data to the new Auto-Backup system on android >=6 because of the proper rules. I admit, that I don't know if adb-backup will work correctly if the rules do not allow any data to be uploaded to google. BTW the new system uses you google-drive account for uploading the backups.

[jedie]

I have try to backup/restore with oandbackup. But this doesn't work,too.

Is this ticket is the right place?!?

Generally, a full backup/restore function, would be great.

[strugee]

Can we get a prioritization call on this? I just tried to restore an adb backup only to find that it didn't include Signal data. So I've lost a couple _months_ of conversations, along with my private key.

There's virtually no way to notice that this is a problem until it's too late. I see this as a data loss issue, so, can we fix it? From what @fajabird said, seems like a simple XML fix.

[2-4601]

Sorry to hear you lost your conversations.
The backup will probably be restored as soon as someone with a Marshmallow device is willing to test that backups don't go to Google.
An example test:

• revert 3bec127020ca9d63f0964672adfc90d5753d3a90
• install the build
• leave something to back up, e.g. send some messages or check your identity key's fingerprint
• uninstall
• install again
• I guess at this point you would either get a registration screen or see your old key and conversations in the case that everything was backed up to Google

[fajabird]

@jedie oandbackup does work for me. If you try to restore on android >=6 you will need to run selinux restore command "restorecon -r ..." on the signal data directory afterwards. There is a unofficial build of oandbackup in the issue tracker which does run the restorecon command automatically here: https://github.com/jensstein/oandbackup/issues/111

Still I would like to see adb back in the backup-game. Helium was for a long time the officially recommended way to backup and restore signal. BTW. Are there plans to finaly implement full export/import in Signal?

[strugee]

@2-4601 thanks, I appreciate it. But mostly I just wanted to highlight the fact that this is a serious issue that can lead to data loss.

[SecUpwN]

@fajabird, I've been running into the exact situation you just described: I updated my ROM from Lollipop to Marshmallow using a dirty flash, obviously that did not work and I got boot loops. I then made a nandroid and did a fresh install of the new Android 6 system, trying to restore only the data partition of my nandroid. That does not work either. So I ended up installing a fresh Android 6 system and trying to restore my apps with oandbackup. Seems like it only restores the apps, but no data. And in the specific case of Signal (I am using the websockets version LibreSignal), it seems as if oandbackup cannot restore the data as well - at least I am prompted with a fresh registration screen. But what is this:

If you try to restore on android >=6 you will need to run selinux restore command "restorecon -r ..." on the signal data directory afterwards. There is a unofficial build of oandbackup in the issue tracker which does run the restorecon command automatically here: jensstein/oandbackup#111.

Which build do you mean? Also I need to restore LibreSignal from my nandroid backup, which oandbackup is currently not able to do - therefore I filed jensstein/oandbackup/issues/114. How to solve this?

[fajabird]

Just follow my link and find it near the bottom. Works flawlessly.

[SecUpwN]

Just follow my link and find it near the bottom. Works flawlessly.

Thanks for letting me know, @fajabird. How to restore from my nandroid with it?

[fajabird]

Does not work with nandroid - for this you could try the pro version of titanium-backup. oandbackup only restores backups created with oandbackup.

[SecUpwN]

Does not work with nandroid - for this you could try the pro version of titanium-backup.

I have tested that with the PRO version of TitaniumBackup, yet I only get back to the registration screen and when re-registering, the data is still gone. I'm currently trying to solve this with the oandbackup dev.

oandbackup only restores backups created with oandbackup.

Since I filed https://github.com/jensstein/oandbackup/issues/114 chances are good that oandbackup may restore nandroids soon.

[atanasi]

I read bits of the AOSP source, and I think full backup includes and excludes specified in a fullBackupContent file affect also adb backups, so that the specified parts are also included or excluded there.
However, if the application has a backup agent, automatic (Google) backups are not done, unless asked for by fullBackupOnly. So I think the solution to try is to include a dummy backup agent.

[eaon]

For those looking for a temporary solution, this is quite dirty and possibly dangerous, but I tried it and it worked for me. I uninstalled Signal 3.9.1 while leaving its data in place (adb uninstall -k org.thoughtcrime.securesms), and installed the last version that allows backups, which is afaik TextSecure 2.28.1 (adb install -r textsecure2.28.1.apk), then pulled a backup also via adb. First installed 2.28.1 on the fresh device, restored the backup, upgraded to the most recent Signal. To make sure push messages work fine I re-registered the number, which reuses the keys.

Looking forward to having adb backup -f … work again! I would hate to have to rejoin the several groups I am part of.

[tsiliakis]

Is there any progress on this? Would be great if we could back up Signal as non-root users !

[SecUpwN]

Is there any progress on this? Would be great if we could back up Signal as non-root users !

@tsiliakis, have you guys tried oandbackup yet?

[mhoran]

A number of users would like a backup solution that doesn't require root (myself, and @tsiliakis included.)

[moxie0]

@tsiliakis Please don't bump issues: https://github.com/WhisperSystems/Signal-Android/blob/master/CONTRIBUTING.md#dont-bump-issues

[fajabird]

@2-4601: I'm willing to test. Can someone send me an old APK Version with backup enabled? My device is running CM13 nightlies and cloud-backup is disabled in the settings. As oandbackup works for me I can try different things very quickly.

[2-4601]

@fajabird Looks like @jan-kiszka has tested this: https://github.com/WhisperSystems/Signal-Android/commit/3bec127020ca9d63f0964672adfc90d5753d3a90#commitcomment-15167013

[pryorda]

Im not understanding the relevance of disabling the ability to back up the apk and data, but you allow a plain text export?

[johanw666]

This is done so Android 6 and above does not backup the messages to Android Drive if you have a Google Play account active. This could lead to messages getting obtained by the NSA/FBI/...

[ghost]

Is there a reliable place I can download the latest version that still allowed backups? I can not find it in the play store (only has the latest version) and I don't know any of the sites that show up when I search for it. I want to try the remove-and-install-older-version trick.

[johanw666]

Removing the old version deletes the stored messages too. And to downgrade an app you need root, in which case there are better options for a backup.

Even then it's probably not going to work, the database format has changed and the old version will probably crash.

However, the backup code is very general: copy just everything from /data/data/org.thoughtcrime.securesms so if you get it installed and it doesn't crash it might work. Fortunately the last version that had encrypted backup (2.0.8) didn't expire or required registration. I put the apk at https://johanw.home.xs4all.nl/TextSecure-2.0.8.apk, please let us know if it works.

[eaon]

If you read more closely, I provided information of backing up with TextSecure 2.28.1. @martijnotto You can get it here https://www.apkmirror.com/apk/open-whisper-systems/signal-private-messenger/textsecure-2-28-1-release/textsecure-2-28-1-android-apk-download/ - if you worry about the source, if you leave the data in place via adb, Android forces you to install an apk with the same signature, otherwise it blocks the installation. Remember I tried this a while ago, there's no guarantees that still works.

[atanasi]

I have used "adb install -r -d" to downgrade. It preserves the data and doesn't need root, only USB debugging. It doesn't matter if the old version supports the database format because the old app does not even have to be started.

So the backup cycle is "adb install -r -d" the old version, "adb backup" and "adb install -r" the current version. For restoring, the middle step is replaced by "adb restore".

[atanasi]

It is better to switch to airplane mode during this so that new messages are not received in the middle.

[ghost]

Thanks for all the suggestions. Downgrading worked perfectly for making the backup. Restoring after a wipe was also as easy as installing the old version through adb, restoring and the installing the most recent version again.

[sicofante]

This is plain crazy. Is an integrated backup being developed as we speak? I can't recommend using Signal to ordinary people if they can't backup their messages. People buy new phones every year. You can't expect them to throw away all their non-plain-text messages.

Please, developers, let us know if something is being done about backups. Signal needs it desperately. Far far more important than a desktop version.

[2-4601]

Please post on this issue if

• you know how to prevent an Android app from sending automatic backups to Google while simultaneously allowing local backups
• you know some workarounds for the current situation

Other posts, such as complaining why nobody hasn't fixed this, are not helping the situation in any way whatsoever

[fajabird]

What about the same way WhatsApp does it - simply save everything to SD-Card/Internal Memory, optionally encrypted by a password. So changing the phone will need to install Signal from scratch and import the backup from SD-Card/Internal Memory. This will not need to allow anything in the manifest which could leak data to google, only writing to SD-card/internal memory is needed. I think Textsecure already had something like this but it has been removed for some reason.

[johanw666]

That code already exists, but is was commented out due to not specified problems who were never fixed. When I uncomment that code it does work though. IMO removig the option gave more problems than it solved.

[2-4601]

Yeah, that's a different issue #1705.

[sicofante]

Today I lost all my messages to oandbackup, and that was a messy procedure by itself. Through the past year I've used three different procedures and lost my messages three times. Enough is enough, guys. This software is not ready for mainstream. I just signed off. Funny that Whatsapp is better than Signal these days. I'll keep an eye on the project and when you understand that keeping messages is important for ordinary people, I'll consider Signal again. (Of course my ~20 contacts in Signal are off as well. They were using it only for me.) Bye.

[BenjaminVanRyseghem]

@atanasi your technic worked like a charm!! Thanks, you saved me a lot of time :smile:

Side, be sure to be offline (flight mode) when you do it, or TextSecure will tend to update itself to signal in the process, making it fail.

[13rac1]

FWIW The workaround process I used to upgrade to a Nexus 5X, thx @eaon and @atanasi :

# 1. Create a plain text backup just in case.
# 2. Put both devices into airplane mode.
# 3. Old phone - Remove Signal and keep data.
adb shell pm uninstall -k org.thoughtcrime.securesms
# 4. Old phone - Install old version.
adb install -r org.thoughtcrime.securesms-2.28.1-156-minAPI9.apk
# 5. Old phone - Create a backup.
adb backup -noapk org.thoughtcrime.securesms
# 6. Old phone - Remove Signal and keep data *again*.
adb shell pm uninstall -k org.thoughtcrime.securesms
# 7. New phone - Install old version.
adb install org.thoughtcrime.securesms-2.28.1-156-minAPI9.apk
# 8. New phone - Restore backup
adb restore backup.ab
# 9. New phone - Disable airplane mode.
# 10. Upgrade Signal using Play Store.
# 11. Shutdown phones to swap SIM card.
# 12. Restart and done.

[Lucent]

I'd like to make a donation or set a bounty to get full database migration from phone to phone implemented. Who can I contact about doing this?

[Trolldemorted]

i don't think the implementation is the problem (as there are most likely developers who would do it for free), but rather the current state of signal-android. If no pull requests fixing bugs or implementing features on the roadmap get merged, nobody sees a reason to start working on it.

[johanw666]

The implementation is easy, just un-comment some code, make a few minor adjustments to make it build and remove the shared library from the backup to be clean.

[ghost]

In Silence is already (re)enabled (https://github.com/SilenceIM/Silence/commit/4d2b4a25575812cd71316576f7689249e4834162)

[perguth]

I experienced a problem applying the backup solution summarized by @eosrei:

I did two backups in a row. device A -> device B -> device C. On Device C I could no longer send messages (always just the bouncing dotts) and I could not link to desktop signal (the camera for scanning didn't activate at all).

To solve this I had to go back to the original backup from device A and use that.

[tvidas]

When performing the workaround listed by @eosrei, you need to ensure that Signal is not currently running when trying to install the downgraded version on the old device (step 4). This error condition might be indicated by [INSTALL_FAILED_VERSION_DOWNGRADE] in logcat.

[perguth]

The workaround does no longer work for me - all messages are missing on the new device. Using Android 7.1.1, Signal 3.29.6, ADB 1.0.36.

Signal keeps my messages safe. So safe I have to let them go when switching devices. Yay.

[spaletta]

@pguth – to be fair, you can take the messages by making a plaintext backup and restoring from it. You just can’t keep your key/identity.

[perguth]

@KillerDiller We use Signal to capture moments of our life in pictures and audio snippets don't we?

[sicofante]

Maybe it's OK for a secure messaging app not to allow backups, but I think it should be stated with big bold letters for new users to see.

Maybe Signal is great for spies or people being oppressed by their governments. For ordinary people willing to preserve moments of our lives -like Per Guth just pointed out-, it's near useless.

Wake me up when the devs understand the needs of ordinary people and provide a solid, secure backup system allowing us to upgrade phones without destroying our memories.

[strugee]

@sicofante the Signal developers don't have unlimited time or money. Remember that WhisperSystems is a non-profit.

If you want to help solve this, see https://github.com/WhisperSystems/Signal-Android/issues/4577#issuecomment-165331721.

[johanw666]

Finally a good excuse to root and use Titanium backup. Or build your own version with encrypted backup put back in.

[sicofante]

Nobody has unlimited time and money, but nope, that's not the point.

They provide an application and us, the users, provide feedback. If backups aren't deemed essential, some users like myself just move on and tons of others don't even bother.

I perfectly know the open source excuse ("just code it yourself", "patches are welcome", etc., etc., etc.). I'm no developer. I'm a user.

The devs must decide if this is a hobby project or a professional project. If it's the latter they should listen to basic reasoning fom the users (who on Earth creates an application that can't backup it's own data??). If it's the former, everything is just great as it is and I wish them unlimited enjoyment from their hobby.

[moxie0]

@sicofante I agree that backups are important, just like many other open issues. I'm working as hard as I can every day, and will get to it eventually. In the mean time if people want to help, I've done my best to outline how. Hope you understand.

[Lucent]

The workaround still works. Just migrated a database from an S7 Edge on Android 7 to an S8+ on Android 7. Reboots were required after each uninstall due to INSTALL_FAILED_VERSION_DOWNGRADE error from adb.

Six more months until I have to migrate again to the Note 8. I'd like to seed the bounty for implementing a full database backup with $100. Someone who's more experienced is welcome to set up the collection and distribution of the bounty.

[ghost]

An upgrade today didn't work for me anymore. It shows all the conversations, but all messages show 'Error decrypting message'.

[strugee]

@martijnotto this is not the appropriate place for a report like that. Search for duplicates and if you don't find any, file a new issue with WAY more detail than you just gave. Make sure to completely fill out the issue template.

[ghost]

@strugee this is actually the place, as this topic is about upgrading without losing the messages. Apparantly the 'usual' way of downgrading, backing up and then installing the old version on the new/upgraded phone and restoring doesn't work. At least it didn't in my case going from 5.1.1 to 7.1.1.

This is relevant information for people in this discussion because people might choose to

• not upgrade their phone/get a new one to avoid losing Signal data
• not waste time mucking about with adb because it doesn't work anyway

[strugee]

@martijnotto uh... you meant a phone upgrade?

You didn't specify that. So we were left to assume you meant upgrade Signal.

[kyanha]

@strugee Your users are not as clueless as you assume they are. This is an issue which has frustrated many, for close to a year and a half. Please consider increasing this issue's priority.

Or, perhaps, create a protocol by which conversations may be securely transferred into another app on the device without having to be copied plaintext through the clipboard. Perhaps some form of permissioned intent to perform an ECDH/curve25519 key and digest negotiation, then a set of intents to actually transfer plaintext conversations under that encrypted cover. I'd create that protocol, but it seems reasonable to assume that you will never trust any protocol for secure message transfer that you don't create yourselves.

Yes, it means that the user must trust the security of the software they use to invoke that function, but your users are capable of more than you think they are, are capable of determining their own threat models, and if you don't provide the function you ultimately end up driving your users to root their phones (which by definition makes them less secure and more open to malware stealing their messages anyway).

(Or, if you've started using/trusting things that other people have built, you might consider the use of SQLCipher, which is used in https://play.google.com/store/apps/details?id=com.twistedplane.sealnote with source at https://github.com/vishesh/sealnote/ to store its notes and export copies of its note database.)

[strugee]

@kyanha I'm not associated with WhisperSystems and I can't reprioritize this issue.

People have stated above what needs to happen for this to be resolved. If it's important enough to you, you can follow those steps.

That being said you should maybe read https://github.com/WhisperSystems/Signal-Android/blob/master/CONTRIBUTING.md#development-ideology again. In my experience running privacy events, etc. users are indeed "clueless" (though I wouldn't use that term). If Signal wants mass adoption it has to be really easy. That's what good UX is.

[kyanha]

When people outline steps that "need to happen", but don't actually fulfill them themselves, I've found that often those steps are impossible to fulfill. "It should be possible to do this!" ... well, it's often not.

Thus, I'm looking at alternatives that still provide a better amount of security than the "export to plaintext and hope the channels/midpoints/endpoints used to transfer the plaintext archive aren't compromised" option that is currently provided, while still keeping the "cannot allow backups to the cloud" policy that Whisper Systems appears to wish to impose.

[strugee]

@kyanha the steps outlined in https://github.com/WhisperSystems/Signal-Android/issues/4577#issuecomment-165331721 are very reasonable.

[kyanha]

Except that the "cannot allow backups to the cloud" policy becomes violated by those steps.

[johanw666]

Well, there used to be an encrypted option that worked quite well (except that it also backupped .so library files but those can be skipped easily). I run a personal version that has this built in again. If you make this option foolproof by resetting all secure sessions after restore I think this is good enough for public release, but Moxie disagrees and stays focussed on using the system sms database and some external tool that uses it.

[haffenloher]

Hey, please keep discussion to the community forums and don't post to the issue tracker unless you have new information to add which could help solving the issue. This helps keeping the issue tracker clean and limits the amount of time everyone watching the repo (>500 people) has to spend reading through email notifications. Thanks!

[shaaati]

I just tried a migration from my Nexus 5 running stock 6.0.1 to a Oneplus 5 with 7.1.1. I followed the process documented by @eosrei and it initially failed, as reported by others in here who tried a restore on a recent Android version. More exactly, some of the settings seemed to have been applied (dark theme, list of linked devices) but not a single conversation was visible.
I am not an Android export but I thought this might be related to SELinux stuff (although SELinux should be in place since Android 5 from what I found).

With some research I found that issuing adb shell restorecon -Rv /data/data/org.thoughtcrime.securesms after importing the backup via adb and before upgrading the app via Play Store did the trick. When following the process from https://github.com/WhisperSystems/Signal-Android/issues/4577#issuecomment-228627416 include this command as step 8.1.

I rooted the Oneplus 5 before doing all this so I do not know if this is a viable solution for non-rooted devices.

[snapfractalpop]

For anyone using the workaround mentioned from #4577 (comment), it was going smooth for me until I hit a snag. On step 8, when performing the backup, BackupManagerService has a moderately fast timeout for individual items. So, if you have a large video, or other bulky file, it may timeout and cause the backup to fail.

I will lose my data as a result of this, however I did do a plaintext backup, and saved various images (they will lose their context). I suppose I will lose group messages. I've wasted a good amount of time on this, and I hope that this can help save someone else from my fate.. if you do try this workaround, beware of the restore timeout (there doesn't seem to be any documented way of overriding it).

[smarek]

Last item in discussion https://whispersystems.discoursehosting.net/t/encrypted-backup/1227 mentions the proposal that can be merged if I provide it in PR, the bounty still available? :smile_cat:

Also @moxie0 can you please confirm that this PR will be merged if implemented as you described previously? My last work on #6886 and ton of other PRs are ready-to-merge for a long time, and there is no activity from your side ...

[automated-signal]

GitHub Issue Cleanup:
See #7598 for more information.