Wire-webapp: Feature request: conversation history backup (import / export)

[maximbaz]

Today Wire can only be used when it is acceptable to lose the entire conversation history with all contacts. A single wrong step (could be a user action or even a bug), and the device is not recognized anymore and is registered as a new device, with all the history gone.

The reasons for not synchronizing the history are well understood, they are described in #473. However, the pain point is still there, and nobody would use Wire if they know that they might require to refer to this conversation history later — it's just too easy to lose it forever.

What I'm proposing is basically a import / export functionality.

• Export will just decrypt every available message using the current device's key and write them in a file.
• Import would read the file, encrypt all messages with the new device's key and send them to the server.

These are heavy operations, and the server can throttle such requests if needed — there's no need to be able to make _blazingly fast_ backups.

For desktop apps this could be improved even further, by creating an --export CLI parameter, that would export everything into a backup file. This way the backup procedure could be automated and even integrated with other backup tools.

Configuration should at least contain a "start date" parameter for people to make incremental backups, that would also drastically reduce the load on servers.

---

Depending on the way the messages are stored in the database, it might get complicated to handle message duplicates. The security whitepaper states that server receives a bundle of messages encrypted for every device of every participant, but it doesn't say if server _stores_ it as a single bundle or as a bunch of separate records. It get's tricky in the former case (we can't update or delete that single record), but should be simple in the latter case (we just add one more message encrypted for one more device). Which one is it, by the way? And is there a mechanism that cleans the database from messages that cannot ever be decrypted anymore, since all the devices that could have decrypted it are already deleted?

[lipis]

What about the deleted/edited messages..? Ephemeral? The whole idea of the privacy is that when I delete a message I know that is deleted from "everywhere".. while I understand that people can take screenshots, copying etc.. having the export/import built in would raise some other issues.

[maximbaz]

I want to export only that what I can see in the app at this moment. If the message was edited, I see it edited in the app, and I want to export it in the edited state as well. If the message is ephemeral or was deleted, I don't see them in the app, so I don't want them to be exported. Simple :)

Like you say, it's exactly like copy-pasting the history, but a nice way of doing it. We can also call this a "migration" scenario, I want to be able to painlessly switch to another device and still have a context of what I was discussing before the switch, without manually scrolling and copy-pasting the history. I don't want the history to be _automatically_ synchronized (for the reasons described in #473), but a manual way to preserve the context is needed.

Frankly, the import feature is to some degree useless without the search functionality made first (https://github.com/wireapp/wire-desktop/issues/218, https://github.com/wireapp/wire-desktop/issues/76), but if the export is done in a text format, I can use other tools to search in it.

[lipis]

Yes I understand that.. my point wasn't what we are going to do or what to backup.. but more philosophical.. Whenever I'm deleting a message.. I don't want this message to be backed up somewhere on your machine! At least I wouldn't like Wire to provide the tools to keep it.. Similar idea of Snapchat. Of course nobody can stop the screenshot.. but at least the original poster would get a notification that the image was stored.. (not sure if it's still happening).

[maximbaz]

Hmm, I see your point... But honestly I don't think someone would use backup functionality to save a compromising message that you had sent, it's much easier to make a screenshot :) Simply because then they have to restore backup, and only then to either copy text or make a screenshot — it's two extra steps comparing to just making a screenshot on the spot.

I'd also question how often people edit/delete messages that were posted _earlier than one hour ago_. If one posted something compromising, one would usually edit/delete it within 15 minutes, I think...? Is it possible for you to fetch such statistics from the database? If it proves to be true, perhaps a sufficient thing would be to limit export functionality, not to export anything posted within the last 60 minutes?

[lipis]

I'm just raising the issue. If we all about security/privacy and we have a built in functionality that exports the data in "plain text", it might not be that private anymore :)

[helenakos]

@maximbaz Thanks for the request! This feature is in the plans and on a longer term roadmap, but unfortunately no committed date yet. I will close the issue due to cleaning up.

[subversive-owl]

@helenakos Just checking in, is there no committed date for this yet?
(If so, are there feature bounties or another way we can push up the priority of this feature?)

[fhamborg]

Any news on this @helenakos ?

[maximbaz]

I've just stumbled upon wire.com/features and discovered History backup (coming in 2018). Great news!

However, this is currently displayed under the "Teams — built for business" section.
Could you please confirm whether this feature is planned to only be available for paid subscriptions?

If so, I'd like to express my disapproval of this decision. I had to change devices a couple of times already, and losing the history was a serious issue for me every time. How ever painful it was, I'm not considering to pay 5€ per month exclusively for history backup.

[lefherz]

I'd also suggest to encrypt the backup file with a passphrase when you are already on it. You can't expect everyone to have an encrypted hard drive for safe storage of the history backup file.

[rkagerer]

Really looking forward to this feature. The inability to migrate my history to a fresh installation is a key impediment to me adopting this app.

The passphrase suggestion is a nice touch. Although it would be handy if it's optional so that chats could be exported to other tooling for more advanced search, to avoid "vendor lock-in", etc.

[starkshaw]

Really looking forward to see if this feature is eventually developed. I have changed my phone and it is very hard to track the old messages that might be useful today...

[maximbaz]

I had this idea on how to make this feature much simpler to implement, I'm curious what you guys think.

How about creating client-side backups only, and not involve server in any way? That is, backup becomes as simple as backing-up IndexDB (e.g. export it as a json file), and import really becomes merging entries from an earlier exported IndexDB into the current database. Entries in IndexedDB are already decrypted, so no need to deal with encryption/decryption at all.

This is not a cross-platform solution obviously and will work only for webapp and desktop apps, but nevertheless it sounds like a much simpler thing to implement.

/cc @lipis, @gregor

[gregor]

@maximbaz Thank you very much for your input. As history backup has been put on our official roadmap, we are discussing ideas on how to approach the user problem we are trying to solve internally. Your feedback and ideas are welcome and will definitely be part of those discussions. At this time we cannot comment on the matter any further than that the feature is on our roadmap and will come. Which flavor and how the implementation is going to look like is not yet finalized - which is also true for the actual realease and the customer base that will benefit from it. For now we have announced it as a business feature to come in 2018.

[lipis]

Just an update: https://medium.com/@wireapp/history-backup-comes-to-wire-cd79e02ec66e

[maximbaz]

Thanks for the update, I'm very excited to try it out on desktop 🙂

Backup files are not cross-platform compatible, for example iOS backup can’t be restored on Android or desktop. However, a Windows backup does work on macOS and the other way round too.

Could you shed some light on whether this is going to be addressed any time soon, or this is a fundamental design limitation?

I'll explain why I'm asking.

Like most people, I use Wire on desktop and on a phone. With non-cross-platform backups, I will have to do regular backups, like once a week or once a month, on all devices.

If backups were cross-platform, I would never have to care about that. It is unlikely that I will lose both devices at the same time, so if backups were cross-platform and I lose my phone, I could instantly export the most up-to-date history from the laptop and import it on a new phone. Same the other way around, should a bug happen and Wire greets me with "You are using a new device", I could export the most recent history from the phone and import it on the desktop.

[KopfKrieg]

Backup files are not cross-platform compatible, for example iOS backup can’t be restored on Android or desktop. However, a Windows backup does work on macOS and the other way round too.

I'm curious and just need to ask: Why? Wouldn't a standardized backup format be better? Or are there some platform dependent limitations I'm not aware of?

And of course: Will this be addressed in the future? I've recently installed Wire on my laptop (Linux, before that I've used the web client), and my phone (Android, got a new phone), and my complete history is now only available on my work station (Windows). It would be nice to create a backup on Windows and import it on my other devices.

[finnoleary]

This solution is bad for two reasons:

1) Media is stored as links. This means that not only do the media files essentially belong to you (Under all circumstances you keep the files), but also if your servers go down, or are blocked, all of the images I have sent someone are lost. I am thus hesitant to send any media files.
(This also prevents me from saving media files and deleting them after N time limit to ensure that old media cannot be passed to $privacy-invasive-agency).

2) As other people have mentioned, the lack of cross-platform backups is a problem. However in my opinion this extends further, it tells me that as a developer you are not committed to keeping backups in a standard format. For all I know, you could introduce breaking changes that break restoration of older backups, which could contain important data to me (Having deleted the original logs for security andor space reasons).

msgpack.org, capnproto.org, developers.google.com/protocol-buffers would easily solve this problem. From a design standpoint there shouldn't be anything you need to store that's implementation-dependent.

[Sigmun]

Indeed, the lack of cross-platform backups is a huge problem. Moreover, like I explained in this ticket (#770), this feature could be used on back-end in order to allow history synchronization. But for that, you need a cross-platform backup!