Signal-desktop: Error: Encrypted export/import is disabled

[501st-alpha1]

I just tried to export my Signal data from the desktop app, so that I can archive it in a Git repo that I sync to my server.

To my surprise after I did

Signal.Crypto.encryptSymmetric = async function (key, plaintext) { return new Uint8Array(plaintext); }
Signal.Backup.exportToDirectory("/tmp", {key: true});

in the console, I got Error: Encrypted export/import is disabled.

I tracked it down to this line, and blame indicates commit 8eeaad8e18823f83fa2c66b82616552b6d2b04e9, which merely says:

Remove unused export/import functionality

I haven't been able to find any discussion of this in the GitHub issues; why was this removed? I noticed that if I comment out the throw line, the export works fine.

Given the interest in #522, #1013, and others, I think this feature was actually in use, though I can't say how widespread it was. I personally don't care as much about having an import available, as long as I can read my data outside of Signal.

Can this please be re-enabled?

[scottnonnenberg-signal]

We'll re-enable it when it's ready for widespread use. I'm glad you were able to find a workaround for the near term.

[betrieb]

I would like to state here that a signal-desktop backup/archive function would be much appreciated. Even if it remains console based.

@501st-alpha1 Could you elaborate on how to edit the js file to disable the error?
I have tried to comment out the above mentioned line in my local app.asar file which is under "C:\Users\'Username'\AppData\Local\Programs\signal-desktop\resources". However any change I make prevents Signal-desktop to open. I get a message "A javascript error occurred in the main process".
I'm on Windows 10.

[501st-alpha1]

@betrieb Rather than edit the actual source code, I did I live edit from the debugger. In the console, I clicked the link near the error I mentioned, and it opened up the line in the backup.js file I linked to. From there you can comment out the throw line and save the file, and then the export will work.

[betrieb]

Thanks! It worked via the console edit. This is great

[IgiMar]

I agree on the possibility of moving Signal Desktop data to another machine.
It was possible via the Chrome extension and the irreversible migration option. But just one time.
Could not be possible to do it a second time now.

[nomeata]

@501st-alpha1 Thanks for pointing how to get past that line. But when I do that, I get an empty attachements folder. That used to work.

Is that the same for you?

[nomeata]

Oh, and also many files have messages = [] when they didn’t before. It seems this is broken :-(

[sideburnie]

this is a bummer. trying to upgrade (ios) phones and only now do i learn my messages may be truly ephemeral... anybody have the latest commands to hack around it being disabled? maybe some way to dump and decrypt the DB?

[moyix]

Yes, this doesn't seem to work – is there any workaround to export messages for backup? Even if there's no way to re-import it it would be nice to have the ability to archive.

[scottnonnenberg-signal]

Where previously you needed to export your data to take it elsewhere (because it was in IndexedDB) you can now just take the database itself. It's SQLCipher, so you need to unlock it with a key, but it's a normal SQLite database after that. As of 1.17.x conversations are there, and since 1.15.x messages have been there. Since spring 2018 the attachments.noindex folder has had all the attachments.

[sideburnie]

I'm using signal desktop 1.16.2 on Ubuntu 16.04 (4.15.0-36-generic #39~16.04.1-Ubuntu SMP). I'm using sqlcipher version 3.8.6 2014-08-15 from apt.

I've assumed the key mentioned is in /home/`whoami`/.config/Signal/config.json
I've assumed the database mentioned is in /home/`whoami`/.config/Signal/sql/db.sqlite

I've tried methods described
here
and here

➜  signal sqlcipher db.sqlite 
SQLCipher version 3.8.6 2014-08-15 11:46:33
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> PRAGMA key = 'my_key_pasted_from_config.json';
sqlite> ATTACH DATABASE 'plaintext.db' AS plaintext KEY '';
Error: file is encrypted or is not a database```
sqlite> .backup plaintext.db
Error: file is encrypted or is not a database

I've also tried DB Browser for SQLite (both the versions in the package manager, and the ones in ppa:linuxgndu/sqlitebrowser, which from what I read, is supposed to be able to read sqlcipher encrypted databases now), and I get the message

Could not open database file.
Reason: file is encrypted or is not a database

I think I must be missing something simple.

[moyix]

You're very close, the only thing you need to change is

PRAGMA key = 'my_key_pasted_from_config.json';

For raw keys the format is (not my real key):

PRAGMA key = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'";

(otherwise it will assume it's a passphrase and do key derivation, as described here).

I was able to successfully dump out the DB as a plaintext SQLite database with this.

[sideburnie]

[sideburnie]

for those interested in exporting to CSV

➜  signal sqlite3 plaintext.db 
SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite> .mode csv
sqlite> .headers on
sqlite> .out messages.csv
sqlite> select * from messages;
sqlite> .quit

Thanks again @moyix!

[nomeata]

Nice, that looks like a proper way of exporting data! And maybe someone soon builds a tool that turns the .csv and the attachements into a nice static HTML page for proper browing … I am using these archives to also for full-text search, which Signal does not offer yet.

[501st-alpha1]

@nomeata To answer your earlier question, I haven't had any issues with the export recently, though I'm stuck on version 1.14.4 due to #2604.

Thanks everyone else for the info about how to export the data in newer versions of the app; I will try that out once I'm able to upgrade.

[nomeata]

Since I was asked by a friend, here again a summary of all steps. This is for a Debian or Ubuntu system.

• (Only once,) install sqlcipher:
  apt get install sqlcipher
• Maybe start in an empty directory where you want to store your archive:
  shell mkdir signal-archive cd signal-archive
• Run
  shell cat ~/.config/Signal/config.json
  The output should look something like
  json { "window": null, "key": "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c" }
• Run
  shell sqlcipher ~/.config/Signal/sql/db.sqlite`
  In that program, enter these commands (using the key you just saw):
  sql PRAGMA key = "x'b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c'"; .mode csv .headers on .out messages.csv select * from messages order by conversationId, sent_at; .quit
  You now should have a file messages.csv in the current directory with all your messages.
• Copy ~/.config/Signal/attachments.noindex/, which contains all media files:
  shell cp -r ~/.config/Signal/attachments.noindex/ attachments

You now have a complete archive in your current directory (although a bit inconvenient to use, unfortunately.)

[patrikhuber]

It's a shame that this was closed and no real solution exists. (@nomeata, thanks for the write-up, very useful, but Signal really needs a first-class citizen way of doing this).

[rph8]

I followed @nomeata's exact instructions on macOS with Signal Desktop 1.22 for macOS and after doing

PRAGMA key = "x'123456789ABCDEF123456789ABCDEF123456789ABCDEF123456789ABCDEF1234'";
.mode csv
.headers on
.out messages.csv
select * from messages;

on the sqlcipher CLI I get

Error: file is not a database

When instead trying .tables right after setting the key, I get the same error message.
I did extract the key from config.json.
What am I doing wrong? Did something change since last year?
Feels like I must be very close.

[five-c-d]

@rph8 , the signalapp github is used for bug-reports rather than for tech-troubleshooting of unofficial workarounds :-) If you can please open a new thread over at https://community.signalUsers.org , or if you prefer just add a comment to https://community.signalusers.org/t/cannot-read-db-sqlite-windows/6580 which already exists, that IS the place where unofficial workarounds and tech-troubleshooting are likely to get figured out.

To answer your question, the problem is that you did not quite follow nomeata's instructions, because they specifically said that you needed Linux, and you are on MacOS... which has very similar signal4desktop codebase, but a different directory-structure. Here are the helpdocs, https://github.com/signalapp/Signal-Desktop/blob/master/CONTRIBUTING.md#the-staging-environment and you can see there MacOS prefers ~/Library/Application Support/Signal/sql/db.sqlite rather than ~/.config/Signal/sql/db.sqlite that is used on Linux.

p.s. No this was not tested on an actual MacOS. But if my guess as to the answer was wrong, please ask in the signalUsers forum, not here on github, which was my main point, somebody there will have tried this on OSX (or be willing to help test).

[mattparlmer]

@nomeata that walkthru was enormously helpful, thanks!

[ionphractal]

Just to complete information here.

@rph8

Error: file is not a database

PRAGMA key = 'YourPasswordKere';
PRAGMA cipher_compatibility = 3;
...

did the trick for me. See https://github.com/sqlcipher/sqlcipher/issues/299#issuecomment-448744465

[ywwg]

PRAGMA key = 'YourPasswordKere';
PRAGMA cipher_compatibility = 3;

This still doesn't work for me:
Error: file is encrypted or is not a database

[zormit]

For me it worked with SQLCipher version 3.30.1 without the cipher_compatibility setting.

[crodjer]

For me it worked with SQLCipher version 3.30.1 without the cipher_compatibility setting.

This worked for me. I had to build sqlcipher from source based on https://www.zetetic.net/sqlcipher/introduction/
and I could simply follow the steps as per: https://github.com/signalapp/Signal-Desktop/issues/2516#issuecomment-442797638
Thanks @nomeata and @zormit

[rubo77]

Signal.Crypto.encryptSymmetric = async function (key, plaintext) { return new Uint8Array(plaintext); }
Signal.Backup.exportToDirectory("/tmp", {key: true});

I got the error in the console (CTRL+Shift+i):

Promise {<rejected>: Error: export is only supported in test mode
    at Object.exportToDirectory (/opt/Signal/resources…}__proto__: Promise[[PromiseStatus]]: "rejected"[[PromiseValue]]: Error: export is only supported in test mode
    at Object.exportToDirectory (/opt/Signal/resources/app.asar/js/modules/backup.js:1215:11)
    at <anonymous>:1:15
/opt/Signal/resources/app.asar/js/modules/backup.js:1215 Uncaught (in promise) Error: export is only supported in test mode
    at Object.exportToDirectory (/opt/Signal/resources/app.asar/js/modules/backup.js:1215)
    at <anonymous>:1:15

[rubo77]

* In that program, enter these commands (using the key you just saw):
  ```sql
  PRAGMA key = "x'b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c'";
  .mode csv
  .headers on
  .out messages.csv
  select * from messages order by conversationId, sent_at;
  .quit
  ```

Can you add those commands somehow to the script at https://unix.stackexchange.com/a/505009/20661 ?

[501st-alpha1]

I tried the new export method again, and I was still getting the error that @ywwg and @rph8 have reported:

Error: file is encrypted or is not a database

I tried the cipher_compatibility command, but that didn't work either, and after digging around in the source code it seems like it shouldn't be needed anyway.

This was after installing the latest version of the Debian package from updates.signal.org (shows as version 1.39.4). I had Signal installed before, but when it launched it prompted me to delete the old data, so the database should be freshly created.

The error occurred after issuing any command after opening the database, even a simple SELECT query or .databases command. I know it wasn't an error copy/pasting the key, because this didn't work either:

sqlcipher ~/.config/Signal/sql/db.sqlite \
  "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; select * from messages limit 1;"

(I also tried the old export method, and it looks like the exportToDirectory function still exists, but even if I disable the test mode check, it gives an error on the key, so the old solution of {key: true} doesn't work.)

Finally I noticed that Debian only has SQLCipher 3, while Signal now uses SQLCipher 4. Rather than compile it from source, I found this Docker image, which contains version 4. I ran it like this:

docker run --rm --volume="$HOME/.config/Signal/sql/db.sqlite:/db.sqlite:rw" \
  --entrypoint sqlcipher yspreen/sqlcipher /db.sqlite \
  "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; select * from messages limit 1;"

and it worked for me. So the complete export command becomes:

docker run --rm --volume="$HOME/.config/Signal/sql/db.sqlite:/db.sqlite" \
  --volume="$(pwd)/plaintext.db:/plaintext.db" --entrypoint sqlcipher yspreen/sqlcipher db.sqlite \
  "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; attach database 'plaintext.db' as plaintext key ''; SELECT sqlcipher_export('plaintext'); DETACH DATABASE plaintext;"

(Note that the file plaintext.db must exist beforehand or it will be mounted as a directory. Do that with touch plaintext.db.)