Signal-desktop: Database startup error: Error: SQLITE_NOTADB: file is not a database

[eppfel]

• [x] I have searched open and closed issues for duplicates
  Related, but outdated: #2712 #2609

Bug Description

I just started Signal and receive the error Database startup error: Error: SQLITE_NOTADB: file is not a database with the option to delete all data and start new.

What is strange is, that I have a Signal and Signal-betafolder in ~/Library/Application Support/. The files in beta however are all created on 2020-01-08 and no changes were made after that. I thought I was using beta but somehow I did not.

Also interesting that it tries to migrate the data with each restart.

I have used Signal on and off on that machine, not on a daily basis.

Screenshots

Platform Info

Signal Version:

1.35.2

Operating System:

OS X 10.14

Linked Device Version:

4.71.2

Link to Debug Log

No link, as App did not start, so here the log file.

{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"app ready","time":"2020-09-12T07:47:54.231Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"starting version 1.35.2","time":"2020-09-12T07:47:54.231Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-09-12T07:47:54.261Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-09-12T07:47:54.263Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-09-12T07:48:24.204Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"app ready","time":"2020-09-12T08:45:16.683Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"starting version 1.35.2","time":"2020-09-12T08:45:16.684Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-09-12T08:45:16.698Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-09-12T08:45:16.700Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-09-12T08:45:21.291Z","v":0}

the config.js holds a key.

[scottnonnenberg-signal]

@eppfel When you see this error, it means that something has gone wrong with your database on disk. It happens most frequently when your computer loses power unexpectedly, or you have to shutdown your computer abruptly. Did anything like that happen recently?

[eppfel]

Thanks for getting back to me. I don't recall it, but the laptop is old so the battery discharges quickly.

[scottnonnenberg-signal]

Prior discussion about this error is here: https://github.com/signalapp/Signal-Desktop/issues/2609

[eppfel]

This seems different in the sense, that it fails migrating the databse in my case. Could this be related to an update?

[scottnonnenberg-signal]

It's definitely the same thing. We are prepared to migrate a database (encryption ciphers or database schema, as needed) on every database open.

[bjdooi]

I'm having the same exact issue (probably due to my computer randomly shutting off during an update). I contacted [email protected] like the previous issues said to do but haven't heard back from them yet

[primeos]

This just happened to me too after a normal reboot (no crash or anything and a FS corruption seems unlikely):

{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"app ready","time":"2020-12-08T14:25:15.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"starting version 1.38.2","time":"2020-12-08T14:25:15.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-12-08T14:25:15.478Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-12-08T14:25:15.481Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-12-08T14:25:19.715Z","v":0}

I'm on GNU/Linux with Signal-Desktop 1.38.2. I'll try to find out more.

Updates:

This is the log of the last application shutdown:

{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:14:51.665Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:15:46.961Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:39.017Z","msg":"Removing notification","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:40.032Z","msg":"SQL channel job 24509 (updateConversations) succeeded in 12ms","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"close event {\"shouldQuit\":false}","time":"2020-12-07T17:16:42.391Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"requestShutdown: Requesting close of mainWindow...","time":"2020-12-07T17:16:42.393Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.077Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"background/shutdown","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"attachment_downloads/stop: disabling","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"MessageReceiver: stopProcessing requested","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"MessageReceiver.close()","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"WebSocketResource.close()","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"drained","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.397Z","msg":"MessageReceiver: unregister batchers","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"requestShutdown: Response received","time":"2020-12-07T17:16:42.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"before-quit event {\"readyForShutdown\":true,\"shouldQuit\":false}","time":"2020-12-07T17:16:42.450Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"close event {\"readyForShutdown\":true,\"shouldQuit\":true}","time":"2020-12-07T17:16:42.450Z","v":0}

There seems to be no indication that anything did go wrong. I've created a backup right after closing Signal-Desktop but the DB (~/.config/Signal/sql/db.sqlite) from the backup has the same hash and is also corrupt (I cannot open it manually using sqlcipher and since it's encrypted I cannot really investigate why...).

A comparison of the file sizes (last working backup vs. corrupt DB):

$ du -h Signal/sql/db.sqlite Signal-corrupt-sqlcipher-db/sql/db.sqlite
56M Signal/sql/db.sqlite
57M Signal-corrupt-sqlcipher-db/sql/db.sqlite
$ du -b Signal/sql/db.sqlite Signal-corrupt-sqlcipher-db/sql/db.sqlite
58351616    Signal/sql/db.sqlite
59084800    Signal-corrupt-sqlcipher-db/sql/db.sqlite

But only the first 16 bytes of db.sqlite seem to be identical in both files (I didn't look at the SQLCipher documentation; not sure if it should look that way or not).

[ryantrinkle]

I'm trying to upgrade from v1.37.3 to v1.39.4, and I"m getting this same error (NOTADB). If I manually start v1.37.3, everything still works and all data is still present.

[ryantrinkle]

I've now narrowed this down a bit: I was able to step through versions up to 1.38.2, which runs successfully. Running 1.39.2 (which seems to be the next release after 1.38.2) results in NOTADB.

Set Windows Application User Model ID (AUMID) { appUserModelId: 'org.whispersystems.signal-desktop' }                                              
NODE_ENV production
NODE_CONFIG_DIR /nix/store/8x3rqj3j9dg2g3f842xwymrhffvi60y6-signal-desktop-1.39.2/lib/Signal/resources/app.asar/config
NODE_CONFIG {}
ALLOW_CONFIG_MUTATIONS undefined
HOSTNAME undefined
NODE_APP_INSTANCE undefined
SUPPRESS_NO_CONFIG_WARNING undefined
SIGNAL_ENABLE_HTTP undefined
userData: /home/ryan/.config/Signal
config/get: Successfully read user config file
x-attr dependency did not load successfully
config/get: Successfully read ephemeral config file
making app single instance
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"app ready","time":"2020-12-24T02:33:44.571Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"starting version 1.39.2","time":"2020-12-24T02:33:44.571Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"media access status undefined undefined","time":"2020-12-24T02:33:44.572Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-12-24T02:33:44.586Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-12-24T02:33:44.587Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-12-24T02:33:48.441Z","v":0}

[scottnonnenberg-signal]

@ryantrinkle It's really surprising that the changes from v1.37 to v1.39 would result in that kind of error, since no low-level database changes are included. How are you installing Signal Desktop?

[primeos]

I hate Signals way of dealing with long term conversations and history archival, it sucks - there are no proper ways of dealing with this. @moxie0

+1, please do something about this.

Finding and fixing the bug might be a challenge but in any case Signal-Desktop should (IMO) at least:
1) Support making backups and importing them without relying on 3rd party tools: https://github.com/signalapp/Signal-Desktop/issues/522
- https://support.signal.org/hc/en-us/articles/360007059752-Backup-and-Restore-Messages ("Manually transferring folders is not supported and may result in errors. Signal uses a stateful protocol, meaning that you cannot simply copy files to another instance of Signal or copy files after linking and using Signal.")
- 3rd party: https://github.com/NixOS/nixpkgs/issues/106038
2) Support fetching(/importing) older messages from the phone (if the user wants it): https://github.com/signalapp/Signal-Desktop/issues/1651

[neonfuz]

I have been getting network errors in signal today so I looked at issues for my distro (nixos) and found this bug, so I quit signal to backup my signal folder, and upon re-opening it I had a corrupted database... very annoying, my phone is broken right now so its going to be a pain getting access to signal now.

[fdietze]

I have the same problem. So what are my options now? Is there a way repair the corrupted database? Do I have to delete my history? Should I wait for a fix?

[backuitist]

You could check the first 16 bytes of your db file:

$ hexdump -C ~/.config/Signal/sql/db.sqlite | head -1

You're supposed to see this:

00000000  53 51 4c 69 74 65 20 66  6f 72 6d 61 74 20 33 00  |SQLite format 3.|

On my broken DB I had this:

00000000  f0 fd 28 28 7e 5b 5a d7  8f 3f d1 ea 60 73 1c 47  |..((~[Z..?..`s.G|

I couldn't find any tool able to repair a DB with a bad header...

[fdietze]

Hmm, I have:

00000000  f7 93 95 e1 ef b4 ad ee  98 3b ac 32 54 15 84 10  |.........;.2T...|

:disappointed:

[mehdibo]

Same error just happened to me today

Database startup error:

Error: SQLITE_NOTADB: file is not a database

[fdietze]

Since I wanted to use Signal on my Desktop again, I just deleted my whole chat history. That felt really bad...

[panicfarm]

This Database startup error: Error: SQLITE_NOTADB: file is not a database on Windows 10 makes the program unusable- it has destroyed my chat history on two occasions. It only happens after the app autoupdate, when I quit and restart the same version of Signal, the DB is opened fine.
When I chose "Delete All Data" on the error dialog, it shows

```Unhandled Promise Rejection

Error: EBUSY: resource busy or locked, unlink 'C:\Users[REDACTED]\AppData\RoamingSignal\sqldb.sqlite'
at Object.unlinkSync (fs.js:930:3)
at Function.rimrafSync [as sync] ([REDACTED]\app.asar\node_modules\rimraf\rimraf.js:306:17)
at removeDB ([REDACTED]\app.asar\app\sql.js:1150:10)
at Object.initialize ([REDACTED]\app.asar\app\sql.js:1123:13)
```

Using Process Explorer, I could not find a process that was locking db.sqlite

[norpol]

This issue is so painful. There are journalists on Twitter reporting that they've lost access to their Signal app & sources due to this bug.

[sonergonul]

Now, this is on Hacker News: https://news.ycombinator.com/item?id=26292299

[justinclift]

@backuitist If Signal is using SQLCipher, as mentioned earlier in this thread, then the header of the encrypted database isn't the standard SQLite header.

For example, this is the header from a SQLCipher database with some generated ~random data for testing purposes:

$ hexdump -C 2.5mb-encrypted-abc123.sqlite | head -1
00000000  ff 3f fc f2 74 1a 61 44  5c c7 44 0e bb 0b a1 42  |.?..t.aD\.D....B|

That database file opens fine with any client that can use SQLCipher version 4.x. eg DB Browser for SQLite

Note, this is the above test database itself (not a Signal one), just in case it's useful for someone to investigate with. Password is "abc123":

• 2.5mb-encrypted-abc123.zip <-- thrown inside a .zip file, so GitHub would accept the upload

[justinclift]

@ryantrinkle

I'm trying to upgrade from v1.37.3 to v1.39.4, and I"m getting this same error (NOTADB). If I manually start v1.37.3, everything still works and all data is still present.

That kind of sounds like the newer version(s) of Signal is using different encryption settings than the older version. Would personally kind of expect the newer version to at least try reading in the "old" database using the "old" settings, and then write out the new database using the "new" settings. Maybe something is going wrong with that process?

@norpol Sounds like the journalists losing their data should revert to the older Signal version until this problem is resolved. Hopefully that's something they can do.

[norpol]

@justinclift Thanks, I'll forward this information. Update: Suddenly just asking to re-link, so the archive is thankfully not gone.

Regarding SQLCipher At least for the my current version (1.39.6) the header of the database is 00000000 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 |SQLite format 3. and using sqlite3-cli it allows me to see actual message contents - just by opening the database. If SQLCipher was in use I wouldn't see that header/be able to extract the database I assume.

[justinclift]

@norpol Yeah, that sounds like the database you're trying there isn't encrypted. At least, not with SQLCipher.

[backuitist]

Thanks @justinclift good to know! I thought there would be some sort of encryption but couldn't find anything in the codebase, and the signal error message was exactly matching that of SQLite.

[gkft]

My signal-desktop (1.40.1) is affected by the same bug, but what looks pretty strange to me the encrypted database opens perfectly fine with sqlcipher and no corruption at all.

sqlcipher --version
3.33.0 2020-08-14 13:23:32 fca8dc8b578f215a969cd899336378966156154710873e68b3d9ac5881b0alt1 (SQLCipher 4.4.2 community)
sqlcipher db.sqlite "PRAGMA key = \"x'$(jq -r '.key' config.json)'\"; attach database 'plaintext.db' as plaintext key ''; SELECT sqlcipher_export('plaintext'); DETACH DATABASE plaintext"

as proposed in https://github.com/signalapp/Signal-Desktop/issues/522#issuecomment-787527821

Maybe the affected version of signal assumes another encryption scheme of the database, a indicator maybe migrateDatabase, which is output from here https://github.com/signalapp/Signal-Desktop/blob/a173a9b33ff8dccbe02b9ad2d5e268ca3b036563/ts/sql/Server.ts#L329

And here the log:

{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"app ready","time":"2021-03-13T16:54:37.550Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"starting version 1.40.1","time":"2021-03-13T16:54:37.550Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"media access status undefined undefined","time":"2021-03-13T16:54:37.551Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2021-03-13T16:54:37.563Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2021-03-13T16:54:37.563Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2021-03-13T16:54:59.511Z","v":0}

Also I tested if re encrypting the database using sqlcipher with no effect.

@scottnonnenberg-signal any updates on this?

[WildPenquin]

I was hit by this problem today, too.

The computer is mostly stable, there has been one (1) hard crash during the last month, but IIRC, Signal was not running at the time (I'm not sure when the "corruption" of the db happened). Either way, a user should not be faced with a situation where only re-linking is possible, and losing all conversation history on the computer. This is also an accessibility problem, since some users might be more dependent on the desktop software (since desktop computers are usually more accessible than smartphones).

This is the terminal output I got when I was hit by this when starting signal-desktop (1.40.1) on Arch Linux:

$ signal-desktop 
Set Windows Application User Model ID (AUMID) { appUserModelId: 'org.whispersystems.signal-desktop' }
NODE_ENV production
NODE_CONFIG_DIR /usr/lib/signal-desktop/resources/app.asar/config
NODE_CONFIG {}
ALLOW_CONFIG_MUTATIONS undefined
HOSTNAME undefined
NODE_APP_INSTANCE undefined
SUPPRESS_NO_CONFIG_WARNING undefined
SIGNAL_ENABLE_HTTP undefined
userData: /home/ville/.config/Signal
config/get: Successfully read user config file
x-attr dependency did not load successfully
config/get: Successfully read ephemeral config file
making app single instance
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"app ready","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"starting version 1.40.1","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"media access status undefined undefined","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2021-03-13T19:57:16.417Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2021-03-13T19:57:16.418Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2021-03-13T19:57:24.718Z","v":0}

Moreover, "copy error and quit" does not actually seem to copy the error - I'm presuming it should copy it to clipboard. If it doesn't - and if that's not what it is supposed to do, the dialog should be more clear since I have no idea where it is copying the error (message?).

I've copied the old signal configuration directory into a safe place in case it might prove useful.

[joukewitteveen]

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

[johackim]

I have the same problem on Arch Linux since I upgrade my system. I have a backup of the ~/.config/Signal directory but even if I restore it, it doesn't work. Any idea ?

EDIT: It work when I downgrade gtk3 :)

[kevenwyld]

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

[richardschuetz]

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

The tracker3 search engine loads SQLite into to process. Signal already loads SQLCipher (specialized build of SQLite with encryption). Both apparently do not play well together. The possibility of this leading to problems was also mentioned by a SQLCipher developer some time ago: https://discuss.zetetic.net/t/is-the-multiple-sqlite-problem-an-issue-for-sqlcipher-for-android/1449/2

There is already a similar problem with OpenSSL discussed in https://github.com/signalapp/Signal-Desktop/issues/2634.

[justinclift]

The tracker3 search engine loads SQLite into to process. Signal already loads SQLCipher ...

Ouch. That's not good.

Sounds like the "tracker3" thing might need changing so it loads the SQLCipher library instead. Then there wouldn't be a conflict.

Maybe screwing with LD_LIBRARY_PATH could make that happen, as a workaround anyway?

[devansh08]

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

On Arch Linux as well, downgrading to gtk3-1:3.24.27-2 worked for me. I was already on gtk3-1:3.24.27-3 when Signal threw this error for me.

[gkft]

I can also confirm, downgrading gtk3 on archlinux resolves the issue.
gtk3 (1:3.24.27-3 -> 1:3.24.26-1)

[Mushoz]

Ran into the same issue with Arch today. Can confirm that downgrading gtk3 fixes the issue.

[fkroener]

Ran into the same issue. Decided I didn't want to delete my data and investigate first whether there was a solution.

Clicked the "X" in the window decoration of the error popup - all my data got purged. Thanks a lot Signal team for your excellent error handling and backup strategy.

[MatejLach]

Can confirm downgrading works but is at best a temporary solution. Does anyone have any links as to why this was enabled now, are there any bug reports on the Arch bug tracker related to this yet?

[real-or-random]

Can confirm downgrading works but is at best a temporary solution. Does anyone have any links as to why this was enabled now, are there any bug reports on the Arch bug tracker related to this yet?

https://bugs.archlinux.org/index.php?do=details&task_id=69980

No need to ask here, this was easy to find...

[gkft]

Maybe the observed behavior is also the reason for https://github.com/signalapp/Signal-Desktop/issues/5097 as now out of a sudden the database is created unencrypted i.e. sqlite instead of the bundled sqlcipher ist used.

[aurelg]

Maybe the observed behavior is also the reason for #5097 as now out of a sudden the database is created unencrypted i.e. sqlite instead of the bundled sqlcipher ist used.

If I understand correctly, a package unrelated to signal-desktop could be able to break it in a way that basically leads to 1/ the loss of existing history, and 2/ history being unencrypted from now on.

If that's the case, I think it is a major security issue: a lack of integrity breaks availability and confidentiality (ref here).

In addition, the benefit of using sqlcipher (with the encryption key in cleartext next to the database) vs sqlite3 should be discussed. Is that documented somewhere?

[gkft]

Well it is just an educated guess from my side at least it could be a possible explanation for https://github.com/signalapp/Signal-Desktop/issues/5097 https://github.com/signalapp/Signal-Desktop/issues/4513.
If the database is encrypted and the key is stored next to the database this doesn't provide additional security, maybe this is for future use like having a local password for signal desktop as the key of the database can be exchanged.

[LukeLR]

I also lost thousands of messages due to this bug. I can confirm that after upgrading to gtk 3.24.27, Signal creates a new plaintext database. After downgrading to 3.24.26 again, Signal creates an encrypted database. I am not sure what to do, as I don't want to stick with the older GTK version, but I also don't want to loose all my messages again when the bug is fixed and Signal expects an encrypted database again. Any ideas?

[gkft]

The dependency of sqlite is caused by tracker3 which requires sqlite. Maybe installing tracker3 only to the local pacman database and not actually to the filesystem fixes this (does not work).
Based on a reddit post

sudo pacman -S tracker3 --dbonly

And adding tracker3 to IgnorePkg list in /etc/pacman.conf as well to avoid updates.
This also may break things.

Until this is fixed I will stick to sudo pacman -Syu --ignore gtk3

[MatejLach]

The dependency of sqlite is caused by tracker3 which requires sqlite. Maybe installing tracker3 only to the local pacman database and not actually to the filesystem fixes this (untested).
Based on a reddit post

sudo pacman -S tracker3 --dbonly

And adding tracker3 to IgnorePkg list in /etc/pacman.conf as well to avoid updates.
This also may break things.

Until this is fixed I will stick to sudo pacman -Syu --ignore gtk3

Already tested. Doesn't work because tracker3 libs are now required at runtime.

[kpcyrd]

Arch maintainer here, we've confirmed the issue and filed a bug on the gtk3 package: https://bugs.archlinux.org/task/69990

[Jaakkonen]

FYI: Pressing Alt+F4 on the error dialog causes DB deletion. Not happy from finding that out...

[seadra]

@Jaakkonen Happened to me as well (I clicked on the close window button) , IMHO Signal Desktop needs a better confirmation UI before performing the drastic action of deleting the database.

[buzo-ffm]

@Jaakkonen Happened to me as well (I clicked on the close window button) , IMHO Signal Desktop needs a better confirmation UI before performing the drastic action of deleting the database.

Even worse, the popup nowhere states that it's coming from Signal Desktop. Today I logged in after an Arch Linux update, several programs started automatically as usual, and then I saw this popup and wondered where it is coming from. Only after I searched the internet for this error message I knew that it's Signal Desktop and that I should not click on the “delete” button. Please fix this UI!

BTW, downgrading gtk3 to 1:3.24.27-2 fixed it for me, too. There is no need to go back to 1:3.24.26-1.

[justinclift]

As a reminder to everyone hitting this... please back up your database, in whatever shape it's in (encrypted or not).

It looks like many of the "lost all my messages" problems can probably be resolved. But that'll only work if you still have your old database file to work with. :smile:

[kpcyrd]

@justinclift the signal protocol is very stateful due to the ratchet design, rolling back to an old database is likely going to cause issues with encrypted chat sessions.

[kpcyrd]

gtk3 1:3.24.27-4 has been released which is expected to resolve this issue. If you've started from scratch while -3 was installed you likely need to relink your device again.

We're still looking for a more permanent solution that works well with gnomes future plans.

[cyorps]

gtk3 1:3.24.27-4 has been released which is expected to resolve this issue.

I can confirm this information. In my case without that I have to relink the device.

[grandchild]

Updating to gtk3 1:3.24.27-4 let me restart Signal with all data intact (luckily I used the button to close the error window).

Note that this is a temporary fix, and the signal-desktop app should still be fixed.

[65a]

I just lost my signal database from this. I'd strongly advocate for a simpler, command line client for Signal, as it's clear the desktop "app" is pretty complicated and unreliable at this point. This would also dramatically reduce the attack surface.

[aurelg]

Updating to gtk3 1:3.24.27-4 showed the same error message as #5097. After clicking on "Delete all data", ~/.config/Signal/sql/db.sqlite is recreated anew, and encrypted. All history is lost again, of course.

[gkft]

Sounds like this time the problem appeared vice versa from unencrypted to encrypted?
If you have a backup of the plain database you may try to encrypt the database using the key inside config.json

[aurelg]

Sounds like this time the problem appeared vice versa from unencrypted to encrypted?

Yes exactly.

If you have a backup of the plain database you may try to encrypt the database using the key inside config.json

I don't have it. It only contained a few messages. Moreover, I backup Signal's history on a regular basis with a script of mine. I haven't lost anything. The only issue is that previous messages are not accessible from the Signal UI, but that's a minor annoyance. :+1:

[gkft]

Did Signal also purge attachment after clicking on the wrong button? If not I tam thinking about some kind of bash alias to first copy the database somewhere and opening signal afterwards

I can confirm signal is working as expected after updating to gtk3-1:3.24.27-4, but this needs to be resolved asap.

[fschwa]

Weird. For me it just happened for the first time on Arch with gtk3-1:3.24.27-4 already up to date.

[thomai]

I got the same error two times after updating Signal on Feb 24th (Signal 1.40, Arch Linux, up-to-date).

[eNTi]

It happend again today. Not sure what changed...

[thomai]

There is an Arch bug report already for this problem: https://bugs.archlinux.org/task/69980?project=5&string=signal-desktop

[bjo81]

I assume you got the error a second time because the first time a new unencrypted database was created.

[thomai]

I assume you got the error a second time because the first time a new unencrypted database was created.

I downgraded to gtk3-1:3.24.27-3 and then it worked w/o data loss. Isn't that a contradiction to your statement?

Do you suggest to update to the newest version of gtk3 and to delete the database again?

[bjo81]

You said you got the error 2 times, so maybe I understood you wrong.
You updated from 3.24.27-2 to 3.24.27-3 and no issue appeared, but 2 times after upgrading to 3.24.27-4? Then my assumption would be wrong, yes.

[thomai]

Yes, the chronological sequence was:

1. Upgrade to 3.24.27-3: Worked
2. Upgrade to 3.24.27-4: Error
3. Deleted database + start Signal + initial Signal setup: Worked (maybe because the application data was set up at that moment)
4. Restart Signal: Error
5. Downgrade to 3.24.27-3: Worked again

[kpcyrd]

If it only works with -3 there's likely a problem with your database:

It's supposed to look like this, which means the database is encrypted:

% file ~/.config/Signal/sql/db.sqlite    
/home/user/.config/Signal/sql/db.sqlite: data

If you're on -3 it likely looks like this, which means your database is NOT encrypted:

% file ~/.config/Signal/sql/db.sqlite    
/home/user/.config/Signal/sql/db.sqlite: SQLite 3.x database, last written using SQLite version 3032002

Keep in mind this note in my comment:

If you've started from scratch while -3 was installed you likely need to relink your device again [with -4].

[thomai]

It worked after deleting the whole .config/Signal/ folder. I don't get why because I tried that before but hey - let's accept that stroke of luck! Unfortunately, I did not find a solution to keep the history.

[kaiyazeera]

It worked after deleting the whole .config/Signal/ folder. I don't get why because I tried that before but hey - let's accept that stroke of luck! Unfortunately, I did not find a solution to keep the history.

That is because when that folder is empty, signal creates a new database (at least I pretty much assume it works this way). If you read the posts above, there is pretty much indicated why this "solved" the problem.

---

Experienced the same error twice this week. I believe it might have been the two updates and therefore the encrypted->unencrypted->encrypted issue. Today, I just updated sqlite and some other applications. (ArchLinux)
Kind of annoying to lose all my history twice for clicking the exit cross instead of any of the two dialog options; so please at least catch the error with a more user-friendly dialog ;)
Anyway, thanks for the work so far.

[GitGangGuy]

Just wanted to chime in, I force shutdown (10 sec power button press) my old laptop running Signal Desktop on Arch GNU/Linux (Filesystem: btrfs, but it should be pretty stable by now and I haven't experienced any corruption) occasionally when it freezes up (some weird kernel issue with Haswell Intel CPUs). After a reboot, I get the infamous prompt. I'm aware that this is not a common case, but it would still be great if Signal could have some kind of protection against forced shutdowns, crashes etc. instead of corrupting the database! ^^

[cburlacu]

I also have this issue (Arch) and downgrading the gtk3 to ..-3 "solved" the issue. Signal is starting again.

[JohnMaguire]

I just ran sudo reboot on Arch after a system-wide update and experienced this issue.

-> % file ~/.config/Signal/sql/db.sqlite
/home/jmaguire/.config/Signal/sql/db.sqlite: data

Edit to add - I was already running 3.24.27-3. I downgraded to 3.24.26-1 which seems to have resolved the issue.