Signal-desktop: Signal 1.15.0 do not start in ArchLinux with Segmentation Fault (core dumped)

[Edu4rdSHL]

• [x] I have searched open and closed issues for duplicates
  

---

Bug description

Signal don't start in ArchLinux after of v1.15.0-beta.9 update, it end with segmentation fault, as i was talking in #2600 it isn't a electron problem, it's a signal problem.

➤➤➤➤ ▶ signal-desktop
NODE_ENV production 
NODE_CONFIG_DIR /usr/lib/signal/resources/app.asar/config 
NODE_CONFIG {} ALLOW_CONFIG_MUTATIONS undefined 
HOSTNAME undefined 
NODE_APP_INSTANCE undefined 
SUPPRESS_NO_CONFIG_WARNING undefined Set Windows Application User Model ID (AUMID) { appUserModelId: 'org.whispersystems.signal-desktop' } userData: /home/sechacklabs/.config/Signal making app single instance {"name":"log","hostname":"SecHackLabs","pid":9268,"level":30,"msg":"app ready","time":"2018-08-03T13:26:14.417Z","v":0} {"name":"log","hostname":"SecHackLabs","pid":9268,"level":30,"msg":"Ensure attachments directory exists","time":"2018-08-03T13:26:14.422Z","v":0} 
/usr/bin/signal-desktop: line 3: 9268 Segmentation fault (core dumped) electron /usr/lib/signal/resources/app.asar $@

Steps to reproduce

1. Upgrade signal to releases after of 1.15.0-beta.8
2. Try to open Signal from terminal emulator (to see error messages)
3. Wait the core dump

Actual result:

Signal crash with Segmentation Fault (core dumped) message

Expected result:

Signal start correctly

Screenshots

N/A

Platform info

Signal version: Tested in 1.15.0-beta.9, 1.15.0-beta.10 and 1.15.0 (latest release)

Operating System: ArchLinux

Linked device version: Signal Android 4.24.7

Link to debug log

Here is the core dumped log of journalctl https://gist.github.com/Edu4rdSHL/d97a4c8d8df3672c6768913dba375a33

[sr-verde]

I have the same problem with version 1.15.1.

[papertigers]

Some more context around the segfault:

[ 746.944534] electron[14485]: segfault at 10 ip 00007f65875c6345 sp 00007f65607253f8 error 6 in libcrypto.so.1.1[7f6587479000+253000]

[breznak]

Confirmed on Ubuntu, 1.15 signal version

 JavaScript error occurred in the main process
Uncaught Exception:
Error: /tmp/.org.chromium.Chromium.lLffkZ: failed to map segment from shared object
    at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:172:20)
    at Object.Module._extensions..node (module.js:671:18)
    at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:186:18)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (module.js:586:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/opt/Signal/resources/app.asar/node_modules/@journeyapps/sqlcipher/lib/sqlite3.js:4:15)
    at Object.<anonymous> (/opt/Signal/resources/app.asar/node_modules/@journeyapps/sqlcipher/lib/sqlite3.js:190:3)

EDIT:
For me, a workaround for the above crash was remounting /tmp as exec: mount -o remount, exec /tmp

Then: distro-package version runs ok, snap still fails.

[Edu4rdSHL]

To signal users: the current Signal core dumped at start is only happening when you compile signal from source (like the signal package that's in the AUR), if you try to install the signal-desktop-bin package that's the binary version compiled for Debian by OWS it works as expected (even in 1.15.0).

[veluria]

@Edu4rdSHL Not true. I'm using signal-desktop-bin and it segfaults for me as well.

[luminoso]

Ok, I'm packaging Signal on Fedora Copr and I also have this core dumped problem. Unsure if this belongs to #2610 or #2600

It also segfaults with the binary debian release on Fedora 28 amd64

[Edu4rdSHL]

Mmm, weird @veluria, i can not reproduce the issue with the -bin version and i haven't idea why.

 ➤➤➤➤ ▶ pacman -Q signal-desktop-bin 
signal-desktop-bin 1.15.0-2

[veluria]

@Edu4rdSHL Are you using testing repos? Because I am.

$ pacman -Q signal-desktop-bin 
signal-desktop-bin 1.15.0-2
$ signal-desktop 
zsh: segmentation fault (core dumped)

[Edu4rdSHL]

@veluria no, i'm not. Also, i tried to reproduce it in a Virtual Machine and i can not reproduce the issue with the -bin version, are you getting exactly the same error message?

[veluria]

@Edu4rdSHL The only thing I'm seeing with the bin version is the segfault I just posted. No further error messages whatsoever.

[lnicola]

@veluria If you get a segfault and only that, it's because of #2600.

[scottnonnenberg-signal]

Note that v1.15.0 introduced an important new binary dependency required for using the app: SQLCipher.

To the person packaging for Arch, it might be that you're compiling on a too-new distribution? I found that I needed to build on an older machine for maximum compatibility.

[ginkel]

@scottnonnenberg-signal AURs are typically compiled from source just before installation, so there is no way to build on an older distribution for Arch Linux.

[AGCaesar]

SQLCipher was missing on my system, I installed it, recompiled the AUR package, still broken :/
signal-desktop-bin on the other hand works perfectly fine!

[Edu4rdSHL]

@scottnonnenberg-signal well, if signal-desktop added sqlcipher (https://www.archlinux.org/packages/community/x86_64/sqlcipher/) as a new dependency, why it is not failing when compiling? Where is signal-desktop checking for the new dependency? Also, if signal is not checking for the SQLCipher dependency we can not wait for the problem to be magically solved by installing sqlcipher (because signal is not searching for that).

[scottnonnenberg-signal]

For security reasons we build OpenSSL and SQLCipher ourselves and statically link them into one final binary. If building it on your own machine isn't working, then the problem is likely our pre-built version of openssl inside of the @journeyapp/node-sqlcipher dependency we forked.

[lnicola]

@Edu4rdSHL I'd totally missed the signal package, as I was looking for signal-desktop. I can now reproduce this issue.

{"name":"log","hostname":"maru.home","pid":27205,"level":30,"msg":"app ready","time":"2018-08-06T17:34:59.222Z","v":0}
{"name":"log","hostname":"maru.home","pid":27205,"level":30,"msg":"Ensure attachments directory exists","time":"2018-08-06T17:34:59.227Z","v":0}
{"name":"log","hostname":"maru.home","pid":27205,"level":30,"msg":"updateSchema: Current schema version: 0; Most recent schema version: 1; SQLite version: 3.20.1; SQLCipher version: 3.4.2;","time":"2018-08-06T17:34:59.231Z","v":0}
{"name":"log","hostname":"maru.home","pid":27205,"level":30,"msg":"updateToSchemaVersion1: starting...","time":"2018-08-06T17:34:59.231Z","v":0}

Thread 33 "electron" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffbb648700 (LWP 27243)]
0x00007fffe2678345 in EVP_MD_CTX_clear_flags () from /usr/lib/libcrypto.so.1.1
(gdb) bt
#0  0x00007fffe2678345 in EVP_MD_CTX_clear_flags () at /usr/lib/libcrypto.so.1.1
#1  0x00007fffe2668b4e in EVP_DigestInit_ex () at /usr/lib/libcrypto.so.1.1
#2  0x00007fffe267f790 in HMAC_Init_ex () at /usr/lib/libcrypto.so.1.1
#3  0x00007fffd85ef141 in sqlcipher_openssl_hmac () at /tmp/.org.chromium.Chromium.fHrcNR
#4  0x00007fffd85ee7d7 in sqlcipher_page_hmac () at /tmp/.org.chromium.Chromium.fHrcNR
#5  0x00007fffd86040a8 in sqlcipher_page_cipher () at /tmp/.org.chromium.Chromium.fHrcNR
#6  0x00007fffd86188a1 in sqlite3Codec () at /tmp/.org.chromium.Chromium.fHrcNR
#7  0x00007fffd86344e1 in pager_write_pagelist () at /tmp/.org.chromium.Chromium.fHrcNR
#8  0x00007fffd863cd20 in sqlite3PagerCommitPhaseOne () at /tmp/.org.chromium.Chromium.fHrcNR
#9  0x00007fffd863cf22 in sqlite3BtreeCommitPhaseOne.part.605 () at /tmp/.org.chromium.Chromium.fHrcNR
#10 0x00007fffd86556d7 in sqlite3VdbeHalt () at /tmp/.org.chromium.Chromium.fHrcNR
#11 0x00007fffd866b3d2 in sqlite3VdbeExec () at /tmp/.org.chromium.Chromium.fHrcNR
#12 0x00007fffd867178f in sqlite3_step () at /tmp/.org.chromium.Chromium.fHrcNR
#13 0x00007fffd85dc9ba in node_sqlite3::Statement::Work_Run(uv_work_s*) () at /tmp/.org.chromium.Chromium.fHrcNR
#14 0x00007ffff7ee39d0 in  () at /usr/lib/electron/libnode.so
#15 0x00007ffff6befa8d in start_thread () at /usr/lib/libpthread.so.0
#16 0x00007fffecf52823 in clone () at /usr/lib/libc.so.6

For security reasons we build OpenSSL and SQLCipher ourselves and statically link them into one final binary.

Is that.. a good idea? I'm not sure what it's trying to mitigate.

[meliber]

I just downgrade signal to 1.14.4

[luminoso]

For security reasons we build OpenSSL and SQLCipher ourselves and statically link them into one final binary. If building it on your own machine isn't working, then the problem is likely our pre-built version of openssl inside of the @journeyapp/node-sqlcipher dependency we forked.

But does these build steps...

yarn install
yarn generate --force
yarn build-release --dir

..take that into consideration? using the forked sqlcipher.

Also: why build against a binary OpenSSL? If I can trust my system's OpenSSL binary for everything else I can also trust it for the Signal Desktop app. It is more strange using a prebuilt binary than not using it! Fedora's packages (including OpenSSL) chain is fully verified and transparent. Requiring a binary when doing a fresh build is not.

[Jakeler]

I did a few tests:
1.15.0 Arch electron 2.0.6 + glibc 2.27= segfault
1.15.0 builtin electron 2.0.2 + glibc 2.27 = WORKS
1.15.0 Arch electron 2.0.6 + glibc 2.28 = segfault
1.15.0 builtin electron 2.0.2 + glibc 2.28 = segfault

(1.15.2 is exactly the same, so i put the pkgbuild updates on hold for now, because it won't run anyway)

Older versions:
1.14.4 Arch electron 2.0.6 + glibc 2.28 = WORKS
1.14.4 builtin electron 2.0.1 + glibc 2.28 = segfault
1.13.0 builtin electron + glibc 2.28 = segfault

I think we are having multiple issues here, that happened at the same time.
First obviously the glibc 2.28 update on Arch Linux, which causes all prebuilt electron binaries to crash. The signal AUR package uses normally the system's electron (so no problem with glibc), but that also doesn't work anymore since 1.15.0, might be because of the slight version mismatch or other included binaries (openssl) and incompatibility from sqlcipher.
The segfault backtrace is the same as @lnicola posted above.

I also agree with @luminoso that it would be better to use the system openssl distribution on Linux, if possible. In addition i have seen that the release build app.asar contains still OpenSSL source code, Windows and macOS binaries, which seems really uneccessary.

[j0ni]

Not a fix, but a workaround: I was able to install a working Signal 1.15.x (and Slack) via flatpak (from flathub).

[papertigers]

I also went the flatpak route since I am already using it for a few other apps.

If anyone else decides to do this you can cp -R ~/.config/Signal ~/.var/app/org.signal.Signal/config to move over your config/messages to where the flatpak application will look for them.

[luminoso]

@j0ni @papertigers well, that's nice to have a workaround, but like Signal being Electron isn't bad enough (#2178) I can't imagine running it via flatpak where even more libs and bloat comes with it. I'm hoping for a fix instead (/offtopic)

[njfox]

I'm hoping for a fix instead (/offtopic)

I'm hoping for an explanation from the maintainers as to why precompiled binaries are necessary to build a private messenger app from source.

[zatrazz]

On Electron [1] project, it seems the issue is related to linker where lld seems to generate wrong relocation. Could be this same issue here?

[1] https://github.com/electron/electron/issues/13972#issuecomment-411416730

[je-vv]

The AUR "signal" package got worked around with setting @journeyapps/sqlcipher dependency manually, BTW. Look for @WaltzOfWoe AUR user comment, referring to https://github.com/signalapp/Signal-Desktop/issues/2634

[Edu4rdSHL]

@je-vv yes, the signal package is working now, but i'm still not closing the issue because it's really a _hack_ using the PKGBUILD (see it commit) and not a real solution by the OWS developers.

[Jakeler]

@Edu4rdSHL I don't really see it as a hack. It switches to the original sqlcipher release with dynamic linking (using the openssl-1.0 package from core) instead of the fork, so the effective difference are these commits that added the linux openssl binaries and changed to static linking.

Most other Linux distributions as far as i know don't provide an easy and secure way to install openssl 1.0.2 alongside. So it might make still sense to supply the binaries there.

On Arch it is usual practice to do some patching to use more system dependencies.
Sure, it would be nice to link against openssl 1.1.0 (also because we already have it in the electron dependency tree multiple times) but that is another issue and would probably break it again on "LTS" distributions...