Invidious: I using this command : ./invidious recive this error why ?

[elekber123]

I using this command : ./invidious
Receive this error why i receive this ???
[invidious7@ns395754 invidious]$ ./invidious
Invalid memory access (signal 11) at address 0x0
[0x977146] ???
[0x45597f] __crystal_sigfault_handler +479
[0x7fbbc0bfb5f0] ???
[0x7fbbc1491e70] ???
[0x7fbbc14d533f] X509_STORE_add_lookup +95
[0x7fbbc14cd9a4] X509_STORE_set_default_paths +20
[0x4d7d8f] ???
[0x55da9a] ???
[0x5bca08] ???
[0x4511ed] ???
[0x0] ???

[Perflyst]

Maybe duplicate of #774

[omarroth]

What command are you using to build Invidious?

[Infinisil]

I have this exact same error, but only when I compile with --release (https://github.com/crystal-lang/crystal/issues/4391 may be related), not using that flag gives a slightly different error, which seems to change depending on the openssl version:

• openssl-1.0.2t:

Invalid memory access (signal 11) at address 0x0
[0xe01f36] *CallStack::print_backtrace:Int32 +118
[0x615e59] __crystal_sigfault_handler +217
[0x7f77191a4860] ???
[0x7f77195554b0] ???
[0x7f77195a3572] X509_STORE_add_lookup +98
[0x7f771959bb14] X509_STORE_set_default_paths +20
[0x1390206] ???
[0x108c42a] *OpenSSL::SSL::Context::Client +10
[0x108c37a] *OpenSSL::SSL::Context::Client +74
[0x108c312] *OpenSSL::SSL::Context::Client#initialize:Nil +34
[0x108c2e1] *OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client +65
[0x114c9a1] *QUIC::Client#initialize<String, (Int32 | Nil), Bool>:Bool +113
[0x114c91f] *QUIC::Client::new<String, (Int32 | Nil), Bool>:QUIC::Client +223
[0x114c566] *QUIC::Client::new<URI, Nil>:QUIC::Client +102
[0x114c4f3] *QUIC::Client::new<URI>:QUIC::Client +19
[0x6e4def] *fetch_decrypt_function<String>:Array(NamedTuple(name: String, value: Int32)) +175
[0x6e4d35] *fetch_decrypt_function:Array(NamedTuple(name: String, value: Int32)) +21
[0x6e4c09] ~procProc(Nil) +41
[0xe998c3] *Fiber#run:(IO::FileDescriptor | Nil) +115
[0x615306] ~proc2Proc(Fiber, (IO::FileDescriptor | Nil)) +6
[0x0] ???

• openssl-1.1.1d

Invalid memory access (signal 11) at address 0x18
[0xe01e36] *CallStack::print_backtrace:Int32 +118
[0x615d59] __crystal_sigfault_handler +217
[0x7feac7dba860] ???
[0x7feac7db4c40] ???
[0x7feac823b0a9] CRYPTO_THREAD_write_lock +9
[0x7feac824c0a6] X509_STORE_add_cert +70
[0x7feac8247d7e] X509_load_cert_crl_file +142
[0x7feac8247ec9] ???
[0x7feac824acd0] X509_STORE_set_default_paths +48
[0x1390106] ???
[0x108c32a] *OpenSSL::SSL::Context::Client +10
[0x108c27a] *OpenSSL::SSL::Context::Client +74
[0x108c212] *OpenSSL::SSL::Context::Client#initialize:Nil +34
[0x108c1e1] *OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client +65
[0x114c8a1] *QUIC::Client#initialize<String, (Int32 | Nil), Bool>:Bool +113
[0x114c81f] *QUIC::Client::new<String, (Int32 | Nil), Bool>:QUIC::Client +223
[0x114c466] *QUIC::Client::new<URI, Nil>:QUIC::Client +102
[0x114c3f3] *QUIC::Client::new<URI>:QUIC::Client +19
[0x6e4cef] *fetch_decrypt_function<String>:Array(NamedTuple(name: String, value: Int32)) +175
[0x6e4c35] *fetch_decrypt_function:Array(NamedTuple(name: String, value: Int32)) +21
[0x6e4b09] ~procProc(Nil) +41
[0xe997c3] *Fiber#run:(IO::FileDescriptor | Nil) +115
[0x615206] ~proc2Proc(Fiber, (IO::FileDescriptor | Nil)) +6
[0x0] ???

I'm using Nix to compile invidious, see https://github.com/NixOS/nixpkgs/pull/67664. So this failure should be very reproducible. From that PR it's possible to do

$ nix-build -A invidious
$ cd some-dir
$ result/bin/invidious

Where some-dir contains config/config.yml. Also make sure that it can connect to the db.

I'll try to bisect this issue, because I know this didn't happen with an earlier set of dependencies and invidious 19.1

[Infinisil]

Yeah no problems with invidious 0.19.1, crystal 0.30.1 and openssl 1.0.2t. I'll further try to bisect into crystal

[Infinisil]

After bisecting I can confirm that invidious commit d46b26e3bc87fbbdfa9550820410041b861bb393 is the cause of the error I mentioned above

[Infinisil]

And looking at the code of lsquic.cr, this makes sense, because in at least Nix (and maybe other package managers), just using random statically linked libraries won't work: https://github.com/omarroth/lsquic.cr/tree/master/src/lsquic/ext

Why are these in there? How are they built?

[Infinisil]

Seems like those are from boringssl and lsquic, I should be able to patch those with Nix-built ones

[omarroth]

lsquic and boringssl do not have pre-built packages available for most distributions, so linking them statically is likely the best solution for now.

If you want to build them yourself I would recommend taking a look at lsquic-alpine and boringssl-alpine for reference.

[Infinisil]

Nix already builds everything itself, and boringssl was already packaged, but I needed to do lsquic myself. See latest commits in https://github.com/NixOS/nixpkgs/pull/67664. Unfortunately even after patching out those libraries with Nix-built ones the same error occurs (with latest openssl):

Invalid memory access (signal 11) at address 0x18
[0xe00e66] *CallStack::print_backtrace:Int32 +118
[0x614d89] __crystal_sigfault_handler +217
[0x7f520d44b860] ???
[0x7f520d445c40] ???
[0x7f520d8cc0a9] CRYPTO_THREAD_write_lock +9
[0x7f520d8dd0a6] X509_STORE_add_cert +70
[0x7f520d8d8d7e] X509_load_cert_crl_file +142
[0x7f520d8d8ec9] ???
[0x7f520d8dbcd0] X509_STORE_set_default_paths +48
[0x108b35a] *OpenSSL::SSL::Context::Client +10
[0x108b2aa] *OpenSSL::SSL::Context::Client +74
[0x108b242] *OpenSSL::SSL::Context::Client#initialize:Nil +34
[0x108b211] *OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client +65
[0x114b8d1] *QUIC::Client#initialize<String, (Int32 | Nil), Bool>:Bool +113
[0x114b84f] *QUIC::Client::new<String, (Int32 | Nil), Bool>:QUIC::Client +223
[0x114b496] *QUIC::Client::new<URI, Nil>:QUIC::Client +102
[0x114b423] *QUIC::Client::new<URI>:QUIC::Client +19
[0x6e3d1f] *fetch_decrypt_function<String>:Array(NamedTuple(name: String, value: Int32)) +175
[0x6e3c65] *fetch_decrypt_function:Array(NamedTuple(name: String, value: Int32)) +21
[0x6e3b39] ~procProc(Nil) +41
[0xe987f3] *Fiber#run:(IO::FileDescriptor | Nil) +115
[0x614236] ~proc2Proc(Fiber, (IO::FileDescriptor | Nil)) +6
[0x0] ???

So it seems that this wasn't it after all :(

[tuxor1337]

I'm running into the same problem. Does this issue mean that you can't succesfully build the current master?

[Perflyst]

@tuxor1337 invidious can be built on latest master
which OS and crystal version are you using?

[tuxor1337]

CentOS 7 with crystal installed from https://github.com/crystal-lang/crystal/releases/download/0.32.1/crystal-0.32.1-1-linux-x86_64.tar.gz

$ crystal --version
Crystal 0.32.1 [41bd18fbe] (2019-12-18)

LLVM: 8.0.0
Default target: x86_64-unknown-linux-gnu

To be clear: The build process (crystal build src/invidious.cr --release) runs fine. But invoking ./invidious yields the error message quoted by @Infinisil above. And this is regardless from the version of boringssl or lsquic I'm using in lsquic.cr.

[Perflyst]

Please run shards install && shards update && crystal build src/invidious.cr
--release is broken

[tuxor1337]

I tried it without --release and it still produces a dysfunctional binary that crashes with the same error message quoted above.

Edit: As noted by @Infinisil, the version of invidious prior to d46b26e works with my setup.

[tuxor1337]

Actually, I found that my error message is slightly different:

Invalid memory access (signal 11) at address 0x0
[0xd807e6] *CallStack::print_backtrace:Int32 +118
[0x609e29] __crystal_sigfault_handler +217
[0x7fc79f8375f0] ???
[0x7fc79fdb61e9] ???
[0x1317467] ???
[0x1036fa1] *OpenSSL::SSL::Context::Client +17
[0x1036f62] *OpenSSL::SSL::Context::Client#initialize:Nil +34
[0x1036f29] *OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client +73
[0x105ee18] *QUIC::Client#initialize<String, (Int32 | Nil), Bool>:Bool +104
[0x105ed9d] *QUIC::Client::new<String, (Int32 | Nil), Bool>:QUIC::Client +221
[0x105ea6b] *QUIC::Client::new<URI, Nil>:QUIC::Client +107
[0x105e9f3] *QUIC::Client::new<URI>:QUIC::Client +19
[0x665723] ~procProc(QUIC::Client) +35
[0x1066ef8] *ConnectionPool(QUIC::Client) +104
[0x1066dea] *ConnectionPool(QUIC::Client) +58
[0x6c84db] *fetch_decrypt_function<String>:Array(Tuple(Proc(Array(String), Int32, Array(String)), Int32)) +123
[0x6c8455] *fetch_decrypt_function:Array(Tuple(Proc(Array(String), Int32, Array(String)), Int32)) +21
[0x6c8329] ~procProc(Nil) +41
[0xe5460d] *Fiber#run:(IO::FileDescriptor | Nil) +109
[0x609326] ~proc2Proc(Fiber, (IO::FileDescriptor | Nil)) +6
[0x0] ???

[tuxor1337]

Apparently, I used a bad combination of environment variables during build so that the linker got confused, but didn't complain (there were two different versions of some libraries, such as openssl, in my paths). Now that I'm making sure that both $PKG_CONFIG_PATH and $LD_LIBRARY_PATH point to the right locations, invidious compiles and runs fine with my custom builds of boringssl and lsquic.

Sorry for the confusion!

Background: My goal was to set up Invidious in a non-root environment as a user on a system where non-compatible versions of libraries might be pre-installed in a global path. So I used Anaconda to create a virtual environment and install all required libraries to the home directory. However, with such a setup, you have to be extremely careful in order to make sure that only the headers/libraries in the conda environment are used by compilers and linkers.

[Infinisil]

Oh interesting, in my case I thought I made sure crystal and invidious used the same versions of libraries. Thanks for that clue, I think I should be able to figure it out for Nix with this.

[jpotier]

@Infinisil I venture this is still broken for us on nix? Did you get the same error?

[Infinisil]

One pseudo-unfortunate effect of Nix's reproducibility is that issues will persist if nothing is changed. And I didn't change anything in my invidious PR (https://github.com/NixOS/nixpkgs/pull/67664), meaning it's still broken :)

The problem is a combination of how nixpkgs builds crystal packages and the fact that invidious needs to link different parts to different libraries with the same name (BoringSSL and OpenSSL). I think nixpkgs crystal builder needs to be adjusted to support that somehow, copying how other distros build it.