Crystal: Crystal and libevent 2.1.11 problem

Created on 5 Aug 2019 · 10Comments · Source: crystal-lang/crystal

With latest crystal 0.30, when I building with --release --static, everything is OK (but very slow), but when I want to just run file with crystal src/something.cr or crystal spec, this errors occurs:

...
Unhandled exception in spawn: Error reinitializing libevent[warn] event_reinit: forked from the event_loop.
 (Exception)
[warn] event_reinit: forked from the event_loop.
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Unhandled exception in spawn:  (Unhandled exception in spawn: Error reinitializing libeventExceptionError reinitializing libevent ()
 (ExceptionException)
)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
Unhandled exception in spawn: Error reinitializing libevent (Exception)
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: [warn] event_reinit: forked from the event_loop.
Error reinitializing libeventUnhandled exception in spawn: [warn] event_reinit: forked from the event_loop.
 (Error reinitializing libeventUnhandled exception in spawn: Exception (Error reinitializing libevent)
Exception ()
Exception[warn] event_reinit: forked from the event_loop.
[warn] event_reinit: forked from the event_loop.
Error reinitializing libeventUnhandled exception in spawn:  (Error reinitializing libeventException ()
Exception[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
...

and there is also tons of dead processes in process list in state piperd:

...
 2382 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2416 pf            1  52    0   619M   494M piperd  0   0:00   0.00% crystal spec
 2443 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2140 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2113 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2436 pf            1  52    0   619M   494M piperd  0   0:00   0.00% crystal spec
 2368 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2384 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2430 pf            1  52    0   619M   494M piperd  3   0:00   0.00% crystal spec
 2401 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2347 pf            1  52    0   619M   494M piperd  3   0:00   0.00% crystal spec
 2357 pf            1  52    0   619M   494M piperd  3   0:00   0.00% crystal spec
 2440 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2377 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2375 pf            1  52    0   619M   494M piperd  0   0:00   0.00% crystal spec
 2371 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2439 pf            1  52    0   619M   494M piperd  1   0:00   0.00% crystal spec
 2376 pf            1  52    0   619M   494M piperd  3   0:00   0.00% crystal spec
 2316 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
 2373 pf            1  52    0   619M   494M piperd  0   0:00   0.00% crystal spec
 2409 pf            1  52    0   619M   494M piperd  0   0:00   0.00% crystal spec
 2298 pf            1  52    0   619M   494M piperd  2   0:00   0.00% crystal spec
...

Versions:
Crystal:

Crystal 0.30.0 (2019-08-05)
LLVM: 6.0.1
Default target: x86_64-portbld-freebsd11.2

libevent:

libevent-2.1.11 (libevent-2.1.so.7)

FreeBSD:

11.2-RELEASE-p11

ldd output:

➤  ldd /usr/local/bin/crystal
/usr/local/bin/crystal:
    libLLVM-6.0.so => /usr/local/llvm60/lib/libLLVM-6.0.so (0x801400000)
    libc++.so.1 => /usr/lib/libc++.so.1 (0x804c59000)
    libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x804f27000)
    libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x805146000)
    libm.so.5 => /lib/libm.so.5 (0x8053e8000)
    libgc-threaded.so.1 => /usr/local/lib/libgc-threaded.so.1 (0x805615000)
    libthr.so.3 => /lib/libthr.so.3 (0x80588b000)
    libevent-2.1.so.7 => /usr/local/lib/libevent-2.1.so.7 (0x805ab3000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x805d04000)
    libc.so.7 => /lib/libc.so.7 (0x805f13000)
    libedit.so.0 => /usr/local/lib/libedit.so.0 (0x8062cf000)
    libz.so.6 => /lib/libz.so.6 (0x806506000)
    librt.so.1 => /usr/lib/librt.so.1 (0x80671e000)
    libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x806924000)
    libncurses.so.8 => /lib/libncurses.so.8 (0x806b27000)
    libdl.so.1 => /usr/lib/libdl.so.1 (0x806d7c000)
    libelf.so.2 => /lib/libelf.so.2 (0x806f7d000)

Source

petr-fischer

👀1

Most helpful comment

Workaround for now: compile with --threads=1 (the problem is related to fork)

asterite on 7 Aug 2019

👍2

All 10 comments

Note: This problem occurs even with official FreeBSD crystal port 0.28 - something weird in libevent?

Also posted here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239655

petr-fischer on 5 Aug 2019

What happens if you don't use --static?

asterite on 5 Aug 2019

@asterite

1) crystal build src/something.cr
Lots of libevent errors, lots of dead piperd state crystal processes.

2) crystal build src/something.cr --release
Everything OK.

For both crystal testing versions, 0.28 and 0.30.

petr-fischer on 5 Aug 2019

Next investigation - it looks like there is problem in transition from libevent <= 2.1.10 to the libevent-2.1.11.

I tested building crystal also on another box with FreeBSD 12 and:

with libevent-2.1.10, everything is OK
with libevent-2.1.11, lot of error messages from libevent, crystal does not work

(binaries of crystal ale builded on a box with libevent-2.1.11)

And as a test script, you need something that utilize fibers/IO (?), like:

require "http/client"

HTTP::Client.get "http://api.icndb.com/jokes/1"

petr-fischer on 6 Aug 2019

Also - there is some ABI changes in libevent - https://raw.githubusercontent.com/libevent/libevent/release-2.1.11-stable/ChangeLog

petr-fischer on 6 Aug 2019

👀1

It's not just FreeBSD, I get exactly the same behaviour in Gentoo Linux.

After building crystal-0.30.0 against libevent-2.1.11 "crystal src/something.cr" no longer works and instead spews a few hundred lines of

[warn] event_reinit: forked from the event_loop.
Unhandled exception in spawn: Error reinitializing libevent (Exception)
Unhandled exception in spawn: Error reinitializing libevent (Unhandled exception in spawn: Error reinitializing libevent (Exception)
Unhandled exception in spawn: Channel is closed (Channel::ClosedError)
Channel is closed (Channel::ClosedError)

in various permutations.

No Gentoo bug opened about this as of yet.