Termux-packages: erlang crashes after running make

Created on 22 Sep 2018 · 11Comments · Source: termux/termux-packages

Erlang runs fine under a fresh install of Termux (Andoid 8.1 on aarch64), but after running make, it crashes whilst starting up.

For those not familar with Erlang, fortunately all you need to do to replicate this is run erl, quit from its REPL by typing q(). and wait the short time for it to shutdown.

Here is a typical session showing the error (take note of that erl can be run without problems multiple times before running make):

$ pkg install erlang make

$ erl
Erlang/OTP 21 [erts-10.0.7] [source] [64-bit] [smp:8:6] [ds:8:6:10] [async-threads:1]

Eshell V10.0.7  (abort with ^G)
1> q().
ok
2> $

$ erl
Erlang/OTP 21 [erts-10.0.7] [source] [64-bit] [smp:8:6] [ds:8:6:10] [async-threads:1]

Eshell V10.0.7  (abort with ^G)
1> q().
ok
2> $

$ make
make: *** No targets specified and no makefile found.  Stop.

$ erl 
erl_child_setup closed

Crash dump is being written to: erl_crash.dump...done

$ erl 
erl_child_setup closed

Crash dump is being written to: erl_crash.dump...done

I recently started to use Termux, so no idea if this particular problem is a recent regression, or has always been there. On two other different phones (one aarch64 and the other Android 7 armv71) I replicated this from a clean install of Termux, and they both see the same problem. This occurs regardless of whether there is a valid Makefile in the current working directory or not.

I did find, by accident that piping STDERR to /dev/null stopped the crashes:

$ erl 2>/dev/null

This prompted me to look at logcat and I found:

09-22 18:24:11.295 26116 26116 W erl_child_setup: type=1400 audit(0.0:1865158): avc: denied { append } for path="/dev/pts/0" dev="devpts" ino=3 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:untrusted_app_devpts:s0:c512,c768 tclass=chr_file permissive=0

The SELinux denys only start to occur after running make.

I have looked to see if make somehow left any environment variables behind, but no change there. I also tried this after unset'ing LD_PRELOAD to stop termux-exec from loading in. Again no change. Nothing stands out as abnormal when running erl or make under strace either.

I am nearly out of ideas on how to push forward on resolving this, so turning to the project owners for some hints as maybe you have see this elsewhere?

Thanks for an otherwise amazing contribution to the Android ecosystem though, top work!

bug report

Source

jimdigriz

All 11 comments

Okay, so it seems SELinux is upset by the append, and it is make that reopens std{in,out,err} fds with the addition of O_APPEND:

$ cat /proc/$$/fdinfo/0
pos:    376
flags:  0400002
mnt_id: 2791
$ cat /proc/$$/fdinfo/1
pos:    413
flags:  0400002
mnt_id: 2791
$ cat /proc/$$/fdinfo/2
pos:    450
flags:  0400002
mnt_id: 2791

$ make
make: *** No targets specified and no makefile found.  Stop.

$ cat /proc/$$/fdinfo/0
pos:    487
flags:  0402002
mnt_id: 2791
$ cat /proc/$$/fdinfo/1
pos:    487
flags:  0402002
mnt_id: 2791
$ cat /proc/$$/fdinfo/2
pos:    487
flags:  0402002
mnt_id: 2791

The flags in fdinfo are described in fcntl.h.

The reason we see this in Erlang and nowhere else is due to its use of SCM_RIGHTS; SELinux refuses to let the fd get passed to the child process and truncates the message:

working comms:

29290 dup(2)                            = 16
----8<----
29290 sendmsg(11, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\16\0\0\0\r\0\0\0\20\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[14, 13, 16]}], msg_controllen=32, msg_flags=0}, 0) = 32
29285 recvmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\16\0\0\0\r\0\0\0\20\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[6, 7, 8]}], msg_controllen=32, msg_flags=0}, MSG_DONTWAIT) = 32

SELinux hates us comms:

29001 dup(2)                            = 16                                                                                                                                                                                                                   
----8<----
29001 sendmsg(11, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\16\0\0\0\r\0\0\0\20\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[14, 13, 16]}], msg_controllen=32, msg_flags=0}, 0) = 32
28998 recvmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\16\0\0\0\r\0\0\0\20\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=24, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[6, 7]}], msg_controllen=24, msg_flags=MSG_CTRUNC}, MSG_DONTWAIT) = 32

Those with a keen eye will see MSG_CTRUNC and that one of the fd's (the dup`d stderr) did not make it over.

This is not a Erlang problem. I am thinking we need something to keep those file descriptors clear of O_APPEND or is this just a problem with make? Thoughts?

jimdigriz on 22 Sep 2018

This patch prevents make reopening std{out,err} with O_APPEND:

https://github.com/jimdigriz/termux-packages/commit/4aedbf2f1964a3110fd6aa7910ecff305d5c4c3b

Erlang seems to stop complaining now.

jimdigriz on 22 Sep 2018

👍1

Nice.
this is something im going to have to look out for and might explain a few issues im having else where. will be looking into how do deal with this generally as well.
I also like how you were out of ideas then a few hours later solved it.

its-pointless on 22 Sep 2018

You may be able to plaster over most of the problems by having termux-exec fix up std{out,err} as its there at execve() anyway which may spare you having to play whack-a-mole.

Anyway, great work on Termux, to everyone involved.

jimdigriz on 22 Sep 2018

Oh, in a day or two, I'll submit a PR request once I am happy with it.

jimdigriz on 22 Sep 2018

👍1

You might want to do: if(!isatty(1))set_append_mode(1); etc, since I can see the O_APPEND behaviour being useful for recording make's output to files.

geocar on 23 Sep 2018

You might want to do: if(!isatty(1))set_append_mode(1); etc, since I can see the O_APPEND behaviour being useful for recording make's output to files.

Good idea, https://github.com/jimdigriz/termux-packages/commit/56758caf6f87e660bf88d9ec6af600ff258353d3

jimdigriz on 23 Sep 2018

👍1

Nice work @jimdigriz, looking forward to your PR!