Nixpkgs: Something between 125ffff..d458029 broke xterm

[matthiasbeyer]

Issue description

I submitted a bug report in https://github.com/NixOS/nixpkgs/issues/17158, then in https://github.com/neovim/neovim/issues/5101 because I thought it would be neovim which crashes, but it wasn't. It was xterm.

I start xterm through i3 with (from my ~/.i3/config):

bindsym Mod4+Return exec xterm -bg grey18 -fg grey90

When I then open a file with nvim, xterm crashes (I do not believe anymore that nvim crashes, because the xterm gets killed as well and I cannot think of any reason why a crashing nvim would kill a xterm). This _also_ happens when setting :set ft=mail in nvim, for some reasons. It might also happen with other filetypes. Here goes my ftplugin for mail, for completeness:

" mails
" -----
setlocal textwidth=70
setlocal wrap

setlocal fo=aw

set colorcolumn=71

I nixos-rebuild switch --rollbacked to my generation 121, which was at unstable channel with commit 125ffff - I was on generation 122 with commit d458029 before. Everything worked again (that's why nixos is awesome, guys! :tada: ).

I diffed the nixpkgs tree between this commits. The only change to the definition of the xterm package itself is a98ae8e1526055c347409540d3d5c9dd104b81cb. Can someone test xterm with reverted a98ae8e1526055c347409540d3d5c9dd104b81cb - to check whether this is the issue? (I'm in the hot phase of my bachelors thesis and cannot do this right now).

If I use git diff 125ffff..d458029 there are a _lot_ of changes which might affect xterm as well... so I'm not sure whether a98ae8e1526055c347409540d3d5c9dd104b81cb is the issue here.

Steps to reproduce

1. Be on nixos-unstable at commit d458029
2. Open xterm via i3 with bindsym Mod4+Return exec xterm -bg grey18 -fg grey90 (It doesn't crash when you open an xterm via a commandline call in your already opened xterm, even if you pass the same parameters :curly_loop: )
3. Open a file with nvim (aliased to vim for me): nvim some_existing_file.
4. If the above doesn't work, try to set ft=mail in nvim.
   

   Technical details

• System: 16.09pre86979.d458029 (Flounder)
• Nix version: nix-env (Nix) 1.11.2
• Nixpkgs version: "16.09pre86979.d458029"

[matthiasbeyer]

I just reproduced this issue on

• System: 16.09pre87355.a18abd2 (Flounder)
• Nix version: nix-env (Nix) 1.11.2
• Nixpkgs version: "16.09pre87355.a18abd2

With an xterm started on 125ffff (I started an xterm, then upgraded, than started nvim on a file in /tmp/ - crash).

I'm not sure anymore that this is xterm. Reproducing with an old xterm but new nvim (I guess nvim is 0.1.4 anyways and there wasn't an update of nvim in this commit range) makes me think about whether this is actually xterm or nvim.

[matthiasbeyer]

I just tried normal vim.

vim doesn't crash.

I'm going to switch my configuration to use vim instead of nvim, and I'll reopen the issue in the neovim project.

[matthiasbeyer]

Reverting https://github.com/NixOS/nixpkgs/pull/16604/commits/a98ae8e1526055c347409540d3d5c9dd104b81cb and rebuilding xterm without the option added in this commit _seems_ to solve my issue. I'm not completely convinced, though.

[matthiasbeyer]

I hate this... I just experienced this issue with plain vim and xterm. Couldn't reproduce after applying #17234, though... so I guess this is actually the problem.

[vrthra]

@matthiasbeyer Having a little trouble reproducing this issue, I have started xterm from i3, and used both vim and nvim top open an non-existant file. Does the crash only occur indeterminately?

[matthiasbeyer]

Nope. Try to :set ft=mail.

Maybe the issue gets only triggered with some special vimrc settings... I'm not sure.

[vrthra]

Just tried with your ftplugin settings and :set ft=mail, could you try with strace vim /tmp/nonexistant and post the logs on crash?

[vrthra]

Also, try without a vimrc, or a dummy vimrc

[matthiasbeyer]

I did strace nvim /tmp/mutt-... 2> /tmp/strace.out. I got a 2mb strace output file, ending with these lines:

write(22, "\1\0\0\0\0\0\0\0", 8)        = 8
futex(0x7fb73ddfee40, FUTEX_WAKE_PRIVATE, 1) = 1
write(22, "\1\0\0\0\0\0\0\0", 8)        = 8
epoll_wait(10, [], 1024, 0)             = 0
epoll_wait(10, [], 1024, 0)             = 0
epoll_wait(10, strace: Process 22781 detached
 <detached ...>
Vim: Caught deadly signal 'SIGHUP'

Vim: Finished.

With no vimrc:

$ strace nvim -u NORC /tmp/mutt-... 2> /tmp/norc-strace.out

everything works.

[vrthra]

Any chance you can bisect your vimrc and see which line is the culprit? Also, could you paste the tail -n 1000 /tmp/strace.out in the paste bin, and post a link here.

[vrthra]

If bisect is too time consuming, you could put the vimrc that causes the bug in a paste bin, and I can try using that to see if it makes my vim dump.

[matthiasbeyer]

bisecting the vimrc is a bit complicated I guess... As I build it into my system (it is part of my configuration.nix).

Paste: http://pastebin.com/0ibwK2k5
strace: http://pastebin.com/nwPeLpNC

[vrthra]

On the strace, could you please paste $ cat /tmp/strace.out | grep -v FUTEX_WAKE_PRIVATE | grep -v 'write\(22, .*, 8) = 8? This seems to be repeating, and drowns out the actual lines.

[matthiasbeyer]

Here we go: http://pastebin.com/b7G3VfG0

(from: cat /tmp/strace.out | grep -v FUTEX_WAKE_PRIVATE | grep -v "'write\(22, .*, 8\) = 8" | tail -n 1000)

[vrthra]

Ouch, that doesn't seem to be of much help :(, could you add |uniq -c | tail -n 1000?

[matthiasbeyer]

And here we go again: http://pastebin.com/1DDYRn11

[vrthra]

If you have time :), could you also paste the same with :set ft=mail?

[matthiasbeyer]

I guess this is already one with set ft=mail, as I opened a mail file when calling strace on nvim!

[vrthra]

Could you confirm if set ft=mail is the culprit? Does the crash occur if set ft=mail is not called? i.e If you open a non existing file outside mutt?

[matthiasbeyer]

No it is actually not... it also crashed for other filetypes and also for nonexisting files and also for existing files without setting the ft.

[vrthra]

Any chance you could post those, I am especially interested in the time when vim/nvim does not crash on startup, but after some time, due to some action such as setting ft=mail or opening a file causes it to dump. Any way of demarcating the strace during that time in the logs would also be really good.

[matthiasbeyer]

It's private data, so no, sorry, I cannot paste these online.

It is funny that this little patch revert fixed the issue for me, though. I just rebuild my system with the patch reverted and it seems to fix the issue completely, with both nvim and vim.

I would say this is a really hard one, as the bug depends on a very specific set of things, like vimrc, xterm compiletime settings and maybe even other things we did not discover yet.

---

I really think that xterm is buggy here. I cannot think of some issue that crashes _both_ vim and nvim so hard that xterm gets killed. The strace output of nvim tells us that it gets a SIGHUB:

Vim: Caught deadly signal 'SIGHUP'

which might be from when xterm crashes. So I really think the compiletime settings of xterm bring in buggy code which gets triggered by vim/nvim sending some commands to xterm.

A funny side-note is also that when I start an xterm by calling xterm on the commandline (inside a non-buggy or buggy xterm, doesn't matter) - I cannot reproduce the issue. Or at least the issue wasn't reproducible this way _by now_.

[matthiasbeyer]

Fuck. The issue just happened with an xterm without the mentioned patch. (and nvim)

[matthiasbeyer]

It appears to be almost random. I just tried to open nvim and vim on a mail file several times with the xterm which _inclues_ the --enable-dec-locator option in the compile-time settings and it worked _every single time_.

[vrthra]

Hmm, I think we can rule out vim and nvim as culprits here as you said. Since it is SIGHUP, it is probably from xterm. The only other experiment I would ask you to do is to start dtach, and start vim from inside that. See if that crashes.

[matthiasbeyer]

@vrthra Sure, tell me the commandline calls I have to do! (As nix-shell is buggy right now I would nix-shell -p dtach --run ... or nix-shell -p dtach --pure to test this... or do you want me to nix-env -iA nixos.dtach it?)

[vrthra]

nix-shell -p dtach -p nvim --pure would do nicely. After that dtach -A /tmp/vim vim to actually start vim. Then once you have made it crash, do dtach -a /tmp/vim

[matthiasbeyer]

I did:

• nix-shell -p dtach -p neovim --pure
• dtach -A /tmp/vim nvim inside the spawned shell

That worked for me. Setting set ft=mail worked as well. Note that this does not load the vimrc.

Calling nvim -u /nix/store/89cx9lx88vcp4hksfv17x2wk1jglsz9s-vimrc inside the dtach worked as well.

[vrthra]

Is your environmental settings causing it? could you do it with nix-env -iA nixos.dtach? and the above?

[matthiasbeyer]

Is your environmental settings causing it?

I'm not sure what you mean by that. My vimrc is baked into my vim via configuration.nix, if you mean that.

I just did the above after nix-env -iA nixos.dtach - same results as reported above.

[vrthra]

My Nix ignorance shows :), so when you do --pure, I assumed you get a new environment with no environmental variables such as those set on .bashrc. So, if there was a difference between nix-env -iA nixos.dtach and nix-env --pure, then I would suspect some environment variable causing it. Since it does not seem to have an impact, perhaps not. And seems I am wrong. .bashrc is still sourced in --pure

[matthiasbeyer]

I guess it could help if I would paste my whole configuration.nix and ~/.vim/~/.nvim, but that's ~29 kloc for configuration.nix and another 7.5 kloc for the vim settings... so I doubt that would be a quick way of finding the issue.

[vrthra]

I am mostly out of ideas now :(. I suspect this is not due to the dec-locator and is only triggered due to the particular conditions. May be turn off the sixel-graphics along with the dec-locator and see if it helps. I will monitor this bug to see if any one else faces a similar issue, so we have more info.

[matthiasbeyer]

So, small update from me: I just experienced this issue again... I used vim on a xterm which does not include the mentioned compiletime flag. As it crashed, I tried vim -u NORC on the file - it worked. Then, I :syn on in vim and vim/xterm crashed.

If I use nvim and call nvim -u NORC it instantly crashes, though.

[matthiasbeyer]

I just rebooted and tested the same file as above. It worked.

[vrthra]

Just to confirm, you got a crash even if both dec-locator and sixel-graphics are disabled?

[matthiasbeyer]

The former yes, I don't know how to disable the latter.

[vrthra]

You could do that by removing "--enable-sixel-graphics" from the configureFlags in xterm.

[vrthra]

@fpletz @matthiasbeyer Did this issue get resolved? I thought the dec-locator was not really connected to this issue.

[matthiasbeyer]

@vrthra Yes, the patch was nice but it wasn't related to the issue apparently.

[vrthra]

@matthiasbeyer @fpletz Shouldn't we reopen this issue?

[fpletz]

Sorry, I seem to have misread your discussion. Should we also revert the commit or does it have any value?

[vrthra]

@fpletz The commit is harmless, and adds one option for configuration. I think that might be useful.

[matthiasbeyer]

I just re-experienced this issue but with a completely unrelated setup: When using the new git options color.diff.new "green italic", xterm crashes when viewing a diff that contains added lines - I guess the problem is the _italic_ output, not vim or something!

[matthiasbeyer]

I just tested whether the xterm decLocator option was the issue and yes, disabling declocator solves the issue.

[matthiasbeyer]

(Note that after nixos-rebuilding the system, the X-Server might need a restart).

I'll close this now.