Ale: Processes not being killed (SIGKILL) in NeoVim

[jwkvam]

Sometimes lately I'll run into the issue where I get errors, that I can't launch any more processes. It turns out that I'll find a bunch of pylint processes. I'm not really sure what the root cause is, I'm hoping someone might help me root cause this or point me in the right direction.

I do tend to save files frequently and if just open a python file and do :wqa, then I can reproducibly get the following:

ps | grep pylint                                                                                                                                             
15670 ttys005    0:00.02 /Users/jacques/miniconda/bin/python /Users/jacques/miniconda/bin/pylint --output-format text --msg-template={path}:{line}:{column}: {msg_id} ({symbol}) {msg} --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvimSnRW2q/3/tfnets.py
15671 ttys005    0:02.99 /Users/jacques/miniconda/bin/python /Users/jacques/miniconda/bin/pylint --output-format text --msg-template={path}:{line}:{column}: {msg_id} ({symbol}) {msg} --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvimSnRW2q/3/tfnets.py
15672 ttys005    0:00.00 /Users/jacques/miniconda/bin/python /Users/jacques/miniconda/bin/pylint --output-format text --msg-template={path}:{line}:{column}: {msg_id} ({symbol}) {msg} --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvimSnRW2q/3/tfnets.py
15673 ttys005    0:00.00 /Users/jacques/miniconda/bin/python /Users/jacques/miniconda/bin/pylint --output-format text --msg-template={path}:{line}:{column}: {msg_id} ({symbol}) {msg} --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvimSnRW2q/3/tfnets.py
15674 ttys005    0:00.00 /Users/jacques/miniconda/bin/python /Users/jacques/miniconda/bin/pylint --output-format text --msg-template={path}:{line}:{column}: {msg_id} ({symbol}) {msg} --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvimSnRW2q/3/tfnets.py

If there's more that I can do to debug this I would appreciate any help.

Here's my ALEInfo

 Current Filetype: python
Available Linters: ['flake8', 'mypy', 'pylint']
  Enabled Linters: ['pylint']
 Linter Variables:

let g:ale_python_pylint_executable = 'pylint'
let g:ale_python_pylint_options = ''
 Global Variables:

let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%s'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
clipboard: error: 
let g:ale_lint_on_enter = 1
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 1
let g:ale_linter_aliases = {}
let g:ale_linters = {'python': ['pylint']}
let g:ale_open_list = 0
let g:ale_set_highlights = 1
let g:ale_set_loclist = 1
let g:ale_set_quickfix = 0
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = '✗'
let g:ale_sign_offset = 1000000
let g:ale_sign_warning = '⚠'
let g:ale_statusline_format = ['⨉ %d', '⚠ %d', '⬥ ok']
let g:ale_warn_about_trailing_whitespace = 1
  Command History:

[w0rp]

It sounds like pylint never stops executing. Does the problem go away if you change call job_stop(a:job) to call job_stop(a:job, 'kill') in autoload/ale/engine.vim?

[jwkvam]

Thanks for the tip. I should have mentioned that I'm using nvim. I tried modifying line 59 to call jobstop(a:job, 'kill') but as you probably know I guess that function doesn't take additional arguments.

I found this in the neovim docs, so I'm not sure if I can directly send sigkill to a process :( I can dig a little more from here though. Maybe this is just a bug with pylint 😕
https://github.com/neovim/neovim/blob/master/runtime/doc/job_control.txt#L142

[w0rp]

Ah, you're on NeoVim. That was a suggestion for the Vim 8 function.

Is there nothing in the Command history for your output? NeoVim is supposed to SIGKILL any process which takes too long to exit, as it says in the documentation.

[jwkvam]

Yea it looks something like this. All the pylint processes should have finished (or at least been killed) by the point I took this snapshot. Seems like it might happen when multiple processes are kicked off at the same time, but that's a really haphazard observation.

  Command History:
(finished - exit code 16) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/3/tfnets.py'
(finished - exit code 16) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/4/tfnets.py'
(finished - exit code 22) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/5/tfnets.py'
(started) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvim4H26Jj/
6/tfnets.py'
(finished - exit code 16) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/7/tfnets.py'
(started) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvim4H26Jj/
8/tfnets.py'
(started) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvim4H26Jj/
9/tfnets.py'
(finished - exit code 22) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/10/tfnets.py'
(started) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000gn/T/nvim4H26Jj/
11/tfnets.py'
(finished - exit code 16) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/12/tfnets.py'
(finished - exit code 16) 'pylint  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n /var/folders/gd/p_8rmvg56j39gd6lnkb7_51h0000
gn/T/nvim4H26Jj/13/tfnets.py'

[w0rp]

One thing that might help you here is the advice in this section: https://github.com/w0rp/ale#5xi-will-this-plugin-eat-all-of-my-laptop-battery-power

You can configure how often linting is done, and that might lead to fewer processes being created.

[wbthomason]

I can confirm that I'm still seeing this issue even when only linting on load and write. I'm also using Neovim, so I think @jwkvam might be right when he says this is a problem with Neovim sending SIGKILL correctly.

[w0rp]

@wbthomason Try pulling again and tell me if you can repeat the issue. I have added some experimental code for manually running kill -9 and the equivalent on Windows for killing the process with after a short delay.

[wbthomason]

@w0rp I seem to be still seeing the issue. What info can I send to help with the debugging process?

[w0rp]

You could start adding :echo lines inside of the s:KillHandler function and you can check if you're hitting the call system('kill -9 ' . l:pid) call. It's hard for me to fix the issue, as I can't repeat the issue on my machine. I might see if I can run a program which I know won't respond to SIGTERM or something.

[wbthomason]

Ok, I'll take a look at that. I've read through the relevant source file, but I'm no vimscript wizard, unfortunately. I'll let you know what I find.

[wbthomason]

I've added echom at lines 70 and 89 of autoload/ale/engine.vim (the start of the KillHandler function and the line just before the kill -9 call), and am seeing nothing. I have tested this by opening a Python file with Pylint enabled and repeatedly saving. I then confirm using ps that there are several pylint processes running, and wait several minutes for these to be killed. ps reports more processes after this time (which is another weird thing, but I haven't looked into it yet), and I never see my echom messages. I have confirmed that I'm correctly inserting echom by also adding a call at the top of the Invoke function in ale; I see that message printed.

[w0rp]

Okay. I'll have to see if I can create a test linter which doesn't respond to SIGTERM and run that in NeoVim then.

[w0rp]

I have reported this issue to NeoVim here: https://github.com/neovim/neovim/issues/6530

I'll remove the milestone for this bug. I think this is a bug in NeoVim, and hopefully the NeoVim developers will be able to fix it soon.

[wbthomason]

Sounds good, thank you. I was never able to find a fix, so I expect that
you're right.

On Sat, Apr 15, 2017, 16:14 w0rp [e-mail] wrote:

I have reported this issue to NeoVim here: neovim/neovim#6530
https://github.com/neovim/neovim/issues/6530

I'll remove the milestone for this bug. I think this is a bug in NeoVim,
and hopefully the NeoVim developers will be able to fix it soon.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/w0rp/ale/issues/380#issuecomment-294315481, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADjp9DYXYmk6SMHUSdhq4RkaKXHg1fhgks5rwSUugaJpZM4MRwiR
.

[devlinzed]

Any news here? I don't want to need to switch from ALE over this -- I really like it.

[w0rp]

It's a bug in NeoVim, and it's being fixed there.

[wbthomason]

@w0rp @devlinzed FWIW, I haven't noticed this behavior when I've checked recently. I'm not sure what would have fixed it, but I haven't been able to reproduce it anymore.

[devlinzed]

I still notice this behaviour with NVIM v0.2.1-291-gcb0abce5, ale's current master, and cargo 0.20.0-nightly (82733b014 2017-05-28).

[w0rp]

The bug hasn't been fixed in NeoVim yet. I personally recommend using Vim 8 instead, which doesn't suffer from the same issue.

[w0rp]

I might see about submitting a patch for fixing this in NeoVim, because it seems like nobody has tried to fix this in NeoVim for months. It's a pretty serious bug.

[w0rp]

I'll leave this to the NeoVim maintainers. In the meantime, consider using Vim 8 instead.

[justinmk]

https://github.com/neovim/neovim/issues/6530 (I hope) is fixed on Nvim HEAD. Pre-release builds are available at https://github.com/neovim/neovim/releases

[w0rp]

@devlinzed There is no mailing list. You need to click the "Unsubscribe" button on the GitHub issue page.

[w0rp]

I'll close this issue now the bug has been fixed in NeoVim. If anyone has this issue, upgrade to NeoVim 0.2.3 when it is out, or grab the development version.