Ale: ALE chooses wrong virtual environment for python

[justfortherec]

Information

VIM version

VIM - Vi IMproved 8.1 (2018 May 17, compiled May 21 2018 11:59:30)
Included patches: 1-10

Operating System:

Linux 4.15.0-29-generic #31~16.04.1-Ubuntu SMP Wed Jul 18 08:54:04 UTC 2018 x86_64

Description:    Ubuntu 16.04.5 LTS



md5-a881fb8640ed4bc1005e724b9792b97a



Current Filetype: python
Available Linters: ['flake8', 'mypy', 'prospector', 'pycodestyle', 'pyflakes', 'pylint', 'pyls']
  Enabled Linters: ['flake8', 'mypy', 'pylint']
 Linter Variables:

let g:ale_python_flake8_executable = 'flake8'
let g:ale_python_flake8_options = ''
let g:ale_python_flake8_use_global = 0
let g:ale_python_mypy_executable = 'mypy'
let g:ale_python_mypy_ignore_invalid_syntax = 0
let g:ale_python_mypy_options = ''
let g:ale_python_mypy_use_global = 0
let g:ale_python_pylint_executable = 'pylint'
let g:ale_python_pylint_options = ''
let g:ale_python_pylint_use_global = 0
 Global Variables:

let g:ale_cache_executable_check_failures = 0
let g:ale_change_sign_column_color = 0
let g:ale_command_wrapper = ''
let g:ale_completion_delay = 100
let g:ale_completion_enabled = 0
let g:ale_completion_max_suggestions = 50
let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%code: %%s'
let g:ale_echo_msg_info_str = 'Info'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_fix_on_save = 0
let g:ale_fixers = {}
let g:ale_history_enabled = 1
let g:ale_history_log_output = 1
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
let g:ale_lint_on_enter = 1
let g:ale_lint_on_filetype_changed = 1
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 'always'
let g:ale_lint_on_insert_leave = 0
let g:ale_linter_aliases = {}
let g:ale_linters = {}
let g:ale_linters_explicit = 0
let g:ale_list_window_size = 10
let g:ale_list_vertical = 0
let g:ale_loclist_msg_format = '%code: %%s'
let g:ale_max_buffer_history_size = 20
let g:ale_max_signs = -1
let g:ale_maximum_file_size = 0
let g:ale_open_list = 0
let g:ale_pattern_options = {}
let g:ale_pattern_options_enabled = 0
let g:ale_set_balloons = 1
let g:ale_set_highlights = 1
let g:ale_set_loclist = 1
let g:ale_set_quickfix = 0
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = '>>'
let g:ale_sign_info = '--'
let g:ale_sign_offset = 1000000
let g:ale_sign_style_error = '>>'
let g:ale_sign_style_warning = '--'
let g:ale_sign_warning = '--'
let g:ale_statusline_format = ['%d error(s)', '%d warning(s)', 'OK']
let g:ale_type_map = {}
let g:ale_warn_about_trailing_blank_lines = 1
let g:ale_warn_about_trailing_whitespace = 1
  Command History:

(executable check - success) <working-directory>/env/bin/flake8
(started) ['/bin/bash', '-c', '''<working-directory>/env/bin/flake8'' --version']
(executable check - failure) mypy
(executable check - success) <working-directory>/env/bin/pylint
(started) ['/bin/bash', '-c', '''<working-directory>/env/bin/pylint''  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n ''<working-directory>/crashreport_stats/management/commands/stats.py''']
(finished - exit code 0) ['/bin/bash', '-c', '''<working-directory>/env/bin/flake8'' --version']
(executable check - failure) mypy
(finished - exit code 22) ['/bin/bash', '-c', '''<working-directory>/env/bin/pylint''  --output-format text --msg-template="{path}:{line}:{column}: {msg_id} ({symbol}) {msg}" --reports n ''<working-directory>/crashreport_stats/management/commands/stats.py''']

<<<OUTPUT STARTS>>>
************* Module crashreport_stats.management.commands.stats
crashreport_stats/management/commands/stats.py:220:0: W0511 (fixme) FIXME Agressively drop duplicates
crashreport_stats/management/commands/stats.py:26:0: C1001 (old-style-class) Old-style class defined.
crashreport_stats/management/commands/stats.py:65:4: E1002 (super-on-old-class) Use of super on an old style class
crashreport_stats/management/commands/stats.py:85:4: E1002 (super-on-old-class) Use of super on an old style class
crashreport_stats/management/commands/stats.py:152:0: C1001 (old-style-class) Old-style class defined.
crashreport_stats/management/commands/stats.py:324:4: E1002 (super-on-old-class) Use of super on an old style class
crashreport_stats/management/commands/stats.py:337:4: E1002 (super-on-old-class) Use of super on an old style class

------------------------------------------------------------------
Your code has been rated at 8.23/10 (previous run: 8.23/10, +0.00)

<<<OUTPUT ENDS>>>

(finished - exit code 0) ['/bin/bash', '-c', '''<working-directory>/env/bin/flake8'' --format=default --stdin-display-name ''<working-directory>/crashreport_stats/management/commands/stats.py'' - < ''/tmp/v5qHriy/6/stats.py''']

<<<NO OUTPUT RETURNED>>>

As apparent in the log, virtual environment env is used while I have a different environment enabled.

What went wrong

Linters installed in the wrong virtual environment are run.

I have several virtual environments for one project (e.g. various versions of python) and I would like the linter of the currently enabled environment to be used. ALE supports reading the path from ${VIRTUAL_ENV} but defaults to well known virtual environment names (defined in ale_virtualenv_dir_names) if they exist.

Reproducing the bug

1. Create virtual environments
   a. virtualenv -p python2 ve-py2
   b. virtualenv -p python3 ve-py3
2. Note that ve-py3 is in ale_virtualenv_dir_names, while ve-py2 is not.
3. Install linters in both environments.
4. Enable the virtual environment without well-known name: source ve-py2/bin/activate
5. Open a python file in vim from a subdirectory of current working directory.

Expected result:

Linters from the active environment are used.

Actual result:

Linters from the environment with name in ale_virtualenv_dir_names are used.

Change request

Change ale#python#FindVirtualenv(buffer) to default to ${VIRTUAL_ENV} if it is available and only fall back to environments in ale_virtualenv_dir_names if the environment variable is not set.

[w0rp]

The git repository virtualenv directories should be preferred over VIRTUAL_ENV, and I can't change this without breaking how some people have ALE set up already. You should configure g:ale_virtualenv_dir_names so it doesn't include the directory name that causes problems for you.

[w0rp]

Alternatively, you can configure the variable so 've-py2' is in your list of directories to look for. What you can't do is break how others have configured ALE so it works better for you.

[w0rp]

I might consider this, actually. I'll have to think about it. I'll have to find out if using VIRTUAL_ENV first will break something. It probably will.

[w0rp]

One issue to take into account is if you do this.

export VIRTUAL_ENV=...
vim
# Navigate to a file in a different project which contains a virtualenv directory

If the VIRTUAL_ENV environment variable is preferred over the local directory, ALE will start checking projects with the programs. If the local directories are preferred over VIRTUAL_ENV, the correct programs will be used.

[w0rp]

Okay, the current behavior is better, and I'm not changing it. Sorry about the comment spam. Change your settings in vimrc or wherever.

[justfortherec]

Fair enough. Thanks for considering it.

[justfortherec]

(although, I still consider it a bug because ALE performs some secret magic without telling me about it. I was seriously confused because ALE showed different output than the linters in the same terminal).

[hwalinga]

ALE supports reading the path from ${VIRTUAL_ENV}

I did my best, but I couldn't find how to configure ALE to read from ${VIRUAL_ENV} instead. Can anybody post how to configure ALE to have this behaviour?

[justfortherec]

What @w0rp wrote works:

You should configure g:ale_virtualenv_dir_names so it doesn't include the directory name that causes problems for you.

I use it with this in my .vimrc:

" Disable auto-detection of virtualenvironments
let g:ale_virtualenv_dir_names = []
" Environment variable ${VIRTUAL_ENV} is always used