Spacevim: pyenv + neovim python package setup not working as expected

[neverfox]

Expected behavior, english is recommend

After following these steps, I should be able to start spacevim (neovim) in any pyenv environment or no environment at all, and it won't complain about missing neovim python package.

Environment Information

• OS: OSX
• vim version: -
• neovim version: 0.2.2
• SpaceVim version: 0.8.0-dev
• SpaceVim status:

On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

The reproduce ways from Vim starting (Required!)

1. Set up virtualenvs and init.vim according to this.
2. Exit all virtualenvs, i.e. pyenv is pointing to system python where no neovim package is installed (it's only installed in the two virtualenvs).
3. Start nvim
4. Get message:
   Floobits error: no neovim python module. Run `pip install neovim` to fix. For more info, :he nvim-python Press ENTER or type command to continue
5. Press ENTER and spacevim fails to start properly:
   ```
   Vim(if):E475: Invalid argument: Channel id must be a positive integer
   Error detected while processing function
   remote#define#AutocmdBootstrap[1]..remote#host#Require[10]..provider#pythonx#Require:
   line 27:
   E605: Exception not caught: Failed to load python host. You can try to see what happened by
   starting nvim with $NVIM_PYTHON_LOG_FILE set and opening the generated log file. Also, the host
   stderr is available in messages.
   Error detected while processing function
   remote#define#AutocmdBootstrap[1]..remote#host#Require:
   line 10:
   E171: Missing :endif
6. Activate either neovim2 or neovim3 virtualenv, start nvim, and spacevim starts fine.
7. Undo everything and instead pip install --user neovim and pip3 install --user neovim and everything works as expected outside of virtualenvs, but now you have packages installed against the system interpreters that you'd rather have in a virtualenv. Also, :checkhealth now says I'm not configured optimally for pyenv and tells me to do I just tried that didn't work:

## Python 2 provider (optional)
  1   - OK: pyenv found: "/usr/local/Cellar/pyenv/1.2.3/libexec/pyenv"
  2   - INFO: `g:python_host_prog` is not set.  Searching for python2 in the environment.
  3   - WARNING: pyenv is not set up optimally.
  4   ┊ - ADVICE:
  5   ┊ ┊ - Create a virtualenv specifically for Neovim using pyenv, and set `g:python_host_prog`.  This will avoid the need to install the Neovim Python module in each version/virtualenv.
  6   - INFO: Executable: /usr/local/bin/python2
  7   - INFO: Python2 version: 2.7.14
  8   - INFO: python2-neovim version: 0.2.4
  9   - OK: Latest python2-neovim is installed: 0.2.4
 10
 11 ## Python 3 provider (optional)
 12   - OK: pyenv found: "/usr/local/Cellar/pyenv/1.2.3/libexec/pyenv"
 13   - INFO: `g:python3_host_prog` is not set.  Searching for python3 in the environment.
 14   - WARNING: pyenv is not set up optimally.
 15   ┊ - ADVICE:
 16   ┊ ┊ - Create a virtualenv specifically for Neovim using pyenv, and set `g:python3_host_prog`.  This will avoid the need to install the Neovim Python module in each version/virtualenv.
 17   - INFO: Executable: /usr/local/bin/python3
 18   - INFO: Python3 version: 3.6.5
 19   - INFO: python3-neovim version: 0.2.4
 20   - OK: Latest python3-neovim is installed: 0.2.4

To add insult to injury, now it doesn't work inside of a virtualenv (unless I install neovim in it).

Output of the :SPDebugInfo!

SpaceVim Options :

g:spacevim_altmoveignoreft = ['Tagbar', 'vimfiler']
g:spacevim_auto_disable_touchpad = 1
g:spacevim_autocomplete_method = 'completor'
g:spacevim_buffer_index_type = 1
g:spacevim_checkinstall = 1
g:spacevim_colorscheme = 'base16-darktooth'
g:spacevim_colorscheme_bg = 'dark'
g:spacevim_colorscheme_default = 'desert'
g:spacevim_custom_color_palette = []
g:spacevim_custom_plugins = [['chriskempson/base16-vim']]
g:spacevim_debug_level = 1
g:spacevim_default_indent = 2
g:spacevim_dein_installed = 1
g:spacevim_denite_leader = 'F'
g:spacevim_disabled_plugins = []
g:spacevim_enable_ale = 0
g:spacevim_enable_cursorcolumn = 0
g:spacevim_enable_cursorline = 1
g:spacevim_enable_debug = 1
g:spacevim_enable_googlesuggest = 0
g:spacevim_enable_guicolors = 1
g:spacevim_enable_insert_leader = 1
g:spacevim_enable_javacomplete2_py = 0
g:spacevim_enable_key_frequency = 0
g:spacevim_enable_language_specific_leader = 1
g:spacevim_enable_neocomplcache = 0
g:spacevim_enable_neomake = 1
g:spacevim_enable_os_fileformat_icon = 1
g:spacevim_enable_powerline_fonts = 1
g:spacevim_enable_statusline_display_mode = 0
g:spacevim_enable_tabline_filetype_icon = 1
g:spacevim_enable_vimfiler_filetypeicon = 0
g:spacevim_enable_vimfiler_gitstatus = 0
g:spacevim_enable_vimfiler_welcome = 1
g:spacevim_enable_ycm = 0
g:spacevim_error_symbol = '✖'
g:spacevim_expand_tab = 1
g:spacevim_filemanager = 'vimfiler'
g:spacevim_filetype_icons = {}
g:spacevim_force_global_config = 0
g:spacevim_gitcommit_issue_icon = ''
g:spacevim_gitcommit_pr_icon = ''
g:spacevim_github_username = ''
g:spacevim_guifont = ''
g:spacevim_hiddenfileinfo = 1
g:spacevim_hosts_url = 'https://raw.githubusercontent.com/racaljk/hosts/master/hosts'
g:spacevim_info_symbol = 'ⓘ'
g:spacevim_keep_server_alive = 1
g:spacevim_language = ''
g:spacevim_leader_guide_default_group_name = ''
g:spacevim_leader_guide_flatten = 1
g:spacevim_leader_guide_hspace = 5
g:spacevim_leader_guide_max_size = 0
g:spacevim_leader_guide_position = 'botright'
g:spacevim_leader_guide_run_map_on_popup = 1
g:spacevim_leader_guide_sort_horizontal = 0
g:spacevim_leader_guide_submode_mappings = {'<C-C>': 'win_close'}
g:spacevim_leader_guide_vertical = 0
g:spacevim_lint_on_save = 1
g:spacevim_lint_on_the_fly = 0
g:spacevim_max_column = 120
g:spacevim_neobundle_installed = 0
g:spacevim_plugin_bundle_dir = '/Users/neverfox/.cache/vimfiles/'
g:spacevim_plugin_manager = 'dein'
g:spacevim_plugin_manager_max_processes = 16
g:spacevim_plugin_name = 'screensaver.vim'
g:spacevim_project_rooter_automatically = 1
g:spacevim_project_rooter_patterns = ['.git/', '_darcs/', '.hg/', '.bzr/', '.svn/', '.clang', 'pom.xml', 'package.json']
g:spacevim_realtime_leader_guide = 1
g:spacevim_relativenumber = 1
g:spacevim_search_tools = ['rg', 'ag', 'pt', 'ack', 'grep']
g:spacevim_sidebar_width = 30
g:spacevim_simple_mode = 0
g:spacevim_smartcloseignoreft = ['tagbar', 'vimfiler', 'SpaceVimRunner', 'SpaceVimREPL', 'SpaceVimQuickFix', 'HelpDescribe', 'VebuggerShell', 'VebuggerTerminal']
g:spacevim_smartcloseignorewin = ['__Tagbar__', 'vimfiler:default']
g:spacevim_snippet_engine = 'neosnippet'
g:spacevim_src_root = 'E:\sources\'
g:spacevim_statusline_inactive_separator = 'arrow'
g:spacevim_statusline_left_sections = ['winnr', 'filename', 'major mode', 'syntax checking', 'minor mode lighters']
g:spacevim_statusline_right_sections = ['fileformat', 'cursorpos', 'percentage']
g:spacevim_statusline_separator = 'arrow'
g:spacevim_statusline_unicode_symbols = 1
g:spacevim_terminal_cursor_shape = 2
g:spacevim_unite_leader = '\f'
g:spacevim_version = '0.8.0-dev'
g:spacevim_vim_help_language = 'en'
g:spacevim_vim_plug_installed = 0
g:spacevim_warning_symbol = '⚠'
g:spacevim_wildignore = '*/tmp/*,*.so,*.swp,*.zip,*.class,tags,*.jpg,*.ttf,*.TTF,*.png,*/target/*,.git,.svn,.hg,.DS_Store,*.svg'
g:spacevim_windows_index_type = 0
g:spacevim_windows_leader = 's'
g:spacevim_windows_smartclose = 'q'

SpaceVim Health checking :

SpaceVim clipboard support check report:
Checking +clipboard:
SUCCEED!
SpaceVim environment check report:
Current progpath: nvim(/usr/local/Cellar/neovim/0.2.2_1/bin/nvim)
version: 800
OS: OSX
SpaceVim lua support check report:
Checking +lua:
Failed : Known issue, neovim do not support lua now.
SpaceVim python support check report:
Checking +python3:
Failed : to support +python3, you need run pip3 install neovim
Checking +python:
SUCCEED!

SpaceVim runtime log :

[ SpaceVim ] : logger file  does not exists, only log for current process will be shown!
[ SpaceVim ] [12:23:15] [ Info ] Skip glob configuration of SpaceVim
[ SpaceVim ] [12:23:15] [ Info ] SpaceVim server startup at:/tmp/spacevim_nvim_server

Screenshots

If you have any screenshots for this issue please upload here. BTW you can use https://asciinema.org/ for recording video in terminal.

[abijr]

_Workaround:_
Try adding

let g:python_host_prog = '/full/path/to/neovim2/bin/python'
let g:python3_host_prog = '/full/path/to/neovim3/bin/python'

to ~/.SpaceVim/init.vim instead of ~/.SpaceVim.d/init.vim

[wsdjeg]

you should not edit ~/.SpaceVim/init.vim

[abijr]

@wsdjeg that's why I marked it as a "workaround", it's the only way I found that made stuff work. Do you have an alternative solution/fix?

[wsdjeg]

adding to ~/.SpaceVim.d/init.vim should works. If it does not. It is a bug.

[abijr]

I'm experiencing the same behavior as OP.

[neverfox]

@wsdjeg @abijr Thanks for both of your replies. I didn't get much from the Neovim folks (was asked if I'd read closed issues. A: of course, but none of that currently works). Is there any more information I could provide that would be of help?

[wsdjeg]

@neverfox Floobits this plugin need python support in neovim, so if you starting nvim in pyenv, you need to install neovim package in it.

[stroxler]

I can confirm that on my system, adding those lines to ~/.Spacevim.d/init.vim did not work; e.g. vim was unable to find the python3 interpreter when I was in a python2 pyenv. But @abijr's workaround did work: as soon as I added the lines to ~/.SpaceVim/init.vim spacevim started finding both the python2 and python3 interpreters regardless of what pyenv I was in.

I determined whether the python config was working in each setup by examining the following two lines in the Spacevim Health Check section:

Checking +python3:
Failed : to support +python3, you need run pip3 install neovim
Checking +python:
SUCCEED!

When the neovim python setup is working properly, both of these should SUCCEED! regardless of the pyenv from which we start neovim.

I'd guess that SpaceVim must be loading code which depends on the python interpreter path sometime prior to loading ~/.Spacevim.d/init.vim. If this behavior is a bug, then perhaps this issue could be more usefully reframed as a bug report

I know some editor plugin projects - including spacemacs if I remember correctly - have separate sections for user config that needs to run before vs after the project code loads; maybe we need a ~/.SpaceVim.d/pre-init.vim or something that gets loaded as the absolute first thing SpaceVim does on startup

[neverfox]

@wsdjeg Thanks for the reply. I understand why it's complaining. What I don't understand is why I'm experiencing the behavior given that I followed what :checkhealth tells me should work without having to do that (which is the same as what the folks at deoplete-jedi said). If I'm understanding it correctly, following those steps should let me start nvim/spacevim with any python plugins (2 or 3) and it just works, no matter what environment I'm in, including no environment at all.

Here's my basic dilemma. Neovim (w/ spacevim) is my $EDITOR, so I use it for every one-off editing task and need it to work anywhere and everywhere. I don't however use it for Python programming, preferring to use either Spacemacs or VS Code when working on a Python project. That said, I often need my basic editor for all kinds of things while I'm working in a Python project (which means I most certainly have a venv activated at those times), e.g. git tasks, quick shell edits, weechat multiline messages while I'm chatting with folks, etc. Currently, I haven't figured out a way to be in a Python virtualenv and use neovim/spacevim without doing what you say, i.e. installing the neovim package in every venv I might find myself in and the system python so I can use it outside of virtualenvs. Not only is that less than ideal for one's $EDITOR but it's also contrary to what the suggested instructions say:

This will avoid the need to install the Neovim Python module in each version/virtualenv.

So the first issue is why is this statement not seemingly holding? That is to say, why can't I, despite doing exactly what it says, run it in any virtualenv that's not the special neovim one? Because right now I can't.

The other issue concerns running it outside of any virtualenv (or in pyenv terms, when the interpreter is "system"). The only way I've been able to do that without complaint is also having the neovim package installed as a system level package (really --user) in addition to the neovim2/3 virtualenvs. That makes sense given how I understand pyenv to work, but I assumed from the instructions that there was some kind of magic in setting g:python_host_prog and g:python3_host_prog that would make it work, especially since I've seen other mention the venv option as an alternative to help you avoid having to do system level pip installs. Maybe all those people are just only using neovim as a python editor so it never occurred to them to worry about the other cases. So the question here is should it work when I'm not in a venv if I set up the two variables to point to special neovim venvs?

[neverfox]

@stroxler Unfortunately, that didn't work for me. The behavior stays the same.

[neverfox]

When in doubt, uninstall, delete everything, and reinstall. Now it works. Python is disabled, vanilla config, but then I never did install any Python-based plugins intentionally. I don't know why my older installation of SpaceVim has plugins that were crying out for Python and not working correctly, but now it's peace and quiet.

[wsdjeg]

you should know how neovim provide +py and +py3 feature, it need a neovim python client, so in pyenv, you need to installl neovim python package via pip/pip3.