Jedi: Jedi, Anaconda and MacPorts. How to specify python binary without virtualenv

[rebelot]

Hello all, first of all, thanks for your great work!

I am having troubles setting up python code completions in neovim. I have my completion sources managed by deoplete and I have tried to work with 2 sources: LanguageClient-neovim and deoplete-jedi.

I was happy for a few seconds with the language client, until I tired to complete numpy.

Issuing import nump<TAB> with one of the two sources at a time:

deoplete-jedi: correctly completes numpy and submodules
language-client: nothing happens
(NOTE: that the language client works "well" despite it fails to list many modules)

I have

❯ python --version
Python 3.6.5 :: Anaconda, Inc.

and

❯ conda list jedi
# packages in environment at /opt/anaconda3:
#
# Name                    Version                   Build  Channel
jedi                      0.12.0                   py36_1
~
❯ conda list python-language-server
# packages in environment at /opt/anaconda3:
#
# Name                    Version                   Build  Channel
python-language-server    0.19.0                    <pip>

Both deoplete-jedi and the language client use jedi as a backend for code completion, the latter through python-language-server. And this made me thought that with deoplete-jedi I have manually set: let g:deoplete#sources#jedi#python_path = "/opt/anaconda3/bin/python", while with the language server I have no such option (at least, I am unable to find how to specify this). Also with VSCode I was able to set up a python path and all was working nicely. Then,

i've tested jedi with the following commands from the command line:

❯ python -c "import jedi; print(jedi.Script('import nump').completions())"
[]
❯ python -c "import jedi, sys; print(jedi.Script('import nump',sys_path=sys.path).completions())"
[<Completion: numpy>, <Completion: numpydoc>]
❯ python -c "import jedi; print(jedi.get_default_environment().executable)"
/opt/local/bin/python3.6
❯ python -c "import sys; print(sys.path)"
['',
'/opt/anaconda3/lib/python36.zip',
'/opt/anaconda3/lib/python3.6',
'/opt/anaconda3/lib/python3.6/lib-dynload',
'/opt/anaconda3/lib/python3.6/site-packages',
'/opt/anaconda3/lib/python3.6/site-packages/aeosa']

Which pretty much gets to what the problem is, but I don't know what to do next! I'd really like to use the language server over deoplete-jedi...

btw, I use MacPorts to maintain various GNU utilities, compilers, libraries and other niceties I am not willing to take care of by myself. It installed python "by itself" as a dependancy for some package, therein

❯ port select python
Available versions for python:
        none (active)
        python26-apple
        python27
        python27-apple
        python36

but it's of no use.

I am not a python developer, I am a bioinformatician and I need python with scientific libraries to make data analysis scripts or other workflow automations. In my spare time I am a vim tweak-addict. What should I do to manage my python in a user friendly way and have the least possible number of issues? should I wave goodbye to anaconda in favour of pipenv? Also, say I want to make a python script that will run with a python distributed by some commercial software, how could I get completions?

[davidhalter]

Thanks for your long issue description.

Since it works with deoplete-jedi (where you specify an executable), it is probably not a Jedi bug. I'm pretty sure that the python-language-server doesn't let you specify an executable to run for Jedi. You should be using something like:

environment = jedi.create_environment('/opt/anaconda3/bin/python')
 jedi.Script('import nump', environment=environment).completions()

This should be enough to get it working. I'm also pretty sure that that's what deoplete is doing. The problem lies probably with the language server. As long as it doesn't allow you to specify an environment, you have a problem :)

[rebelot]

From what I have learned, if a sys_path is not specifically issued to jedi, it will look first for an active virtualenv, then for the first suitable python binary in PATH. As I did not want to put /opt/anaconda3 ahead of /opt/local in my $PATH (it felt like an inconsistent, side-effects–prone workaround), I have come up with the following solution, albeit relying on virtualenvs:

1. install anaconda python (I already had this)
2. conda activate base
3. conda install virtualenv
4. virtualenv --system-site-packages ~/venv

the following lines in .zshrc:

source /opt/anaconda3/etc/profile.d/conda.sh 
[[ -z $TMUX ]] || conda deactivate; conda activate base 
source $HOME/venv/bin/activate

The result is a virtualenv wrapped around the current anaconda environment: I guess, if I change the current conda environment virtualenv will have access to different modules as well. conda installs will work as usual and will be accessible by the venv, and pip installs will be venv-specific, unless I deactivate the venv, the they will be conda-env specific, which is doable.

Is it acceptable or is it a terrible mess which will force me to set my computer on fire someday? Python ways are tortuous.

Anyway, Thank You ^^

[davidhalter]

Since you obviously found a way that works for you and I don't see what we should change in Jedi, I'm going to close this one. Generally speaking I think there are nice ways to specify environments in Jedi, even though you have to write a little bit of code for that to make it work, but it's at least consistent and explicit.

The latest master branch in jedi contains a lot of improvements to make life with anaconda and other interpreters easier.

[samwindels]

From what I have learned, if a sys_path is not specifically issued to jedi, it will look first for an active virtualenv, then for the first suitable python binary in PATH. As I did not want to put /opt/anaconda3 ahead of /opt/local in my $PATH (it felt like an inconsistent, side-effects–prone workaround), I have come up with the following solution, albeit relying on virtualenvs:

1. install anaconda python (I already had this)
2. conda activate base
3. conda install virtualenv
4. virtualenv --system-site-packages ~/venv

the following lines in .zshrc:

source /opt/anaconda3/etc/profile.d/conda.sh 
[[ -z $TMUX ]] || conda deactivate; conda activate base 
source $HOME/venv/bin/activate

The result is a virtualenv wrapped around the current anaconda environment: I guess, if I change the current conda environment virtualenv will have access to different modules as well. conda installs will work as usual and will be accessible by the venv, and pip installs will be venv-specific, unless I deactivate the venv, the they will be conda-env specific, which is doable.

Is it acceptable or is it a terrible mess which will force me to set my computer on fire someday? Python ways are tortuous.

Anyway, Thank You ^^

Hi @rebelot,

did this approach actually work for you? I got virtual env to wrap my conda enviroment per your instructions. However, in my case jedi-vim picked up my tags file but nothing more. I.e. the functions\classes\modules I personally defined get auto-completed properly but not those defined in standard libraries (e.g. numpy, matplotlib, which are installed in my conda environment).

Kind regards,

Sam

[rebelot]

@samwindels yeah it’s working fine, be aware that python -m venv is recommended over virtualenv.
Also make sure to activate the venv from the instance from where you’ll use Jedi.
If you are using Jedi-vim, however, there should be an option to specify the python binary path so you’ll have no problems

[davidhalter]

This issue is kind of old and I wrote that I was closing it. That didn't happen. I'm closing it now. If there still is an issue, please describe it and I will reopen.