Jedi-vim: numpy, scipy, pandas all very slow (no python-mode here and seemingly no caching)

[memeplex]

After doing import numpy or import pandas or import scipy package level completion numpy., pandas., scipy. takes several seconds (up to more than 20 seconds for pandas). Successive completions take the same time, so maybe the cache is not working. I'm not using python-mode, my bundles are:

base16-vim  tabular         vim-markdown  vim-python-pep8-indent
jedi-vim    vim-commentary  vim-pathogen  vim-shebang

[memeplex]

No suggestion in https://github.com/davidhalter/jedi-vim/issues/163 seems to apply here.

[Guzzii]

+1

[davidhalter]

Maybe related to https://github.com/davidhalter/jedi/issues/915.

[Rmano]

+1

[blueyed]

@Rmano
Instead of +1 you might want to help out in a more productive way, e.g. by looking into why it is so slow..!
After all (all of you) should be Python developers, right?

[Rmano]

@blueyed --- I just wanted to add that I see the problem too. No, I am not a Python nor a Vim developer, unfortunately (I'd like).
I use Python with numpy, scipy, and pandas to do my job of data analysis and processing (it's my "free matlab" option --- I am an electronic, analog-hardware, engineer), but I am not skilled enough to go deep inside --- the linked issue, davidhalter/jedi#915, it's way above my developer (meager) abilities.
But if given detailed instruction I will try to help (test, etc.) in the measure I can; so yes, instead of +1, I should have written this message. Thanks for helping in better communication.

[davidhalter]

@Rmano It's actually not that easy to solve. I've tried to fix this before and it seems pretty hard. Do you have an SSD? Fairly new computer?

[Rmano]

OI Imagine it should be difficult. They are big libraries and probably the only way would be to have a kind of on-disk cache... with all the related keep-in-sync problem.

I have an i5, normal HDD, 16g ram. It's about a 10 second delay.

Thanks for the nice package, BTW!

[sobfiggis]

i5-4960k 4.6Ghz running off SSD

https://asciinema.org/a/OEHrTdJ9Pnf7Kq4ZeXWskMVx9

Even makes my fans turn on :P. What can we provide to help?

[riazrizvi]

I found airline was slowing the autocomplete popup to a crawl (as well as hampering cursor movement). I knew it was a vim specific problem and not due to the size of scipy because the autocomplete popup was near instant in terminal ipython. So try commenting out all your questionable plugins in .vimrc and then restart vim. If the problem goes away, a plugin is the culprit, and you can figure out which through a process of elimination.

[memeplex]

I use almost no plugin and the few I use are extremely conventional: some highlighting, indentation, etc.

Nevertheless I'm not experiencing too slow autocompletion for np, pd, etc. anymore. It's not super fast but it's ok if you open the completion menu on demand.

[memeplex]

Ah... and it's not super fast in ipython also.

[davidhalter]

Ok. I think I'm closing, because the most serious issues have been fixed.

There are still slow libraries, but numpy is probably fine. pandas is a bit of a bigger problem and tensorflow... Well that one is just really slow. I have ideas how to fix it. If you want to file an issue, please use the jedi issue tracker and not this one.

[sobfiggis]

Just confirming that autocomplete for those libraries load faster. Not insanely fast due to their size, but it's much faster then the video I linked above.

[jogardi]

I also had issues with those big libraries being slow to autocomplete. But for most autocompletes, vim's built in path and keyn completion methods work very well. So I set it up to just use path and keyn automatically as I type and then jedi doesn't activate unless I manually hit or type a dot.

The https://github.com/lifepillar/vim-mucomplete plugin helped me set this up. Here is the code I had to put in my vimrc.

call plug#begin('~/.vim/plugged')
........
Plug 'davidhalter/jedi-vim'
Plug 'lifepillar/vim-mucomplete'
call plug#end()

set completeopt+=longest,menuone,noinsert
let g:mucomplete#enable_auto_at_startup = 1
let g:mucomplete#completion_delay = 300
let g:mucomplete#chains = { 'python': ['path', 'keyn']}

If that still doesn't make it fast enough try the settings below,
let g:pymode_rope = 0
let g:jedi#show_call_signatures = "0"
let g:jedi#popup_on_dot = 0

[davidhalter]

The problem with vim's builtin library is that it just randomly executes code. It might not be what you want. (That's also how it's fast).

[jogardi]

I know iPython and jupyter lab completion works that way but the built in methods I'm using are very simple and don't execute Python code. I'm using Path and keyn. Path looks for paths on your filesystem. keyn looks for keywords in the file. I still rely on jedi vim for autocompleting anything that isn't already used or defined in my current buffer.