Vim-plug: Eliminate plug#begin(), plug#end()

[justinmk]

What are the technical reasons that require the plug#begin/plug#end dance? After a cursory look at the code, here are some reasons for the current approach, with possible solutions:

1. plug#begin() is needed because vim-plug is installed to autoload/
   
   
   
   • Solution: install to plugin/ instead. vim-plug always runs on startup, so autoload/ gains nothing.
   
   
2. plug#begin() is needed to initialize vim-plug
   
   
   
   • Solution: This could be done in the first call to :Plug, and cached.
   
   
3. plug#end() is needed to call syntax enable
   
   
   
   • Could be done on VimEnter instead. Or in the case of Neovim, don't do it at all.
     
     
     
     
     
     • This causes confusion with some configs: https://github.com/neovim/neovim/issues/4684
     
     
     
   
   
4. plug#end() is needed to reorder 'rtp' and do some initialization
   
   
   
   • The could be done incrementally in each :Plug call, and wouldn't be slower because plug#end() is doing a "big loop" anyways.
   
   

Probably I'm missing something (which is why I created this issue).

[junegunn]

Thanks, you made some good points, I would also like vim-plug to require less ceremony.

install to plugin/ instead

Scripts in plugin directory are loaded after vimrc, so we would still need a line for loading plug.vim before the first appearance of Plug, like runtime plugin/plug.vim, and I can't say that it's aesthetically better or more straightforward to the users than call plug#begin(), which has the added benefit of allowing custom plugin directory in one line without having to resort to let g: something.

vim-plug was modeled after vundle, and admittedly it was when I had a limited understanding of plugin mechanism of Vim. The primary reason I introduced begin/end scheme was to abstract away filetype off and filetype on around the plugin declarations Vundle still requires on its documentation to this day. I haven't checked if the toggling is still required on the latest versions of Vim, but anyway I didn't like the fact that the users had to put those lines without fully understanding why they are required. So I thought it was a good idea to provide higher-level API that hides such gory details.

After some time, Vundle also decided to use begin/end pairs (https://github.com/VundleVim/Vundle.vim/pull/418#issuecomment-39647638), not because of this as they still require filetype off, but mainly because of performance overhead of incremental update of &runtimepath.

But it was three years ago, maybe it's time that we reassess the performance benefit of one-time update of &rtp on modern systems.

If we put aside the idea of incremental &rtp update, one way to avoid explicit plug#end is to fire it on VimEnter if plug#begin() was called but plug#end() was not at the point, which is actually a common mistake of newcomers to vim-plug.

[vheon]

one way to avoid explicit plug#end is to fire it on VimEnter if plug#begin()

But by the time we fire the VimEnter, plugins are already loaded, which means that we should manually reload all plugin at that point.

[justinmk]

Scripts in plugin directory are loaded after vimrc, so we would still need a line for loading plug.vim before the first appearance of Plug

So that issue could be resolved by having a built-in :plug command (which could be served by different plug managers via e.g. `'pluginmanager' option or whatever).

maybe it's time that we reassess the performance benefit of one-time update of &rtp on modern systems.

Is the performance concern mostly about re-scanning the runtimepath on the filesystem? Or something else?

But by the time we fire the VimEnter, plugins are already loaded, which means that we should manually reload all plugin at that point.

@vheon Not "reloaded", but loaded, right? That's when the 'rtp' would (could) be modified and :runtime applied.

[vheon]

@vheon Not "reloaded", but loaded, right? That's when the 'rtp' would (could) be modified and :runtime applied.

@justinmk yeah, sorry. I meant that we should explicitly tell vim to runtime the plugins that we just added, while right now we just modify rtp and then vim do the works on its own.

[junegunn]

@vheon Good point, you're right. I was confused.

@justinmk I wasn't concerned about performance when I introduced begin-end scheme, so I don't have any data. You might want to read through the discussion in the linked Vundle issue.

Incremental update of &rtp can actually be a bit tricky as we can't just += or ^= due to ordering issues.

• head,...,tail
  
  
  
  • See #34
  
  
• head,p1,...,p1/after,tail
• head,p1,p2,...,p1/after,p2/after,tail
• head,p1,p2,p3,...,p1/after,p2/after,p3/after,tail

So I wouldn't be surprised if it adds a few milliseconds if there are like 100+ plugins.

[vheon]

@junegunn well if you go the VimEnter route then you could call the plug#end() function in the VimEnter handler. That way you would be sure that plug#end() will be called at the right time (?) and you would not have the the performance and problem that incremental rtp manipulation would bring.

[mgutz]

Could it be simplified to plug#load('~/.vim/Plugfile'), where Plugfile is a manifest file like

Plug 'fatih/vim-go'

This is basically what I do now except I source it. FWIW, seems like a manifest file say in JSON/YAML makes more sense than writing vimscript. That's how most package managers work.

[justinmk]

Redirection isn't simplification.

[chreekat]

I don't have a horse in this race, but @mgutz' solution could get rid of the redirection with

call plug#load(
    '~/.vim/bundle',
    ['tpope/vim-fugitive'
    ,{ 'plug': 'mbbill/undotree', 'on': 'UndotreeToggle' }
    ]
)

[chreekat]

Lol, I forgot I'd need a bunch of backslashes, how ugly.

function! PlugLoad(loc, pluglist)
    call plug#begin(a:loc)
    for p in a:pluglist
        if type(p) == v:t_string
            let cmd = "Plug '" . p . "'"
        else
            let pname = remove(p, 'plug')
            let cmd = "Plug '" . pname . "', " . string(p)
        endif
        exec cmd
    endfor
    call plug#end()
endfunction

call PlugLoad('~/.vim/bundle',
    \[{'plug': 'chrisbra/NrrwRgn', 'on': 'NR' }
    \,{'plug': 'junegunn/goyo.vim', 'on': 'Goyo' }
    \,{'plug': 'mbbill/undotree', 'on': 'UndotreeToggle' }
    \,'tpope/vim-abolish'
    \,'tpope/vim-eunuch'
    \,'tpope/vim-flagship'
    \,'tpope/vim-fugitive'
    \,'ultimatecoder/goyo-doc'
    \,'~/LoByMyHand/vim-simple-md'
    \,'~/LoByMyHand/passhole'
    \,'~/LoByMyHand/vimin'
    \,'vim-scripts/VisIncr'
    \,'~/src/Haskell/intero/vim'
    \])

[SevereOverfl0w]

I've had a go at this today, it's very possible I've done a bad job of it, but despite my best efforts, on my [email protected] I've added 15-20ms to plug-all.vim and 10ms to plug.vim (from vim-startup-benchmark).

Before:

After:

A lot of time is spent on setting the 'rtp' variable.

[agriffis]

I'm interested in the outcome of this issue for a reason that I don't think has been mentioned. With the existing approach, it's easy to get confused about the order in which configuration will be applied. For example:

call plug#begin('~/.vim/plugged')
Plug 'one/vim-foo'
autocmd FileType javascript setl comments=s1:/*,mb:*ex:*/,://
call plug#end()

Instinctively, it seems like the autocmd will register between the first and second plugs. But in fact nothing has happened yet, so any autocmds that one/vim-foo registers will run later.

It's not a big deal in the toy example above. In a large configuration, it results in a split brain between "here are the plugins I load for javascript" and later on, "here is my configuration for javascript." It's so much nicer if these things can be colocated.

So I'm in favor of incremental :Plug even if it slows things down slightly for &rtp shuffling.

[agriffis]

Also, @junegunn said:

Scripts in plugin directory are loaded after vimrc

Would it be reasonable for vim/nvim to add another directory, something like early which loads before vimrc? Then :Plug could be available when vimrc is read

[SevereOverfl0w]

@agriffis My patches allow me to load & configure plugins incrementally.

Putting my patch here in case I lose it, and for critique of what it breaks:

diff --git a/plug.vim b/plug.vim
index 52c6b2a..3558442 100644
--- a/plug.vim
+++ b/plug.vim
@@ -193,29 +193,26 @@ function! s:ask_no_interrupt(...)
   endtry
 endfunction

-function! plug#end()
+function! plug#increment(name)
   if !exists('g:plugs')
     return s:err('Call plug#begin() first')
   endif

-  if exists('#PlugLOD')
-    augroup PlugLOD
-      autocmd!
-    augroup END
-    augroup! PlugLOD
-  endif
   let lod = { 'ft': {}, 'map': {}, 'cmd': {} }

   if exists('g:did_load_filetypes')
     filetype off
+    let g:plug_filetype_off = 1
   endif
-  for name in g:plugs_order
+
+  for name in [a:name]
     if !has_key(g:plugs, name)
       continue
     endif
     let plug = g:plugs[name]
     if get(s:loaded, name, 0) || !has_key(plug, 'on') && !has_key(plug, 'for')
       let s:loaded[name] = 1
+      call plug#s_increment_rtp(a:name)
       continue
     endif

@@ -274,8 +271,101 @@ function! plug#end()
             \ ft, string(ft), string(names))
     augroup END
   endfor
+  " filetype plugin indent on
+  " if has('vim_starting')
+  "   if has('syntax') && !exists('g:syntax_on')
+  "     syntax enable
+  "   end
+  " else
+  "   call s:reload_plugins()
+  " endif
+endfunction

-  call s:reorg_rtp()
+function! plug#end()
+  if !exists('g:plugs')
+    return s:err('Call plug#begin() first')
+  endif
+
+  if exists('#PlugLOD')
+    augroup PlugLOD
+      autocmd!
+    augroup END
+    augroup! PlugLOD
+  endif
+  " let lod = { 'ft': {}, 'map': {}, 'cmd': {} }
+
+  " if exists('g:did_load_filetypes')
+  "   filetype off
+  " endif
+
+  " for name in g:plugs_order
+  "   if !has_key(g:plugs, name)
+  "     continue
+  "   endif
+  "   let plug = g:plugs[name]
+  "   if get(s:loaded, name, 0) || !has_key(plug, 'on') && !has_key(plug, 'for')
+  "     let s:loaded[name] = 1
+  "     continue
+  "   endif
+
+  "   if has_key(plug, 'on')
+  "     let s:triggers[name] = { 'map': [], 'cmd': [] }
+  "     for cmd in s:to_a(plug.on)
+  "       if cmd =~? '^<Plug>.\+'
+  "         if empty(mapcheck(cmd)) && empty(mapcheck(cmd, 'i'))
+  "           call s:assoc(lod.map, cmd, name)
+  "         endif
+  "         call add(s:triggers[name].map, cmd)
+  "       elseif cmd =~# '^[A-Z]'
+  "         let cmd = substitute(cmd, '!*$', '', '')
+  "         if exists(':'.cmd) != 2
+  "           call s:assoc(lod.cmd, cmd, name)
+  "         endif
+  "         call add(s:triggers[name].cmd, cmd)
+  "       else
+  "         call s:err('Invalid `on` option: '.cmd.
+  "         \ '. Should start with an uppercase letter or `<Plug>`.')
+  "       endif
+  "     endfor
+  "   endif
+
+  "   if has_key(plug, 'for')
+  "     let types = s:to_a(plug.for)
+  "     if !empty(types)
+  "       augroup filetypedetect
+  "       call s:source(s:rtp(plug), 'ftdetect/**/*.vim', 'after/ftdetect/**/*.vim')
+  "       augroup END
+  "     endif
+  "     for type in types
+  "       call s:assoc(lod.ft, type, name)
+  "     endfor
+  "   endif
+  " endfor
+
+  " for [cmd, names] in items(lod.cmd)
+  "   execute printf(
+  "   \ 'command! -nargs=* -range -bang -complete=file %s call s:lod_cmd(%s, "<bang>", <line1>, <line2>, <q-args>, %s)',
+  "   \ cmd, string(cmd), string(names))
+  " endfor
+
+  " for [map, names] in items(lod.map)
+  "   for [mode, map_prefix, key_prefix] in
+  "         \ [['i', '<C-O>', ''], ['n', '', ''], ['v', '', 'gv'], ['o', '', '']]
+  "     execute printf(
+  "     \ '%snoremap <silent> %s %s:<C-U>call <SID>lod_map(%s, %s, %s, "%s")<CR>',
+  "     \ mode, map, map_prefix, string(map), string(names), mode != 'i', key_prefix)
+  "   endfor
+  " endfor
+
+  " for [ft, names] in items(lod.ft)
+  "   augroup PlugLOD
+  "     execute printf('autocmd FileType %s call <SID>lod_ft(%s, %s)',
+  "           \ ft, string(ft), string(names))
+  "   augroup END
+  " endfor
+
+  " call s:reorg_rtp()
+"
   filetype plugin indent on
   if has('vim_starting')
     if has('syntax') && !exists('g:syntax_on')
@@ -418,6 +508,40 @@ function! s:reorg_rtp()
   endif
 endfunction

+function! plug#s_increment_rtp(loaded_name)
+  if !empty(s:first_rtp)
+    execute 'set rtp-='.s:first_rtp
+    execute 'set rtp-='.s:last_rtp
+  endif
+
+  " &rtp is modified from outside
+  if exists('s:prtp') && s:prtp !=# &rtp
+    call s:remove_rtp()
+    unlet! s:middle
+  endif
+
+  let s:middle = get(s:, 'middle', &rtp)
+  let s:rtps = get(s:, 'rtps', '')
+  let new_rtp     = s:rtp(g:plugs[a:loaded_name])
+  let s:afters = get(s:, 'afters', '')
+  let new_afters = globpath(new_rtp, "after", 0, 1)
+
+  let s:rtps .= "," . escape(new_rtp, ",")
+  let s:afters .= "," . join(map(new_afters, 'escape(v:val, ",")'), ',')
+
+  let rtp      = s:rtps
+                 \ . ','.s:middle.','
+                 \ . s:afters
+  let rtp_l = substitute(substitute(rtp, ',,*', ',', 'g'), '^,\|,$', '', 'g')
+  let s:prtp   = rtp_l
+
+  if !empty(s:first_rtp)
+    let &rtp = s:first_rtp . ',' . rtp_l . ',' . s:last_rtp
+  else
+    let &rtp = rtp_l
+  endif
+endfunction
+
 function! s:doautocmd(...)
   if exists('#'.join(a:000, '#'))
     execute 'doautocmd' ((v:version > 703 || has('patch442')) ? '<nomodeline>' : '') join(a:000)
@@ -553,6 +677,7 @@ function! plug#(repo, ...)
     endif
     let g:plugs[name] = spec
     let s:loaded[name] = get(s:loaded, name, 0)
+    call plug#increment(name)
   catch
     return s:err(v:exception)
   endtry

[agriffis]

@SevereOverfl0w Thanks, I think you should just make an RFC PR instead of putting the patch here. And delete the old code instead of commenting it... it will be clearer in the PR diff.

[chemzqm]

Compare to several milliseconds differences above, let me tell what really would make your neovim slow.

https://github.com/neovim/neovim/blob/master/runtime/autoload/provider.vim#L10

The rpcrequest call here would cause about 150ms for each remote plugin host on my mac (2.6 GHz Intel Core i7)

https://github.com/neovim/neovim/blob/9afed40ea684127056bcd5019533522aa67cbb57/runtime/autoload/provider/node.vim#L31

The system command like this which trying to find npm global module folder would cause more than 200ms on my mac.

It's reasonable that lazy load would make vim slower sometimes since neovim already doing some lazyload stuff for remote plugins(including the python ones).