Vimtex: Problem with continuous compilation (macVim + Skim)

[AlekseiSmirnov]

Recently I occasionally updated the vimtex plugin and it seems that it doesn't work with skim anymore.
I use macVim + skim bundle to create latex docs at macOs.

Since one of the recent updates, text isn't updated anymore during continuous compilation even despite there is Compiler: success label. In order to see compiled text I have to pres \lv each time.

I'd be thankful if you have any ideas on how to fix that. Here is code snipper for Skim from my .vimrc file:

let g:vimtex_view_general_viewer
      \ = '/Applications/Skim.app/Contents/SharedSupport/displayline'
let g:vimtex_view_general_options = '-r @line @pdf @tex'

" This adds a callback hook that updates Skim after compilation
let g:vimtex_latexmk_callback_hooks = ['UpdateSkim']
function! UpdateSkim(status)
  if !a:status | return | endif

  let l:out = b:vimtex.out()
  let l:tex = expand('%:p')
  let l:cmd = [g:vimtex_view_general_viewer, '-r']
  if !empty(system('pgrep Skim'))
    call extend(l:cmd, ['-g'])
  endif
  if has('nvim')
    call jobstart(l:cmd + [line('.'), l:out, l:tex])
  elseif has('job')
    call job_start(l:cmd + [line('.'), l:out, l:tex])
  else
    call system(join(l:cmd + [line('.'), shellescape(l:out), shellescape(l:tex)], ' '))
  endif
endfunction

[lervag]

Try to use g:vimtex_compiler_callback_hooks. I've made some big changes
to the compiler layer. I think you should see it in the docs.

[AlekseiSmirnov]

Thanks for your help. Just replaced

let g:vimtex_latexmk_callback_hooks = ['UpdateSkim']

to

let g:vimtex_compiler_callback_hooks = ['UpdateSkim']

and evetything works fine again. Probably it worth to replace it in vimtex help section about skim as well.

[lervag]

Great, I'll update later today.

[clason]

Would it make sense to have vimtex provide the update function (and the standard options) in, e.g., view/skim.vim? Skim doesn't require as much of a backend as mupdf or zathura, but it still might be nice to not have to copy&paste the function into a .vimrc if there's nothing to customize.

[lervag]

I've added an initial version now. Please test. I'll update the docs when things work (in the meantime people shouldn't use this). Note: I don't own a Mac, so I rely on testing by you guys.

The point is that things should work with simply let g:vimtex_view_method = 'skim' and nothing else.

[clason]

Yes, works nicely both from MacVim (8.0.525) and terminal neovim (master), thanks! I will report should something break down the line. (It still doesn't automatically open the viewer after the first successful compilation unless Skim is already running, but that was already the case before and can be addressed via a quick <leader>lv.)

[lervag]

It should automatically open the viewer. Could you try to change line 64 of the skim.vim file to pgrep -f Skim? You could also do some echo-debugging to verify if this part is right. If I understand correctly, we should add -g only if there already is a running viewer.

[AlekseiSmirnov]

Here is wat I see if I only add let g:vimtex_view_method = 'skim'

vimtex warning: viewer skim does not exist!
- Please see :h g:vimtex_view_method

[clason]

That doesn't work either, unfortunately. The if clause there seems to be correct, but it only ever updates Skim if the file(!) is opened already, either manually or with lv. (Just having Skim open is not enough.)

I think that behavior started a while ago; unfortunately I don't remember when. It could also have something to do with my OSX version (10.12).

Feel free to ignore this for now, it's not particularly bothersome. If I get an idea what the problem could be, I'll post a new issue.

[lervag]

@AlekseiSmirnov Did you update vimtex?

[AlekseiSmirnov]

Just so you know, I tried to take code snippet from the help and it doesn't update skim, moreover, after few compiles I'm getting corrupted pdf that can't be open by any viewer. Reverted back to code snippet I used to have previously and it seems that it does the trick:

""set up default pdf viewer
let g:vimtex_view_general_viewer
      \ = '/Applications/Skim.app/Contents/SharedSupport/displayline'
let g:vimtex_view_general_options = '-r @line @pdf @tex'
"
" This adds a callback hook that updates Skim after compilation
let g:vimtex_compiler_callback_hooks = ['UpdateSkim']
function! UpdateSkim(status)
  if !a:status | return | endif

  let l:out = b:vimtex.out()
  let l:tex = expand('%:p')
  let l:cmd = [g:vimtex_view_general_viewer, '-r']
  if !empty(system('pgrep Skim'))
    call extend(l:cmd, ['-g'])
  endif
  if has('nvim')
    call jobstart(l:cmd + [line('.'), l:out, l:tex])
  elseif has('job')
    call job_start(l:cmd + [line('.'), l:out, l:tex])"let g:vimtex_view_general_viewer
  else"      \ = '/Applications/Skim.app/Contents/SharedSupport/displayline'
    call system(join(l:cmd + [line('.'), shellescape(l:out), shellescape(l:tex)], ' '))"let g:vimtex_view_general_options = '-r @line @pdf @tex'
  endif
endfunction

[AlekseiSmirnov]

yeah, sorry. Just updated vimtex. Now only have let g:vimtex_view_method = 'skim' in my .vimrc and skim is not updated automatically with this setting. Have to press \lv each time to see updates in pdf

[lervag]

Strange. I don't get it. This is more or less exactly what is in the new skim implementation I pushed today. In any case, the implementation is pretty straighforward. I'll let it be for now, and if we get a version that seems robust and good for more people, then I'll update the docs to suggest the new method instead of a customized general method.

[clason]

Just to make sure: for me, Skim does update automatically once the file is open even without pressing \lv. It just doesn't open Skim with the file by itself without \lv.

Also, note that the code in skim.vim in contrast to the snippet from the help (and yours) doesn't highlight the current line in Skim with a red dot after updating, so it might be easy to miss a small update. (I actually prefer the skim.vim behavior.)

[lervag]

Ok, I've tried to improve things slightly, but I am not sure if it works. Please test. The code should be even simpler to understand, I think.

Also, the VimtexInfo command should show some more relevant information.

[clason]

Thanks! Now it works with neovim, both with and without Skim open. (Although I personally find the strong highlight a bit distracting.)

Curiously, it doesn't work anymore with MacVim, even if Skim is opened (i.e., I now have the same issue as @AlekseiSmirnov).

\li says

viewer: Skim
    process: process 72745 dead

Everything else seems identical apart from the obvious, in particular the backend.

[lervag]

Yes, this is very strange. Perhaps you could try some alterations of the command string and report back why it doesn't work? Could it be the ...#shellescape-thing?

[clason]

It's exactly the same command string, which works if I execute it with ! -- either from (vim's) command-line or the skim.vim script in place of self.process. So it might in fact be tied in some way to the job start API changes. Weird.

[lervag]

Hmm. What happens if you comment out the elseif has('job') clause, i.e. depend only on the vimtex processes module (and old style :! / system())?

[clason]

Then it says vimtex warning: Can't run empty command.

Replacing
let self.process = job_start(l:cmd)
with
execute "!" l:cmd
works though (although not well, obviously).

[lervag]

Huh, strange. Could you echo the command before it is executed and report the output here? (Use echom, then :messages if the output dissappears immediately. You can also use sleep x (x is number of seconds).)

[clason]

Hmm, I don't get any output on this. Just to make sure, could you give me the exact code that should go into the else clause? (I don't know any vimscript, so this is poking blindly on my part.)

[lervag]

function! s:skim.compiler_callback(status) dict " {{{1
  if !a:status | return | endif

  if g:vimtex_view_use_temp_files
    call self.copy_files()
  endif

  let l:cmd = join([
        \ self.path,
        \ '-r',
        \ '-b' . (!empty(system('pgrep Skim')) ? ' -g' : ''),
        \ line('.'),
        \ vimtex#util#shellescape(self.out()),
        \ vimtex#util#shellescape(expand('%:p'))
        \])

  " You can also try this (without the shellescapes)
" let l:cmd = join([
"       \ self.path,
"       \ '-r',
"       \ '-b' . (!empty(system('pgrep Skim')) ? ' -g' : ''),
"       \ line('.'),
"       \ self.out(),
"       \ expand('%:p')
"       \])

  if has('nvim')
    let self.process = jobstart(l:cmd)
"  elseif has('job')
"    let self.process = job_start(l:cmd)
  else
    let self.process = vimtex#process#start(join(l:cmd))
    echom join(l:cmd)
  endif
endfunction

[clason]

With this, I just see 0 in the status line and the :messages list... (The shell escapes make no difference.)

[lervag]

Hmm. Try to remove the join from join(l:cmd).

[clason]

Yep, I was just doing that -- then I get the correct command.

[lervag]

Great! So, let's get back to the job part. What happens if you remove the join from the let l:cmd part? Does the job_start(l:cmd) work then?

[clason]

Yes, exactly, with or without the file already opened.

[lervag]

Ok, does this mean we have a solution that works with all three executioners? If so, could you post the snippet here?

[clason]

So from what I could test:

1. The nvim branch works in neovim master as is (let self.process = jobstart(l:cmd))

2. The jobs branch doesn't seem to work on MacVim 8.0.525 (which reports +job)

3. let self.process = vimtex#process#start(l:cmd) does work on MacVim 8.0.525

I couldn't test any other combination. Still a bit confused why it doesn't work.

[lervag]

Ok, could you test nvim and jobs with the newest version that I just pushed?

[lervag]

Btw: I think the third branch should work in all cases, also for neovim. One possibility is to just use this always. I can't really see a good reason to use the job API for this, because we don't really want to control the process, only start it. Thus, we could perhaps fix all problems simply by changing:

  if has('nvim')
    let self.process = jobstart(l:cmd)
  elseif has('job')
    let self.process = job_start(l:cmd)
  else
    let self.process = vimtex#process#start(join(l:cmd))
  endif

to

    let self.process = vimtex#process#start(join(l:cmd))

Could you test if that works for you with both neovim and MacVim?

[clason]

With the fix you pushed, neither neovim nor MacVim work...

let l:cmd = join([
          \ self.path,
          \ '-r' . (!empty(system('pgrep Skim')) ? ' -g' : ''),
          \ line('.'),
          \ vimtex#util#shellescape(self.out()),
          \ vimtex#util#shellescape(expand('%:p'))
          \])

let self.process = vimtex#process#start(l:cmd)

does work for me on both neovim and MacVim. (I took the liberty of disabling the reading bar; I find the red dot enough.)

I think there might be something to do with the options for job_start, but I agree with you that job control is overkill for this specific use case.

[lervag]

Ok, then we'll use this for now. I've pushed the change. Let me know if there is more that can be done to make it work better.

[clason]

Yes, that seems to work fine now. Thank you!

[lervag]

No problem, and thank you for your patience and help in testing! :)

[lervag]

@AlekseiSmirnov I realize it's too much to read through for you now, but could you report whether the updated version works for you now?

[AlekseiSmirnov]

Just checked the last version. Now it's better and even work with skim using 1 line in .vimrc but still have some issues. Sometimes the text is not updated in pdf after compilation. Please check this gif for clarity

[AlekseiSmirnov]

Regarding the problem with the line number - it seems to be shifted by number of lines in document header

[lervag]

@AlekseiSmirnov Thanks for the gif, a nice way to explain. However, just to be sure: The document is only compiled when you save the file, right? It seemed you either saved very often, or that you had some kind of autowrite enabled that started compilation more often than I would expect.

As said before, it is very difficult for me to help with debugging/developing for OSX, as I don't have any means of testing. Thus I would very much appreciate if you could try to look at the source file and find a more optimal implementation yourself.

Of course, you may just use the general method that seems to work as expected, but in principle, I don't see any difference between what I've implemented and what you've provided as the Skim settings for the general viewer. :\

[AlekseiSmirnov]

I disabled autowrite and still have this problem even if I save document manually. Each time document is not updated I see firstly Compiler: Failed and a few seconds later Compiler: Success. Output pdf is corrupted, so I even can't open it with any outer pdf reader

Is there any way to see verbose compiler logs that could help to figure out what's going on?

[lervag]

Strange. I was also curious if this happens with the old customization of the general viewer, or if this only happens with the new skim viewer?

Perhaps you could provide a minimal vimrc file that reproduces your problem?

There are two logs/outputs to view: VimtexInfo will show you vimtex state data, which might be helpful. Also, VimtexOutput will show you the latexmk output. I don't think neither of these will be enough to pinpoint the problem, though.

[AlekseiSmirnov]

When I enter VimtexOutput command I'm getting

E492: Not an editor command: VimtexOutput

Tried to reproduce on some simple example and all in vain. Finally checked all the warnings and experimentally found that sometimes corrupted pdf were produced due to bad citation. There was the following warning:

|76 warning| LaTeX Warning: Citation `shiryaev1' on page 2 undefined on input line 76.
|| LaTeX Warning: There were undefined references.
|| LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.

Removed this citation and now it seems to be compiled well, at least without corrupted pdfs, despite
I still see Compiler: Fail and then Compiler: Success

UPD: noticed that if I compile from main file I always see Compiler: Success so I guess the problem above is somehow connected with subfiles

[lervag]

Ok, first: It seems your particular case is very complex, and thus it is almost impossible for me to know what could be wrong. This is _exactly_ why I ask people to create minimal examples: It helps me to reproduce the issue, but it also helps users to actually isolate/locate the problem.

My conclusion so far is that there is nothing really wrong with vimtex, at least nothing that I can reproduce.

Please don't mistake this post as hostile, that is not intended! If you should be able to provide a simple example that I can reproduce, or somehow provide information that we can use to locate an issue, then we'll continue work on improving vimtex. I will be happy to help!

[AlekseiSmirnov]

Of course, I understand. This case is indeed unusual and hard to reproduce.
I'm still investigating the issue and will be back once I have more info