Vimtex: Smarter handling of package syntax rules

[lervag]

Some months ago I improved (IMHO) the way Vimtex loads syntax rules. In particular, since syntax highlighting is expensive in Vim, I ensured that we don't load package related rules that are not relevant (cf. #1433). This works by parsing the loaded packages, either by looking at the preamble or by parsing the .fls file if it is available.

This does work the way it is intended. However, there have been several issues related to this because a lot of packages are loaded implicitly. So the syntax rules are only loaded when the .fls file is available. E.g., see #1545, where mathtools is loaded which implicitly loads amsmath. The syntax rules for amsmath related environments such as gather will only be loaded if the document is compiled and the .fls file is available.

So: I'm wondering if there is a good way to make things more intuitive. I'm thinking about a couple of alternatives:

1. An option to avoid the package guards, i.e. always load all syntax rules. Perhaps a lot of people will not care that this might be inefficient. This now exists, see :help g:vimtex_syntax_autoload_packages and #1680.

2. Add a flexible way to load syntax rules after the buffer is opened and after the document is compiled, perhaps automatically. Something like this:

   • Add method to reparse the list of loaded packages.
   • Add method to (re)load syntax rules at will.
   • Perhaps: Add hook after compilation for reloading syntax rules. Do this so that it does not reload syntax rules; only add rules for packages that were added.

I think alternative 2 is a viable solution that should be possible to implement. I also think that the compilation hook would allow this to work so that it would dynamically load the syntax rules of packages that are "discovered" when the .fls file becomes available. However, I am unsure of how interesting this feature is. For me, things work well now.

Other related issues: #1428, #1471, and more.

I'll be glad to get some opinions. I'm therefore tagging names from the related issues: @Konfekt @927589452 @ruifm @kiryph @clason @juw

[927589452]

What about extending the loading for now by adding a few package specific rules, eg.

• mathtools implicates amsmath
• glossaries-extra implicates glossaries

[clason]

@lervag First of all, will <leader>lx retrigger parsing as well? I use this as a workaround for adding mode lines, and it might be "offically" recommended as a way around. (I have a sense of déja vu here -- it's not possible for lx to stop any running compilation first, right? That's the only thing that makes it an "unsafe" advanced option in my experience.)

Otherwise your program 2 seems like the right way to do this properly, but I also like @927589452's lazy option. Maybe it would make sense to see how many packages this affects anyway? Otherwise it might be overkill.

We could make a poll here, but I'm skeptical that this would give a good coverage. Maybe adding the named rules (plus possibly a biblatex* regex) and waiting for further issues to switch to the "full solution"?

(This is assuming the rules wouldn't have much performance overhead; that might be wrong, of course.)

[lervag]

Sorry for the very late reply!

What about extending the loading for now by adding a few package specific rules, eg.

• mathtools implicates amsmath
• glossaries-extra implicates glossaries

Good idea! But: there are no rules for glossaries, nor glossaries-extra. I'll add dummy files for them in case there should ever be something, though.

---

I'm currently looking into a way to load missing package syntax on compiler callbacks. I think that should work and it should solve this issue.

[lervag]

I think that's it. It works on my simple tests, at least. Feel free to test and let me know how it works for you.

[lervag]

@clason Sorry, I forgot to respond to your comment!

Otherwise your program 2 seems like the right way to do this properly, but I also like @927589452's lazy option. Maybe it would make sense to see how many packages this affects anyway? Otherwise it might be overkill.

I've recently added the option g:vimtex_syntax_autoload_packages (see #1680), which allows to specify syntax packages that are always loaded. In addition, I've also added some "lazy" stuff like mathtools loading amsmath and similar. I find it overkill to add too much of this type, though.

So, this means I've added both of my originally suggested points, and I think this part of vimtex now works quite well!

@lervag First of all, will <leader>lx retrigger parsing as well? I use this as a workaround for adding mode lines, and it might be "offically" recommended as a way around. (I have a sense of déja vu here -- it's not possible for lx to stop any running compilation first, right? That's the only thing that makes it an "unsafe" advanced option in my experience.)

Good question... I found a minor bug in the reload function. When I fixed it, it seems like \lx should reload the syntax as expected. Feel free to open issues regarding this feature as well. I find it very useful sometimes, and I think it should work mostly as expected. Not quite sure what you mean with "it's not possible for lx to stop any running compilation first, right?" - that is, I think it does stop running compilations first, as long as they were started by vimtex.

[clason]

@clason Sorry, I forgot to respond to your comment!

No worries! Most of it was tangential anyway.

So, this means I've added both of my originally suggested points, and I think this part of vimtex now works quite well!

Agreed! 👍

Good question... I found a minor bug in the reload function. When I fixed it, it seems like \lx should reload the syntax as expected. Feel free to open issues regarding this feature as well. I find it very useful sometimes, and I think it should work mostly as expected. Not quite sure what you mean with "it's not possible for lx to stop any running compilation first, right?" - that is, I think it does stop running compilations first, as long as they were started by vimtex.

I think it didn't, at least some of the times, when I tried it (the latexmk process kept running in the background, so starting compilation again lead to the expected problems). It's not critical, though; I'll keep an eye out and will raise an issue when it crops up (and I can reproduce it) again!

[lervag]

I'll keep an eye out and will raise an issue when it crops up (and I can reproduce it) again!

Great, thanks! :)