Syntastic: Consider relying on vim-dispatch

[Diablo-D3]

tpope has a new vim plugin out for async jobs. Watch the introductory video.

http://vimeo.com/63116209

https://github.com/tpope/vim-dispatch#readme

[lcd047]

@scrooloose is still missing in action, but I'm pretty sure he mentioned vim-dispatch in the past. So yes, we're aware of it.

My personal take is that vim-dispatch is nice and very well written, but it only supports quickfix lists, while syntastic needs loclists. An older patch adding support for async checking (see #370) uses AsyncCommand instead, which can handle both quickfix lists and loclists. I'd still prefer vim-dispatch, mainly because the code is cleaner. But for that to happen we'd have to either persuade @tpope to support loclists, or change syntastic to use quickfix lists (which in turn would make it conflict with :make and friends), or do some ugly dance to bridge quickfix lists with loclists (and still conflict with :make).

Well, we should wait for @scrooloose to get back.

[tpope]

I don't really see these two plugins as a great fit together. Can you imagine a tmux pane opening for a split second every time you write a file? Gross.

[lcd047]

If it happens in background, why not? :)

[tpope]

You mean other than two resizes and a potential view port shift?

[lcd047]

Ah right, in tmux parlance "panes" are equivalents to Vim's windows. Ok, would that work better if the checkers would be sent to run in a background tmux window (equivalent to a background tab in Vim parlance)?

[tpope]

Dispatch relies on the resize event to trigger the callback. Without that, there's no point in bothering with tmux at all.

[justinmk]

What about vimproc? https://github.com/Shougo/vimproc.vim

[lcd047]

@justinmk After scratching my head about it for a while, I don't see any useful way to interface vimproc with syntastic. Assuming I decrypted it properly, vimproc's main use of backgrounding is about running several things in parallel and polling for results. For syntastic we'd need a mechanism to run a single process in background, and get an event when the process is done. These are pretty different use cases.

[lcd047]

On a related topic: I played a little with AsyncCommand, and it isn't very robust. It appears to leave background processes behind, which can happily eat 99% of CPU cycles. And I wasn't even _trying_ to crash it. :)

[justinmk]

@lcd047 I'm surprised that vimproc doesn't fit this use case, but I haven't looked closely at it. AFAIK, there is no way to get passively notified of an external event in vim (without --remote which requires gvim). But using vim's &updatetime and CursorHold/CursorHoldI have been used by other plugins to reasonable effect. Why wouldn't that work for syntastic?

FWIW, I just bumped into another option: xolox's vim-misc combined with vim-shell. It is used by easytags to provided asynchronous behavior. See xolox#misc#os#exec(options) for usage of the vim-shell asynchronous capability.

[lcd047]

Well, vim-dispatch mentioned above uses tmux to send Vim a SIGWINCH when a background command finishes. Also, I believe --remote requires +clientserver, not a GUI.

As for CursorHold, I know about people using it in creative ways to achieve async effects, but I never bothered to find out exactly how that works. If it's possible to emulate a poll loop that way, I'm not familiar with it. The burden of proof is on you, not me. :)

Anyway, personally I'm not going to waste any more brain cycles on this. Async make should be implemented in Vim, not emulated by some horrible contortions in third-party plugins. It's 2013, job control has been understood really, really well by now. Patching Vim to do that would be relatively straightforward --- except qf_init_ext() is ~700 lines long, and the chances of such a patch becoming official any time soon seem pretty slim. And, last but not least, I have exactly zero use for all this, I'm pretty happy with the sync checks. _shrug_

[justinmk]

@lcd047 fair enough, I mostly agree.

For posterity: --remote does appear to require a GUI: https://code.google.com/p/macvim/issues/detail?id=431

[lcd047]

@justinmk: Apparently it requires either OLE (that is, Windows), or an X-like clipboard (UNIX, Mac OS, or VMS). This means Vim needs to run in a (specific) GUI, but Vim itself doesn't need to be compiled with GUI support. I'm using this:

VIM - Vi IMproved 7.4b BETA (2013 Jul 28, compiled Jul 29 2013 09:10:27)
Compiled by root@euler
Huge version without GUI.  Features included (+) or not (-):

It can do remote commands as long as both the server and the client run in xterms, but not in console.

[justinmk]

@lcd047 Very interesting! Good to know. It _does_ work on Windows using any combination of {vim.exe, gvim.exe}.

Well, since it appears this github issue has become an accidental comprehensive survey of the state of "vim async support", perhaps it will save others the trouble in the future.

[maksimr]

+1 Need async for javascript :)

[idbrii]

@lcd047 : Any chance you could file a bug to AsyncCommand for how you produced abandoned background processes ("which can happily eat 99% of CPU cycles.")? AsyncCommand doesn't do any process killing, so if you do something like AsyncShell yes, then it will run forever (but isn't that what you'd expect)?

---

On topic:

Assuming I decrypted it properly, vimproc's main use of backgrounding is about running several things in parallel and polling for results. For syntastic we'd need a mechanism to run a single process in background, and get an event when the process is done. These are pretty different use cases.

Without using something like AsyncCommand (where you use --remote to send a command to vim), I don't think you can send an event, without polling for completion. So essentially, those are the same use cases except syntastic only fires one job. (Unless you mean vimproc blocks until all parallel jobs complete?)

I can't find the SIGWINCH call @lcd047 mentioned, but dispatch#callback is also using --remote. Maybe I'm missing something.

@tpope Wouldn't using syntastic and vim-dispatch's headless strategy make sense? I guess that would require adding a way to force a specific strategy to vim-dispatch? That seems like something that would be generally useful for other plugins to depend on too. (You could use it for an auto-exec repl, doc or tags lookup, and of course syntax checking.) Anywhere the user wants parsed output and not raw output.

[justinmk]

@pydave I would keep an eye on tarruda's Vim fork to see where it goes. It's showing more promise than the floobits "timers" fork from October.

[lcd047]

Any chance you could file a bug to AsyncCommand for how you produced abandoned background processes ("which can happily eat 99% of CPU cycles.")?

Vanishingly small chances. As I said above, this proved to be a huge time sink, and I'm not going to waste any more brain cycles on it. If you care about it, I suppose you can retrieve the relevant setup in #370 and do your own testing. If you revive that patch (or if you come up with some other working solution), people test it extensively, and it proves to be stable, then I'll gladly include it. Until then, I'd rather spend my time doing something else.

I can't find the SIGWINCH call @lcd047 mentioned

The term to look for is VimResized. SIGWINCH is the signal that makes it possible.

[mitchmindtree]

Hey guys, any further word on a solid solution for async checking? If not, what is the best (/ least terrible) 3rd party solution that you've come across? I've recently been converted from Xcode to Vim land (a _very_ liberating move), yet the one thing I miss a lot is Xcode's excellent multi-file async checking. Syntastic is already incredible, but I just wanted to add another "with async it would be totally awesome!". I hope the passion for this feature hasn't died yet :-) Or even better, if this has already been addressed with a recent feature that I've somehow managed to miss, let me know!

[lcd047]

If you're happy with the set of languages it supports, YouCompleteMe is pretty good at what it does. And if you're not married to Vim, InteliJ IDEA, PyCharm, and friends are rather useful, too.

[gilligan]

@lcd047 Sorry for reviving this but I was wondering if you are following neovim development and if you can say to what degree the async stuff they have included will be of use for syntastic at some point ?

[lcd047]

Sorry, I haven't followed neovim development recently. I'll probably look at it again when / if they actually produce some kind of release.

[s0undt3ch]

A look for the support if syntastic background checking led me to this ticket. Support for this would make syntastic even awesome'er!

[idbrii]

Awhile back I found a fork of syntastic that tried to use idbrii/AsyncCommand to run syntax checking in the background. The initial and only commit stgpetrovic/syntastic-async@b3e3a9ba22b54aac6483e3b5ea545363d10fb304 (from a year ago) says:

First version that is workable with.

Will improve everything this is just the initial hack.

@s0undt3ch: if you're interested, you could try figuring out the problems with it. (Or whether it could work with vim-dispatch.)

(I haven't had time to look into VimResized that lcd047 mentioned above or actually mess with syntastic-async myself.)

[lcd047]

I suppose I should update you guys on this. The short version is, it should be possible to make syntastic do async checks using vimproc alone, but there would be a lot of work involved, most checkers would have to be refactored to take advantage of it, and the result is likely to be pretty fragile. Also speed would likely suck for some checkers, despite them being run asynchronously. A longer version follows, if you care about that kind of details.

The discovery process went something like this: vimlint pointed me to vim-watchdogs, which does async checks. Its approach is very different from syntastic, so the trick there can't be applied directly to syntastic; however, it pointed me to vim-quickrun, which is essentially a higher level frontend to vimproc. This again isn't directly useful for syntastic, but digging through the sources reveals how to do polling for an external event from Vim, using CursorHold and feeding it invisible keystrokes with feedkeys(). So it would be possible (at least in principle) to run the checkers with vimproc, poll for results, and trigger syntastic notifications accordingly. This seems better than all other options listed above, for many technical reasons.

Now there are two main problems with this approach. First, we'd need an entire framework for dealing with tasks, processes, and queues. This can be done, but it's a lot of work, and the documentation for vimproc is less than useful.

Second, one can only run _processes_ in background, not Vim functions. Since the results of most checkers are further munged by Vim before being put in the final loclist, dealing with this would involve a major rewrite of the entire checking mechanism. Also, having a single *_GetLocList() function per checker would no longer be enough, it would have to be split into a function that would submit the checker command line to vimproc, and another function that would post-process the results. Furthermore, this post-processing stage would be run in foreground (since it would be a Vim function, not a process), in a CursorHold handler. The same CursorHold handler would also be responsible for task bookkeeping. That is likely to suck, badly. A possible solution to speed things up would be to do the post-processing in something like Python instead of VimL, but that would mean re-writing all checkers to use Python, which would suck even worse, albeit in a different way.

Last but not least: using feedkeys() is almost guaranteed to be fragile. If you're a fast typist and you already run into Vim occasionally misreading some key combinations while you type, feedkeys() would complicate this problem for you exponentially. _shrug_

[spacepluk]

I mess up the screen pretty often trying to do stuff while syntastic is working. If there's a way to prevent that from happening maybe it's not such a big deal if the checkers are not async... just a thought.

[lcd047]

@spacepluk As I explained in #822, this is a bug in your environment, and syntastic has absolutely no control over it.

[spacepluk]

@lcd047 thanks, that's very useful info.

[q0LoCo]

I write an async version using clientserver feature.
if anyone has interest, you can checkout my fork.

[treyhunner]

@q0LoCo I tried your async branch and it seems to work with flake8 for Python but not with ESLint for JavaScript. I'd open an issue there but you don't have the issue tracker enabled on your repository.

[q0LoCo]

@treyhunner it's opened now.

[acornejo]

since syntactic relies on external program's to check syntax, what is the main hurdle in doing something like this (oversimplifying, but want to understand if this approach is fundamentally broken):

1. run syntax checker external command forking it to the background and redirecting the output to to a file.
2. use cursorhold to trigger a periodicvim function which checks if the file (created in the above step) has changed, and if so it does whatever syntastic already is doing to parse the output and put it in the location list.

[s0undt3ch]

@q0LoCo your branch often fails to update the UI. I'm using powerline(if that matters) and mostly pylint as the lint checker...

[lcd047]

@acornejo I believe the thread above answers your point: it's possible in principle, but it's less simple than you seem to believe, and the result is likely to suck. Please feel free to prove me wrong though. :)

[lcd047]

@s0undt3ch Can you please take this to the appropriate place? Thank you!

[chemzqm]

I've tried to use other async check plugins, but turns out I miss the features of syntastic so much.
Finally I realized the problem is not with syntastic, the actual reason is the lint tool runs so slowly.
So I made a plugin to check javascript syntax error only https://github.com/chemzqm/jscheck
it uses notify functions from syntastic which are very well written :+1:

[bendavis78]

Out of curiosity, could we not somehow use vim's client/server model? That is, I could run my main vim session as a server, run the syntastic check in a separate client, and when checking is done have the signs and errors sent to the server.

[idbrii]

@bendavis78: There's not much benefit to initiating the check from another instance of vim. Most of the work is done by the external checker. Vim does some work to process the checker's output (see :help efm), but I don't see how we could send that from one vim session to another.

However, using vim's client/server model could work but there are concerns about requiring it for syntastic. AsyncCommand uses vim's client/server model to do async tasks: Run the checker in the background and when it completes, use --remote to send a signal to vim (but there's only been a proof of concept syntastic integration: stgpetrovic/syntastic-async@b3e3a9b). vim-dispatch works similarly, but uses a different signal to notify vim of completion. (See the above conversation about SIGWINCH.) I think the trickiest part is figuring out how to feed the output file into syntastic (probably more complex than :cgetfile).

[mtglsk]

Just so you guys know, version 7.4.1274 of vim added job functionality.

https://groups.google.com/forum/#!topic/vim_dev/_SbMTGshzVc

[justinmk]

Yep. So now everyone can stop pretending that job control isn't useful :)

[lcd047]

@mtglsk The interface is still under development, please see :h job-options. Either way, a number of things need to change in syntastic before it could take advantage of background jobs without having to rewrite the existing 202 checkers. Neovim had a similar interface for a while, and it inspired neomake, with a different approach than syntastic (see also #1253).

The forthcoming job interface in Vim will be similar to the one in Neovim, but it probably won't be compatible to it, despite several people (including Neovim's author) trying to reconcile the two APIs. At least that's my understanding of Bram's position, at the time of this writing.

@justinmk Working PRs are still welcome.

[chemzqm]

Neovim could parse result of async make command by using 'errorformat' option automatically, but I'm wondering vim will have this feature or not.

[q0LoCo]

There is job_status() function in new job api. Vim can use autocmd to parse result if there is no way to pass a callback function to job_start().
A checker may do somethings on result, the problem is that current buffer may not be the same one it just checked.

[lcd047]

A checker may do somethings on result, the problem is that current buffer may not be the same one it just checked.

Yup, that's one of the problems.

[parshap]

Vim 8 is out with "native" support for async functions via channels! Does this make sense for Syntastic to use?

[lcd047]

Yes, async checking shall be implemented, eventually.

[shinzui]

@lcd047 is there a separate issue for that?

[lcd047]

@shinzui Not that I know of.

[lencioni]

In the meantime, it looks like validator does async syntax checking. https://github.com/maralla/validator.vim

I haven't tried it yet, so I don't know if it is any good.

[lcd047]

In the meantime, it looks like validator does async syntax checking.

@lencioni Right. And so does YouCompleteMe. And vim-go. And neomake. And LaTeX-Box. And pyflakes-vim. And ghcmod-vim. And other tools mentioned in related threads. And many others I can't remember right now. Yet others are getting there. All of these projects have features that syntastic doesn't. I strongly encourage you to use them instead of syntastic.

[blueyed]

@lcd047 as the de-facto current maintainer of Neomake I've often thought that my time would have been better spent by working on an async feature for syntastic, rather than fixing issues there all the time..
By now we have a test suite, but there are still fundamental issues, and all the groundwork already did and still does, and features it has are not available there.

[justinmk]

@lcd047 What are your thoughts on decoupling syntastic's library of checkers (similar to how snippets are separated from Ultisnips), so that other checker "engines" can use them? That would be 100x more valuable than making syntastic itself asynchronous. It seems inevitable that people are going to re-invent these types of plugins, but if a library of checkers existed it would practically guarantee that the checkers themselves would no longer be re-invented.

By "decouple" I don't necessarily mean moving the checkers to a separate repo, but just structuring the checkers so that they are isolated.

[lcd047]

@blueyed You (and many people above) assume that, now that Vim has support for jobs, what prevents syntastic from doing async checks is that it doesn't call checkers asynchronously. That's not the case. That is, it's true, but it's the least of the problems to be solved. What _actually_ prevents syntastic from doing async checks right now is that half of its functions assume that bufnr('') returns the number of the buffer currently being checked, and that b: variables are local to the same buffer. That problem was kind of lurking in syntastic before I came to the picture, but it was cast in stone by a design mistake that I made when I allowed checkers to have per-buffer overrides. That happened long before Neovim, and before the above discussions about async checks.

So the problem I'm facing right now is how to fix the internal API, preferably without breaking compatibility with all third-party checkers out there. It's a design problem, and no, simply breaking backwards compatibility won't solve it. _shrug_

[lcd047]

@justinmk Are we talking philosophy or logistics?

Philosophically, I think that:

1. Syntastic has built-in checkers; this has been a mistake;
2. Syntastic is using errorformat and the makeprg model; this has been a mistake;
3. Syntastic is trying (and failing) to solve some unreasonable parsing problems; this has been a mistake.

Logistically, I think that:

1. You can't meaningfully talk about a "library" without defining an API. Syntastic did try to define such an API. It failed, in that there are at least four methods of producing loclists, and they are only superficially integrated:
   
   
   
   • makeprg / lgetexpr (most checkers)
   
   
   • the preprocess machinery (used f.i. by pydocstyle)
   
   
   • the syntastic#c#GetLocList machinery (used by most "real" compilers, such as gcc)
   
   
   • generating loclists from the output of other plugins (used by vimlint).
     
     
     Integrating all these in a library would involve a major rewrite.
   
   
2. It won't do you much good to have "decoupled" checkers, simply because syntastic goes way beyond returning a list of errors. It also compensates for a myriad of tiny flaws in Vim, and in the design of the 200+ supported linters (despite trying hard to limit those workarounds to a minimum). Some time ago I browsed through neomake's issues, and as far as I can tell this is what its authors failed to take into account. _shrug_
3. You _could_ grep the code for errorformats and put them into a library. Some people take the view that syntastic is useless, and everybody (especially new users) should learn to use the compiler features builtin to Vim. I presume those users would find such a library useful, or at least an useful start. Syntastic comes from there, and it's heading to the opposite direction.
4. VimL is a crappy language. Among other things, it doesn't have named captures for regexps. This could be overcome by relying on python and / or python3 for parsing errors. That would be a worthwhile project. It would still not result in a library; just a major, very nice overhaul of three of the four methods mentioned at 1.
5. The bottleneck for implementing all coupling, decoupling, async checks, and whatever other features you guys are asking for has a name: yours truly. I'm the only maintainer left (for whatever reasons), and these days I can afford to dedicate only a very small amount of resources to syntastic. I expect to be able to afford even less in the near future, despite the pressure you guys are building here. I encourage you to fork syntastic and change it to fit your needs. I'm an old fart, and I have my own priorities in the real life. Don't wait for me.

[lencioni]

despite the pressure you guys are building here

@lcd047 I didn't intend to build any pressure for you--simply to point people who are looking for this to something that might currently fit their needs. I rely on syntastic every day and I really appreciate all of your hard work that has gone into it.

I'm the only maintainer left

I wonder if there are any folks in this thread who would be interested in helping out more?

[blueyed]

@lcd047

it was cast in stone by a design mistake that I made when I allowed checkers to have per-buffer overrides

Neomake has those, too: the basic idea there is that the bufnr and winnr (preferably via the new win_getid function family, but otherwise using a window-local var) will be remembered for the job that is being run. And then e.g. location list results will only be opened when you are in the matching window/buffer (in case you move around while the job runs). (There are still issues in that regard though, especially with chained makers/jobs, that get run after each other)
Any settings that are relevant should be retrieved when the job starts, or from the corresponding buffer later.

[justinmk]

@lcd047 Thanks for that valuable insight. Personally my interest is (was) to avoid redundant effort in the Vim ecosystem, but based on your answer here and at SO the situation appears grim.

Some people take the view that syntastic is useless, and everybody (especially new users) should learn to use the compiler features builtin to Vim.

The reason IDEs work is because they ship with the entire stack. Plugins like quickrun and syntastic attempt to use tools found in arbitrary environments, configured (or not) by clueless users, to provide a light-weight IDE. And that's a recipe for endless bugs.

My preference would be to use the "compiler" feature of Vim. But it is incomplete, as you noted...

[lcd047]

I didn't intend to build any pressure for you--simply to point people who are looking for this to something that might currently fit their needs.

@lencioni So you started to make a list at the ~50th message in this thread. How inspiring.

Well, now that you mention it, a list with other linter projects _could_ be pretty useful. I'd suggest you create a wiki page for them and start collecting names, links, and basic characteristics (perhaps languages supported, async or not, etc.). If / when this list starts getting substance I'll merge it into syntastic, to save GitHub the trouble of rendering it.

Any settings that are relevant should be retrieved when the job starts, or from the corresponding buffer later.

@blueyed Right, which is why bufnr('') references are fine in checkers and notifiers. They're less than great in the checker API, in code that runs after the fact, in most of the loclist API, and in a few other autocmd-related places. Since Vim 7.4.1640 they also boobytrap check_on_open. _shrug_

My preference would be to use the "compiler" feature of Vim. But it is incomplete, as you noted...

@justinmk The errorformat plus makeprg model is flawed, not incomplete.

makeprg is broken because it makes it impossible to quote filenames robustly. Syntastic found that out the hard way, and it switched to system() plus lgetexpr. Running makeprg is also boobytrapped by QuickFixCmdPre and QuickFixCmdPost autocmds (meaning, you'll end up calling code from other plugins if you aren't careful).

errorformat is flawed because:

1. it doesn't expose the parsing loop to the user
2. it doesn't allow adding extra fields to the standard line and column numbers
3. it depends on isfname in a way that makes it impossible to match certain combinations of filenames and errors
4. the interaction between parsing rules is less than useful, and
5. it has arbitrary (undocumented) limitations.

makeprg and errorformat together are also flawed because they are essentially impossible to debug. If there is anything that syntastic has done right over Vim's compiler sets, it has been logging. It's far from perfect. But at least it can show you Vim's idea of what a linter should be doing.

Now, if you can live with these limitations then good for you. They're certainly fine for many simple cases. Some of us however have to make the best out of some hairy situations. That's why syntastic exists in the first place.

[lencioni]

So you started to make a list at the ~50th message in this thread. How inspiring.

@lcd047 I'm not really sure I understand what you are referring to about starting a list here. I just learned about this other project, thought "I bet the fine folks who work on syntastic are working on async too", found this issue, noticed folks talking about async, and thought they might find the thing I just learned about to be useful. I'm not trying to make any lists or inspire anyone, I just wanted to help out and share some info. 😕 I apologize if I've upset you--that was certainly not my intention.

[bounceme]

I think any attempt to make on-the-fly checking would be a bad move, rather than having the neomake-like bufwrite execution. I saw a few different versions for on-the-fly(piping, making and writing and switching between temp buffers) and they all look flawed.

[lcd047]

Here's an update for you guys. The plan goes something like this:

• I'll make a 3.8 release, provided that no new bugs are reported the next few days. This shall be the stable branch. If new bugs are reported after the release I'll fix them, but I won't be adding any new features (or new checkers, for that matter). This may eventually evolve into a stable 3.9 if needed.
• Right now I'm fixing the API for the checkers. Checkers will be able to use custom parsing functions instead of errorformat. Right now that means moving all JSON parsers from preprocess to the new parser sections of each checker, but there will be new parsing functions based on if_python and if_perl. This means some checkers will depend on Vim being compiled with Python or Perl.
• Most internal checkers (hopefully all) will be migrated to the new API, but external third-party checkers will stop working. In particular checkers that rely on calling a Vim function instead of running a program (such as tsuquyomi and omnisharp) will break. I plan to allow old-style checkers to run synchronously (along with the async checkers), and I'll eventually provide some upgrade path for checkers running Vim functions, but neither would happen at the beginning.
• The async functions still require a lot of bookkeeping. I'll upload a new branch named async when I have them actually doing something useful. They'll run with Vim 8.0, but they'll require a more recent Vim snapshot to work _reliably_ (a few critical problems have been fixed in Vim in the mean time). At first they won't run with Neovim. There will be hooks for Neovim though, provided that somebody in the know is willing to fill in the necessary code.
• A number of defaults will change. If you want to suggest changes to defaults to your favourite checkers, now is the time to do it. When the new branch gets released the defaults will again get locked for the next few years. :smile: Of course, I'll keep a detailed log with all incompatible changes, and I'll eventually update the docs, write an upgrade guide, etc.
• There are a number of other problems to be addressed, f.i. how to handle the IsAvailable() functions. This is important because some checkers (hello, rubocop) take forever and a day to return a version. So version queries might need to be run asynchronously -- but doing so would turn the API for IsAvailable() into a pain in the rear. Let's hope some useful approach pops up eventually.
• If you plan to write new checkers for syntastic after the 3.8 release, you'll have to use to the new API. And you'll have to do that based on whatever docs are available at the time. I don't plan to offer much hand-holding until the API stops changing. Sorry about that. Please feel free to explore using other checker engines in the mean time.
• When the new API in the async branch stops changing I'll call the result syntastic 4.0, and merge it to master. All major issues would have to be fixed for that to happen. And you probably want to stay with the stable branch in the mean time.

That's the basic outline. Of course, that's just a plan. Plans do typically change.

[blueyed]

@lcd047
Thanks for the update and plans for a bright future.

Have you thought about using Neomake as a base for the new Syntastic?
Syntastic could be kept in maintenance mode as-is, and instead its features would get ported over to Neomake. Neomake has a quite decent testing framework already, and has support for both Neovim and Vim's async modes.

I'd rather not waste development and support effort. So if you do not like going the Neomake route then I might just put Neomake (from my side) into maintenance mode and help you out.. :)
But please consider that Neomake had been created (not by me though) as a proof-of-concept to make Syntastic async.

[lcd047]

@blueyed

Thanks for the update and plans for a bright future.

Oh absolutely, maintaining a healthy dose of skepticism is always a good idea. :smile:

Have you thought about using Neomake as a base for the new Syntastic?

I have. I still think syntastic is a lot easier as a foundation for what I'm trying to do than Neomake would be. The vast majority of syntastic is also my code, so I'm already pretty familiar with it.

Neomake has a quite decent testing framework already,

On a side note, syntastic doesn't have tests for a single reason: I'm not the owner of this repository, and activating Travi-CI for it would require the intervention of the initial author. I contacted him in the past and, while he was always nice, I did get a feeling he wants to put all this behind him, and he wouldn't appreciate being bothered with anything related to Vim these days. Activating Travis-CI is simple but not completely trivial if you haven't done it in a long time, it might take a few exchanges back and forth.

and has support for both Neovim and Vim's async modes.

I'm not worrying about Neovim at this point. With the design I'm going to use adding support for Neovim shall be either trivial, or very hard. I'm reasonably confident it's going to be trivial rather than very hard.

But please consider that Neomake had been created (not by me though) as a proof-of-concept to make Syntastic async.

As far as I can tell the point of Neomake was to create an asynchronous :make for Neovim, completely integrated with compiler sets and directly extending them. I think the author of Neomake considered syntastic grossly over-engineered. I don't think Neomake was ever meant to be an async syntastic.

Now, is syntastic grossly over-engineered? I'm sure it is. I'm a scientist that happens to know a few things about computers, not a programmer, and when I started all this I didn't know substantially more about Vim than the average user (despite having already used Vim for 15+ years at the time). It probably shows. However, there is a number of things in syntastic that are the way they are for a number of reasons. The author of Neomake was not aware of these reasons (which is by no means surprising, since errorformat and friends are a mine field), and that also shows. I suppose people these days address this problem by writing blog posts about their projects, where they basically keep chronicles about their reasons and their designs. But I'm an old fart, and blogs are way past my time. :smile:

[mgedmin]

On a side note, syntastic doesn't have tests for a single reason: I'm not the owner of this repository, and activating Travi-CI for it would require the intervention of the initial author. I contacted him in the past and, while he was always nice, I did get a feeling he wants to put all this behind him, and he wouldn't appreciate being bothered with anything related to Vim these days.

It might be best to create a GitHub organization and ask the original maintainer to transfer repository ownership to the new org.

GitHub redirects old URLs when you transfer ownership, so existing links (and, more importantly, git checkouts) will continue to work.

[lcd047]

@mgedmin I'm not particularly unhappy with the current arrangement, and there are more pressing issues at the moment than enabling Travis-CI. I might consider it later though.

[scrooloose]

Hey guys, I have also been thinking it would be a good idea to move syntastic out into an org - for a while now. @lcd047: you have been the main maintainer for several years now and it is wrong for it be under my name after all this time. For one, I am getting credit for your hard work, and two, for the practical reasons you outlined above.

Thoughts?

One thing I would consider is putting the checkers and the core in different repos. Then you could have a very open policy with adding maintainers to the checker repo. For me, it was the tedium of maintaining the checkers that I burned out on. Perhaps that is a solution if you feel the same.

[lcd047]

@scrooloose You do deserve the credit, syntastic has been a very useful plugin for a years when I came around. I just tried to make it more maintainable, with some optimisations, improving the docs, and by adding some obvious features. But the overall design is yours, and it's beyond my league in creativity.

I'd be fine with whatever you decide, including a GitHub organisation. I'd also be fine with leaving it as it is, but yeah, there are a few practical problems that can only be solved by the owner of the project.

As for splitting the checkers from the core, that's probably a good idea. It would make sense to split them by groups of related languages, but that would leave the less popular ones in no man's land. In fact the long tail is probably more important than the main languages, since things like C/C++, Python, and Go already have plugins that can do much better than syntastic, simply by being more specialised.

[scrooloose]

OK, well that was scary but seems to have worked fine.

@lcd047 I have created the vim-syntastic org and added you as an owner (i.e. the highest permissions). I have then transferred syntastic into this org. All the links are redirecting fine (AFAICS) so it looks like a win.

Thanks for the kind words :-) I wouldn't undersell your hard work and persistence though. I think the syntastic project was very lucky that you came along just as I was running out of steam.

[lcd047]

@scrooloose Thank you, it seems to be working fine for me too.

[unphased]

Someone mentioned instability concerns; I use YCM with javascript and it already leaves tern stomping out 100% CPU semi-regularly. This is in no way encouraging this as something that is okay, because it so absolutely is not.

I really would prefer syntastic to stop blocking the UI because I have tmux binds that need Vim to stay responsive for the keybinds to stay responsive.

In fact, I'm so twitchy that even if I'm not using the binds I'll still end up typing movement characters or something, and it invariably makes the command line shift up and it garbles up vim's TUI render state, and all of this is because of blocking the UI.

Apologies for not reading the thread to the letter as that'd take the better half of a day which I'd be happy to devote if I could afford that at the moment. It sounds like architectural changes that will make syntastic even more amazing than it already is are in the pipes and they'll include robust async checkers. 👍

🎉

[mgedmin]

ALE is a Syntastic-like Vim plugin that works asynchronously. AFAICT it supports fewer languages than Syntastic, but you could check it out today instead of waiting for Syntastic to get support for async.

[JohanBjoerklund]

Neomake is also an alternative

[lcd047]

@unphased Sorry to disappoint you, but for the last few months I've been struggling with health problems. I'm a sick, old fart, and my priorities are those of a sick, old fart. Support for async will be ready when / if I can get myself together enough to finish it. Until then, you shouldn't really expect more than basic support for the existing code. In the mean time you could always:

1. submit a patch to add support for async,
2. fork syntastic, or
3. use something else.

Not sure what else I could tell you. _shrug_

[gilbertwyw]

@lcd047 Hope you get well soon, i'm grateful for your great work on syntastic. 🙏

[unphased]

@lcd047 Absolutely, your contributions to the Vim ecosystem have NOT gone unnoticed!

[bradleyfalzon]

I know +1s are frowned upon, but I also wanted to say how much I loved and used Syntastic, it was one of the first plugins I used with vim, and we used it extensively. Love your work, and thank you.

[wshanks]

Thank you, @lcd047 for all your work on Syntastic. I hope you feel better soon. If working on Syntastic adds stress to your life and asynchronous support will be a lot of work, you might want to try out ALE that @mgedmin suggested. I have been testing it recently and it works well -- for the languages I use it is like a drop-in replacement for Syntastic except it doesn't hang my terminal when I save. I worry that by the time Syntastic has asynchronous support that most of the userbase will have migrated to something else like it.

[bounceme]

https://github.com/vim-syntastic/syntastic/issues/699#issuecomment-285350988
there have already been several async linter runners made. ALE is one of the newest and will have the most changes and activity, sure, but tons of people have used syntastic for a long time and there is a huge amount of linter support here.

And, last but not least, I have exactly zero use for all this, I'm pretty happy with the sync checks. shrug

[ErikBjare]

I just switched from syntastic to ALE which went pretty easy. I figured someone might wonder how to do that in a reasonable manner, so here's a basic port of my config: https://github.com/ErikBjare/dotfiles/commit/2b946b5d0f95ba056ce99bf0487c3ff414ce3c51

[vitaly-zdanevich]

Still blocking only, with waiting a few seconds after file saving? :(