Doom-emacs: lsp-mode gets out of sync with buffer causing false errors

[jsravn]

What did you expect to happen?
lsp-mode should correctly synchronize the buffer changes with the lsp server.

What actually happened?
When using lsp-mode, the lsp server seems to get frequently out of sync with the buffer. This has been reported by many users on Discord, and I have experienced it since I started using Doom across multiple lsp language servers.

I am using a function from the lsp-mode test suite which adds advice on the lsp function that triggers on modification events, and mirrors what lsp-mode receives on changes and then sends to the lsp server. [0]

Using this mirror buffer, I can observe that lsp-mode is not receiving all the buffer changes.

Here is an example where it happened. Here is the original buffer:

const char *lookupError(int err) {
  switch (err*-1) {
    case EAI_BADFLAGS:
      return "EAI_BADFLAGS";
  }
  return "unknown";
}

And the mirror buffer has:

const char *lookupError(int err) {
switch (()err*-1) {
    case EAI_BADFLAGS:
      return "EAI_BADFLAGS";
  }
  return "unknown";
}

() is inserted by smart-parens-mode, but what happens is the left paren gets immediately deleted in the buffer, and this deletion should show up in the mirror buffer. I also formatted the code using SPC c f which indented the function body in the buffer, but as can be seen lsp did not get the indent change either.

I still haven't been able to find an easy way to reproduce this. It seems to happen almost randomly over the course of an editing session (about once every hour or so).

From talking w/ the lsp-mode maintainer about this, he suggests that it looks like something is setting inhibit-modification-hooks, which prevents lsp from getting the buffer changes sometimes. I'm guessing there is some small race condition somewhere leading to this.

I've spent a lot of time trying to debug this so any help is appreciated!

Additional details:

• https://github.com/jsravn/emacs-config
• No errors.

System information:

emacs   version    27.0.91
        features   XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER GMP
        build      Jun 01, 2020
        buildopts  (--prefix=/nix/store/c05kfadkps7k7qzclcq0flg117jmdrvy-emacs-27.0.91 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft CFLAGS=-DMAC_OS_X_VERSION_MAX_ALLOWED=101200)
        windowsys  x
        daemonp    server-running
doom    version    2.0.9
        build      HEAD -> develop 7ed96590e 2020-05-26 21:45:31 -0400
        dir        ~/.config/doom/
system  type       gnu/linux
        config     x86_64-pc-linux-gnu
        shell      /run/current-system/sw/bin/zsh
        uname      Linux 5.4.43 #1-NixOS SMP Wed May 27 15:46:53 UTC 2020 x86_64
        path       (~/.config/emacs/bin/../bin/ /nix/store/zrwinq0diavisjyl2q6pw2arhqswf9df-kitty-0.16.0/bin/ /nix/store/pg4ghlqlmmg3zr6j6faqc1g74d3bg2fz-imagemagick-6.9.10-71/bin/ /nix/store/bdvvgigcqm0biz070yq7ds4ycirxnjq7-xsel-unstable-2019-08-21/bin/ /nix/store/dyfb5yvmqfprr5fzgx3d117gwypp2bl6-ncurses-6.1-20190112-dev/bin/ /nix/store/rk5nr8s3bcp3jdb0y8rsmpd476xl6als-swaybg-1.0/bin/ /run/wrappers/bin/ ~/.config/emacs/bin/ ~/go/bin/ ~/.nixfiles/bin/ ~/.nix-profile/bin/ /etc/profiles/per-user/james/bin/ /nix/var/nix/profiles/default/bin/ /run/current-system/sw/bin/ /nix/store/c05kfadkps7k7qzclcq0flg117jmdrvy-emacs-27.0.91/libexec/emacs/27.0.91/x86_64-pc-linux-gnu/)
config  envfile    envvar-file
        elc-files  0
        modules    (:completion (company +childframe) (ivy +fuzzy +icons) :ui doom doom-dashboard hl-todo indent-guides (modeline +light) nav-flash ophints (popup +all +defaults) (pretty-code +pragmata-pro) treemacs vc-gutter vi-tilde-fringe window-select workspaces zen :editor (evil +everywhere) fold format multiple-cursors rotate-text snippets word-wrap :emacs dired electric vc :term term :checkers syntax spell :tools ansible docker (debugger +lsp) direnv editorconfig (eval +overlay) (lookup +docsets +dictionary) (lsp +peek) (magit +forge) make terraform :lang (cc +lsp) (clojure +cider +lsp) common-lisp data emacs-lisp (go +lsp) (java +lsp) javascript markdown nix (org +dragndrop +journal +roam) (python +lsp +pyenv) (scala +lsp) sh scheme (yaml +lsp) :email mu4e :config literate (default +bindings +smartparens))
        packages   ((atomic-chrome) (org-roam :pin 4af4d2e4d5f18422fe0f90d5d83fe4c10ef1a20f) (org-msg) (org-alert) (caddyfile-mode))
        unpin      (lsp-mode)
        elpa       (n/a)

(defun lsp-notify-wrapper (params)
  (let ((lsp--virtual-buffer-mappings (ht)))
    (pcase (plist-get params :method)
      (`"textDocument/didChange"
       (setq my/params params)
       (-let [(&plist :params
                      (&plist :textDocument (&plist :uri :version)
                              :contentChanges [(&plist :range (&plist :start :end )
                                                       :text)]))
              params]
         (with-current-buffer (get-buffer-create (format "*%s*" (f-filename (lsp--uri-to-path uri))))
           (let ((start-point (if start
                                  (lsp--position-to-point (ht ("line" (plist-get start :line))
                                                              ("character" (plist-get start :character))))
                                (point-min)))
                 (end-point (if end
                                (lsp--position-to-point (ht ("line" (plist-get end :line))
                                                            ("character" (plist-get end :character))))
                              (point-max))))
             ;; (display-buffer-in-side-window (current-buffer) ())
             (delete-region start-point end-point)
             (goto-char start-point)
             (insert text)))))
      (`"textDocument/didOpen"
       (-let [(&plist :params (&plist :textDocument
                                      (&plist :uri
                                              :version
                                              :text)))
              params]
         (with-current-buffer (get-buffer-create (format "*%s*" (f-filename (lsp--uri-to-path uri))))
           ;; (display-buffer-in-side-window (current-buffer) ())

           (delete-region (point-min) (point-max))
           (insert (or text ""))))))))
(advice-add 'lsp--send-notification :before 'lsp-notify-wrapper)

[hlissner]

Does disabling ws-butler-mode or the :editor format module fix this?

(remove-hook 'doom-first-buffer-hook #'ws-butler-global-mode)

[jsravn]

I will give that a try. Right now it's a process of elimination so it will take a while to test each thing.

[ericdallo]

For me, this happens when I change of workspace, example:

1. lsp running fine on workspace A
2. change to workspace B (then lsp prints LSP :: Shutdown clojure-lsp:27936 since folder /home/ericdallo/dev/project is removed...)
3. when I go back to workspace A, lsp is not running anymore.

[ericdallo]

Disabling lsp-treemacs-sync-mode fixed my problem

[jsravn]

@ericdallo is that the same as the sync issue I'm describing? When it happens to me, it gives incorrect flycheck errors and I have to restart the LSP session to fix things.

[ericdallo]

No, sorry, my problem is that my lsp session was dying but without any flycheck issues.

[jsravn]

Update: I thought doom-modeline might be the culprit (as it has some inhibit hooks), so I swapped to (modeline +light) for the last day but just experienced the same desync. This time it's a missing space, so maybe it is ws-butler.

Edit buffer:

Mirror buffer:

Although there's also a duplicate "r" so I'm not sure. Maybe it's a combination of things. I'll try disabling ws-butler now for a couple days and report back.

[jdnavarro]

I'm seeing the same issue. Disabling ws-butler-mode or the :editor format didn't fix the issue.

If I insert a new blank line and save the buffer flycheck shows the correct errors.

[ghost]

Issue appeared after https://github.com/hlissner/doom-emacs/commit/fdf9c408e8591f2a3c6100b9cac89ec3ffd47de2

[hlissner]

Does (setq lsp-flycheck-live-reporting t) fix the issue for you?

[wedens]

@hlissner That option doesn't exist anymore https://github.com/emacs-lsp/lsp-mode/pull/1701. And this change might be related to the issue.

[jsravn]

I have lsp-flycheck-live-reporting enabled, but it did not fix the problem.

However I haven't had any desyncs since disabling ws-butler last week. So far that looks like the culprit for me. I'll report back if it happens again.

[wedens]

I've noticed the same problem with lsp in scala. If I run flycheck-buffer, no longer relevant errors disappear. I've tried disabling ws-butler, but it doesn't help.

[wedens]

I'm gonna try using the default value for flycheck-check-syntax-automatically which is '(save idle-change new-line mode-enabled). Doom sets it to '(save mode-enabled idle-buffer-switch) which may delay checking for too long.

Also related upstream issue: https://github.com/emacs-lsp/lsp-mode/issues/1792

[jsravn]

I don't think this will help in my case. As seen in the lsp mirror buffer, lsp-mode isn't receiving all buffer changes. So it won't matter when flycheck runs, the lsp server itself will not have a correct view of the buffer.

[wedens]

Yeah, that sounds like we have different issues then. In my case LSP server receives the changes but flycheck often shows outdated diagnostics.

[babariviere]

The suggestion of @wedens fixes the issue for me. There is no more lagging issue and lsp is showing me the correct errors :+1:

[jsravn]

I haven't had any issues since disabling ws-butler. So it seems like ws-butler is doing some weird things causing the LSP on change hook to not get triggered sometimes. I haven't used LSP as much in the last month, so not 100% sure, but I'll keep this issue updated if I see a problem again.