Lsp-mode: Port existing packages to lsp.el

[yyoncho]

List of the packages to migrate:

• [x] C (ccls) emacs-ccls
• [x] C (clangd) lsp-clangd
• [x] C (cquery) emacs-cquery
• [x] Haskell lsp-haskell
• [x] Java (Eclipse JDT) lsp-java
• [x] OCaml lsp-ocaml
• [x] PHP lsp-php
• [x] Ruby lsp-ruby
• [x] Rust lsp-rust
• [x] Scala lsp-scala
• [x] Vue lsp-vue

Here it is example from lsp-java

(eval-after-load 'lsp '(lsp-register-client
    (make-lsp--client
     :new-connection (lsp-stdio-connection 'lsp-java--ls-command)
     :major-modes '(java-mode)
     :server-id 'jdtls
     :multi-root t
     :notification-handlers (lsp-ht ("language/status" 'lsp-java--language-status-callback)
                                    ("language/actionableNotification" 'lsp-java--actionable-notification-callback)
                                    ("language/progressReport" 'lsp-java--progress-report)
                                    ("workspace/notify" 'lsp-java--workspace-notify))
     :action-handlers (lsp-ht ("java.apply.workspaceEdit" 'lsp-java--apply-workspace-edit))
     :uri-handlers (lsp-ht ("jdt" 'lsp-java--resolve-uri)
                           ("chelib" 'lsp-java--resolve-uri))
     :initialization-options (lambda ()
                               (list :settings (lsp-java--settings)
                                     :extendedClientCapabilities (list :progressReportProvider t
                                                                       :classFileContentsSupport t)
                                     :bundles (lsp-java--bundles)))
     :library-folders-fn (lambda (_workspace) (list lsp-java-workspace-cache-dir))
     :before-file-open-fn (lambda (workspace)
                            (let ((metadata-file-name (lsp-java--get-metadata-location buffer-file-name)))
                              (setq-local lsp-buffer-uri
                                          (when (file-exists-p metadata-file-name)
                                            (with-temp-buffer (insert-file-contents metadata-file-name)
                                                              (buffer-string)))))))))

[MaskRay]

Since lsp-mode.el -> lsp.el is breaking, have you considered renaming extra-init-params to initialization-options?

[yyoncho]

IMO it is a good chance to do all the cleanup that we didn't want to do due to backward compatibility issues.

[shackra]

I got this from lsp-javascript-typescript:

Error (use-package): lsp-javascript-typescript/:catch: Symbol’s function definition is void: lsp-make-traverser

The mode comes from https://github.com/emacs-lsp/lsp-javascript/ so it should be add to the list of clients to migrate, no?

[yyoncho]

If you are using lsp.el you do not need lsp-javascript package at all. You have to require lsp-clients.el and call lsp directly.

[shackra]

Oh, so, is a matter of adding lsp to *-mode-hook.

The same applies for lsp-css? it isn't listed.

[yyoncho]

Yes, check what is available in lsp-clients . The documentation is wip and we will apreciate any help.

[shackra]

Thanks!

@yyoncho I have opened https://github.com/emacs-lsp/lsp-mode/issues/489 because I have a couple of more questions and feels like we are hitting a bug on the documentation currently available regarding the changes made in the project.

The documentation is wip and we will apreciate any help.

OH! okay, in such case I'll wait because honestly I don't know what I'm doing right now

[seagle0128]

Hi @yyoncho , When will lsp-mode.el be disabled?

[yyoncho]

@seagle0128
When:

1. All (or at least all major) language packages are ported to lsp.el
2. Spacemacs layer is ported to lsp.el
3. dap-mode is ported to lsp.el
4. lsp.el is considered stable.

Do you have something in mind?

[seagle0128]

@yyoncho It's fine with me. Eager to get the new features in lsp. Thanks a lot! Great work!

[yyoncho]

You may use it even now, refer to https://github.com/emacs-lsp/lsp-mode/blob/master/README-NEXT.md (if you are using melpa wait for https://github.com/tigersoldier/company-lsp/pull/69 was just pushed). We are looking for feedback on what we currently have.

lsp.el is implemented so once you require it lsp-ui will start working against it. But you cannot use both lsp-mode.el and lsp.el.

[alanz]

I have just realised this thing exists. What do I need to do to migrate lsp-haskell to it?

[alanz]

never mind, looking into it. PR some time soon, I hope.

[yyoncho]

I believe the minimal configuration will be sufficient for lsp-haskell. Here it is the python example:

(lsp-register-client
 (make-lsp-client :new-connection (lsp-stdio-connection "pyls")
                  :major-modes '(python-mode)
                  :server-id 'pyls))

I will update lsp-language-id-configuration include map haskell-mode to haskell.

[alanz]

Ok, just looking into it. We have various configuration options as well, I guess they can come over similar to the ones for go-lang?

[alanz]

and is there a standard way of dealing with configuration settings now? For the lsp--set-configuration message.

[yyoncho]

The lsp--set-configuration is still present, that part is not changed. We have added few more extension points in registration client registration and some of the configuration like doc renderers are handled out of the box.

[alanz]

Ok, I think I will take the java approach to start with, and keep it in lsp-haskell, to get a feel for it.

Then as a later stage it can come into the main repo if it makes sense.

[yyoncho]

I agree, I think this is the best approach.

[yyoncho]

@alanz make sure that you are using lsp.el version having 1fd6b34072ac5fa6d6774820029e5c305b9427c0

[alanz]

Does the new method no longer allow for providing a custom function to find the project root?

[yyoncho]

Yes. lsp.el will interactively ask once for the project root and the default suggestion will be provided by projectile/project.el . Once you select a project root it will be persisted and used for the upcoming sessions. In works like vscode but you are not forced to add the workspace folders upfront.

[MaskRay]

Haskell is not listed in https://microsoft.github.io/language-server-protocol/specification but I hope the value of languageId does not matter too much

lsp-java.el currently does:

(require 'lsp-methods)
(eval-after-load 'lsp
  '(lsp-register-client
    (make-lsp--client

emacs-ccls/ccls.el currently does https://github.com/MaskRay/emacs-ccls/commit/7ab406d48da7c6122bc95a4fe9a0fda53a5dc299

(require 'lsp)

(lsp-register-client
 (make-lsp-client

I have a PR to fix emacs-cquery

[yyoncho]

@MaskRay you may delete the ccls--make-renderer as well.

[alanz]

Ok, the basics seem to be working for haskell, but the workspace/executeCommand is blowing up.

Debugger entered--Lisp error: (wrong-type-argument hash-table-p "7744:hie:fallbackCodeAction")
  gethash("command" "7744:hie:fallbackCodeAction")
  (lsp--send-execute-command (gethash "command" command) (gethash "arguments" command nil))

I think the problem is that the code action should be indexed by title, not command. We use the same command each time in hie, with different arguments.

The codeActions message from the server is

{
  "result": [
    {
      "command": "7744:hie:fallbackCodeAction",
      "arguments": [
        {
          "fallbackWorkspaceEdit": {
            "changes": {
              "file:///home/alanz/tmp/hie-demo-project/src/B.hs": [
                {
                  "range": {
                    "start": {
                      "line": 23,
                      "character": 0
                    },
                    "end": {
                      "line": 23,
                      "character": 0
                    }
                  },
                  "newText": "ff :: IO ()\n"
                }
              ]
            }
          },
          "fallbackCommand": null
        }
      ],
      "title": "Add signature: ff :: IO ()"
    },
    {
      "command": "7744:hie:fallbackCodeAction",
      "arguments": [
        {
          "fallbackWorkspaceEdit": null,
          "fallbackCommand": {
            "command": "7744:hare:liftonelevel",
            "arguments": [
              {
                "pos": {
                  "line": 23,
                  "character": 0
                },
                "file": "file:///home/alanz/tmp/hie-demo-project/src/B.hs"
              }
            ],
            "title": "Lift ff one level"
          }
        }
      ],
      "title": "Lift ff one level"
    },
....

[alanz]

And I think the problem is in the body of lsp-execute-code-action

  (when-let ((edit (gethash "edit" action)))
    (lsp--apply-workspace-edit edit))

The action does not have an "edit" value, and a message inserted after this never shows up in the Messages.

[alanz]

Nope, https://github.com/emacs-lsp/lsp-mode/pull/495

[alanz]

So I found you can do

(eval-after-load 'lsp
  '(add-to-list 'lsp-language-id-configuration '(inc-mode . "inc") ))

in the language-specific config file, so the mode start-up works as for the built-in clients.

[yyoncho]

@seagle0128 I saw a comment from you in emacs-china that you had an issues with lsp.el, can you file a corresponding issues.

Apart from that, I saw that you are maintainer of https://github.com/seagle0128/doom-modeline . I wonder whether doom-modeline could provide lsp-mode specific segments. Currently (lsp-mode-line) gives an info what are the running servers. Also, in addition to flycheck info it would be good if we display errors/warnings/infos for the current workspace(s).

[seagle0128]

Hi @yyoncho,

I just tried new lsp interface for a few hours. Some behaviors are changed and I am not sure if they are as designed. When I was working on a complicated project including Python, Golang, HTML/CSS, Javascript and Shell, it confused me. Which client should be started? or all clients? But I observed not all clients were started successfully. Moreover, some clients haven't been ported. So I reverted to lsp-mode for development. lsp is a good direction, and it simplifies the configurations like eglot. But needs some time to stable IMO. I will try later if I have some time. Thanks for your great work!

Regarding doom-modeline, I'm pleased to add more features for lsp-mode. I can add new segment for lsp-mode. Currently doom-modeline displays flycheck info for current buffer, not for all workspace. I'd like keep the thing simple and fast. If the flycheck info of the whole workspace is easy and fast to get, I think I can add a new segment or replace the old one (depends on user's option). Your thoughts?

[yyoncho]

Thanks, few comments:

When I was working on a complicated project including Python, Golang, HTML/CSS, Javascript and Shell, it confused me. Which client should be started?

When you open Python file it will start the python server and so on. There is a command lsp-describe-session which will list the folders and the corresponding servers and buffers associated to it. Also, as I mentioned doom-modeline does not display the info that you need in order to figure out what are the associated servers. Here it is what lsp UI is displaying.

But I observed not all clients were started successfully. Moreover, some clients haven't been ported.

All of the clients you have mentioned are ported.

But needs some time to stable IMO.

In order that to happen we have have concrete PR-s to work on. At this point, there are no lsp.el specific PRs.

Regarding doom-modeline, I'm pleased to add more features for lsp-mode. I can add new segment for lsp-mode. Currently doom-modeline displays flycheck info for current buffer, not for all workspace. I'd like keep the thing simple and fast. If the flycheck info of the whole workspace is easy and fast to get,

As you might have noticed, I want to create a IDE-like experience around lsp-mode with integration with the following components:

• lsp-ui for UI sugar
• Flycheck/Flymake for listing buffer errors
• treemacs for project browser (check https://github.com/Alexander-Miller/treemacs/issues/263 @Alexander-Miller was very helpful)
• helm/ido/ivy for management of errors/folders/workspace symbols and so on.
• dap-mode for debugger
• modeline?

I like doom-modeline but without some of the information I mentioned in the previous comment it wont be a good fit for lsp-mode.

I want to have the following components in the modeline:

1. Information about servers running(check screenshot)
2. Information about the state of the workspace(e. g. number of errors/warnings/infos)
3. A place to list progress tasks which will consist of percent + task description.

If you want to keep doom-modeline minimalistic, that's fine.

I think I can add a new segment or replace the old one (depends on user's option). Your thoughts?

I believe that we need a separate one

[alanz]

A general comment on lsp.el approach.

I just opened a markdown file, and LSP prompted for the project root. But I do not have the markdown lsp server installed.

So perhaps we should have some way of configuring which lsp servers are available on a given machine.

[seagle0128]

@yyoncho I love this awesome plan!!! It will make Emacs more fancy definitely! lsp is fantastic, and the debugger is also important to the developers. I went through dap-mode a few days ago, and like the design. Hope it gets more powerful! Thank you!

treemacs is another powerful package I love. I use it instead of neotree I used for a long time. It's integrated to Centaur Emacs by default. Really awesome although I don't use it frequently, especially in terminal. But it's very useful to read a new big project.

I also have a comment about workgroup/layout, like Eclipse perspectives or VSCode layouts. The similar package for Emacs is persp-mode . Is it possible adding this into the plan list?

Well, I will take some time to think how to integrate it to doom-modeline. I'd love do that, but as you mentioned, I also want to keep it minimalistic and fast. I will make the decision after the investigation anyway.

[yyoncho]

@alanz this looks like a bug in markdown which does not delay-mode-hooks when it tries to fontlock a language. I tried to reproduce it on my side but it does not reproduce. As a reference, here it is how we font-lock a language without running major mode hooks. I guess putting (debug) at lsp will reveal what the problem is.

  (condition-case nil
      (with-temp-buffer
        (delay-mode-hooks (java-mode))
        (insert str)
        (font-lock-ensure)
        (buffer-string))
    (error str))

[seagle0128]

@yyoncho I implemented a simple indicator of LSP for doom-modeline.

Since there are two interfaces of getting status, I leverage lsp-mode-line directly for now. It's same format and face as the lighter.

If lsp is stable and lsp-mode is removed, I can improve it with new APIs. For example, querying status and show different icons. Anyway I think it looks good so far. Your thoughts?

[yyoncho]

Looks good! I like the idea of showing the status in different colours. I am open also for improvements.

At this point, I am considering lsp.el to be stable based on the lack of lsp.el specific PRs (excluding the errors caused by lsp-mode.el <> lsp.el messing up). I will try to move lsp-mode.el deletion forward by the end of the week.

[alanz]

And I think you can mark haskell off as ported/supported.

[yyoncho]

@seagle0128

I also have a comment about workgroup/layout, like Eclipse perspectives or VSCode layouts. The similar package for Emacs is persp-mode . Is it possible adding this into the plan list?

I have missed that question: sure, we can do that. Longterm we can do mouse/toolbar integration in order to attract more "traditional" users to emacs. . Treemacs already have mouse support.
I am planning to implement VScode/Eclipse/Intelij/etc icon themes too, dap-mode and lsp-java do support themes.

[seagle0128]

@yyoncho The plan will help attracting more users, especially beginning users. VSCode is a role model of the modern editors. Advance users prefer keyboard. But for me I don't reject mouse :-)

I checked dap-mode and I'm very interested in it. Debugging in emacs is a missing feature in Emacs IMO, at least not good before. My experiences tell me only gdb works well in Emacs, others sucks. Sorry to say that! But dap-mode may open a new door. Microsoft is doing good things, like lsp and dap from VSCode. Awesome ideas! All editors will benefit from VS/VSC.

BTW, regarding lsp mode-line, I may suggest putting it in global-mode-string instead of minor-mode or providing an option. Because many emacsers don't like too many minor modes (like me?) and disable them by diminish or delight. While the most of modelines keep global-mode-string. I know display-time-mode and org-pomodoro use it to display status and information. Of cause, it should be disabled by users if the standalone segments for lsp is provided in modeline, like doom-modeline.

[stardiviner]

I prompt an idea that hope lsp can provide troubleshooting of lsp itself. Let user know where is wrong. and auto generate info let developer can diagnose.

[stardiviner]

And there is other lsp clients like:

• [x] lsp-html
• [x] lsp-css

[yyoncho]

html/css are already ported.

I prompt an idea that hope lsp can provide troubleshooting of lsp itself. Let user know where is wrong. and auto generate info let developer can diagnose.

I have tried to improve process error handling + error messages. You make take a look at lsp-describe-session too + the new lsp-mode lighter which lists what are the language servers connected to the current buffer.

[yyoncho]

@eubarbosa I am not (e)lisp expert but in general having one big file is not considered as bad in Lisp based languages.

[nykma]

Hi, I want to contribute a lsp-client config of Dart language (which is not listed in the checklist above). I've tested it on my machine. Can I just open a PR or give you more information?

[yyoncho]

@nykma PR would be fine. If you are interested you may also consider contributing a debugger as well.

[yyoncho]

Closing this PR, the remaining servers will be tracked by:
Vue - https://github.com/emacs-lsp/lsp-vue/issues/20
Scala - https://github.com/rossabaker/lsp-scala/pull/9