Spyder: Kite (and LSP) completions frequently not displayed

[metalogical]

Problem Description

Kite (and LSP) completions frequently don't show up:

1. start Spyder in debug mode with Kite installed & running
2. type the following

import numpy as np

np.ar‸

1. backspace and type r repeatedly

• sometimes no completions show up
• sometimes ArithmeticError shows up (presumably from LSP?)
• sometimes Kite completions show up
• in all cases, you can see the Kite completions request returns very quickly, and the response is shown in the debugging output

I expect Kite completions (and LSP?) to always show up. As a separate issue, the completions ordering returned by Kite is not respected: I will create a separate issue to track this.

Versions

• Spyder version: Spyder 4.0.0.dev0 (Commit: 11611776d)
• Python version: 3.7
• Operating System name/version: macOS

[ccordoba12]

This probably happens because LSP completions take some time to be computed.

@andfoy, please take a look at this one.

[andfoy]

I don't know how to address this, due to the different character of the requests being made, which have different times. The only solution would be like using a single completion service, either LSP or Kite (Fallback should be always on)

[ccordoba12]

The only solution would be like using a single completion service, either LSP or Kite

This is a good idea! We could make Kite and LSP mutually exclusive so one doesn't affect the other.

@goanpeca, could you do that in PR #10169?

[goanpeca]

This is a good idea! We could make Kite and LSP mutually exclusive so one doesn't affect the other.

Yes!

[metalogical]

I generally agree with this as well. Another (very similar) option that might be more robust:

Define a strict ordering of clients (prefer Kite to LSP, or prefer LSP to Kite) configurable by the user. We try each client in order, and if the client "fails fast" (for Kite, that means ConnectionError), we try the next client in the list. If the client returns completions, we stop.

That way, if the user has "enable Kite" selected (or "prefer Kite to LSP"), but the Kite daemon is not running, we could still fall back to LSP. If Kite returns completions, we don't try LSP.

The downside is that this would require that the LSP server is always running, even if we never end up using it due to Kite providing completions.

Let me know what you think.

[andfoy]

We try each client in order, and if the client "fails fast" (for Kite, that means ConnectionError), we try the next client in the list. If the client returns completions, we stop.

We could do this only with synchronous servers, but LSP is asynchronous, so we don't know if it's going to finish nor we can assert any waiting time for response

[metalogical]

I see. In that case, we could just make it concrete, and say:

• if Kite is preferred over LSP, fall back on ConnectionError to LSP.
• if LSP is preferred over Kite, always use LSP.

[andfoy]

This is a good idea! We could make Kite and LSP mutually exclusive so one doesn't affect the other.

@goanpeca, please keep in mind that there are requests that only can be attended by LSP, such as go-to-definition and workspace-related calls

[ccordoba12]

@andfoy, that means that LSP can't be turned off completely?

[andfoy]

Yes, for instance if we enable code folding support from LSP, that cannot be solved by Kite

[ccordoba12]

You're right. So how can we turn off completions without disabling the rest?

[andfoy]

The proper way to manage this would be to have a configuration settings per plugin that indicates its capabilities in form of a set. Then we would merge all the capabilities, and depending on the availability of a given client and the preference of the user, the call would be redirected to the corresponding client, as opposed to sending the call to all clients.

KITE_HANDLES = {'textDocument/didOpen', 'textDocument/didChange', 'textDocument/hover', 'textDocument/completion'}
LSP_HANDLES = {'textDocument/didOpen', 'textDocument/didChange', 'textDocument/hover', 'textDocument/completion', 'textDocument/foldingRange', 'workspace/didChangeConfiguration', 'workspace/didChangeWorkspaceFolders'}

An exception here would be with respect to the fallback, which should be always be called on completion calls