Spyder: Completions sometimes insert extraneous text

[its-dhung]

Problem Description

When using a completion, sometimes extraneous text is inserted into the buffer.

What steps reproduce the problem?

See the video below:

What is the expected output? What do you see instead?

__Expected:__ this_message should be inserted into the buffer.

__Actual:__ In this case, this_message$is was inserted into the buffer, where the $ denotes the final location of the cursor.

This behavior seems intermittent. Sometimes the correct text is inserted, sometimes the extraneous text is inserted.

Versions

Python 3.7.0 64-bit | Qt 5.9.6 | PyQt5 5.9.2 | Darwin 19.0.0

Dependencies

cloudpickle >=0.5.0          :  1.2.2 (OK)
pygments >=2.0               :  2.4.2 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
sphinx >=0.6.6               :  2.2.1 (OK)
pylint >=0.25                :  2.4.4 (OK)
psutil >=0.3                 :  5.6.5 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
zmq >=17                     :  18.1.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
numpydoc >=0.6.0             :  0.9.1 (OK)
spyder_kernels >=1.8.0;<2.0.0:  1.8.0 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
watchdog                     :  None (OK)
keyring                      :  None (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pympler                      :  None (OK)
sympy >=0.7.3                :  None (NOK)
cython >=0.21                :  None (NOK)
IPython >=4.0                :  7.9.0 (OK)
matplotlib >=2.0.0           :  None (NOK)
pandas >=0.13.1              :  None (NOK)
numpy >=1.7                  :  None (NOK)
scipy >=0.17.0               :  None (NOK)
pyls >=0.31.0;<0.32.0        :  0.31.1 (OK)
applaunchservices >=0.1.7    :  0.2.1 (OK)
rtree >=0.8.3                :  0.8.3 (OK)

[andfoy]

Thanks for reporting this one @its-dhung , however, as you mention this issue is difficult to reproduce; at least I couldn't reproduce it on my machine by trying the aforementioned completion scenario

[andfoy]

What do you think about this one @ccordoba12

[ccordoba12]

I'm leaving this for 4.1.0 to try to find a reproducible use case.

[melsophos]

I have exactly the same bug.

Version

Spyder 4.0.0 with Kite, Python 3.6.9, Fedora 31.

Dependencies

cloudpickle >=0.5.0 : 1.2.2 (OK)
pygments >=2.0 : 2.5.2 (OK)
qtconsole >=4.6.0 : 4.6.0 (OK)
nbconvert >=4.0 : 5.6.1 (OK)
sphinx >=0.6.6 : 2.3.1 (OK)
pylint >=0.25 : 2.4.4 (OK)
psutil >=0.3 : 5.6.7 (OK)
qtawesome >=0.5.7 : 0.6.0 (OK)
qtpy >=1.5.0 : 1.9.0 (OK)
pickleshare >=0.4 : 0.7.5 (OK)
zmq >=17 : 18.1.1 (OK)
chardet >=2.0.0 : 3.0.4 (OK)
numpydoc >=0.6.0 : 0.9.2 (OK)
spyder_kernels >=1.8.1;<2.0.0: 1.8.1 (OK)
qdarkstyle >=2.7 : 2.7 (OK)
atomicwrites >=1.2.0 : 1.3.0 (OK)
diff_match_patch >=20181111 : 20181111 (OK)
intervaltree : None (OK)
watchdog : None (OK)
keyring : None (OK)
pexpect >=4.4.0 : 4.7.0 (OK)
pympler : None (OK)
sympy >=0.7.3 : 1.5 (OK)
cython >=0.21 : 0.29.14 (OK)
IPython >=4.0 : 7.10.2 (OK)
matplotlib >=2.0.0 : 3.1.2 (OK)
pandas >=0.13.1 : 0.25.3 (OK)
numpy >=1.7 : 1.18.0 (OK)
scipy >=0.17.0 : 1.4.1 (OK)
pyls >=0.31.2;<0.32.0 : 0.31.4 (OK)
xdg >=0.26 : 0.26 (OK)

[melsophos]

I may have a hint of the error origin. I have a class with a method self._corr_fig without docstring (not sure if it does matter). When I hit tab after having written self._cor, instead of some additional characters, I get self._corr_fig($1)$0. My suspicion is that some of the last written characters go into this $0 sometimes.

[ccordoba12]

@andfoy, please take a look at this one.

[andfoy]

I may have a hint of the error origin. I have a class with a method self._corr_fig without docstring (not sure if it does matter). When I hit tab after having written self._cor, instead of some additional characters, I get self._corr_fig($1)$0. My suspicion is that some of the last written characters go into this $0 sometimes.

I think this is a separate issue, which I can reproduce

[andfoy]

@melsophos, PR #11400 should fix your aforementioned issue

[andfoy]

@its-dhung @ccordoba12 I'm still unable to reproduce this issue:

[its-dhung]

@andfoy I can reproduce this reliably.

In the video above, the first 2 usages of the completion json result in extra text being inserted. In the 3rd usage, the right text is inserted after moving the cursor around. Not sure if moving the cursor around has anything to do with the root cause though.

Let me know if there's any info I could be logging to help you debug.

[ccordoba12]

@dalthviz, could you reproduce this one?

[dalthviz]

@ccordoba12 I was able to reproduce this by changing the number of characters for automatic completion to 1 and decreasing the show automatic completions keyboard idle to 50:

Seems like the extra characters are the ones typed before the automatic request for completion is done and shown in the completion widget for the first typed letter(s). So for example, if I type j a request will be made and the completion for json will complete son next to the j, but then the rest of the elements typed like s will not be taken into account in the completion and will end up at the end of the completion insert:

import j|  # Autocompletion request is made
---------------------------------------------------------------------------------------------
import js|  # 's' is typed and the completion widget pops up but it's actually showing  
            # completions for 'j'. Accepting the completion would insert elements at 
            # 'import j|'
---------------------------------------------------------------------------------------------
import json|s  # Insertion at 'j|'. End cursor before '|s'

[ccordoba12]

Great work @dalthviz! So this is sort of a race condition in the completion widget.

I think the problem is that Spyder takes some time to display the completion widget, so the additional characters the user types are not passed to the widget. And until the widget appears, they are not passed to the editor either.

A possible solution would be to display the completion widget immediately when users select 1 as the minimum number of characters to show completions. @its-dhung, what do you think?

[ok97465]

If multiple requests occur before displaying a widget, how about displaying the widget only for the last request?

I'm not sure my code is the solution about this problem.
I changed send function of _maybe_send_locked of plugin.py of completion to my code.
I've been using the following code for months.

        def send():
            max_req_id = max(
                [key for key, item in self.requests.items()
                 if item['req_type'] == self.requests[req_id]['req_type']],
                default=-1)

            del self.requests[req_id]

            # Response only to recent requests.
            if req_id == max_req_id:
                self.gather_and_send(request_responses)

[ccordoba12]

If multiple requests occur before displaying a widget, how about displaying the widget only for the last request?

That makes a lot of sense, thanks @ok97465!

@dalthviz, please take a look at @ok97465's patch to see if it solves this issue.

[dalthviz]

@ccordoba12 the patch of @ok97465 fixes the issue for me 🎉

@ok97465 could you open a PR with your patch, please? or if you prefer I could create the PR, whatever works best for you.

Again, thanks for the fix suggestion and feedback @ok97465 !

[ok97465]

@dalthviz Thank you for your review. I'll open a PR.