Spyder: Crash in fallback plugin when typing character

[suzannejb]

Description

What steps will reproduce the problem?

I have an animated gif available. I have many things loaded into memory and just type a character and Spyder crashes.

Traceback

  File "C:\Users\suzbm\Anaconda3\lib\site-packages\spyder\plugins\completion\fallback\actor.py", line 155, in handle_msg
    msg['current_word'])
  File "C:\Users\suzbm\Anaconda3\lib\site-packages\spyder\plugins\completion\fallback\actor.py", line 62, in tokenize
    valid = is_prefix_valid(text, offset, language)
  File "C:\Users\suzbm\Anaconda3\lib\site-packages\spyder\plugins\completion\fallback\utils.py", line 125, in is_prefix_valid
    current_pos_text = text[offset - utf16_diff - 1]
IndexError: string index out of range

Versions

• Spyder version: 4.1.2
• Python version: 3.7.4
• Qt version: 5.12.5
• PyQt5 version: 5.12.3
• Operating System: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0           :  1.3.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.3.0 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.12.0 (OK)
jedi =0.15.2                   :  0.15.2 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  0.9.2 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso =0.5.2                   :  0.5.2 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.0 (OK)
pygments >=2.0                 :  2.5.2 (OK)
pylint >=0.25                  :  2.4.4 (OK)
pyls >=0.31.9;<0.32.0          :  0.31.9 (OK)
qdarkstyle >=2.8               :  2.8 (OK)
qtawesome >=0.5.7              :  0.7.0 (OK)
qtconsole >=4.6.0              :  4.6.0 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
rtree >=0.8.3                  :  0.9.4 (OK)
sphinx >=0.6.6                 :  2.4.3 (OK)
spyder_kernels >=1.9.0;<1.10.0 :  1.9.0 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.0 (OK)

# Optional:
cython >=0.21                  :  0.29.15 (OK)
matplotlib >=2.0.0             :  3.1.3 (OK)
numpy >=1.7                    :  1.18.1 (OK)
pandas >=0.13.1                :  1.0.1 (OK)
scipy >=0.17.0                 :  1.4.1 (OK)
sympy >=0.7.3                  :  1.5.1 (OK)

[suzannejb]

[ccordoba12]

Thanks a lot the detailed report @suzannejb! Could you upload the preprocessing.py file that you're showing above (not its contents) here, so we can try to reproduce the bug in our side? Thanks!

[ccordoba12]

Pinging @suzannejb again. Could you help us here by providing preprocessing.py?

[demattox]

I have the exact same issue arising when typing any character in a ~400ish line python script:

File "/Users/dmattox/anaconda3/lib/python3.7/site-packages/spyder/plugins/completion/fallback/actor.py", line 155, in handle_msg
msg['current_word'])
File "/Users/dmattox/anaconda3/lib/python3.7/site-packages/spyder/plugins/completion/fallback/actor.py", line 62, in tokenize
valid = is_prefix_valid(text, offset, language)
File "/Users/dmattox/anaconda3/lib/python3.7/site-packages/spyder/plugins/completion/fallback/utils.py", line 125, in is_prefix_valid
current_pos_text = text[offset - utf16_diff - 1]
IndexError: string index out of range

In probing a little more, this occurs when I type anything into any of the three scripts I have open as tabs. Tried closing spyder and error occurs again on reopening.

• Spyder version 4.1.3 [conda installed]
• Python version 3.7.0
• qt 5.12.5
• pyqt 5.12.3
• Mac OS Catalina 10.15.3

[ccordoba12]

@demattox, please upload one of your scripts here to see if we can reproduce this problem in our side.

[demattox]

@ccordoba12 Sorry, where should I upload it? I tried dragging and dropping it here and it says it is a unsupported file type. It is available on github already if that works: https://github.com/demattox/plip/blob/stable/plip/modules/plipxml.py

[ccordoba12]

It is available on github already if that works

Thanks, that works great!

Could you also create an animated gif (for instance with the Licecap program) and upload it here (there shouldn't be issues with that)? That way I could see at what exact point (line and column) you're getting the error.

Finally, please let us know what operating system, Python, Qt and PyQt versions you're using. Thanks!

[demattox]

Spyder version 4.1.3 [conda installed]
Python version 3.7.0
qt 5.12.5
pyqt 5.12.3
Mac OS Catalina 10.15.3

[demattox]

@ccordoba12 Sorry, that GIF got sped up from the screen capture, not sure how. But hopefully it shows that typing a comment did not produce an error, but then typing any random characters on the same line gives the error. And then picking another line at random and typing gives the error, as well as backspacing.

[ccordoba12]

No worries, i think it's fine.

[demattox]

@ccordoba12 So I made a potentially interesting and hopefully helpful observation. It seems this error does not arise when I've split horizontally, but after closing the split panel, the error occurs again. And I noticed the tab selector button options (not sure the name for them, the square buttons to the left and right of the tab names) are highlighted when working in a single pane, but look normal in the split pane. Might be unrelated though.

Please let me know if there's anything else I can do.

[atiyabzafar]

@demattox I have same issue here in spyder
@ccordoba12 Please take note of the issue. It started happening just an hour ago after a restart. As per my knowledge I have not updated anything since my last start.

Spyder version 4.1.2
Python 3.7.7 64-bit | Qt 5.9.6 | PyQt5 5.9.2 | Windows 10
Using Kite for indexing

Here is the gif showing the error.

Also i tried it with horizontal splitting and the problem goes away.

[demattox]

Thank you for your work! I'm a big fan of spyder :)

[ccordoba12]

@demattox, thanks for your kind words! And also thanks for your patience while we fixed this bug.