Spyder: Spyder 4.0.1 editing *really* laggy

[durack1]

Issue Report Checklist

• [x] Searched the issues page for similar reports
• [x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• [x] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• [ ] Could not reproduce inside jupyter qtconsole (if console-related)
• [x] Tried basic troubleshooting (if a bug/error)
  
  
  
  • [x] Restarted Spyder
  
  
  • [ ] Reset preferences with spyder --reset
  
  
  • [x] Reinstalled the latest version of Anaconda
  
  
  • [x] Tried the other applicable steps from the Troubleshooting Guide
  
  
• [x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

After updating from the beta4 to 4.0.1 I am experiencing very laggy keyboard responses in the editor. I have turned off Kite and this doesn't seem to solve the problem, so it appears to be part of the Spyder default (rather than Kite addon) configuration

What steps reproduce the problem?

1. Update spyder to latest 4.0.1
2. Open a new session
3. Start editing a ~1500 line *.py script and watch as key presses are not synced with characters appearing in the editor

What is the expected output? What do you see instead?

Instant keyboard, character appearance

Versions

• Spyder version: 4.0.1
• Python version: 3.7.6
• Qt version: 5.9.7
• PyQt version: 5.9.2
• Operating System name/version: MacOS 10.14.6

Dependencies

applaunchservices >=0.1.7    :  0.2.1 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
cloudpickle >=0.5.0          :  1.2.2 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
intervaltree                 :  None (OK)
IPython >=4.0                :  7.11.1 (OK)
jedi =0.14.1                 :  0.14.1 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
numpydoc >=0.6.0             :  0.9.2 (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
psutil >=0.3                 :  5.6.7 (OK)
pygments >=2.0               :  2.5.2 (OK)
pylint >=0.25                :  2.4.4 (OK)
pyls >=0.31.2;<0.32.0        :  0.31.2 (OK)
zmq >=17                     :  18.1.0 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
rtree >=0.8.3                :  0.8.3 (OK)
sphinx >=0.6.6               :  2.3.1 (OK)
spyder_kernels >=1.8.1;<2.0.0:  1.8.1 (OK)
watchdog                     :  None (OK)
cython >=0.21                :  None (NOK)
matplotlib >=2.0.0           :  None (NOK)
numpy >=1.7                  :  1.17.5 (OK)
pandas >=0.13.1              :  None (NOK)
scipy >=0.17.0               :  None (NOK)
sympy >=0.7.3                :  None (NOK)

[ccordoba12]

This problem should have been fixed in 4.0.1 (at least it was not reported again after that release).

@steff456, could you test how things are in Spyder 4.0.1 and macOS?

[durack1]

@ccordoba12 @steff456 happy to provide some more diagnostic input if it's required. It's a bummer, I was looking forward to moving to the shiny new full release (from b4) and this happened :(

[gabrielclow]

This problem should have been fixed in 4.0.1 (at least it was not reported again after that release).

@steff456, could you test how things are in Spyder 4.0.1 and macOS?

I was the original poster of this issue on Spyder 4.0.0 and I can comment that it is now usable for me but still not ideal. Still experience lag but I'd rather have the dark mode than go back to Spyder 3.

[ccordoba12]

Thanks for chiming in @gabrielclow! You said:

but still not ideal

I'm curious to know under what circumstances this still happens. Could you be more specific?

[gabrielclow]

Thanks for chiming in @gabrielclow! You said:

but still not ideal

I'm curious to know under what circumstances this still happens. Could you be more specific?

It feels like there is still lag in the same situations as before. Adding lines seems to be the most affected, but I haven't tested much specifically. It is smoother than 4.0.0, though. I haven't experienced Spyder freezing because of this anymore, for instance.

[ccordoba12]

Adding lines seems to be the most affected, but I haven't tested much specifically

Do you use the Outline pane? This problem should not appear if that pane is not visible.

[gabrielclow]

Adding lines seems to be the most affected, but I haven't tested much specifically

Do you use the Outline pane? This problem should not appear if that pane is not visible.

I do use it. I just tested without and indeed the lag is much lower. But I just noticed something else: minimizing the Outline pane on new Editor windows does not eliminate the lag. It's as if the Outline pane is still open. Closing the Outline pane will only work in the original Editor pane or using it undocked (to be able to drag it to another display).

[ccordoba12]

But I just noticed something else: minimizing the Outline pane on new Editor windows does not eliminate the lag

@bcolsen, could you take a look at this?

[bcolsen]

I can confirm that there is no way to "hide" the outliner in a new editor window. There needs to be an option for that in the editor window view menu.

[durack1]

This is potentially another issue, but is it by design that you can’t shrink/resize a window, so for e.g. expand the editor to take up 95% of the app screen real estate? When I’ve tried to do this in 4.0.1 I can only expand the editor to 50%, and the terminal/console (which I’m not using) just sits there taking up precious space.

[ccordoba12]

I can confirm that there is no way to "hide" the outliner in a new editor window. There needs to be an option for that in the editor window view menu.

That's true. The Outline pane is always visible in new windows.

[impact27]

@ccordoba12 @steff456 happy to provide some more diagnostic input if it's required. It's a bummer, I was looking forward to moving to the shiny new full release (from b4) and this happened :(

The best way to diagnose this in my opinion is to try to profile spyder:
0 - Make sure you have the most up-to-date version of spyder
1 - Disable "Single instance" in the preferences > general > advanced settings
2 - Create a file with:

from spyder.app import start
start.main()

3 - profile with: Run > Profile (Or F10)
4 - Use the new spyder that appears. Try to use it for a short while (3 min or so) and only do what you think is slow. For example if you think key presses is laggy, spend 3 min typing gibberish in the editor. The idea is that if you repeat enough times, whatever is laggy will take a lot of time and will be noticable in the profiling results. Then you can quit the new spyder.
5 - When the new spyder successfully exited, you can save the profiling results.
6 - These profiling results together with the exact version of spyder can help us understand what is slow. You can have a look yourself but they can be a bit hard to interpret.

What I usually look at is:
main > main > run_spyder > built-in method exec_ > ...

[durack1]

@impact27 the notes above (https://github.com/spyder-ide/spyder/issues/11412#issuecomment-579630720) look very windows-ish, can I assume this doesn't map directly to macOS?

[impact27]

@impact27 the notes above (#11412 (comment)) look very windows-ish, can I assume this doesn't map directly to macOS?

I haven’t tried on windows but I would expect this to work on windows and Linux as well, it only uses spyder features.

[durack1]

I haven’t tried on windows but I would expect this to work on windows and Linux as well, it only uses spyder features.

What profiler are you using? The F10 is not something that I thought would map on MacOS

[impact27]

I haven’t tried on windows but I would expect this to work on windows and Linux as well, it only uses spyder features.

What profiler are you using? The F10 is not something that I thought would map on MacOS

The spyder profiler. You can find it under Run > Profile, or with a keyboard shortcut. The default keyboard shortcut if F10, which on mac is "fn" key + the "no sound" key on your keyboard (It should have a little "F10" on it as well.)

[durack1]

@impact27 @ccordoba12 attached is the output of the profiler, I have no idea what I am looking at, so let me know if I'm not on the right track here
200202_Spyder401Profile.Result.zip

[impact27]

@durack1 are you sure you are profiling spyder? It looks like you are profiling some file called /sync/git/CMIP6_CVs/src/writeJson.py.
You should:

• Profile a file containing:

from spyder.app import start
start.main()

• A new instance of spyder should appear when you click "Profile"
• After using this new instance of spyder for a few 10s of seconds doing exclusively what you think takes too much time, close the new instance of spyder.
• Then save the profiling results.

[AdityaSavara]

Like others, I've had laggy behaviour with 4.0+ but not with 3.0+.
I am working on a file that is 5000+ lines.
My experience is that it was worst when I was editing that file in another editor (notepad++) and then saving it. Spyder would reload the file (which is correct behaviour) but would be very slow in doing so. Also slow in letting me make edits after that.

That was earlier in the day. Just now I tried taking a quick profile after only a few minutes of editing hoping to be able to show you the issue but I don't think this profile shows anything that would be useful since I could not reproduce the behavior noticeably in this quick profile attempt. I'll try profiling spyder again later if I see the issue again.

I am using Spyder 4.0.1 with Windows 10. Python 3.7.
200205ProfileOfFewMinutes.zip

Based on what I have observed, and given the new highlighting features etc. of spyder, my suspicion is it's either the parsing of the file or the graphics in rendering.

One strategy I can suggest is that perhaps some of the GUI refreshing should happen in a way that is proportional to file length and/or some other metrics. If a 10 line file has the GUI refreshing every minute, people will think that's strange. But if that navigation pane or whatever it's called refreshes only over 1 minute or few minutes on my 5000+ file, I won't mind.

Perhaps also add a hotkey that exists under the view menu to refresh the GUI. That way if I happen to be trying to trace a bug and want to force it to refresh faster than a minute after I've typed something, I can press ctrl + (whatever you guys think makes sense).

[impact27]

@AdityaSavara thank you for the profile file.

@ccordoba12 @bcolsen
I could see two things in the profile that looks like it takes too much time. Any ideas?
The LSPRequestTypes.DOCUMENT_FOLDING_RANGE process time is bad. (17 seconds for 27 events):
spyder/plugins/editor/widgets/codeeditor.py::CodeEditor.handle_folding_range (16.6s)
-> spyder/plugins/editor/panels/codefolding.py::FoldingPanel.update_folding (16.2s)

The SnippetsExtension takes some time on every keyPressEvent:
spyder/plugins/editor/extensions/snippets.py::SnippetsExtension.cursor_changed (5 sec for 151 events)

[impact27]

We should maybe add an option to disable folding ranges or process them in a thread?

@AdityaSavara one thing you can try is to locate in C:\\Users\\fvs\\AppData\\Local\\Continuum\\anaconda3\\envs\\uncertainties\\lib\\site-packages\\spyder\\plugins\\editor\\widgets\\codeeditor.py:

    @handles(LSPRequestTypes.DOCUMENT_FOLDING_RANGE)
    def handle_folding_range(self, response):
        """Handle folding response."""
        try:

And add a return:

    @handles(LSPRequestTypes.DOCUMENT_FOLDING_RANGE)
    def handle_folding_range(self, response):
        """Handle folding response."""
        return
        try:

This will disable folding ranges, but might help with the lag. Could you let us know it that solves your problem?

[AdityaSavara]

Thank you for your volunteered time and for taking this seriously since it does affect the user experience for some users!

a) I disabled the folding_range as suggested. I believe there was a big speed improvement that I could notice directly. I agree that a separate thread ( or turned off for long files) would be appropriate.

Another option might be a popup saying "folding ranges and other GUI features will be disabled due to the length of your file. Click OK below or click KEEP ALL GUI FEATURES ON . Remember this decision? You can change this setting at a later time under ___."

b) In this profile, I also reproduced one of the laggy behaviours that is new to spyder 4.0, and it occurs even when folding ranges are off. From outside of spyder, I saved over the python file I was editing. Spyder correctly reloaded the file, but practically froze up for a couple of seconds while doing so and then moved me back to the top of the file. (I don't think the position used to move in Spyder 3.0, I think spyder retained where I was working in Spyder 3.0. The screen would just blink is my memory).

c) though I don't find this as critical, it is worth noting that even just typing in text in spyder 4.0 for that large file is lagging. Right now it's not enough for me to really complain about, but it's noticeable. While the computer I'm working on isn't fast, it's not really a slouch. it's an i7-7600 with 16GB of ram and I don't have much else open when I'm working.

Attached is the new profile from again playing around a bit with folding_range off.
200206ProfileOfFewMinutes.zip

For anyone else who tries to make profile files like this... I kept having to delete spyder.lock and make sure you run your new spyder instance by the profile run and not a regular run.

[luwil6man]

I have experienced severe lagging since upgrading from Spyder 3 to Spyder 4 on ONLY 1 of 3 computers on which I frequently use Spyder. Hopefully some of the following information based on observation is of some use, let me know if any technical information is desired.

• All 3 computers are PCs and have Spyder 4.0.1 installed, and are used for the same tasks
• The highest performing PC is the one with lag.
• number of lines in the file, and the number of files open does not effect the severity of lag (this is inconsistent with other comments, although these comments say the lag is "mild", and I have described it as severe)
• lagging gets progressively worse until ~ 1 hour after launching, which is when the degree of lag seems to reach a "steady-state".
• the lag is experienced for all keyboard use in the editor (tabbbing, spacing, backspacing, new lines regular character typing
• it is not experienced in the console
• disabling all linting and completion in addition to closing panes (with the exception of the editor and console) does not effect the severity

[AdityaSavara]

@luwil6man
You wrote:

• disabling all linting and completion in addition to closing panes (with the exception of the editor and console) does not effect the severity

Do you mean that you tried disabled the folding pane by editing the source code, by adding a return statement as described in the comment up above by impact27? If not, can you try doing that?

[luwil6man]

@AdityaSavara
Since adding the return statement, Spyder's performance has significantly improved. I There is no longer a noticeable lag.

[durack1]

@ccordoba12 which release will this fix likely land (and when)?

[goanpeca]

@durack1 4.1 (end of next week)

[durack1]

@ccordoba12 @goanpeca thanks for solving this with 4.1.0/4.1.1 folks, I have just noted a bunch of other issues have popped up, but lagginess for me is a thing of the past!

[ccordoba12]

Great to know! Thanks for letting us know about it!

[ccordoba12]

I have just noted a bunch of other issues have popped up

Please open new issues about them.

[durack1]

@ccordoba12 to be honest, after many minutes of use some lagginess started to return again. It seems the mac laptop was under load. I'll have to see if this starts to niggle at me again, and if so, I'll open a new issue

[davidlee321]

Experiencing a lot of lag editing a (short) .md file on Spyder 4.2.0.

Spyder 3.3.6 feels fine.

OS: MacOS Big Sur. Macbook Pro 2019 (Intel chip)

[ccordoba12]

@davidlee321, that's a bug with Big Sur, which was fixed in 4.2.1