Spyder: Spyder does not realease memory used for matplotlib plots

[vahid-sb]

This is more of a reminder than an issue I hope to see fixed soon. When I run a code that involved plotting and saving hundreds or thousands of matplotlib figures, Spyder causes a memory leak. In spite of all my efforts in closing the figures, using GC library, etc. But if I run the code from the command prompt, the memory leak is not there.
I asked a question about it in Stackoverflow and based on the comment of one of Spyder developers, I understand this is something you have inherited from Jupyter and might not be solved easily. But maybe in future versions, you can keep that issue in mind and find a solution.
I use Spyder for all my work for various reasons. It is the best IDE that I know of. And this issue is its most serious drawback. And that is a real pity.
https://stackoverflow.com/questions/57409470/spyder-does-not-realease-memory-for-matplotlib-plots

[goanpeca]

Hi @vahid-sb, thanks for the kind words and the reminder.

We will keep track of this.

Cheers

[ccordoba12]

@vahid-sb, I have some questions for you (some of which are present in our issue template, so please don't dismiss it for next time):

• What backend are you using to generate your figures? Inline, Automatic or another one?
• Could you give us a simple code that shows this problem for you?
• What operating system (Windows, Linux, macOS) are you using?
• What Qt/PyQt and Spyder versions are you using?
• What's your Matplotlib version?

[vahid-sb]

@ccordoba12, for sample code, I am not sure if I can. I mostly encounter it while analysing fMRI data using a library I have developed called mitfat which is available on PyPi. And I am not sure it is because of the code itself, or the fact that scripts I use based on this code generate huge number of plots in high resolutions. Nevertheless, since the data I used belongs to a research group who do not want to share their data, I cannot really give you a running code to so obviously see the error. But if I could generate an example with the sample dataset they have let me upload alongside mitfat library, I'd let you know. But if you know of a method to reliably track RAM usage, you should see this issue if you run the sample script and sample dataset that comes with mitfat, only in a smaller scale. Your other questions are answered below. If anything needs more explanations, let me know.

@vahid-sb, I have some questions for you (some of which are present in our issue template, so please don't dismiss it for next time):

* What backend are you using to generate your figures? Inline, Automatic or another one?

Always inline.

* What operating system (Windows, Linux, macOS) are you using?

Ubuntu Bionic

* What Qt/PyQt and Spyder versions are you using?
* What's your Matplotlib version?

I have faced this issue for at least 1.5 years. And I always have all libraries updated to the latest versions, including matplotlib. And I have faced this issue in both Spyder 3 and 4, up to the latest update. So i am pretty sure this is not an issue with only one specific version of matplotlib or spyder. For Qt/PyQt, I am not sure, since I never explicitely use it in my codes.

[ccordoba12]

Nevertheless, since the data I used belongs to a research group who do not want to share their data, I cannot really give you a running code to so obviously see the error

Ok, no problem.

Your other questions are answered below. If anything needs more explanations, let me know

Thanks for your answers. I have one last question for you: what is the process that is accumulating memory without releasing it? The kernel or the Spyder interface process? A kernel process appears like python -m spyder_kernels.console -f ... in top or htop.

[ccordoba12]

@vahid-sb, it seems this is a known issue with the inline backend. See here

https://github.com/ipython/ipython/issues/7270

The proposed solution is

Now to the fix/workaround: call matplotlib.interactive(False) before [your code]

Could you try that and let us know if it works for you? That shouldn't fix completely the problem, but it should help a lot to dimish the leakage.

[vahid-sb]

@ccordoba12 Thanks for letting me know. I will try it, but since these days I work on a completely different problem, it may be a while before I can find some time to run that code. I'd let you know the result as soon as I try this fix.

[je-santos]

@ccordoba12
I'm facing a similar issue, after creating a few thousand plots spyder freezes. Is there a way to limit the number of plots in the plot tab?

Other than that, the new spyder is absolutely fantastic. Thanks a lot for all the hard work.

[ccordoba12]

@je-santos, please read my https://github.com/spyder-ide/spyder/issues/12847#issuecomment-635471875 above. The temporary solution is to add matplotlib.interactive(False) to your code.

Please also let us know if that works for you, so we can start thinking how to include that workaround in our code.

[dsmic]

For me this does not solve the problem. I prepared a simple demonstration code, which I run in spyder 4.1.4 on anaconda. console info:
Python 3.7.7 (default, May 7 2020, 21:25:33)
Type "copyright", "credits" or "license" for more information.

IPython 7.16.1 -- An enhanced Interactive Python.

import matplotlib.pyplot as plt
plt.rcParams['figure.dpi'] = 300

for _ in range(1000):
    plt.interactive(False)
    plt.plot([1,2,3], [1,2,2])
    plt.interactive(False)
    plt.show()
    plt.close()

problem is also present without plt.rcParams['figure.dpi'] = 300 (which increases the resolution of the plots), but uses less memory.

If I remove all plots within spyder ide (the three crosses Ctrl-Shift-W) , the memory is not freed.

Maybe it is really the images, which use the memory in spyder. A rough calculation suggest: 1000 images used 6.1GB, the size of the images were 1500*1035 which would be 4 bytes per pixel?

Thanks a lot for the great IDE, Detlef

[telluroid]

Hi, I have the same issue running miniconda on Linux Mint.
If I run my code through the console or if I run it using IPython inside a console I notice no significant increase in memory usage. If I run the code through spyder though, it will eventually fill my ram until the computer freezes or the kernel restarts.

A kernel process appears like python -m spyder_kernels.console -f ... in top or htop.

Yes, this is what shows up in my htop and what takes up more and more memory.

The temporary solution is to add matplotlib.interactive(False) to your code.

This seems to work for my code.