Cura 3.5.1 Excessive Memory use with a running print and just the webcam stream open.

[DuckY1987]

Application Version

Cura 3.5.1

Platform

Windows 10 Latest version.

Printer

Octoprint

Steps to Reproduce

I'm not quiet sure how to reproduce it, but to my printer is a webcam attached, when i leave cura open just with the webcam and an object in print after a while the memory starts getting used excessive.

Actual Results

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: Cura.exe (12704) consumed 51035406336 bytes

And not just the pagefile is getting excessive used, my ram is completely used by cura too.

Windows restarts different applications others are crashing incl. cura.

Expected results

Additional Information

[fieldOfView]

See also https://github.com/fieldOfView/OctoPrintPlugin/issues/88

[fieldOfView]

I have found steps to reproduce, see the issue linked above.

[tenpaiyomi]

+1 on this as well. Windows 10, 16GB of RAM, i7 8700, and leaving Cura open for an extended period of time after starting a print will bring my whole system crawling eventually.

If there is anything I can send or do to help, feel free to ask.

[mvrahden]

Same also happening on MacOS.

It floated my RAM and HDD (Swap Memory exploded) and forced me to make a hard reboot.

System Specs:
OS Version: macOS High Sierry 10.13.6
Processor: 3GHz Intel Core i7
Memory: 16GB DDR3 1600MHz

Cura Versions (with observed Memory Leak):
3.5.0 & 3.5.1

How to reproduce?:
Run a print with OctoPrint plugin and have the Webcam stream running. After a while memory starts bloating (>>50GB).

[fieldOfView]

After a lot of debugging, it looks like this is an issue in Qt 5.10; I can reproduce it with Qt 5.10, but once I upgrade to Qt 5.11 the leak does not seem to happen.

Are there plans to upgrade to PyQt 5.11 for Cura 3.6?

[ChrisTerBeke]

For 3.6 probably not as there's too little time, but maybe for the release after that?

[fieldOfView]

So 3.6 will leak heaps of memory too :-/

Note that I see no reason in the code why this wouldn't also affect Cura Connect.

[LipuFei]

@fieldOfView There won't be a Qt upgrade for 3.6. We already ran into some problems upgrading to 5.10, because 5.10 dropped support for older Mac OS X versions (< 10.10). According to this page, Qt 5.11 only supports OS X 10.11+, so this is also a big concern in upgrading Qt.

What I found so far is that the using the Image widget in QML to refresh image frames will cause this problem. I made an alternative implementation for the camera view in this branch https://github.com/Ultimaker/Cura/tree/CURA-5821_fix_camera_memory_leak, but it's not perfect: It doesn't take much memory but does use more CPU power. We will discuss this with the printer cluster team as well.

[fieldOfView]

Thanks for letting me know you're on it too. I have just started to look in to passing base64 encoded data URIs to QML to skip the QQuickImageProvider, but that leaks too. I will see if I can implement your workaround in the OctoPrint plugin.

[fieldOfView]

Another way to solve the issue is to use Qt Multimedia to play the motion jpeg stream instead of picking the motion jpeg stream apart into jpeg frames. Last I tried that, I ran into an issue that in PyQt on Windows, Qt Multimedia is missing a library.

[LipuFei]

Yes, that will be a proper solution. I couldn't find any builtin support for MJPEG streams so far except using GStream. I guess that's the one you miss on Windows.

[LipuFei]

My findings are that the QQuickImageProvider for providing image://camera/* and QImage.loadFromData() actually don't cause problems, but as soon as you change the source of the Image widget in an QML, it will occupy a lot more memory. More details: I lower the frame rate to ~3/sec, and each frame is a JPG image of ~160K, and every time the source gets changed, it takes ~3MB more memory.

[LipuFei]

JIRA ticket: CURA-5821

[fieldOfView]

Just trying to import QtMultimedia gives me this error: plugin cannot be loaded for module "QtMultimedia": Cannot load library ...\Python35\lib\site-packages\PyQt5\Qt\qml\QtMultimedia\declarative_multimedia.dll: The specified module could not be found.

[LipuFei]

In our build system, we skip qtmultimedia in building qt for Linux and Mac. For windows we don't compile Qt but run pip install PyQt5==5.10, so I'd expect it to be there on Windows.

[Razwer]

@fieldOfView There won't be a Qt upgrade for 3.6. We already ran into some problems upgrading to 5.10, because 5.10 dropped support for older Mac OS X versions (< 10.10). According to this page, Qt 5.11 only supports OS X 10.11+, so this is also a big concern in upgrading Qt.

What I found so far is that the using the Image widget in QML to refresh image frames will cause this problem. I made an alternative implementation for the camera view in this branch https://github.com/Ultimaker/Cura/tree/CURA-5821_fix_camera_memory_leak, but it's not perfect: It doesn't take much memory but does use more CPU power. We will discuss this with the printer cluster team as well.

Interesting approach. Keeping support for old stuff while people with latest and greatest and willing to invest into hardware can suffer? I know this sounds blunt and very black and white. But I mean, come on. Technology progresses and not progressing is going backwards.
Memory leaks are serious issues, also from a security vulnerability perspective.

Granted, Cura is free to use and thus I have no rights to make demands, but I can voice my opinion :)

[fieldOfView]

We're not out of options to work around the memory leak yet.

[fieldOfView]

I have created a new component that just takes a source url for an mjpeg stream, and does not rely on NetworkCamera or signalling between PyQt and QML to update the image. This seems to perform as well as - or better than - the original NetworkCamera -> CameraImageProvided -> QML Image route (both in terms of CPU as RAM usage):
https://github.com/fieldOfView/OctoPrintPlugin/commit/8d2003f0cf9313f96aa13e4980b02df92c63330d#diff-dfed51f60d30f888e177003c4dd98e48

I have implemented it inside OctoPrintPlugin, so it can be more easily tested (without requiring private builds). If it works, I can create a PR to include NetworkMJPGImage in Cura (and remove NetworkCamera and CameraImageProvider).

[Razwer]

I have created a new component that just takes a source url for an mjpeg stream, and does not rely on NetworkCamera or signalling between PyQt and QML to update the image. This seems to perform as well as - or better than - the original NetworkCamera -> CameraImageProvided -> QML Image route (both in terms of CPU as RAM usage):
fieldOfView/OctoPrintPlugin@8d2003f#diff-dfed51f60d30f888e177003c4dd98e48

I have implemented it inside OctoPrintPlugin, so it can be more easily tested (without requiring private builds). If it works, I can create a PR to include NetworkMJPGImage in Cura (and remove NetworkCamera and CameraImageProvider).

i just updated my octoprint plugin from your github master branch. My Octoprint instance on my CR-10S where i have the Fullscreen Plugin (0.0.4) installed it works fine.
My second octoprint instance (Ender3) i do not have this plugin installed and the monitor window just shows the status but no webcam.

[fieldOfView]

@Razwer, let's continue this discussion on the OctoPrintPlugin issue queue

[Razwer]

@Razwer, let's continue this discussion on the OctoPrintPlugin issue queue

ok, any further issues i will discuss there. I didn't verify if the webcam checkbox was still checked (and it wasn't, got unchecked during the upgrade process) so please ignore my foolish post.

[fieldOfView]

A testing version of OctoPrintPlugin with the abovementioned fix can be downloaded here:
http://files.fieldofview.com/cura/OctoPrintPlugin-v5-2018-10-25T13_58_45Z.curapackage
After downloading, drop the file into a running Cura application and restart Cura. It would be nice to hear if this fixes the memory leak for you when using the OctoPrint Connection plugin.

[fieldOfView]

I have submitted #4656 to apply the same fix in a more generalized manner to Cura Connect.

[diegopradogesto]

This should be fixed in the upcoming Ultimaker Cura 3.6