Uwsgi: Python memory leak in uWSGI 2.0.18

[gnb]

After upgrading some dependencies, including uWSGI from 2.0.17.1 to 2.0.18, a couple of months back we noticed our container memory usage growing. Using pyrasite and meliae we narrowed it down to many thousands of copies of the app dict and containing tuple, created by this code https://github.com/unbit/uwsgi/blob/d1338dfefe3d9a2de466a9a695cb61fdd055a295/plugins/python/uwsgi_pymodule.c#L2278
One process had accumulated 955313 copies of this dict over 12 days.

[xrmx]

AFAICT this code is only called by external code. Is your code calling uwsgi.workers()?

[gnb]

Hi Riccardo, it turns out we are. There was a commit in our uwsgi wrapper library which changed some code for emitting metrics on uwsgi worker status, from using the JSON dump from the uwsgi stats socket to calling uwsgi.workers() and parsing the output of that. We got that commit in the same upgrade.

Nevertheless I don't see why that data structure should leak. Is there a refcounting issue in uwsgi.workers() ?

[awelzel]

Hey @gnb - seems like uwsgi.workers() does in fact leak. Would you be able to test the referenced commit?

@xrmx - is there an easy way to add an app that acts as a "regression test" to the travis setup?

[xrmx]

@awelzel the easy way is cut and paste what tests/travis.sh. Would be nice to read the regression tests to run from a file though.

[awelzel]

@xrmx - thanks for the pointer. I've integrated the test into travis.sh in #2057 (3904619).

[gnb]

Hi @awelzel I can't test this in production, we've already pushed a change which stops calling uwsgi.workers(). The best I can do is some throw-away code that I could run in a dev environment which calls that method in a loop and measures the memory impact. I don't think that would provide any value over your unit test. Sorry.

The change looks good BTW.

[gnb]

@awelzel I don't see your commits in the main repo, is that going to happen sometime?