Rpcs3: RPCS3 not freeing memory after closing a game

[RainbowCookie32]

After closing a game, RPCS3 doesn't seem to free all the memory that was used by it. This causes memory usage to continue growing until the emulator is restarted, which can end up filling RAM after running some games a few times. My test subject here was getting to the menu in Spec Ops The Line, but it seems to happen in all the games I've tested that don't cause an emu crash after closing.

First boot:
Usage ingame: ~1.8GB
Usage after closing the game: ~1.5GB

Second boot:
Usage ingame: ~2.6GB
Usage after closing the game: ~2.3GB

...and so on until you either notice, or you fill memory.

Log in case it's of any use:
RPCS3.log.gz

This is running on popOS by the way, haven't checked this behavior on Windows, but I'd assume it also happens there.

[elad335]

This sounds a regression to me, consider trying random builds from months ago.

[psennermann]

On Windows I can't replicate this problem, anyway 70% of the time that I close a game the emu crashes, so it wouldn't even be a big problem for me ;-)

[incognitoh]

anyway 70% of the time that I close a game the emu crashes, so it wouldn't even be a big problem for me ;-)

same here. 8 out of ten times when i hit the stop button, it freezes and i have to close rpcs3 manually through windows task manager. dont know what they did that ruined the emulator, but it gets frustrating quickly

[RainbowCookie32]

Alright, I'll start the regression testing. The leak doesn't seem to happen in all games by the way, since last night I was testing Battlefield 3 and it didn't have the issue, or at least to the same degree.

I was also able to reproduce with Persona 5, Catherine (to a smaller degree), and Diablo 3. Probably other games too, but I'll go and test old builds now

[RainbowCookie32]

Well, I went back to the oldest AppImage available (0.0.5-7644) and the issue is still there. While overall usage is lower, it still stacks up on each boot.
My guinea pig for this testing was Persona 5, since it ended up being the most stable title so RPCS3 wouldn't crash on exit (Spec Ops The Line no longer met that requirement).

Memory usage after closing P5 (First run):

Memory usage after closing P5 (After 4 more runs):

[kd-11]

Did you try OpenGL?

[RainbowCookie32]

I didn't, will try in a bit.

[RainbowCookie32]

Yes, it also happens with OpenGL.
Memory usage scales to ~1.5GB after booting P5 3 times on latest master using OpenGL, and it keeps growing on further runs

[Darthagnon]

RPCS3 sometimes doesn't free memory on Windows, either: #3080 (had it happen to me: left 2 processes running after closing and it prevented my PC from shutting down, as the processes left behind could not be terminated)

[RainbowCookie32]

That's different though. In that issue, you get a zombie unresponsive process. Here rpcs3 works just fine, but its memory usage keeps going up on each game boot