Streetmerchant: Getting JavaScript memory leak error

[ragermac]

Description

Possible solution

[tcosculluela]

can confirm

<--- Last few GCs --->

[12880:0000019DFD18BC20] 3530069 ms: Mark-sweep 2046.1 (2061.7) -> 2044.7 (2060.3) MB, 39.7 / 0.0 ms (+ 0.0 ms in 1 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 40 ms) (average mu = 0.150, current mu = 0.018) finali[12880:0000019DFD18BC20] 3530111 ms: Mark-sweep 2045.7 (2061.3) -> 2045.6 (2062.3) MB, 40.6 / 0.0 ms (average mu = 0.105, current mu = 0.050) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 00007FF61097DD6C]

Security context: 0x009739000921
1: getData [00000165A8242B99] [H:\git\nvidia-snatcher\node_modules\ws\lib\receiver.js:~337] pc=000001D507DEE92D
2: socketOnData [0000005FF7AC0B29] [H:\git\nvidia-snatcher\node_modules\ws\lib\websocket.js:~871] [pc=000001D...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 00007FF60FDE0C3F napi_wrap+126287
2: 00007FF60FD7DF76 v8::base::CPU::has_sse+34950
3: 00007FF60FD7EC46 v8::base::CPU::has_sse+38230
4: 00007FF6105A803E v8::Isolate::ReportExternalAllocationLimitReached+94
5: 00007FF61058F791 v8::SharedArrayBuffer::Externalize+785
6: 00007FF6104567FC v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1468
7: 00007FF610461B40 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
8: 00007FF61045E634 v8::internal::Heap::PageFlagsAreConsistent+3188
9: 00007FF610453D43 v8::internal::Heap::CollectGarbage+1283
10: 00007FF6104524D4 v8::internal::Heap::AllocateExternalBackingStore+1396
11: 00007FF61047B888 v8::internal::Factory::NewRawOneByteString+216
12: 00007FF61047EA32 v8::internal::Factory::NewStringFromUtf8+130
13: 00007FF6105A494A v8::String::NewFromUtf8+298
14: 00007FF60FCE978F node::tracing::TraceEventHelper::SetAgent+40943
15: 00007FF60FD95F95 v8::internal::Malloced::operator delete+2021
16: 00007FF61097DD6C v8::internal::SetupIsolateDelegate::SetupHeap+44124
17: 000001D507DEE92D
npm ERR! code ELIFECYCLE
npm ERR! errno 134

[jef]

What commit are you guys on? This problem I believe was solved, but may have been wiped.

[tcosculluela]

What commit are you guys on? This problem I believe was solved, but may have been wiped.

Wow, could've sworn I was up to date but I'm 36 commits behind 🤡

[jef]

Gonna close for now unless this continues to be a problem on latest main.

Thanks!

[amarshall]

I still get OOM on fd294d2baa06a1c0a68852497889a0412dea492e; I’m using low bandwidth mode (mentioning since previously ad blocker seems to have been an issue). Can open a new issue if you’d like. I work around it by configuring the container to restart on failure, but obviously that’s not ideal.

[jef]

I'm also getting OOM from time to time. Have to look more into it. Probably some pages not closing overtime.

[FrankFioretti]

I, too, am on the latest commit, am using low bandwidth mode, and got the exact error above.

[jef]

Yeah, definitely an issue. Probably something not getting closed. I'll look into it more.

[EchoJoker]

Been running into this issue recently. Happens around 20-ish hours after I start the script. I attached are the logs and my current configuration.

[jef]

@EchoJoker I updated your comment and remove the log, as it contained personal information. Thanks for the log though and will look into it!

[EchoJoker]

@jef It did? I thought I modified all of that out. Eh oh well, hopefully, it helps.

[LuPro]

I'm also getting some fairly consistent memory leaks - it did crash once or twice a while ago but I unfortunately forgot to store the log.
I'm running streetmerchant in a ubuntu server container in a proxmox hypervisor and gave it 2GB of RAM, right when starting the program I'm using around ~400MB RAM but over time it fills my 2GB very consistently - it can run for about ~9 hours before getting close to the limit:

That's RAM overview over one hour - the high part in the beginning was while it was running for a while, then killed it (drop) then started it again (jump to ~400M) and then slowly rising

here the overview over the last ~day

again, this is a clean ubuntu container, there is absolutely nothing else running on it besides street merchant and it's freshly installed 3 days ago (though it is ubuntu server 18.04lts, have been too lazy to download the 20.04lts)

for now I killed the program regularly and restarted it to "reset" ram usage, but I can leave it running to get another crash out of it to see if I can get some other log data for you all

Edit: It crashed at ~1.7GB ram usage (for the entire system), got a very similar error message as posted above by tcosculluela
Edit 2: Now 9 hours later it ran out of RAM again (so about 9 hours for filling ~1.5GB of ram)