Go-whatsapp: Memory usage with session restoration

[x00b]

Hello guys,

I am having a _weird_ problem after making some experiments with the library(the current master branch)..

I have a session stored and a function that keeps restoring(every X minutes) this session with:

newSession, err = wac.RestoreWithSession(storedSession)

but if err != nil(cellphone offline or app was unauthorized in wpp app) , the cpu usage goes up(and doesn't go down afterwards, only if app os.Exit()) and as i mentioned, this restore process runs forever and this is causing CPU usage to go up to almost 100% sometimes and the process gets killed by the kernel.

And just to mention, if err == nil things run just fine and cpu keeps normal.

Does anyone ever had some similar problem or an idea of why this is happening?

[cht368]

Yes I have this issue too, memory and goroutine keeps leaking.

Edit: I tried wac.Disconnect() before return and it seems to lessen the leak

[beshoo]

+1

[x00b]

Yes I have this issue too, memory and goroutine keeps leaking.

Edit: I tried wac.Disconnect() before return and it seems to lessen the leak

That's a good temporary semi-solution. But we really have to find where exactly this bug is happening on the go-whastapp. Does anyone have any clue?

Ps: I thought about calling debug.FreeOSMemory() where you called wac.Disconnect(). But again, this would be a hack not a real solution for the bug.

[cht368]

I looked for the leak and it seems keepAlive() and readPump() didn't closed when connection closed

[x00b]

I looked for the leak and it seems keepAlive() and readPump() didn't closed when connection closed

Thats weird.

defer func() {
    wac.wg.Done()
    _, _ = wac.Disconnect()
}()

Disconnect() should've close the websocket. Right?

Are using pprof to analyze it?

[cht368]

Yes, I do.

    stack := debug.Stack()
    w.Write(stack)
    pprof.Lookup("goroutine").WriteTo(w, 2)

both keepAlive and readPump keep stacking over time if don't disconnect the conn

[x00b]

I see.. It seems to work as you said _wac.Disconnect() before return seems to lessen the leak_.
Gonna keep an eye on this for next days

Thanks! :+1:

[beshoo]

@x00b but would you think this has to be implemented in the go-whatsapp lib, not in the user app!
And if true, why https://github.com/Rhymen/go-whatsapp/issues/328#issuecomment-613501616 is not fired!

[cht368]

It seems readPump() didn't returned and https://github.com/Rhymen/go-whatsapp/issues/328#issuecomment-613501616 never get executed.
I don't know why yet but wac.ws.close didn't received anything when I tried to restore with session.

[beshoo]

I believe this has to be fixed on the base lib.
Work around solutions will not help.

[x00b]

I believe this has to be fixed on the base lib.
Work around solutions will not help.

Yes, i agree. This can lead to a stack overflow if the Restore process runs forever.

For what i tested yesterday the wac.Disconnect() after wac.RestoreWithSession() really keep things normal so we got find why the readPump() is not closing when called by Restore(and the session cannot be restored).

Anyone have any clue?

[juliowerner]

I was using this approach of disconnect but sometimes it got stuck on disconnect...

[beshoo]

I was using this approach of disconnect but sometimes it got stuck on disconnect...

Agree, i have same problem, that is why we need to focus on the base lib. @Rhymen

[juliowerner]

Maybe read pump get also stuck on disconnect, but I can't figure out why...

[beshoo]

@juliowerner, the PL closed, what is that mean? Is it accepted or not?

[juliowerner]

Unfortunately, The PL is not working, I ran out more tests and disconnect function still getting stuck. :(

[beshoo]

Question: if we have a server with 24 cpu core, how can this help this lib? As long as we are talking about cpu useg.

Can this api use all cpu thirds?
Is there any load balance method that handles this?

@Rhymen

[juliowerner]

I think we are talking about memory usage at least the title was changed.
Issue #282 pointed out already about why so huge buffer is used.
Currently, I set the buffer to zero for testing and everything is going fine...

[beshoo]

We have to find why this happens, I have one session, and the supervisor is monitoring the service
look how many times Linux killed the proc.

2020-04-11 12:34:45,185 INFO spawned: 'whatsapp_api_00' with pid 18456
2020-04-11 12:43:58,894 INFO spawned: 'whatsapp_api_00' with pid 32226
2020-04-11 12:52:48,194 INFO spawned: 'whatsapp_api_00' with pid 13332
2020-04-11 13:39:39,063 INFO spawned: 'whatsapp_api_00' with pid 19236
2020-04-11 14:37:15,471 INFO spawned: 'whatsapp_api_00' with pid 12711
2020-04-11 14:47:01,607 INFO spawned: 'whatsapp_api_00' with pid 31240
2020-04-11 14:51:17,522 INFO spawned: 'whatsapp_api_00' with pid 6908
2020-04-11 15:01:05,498 INFO spawned: 'whatsapp_api_00' with pid 23252
2020-04-12 03:33:18,424 INFO spawned: 'whatsapp_api_00' with pid 1799
2020-04-12 03:39:30,516 INFO spawned: 'whatsapp_api_00' with pid 12986
2020-04-12 03:41:44,715 INFO spawned: 'whatsapp_api_00' with pid 16910
2020-04-12 03:42:39,940 INFO spawned: 'whatsapp_api_00' with pid 18542
2020-04-14 07:58:50,431 INFO spawned: 'whatsapp_api_00' with pid 15192
2020-04-16 16:33:34,442 INFO spawned: 'whatsapp_api_00' with pid 11353
2020-04-16 17:39:36,882 INFO spawned: 'whatsapp_api_00' with pid 29558
2020-04-16 17:49:29,762 INFO spawned: 'whatsapp_api_00' with pid 14900
2020-04-16 20:20:11,328 INFO spawned: 'whatsapp_api_00' with pid 19132
2020-04-16 21:28:17,911 INFO spawned: 'whatsapp_api_00' with pid 28063
2020-04-16 21:42:01,391 INFO spawned: 'whatsapp_api_00' with pid 20580
2020-04-17 02:44:31,816 INFO spawned: 'whatsapp_api_00' with pid 14435
2020-04-17 04:23:30,306 INFO spawned: 'whatsapp_api_00' with pid 32387
2020-04-17 04:44:35,750 INFO spawned: 'whatsapp_api_00' with pid 4088
2020-04-18 14:23:47,408 INFO spawned: 'whatsapp_api_00' with pid 16948
2020-04-18 14:25:54,277 INFO spawned: 'whatsapp_api_00' with pid 21232
2020-04-18 14:30:28,252 INFO spawned: 'whatsapp_api_00' with pid 29981
2020-04-18 15:35:16,658 INFO spawned: 'whatsapp_api_00' with pid 31533
2020-04-20 17:44:23,203 INFO spawned: 'whatsapp_api_00' with pid 27712
2020-04-20 21:20:22,914 INFO spawned: 'whatsapp_api_00' with pid 9276
2020-04-20 21:56:24,358 INFO spawned: 'whatsapp_api_00' with pid 6460
2020-04-21 04:36:23,852 INFO spawned: 'whatsapp_api_00' with pid 22379
2020-04-21 08:27:38,265 INFO spawned: 'whatsapp_api_00' with pid 18387
2020-04-21 16:03:50,016 INFO spawned: 'whatsapp_api_00' with pid 1648
2020-04-23 16:27:50,806 INFO spawned: 'whatsapp_api_00' with pid 4021

[juliowerner]

Are you using CheckCurrentServerVersion ?

[beshoo]

Are you using CheckCurrentServerVersion ?

No,