Core: Memory Leak in HandleGetUpdates() ???

[jonofe]

Hi,

just implemented a small daemon based on php-telegram-bot to receive text messages sent to my bot.

I realized, that the memory usage is steadily growing, up to 100% if you are waiting for a while (depending on the memory you have installed).

Here is my code:

<?php 

set_time_limit(0);

require (dirname(__FILE__)."/php-telegram-bot/vendor/autoload.php");

use Longman\TelegramBot\Request;
use Longman\TelegramBot\Telegram;
use Longman\TelegramBot\Entities\Update;
use Longman\TelegramBot\Entities\Message;
use Longman\TelegramBot\Entities\InlineQuery;


$apiKey = '#######################################################';
$botName = '*********';

$telegram = new Telegram($apiKey, $botName);

// Enable MySQL
$mysql_credentials = array(
    'host'     => '127.0.0.1',
    'user'     => 'root',
    'password' => '########',
    'database' => 'Telegram',
);
$telegram->enableMySQL($mysql_credentials);

// polling for new messages
while (1)
{
    $serverResponse = $telegram->handleGetUpdates();
    if ($serverResponse->isOk()) 
    {
        $receivedTelegrams = $serverResponse->getResult();
        if (count($receivedTelegrams))
        {
            foreach ($receivedTelegrams as $tg)
            {
                $updateContent = $tg->getUpdateContent();
                if ($updateContent instanceof Longman\TelegramBot\Entities\Message)
                {
                    $text = preg_replace('/\s+/', ' ',trim(strtoupper($updateContent->getText())));
                    $chatId = $updateContent->getFrom()->getId();
                } else if ($updateContent instanceof Longman\TelegramBot\Entities\InlineQuery)
                {
                    $text = preg_replace('/\s+/', ' ',trim(strtoupper($updateContent->getQuery())));
                    $chatId = $updateContent->getFrom()->getId();
                } else if ($updateContent instanceof Longman\TelegramBot\Entities\ChosenInlineResult)
                {
                    $text = preg_replace('/\s+/', ' ',trim(strtoupper($updateContent->getQuery())));
                    $chatId = $updateContent->getFrom()->getId();
                }
                echo "$chatId: $text\n";
            }
        }
    }
    usleep(100000);
}


?>

Any ideas, where could be the problem?

Best regards
André

[noplanman]

I don't think it's the root cause, but do you really need to run the update commands 10 times per second?
usleep takes microseconds, not milliseconds!

[jonofe]

I only reduced it to 100ms to reproduce the memory problem in a shorter time period.
Normally I do it once per second. Because I'm using it for home automation, i.e. switching lights or something else with a telegram message.

What else could be the reason? I'm executing only that code above, without receiving any telegrams and with top or ps u can see the memory is steadily growing. And I experience the same, even if I'm commenting out the entire "if" statement, i.e. only having

$serverResponse = $telegram->handleGetUpdates();

in an endless loop. I have no idea how to find the root cause. :(

[noplanman]

Searching online I found this old article discussing memory leakage in objects. Maybe this could be a part of the problem, not sure.

@MBoretto @akalongman @jacklul
You guys have any experience with this?

[jonofe]

Interesting is, that

memory_get_usage();

inside the endless loop does not show a memory leak. It is constant as long as no telegram is received.
But anyway, top and ps shows the increasing memory usage of the corresponding php process.

[MBoretto]

So if i understand correctly, start run out of memory just when you receive updates?

[noplanman]

@MBoretto That's how I understand it too.
Take a look at this comment. Maybe if we _destroy_ the objects "properly" it wouldn't happen? Haven't tested this yet.

[jonofe]

It also happens if I do not receive updates.
My comment was only related to the memory_get_usage() functions, which do not show additional memory usage, but top and ps still do. And that's valid even without receiving updates.

[jonofe]

any news or ideas on that issue?

[MBoretto]

Need time time to dive into the code!

[UnsupportedCallbackException]

@jonofe why you ignore WebHook way? With self-signed certificate I think it's best approach. You get your script running only when you have updates.

Try to use cron task to start your script once per minute and limit life time for your script for same period.

For current code I also suggest you move following lines inside loop:

$telegram = new Telegram($apiKey, $botName);
$telegram->enableMySQL($mysql_credentials);

Or at least recreate $telegram each 1000 iteration or once per minute. It's should release all resources as long as it's only object that may cause leak in your snippet.

[jonofe]

I can't use WebHook, because it is a plugin in home control system, which is security sensitive, so I cannot expose this server to the internet.

Second option is exactly what I've chosen so far. I'm restarting the script on regular basis. I consider that as a work around. Better would be an implementation without memory leak. :)

Last option does not work. Recreating the telegram object does not free the ressources, which were allocated due to the memory leak. It only destroys the telegram object. But the reason for the memory leakage is most probably in some of the sub-objects.

[UnsupportedCallbackException]

@MBoretto according to this page I think you need to close $verbose temp file.

I also suggest close curl handler before throwing TelegramException. I can catch that exception, but $ch still opened

But not sure if it only the reason

[MBoretto]

@commfort thanks for your suggestion! I pushed in develop fix for what concern the curl handler.
You are right also for $verbose temp file but in @jonofe case logging is not used.
Anyway I will fix it in the monolog brach.

@jonofe what happen if you don't use the database?
Can you quantify if the memory grow faster with/without database?
Thanks for you patience

[noplanman]

@jonofe Are you still experiencing the memory leak with the latest version of the library, 0.33?

[jonofe]

Yes, still the same. Yesterday I tested with the most current release. in 12h it grow up to 1.1GB memory allocation according to what I can see in "top".

[jonofe]

@MBoretto: How can I use it without database? My understanding was, when using in polling mode, I have to use the database, right?

[noplanman]

Yes, using getUpdates requires the database to store the updates, so that it doesn't fetch the same ones again and again.

I'll dig deeper into this and try to find out where the leak is occurring.
Sorry that it's taking this long 😕

[noplanman]

I can't seem to reproduce the problem, my PHP is cleaning up nicely and the memory usage is constant.

Could you please let us know what PHP version you're using and anything else that would help to create a similar environment to what you have.

[jonofe]

As I have to use for my specific purpose a php 5.3.3, I had to backport php-telegram-bot (especially the array declarations). But to cross check, that my backport is not the reason for the memory leak, I tried it on another machine with a newer PHP and the official php-telegram-bot version (latest one):

Here my php Version:

PHP 5.6.10 (cli) (built: Nov 14 2015 12:27:38)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies

Same effect on that machine wrt the memory leak and I used exactly the source code I posted above.

Here an example output of a "ps" every second:

root 24028 1.1 0.4 347404 33344 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347404 33344 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347536 33344 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347536 33344 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347536 33344 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347668 33608 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347668 33608 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347800 33608 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347800 33608 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347800 33608 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347932 33872 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 347932 33872 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348064 33872 pts/1 R+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348064 33872 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348064 33872 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348196 34136 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348196 34136 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348328 34136 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348328 34136 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348328 34136 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348460 34400 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348460 34400 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348592 34532 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348592 34532 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348592 34532 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348720 34656 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348720 34656 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 348828 34768 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 348828 34768 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 348544 34604 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349068 34864 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 349068 34864 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 349068 34864 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 349068 34864 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.2 0.4 349200 35128 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349200 35128 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349200 35128 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349332 35128 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349332 35128 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349464 35392 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349464 35392 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349464 35392 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349596 35392 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349596 35392 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349728 35656 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php
root 24028 1.1 0.4 349728 35656 pts/1 S+ 16:34 0:04 /opt/php5/bin/php ./telegram_receiver.php

When I start my script it uses ~16MB. The log starts after around 5 minutes and uses already 33,444MB and at the end of the log 35,656MB. Every 5 seconds the memory usage increases by exactly 264kb.

How did you monitor your memory usage?

[noplanman]

Ok, I used the same PHP version, 5.6.10.

For profiling I used xhprof and xdebug.
Inside the while() loop I would log the usage every other cycle. Comparing the first log to the last one (letting it run for about 10 mins), there was no significant change.

Also, I checked the usage of the process on my Mac and it stayed constant.

Additionally, I echoed memory_get_usage(1); from within the loop and it stayed constant, meaning (as far as I know) that the garbage collector did it's job properly.

I'm still new to profiling and am desperately trying to learn more about it, thanks for being so patient 😊

Are you sure it's the physical memory usage that's growing? What happens when the process reaches the RAM limit, have you gone that far?

Also, you could you try running it with php-cgi instead of php to see if that makes a difference.

[jonofe]

I also did memory_get_usage(true) before and right after the handleGetUpdates() and it stays constant. That was also a surprise for me, because when looking at top the memory (RES) is increasing permanently.

According to the definition of RES it is the physical memory used:

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming.

I would not completely exclude that it could be a also a misinterpretation or mistake on my side. And as I have a working work-around, I'll will stay patient and try to help if I can. :blush:

Thanks for the support so far. It's really a great API implementation for telegram.

[jonofe]

using php-cgi makes no difference.
Did not go up to now to the RAM limit.

[noplanman]

I have a working work-around

What is your work around then? Maybe we can compare and see why it's happening in one place but not the other?

Thanks! We're really grateful for people like you who help debug and solve problems, so thank YOU too!

[jonofe]

The work-around is only automatically restarting the script after 1h. So nothing which can help, I suppose.

[noplanman]

@jonofe Memory leaks still a problem for you?