Framework: DB memory leak (querylog disabled)

[dirkpostma]

Problem description

With certain Fluent queries, laravel is leaking memory.

When I do:

\DB::connection()->disableQueryLog();
for($i= 0; $i<10000000;$i++) {
    \DB::table('test')->insert(['somefield' => $somestring ]);
}

...memory usage is growing linear.

When I perform exactly the same query, but using PDO directly:

for($i= 0; $i<10000000;$i++) {
    $query = "insert into `test` (`somefield`) values ('{$somestring}')";
    \DB::connection()->getPdo()->query($query);
}

...memory usage is constant, no problems.

Therefore, I think there must be some kind of leak in Fluent.
I tried to locate the problem, but didn't find it yet.

Reproduction

Create command file in /app/commands/Leaktest.php
http://pastebin.com/LJUbaYfp

Register command in app/start/artisan.php

Artisan::add(new Leaktest);

Create table in database:

CREATE TABLE `test` (
    `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
    `somefield` varchar(255) DEFAULT NULL,
     PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Run commands in CLI:

$ art leaktest leak_example
#executions = 10000 - mem: 16523992
#executions = 20000 - mem: 23295240
#executions = 30000 - mem: 29935360
#executions = 40000 - mem: 36837608
#executions = 50000 - mem: 43477728
#executions = 60000 - mem: 50117848
#executions = 70000 - mem: 57282248
#executions = 80000 - mem: 63922368
#executions = 90000 - mem: 70562480
READY
Ready

$ art leaktest noleak_example
#executions = 10000 - mem: 9402032
#executions = 20000 - mem: 9402064
#executions = 30000 - mem: 9402064
#executions = 40000 - mem: 9402064
#executions = 50000 - mem: 9402064
#executions = 60000 - mem: 9402064
#executions = 70000 - mem: 9402064
#executions = 80000 - mem: 9402064
#executions = 90000 - mem: 9402064
READY
Ready

I'm usin updated

 "laravel/framework": "4.0.*",

Anyone a clue of the cause of this memory leak?

[ipalaus]

Using sqlite (in MBA):

$ php -v
PHP 5.4.13 (cli) (built: Apr 24 2013 00:30:37)
...
$ php artisan leaktest leak_example
#executions = 10000 - mem: 8794208
#executions = 20000 - mem: 8794240
#executions = 30000 - mem: 8794240
#executions = 40000 - mem: 8794240
#executions = 50000 - mem: 8794240
#executions = 60000 - mem: 8794240
#executions = 70000 - mem: 8794240
#executions = 80000 - mem: 8794240
#executions = 90000 - mem: 8794240
READY
Ready
$ php artisan leaktest noleak_example
#executions = 10000 - mem: 8576232
#executions = 20000 - mem: 8576264
#executions = 30000 - mem: 8576264
#executions = 40000 - mem: 8576264
#executions = 50000 - mem: 8576264
#executions = 60000 - mem: 8576264
#executions = 70000 - mem: 8576264
#executions = 80000 - mem: 8576264
#executions = 90000 - mem: 8576264
READY
Ready

Using MySQL (Debian 7.0 in a Vagrant guest):

$ php -v
PHP 5.4.4-14 (cli) (built: Mar  4 2013 14:08:43) 
...
$ php artisan leaktest leak_example --env=development
#executions = 10000 - mem: 10536608
#executions = 20000 - mem: 10536640
#executions = 30000 - mem: 10536640
#executions = 40000 - mem: 10536640
#executions = 50000 - mem: 10536640
#executions = 60000 - mem: 10536640
#executions = 70000 - mem: 10536640
#executions = 80000 - mem: 10536640
#executions = 90000 - mem: 10536640
READY
Ready
$ php artisan leaktest noleak_example --env=development
#executions = 10000 - mem: 10279448
#executions = 20000 - mem: 10279480
#executions = 30000 - mem: 10279480
#executions = 40000 - mem: 10279480
#executions = 50000 - mem: 10279480
#executions = 60000 - mem: 10279480
#executions = 70000 - mem: 10279480
#executions = 80000 - mem: 10279480
#executions = 90000 - mem: 10279480
READY
Ready

[taylorotwell]

Wait, so you're not getting a leak @ipalaus?

[ipalaus]

@taylorotwell not at all, you?

@dirkpostma which PHP version are you using?

[taylorotwell]

I haven't tried yet. OP: do you have any packages installed or anything or is this a totally clean install?

On Jun 13, 2013, at 7:39 PM, Isern Palaus [email protected] wrote:

@taylorotwell not at all, you?

@dirkpostma which PHP version are you using?

—
Reply to this email directly or view it on GitHub.

[ipalaus]

The first one is a clean installation without any additional packages. The second one running in Debian has some packages installed.

[dirkpostma]

Thanks for trying to reproduce. Good to see you don't have the problem. I'm not behind computer right now (iphone), but now I think my problem is caused by a package, forgot to exclude these in my tests, sorry for that. Later this day (i'm in The Netherlands), I'll try to find the cause and probably will open an issue at some package's hub.

[dirkpostma]

https://github.com/loic-sharma/profiler

FYI: disabling this package solved the leak.

[ipalaus]

That's why: https://github.com/loic-sharma/profiler/blob/master/src/Profiler/ProfilerServiceProvider.php#L85

When you get the PDO object laravel won't fire the event. :)

[pabloleone]

I have this issue and the only package I've installed is Sentry 2. Is a clean installation of Laravel 4.0.

I'm trying to get all rows from a table like Table::all(); and I get this memory issue. WHY?

I don't understand... If I add DB::connection()->disableQueryLog();, works but not in all cases... I fix a bulk with that line to avoid memory issues, but seems not working with db selects.

Ideas??

[ipalaus]

Well... how many results are you fetching from the DB?

[dirkpostma]

I see documentation states

DB::connection()->disableQueryLog();

However, I always use

DB::disableQueryLog();

Can you try that? (May be this is a mistake in documentation?)

[JoostK]

It can't be the query log, as Table::all executes only one query, so storing that in the query log won't consume memory. The data he queries does, and that is probably the culprit.

[dirkpostma]

I think this is not the place to discuss. Craete a new issue, with a reproduction of the bug in a clean project.

[Zae]

I resolved my memory leak by disabling https://github.com/barryvdh/laravel-debugbar, even tho I did disableQueryLog, debugbar was probably still logging my queries.

[slampenny]

@Zae's solution worked, but I had to additionally remove 'Clockwork\Support\Laravel\ClockworkServiceProvider' . There's still a small leak, but it's acceptable.

[Daniel-HBK]

I resolved my memory leak by disabling https://github.com/barryvdh/laravel-debugbar, even tho I did disableQueryLog, debugbar was probably still logging my queries.

u saved my day!
my chrome tab was taking 2GB of ram and crash after importing 5k row of excel/csv file with Laravel Excel 2.1 although it finishing inserting them within 20 second but after redirecting with session flash chrome tab keeps loading and memory leak happen from 10mb to 2gb! ...
Thanks again