While working on a console app to import a csv file (about 2.5 million lines) into a database I was running out of memory.
After a bit of delving into the code I found the memory was getting eaten by the Logger
on the Yii
class as each query was adding to the messages.
The only way I could work around it was to create an empty Logger
class that implements the Log
method but does nothing.
It would be handy if you could turn off logging at certain times where large amounts of queries may occur.
Did you configure any log targets? Logger will flush logged messages to targets every 1000 messages. So it's not likely it will eat up your memory if you have no log targets.
You can setup Logger::flushInterval
to some minor value like '10' or even '1', allowing log messages be cleared from the memory.
Also you can disable debuf mode setting YII_DEBUG = false
to reduce the amount of log messages excluding trace ones.
For reference it could be I'm running on an outdated Yii
console entry script.
It was merging all the configs and pulling in the debug toolbar which used LogTarget
.
so , how do you fix this problem? It bothering me now.
Im doing some performance debugging at the moment and to my surprise I noticed that the Yii Logger eats 2%. Within a heavily used local test environment, every X% is important and this logging about basic stuff does not always make sense (to give you an idea, this test suite is run about >500 times a day). I would suggest to defer the choice of this logging to the user.
Even though closed, I must confirm that logger is a memory killer. Flushing is not a solution e.g. on batch inserts every 1000 queries - eats up 100 MB+ memory .. Unfortunately, logger object cannot be turned off. Disabling log targets won't help - logger still tracks messages into memory :-( Something like
Yii::$app->log = null;
would be nice.
@lubosdz Set $exportInterval
to 1? https://github.com/yiisoft/yii2/blob/master/framework/log/Target.php#L85
Yii::getLogger()->flushInterval = 1;
would be more appropriate if no targets are defined in Yii::$app->log, which is an instance of yii\log\Dispatcher
. The yii\log\Logger
is in Yii::getLogger()
.
Could be enhanced by adding an enabled/disabled switch.
Yes, combination of Yii::getLogger()->flushInterval = 1
and Yii::$app->log->targets = [];
seems to solve the memory issue. Unfortunatelly, this is a lot of useless overhead operations and also I am wondering how many users will come to this solution. It took me time to search documentation, forums, ...
Simple and documented possibility to turn off logging would be more welcome solution.
there is now an option to disable logging for DB queries: #14056
This issue should document or implement methods to disable logger in general.
Won't be implemented in 2.0.
Most helpful comment
Yes, combination of
Yii::getLogger()->flushInterval = 1
andYii::$app->log->targets = [];
seems to solve the memory issue. Unfortunatelly, this is a lot of useless overhead operations and also I am wondering how many users will come to this solution. It took me time to search documentation, forums, ...Simple and documented possibility to turn off logging would be more welcome solution.