Mautic: 2.15.0 and 2.15.1 mautic.CRITICAL: OutOfMemoryException even with 896MB on low traffic domains

[onlineexpert]

Bug Description

I have 2 Mautic configurations. One is still 2.15.0 and another upgraded from 2.15.0 to 2.15.1 on recent release. Both were always running smoothly side by side on the same Ubuntu server for two different low traffic domains.

Today BOTH configurations suddenly started to give mautic.CRITICAL errors. ** 1

Cron jobs can still be executed without any problem,
but login in dashboard and viewing landing pages is no longer possible with time-outs.

After first trying to increase php memory_limit in php.ini and max_execution_time I moved the 15.1 configuration to a completely new server to exclude any performance issues. After migration I disabled the 2.15.1 configuration on the initial server so only the 2.15.0 is running.

But both servers keep having problems. There seems to be some strange memory leak?!

The following steps did not help:

• increasing memory_limit in php.ini from 128M to 250M on the (now only) 2.15.0 server ** 1
• increasing max_execution_time in php.ini from 30 to 240 (because increasing memory_limit also resulted in timeout errors) ** 2
• even further increasing memory on both the initial and the new 2.15.1 server to 512M and 896M keeps resulting in the same mautic.CRITICAL errors for even more memory. There seems to be some strange memory leak?! ** 3
• just to be sure also replaced all files on NEW 2.15.1 server with completely fresh 15.1 download and restored a backup from working 2.15.1 configuration of yesterday
• also removed Mautic cache countless times, restarting nginx and php7.1-fpm, rebooting, ...

Still no luck and no idea at all what to do or test to get both servers working again.

Since server and files for the 2.15.1 configuration are now fresh I suspect something slipped in database today though there is also still an issue after restoring old working backup.

| Q | A
| --- | ---
Mautic versions | 2.15.0 and 2.15.1
PHP versions | 7.1 (and 7.0)
Browser | Chrome 73.0.3683.103

Steps to reproduce

Both on AWS EC2 and Lightsail
1 GB RAM, 1 vCPU, 40 GB SSD
Ubuntu 18.04.2 LTS
nginx version: nginx/1.14.0 (Ubuntu)
php7.1-fpm (even tried php7.0 though 7.1 was working good long time)

Log errors

** 1
mautic.ERROR: PHP Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 134217736 bytes) - in file /var/www/prod/vendor/symfony/intl/Data/Bundle/Reader/BundleEntryReader.php - at line 134 [] [] mautic.ERROR: Symfony\Component\Debug\Exception\OutOfMemoryException: Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 134217736 bytes) - in file /var/www/prod/vendor/symfony/intl/Data/Bundle/Reader/BundleEntryReader.php - at line 134 [] [] mautic.CRITICAL: Uncaught PHP Exception Symfony\Component\Debug\Exception\OutOfMemoryException: "Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 134217736 bytes)" at /var/www/prod/vendor/symfony/intl/Data/Bundle/Reader/BundleEntryReader.php line 134 {"exception":"[object] (Symfony\\Component\\Debug\\Exception\\OutOfMemoryException(code: 0): Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 134217736 bytes) at /var/www/prod/vendor/symfony/intl/Data/Bundle/Reader/BundleEntryReader.php:134)"} []

** 2
FastCGI sent in stderr: "PHP message: PHP Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/prod/vendor/symfony/intl/Data/Bundle/Reader/JsonBundleReader.php on line 39" while reading response header from upstream

** 3
'Allowed memory size of 536870912 bytes exhausted (tried to allocate 536870920 bytes)'

[nfwade]

I'm having the same issue. I updated php a few days ago and I believe it's related.

[onlineexpert]

Since php7.1-fpm is released Dec. 1 2016 I'm curious how a php update can be related?
Meanwhile some extra information from testing:

• after cache delete most of the time the Mautic login screen is shown but with error in Chrome developer console and timeout
  Unchecked runtime.lastError: The message port closed before a response was received.
• successively increasing memory only extends the execution time with always the same result on both servers in nginx and mautic logs:
  Allowed memory size of 262144000 bytes exhausted (tried to allocate 268435464 bytes)
Allowed memory size of 536870912 bytes exhausted (tried to allocate 536870920 bytes)
Allowed memory size of 805306368 bytes exhausted (tried to allocate 536870920 bytes)
• it's always /vendor/symfony/intl/Data/Bundle/Reader/BundleEntryReader.php line 134
  // Remember which locales we tried
$testedLocales[] = $currentLocale;

I'm not familiar with Symfony programming so can someone explain what this file and line is trying to accomplish?
Is there a way to disable or override this?

[onlineexpert]

https://www.php.net/ChangeLog-7.php
php 7.1.27 was released on March 7 2019 and 7.1.28 on April 4 2019 so possibly it's related
though testing with 7.0 gives the same problems and latest 7.0.33 dates December 6 2018.
Server patches are always implemented relatively fast and it's only yesterday that both configurations started to show OutOfMemoryExpection so I doubt these minor bug releases can have such a big impact.

[escopecz]

Is this only on log in or other requests/commands as well? One idea that comes to mind is that on login Mautic communicates with Mautic ORG servers to load some news and check if there is a new version to upgrade to. Maybe the Mautic ORG servers has troubles to load or send too much data?

The error is incomplete. If you could get some more details like stack trace it would help to debug what's going on. Try to put 'debug' => true, into your app/config/local.php for short period of time (not good option for production long term) to catch more detailed error messages. Clear the cache to apply the change in the config file.

[TheBags]

Same problem here.
Seems a memory problem, but is not.
updated from PHP7.1.26 to 7.1.28
Doing an strace of the apache pid i get:
access("/var/www/mautik/vendor/symfony/intl/Resources/data/regions/.json", F_OK) = -1 ENOENT (No such file or directory)
I'm trying to downgrade to 7.1.26 but is not so simple....

[josaiasmoura]

This bug was reported in https://github.com/symfony/symfony/issues/31089 , fix it with upgrading Symfony version to 4.2.5 :)
The problem and solution can be found in this file https://github.com/symfony/symfony/blob/2f73c2f66b54b376647b605ebd4fbd7f399b0ff9/src/Symfony/Component/Intl/Locale.php#L90

[TheBags]

@josaiasmoura this solve for me.
tnx a lot

TheBags

[nfwade]

@josaiasmoura 's fix worked for me as well. Thanks a lot

[AndrassGray]

Yes, that hack worked for me as well. I can equally confirm that this bug was triggered by an update on my Ubuntu 18.04.2 LTS. It concerned both php7.0 and php7.1

[onlineexpert]

Thanks for all the quick responses and the fix!! Since I'm only familiar with standard Mautic upgrade procedure, what is the best method to apply this bug fix?

[josaiasmoura]

Thanks for all the quick responses and the fix!! Since I'm only familiar with standard Mautic upgrade procedure, what is the best method to apply this bug fix?

The easy way to fix the bug is updating only the Locale.php file :) Run following bash command inside of your mautic installation folder :

curl -o ./vendor/symfony/intl/Locale.php https://raw.githubusercontent.com/symfony/symfony/2f73c2f66b54b376647b605ebd4fbd7f399b0ff9/src/Symfony/Component/Intl/Locale.php

And restart the php service after that.

PS: It's work for PHP 7.1+, for PHP 7.0 you will have to edit it manually to apply the diff.

[kmanojkumar]

https://github.com/mautic/mautic/issues/7408#issuecomment-482804001 - This solution worked for me, this command

curl -o ./vendor/symfony/intl/Locale.php https://raw.githubusercontent.com/symfony/symfony/2f73c2f66b54b376647b605ebd4fbd7f399b0ff9/src/Symfony/Component/Intl/Locale.php

shared by @josaiasmoura has to be executed in outside vendor folder in the mautic directory, probably with sudo permissions if you get a permission denied error.

[tenil]

How to fix in PHP 7.0?
PHP 7.0.33-6 + ubuntu 16.04.1

[escopecz]

I prepared the fix that can go to the next Mautic version. Please test https://github.com/mautic/mautic/pull/7470

[luizeof]

@escopecz tested with PHP 7.1.27 and PHP 7.2+ .. this bug doesn't happen

[escopecz]

@luizeof could you please write your comment to the PR #7470? It will help to get it to the core asap.

[onlineexpert]

This seems to have solved it. Thx a lot!!

[wittwerch]

We had this issue as well and solved it by simply deactivating the intl module in PHP.

https://github.com/symfony/symfony/issues/31087

[npracht]

Fixed when #7470 merged.

[diegofcv]

I had a similar problem with v2.15.1 and PHP Version 7.2.20, but the error was:
mod_fcgid: stderr: PHP Fatal error: Maximum execution time of 90 seconds exceeded in .../vendor/symfony/intl/Data/Bundle/Reader/JsonBundleReader.php on line 38, referer: .../s/segments
So, when I was trying to create a new segment on a very new instance of Mautic.
I found that bug and removed php7.2-intl, what made the system flow again.