Laravel-backup: allowed memory exhausted on backup:run

[racastellanosm]

Hi,
When i trying to run a backup through artisan command, i got the following error

Starting backup...
Determining files to backup...
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 4304 bytes) in /home/xxx/sites/domain.com/www/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php on line 79

My config exclude vendors and others folders from backup. I think is a ZIP problem, but i'm not sure.
My box has 4GB of RAM

Thanks for your help

[timleland]

I'm having the same issue.

[freekmurze]

Could you please let me know how many files there are in the directories you are including and excluding. Could you also post the contents of the used config file (leaving out any info you consider sensitive)?

[racastellanosm]

Yeah sure,

This is my laravel-backup config file:

return [
    'backup' => [
        'name' => env('APP_URL'),
        'source' => [
            'files' => [
                'include' => [
                    base_path(),
                ],
                'exclude' => [
                    base_path('vendor'),
                    base_path('node_modules'),
                    base_path('db-scripts'),
                    base_path('tests'),
                    storage_path(),
                ],
            ],
            'databases' => [ ],
        ],

        'destination' => [
            'disks' => [
                's3',
            ],
        ],
    ],

    'cleanup' => [
        'strategy' => \Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy::class,
        'defaultStrategy' => [
            'keepAllBackupsForDays' => 2,
            'keepDailyBackupsForDays' => 2,
            'keepWeeklyBackupsForWeeks' => 1,
            'keepMonthlyBackupsForMonths' => 1,
            'keepYearlyBackupsForYears' => 1,
            'deleteOldestBackupsWhenUsingMoreMegabytesThan' => 5000
        ]
    ],

    'monitorBackups' => [
        [
            'name' => 'Samsung -> e-retailers',
            'disks' => ['s3'],
            'newestBackupsShouldNotBeOlderThanDays' => 1,
            'storageUsedMayNotBeHigherThanMegabytes' => 5000,
        ],
    ],

    'notifications' => [
        'handler' => Spatie\Backup\Notifications\Notifier::class,
        'events' => [
            'whenBackupWasSuccessful'     => ['log', 'slack'],
            'whenCleanupWasSuccessful'    => ['log', 'slack'],
            'whenHealthyBackupWasFound'   => ['log'],
            'whenBackupHasFailed'         => ['log', 'slack'],
            'whenCleanupHasFailed'        => ['log'],
            'whenUnhealthyBackupWasFound' => ['log', 'slack'],
        ],
        'mail' => [
            'from' => '[email protected]',
            'to'   => '[email protected]',
        ],
        'slack' => [
            'channel'  => '#projectname',
            'username' => 'Backup bot SIS',
            'icon'     => ':robot_face:',
        ],

        'pushover' => [
            'token'  => env('PUSHOVER_APP_TOKEN'),
            'user'   => env('PUSHOVER_USER_KEY'),
            'sounds' => [
                'success' => env('PUSHOVER_SOUND_SUCCESS','pushover'),
                'error'   => env('PUSHOVER_SOUND_ERROR','siren'),
            ],
        ],
    ]
];

Thanks for your help

[freekmurze]

I highly suspect it'll have something to do with the node_modules directory. I probably contains many many files. Could you confirm my suspicion by temporary moving the node_modules out of your project and taking a backup then? (You probably should do this in your development environment).

[timleland]

@freekmurze Even if it's in the exclude?

[racastellanosm]

Ok,

Moving node_modules out of the project and my exclude array (laravel-backup config) finish with this exclude directories:

 'exclude' => [
                    base_path('vendor'),
                    base_path('db-scripts'),
                    base_path('tests'),
                    storage_path(),
                ],

the tree command with excluded folders:

tree . -I 'vendor|storage' | tail -1
161 directories, 2268 files

I runing again php artisan backup:run and the same error occurs, i think the exclude pattern in my config file is no taking effect, because the tree command without exclude the storage folders return:

62345 directories, 203401 files

¿Maybe something wrong in my version?
Laravel-Backup version = 3.3.3

Thanks again for your help

[freekmurze]

Could you trying upgrade to the latest version of the package? Which PHP version / platform are you using?

[racastellanosm]

Ok,

My WebServer is Nginx + php-fpm processor
My PHP Versión:

PHP 5.5.34 (cli) (built: Apr  2 2016 09:34:42) 
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

And my Laravel-Backup Versión is 3.3.3 ¿What version of LB can i use?

Thanks

[timleland]

I'm using Backup v: 3.5.0
Php: 7.0.5-3

[freekmurze]

I can't seem to recreate this issue.

Could you both try backing up only one file and see if that works? Keep on adding directories until it breaks.

[freekmurze]

I've tried recreating this issues by backing up a lot of files, but it all seems to work.

I guess your problems are related to a specific os / config setting. Be sure to leave any hints on the cause/solution of this problem in this issue.

[timleland]

Are you doing it locally? My issue occurs on my digital ocean droplet that has 2gb of ram? Could that be the issue?

[timleland]

Just tested and the backup ran fine in homestead locally.

[racastellanosm]

MI issue is in a DigitalOcean Droplet of 4GB of ram.. my filesizes are 74MB... maybe something wrong with PHP config. I will check all my settings and try again. Thanks @freekmurze @timleland

[timleland]

@racastellanosm Are you using Forge?

[racastellanosm]

No @timleland

[timleland]

I believe I found out what was causing my issue. I've been using the file driver for cache and sessions which was causing the memory issues. I changed it to use memcache and backups work. A possible fix would be to not read all the files/folders in the exclude section?

[freekmurze]

Hi all,

@sebastiandedeyne did some refactoring in an attempt to reduce memory usage. These changes are present in the newly tagged version 3.7.2. Could you try upgrading to that version and report back if you're still having memory issues?

[eleftrik]

Hi @freekmurze, I'm running into the same issue.
Version 3.7.2, I'm trying locally on Homestead. My config file:

        'source' => [

            'files' => [

                /*
                 * The list of directories that should be part of the backup. You can
                 * specify individual files as well.
                 */
                'include' => [
                    base_path(),
                    storage_path('documents'),
                ],

                /*
                 * These directories will be excluded from the backup.
                 * You can specify individual files as well.
                 */
                'exclude' => [
                    base_path('vendor'),
                    storage_path(),
                    base_path('bower_components'),
                    base_path('node_modules'),
                ],
            ],

When I run php artisan backup:run, I got

Starting backup...
Dumping database meetmount...
Determining files to backup...
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 4304 bytes) in /home/vagrant/Code/xxxx/yyyy/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php on line 79

Thank you

[abenevaut]

php -d memory_limit=256M artisan backup:run

Or edit php.init.

• Could not work on shared env.

[eleftrik]

@42antoine I'm not on a shared environment, but on Homestead.
Anyway, I solved removing "base_path()" from include, leaving "exclude" empty and adding to "include" only the subfolders thath I want to backup.

[schulzefelix]

I had the same issue with the spatie/laravel-medialibrary with around 67k files. I just added to the include the single path public_path('media'). The memory_limit is set to 512MB.

[EventFellows]

Any solution for this yet?
I am running into exactly the same issue on homestead on windows, too

Laravel 5.2.39
Laravel-Backup: 3.8.0
PHP 7.0.1-1+deb.sury.org~trusty+2 (cli) ( NTS )

Here is the error I get when running php artisan backup:run

vagrant@homestead  ~/Code/eventfellows (master)-> php artisan backup:run
Starting backup...
Dumping database eventfellows...
Determining files to backup...
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 8192 bytes) in /home/vagrant/Code/eventfellows/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php on line 79
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 32768 bytes) in /home/vagrant/Code/eventfellows/vendor/symfony/debug/Exception/FatalErrorException.php on line 1

Additional observations:

• there is no difference in error whether or not I list base_path('vendor') and base_path('node_modules') in the exclude array
• it works if I only work with inlcude and list all the different folders in seprate base_path('xyz'), without vendor and node_modules

[freekmurze]

The newly tagged version 3.8.1 should resolve most memory use and speed issues. Upgrade to that version and let me know if your problems persist or not.

[abenevaut]

Working fine for me 👍

Now, no need to specify the memory_limit.

[freekmurze]

Closing this for now. Reopen if you're having troubles with this in latest versions of v3 or v4 of the package.

[t0n1zz]

hello i just now get this error too
Allowed memory size of 134217728 bytes exhausted (tried to allocate 56819713 bytes)

just running it on laravel valet with default configuration.

and i enabled withResponsiveImages and also add this into my model

public function registerMediaCollections()
    {
        $this->addMediaCollection('admin_foto')
            ->singleFile()
            ->registerMediaConversions(function(Media $media){
                $this->addMediaConversion('thumb')
                    ->width(50)
                    ->height(50);
            });
    }

i tried to upload images taken from my iphone 5c with total size 1.2mb... any idea why? i am not testing it on production server yet since the overall development not done yet..

[abenevaut]

Do you see that problem on laravel-backup ? Or on laravel-medialibrary vendor (=> https://github.com/spatie/laravel-medialibrary) ?

[t0n1zz]

oh i am sorry i posted on the wrong project, i see the problem on laravel-medialibrary

[ILABUTK]

Getting the error again while composer require spatie/laravel-backup

"spatie/laravel-backup": "^6.11",
"laravel/framework": "^7.0",
"php": "^7.2.5",

PHP Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 4096 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52