Laravel-backup: now that spatie/db-dumper supports mongodb dumps.. could you pretty please add it to this awesome package?

[vesper8]

with a cherry on top? pleeeeaase

I've starred over 50 of your repos! ;-))

I <3 spatie

[vesper8]

maybe @lukzgois can help :))

[freekmurze]

😍

Let me take care of this real quick.

[freekmurze]

Tagged a new version that supports mongo. 🙌

[vesper8]

awesome!! thanks!!

[vesper8]

Updated it and added 'mongodb' in the list of databases to backup but getting an error connecting

laravel-backup.php

            /*
             * The names of the connections to the databases that should be backed up
             * MySQL, PostgreSQL, SQLite and Mongo databases are supported.
             */
            'databases' => [
                'mysql',
                'mongodb',
            ],

getting error

Backup failed because The dump process failed with exitcode 1 : General error : 2017-05-10T11:08:49.244+0800    Failed: error connecting to db server: server returned error on SASL authentication step: Authentication failed.

I assumed it would just grab the connection settings from my database.php, which I assume is what happens for mysql

here's my .env file

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=mydb
DB_USERNAME=homestead
DB_PASSWORD=secret

DB_MONGO_CONNECTION=mongodb
DB_MONGO_HOST=127.0.0.1
DB_MONGO_PORT=27017
DB_MONGO_DATABASE=mydb
DB_MONGO_USERNAME=homestead
DB_MONGO_PASSWORD=secret

and my database.php

    'connections' => [

        'mysql' => [
            'driver' => 'mysql',
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('DB_DATABASE', 'forge'),
            'username' => env('DB_USERNAME', 'forge'),
            'password' => env('DB_PASSWORD', ''),
            'unix_socket' => env('DB_SOCKET', ''),
            'charset' => 'utf8mb4',
            'collation' => 'utf8mb4_unicode_ci',
            'prefix' => '',
            'strict' => true,
            'engine' => null,
        ],

        'mongodb' => [
            'driver' => 'mongodb',
            'host' => env('DB_MONGO_HOST', 'localhost'),
            'port' => env('DB_MONGO_PORT', 27017),
            'database' => env('DB_MONGO_DATABASE'),
            'username' => env('DB_MONGO_USERNAME'),
            'password' => env('DB_MONGO_PASSWORD'),
            'options' => [
                'database' => 'admin' // sets the authentication database required by mongo 3
            ]
        ],

@freekmurze how do I get this to work? Or am I supposed to pass the mongodb connection parameters via the command line?

[freekmurze]

If would think that because db-dumper supports Mongo and we just use the default way of handling credentials this should have just worked.

This is the code that gets the credentials.

Unfortunately I'm not a Mongo user so can't test on a live installation easily. Could you go through this and let me know what's wrong?

@lukzgois could you take a look at this too?

[vesper8]

Been trying to debug it and get it to work for almost an hour and haven't figured it out yet.

I am getting the same error if I try to use the DB Dumper standalone too using the command I made below

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;

class DumpMongoDB extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'dump:mongodb';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Manually dump the mongodb database';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {
        $dbConfig = config("database.connections.mongodb");

        $dbHost = array_get($dbConfig, 'read.host', array_get($dbConfig, 'host'));

        $dbDumper = \Spatie\DbDumper\Databases\MongoDb::create()
            ->setHost($dbHost ?? '')
            ->setDbName($dbConfig['database'])
            ->setUserName($dbConfig['username'] ?? '')
            ->setPassword($dbConfig['password'] ?? '');

        if (isset($dbConfig['port'])) {
            $dbDumper = $dbDumper->setPort($dbConfig['port']);
        }

        d($dbDumper);

        $path = storage_path("app/laravel-backup/temp/db-dumps/{$dbConfig['database']}.gz");

        d("saving to $path");

        $dbDumper->dumpToFile(storage_path("{$dbConfig['database']}.gz"));
    }
}

I thought maybe it was an oddity with mongodump on OSX, but I now tried it on my Laravel Forge droplet and I get the same error. I am also printing out the connection settings to make sure they are correct and indeed they are.

I obviously can connect to my mongodb instances from Laravel itself.

I thought it might be a problem with with php-cli vs php-fpm, but I ran the above code from a controller and same issue

I thought it might have to do with the part that says

            'options' => [
                'database' => 'admin' // sets the authentication database required by mongo 3
            ]

But I am able to connect even if I comment out those lines since 'admin' is the default place that mongodb looks for authentication. Commenting those lines doesn't help with the problem however

am hoping @lukzgois has more insights :)

[lukzgois]

Hi guys, sorry for the delay in answering. I'll make some tests here to try to understand what may be happing.

[freekmurze]

Ok, thanks to you both for investigating this.

[lukzgois]

I've made a test here @vesper8 and the SASL authentication failure occurred when my user didn't have access to the database. If I gave access to my user, the backup was OK.

When testing on an unprotected mongdb database (without the username and password keys on database config file), I've noticed the package was still inserting the --username and --password arguments on the command. I'll try to fix this when I get home today.

[vesper8]

@lukzgois

ok I'm really confused about something. You've seen my config above and I've been able to have my laravel apps connect to mongo using this config.

what I've done to make this work is add the user homestead/secret on the 'admin' database and grant it the userAdminAnyDatabase@admin role

I've just done some tests and for sure without a doubt if I remove this role then my app can no longer connect to mongodb.

I've also now gone ahead and add a homestead/secret user on the database I want to dump in question and if I grant it the dbOwner@mydb or dbAdmin@mydb then yes I am now able to dump the database using db-dumper or laravel-backup

However I've taken a step back and removed that user on the mydb database and tried adding the dbAdminAnyDatabase@admin role on the admin database instead to see if it works (and removed the userAdminAnyDatabase@admin role for testing

My app can still access mongodb if I do this but I cannot use db-dumper

so I'm confused about this. I can understand how dumping requires dbOwner or dbAdmin roles and userAdmin is not sufficient

However I don't understand why I need to add the user and role directly on the database. Shouldn't I be able to add dbAdminAnyDatabase@admin role on the admin database and still be able to run db-dumper?

[freekmurze]

@lukzgois any news on this?

[freekmurze]

Closing this due to inactivity.

[eboye]

Hi all,

I've found this issue while searching for the same error. I'm not sure where to edit the mongodb dump command, but the needed parameter is missing.

--authenticationDatabase admin

that admin parameter should be grabbed from mongodb.options.database configuration.

Mongo 3 uses additional table for user/pass ... so if someone can add this paramater it would be great. db-dumper has this option here

it just needs to be set ...

[AlexVanderbist]

Hi @eboye, this package uses the spatie/db-dumper you mentioned. However the authenticationDatabase parameter was only added in v2.8.0 whilst this package is locked to ^2.7 in composer.json.

Edit: the parameter you mentioned should be in there... Have you tried running composer update?

[ReeceM]

I Believe the issue is that this is not clearly stated in the documentation that one needs to add another entry for the auth database in the dump parameters, as it does for the mysql.

https://github.com/spatie/laravel-backup/blob/c44b2531448e06c8cf151b7fd6eb601ca52cf35c/src/Tasks/Backup/DbDumperFactory.php#L38

 $dbDumper->setAuthenticationDatabase(config('database.connections.mongodb.dump.mongodb_user_auth') ?? '');

mongodb' => [
            // normal mongodb config data...
            'options'  => [
                'database' => 'admin' // sets the authentication database required by mongo 3
            ],
            'dump' => [
                'mongodb_user_auth' => 'admin' // <- the missing parameter which has been omitted in docs ?
             ]  
        ],

[eboye]

@ReeceM that was the issue :D ... it's working now for me ... so I can remove my custom sh script for mongo backup :)