Homestead: Error when attempting to backup disabled database

[BenceSzalai]

Versions

• Vagrant: v2.2.9
• Provider: vagrant-parallels v2.0.1
• Homestead: v11.3.3

Host operating system

MacOS v10.14.6 (irrelevant)

Homestead.yaml

#...
databases:
  - homestead
backup: true
features:
  - mysql8: true
  - postgres: false
#...

Expected behavior

Homestead should only try to backup the enabled databases.

Actual behavior

Homestead attempts to backup both mysql and postgres without checking if they are enabled. This causes a pg_dump: error: connection to database "homestead" failed: could not connect to server: Connection refused error when destroying.

Steps to reproduce

1. Add backup: true and false for one of the mysql8 or postgres databses under the features array in Homestead.yaml (see example above).
2. run vagrant destroy

[BenceSzalai]

Here a check should be included against settings['features'] to ensure mysql8 (or mariadb i think) and postgres are not set to false, before attempting the backup.
https://github.com/laravel/homestead/blob/40f4dbe4619a592336cec017d461fbdfa033a09c/scripts/homestead.rb#L610-L616

Provided that a nice enabled_databases array is already built at
https://github.com/laravel/homestead/blob/40f4dbe4619a592336cec017d461fbdfa033a09c/scripts/homestead.rb#L519-L534

A simple fix can be:

    if settings.has_key?('backup') && settings['backup'] && (Vagrant::VERSION >= '2.1.0' || Vagrant.has_plugin?('vagrant-triggers'))
      dir_prefix = '/vagrant/'
      settings['databases'].each do |database|
        if ( enabled_databases.include? 'mysql8' ) || ( enabled_databases.include? 'mariadb' )
            Homestead.backup_mysql(database, "#{dir_prefix}/mysql_backup", config)
        end
        if enabled_databases.include? 'postgres'
            Homestead.backup_postgres(database, "#{dir_prefix}/postgres_backup", config)
        end
      end
    end

I'd submit a PR, but it's not exactly clear to me if i should do against both master and 20.04 branches?

[countless-integers]

I'd love to see it in version 10 as well🤞

[svpernova09]

Database backups might not even work on v11.x It's certainly an issue I'd call "Likely broken". The reason for this is because v10 used LVM to allow this to happen pretty seamlessly. I'm not an LVM expert and leaned heavily on others to implement this. When I was working on 20.04 I reached out and didn't get any feedback or response so I had to move forward without the LVM configuration in v11 to support it.

So if it works, great. If it doesn't....we should revisit "Do we need LVM to provide user backups?". From there I would need assistance working on getting the functionality into 20.04 on Settler, and then we can debug how it interacts here.

I'm going to leave this issue open for now, I'm going to check in on your suggestioned fix. thanks.

I'd love to see it in version 10 as well🤞

DB backup originated in v10 and should still work there since the base box is still using a customized LVM configuration to have MySQL use a special volume for it's data.

[BenceSzalai]

As far as I can see the db data is not accessed over the filesystem during the backup operation. MySQL uses the standard mysqldump utility, which I think connects to the running MySQL system service directly (or maybe over a local tcp connection), while Postgress uses pg_dump which connects to port 5432 on localhost. So none of these are dependent on the FS configuration.

What may depend is the way the generated exports are moved from the guest to the host machine. Apparently they are written under /vagrant/mysql_backup and /vagrant/postgres_backup. This /vagrant/ may be a tricky mount point that actually points to the folder where Homestead is launched from, i.e. the project folder on the host machine.

Fun fact that if in Homestead.yaml under folders the map: is not an absolute path, but for example set to just . all seems to work fine with vagrant, however the database backups never appear on the host machine. This also suggests the backups are copied from the guest machine to the host in some special way, which only works using absolute path, while other things work with relative too.

I'm wondering at the same time if the project folder is mounted under /home/vagrant/code in the guest, why not just copy the backups to /home/vagrant/code/*_backup instead of /vagrant/*_backup? I think the files under /home/vagrant/code are two way synced already anyway, aren't they?

[MichaelBelgium]

I have the same error with mariadb: true and mysql8: false

[svpernova09]

Fun fact that if in Homestead.yaml under folders the map: is not an absolute path, but for example set to just . all seems to work fine with vagrant, however the database backups never appear on the host machine. This also suggests the backups are copied from the guest machine to the host in some special way, which only works using absolute path, while other things work with relative too.

https://laravel.com/docs/8.x/homestead

You should never mount . (the current directory) when using Homestead. This causes Vagrant to not map the current folder to /vagrant and will break optional features and cause unexpected results while provisioning.

[svpernova09]

As far as I can see the db data is not accessed over the filesystem during the backup operation. MySQL uses the standard mysqldump utility, which I think connects to the running MySQL system service directly (or maybe over a local tcp connection), while Postgress uses pg_dump which connects to port 5432 on localhost. So none of these are dependent on the FS configuration.

What may depend is the way the generated exports are moved from the guest to the host machine. Apparently they are written under /vagrant/mysql_backup and /vagrant/postgres_backup. This /vagrant/ may be a tricky mount point that actually points to the folder where Homestead is launched from, i.e. the project folder on the host machine.

I'm wondering at the same time if the project folder is mounted under /home/vagrant/code in the guest, why not just copy the backups to /home/vagrant/code/*_backup instead of /vagrant/*_backup? I think the files under /home/vagrant/code are two way synced already anyway, aren't they?

So the v10.x configuration is these /vagrant/ mounts where LVM volumes. You could try adding folder mappings to /vagrant/mysql_backup but it wouldn't surprise me if the virtualization freaks out about it.

Sounds like the better path forward might be for Homestead to automatically mount "safe" paths for these to be shared to your local machine? An option in Homestead.yaml would be database_backup_path: "path/to/database_backup" and Homestead would map path/to/database_backup to /database_backup in the VM and use that for the export path. Could work.

[BenceSzalai]

https://laravel.com/docs/8.x/homestead

You should never mount . (the current directory) when using Homestead. This causes Vagrant to not map the current folder to /vagrant and will break optional features and cause unexpected results while provisioning.

If the lack of /vagrant breaks other things too and is clearly a non supported setup (i didn't noticed this warning before), than should we really care to make sure db backup works, when /vagrant is not mounted correctly? Probably a better approach would be to check if /vagrant is available and error out if not.

Actually it should not be too difficult to adopt

https://github.com/laravel/homestead/blob/40f4dbe4619a592336cec017d461fbdfa033a09c/scripts/homestead.rb#L179-L215

to set up an additional synced_folder dedicated to database backups.

But he system would still need to default to use /vagrant if database_backup_path was not configured. Or maybe just use the first configured path from folders config as a fallback.

---

Meanwhile this is in fact independent of this issue, for which I think the suggested fix will do the required job. Maybe a separate issue would need to be opened about the LVM problems... I haven't tried v11 myself, so cannot tell.

[svpernova09]

https://laravel.com/docs/8.x/homestead

You should never mount . (the current directory) when using Homestead. This causes Vagrant to not map the current folder to /vagrant and will break optional features and cause unexpected results while provisioning.

If the lack of /vagrant breaks other things too and is clearly a non supported setup (i didn't noticed this warning before), than should we really care to make sure db backup works, when /vagrant is not mounted correctly? Probably a better approach would be to check if /vagrant is available and error out if not.

/vagrant always exists. We should (and do) always assume /vagrant/ is same as Homestead's root.

[svpernova09]

@BenceSzalai Give 20.04-dev a spin. I've added https://github.com/laravel/homestead/commit/bf44efa8df2368beace6b7669becc7b4c5b284cf which is a bit of rework. Homestead will only backup databases for enabled features So to backup MySQL/MariaDB databases you would have this in Homestead.yaml:

backup: true
features:
    - mysql: true
databases:
    - homestead
    - vcdt

Running vagrant destroy:

$ vagrant destroy
==> homestead: Running action triggers before destroy ...
==> homestead: Running trigger...
==> homestead: Backing up mysql database homestead...
    homestead: Running: inline script
==> homestead: Running trigger...
==> homestead: Backing up mysql database vcdt...
    homestead: Running: inline script
    homestead: Are you sure you want to destroy the 'homestead' VM? [y/N] n
==> homestead: The VM 'homestead' will not be destroyed, since the confirmation
==> homestead: was declined.

The databases will be saved to /vagrant/.backup which will be .backup to the Homestead root path or per project root. This is a slight path change from how backups work on 10.x as they back up to mysql_backup and postgresql_backup.

I do not intend to update/change the Database backup feature in Homestead v10.x. This change will only affect Homestead 11.x+

[countless-integers]

I do not intend to update/change the Database backup feature in Homestead v10.x. This change will only affect Homestead 11.x+

Would you mind if I opened a PR with the same change against v10? I'd argue that if one disables postresql, they don't expect homestead to take backups of it and crash on vagrant destroy (which is my gripe with it). It'd basically be the same patch you offered (not sure why you create an additional array there to check if feature is disabled, but then I'm not really coding in ruby or familiar with this project so I'd defer to your judgement on that one).

[svpernova09]

I do not intend to update/change the Database backup feature in Homestead v10.x. This change will only affect Homestead 11.x+

Would you mind if I opened a PR with the same change against v10? I'd argue that if one disables postresql, they don't expect homestead to take backups of it and crash on vagrant destroy (which is my gripe with it). It'd basically be the same patch you offered (not sure why you create an additional array there to check if feature is disabled, but then I'm not really coding in ruby or familiar with this project so I'd defer to your judgement on that one).

Ah, I completely forgot. Yes, v10 should not throw errors. The reason I duplicated that array config is because it was only the 2nd time in the class we need to get that, so it doesn't hit my 3 time threshold of extracting the logic. I'm fine with the duplication because IMHO the code is much more readable.

[svpernova09]

@countless-integers Give 1a13308406079cc4585a00ad6eb2711d840ea03c for this feature in v10. I'll probably release this coming week.

[countless-integers]

Splendid, much appreciated @svpernova09 . (Also thanks for explaining)