Vagrant: Vagrant does not display pv (pipe viewer) command output when provisioning

[aaronjwood]

I have very large data sets that I need to do an import of as part of my provisioning process. It's incredibly helpful to see the progress of them in realtime using pv. For some reason Vagrant doesn't display anything from pv like it should. It all still works as intended so it's really just a visual thing. Is this due to the way Vagrant internally handles output?

Here is an example of how I'm using the pv command in my provisioning script:
pv -i 1 -p -t -e /path/to/sql/file.sql | mysql -u username -ppassword

[mitchellh]

Could you get a minimal working (failing... really) Vagrantfile to show this bug? That would make debugging easier if I can repro it easily.

[mitchellh]

[Closing until reply]

[aaronjwood]

Sure, here's something that should show the issue. For the mysqldump part I guess you'll have to have access to some database somewhere you can dump from...? Let me know if you also need me to provide some sort of dummy data set.

I'm running Vagrant 1.5.3 by the way.

#!/usr/bin/env bash

#Enable EPEL (pv is in here)
rm -f epel-release-6*.rpm
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh epel-release-6*.rpm

yum install -y mysql-server mysql pv

#Remove all data so that the mysqladmin, import, and export steps will work if the user wants to REPROVISION
rm -rf /var/lib/mysql/*

service mysqld restart

mysqladmin -u root password 'YourPassword'

mysqldump --verbose -u yourUserName -pYourPassword -h yourHost --databases databasename > ~/file.sql

pv -i 1 -p -t -e ~/file.sql | mysql -u yourUserName -pYourPassword

[coconitro]

Can confirm this is happening for me. Running into a similar issue utilizing Vagrant, pv and restoring a sample (but still large enough that progress output is nice) database into the server running in the vagrant instance for development purposes.

Could it be how pv is spitting its output?

[aaronjwood]

I should also mention that Vagrant does not show the current download progress when doing yum update or yum install whateverpackageshere. Seems to be the same kind of issue.

[aaronjwood]

I just realized the pv docs state: "...Its standard input will be passed through to its standard output and progress will be shown on standard error." I'm betting that's whats causing the issue.

[konstantinast]

Using 1.8.1 Vagrant version and still experience'ing the same problem with pv (pipe viewer).

As I was not able to get better solution to this problem my workaround is to use --force parameter with pipe viewer, but that result in a "spammed" vagrant up log (every second pipe viewer prints a new line with current progress and when a file is very big, then the log can get extra of hundreds or even thousands ofnot very necessary lines). To fight this unnecessary verbosity I add extra pipe viewer parameter --interval.

To wrap it up my currently used pipe viewer and mysql dump import command combo is:

pv --interval 5 --force /tmp/dump.sql | mysql

If there is a more elegant solution to this problem, I would be very glad to hear it.

[keshavkaul]

Why is this still closed?

[Kamilius]

Any progress on this one? Without pv importing big dumps simply looks more like a terminal hang.

[PatchRanger]

+1 for re-opening this issue, it is still valid. Workaround https://github.com/hashicorp/vagrant/issues/3543#issuecomment-185430702 works - but nevertheless it is a workaround, temporal solution, not resolution.

[Zoltanpetrik]

@mitchellh This is still an issue, I've put together a Vagrantfile which demonstrates it without any external dependencies like mysql dumps..

My environment where the issue exists:

• Microsoft Windows [Version 10.0.17763.678]
• Using built-in Windows cmd to execute the vagrant up command
• Vagrant 2.2.5
• VirtualBox 6.0.6 r130049
• ubuntu/xenial64 Box version 20190807.0.0

Vagrantfile:

$script = <<-SCRIPT
#!/usr/bin/env bash

sudo su
apt-get update
apt-get install pv -y

SIZE=200000000

echo === Starting pv with workaround
head -c $SIZE </dev/urandom | pv -s $SIZE --interval 1 --force | gzip -c > /dev/null 2>&1
echo === pv with workaround finished

echo === Starting pv normally, it takes the same amount of time but the shell hangs
head -c $SIZE </dev/urandom | pv -s $SIZE | gzip -c > /dev/null 2>&1
echo === pv finished
SCRIPT

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.define "pv-bug"
  config.vm.provision "shell", inline: $script
end

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.