Vagrant: Add some way to increase disk space from Vagrantfile

[peterbraden]

At the moment you can increase memory in the vagrantfile, but if you want to increase disk space you have to use the VBoxManage tool to enlarge the image. It would be great to be able to specify this in the vagrantfile.

[mitchellh]

You can do this already.

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end

It just isn't documented yet. But will be soon. Sorry!

[peterbraden]

what would I put in the pre-boot list - does it take the same commands as the VBoxManage tool?

[mitchellh]

Yes, it is VBoxManage.

[peterbraden]

This gives me:

"Bringing machine 'default' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

VirtualBox Provider:

• virtualbox is not a valid event for customization. Valid events
  are: pre-import, pre-boot, post-boot"

[tmatilai]

The example should be:

config.vm.provider "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end

[haf]

What are you supposed to put in there?

I tried

v.customize 'pre-boot', ['modifyhd', :id, '--resize', '15360']

Giving

[app] Running 'pre-boot' VM customizations...
A customization command failed:

["modifyhd", "f7ee3447-5748-4562-9ca2-87639ec39515", "--resize", "15360"]

The following error was experienced:

VBoxManage: error: The given path 'f7ee3447-5748-4562-9ca2-87639ec39515' is not fully qualified
VBoxManage: error: Details: code VBOX_E_FILE_ERROR (0x80bb0004), component Medium, interface IMedium, callee nsISupports
VBoxManage: error: Context: "OpenMedium(Bstr(pszFilenameOrUuid).raw(), enmDevType, enmAccessMode, fForceNewUuidOnOpen, pMedium.asOutParam())" at line 178 of file VBoxManageDisk.cpp


Please fix this customization and try again.

[odino]

hey @haf did you find a way out? :)

[haf]

Hi @odino, no shortly afterwards VMWare blue-screened my Mac and corrupted the VM anyway so I had to redo all the work. This time I chose a larger size initially.

[tmatilai]

That :id will expand to the machine ID, while modifyhd expects ID or file system path of the disk image.
But I don't know how to get access to those in Vagrantfile context.

[snowch]

You can use vboxmanage showvminfo UUID to find the disk UUID

vboxmanage showvminfo e47cb0df-0799-4de7-8073-58239da86216
...
SATA Controller (0, 0): /home/me/VirtualBox VMs/mybox_1389356990435_59717/box-disk1.vmdk (UUID: 6eb5858a-a486-4b41-a1fb-1cb9f0eb7e5f)
...

I was hoping that the showvminfo command could be executed by vb.customize and the response parsed.

vb.customize [ "showvminfo", :id ]

Howver, vb.customize does not return a response.

[kikitux]

for virtualbox, if the disk is not VDI, then is not easy to expand the disk.

you should turn the guest off, clone the disk from vmdk to vdi, dettach
vmdk, attach vdi, expand vdi, then turn the machine on, and expand the
filesystem on the disk.

What OS you are using ?? I think will be way easier create a new VM with
the desired disk with packer and then use it in vagrant.

Alvaro.

On Wed, Jan 22, 2014 at 11:50 AM, snowch [email protected] wrote:

You can use vboxmanage showvminfo UUID to find the disk UUID

vboxmanage showvminfo e47cb0df-0799-4de7-8073-58239da86216
...
SATA Controller (0, 0): /home/me/VirtualBox VMs/mybox_1389356990435_59717/box-disk1.vmdk (UUID: 6eb5858a-a486-4b41-a1fb-1cb9f0eb7e5f)
...

I was hoping that the showvminfo command could be executed by
vb.customize and the response parsed.

vb.customize [ "showvminfo", :id ]

Howver, vb.customize does not return a response.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2339#issuecomment-32973372
.

[snowch]

@kikitux it's not easy, but it should be possible to automate shouldn't it?

The guest OS is CentOS 6.4. The host OS could be any OS supported by Vagrant.

The driver behind this requirement to increase the disk size of an existing box is that many security teams will only the use a 'trusted' third party boxes. It is better for me to expand a trusted image than create and make available my own non trusted image.

I've just taken a look at packer - thanks for the tip! I'll take a look and see if that solves the problem.

[kikitux]

When you say my box, then i understand you are creating in some way the box.

I will say the easiest will be when you create the box, create a BIG disk,
say 50GB, as the disk will be an sparse disk, it will use only the size
that is actually used.

I do custom boxes for my test and I use 4 disks of 32GB, so should use
128GB, but the vagrant box is just 700MB for the OS, or couple of gigs when
I install the software.

If this approach work for you, then I will suggest import the box to
vagrant, do the expansion manually (or by some script outside vagrant) and
then export the box and you will get the new machine.

If, the boxes are not yours, and are 3rd party (created, managed by
others), then I will suggest teach the users how to do it, (teach them to
catch fish), or you will be the man in the middle all the time.

I don't think this is an Vagrant issue, so I think the best is to talk in
the mailing list..

I am more than happy to assist if you need any help with the
commands/order/etc, in the mailing list.

(vagrantup.com -> about -> support -> check around for the mailing list)

Alvaro.

On Wed, Jan 22, 2014 at 7:34 PM, snowch [email protected] wrote:

@kikitux https://github.com/kikitux it's not easy, but it is possible
to automate isn't it?

The guest OS is CentOS 6.4. The host OS could be any OS supported by
Vagrant.

The driver behind this requirement is that many security teams will only
the use a 'trusted' third party boxes. It is better for me to expand a
trusted image than create and make available my own non trusted image.

I've just taken a look at packer - thanks for the tip! Packer may overcome
the issue of automating the image, but it looks as though it adds another
installation step for users of my box. Asking my users to install vagrant,
virtualbox, cygwin/putty (windows users), 3 vagrant plugins, and packer is
a big barrier to people using my box.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2339#issuecomment-32996903
.

[snowch]

@kikitux I'm not creating a box. I don't want to do this because my security team don't trust boxes that are available in places like dropbox.

I'm using a box from a trusted place (the specific box is http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210.box). Someone raised an issue against them 4 months ago to increase the box size:

https://github.com/puppetlabs/puppet-vagrant-boxes/pull/33

The pull request seems to still be open :(

Thanks for your feedback and kind offer of assistance.

[kikitux]

Hello, I will give you my best answer based on my expertise on linux I have.

The box came out of the box like this:

[root@localhost ~]# df -Ph
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 8.4G 1.1G 6.9G 14% /
tmpfs 230M 0 230M 0% /dev/shm
/dev/sda1 485M 33M 427M 8% /boot
/vagrant 725G 475G 251G 66% /vagrant
[root@localhost ~]#

Plenty of space for the OS.

So, my suggestion for your problem will be:

• Start with this machine as came from the vendor
• Download it, import it into vagrant.
• Start it up, add a 2nd hard disk, expand the VG, expand the LV
• Update to the latest (yum -y update)
• Package and make available internally

as the box upstream is not suitable, and the vendor is performing slow on
this, you should do what I suggest.

If there is any linux admin there or any linux standard, then the 2nd disk
may be used with a new volume group and create a new mount point for your
applications, say /opt /srv /usr/local , etc.

Hope this helps.

Alvaro.

On Wed, Jan 22, 2014 at 8:15 PM, snowch [email protected] wrote:

@kikitux https://github.com/kikitux I'm not creating a box. I don't
want to do this because my security team don't trust boxes that are
available in places like dropbox.

I'm using a box from a trusted place (the specific box is
http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210.box).
Someone raised an issue against them 4 months ago to increase the box size:

puppetlabs/puppet-vagrant-boxes#33https://github.com/puppetlabs/puppet-vagrant-boxes/pull/33

The pull request seems to still be open :(

Thanks for your feedback and kind offer of assistance.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2339#issuecomment-32998454
.

[snowch]

@kikitux your recommendation is pretty much what I am doing at the moment until automated disk resizing is figured out:

In may vagrantfile:

file_to_disk = File.realpath( "." ).to_s + "/disk.vdi"

        if ARGV[0] == "up" && ! File.exist?(file_to_disk) 
           puts "Creating 30GB disk #{file_to_disk}."
           vb.customize [
                'createhd', 
                '--filename', file_to_disk, 
                '--format', 'VDI', 
                '--size', 300000 * 1024 # 30 GB
                ] 
           vb.customize [
                'storageattach', :id, 
                '--storagectl', 'SATA Controller', 
                '--port', 1, '--device', 0, 
                '--type', 'hdd', '--medium', 
                file_to_disk
                ]
        end

and then:

carbon.vm.provision "shell", path: "scripts/add_new_disk.sh"

https://github.com/snowch/carbon-products-development-environment/blob/master/scripts/add_new_disk.sh

[kikitux]

couple of suggestion.

If you change the script to:

pvcreate /dev/sdb
vgextend VolGroup /dev/sdb

as the new disk is VDI, you avoid creating partitions on it, and then
will be able to add more space in the future if needed.

you could expand the VDI, rescan the base disk, pvscan, pvresize, and
the vg will grow on the fly.

other tip, if you do this:

lvextend -l +$(pvdisplay /dev/sdb -c | cut -d: -f11) /dev/VolGroup/lv_root

you will extend for the exact number of FREE extents that are
available in the new disk.

note is +number to add the number..

Other than that, i think your scripts are fantastic good.

Alvaro.

On Thu, Jan 23, 2014 at 9:36 AM, snowch [email protected] wrote:

@kikitux https://github.com/kikitux your recommendation is pretty much
what I have done so far:

In may vagrantfile:

file_to_disk = File.realpath( "." ).to_s + "/disk.vdi"

    if ARGV[0] == "up" && ! File.exist?(file_to_disk)
       puts "Creating 30GB disk #{file_to_disk}."
       vb.customize [
            'createhd',
            '--filename', file_to_disk,
            '--format', 'VDI',
            '--size', 300000 * 1024 # 30 GB
            ]
       vb.customize [
            'storageattach', :id,
            '--storagectl', 'SATA Controller',
            '--port', 1, '--device', 0,
            '--type', 'hdd', '--medium',
            file_to_disk
            ]
    end

and then:

carbon.vm.provision "shell", path: "scripts/add_new_disk.sh"

https://github.com/snowch/carbon-products-development-environment/blob/master/scripts/add_new_disk.sh

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2339#issuecomment-33064917
.

[snowch]

Very nice. Thank you!

[bechampion]

no news about increasing the size on pre-boot yet ? or some what to get the full path to the vdi from the :id (UUID) ?

[kikitux]

that won't work, as when the machine is created from the base box, the disk are not VDI, hence can't be extended.

[xacaxulu]

Does anyone have the remotest idea about how to put this into a .kitchen.yml format?

[renan]

My solution was to resize the base box itself, like follows:

vagrant box add ubuntu/trusty64 --box-version 20150609.0.9
cd ~/.vagrant.d/boxes/ubuntu-VAGRANTSLASH-trusty64/20150609.0.9/virtualbox/
VBoxManage clonehd box-disk1.vmdk tmp-disk.vdi --format vdi
VBoxManage modifyhd tmp-disk.vdi --resize 61440
VBoxManage clonehd tmp-disk.vdi resized-disk.vmdk --format vmdk
rm tmp-disk.vdi box-disk1.vmdk
mv resized-disk.vmdk box-disk1.vmdk

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        60G  1.8G   55G   4% /

With 60GB instead of the base 40GB.

[hbrls]

I followed this blog post, and it works well.

http://blog.lenss.nl/2012/09/resize-a-vagrant-vmdk-drive/

[BenMorganIO]

@mitchellh Do you know what's supposed to go into those ellipses?

[dvfabbri]

@BenMorganIO did you determine what goes in those ellipses to set disk size?

[BenMorganIO]

Nope...

[dvfabbri]

@mitchellh Can you provide an example of the commands required to create a VM with 500GB disk in the Vagrant file? I have been looking around to set this up, but I currently am just stuck using Gparted to resize post creation.

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["..."]
end

[dimasdanz]

The command should be something like this

config.vm.customize "virtualbox" do |vb|
  vb.customize "pre-boot", ["modifyhd", :id, "--resize", "32678"] #32678 = 32GiB
end

But, this would not work because:

1. UUID passed by Vagrant to VBoxManage is not hdd's UUID listed by VBoxManage list hdds so modifyhd is not possible. It displays error like this

Stderr: VBoxManage.exe: error: The given path <UUID-here> is not fully qualified

1. Even if we can get hdd UUID to execute modifyhd, storage format of boxes downloaded by Vagrant is VMDK. modifyhd command cannot be performed on VMDK format

So, as of now, I think the only way to increase disk space is @renan's way
You can also modify it post creation but you have to remove attached hdd and reattach the new one.

[jhgorse]

so this what we need?

config.vm.provider "virtualbox" do |v|
  vmdk_path = get_vagrant_box_path()
  tmp1 = vmdk_path."tmp1"
  tmp2 = vmdk_path."tmp2"

  v.customize "pre-boot", ["clonehd",  vmdk_path,  tmp1, "--format vdi"]
  v.customize "pre-boot", ["modifyhd", tmp1, "--resize", "32678"]
  v.customize "pre-boot", ["clonehd",  tmp1,       tmp2, "--format vmdk"]

  rm vmdk_path tmp1
  mv tmp2 vmdk_path
end

[mleopold]

Hi,
@mitchellh The issue was closed in 2013, is there any chance of reopening it? it seems this issue is still around and the suggestions are inconvenient at best. A option for the disk-size at the first creation or more guided resize later sounds like it would be appreciated.

Br,
Martin

[Shuliyey]

@jhgorse thanks for the script.

Just a question I got the below error around the below two lines

  tmp1 = vmdk_path."tmp1"
  tmp2 = vmdk_path."tmp2"

There is a syntax error in the following Vagrantfile. The syntax error
message is reproduced below for convenience:


/mnt/sdb/VirtualBox/dye/Vagrantfile:55: syntax error, unexpected tSTRING_BEG, expecting '('
    tmp1 = vmdk_path."tmp1"
                      ^
/mnt/sdb/VirtualBox/dye/Vagrantfile:56: syntax error, unexpected tSTRING_BEG, expecting '('
    tmp2 = vmdk_path."tmp2"

[skortchmark9]

@renan's solution seems to be working for me - However, I also needed to edit the box.ovf file, where it says ovf:capacity

[snowiow]

I had the same problem and thanks to @renan 's solution I got started. But it wasn't the complete solution for me, because I had an UUID conflict. I have written up my complete solution path here. I hope it is of any help for anybody.

[BenMorganIO]

@snowiow Thankyou :heart:

[ogero]

I've just made a Python script that does the VMDK resize and deal with UUID all the way..
But regrettably, it can't be automatically used inside Vagrantfile.. the steps are something like this:

>vagrant up --no-provision
>vagrant halt
>wget https://gist.githubusercontent.com/ogero/14692edf0702bfefe0455d6e7a9dc6de/raw/849c39404b649abaf6985901505170cc2f366c46/expand_disk.py
>chmod +x expand_disk.py
>python expand_disk.py
>vagrant up --no-provision
>vagrant ssh --command "sudo resize2fs /dev/sda1"
>vagrant provision

[Doume3]

Hi, I saw this contribution which help me a lot :+1: https://github.com/sprotheroe/vagrant-disksize

[steveswinsburg]

With the vagrant-disksize plugin I get a resized disk but the partition underneath is still the original so has to be resized within the guest. This makes it hard to automate.

[afturner]

If you'd like an automated solution, I did this:

config.disksize.size = '10GB'
config.vm.provision "shell", inline: <<-SHELL
    sudo resize2fs /dev/sda1
SHELL

And install vagrant-disksize:

vagrant plugin install vagrant-disksize

YMMV. sda1 was the right one for me, but you may need to ssh and double check.

[CristianCantoro]

@Doume3 said:

Hi, I saw this contribution which help me a lot +1 https://github.com/sprotheroe/vagrant-disksize

I confirm that using the vagrant-disksize plugin with a Ubuntu base box solved the issue for me. It also seems that I had nothing else to do (no resize2fs, no lvextend, etc.) and it worked "automagically".

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.