Vagrant: Default gateway for guest

[flaccid]

Requesting to re-open https://github.com/mitchellh/vagrant/issues/436
Happy to provide more information or simply use this new issue.

[mitchellh]

Please provide more information. Why should this be a top-level feature if it can be done with shell provisioning?

[flaccid]

Hi Mitchell, I guess a lot of top level features can be done with shell
provisioning.

I found that when using just public_network e.g.
config.vm.network :public_network

This effectively just adds another default route, so the one to the host
remains. You would expect that by configuring network in this manner, the
guest would act like any other dhcp client on the LAN.
The guest can for example continue to reach out to the internet, but
packets forward through the NAT route wouldn't make it back to the remote
host.

On Fri, Oct 18, 2013 at 12:02 PM, Mitchell Hashimoto <
[email protected]> wrote:

Please provide more information. Why should this be a top-level feature if
it can be done with shell provisioning?

—
Reply to this email directly or view it on GitHub.

[mitchellh]

Yeah, this is tricky because it is so guest-specific. I'd say this should remain in the domain of the shell provisioner.

[niccolox]

this worked for me

bridged, static ip, public dns, vagrant as poor mans cloud

config.vm.network "public_network", :bridge => "eth0", ip:"79.37.142.40", :auto_config => "false", :netmask => "255.255.255.0"

config.vm.provision :shell, inline: <<-SCRIPT
sudo ifconfig eth1 79.37.142.40 netmask 255.255.255.0 up
sudo route add default gw 79.37.142.1

[flaccid]

Guest specific it is. net-tools is deprecated. I do it like this:

config.vm.network :public_network, ip: "192.168.0.50"     # with static IP, useful for server sharing

# IP address of your LAN's router
default_router = "192.168.0.1"

# change/ensure the default route via the local network's WAN router, useful for public_network/bridged mode
config.vm.provision :shell, :inline => "ip route delete default 2>&1 >/dev/null || true; ip route add default via #{default_router}"

[stanleygu]

A problem with these shell provisioning methods is that they don't persist after a vagrant reload.

[niccolox]

depends on the scripts, probably ansible or something idempotent

[kikitux]

can you provide more info?

On Thu, Apr 3, 2014 at 1:43 PM, Stanley Gu [email protected] wrote:

A problem with these shell provisioning methods is that they don't persist
after a vagrant reload.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2389#issuecomment-39401414
.

[stanleygu]

In an inline shell provisioner, I run

routes del default
routes add default gw mygatewayip eth0

However, after a restart, the gateway is lost so I end up needing to
restart with vagrant reload --provision.
On Apr 2, 2014 9:38 PM, "Alvaro Miranda" [email protected] wrote:

can you provide more info?

On Thu, Apr 3, 2014 at 1:43 PM, Stanley Gu [email protected]
wrote:

A problem with these shell provisioning methods is that they don't
persist
after a vagrant reload.

Reply to this email directly or view it on GitHub<
https://github.com/mitchellh/vagrant/issues/2389#issuecomment-39401414>
.

Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2389#issuecomment-39412100
.

[kikitux]

eth0 is nat, why the deault gateway setup in eth0 doesnt work ?

nat mean the host computer will do all the work in route..

nat ip is on 10.2....

can you paste the following commands, before and after you do your changes?

from the guest

ethtool eth0
route -n
ifconfig -a

grep network /vagrant/Vagrantfile

On Thu, Apr 3, 2014 at 5:49 PM, Stanley Gu [email protected] wrote:

In an inline shell provisioner, I run

routes del default
routes add default gw mygatewayip eth0

However, after a restart, the gateway is lost so I end up needing to
restart with vagrant reload --provision.
On Apr 2, 2014 9:38 PM, "Alvaro Miranda" [email protected] wrote:

can you provide more info?

On Thu, Apr 3, 2014 at 1:43 PM, Stanley Gu [email protected]
wrote:

A problem with these shell provisioning methods is that they don't
persist
after a vagrant reload.

Reply to this email directly or view it on GitHub<
https://github.com/mitchellh/vagrant/issues/2389#issuecomment-39401414>
.

Reply to this email directly or view it on GitHub<
https://github.com/mitchellh/vagrant/issues/2389#issuecomment-39412100>
.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2389#issuecomment-39412474
.

[stanleygu]

Sorry, I was replying on my phone and meant to say I added the gateway to eth1.

My vagrant file networking is set up as:

config.vm.network :public_network, :bridge => 'eth1'

And I have an inline shell provisioner script that adds the gateway IP:

route del default
route add default gw myGatewayIp eth1

Everything works as expected, with a dynamic bridged IP when vagrant first starts starts. Running route -n, I see

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         myGatewayIp  0.0.0.0         UG    0      0        0 eth1

However, if I run vagrant reload, I lose the gateway ip that was previously set, route -n shows:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.2.2        0.0.0.0         UG    0      0        0 eth0

So every time I ended up needing to re-provision, with something like vagrant reload --provision.

[niccolox]

auto lo eth0
iface lo inet loopback
iface eth0 inet static
address IP.FAIL.OVER
netmask 255.255.255.255
broadcast IP.FAIL.OVER
post-up route add IP.YOUR.DEDI dev eth0
post-up route add default gw IP.YOUR.DEDI
post-down route del IP.YOUR.DEDI dev eth0
post-down route del default gw IP.YOUR.DEDI

[kikitux]

you are using a setup that is not the default, and you are using a
provisioner to setup a network, so next time you load the machine if there
is no provision your script doesn't get executed.

What you an do is add your script to the next execution, not sure what OS
is, so you need to check the specifics..

On Fri, Apr 4, 2014 at 10:15 AM, Stanley Gu [email protected]:

Sorry, I was replying on my phone and meant to say I added the gateway to
eth1.

My vagrant file networking is set up as:

config.vm.network :public_network, :bridge => 'eth1'

And I have an inline shell provisioner script that adds the gateway IP:

route del default
route add default gw myGatewayIp eth1

Everything works as expected, with a dynamic bridged IP when vagrant first
starts starts. Running route -n, I see

Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 myGatewayIp 0.0.0.0 UG 0 0 0 eth1

However, if I run vagrant reload, I lose the gateway ip that was
previously set, route -n shows:

Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.2.2 0.0.0.0 UG 0 0 0 eth0
0.0.0.0 10.0.2.2 0.0.0.0 UG 0 0 0 eth1

So every time I ended up needing to re-provision, with something like vagrant
reload --provision.

—
Reply to this email directly or view it on GitHubhttps://github.com/mitchellh/vagrant/issues/2389#issuecomment-39505891
.

[flaccid]

I think some of the cases seen in this issue provide good support on why this should be af vagrant feature. For reliable persistence, Chef or Puppet should configure the network on the guest by configuration file so reloads etc. render the same state. The problem there is that its a egg before chicken situation on the first boot/provision - changing the gateway during a chef or puppet run could have some undesired problems or maybe not (I guess with chef server or puppet master the existing TCP connection would still work)?

[kikitux]

@stanleygu can you please put the following outpu when you have time? I got curious now.

please do it for when you set the gw manually and when the gw is not set, like Vagrant does.

route -n
tracert ip where ip is the ip you want to reach using the eth1 nic

with that will be able to see the routes are using to try to understand.

thanks

[stanleygu]

@kikitux

The default vagrant configuration is:

➜  ~  route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.2.2        0.0.0.0         UG    0      0        0 eth0
10.0.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
128.208.17.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

I am able to from the VM to external IPs just fine. However, I am unable to reach the server from an external IP without setting up the gateway, which I do with the shell provisioner:

route del default
route add default gw 128.208.17.100 eth1

Which results in the following route tables, that do allow me to reach the VM from an external IP to its dynamically assigned IP.

➜  ~  route -n 
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         128.208.17.100  0.0.0.0         UG    0      0        0 eth1
10.0.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
128.208.17.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

[livid]

@stanleygu This worked for me.

So it would be better if we can specify gateway and default route in Vagrantfile for public_network.

[flaccid]

+1

[atrauzzi]

This still isn't a great solution because you have to know the IP of the gateway. This won't work on a DHCP network.

[flaccid]

@atrauzzi Thats ok, I'm sure you could put that in :public_network or similar.

[atrauzzi]

@flaccid Not really, I have no way of knowing what the gateway's IP is, ever. It changes based on what network I'm running on.

[flaccid]

@atrauzzi Just look at the gateway IP your host computer gets from DHCP, I guess vagrant could be clever and use that if one isn't specified too.

[drocsid]

+1

I was going to solve this the same way. I think setting a default route would make a nice feature for public_network when a static IP is used. Can we do as @flaccid said and get the dhcp gateway, even if we are using a static address?

[niccolox]

it was this issue, that put me off Vagrant entirely

I have been looking into oVirt, Docker and alternatives

it feels to me that Vagrant open source leaves this unsolved so as to cross sell the proprietary Vagrant Share and Vagrant Cloud

if I remember correctly, the Virtualbox provider has more features than the VmWare provider for networking in Vagrant

but, Vagrant and public global static IP i.e. a universal cloud provider doesn't work

as Vagrant Share proves, it could be super easy

instead, its super hard, so, Vagrant, good bye

[delitescere]

For those who find this "too hard", here is a solution where, given the assumption you're connecting via the WiFi interface on OS/X, the _only_ thing you have to supply is a "public" IP address ("public" usually means an address on your LAN). In this example, it is via the UBUNTU_VM_PUBLIC_IP environment variable.

Yes, it is inconvenient, but remember for it to be declarative (and thus be a simple key/value or flag in the Vagrantfile), it needs to have implementations for all host OS's, all guest OS's, and all providers that are supported by Vagrant.

In the mean time, remember, a Vagrantfile is a Ruby source file too.

So, if you have an OS/X host with WiFi as the main network interface (which you probably do because otherwise you probably don't have a problem with bridged network interfaces because of WiFi router and MAC address issues), an Ubuntu (or other Linux-with-upstart) guest, the following will be useful, and other guests will be fairly easily provisioned too. If you're using the Virtualbox provider, then the lines that make the NIC use the paravirtual device also improves things, but are optional.

Enjoy a simplified version of my system, which also has other (totally optional) goodies about auto-sizing the VM based on your host's capacity, and (the essential in any case) disabling grub after an interrupted reboot.

Note that the public IP address must be in the same network as your host's address as this example also auto-discovers the host's default router and the host's WiFi interface's netmask. The address needs to be outside of your LAN's DHCP pool. I happen to have a subnet set aside which is the same size as the DHCP pool and I can make a calculation on the IP address (e.g. increment the 3rd octet) of my en0 to get an IP address for my VM (so the DHCP pool is "mirrored" with any Ubuntu VMs from hosts).

The Vagrantfile (notice the Ruby code!):

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"

def shell(*args)
  x=%x(#{args.join(" ")} 2>/dev/null).strip
  !x.empty? or nil and x
end

HOST_HOSTNAME = ENV["HOSTNAME"] || shell(:hostname) || ENV["USER"] || shell(:whoami) || abort("How do we make a hostname?")
HOST_CPU_COUNT = shell("sysctl -n hw.ncpu").to_i || 1
(GATEWAY, HOST_INTERFACE) = shell(%{netstat -rn | awk '/^default/ {print $2 " " $6}'}).split
VM_NETMASK = shell("ipconfig getpacket #{HOST_INTERFACE} | awk '/^subnet_mask/ {print $3}'")
VM_BRIDGE = ENV["VAGRANT_BRIDGE"] || "en0: Wi-Fi (AirPort)"
VM_PUBLIC_IP = ENV["UBUNTU_VM_PUBLIC_IP"] || abort("You must specify a static IP address for the bridged network interface")
VM_CPU_COUNT = (HOST_CPU_COUNT * 0.75).ceil # use around 75% of the logical CPUs
VM_CPU_CAP = 80 # use up to 80% of those CPUs
VM_MEM_SIZE = (shell("sysctl -n hw.memsize").to_i * 0.4).ceil / 1024 / 1024 # use around 40% of physical memory

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.ssh.forward_agent = true
  config.ssh.forward_x11 = true
  config.vm.box = "phusion/ubuntu-14.04-amd64"
  config.vm.box_check_update = false
  config.vm.hostname = "#{HOST_HOSTNAME}-ubuntu"

  config.vm.network "public_network", bridge: VM_BRIDGE, ip: VM_PUBLIC_IP, :auto_config => "false", :netmask => VM_NETMASK

  config.vm.provider "virtualbox" do |v|
     v.memory = VM_MEM_SIZE
     v.cpus = VM_CPU_COUNT
     v.customize ["modifyvm", :id, "--cpuexecutioncap", "#{VM_CPU_CAP}"]
     v.customize ["modifyvm", :id, "--nictype1", "virtio"]
     v.customize ["modifyvm", :id, "--nictype2", "virtio"]
     v.customize ["storagectl", :id, "--name", "SATA Controller", "--hostiocache", "off"]
  end

  config.vm.provision :shell, path: File.join(File.dirname(__FILE__), "provision.sh"), args: [GATEWAY]
end

The provision.sh script:

declare -r GATEWAY=${1:-}
# set default gateway for bridged interface, and make it happen at reboot
if [[ -n "${GATEWAY}" ]]; then
  echo "Setting default gateway through bridged interface"
  ip route delete default 2>&1 >/dev/null || true
  ip route add default via ${GATEWAY}
  echo "GATEWAY=${GATEWAY}" >/etc/default/vagrant-bridge
  [ -r /vagrant/vagrant-bridge.conf ] && cp /vagrant/vagrant-bridge.conf /etc/init/ # symlinks don't work
  initctl reload-configuration
fi

grub_updated() {
  grep -q '^GRUB_TIMEOUT=0$' /etc/default/grub && grep -q '^GRUB_DISABLE_RECOVERY="true"$' /etc/default/grub
}

if ! grub_updated; then
  sed -i -e 's/^GRUB_TIMEOUT=10$/GRUB_TIMEOUT=0/' -e 's/^#GRUB_DISABLE_RECOVERY="true"$/GRUB_DISABLE_RECOVERY="true"/' /etc/default/grub && update-grub
fi

The vagrant-bridge.conf upstart file to re-establish routes at reboot:

# vagrant-bridge - configure vagrant bridged network interface
description "configure vagrant bridge"
start on (net-device-up IFACE!=lo)
task
script
  [ -r /etc/default/vagrant-bridge ] && . /etc/default/vagrant-bridge
  [ -n "${GATEWAY}" ] || { stop; exit 1; }
  ip route delete default 2>&1 >/dev/null || true
  ip route add default via ${GATEWAY}
end script

[jonancm]

I feel that this doesn't solve the problem yet if you use DHCP, as @atrauzzi mentioned before.

I'm testing this on a trusty64 (Ubuntu Server) VM where I have the NAT interface with port forwarding for SSH (eth0) that Vagrant creates by default, plus a bridged interface to access my LAN (eth1), which I add like this:

config.vm.network "public_network"

I've noticed that Vagrant appends the following lines to /etc/network/interfaces:

#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
auto eth1
iface eth1 inet dhcp
    post-up route del default dev $IFACE
#VAGRANT-END

whereas the NAT interface (eth0) is set up in /etc/network/interfaces.d/eth0.cfg:

# The primary network interface
auto eth0
iface eth0 inet dhcp

I have the impression that the source of the problem is the post-up hook that removes the default gateway from eth1.

Wouldn't it be most convenient to have a way of telling Vagrant in the Vagrantfile not to add this post-up hook to eth1 and have the issue solved in that way?

PS: Manual configuration or shell provisioning are not feasible because I won't ever know in advance what IP my VM will get or what IP the gateway has. Using static IPs is not an option, and I can't disable or control the DHCP settings.

[flaccid]

@mitchellh I think we have enough evidence here for vagrant to solve this in a proper way instead of Vagrantfile messyness :+1:

[gregtampa]

need to add a gateway NOT using DHCP as it is a static IP at a datacenter..
will try to get those scripts working in one of the answers. Making my life harder.. might change over to LXC

[niccolox]

I've been adding a second public gateway on reload and it bypasses all
these hacks

config.vm.network "public_network", bridge: "eth0"

end
end

On Wed, May 20, 2015 at 2:00 PM, Gregory Hanis [email protected]
wrote:

need to add a gateway NOT using DHCP as it is a static IP at a
datacenter..
will try to get those scripts working in one of the answers. Making my
life harder.. might change over to LXC

—
Reply to this email directly or view it on GitHub
https://github.com/mitchellh/vagrant/issues/2389#issuecomment-104036706.

Nicholas Roberts
www.niccolox.org

[clay584]

Virtualbox, etc will assign a default gateway because that is the default behavior to give clients a default route so they can access the internet. This is not needed if you are bridging, so you can just add this to your vagrantfile in order to not install the NAT default gateway into the guest route table.

This will fix the issue for DHCP, bridged interfaces.

config.vm.provision "shell", run: "always", inline: <<-SHELL
    sudo sed -i s/DEFROUTE=yes/DEFROUTE=no/g /etc/sysconfig/network-scripts/ifcfg-<NAT_INTERFACE>
    sudo systemctl restart network
    ifconfig <BRIDGED_INTERFACE> | grep 'inet ' | awk '{print $2}' | xargs -I IPADDR echo "The bridged IP is IPADDR"
SHELL

[gregtampa]

don't know why you closed this issue, I will not be using vagrant till it has full network routing configurable

[anthonymanzo]

I've been trying to set as static as well (like gregtampa). I'm new to Vagrant and Chef so this may not be the right way. So far, I've only gotten partial success using the https://github.com/target/network_interfaces_v2-cookbook cookbook. It assigns the ip and gw but after that it hangs the 'vagrant up ' process becuase the ip has changed. I need to figure out how to overcome this in a best practices sort of way. Any suggestions ??

[anthonymanzo]

By the way, here's my recipe I'm executing for the static:

network_interface 'eth0' do
  bootproto 'static'
  address node['network_interfaces_v2']['address'] 
  netmask node['network_interfaces_v2']['netmask']
  gateway node['network_interfaces_v2']['gateway']
end

[EvanPurkhiser]

Using @clay584's shell provisioning I've had better luck on Centos 7 with

sudo systemctl restart NetworkManager

[neyfrota]

My vagrant file looks like this

...
config.vm.network "private_network", ip: "192.168.10.15"
config.vm.network "public_network"
config.vm.provision "shell", run: "always", inline: "/sbin/dhclient -r eth2; /sbin/route del default; /sbin/dhclient eth2"
...

lines 1 and 2 make sure i have this adapter setup

• Adapter 1: eth0 nat (for vagrant use)
• Adapter 2: eth1 hostonly (internal network to talk with other vms)
• Adapter 3: eth2 bridged (that we answer bridge adapter and guest dhcp a ip at same network as host)

At this point, guest default gw still point to network at eth0

Line 3 is the best way to modify default gw without know router ip. we release eth2 ip, remove default router, renew eth2 ip and by consequence fill default gw with host router ip

Done. guest default gw point to host router

[belonesox]

My workaround without hardcoding to IP/interface names, etc:
http://wiki.4intra.net/Blog:StasFomin/Easy_Networking_with_Vagrant_and_VirtualBox

[nileshvmware]

Hi team,
Need to get gateway IP details on VMs from scripts
Pls provide me

[vioan]

I have the same issue with vagrant on a FreeBSD host

[marcellos]

A provisioning workaround in my vagranfile for debian guests (using NAT on eth0 and private network default gw on eth1 surviving reboots):

node.vm.network "private_network", type: "dhcp", virtualbox__intnet: "mylocalnet", auto_config: true
ifpostcmd="/sbin/dhclient -r eth1; /sbin/ip route delete default 2>&1 >/dev/null || true; /sbin/dhclient eth1"
  node.vm.provision "shell", run: "always", inline: <<-SHELL
     #{ifpostcmd}
     grep -q -e "post-up #{ifpostcmd}" /etc/network/interfaces|| sed -i "/#VAGRANT-END/apost-up #{ifpostcmd}" /etc/network/interfaces
  SHELL

[pedroarthur]

In my case, it happens the default route via NAT has the same metric as the default route via the public network. Hence, the requests are received through the public network, but responses go through the NAT interface to get dropped.

For now, I am overriding the configuration by provisioning the following:

sed s/RouteMetric=100/RouteMetric=200/ \
    /run/systemd/network/10-netplan-enp0s3.network \
    | tee /etc/systemd/network/10-netplan-enp0s3.network \
    ;

In a Ubuntu Bionic box, this will change the NAT default route metric to 200 (the stack favors lower values; other routes have metric 100; hence, they get picked). I did this way cause it is easier to change NAT interface parameters than changing DHCP-enabled public networks parameters. However, a proper solution should encompass new configuration item for DHCP. Something like:

host.vm.network "public_network",
    use_dhcp_assigned_default_route: true,
    dhcp_route_metric: 50

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.