Packer: Builder not waiting for specified ip_wait_timeout

Created on 16 Sep 2020  路  10Comments  路  Source: hashicorp/packer

Related to #9143
As with the other ticket I am also on a VPN connecting to the datacenter that is hosting our vCenter. I have been able to successfully build a Debian 10 VM without any issues.

Packer fails after waiting for IP for approx 1m30s. It's not consistent across attempts, but is always between 1m20s and 1m35s.

Packer: 1.6.2
Builder: vsphere-iso

Relevant portion of packer file

  "builders": [ 
    { 
      "type": "vsphere-iso",
      "CPUs": 2,
      "RAM": 4096,
      "boot_wait": "2s",
      "ip_wait_timeout": "3600s",
      "ip_settle_timeout": "3600s",

Collected PACKER_LOG and found the following. Looks as though it accepts the time provided, but does not actually wait.

2020/09/16 10:50:09 packer-builder-vsphere-iso plugin: [INFO] Waiting for IP, up to total timeout: 1h0m0s, , settle timeout: 1h0m0s
==> vsphere-iso: Waiting for IP...
==> vsphere-iso: Clear boot order...
==> vsphere-iso: Power off VM...
==> vsphere-iso: Destroying VM...
2020/09/16 10:50:59 [INFO] (telemetry) ending vsphere-iso
==> Wait completed after 1 minute 24 seconds
2020/09/16 10:50:59 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2020/09/16 10:50:59 machine readable: vsphere-iso,error []string{"unable to find an IP"}
Build 'vsphere-iso' errored after 1 minute 24 seconds: unable to find an IP

==> Builds finished but no artifacts were created.
bug buildevsphere

Most helpful comment

I figured it out.
I had a bad network setting in the cloud-config file. I had predicted the wrong predictable network interface name. (I still prefer eth0, eth1, eth2....)
The problem was that when the boot command ran it would boot up to the point where it got a network connection, pulled the cloud-config file using the seedfrom parameter, and immediately messed up the network configuration because I had ens33 and I needed to have ens192.

All 10 comments

Hey there, thanks for reaching out. Have you tried setting "ip_wait_timeout": "60m" or "ip_wait_timeout": "1h" to see what happens?
Anyways, "3600s" should work. We will have a look when we get a chance.

Yes, I tried all kinds of things: "30m", "60m", "1h", "1h30m". I tried removing it altogether because the docs that the default is 30m.
What I posted is just the last in a long line of attempts to get it working.

Hi, this happens when the address the vm obtains doesn't match the desired network range.

I was just talking to someone who came across this and it turned out they could fix it by disabling ipv6 on their vSwitch in vSphere.

After further testing I found the problem is in setting the boot_wait to 2s. When I remove boot_wait from my packer file, Packer will wait for the IP address. However, this screws up the timing on the boot command, it enters the boot command too late and leaves the VM sitting at a the manual installation screen.

So when the boot command is typed nothing happens with the machine? Do you mind sharing with us the complete template? Please remove any sensitive data.

I figured it out.
I had a bad network setting in the cloud-config file. I had predicted the wrong predictable network interface name. (I still prefer eth0, eth1, eth2....)
The problem was that when the boot command ran it would boot up to the point where it got a network connection, pulled the cloud-config file using the seedfrom parameter, and immediately messed up the network configuration because I had ens33 and I needed to have ens192.

I figured it out.
I had a bad network setting in the cloud-config file. I had predicted the wrong predictable network interface name. (I still prefer eth0, eth1, eth2....)
The problem was that when the boot command ran it would boot up to the point where it got a network connection, pulled the cloud-config file using the seedfrom parameter, and immediately messed up the network configuration because I had ens33 and I needed to have ens192.

Ran into the same issue, thanks for sharing!

Nice that you figured out what was happening. Can we close the issue?

Hi, this happens when the address the vm obtains doesn't match the desired network range.

This could explain the similar issue I'm seeing, where my VM will get an APIPA address first before I can get into set it to an address that is within the ip_wait_address setting.

I am seeing the issue where packer (1.6.5) in only waiting approximately 1 min for the IP before timing out. Notice that the IP timeout is set to 20 min. It only waits about 1 min 10 sec or so. The VM IP is static (specified) in a cloud-init user-data file. If I wait for cloud-init to finish and reboot, click retry, the IP is found.

2020/11/06 13:05:11 packer-builder-vsphere-iso plugin: [INFO] Waiting for IP, up to total timeout: 20m0s, settle timeout: 5s
2020/11/06 13:05:11 ui: ==> vsphere-iso: Waiting for IP...
2020/11/06 13:06:01 ui error: ==> vsphere-iso: unable to find an IP
2020/11/06 13:06:01 ui: ==> vsphere-iso: Step "StepWaitForIp" failed
2020/11/06 13:06:01 ui: ask: ==> vsphere-iso: [c] Clean up and exit, [a] abort without cleanup, or [r] retry step (build may fail even if retry succeeds)?
2020/11/06 13:27:42 packer-builder-vsphere-iso plugin: [INFO] Waiting for IP, up to total timeout: 20m0s, settle timeout: 5s
2020/11/06 13:27:42 ui: ==> vsphere-iso: Waiting for IP...
2020/11/06 13:27:43 packer-builder-vsphere-iso plugin: VM IP aquired: 10.255.70.144
2020/11/06 13:27:43 packer-builder-vsphere-iso plugin: VM IP is still the same: 10.255.70.144
2020/11/06 13:27:44 packer-builder-vsphere-iso plugin: VM IP is still the same: 10.255.70.144
2020/11/06 13:27:46 packer-builder-vsphere-iso plugin: VM IP is still the same: 10.255.70.144
2020/11/06 13:27:47 packer-builder-vsphere-iso plugin: VM IP is still the same: 10.255.70.144
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: VM IP is still the same: 10.255.70.144
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: VM IP seems stable enough: 10.255.70.144
2020/11/06 13:27:49 ui: ==> vsphere-iso: IP address: 10.255.70.144
2020/11/06 13:27:49 ui: ==> vsphere-iso: Using ssh communicator to connect: 10.255.70.144
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [INFO] Waiting for SSH, up to timeout: 2h46m40s
2020/11/06 13:27:49 ui: ==> vsphere-iso: Waiting for SSH to become available...
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [INFO] Attempting SSH connection to 10.255.70.144:22...
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [DEBUG] reconnecting to TCP connection for SSH
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [DEBUG] handshaking with SSH
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [DEBUG] handshake complete!
2020/11/06 13:27:49 packer-builder-vsphere-iso plugin: [DEBUG] Opening new ssh session
2020/11/06 13:27:50 packer-builder-vsphere-iso plugin: [INFO] agent forwarding enabled
2020/11/06 13:27:50 ui: ==> vsphere-iso: Connected to SSH!

Boot Command

 "boot_command": [
        " <wait><enter><wait>",
        "<f6><esc>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs>",
        "/casper/vmlinuz ",
        "initrd=/casper/initrd ",
        "autoinstall ",
        "ds=nocloud-net;",
        "<enter>"
      ],

Network portion of user-data

network:
    network:
      version: 2
      ethernets:
          ens192:
            dhcp4: no
            dhcp6: no
            addresses: [10.255.70.144/26]
            gateway4: 10.255.70.129
            nameservers:
              addresses: [10.255.0.1] 
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mwhooker picture mwhooker  路  3Comments

PartyImp picture PartyImp  路  3Comments

frezbo picture frezbo  路  3Comments

s4mur4i picture s4mur4i  路  3Comments

sourav82 picture sourav82  路  3Comments