Packer: virtualbox-vm fails with "unexpected EOF" / "panic: runtime error: invalid memory address / nil pointer dereference"

[harisund]

Overview of the Issue

In #8692 I was unable to use a OVA directly. As a continuation, I thought I would use the virtualbox-vm builder instead. So I started off by creating a VM with the VMDK file I extracted from the OVA archive. However, I am now running into a new error.

Reproduction Steps

Begin by acquiring the VMDK file.

• You can either download the ova from here and extract it ...
• Or you can download the vmdk directly from here

Then, build a virtual machine in VirtualBox. I have provided the commands here for easier reproduction

Build a quick virtual machine in VirtualBox

vboxmanage createvm --name for_packer --register
vboxmanage modifyvm for_packer\
    --memory 512 --boot1 disk --acpi on\
    --nic1 nat\
    --uart1 0x3F8 4 --uartmode1 file serialport.txt

vboxmanage storagectl for_packer\
    --name SATA --add sata --portcount 1

vboxmanage storageattach for_packer\
    --storagectl SATA --port 0 --device 0\
    --type hdd --medium ubuntu-18.04-server-cloudimg-amd64.vmdk

vboxmanage snapshot for_packer take snapshot-one

Now, create a packer build json. The key portions are basically just the snapshot name and the VM name , as far as I can understand reading the manual

packer build json file

{
  "variables": {
    "PACKER_OUTPUT_DIR": "{{env `PACKER_OUTPUT_DIR`}}"
  },
  "builders": [
    {
      "type": "virtualbox-vm",
      "name": "packer_vm",
      "vm_name": "for_packer",
      "attach_snapshot": "snapshot-one",
      "keep_registered": "false",
      "skip_export": "false",
      "headless": "true",
      "output_directory": "{{ user `PACKER_OUTPUT_DIR` }}",

      "http_directory": "cdata",

      "pause_before_connecting": "2s",
      "ssh_username": "ubuntu",
      "ssh_timeout": "2m",
      "ssh_private_key_file": "/home/harisund/.ssh/id_rsa",

      "shutdown_command": "sudo poweroff"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "hostname",
        "whoami",
        "ip -4 -br a",
        "lsb_release -a",
      ]
    }
  ]
}

Packer version

1.5.1

Simplified Packer Buildfile

Mentioned above.

Operating system and Environment details

OS: Tried it on both Win10 and Ubuntu x86_64
Environment variable: PACKER_OUTPUT_DIR is defined to be a directory

Log Fragments and crash.log files

2020/02/05 18:27:23 ui: packer_vm: output will be in this color.
2020/02/05 18:27:23 ui:
2020/02/05 18:27:23 Build debug mode: false
2020/02/05 18:27:23 Force build: false
2020/02/05 18:27:23 On error:
2020/02/05 18:27:23 Preparing build: packer_vm
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: panic: runtime error: invalid memory address or nil pointer dereference
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x294d56d]
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: goroutine 51 [running]:
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/builder/virtualbox/vm.(*Config).Prepare(0x0, 0xc0004161a0, 0x2, 0x2, 0x5e, 0xaa420000aa40, 0xaa4b0000aa44, 0xaa760000aa60, 0x8)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /Users/mmarsh/Projects/packer/builder/virtualbox/vm/config.go:50 +0x8d
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/builder/virtualbox/vm.(*Builder).Prepare(0xc0000cc1e0, 0xc0004161a0, 0x2, 0x2, 0x2, 0x2, 0x0, 0x0, 0xab0e0000ab09, 0xab160000ab11, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /Users/mmarsh/Projects/packer/builder/virtualbox/vm/builder.go:26 +0x50
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/packer/rpc.(*BuilderServer).Prepare(0xc0000b3bc0, 0xc000416160, 0xc0000e0100, 0x0, 0x0)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /Users/mmarsh/Projects/packer/packer/rpc/builder.go:97 +0xa0
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: reflect.Value.call(0xc0000dec00, 0xc00000e738, 0x13, 0x41034fe, 0x4, 0xc0001a5f18, 0x3, 0x3, 0x0, 0x0, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /usr/local/go/src/reflect/value.go:460 +0x5f6
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: reflect.Value.Call(0xc0000dec00, 0xc00000e738, 0x13, 0xc000073f18, 0x3, 0x3, 0xf1c1df, 0xc000164240, 0x0)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /usr/local/go/src/reflect/value.go:321 +0xb4
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: net/rpc.(*service).call(0xc0000b3c00, 0xc0004202d0, 0xc0000d8060, 0xc0000d8070, 0xc0000ec800, 0xc000416100, 0x3373b00, 0xc000416160, 0x16, 0x3373b40, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /usr/local/go/src/net/rpc/server.go:377 +0x16f
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: created by net/rpc.(*Server).ServeCodec
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:        /usr/local/go/src/net/rpc/server.go:474 +0x42b
2020/02/05 18:27:23 Build 'packer_vm' prepare failure: unexpected EOF
2020/02/05 18:27:23 ui error: unexpected EOF
2020/02/05 18:27:23 /usr/local/bin/packer: plugin process exited
2020/02/05 18:27:23 [INFO] (telemetry) Finalizing.
2020/02/05 18:27:24 waiting for all plugin processes to complete...
2020/02/05 18:27:24 /usr/local/bin/packer: plugin process exited

The full log is here, in case interested

Full log

2020/02/05 18:27:23 [INFO] Packer version: 1.5.1 [go1.13.5 linux amd64]
2020/02/05 18:27:23 Attempting to open config file: /root/.packerconfig
2020/02/05 18:27:23 [WARN] Config file doesn't exist: /root/.packerconfig
2020/02/05 18:27:23 Setting cache directory: /root/code/packer-reference/packer_cache.dir
2020/02/05 18:27:23 Plugin could not be found at /usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-builder-virtualbox-vm (exec: "/usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-builder-virtualbox-vm": stat /usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-builder-virtualbox-vm: no such file or directory). Checking same directory as executable.
2020/02/05 18:27:23 Current exe path: /usr/local/bin/packer
2020/02/05 18:27:23 Creating plugin client for path: /usr/local/bin/packer
2020/02/05 18:27:23 Starting plugin: /usr/local/bin/packer []string{"/usr/local/bin/packer", "plugin", "packer-builder-virtualbox-vm"}
2020/02/05 18:27:23 Waiting for RPC address for: /usr/local/bin/packer
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: [INFO] Packer version: 1.5.1 [go1.13.5 linux amd64]
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: Attempting to open config file: /root/.packerconfig
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: Setting cache directory: /root/code/packer-reference/packer_cache.dir
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: args: []string{"packer-builder-virtualbox-vm"}
2020/02/05 18:27:23 Received unix RPC address for /usr/local/bin/packer: addr is /tmp/packer-plugin012696962
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: Plugin address: unix /tmp/packer-plugin012696962
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: Waiting for connection...
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: Serving a plugin connection...
2020/02/05 18:27:23 Plugin could not be found at /usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-shell (exec: "/usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-shell": stat /usr/local/bin/packer-PACKERSPACE-plugin-PACKERSPACE-packer-provisioner-shell: no such file or directory). Checking same directory as executable.
2020/02/05 18:27:23 Current exe path: /usr/local/bin/packer
2020/02/05 18:27:23 Creating plugin client for path: /usr/local/bin/packer
2020/02/05 18:27:23 Starting plugin: /usr/local/bin/packer []string{"/usr/local/bin/packer", "plugin", "packer-provisioner-shell"}
2020/02/05 18:27:23 Waiting for RPC address for: /usr/local/bin/packer
2020/02/05 18:27:23 packer-provisioner-shell plugin: [INFO] Packer version: 1.5.1 [go1.13.5 linux amd64]
2020/02/05 18:27:23 packer-provisioner-shell plugin: Attempting to open config file: /root/.packerconfig
2020/02/05 18:27:23 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2020/02/05 18:27:23 packer-provisioner-shell plugin: Setting cache directory: /root/code/packer-reference/packer_cache.dir
2020/02/05 18:27:23 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2020/02/05 18:27:23 Received unix RPC address for /usr/local/bin/packer: addr is /tmp/packer-plugin000286918
2020/02/05 18:27:23 packer-provisioner-shell plugin: Plugin address: unix /tmp/packer-plugin000286918
2020/02/05 18:27:23 packer-provisioner-shell plugin: Waiting for connection...
2020/02/05 18:27:23 packer-provisioner-shell plugin: Serving a plugin connection...
2020/02/05 18:27:23 ui: packer_vm: output will be in this color.
2020/02/05 18:27:23 ui: 
2020/02/05 18:27:23 Build debug mode: false
2020/02/05 18:27:23 Force build: false
2020/02/05 18:27:23 On error: 
2020/02/05 18:27:23 Preparing build: packer_vm
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: panic: runtime error: invalid memory address or nil pointer dereference
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x294d56d]
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: 
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: goroutine 51 [running]:
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/builder/virtualbox/vm.(*Config).Prepare(0x0, 0xc0004161a0, 0x2, 0x2, 0x5e, 0xaa420000aa40, 0xaa4b0000aa44, 0xaa760000aa60, 0x8)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /Users/mmarsh/Projects/packer/builder/virtualbox/vm/config.go:50 +0x8d
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/builder/virtualbox/vm.(*Builder).Prepare(0xc0000cc1e0, 0xc0004161a0, 0x2, 0x2, 0x2, 0x2, 0x0, 0x0, 0xab0e0000ab09, 0xab160000ab11, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /Users/mmarsh/Projects/packer/builder/virtualbox/vm/builder.go:26 +0x50
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: github.com/hashicorp/packer/packer/rpc.(*BuilderServer).Prepare(0xc0000b3bc0, 0xc000416160, 0xc0000e0100, 0x0, 0x0)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /Users/mmarsh/Projects/packer/packer/rpc/builder.go:97 +0xa0
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: reflect.Value.call(0xc0000dec00, 0xc00000e738, 0x13, 0x41034fe, 0x4, 0xc0001a5f18, 0x3, 0x3, 0x0, 0x0, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /usr/local/go/src/reflect/value.go:460 +0x5f6
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: reflect.Value.Call(0xc0000dec00, 0xc00000e738, 0x13, 0xc000073f18, 0x3, 0x3, 0xf1c1df, 0xc000164240, 0x0)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /usr/local/go/src/reflect/value.go:321 +0xb4
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: net/rpc.(*service).call(0xc0000b3c00, 0xc0004202d0, 0xc0000d8060, 0xc0000d8070, 0xc0000ec800, 0xc000416100, 0x3373b00, 0xc000416160, 0x16, 0x3373b40, ...)
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /usr/local/go/src/net/rpc/server.go:377 +0x16f
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin: created by net/rpc.(*Server).ServeCodec
2020/02/05 18:27:23 packer-builder-virtualbox-vm plugin:    /usr/local/go/src/net/rpc/server.go:474 +0x42b
2020/02/05 18:27:23 Build 'packer_vm' prepare failure: unexpected EOF
2020/02/05 18:27:23 ui error: unexpected EOF
2020/02/05 18:27:23 /usr/local/bin/packer: plugin process exited
2020/02/05 18:27:23 [INFO] (telemetry) Finalizing.
2020/02/05 18:27:24 waiting for all plugin processes to complete...
2020/02/05 18:27:24 /usr/local/bin/packer: plugin process exited

I see the exact same behavior on Windows. If you want I can attach the log file from a Windows run.

[SwampDragons]

Thanks for opening this! We'll take a look as soon as we can.

[sylviamoss]

Hi @harisund! I've tested your template on our nightly version and it worked fine, differently from the v1.5.1 which also failed for me. This is going to be fixed in the next release specifically by this commit: https://github.com/hashicorp/packer/commit/7b5a868de2e72049a17ee5a575b39ed34f5e883c

Before then, you can use the nightly binaries and you can find them here: https://github.com/hashicorp/packer/releases/tag/nightly

Let me know if need more information about it 😄

[harisund]

Thank you very much @sylviamoss ! This is very cool, a "small step for the packer team, a huge leap for me" if you will. I am glad you were able to reproduce this or I would have felt guilty for making y'all waste time for nothing.

I can confirm the new binary works. I now run into the same missing HTTPIP/HTTPPort error but I see you already have a fix for #8692, so thank you once again !

[sylviamoss]

Good to hear that it worked for you too! I’m going to share the binaries for the solution of the other issue so you can test it as well, ok?

[harisund]

Absolutely

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.