Terraform-provider-libvirt: CentOS7: 'libvirt_domain.domain-ubuntu' does not have attribute 'network_interface.0.addresses.0'

Created on 29 Jun 2018  路  5Comments  路  Source: dmacvicar/terraform-provider-libvirt

Version Reports:

Distro version of host:

cat /etc/*release | grep PRETTY_NAME
PRETTY_NAME="Ubuntu 16.04.4 LTS"

Terraform Version Report

terraform --version
Terraform v0.11.7
+ provider.libvirt (unversioned)
+ provider.template v1.0.0

Libvirt version

virsh --version
1.3.1

echo $CGO_ENABLED
1

terraform-provider-libvirt plugin version (git-hash)

https://github.com/dmacvicar/terraform-provider-libvirt/commit/b11e5b79ff40dc8f03d762d92226fabe315955dc

Description of Issue/Question

The problem may be CentOS7 image related, any hints what could be causing it are welcomed.

The default example https://github.com/dmacvicar/terraform-provider-libvirt/blob/21c507a4c44b510196b8a4a527aca921e9cd9b11/examples/ubuntu/ubuntu-example.tf is modified due to https://github.com/dmacvicar/terraform-provider-libvirt/issues/311, and uses CentOS7 image instead of Ubuntu:

cat ubuntu-example.tf 
# instance the provider
provider "libvirt" {
  uri = "qemu:///system"
}

# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "ubuntu-qcow2" {
  name = "ubuntu-qcow2"
  pool = "default"
  source = "https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1805.qcow2"  # fails
  #source = "https://dl.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2"  # works
  format = "qcow2"
}

# Create a network for our VMs
resource "libvirt_network" "vm_network" {
   name = "vm_network"
   addresses = ["10.0.1.0/24"]
}

# Use CloudInit to add our ssh-key to the instance
resource "libvirt_cloudinit" "commoninit" {
          name           = "commoninit.iso"
          ssh_authorized_key = "<ssh-key-here>"
        }


# Create the machine
resource "libvirt_domain" "domain-ubuntu" {
  name = "ubuntu-terraform"
  memory = "512"
  vcpu = 1

  cloudinit = "${libvirt_cloudinit.commoninit.id}"

  network_interface {
    network_name = "vm_network"
  }

  # IMPORTANT
  # Ubuntu can hang is a isa-serial is not present at boot time.
  # If you find your CPU 100% and never is available this is why
  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
      type        = "pty"
      target_type = "virtio"
      target_port = "1"
  }

  disk {
       volume_id = "${libvirt_volume.ubuntu-qcow2.id}"
  }
  graphics {
    type = "spice"
    listen_type = "address"
    autoport = "true"
  }
}

# Print the Boxes IP
# Note: you can use `virsh domifaddr <vm_name> <interface>` to get the ip later
output "ip" {
  value = "${libvirt_domain.domain-ubuntu.network_interface.0.addresses.0}"
}

TF_LOG=debug terraform apply -auto-approve
...
2018-06-29T16:35:49.364+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:49 [DEBUG] Network ID bd751399-a1e1-4cc6-b797-ac8b41a7fac4 successfully read
libvirt_network.vm_network: Creation complete after 8s (ID: bd751399-a1e1-4cc6-b797-ac8b41a7fac4)
2018-06-29T16:35:49.573+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:49 [DEBUG] sending command to qemu-agent
2018-06-29T16:35:51.223+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] qemu-agent response: {"return":[{"name":"lo","ip-addresses":[{"ip-address-type":"ipv4","ip-address":"127.0.0.1","prefix":8},{"ip-address-type":"ipv6","ip-address":"::1","prefix":128}],"hardware-address":"00:00:00:00:00:00"},{"name":"eth0","hardware-address":"6e:29:1a:02:49:e4"}]}
2018-06-29T16:35:51.223+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] Parsed response {Interfaces:[{Name:lo Hwaddr:00:00:00:00:00:00 IPAddresses:[{Type:ipv4 Address:127.0.0.1 Prefix:8} {Type:ipv6 Address:::1 Prefix:128}]} {Name:eth0 Hwaddr:6e:29:1a:02:49:e4 IPAddresses:[]}]}
2018-06-29T16:35:51.223+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] Interfaces obtained via qemu-agent: []
2018-06-29T16:35:51.223+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] getting domain addresses from networks
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] Interfaces: ([]libvirt.DomainInterface) <nil>
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] no interfaces could be obtained with qemu-agent: falling back to the libvirt API
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] Interfaces info obtained with libvirt API:
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: ([]libvirt.DomainInterface) {
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: }
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] read: addresses for '6E:29:1A:02:49:E4': []
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/06/29 16:35:51 [DEBUG] read: ifaces for 'ubuntu-terraform':
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: ([]map[string]interface {}) (len=1 cap=1) {
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:         (map[string]interface {}) (len=10) {
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=14) "wait_for_lease": (bool) false,
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=9) "addresses": ([]string) <nil>,
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=12) "network_name": (string) (len=10) "vm_network",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=6) "bridge": (string) "",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=7) "macvtap": (string) "",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=11) "passthrough": (string) "",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=3) "mac": (string) (len=17) "6E:29:1A:02:49:E4",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=8) "hostname": (string) "",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=10) "network_id": (string) (len=36) "bd751399-a1e1-4cc6-b797-ac8b41a7fac4",
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:                 (string) (len=4) "vepa": (string) ""
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt:         }
2018-06-29T16:35:51.224+0200 [DEBUG] plugin.terraform-provider-libvirt: }
libvirt_domain.domain-ubuntu: Creation complete after 6s (ID: ba7777bb-02b0-4d7d-84ac-6e4ed3b90459)
2018/06/29 16:35:51 [ERROR] root: eval: *terraform.EvalWriteOutput, err: Resource 'libvirt_domain.domain-ubuntu' does not have attribute 'network_interface.0.addresses.0' for variable 'libvirt_domain.domain-ubuntu.network_interface.0.addresses.0'
2018/06/29 16:35:51 [ERROR] root: eval: *terraform.EvalOpFilter, err: Resource 'libvirt_domain.domain-ubuntu' does not have attribute 'network_interface.0.addresses.0' for variable 'libvirt_domain.domain-ubuntu.network_interface.0.addresses.0'
2018/06/29 16:35:51 [ERROR] root: eval: *terraform.EvalSequence, err: Resource 'libvirt_domain.domain-ubuntu' does not have attribute 'network_interface.0.addresses.0' for variable 'libvirt_domain.domain-ubuntu.network_interface.0.addresses.0'
2018/06/29 16:35:51 [TRACE] [walkApply] Exiting eval tree: output.ip
2018/06/29 16:35:51 [DEBUG] plugin: waiting for all plugin processes to complete...

Error: Error applying plan:

1 error(s) occurred:

* output.ip: Resource 'libvirt_domain.domain-ubuntu' does not have attribute 'network_interface.0.addresses.0' for variable 'libvirt_domain.domain-ubuntu.network_interface.0.addresses.0'

The CentOS7 VM is started and running properly (virsh domifaddr ubuntu-terraform) and is accessible over ssh if an ssh pub key is injected.

https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1805.qcow2 fails and https://dl.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2 works.

bug
Source
picture marcindulak

Most helpful comment

can you try adding wait_for_lease = "true" under your network_interface?

picture night199uk on 21 Jul 2018
❤1 👍1

All 5 comments

@marcindulak how many network interface s does it haven the centos7 and the fedora cloud?

can you try to use this:

"${libvirt_domain.admin.network_interface.0.addresses[0]}"

instead of 

value = "${libvirt_domain.domain-ubuntu.network_interface.0.addresses.0}"
MalloZup picture MalloZup on 2 Jul 2018 
value = "${libvirt_domain.domain-ubuntu.network_interface.0.addresses[0]}"

does not help:

* output.ip: At column 61, line 1: list "libvirt_domain.domain-ubuntu.network_interface.0.addresses" does not have any elements so cannot determine type. in:

${libvirt_domain.domain-ubuntu.network_interface.0.addresses[0]}

As shown in the TF_LOG above addresses are empty. Terraform seems to be returning more quickly than in the case of the images (ubuntu, fedora) that start properly, it seems it is not waiting for the full creation of the vm.

The host interfaces look normal and are functional:

virsh domifaddr ubuntu-terraform
 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet0      ee:d5:1e:4a:c2:19    ipv4         10.0.1.88/24

[centos@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 7a:57:79:46:8d:04 brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.88/24 brd 10.0.1.255 scope global dynamic eth0
       valid_lft 3534sec preferred_lft 3534sec
    inet6 fe80::7857:79ff:fe46:8d04/64 scope link 
       valid_lft forever preferred_lft forever
marcindulak picture marcindulak on 5 Jul 2018

can you try adding wait_for_lease = "true" under your network_interface?

night199uk picture night199uk on 21 Jul 2018
❤1 👍1

https://github.com/dmacvicar/terraform-provider-libvirt/commit/67f5aa77d92615f5633fe2211566b531e57141d0 still failing on output using:

  network_interface {
    network_name = "vm_network"
    wait_for_lease = "true"
  }

but this time terraform apply takes longer and actually the assigned ip address is present in the log, but not in terraform.tfstate:

2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt: 2018/07/21 17:00:32 [DEBUG] read: ifaces for 'ubuntu-terraform':
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt: ([]map[string]interface {}) (len=1 cap=1) {
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:     (map[string]interface {}) (len=10) {
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=10) "network_id": (string) (len=36) "8766c64f-2c6c-4c7d-a64c-7e33ac515d54",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=12) "network_name": (*libvirtxml.DomainInterfaceSourceNetwork)(0xc42028bce0)({
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:             Network: (string) (len=10) "vm_network",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:             PortGroup: (string) ""
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         }),
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=7) "macvtap": (string) "",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=11) "passthrough": (string) "",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=3) "mac": (string) (len=17) "AA:ED:18:AE:E4:65",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=8) "hostname": (string) "",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=14) "wait_for_lease": (bool) true,
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=9) "addresses": ([]string) (len=2 cap=2) {
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:             (string) (len=10) "10.0.1.222",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:             (string) (len=25) "fe80::a8ed:18ff:feae:e465"
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         },
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=6) "bridge": (string) "",
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:         (string) (len=4) "vepa": (string) ""
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt:     }
2018-07-21T17:00:32.238+0200 [DEBUG] plugin.terraform-provider-libvirt: }
marcindulak picture marcindulak on 21 Jul 2018
❤1

This is due to this:

"network_name": (*libvirtxml.DomainInterfaceSourceNetwork)(0xc42028bce0)

It is fixed on master (210a4ce2235f07017ed708a6792fd3b86a8c511d) and released in 0.4.1.

dmacvicar picture dmacvicar on 28 Jul 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jlausuch picture jlausuch  路  3Comments
muroj picture muroj  路  7Comments
tormath1 picture tormath1  路  3Comments
MalloZup picture MalloZup  路  6Comments
mtuvikene picture mtuvikene  路  7Comments