Terraform-provider-libvirt: Adding a block device to a domain via xslt override causes terraform crash

Created on 31 Oct 2019  路  7Comments  路  Source: dmacvicar/terraform-provider-libvirt

System Information

Linux distribution

Ubuntu 18.04.3 LTS

Terraform version

terraform -v
Terraform v0.12.12
+ provider.libvirt (unversioned)
+ provider.template v2.1.2

Provider and libvirt versions

terraform-provider-libvirt -version
./terraform-provider-libvirt 0.6.0+git.1569597268.1c8597df
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
2019/10/31 15:48:57 virError(Code=38, Domain=7, Message='Failed to connect socket to '/run/user/1000/libvirt/libvirt-sock': No such file or directory')

Checklist

  • [ ] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?

    • [ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
    • [ ] If it is a very special case, consider using the _XSLT_ support in the provider to tweak the definition instead of opening an issue
    • [ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

  • [x] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Setup

I want to boot a domain from a block device on the host. According to domain doc it doesn't look like block-based disk devices are directly supported only files, urls, and volumes. So I am trying to use xslt to insert block devices into domain dynamically.

Here is example xml of what I am looking to do:

<disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/disk/by-id/dm-uuid-mpath-20017380030bb13be'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
    </disk>

main.tf

provider "libvirt" {
    uri = "qemu+ssh://kvmuser@kvmhost/system"
}

variable "scsi_disks" {
  description = "Disks to attach to the domain as block devices"
  type = map
  default = {

    disks = [
      { 
        target = "vda",
        blockdev = "/dev/disk/by-id/dm-uuid-mpath-20017380030bb13be"
      },
      {
        target = "vde",
        blockdev = "/dev/disk/by-id/ccw-0XBC23"
      }
    ]
  }
}

resource "libvirt_domain" "lun_guest" {
  name = "joe_terraform_lun_guest"
  vcpu = "2"
  memory = "2048"
  running = "true"

  network_interface {
    macvtap = "vlan1310"
  }

  console {
      type = "pty"
      target_port = "0"
  }

  xml {
    xslt = templatefile("${path.module}/xsl/xml_overrides.tmpl", var.scsi_disks)
  }
}

xslt override

<?xml version="1.0" ?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes" indent="yes"/>
  <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
  </xsl:template>

  <xsl:template match="/domain/devices">
  <xsl:copy>
  <xsl:apply-templates select="node()|@*"/>
  %{ for disk in disks ~}
    <xsl:element name="disk">
      <xsl:attribute name="type">block</xsl:attribute>
      <xsl:attribute name="device">disk</xsl:attribute>
      <xsl:element name="source">
        <xsl:attribute name="dev">${disk.blockdev}</xsl:attribute>
      </xsl:element>
      <xsl:element name="target">
        <xsl:attribute name="dev">${disk.target}</xsl:attribute>
      </xsl:element>
    </xsl:element>
  %{ endfor ~}
  </xsl:copy>
  </xsl:template>

  <!-- ide device are not supported on s390x, override to use scsi -->
  <xsl:template match="/domain/devices/disk[@device='cdrom']/target/@bus">
          <xsl:attribute name="bus">scsi</xsl:attribute>
  </xsl:template>
</xsl:stylesheet>

Running terraform apply results in a crash, with error Error: rpc error: code = Unavailable desc = transport is closing; however, the domain does in fact get created on the host, and boots successfully. My attempts to import the guest with terraform import libvirt_domain.lun_guest <uuid> result in a terraform crash as well.

Steps to Reproduce Issue

terraform init
terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # libvirt_domain.lun_guest will be created
  + resource "libvirt_domain" "lun_guest" {
      + arch        = (known after apply)
      + emulator    = (known after apply)
      + fw_cfg_name = "opt/com.coreos/config"
      + id          = (known after apply)
      + machine     = (known after apply)
      + memory      = 2048
      + name        = "joe_terraform_lun_guest"
      + qemu_agent  = false
      + running     = true
      + vcpu        = 2

      + console {
          + source_host    = "127.0.0.1"
          + source_service = "0"
          + target_port    = "0"
          + type           = "pty"
        }

      + network_interface {
          + addresses    = (known after apply)
          + hostname     = (known after apply)
          + mac          = (known after apply)
          + macvtap      = "vlan1310"
          + network_id   = (known after apply)
          + network_name = (known after apply)
        }

      + xml {
          + xslt = "<?xml version=\"1.0\" ?>\n<xsl:stylesheet version=\"1.0\"\n                xmlns:xsl=\"http://www.w3.org/1999/XSL/Transform\">\n  <xsl:output omit-xml-declaration=\"yes\" indent=\"yes\"/>\n  <xsl:template match=\"node()|@*\">\n     <xsl:copy>\n       <xsl:apply-templates select=\"node()|@*\"/>\n     </xsl:copy>\n  </xsl:template>\n\n  <xsl:template match=\"/domain/devices\">\n  <xsl:copy>\n  <xsl:apply-templates select=\"node()|@*\"/>\n      <xsl:element name=\"disk\">\n      <xsl:attribute name=\"type\">block</xsl:attribute>\n      <xsl:attribute name=\"device\">disk</xsl:attribute>\n      <xsl:element name=\"source\">\n        <xsl:attribute name=\"dev\">/dev/disk/by-id/dm-uuid-mpath-20017380030bb13be</xsl:attribute>\n      </xsl:element>\n      <xsl:element name=\"target\">\n        <xsl:attribute name=\"dev\">vda</xsl:attribute>\n      </xsl:element>\n    </xsl:element>\n      <xsl:element name=\"disk\">\n      <xsl:attribute name=\"type\">block</xsl:attribute>\n      <xsl:attribute name=\"device\">disk</xsl:attribute>\n      <xsl:element name=\"source\">\n        <xsl:attribute name=\"dev\">/dev/disk/by-id/ccw-0XBC23</xsl:attribute>\n      </xsl:element>\n      <xsl:element name=\"target\">\n        <xsl:attribute name=\"dev\">vde</xsl:attribute>\n      </xsl:element>\n    </xsl:element>\n    </xsl:copy>\n\n  </xsl:template>\n\n   <!-- terraform-libvirt provider doesn't support vlan attributes, so we need to inject them -->\n  <xsl:template match=\"/domain/devices/interface[@type='bridge']/source[@bridge='ovsbridge1']\">\n      <xsl:copy-of select=\".\"/>\n      <xsl:element name=\"virtualport\">\n          <xsl:attribute name=\"type\">openvswitch</xsl:attribute>\n      </xsl:element>\n      <xsl:element name=\"vlan\">\n          <tag id='1316'/>\n      </xsl:element>\n  </xsl:template>\n\n  <!-- ide device are not supported on s390x, override to use scsi -->\n  <xsl:template match=\"/domain/devices/disk[@device='cdrom']/target/@bus\">\n          <xsl:attribute name=\"bus\">scsi</xsl:attribute>\n  </xsl:template>\n \n</xsl:stylesheet>\n"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

libvirt_domain.lun_guest: Creating...

Error: rpc error: code = Unavailable desc = transport is closing


panic: runtime error: invalid memory address or nil pointer dereference
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xde6be6]
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: goroutine 26 [running]:
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/libvirt.resourceLibvirtDomainRead(0xc4200ff110, 0xe994a0, 0xc4202ce2c0, 0x0, 0x0)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/libvirt/resource_libvirt_domain.go:809 +0x1166
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/libvirt.resourceLibvirtDomainCreate(0xc4200ff110, 0xe994a0, 0xc4202ce2c0, 0x0, 0x0)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/libvirt/resource_libvirt_domain.go:566 +0x122f
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).Apply(0xc420432600, 0xc4200bcfa0, 0xc420596300, 0xe994a0, 0xc4202ce2c0, 0xc4204d6401, 0xc420240060, 0x1a3e390)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:286 +0x36c
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).Apply(0xc420432e80, 0xc4203c79a8, 0xc4200bcfa0, 0xc420596300, 0xc4205db288, 0xc4200ba7b0, 0xf4d360)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:285 +0xa4
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc42000e088, 0x1204960, 0xc42033fe60, 0xc4201e4480, 0xc42000e088, 0xc42033fd70, 0xf5b100)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/helper/plugin/grpc_provider.go:842 +0x864
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x1086280, 0xc42000e088, 0x1204960, 0xc42033fe60, 0xc4200bca50, 0x0, 0x0, 0x0, 0xc42036a000, 0x13e5)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/github.com/hashicorp/terraform/internal/tfplugin5/tfplugin5.pb.go:3019 +0x241
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc420095200, 0x120aae0, 0xc420095c80, 0xc4201ea300, 0xc420121b90, 0x1a3e7c0, 0x0, 0x0, 0x0)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc/server.go:966 +0x4bc
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc.(*Server).handleStream(0xc420095200, 0x120aae0, 0xc420095c80, 0xc4201ea300, 0x0)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc/server.go:1245 +0xd69
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc4200b6190, 0xc420095200, 0x120aae0, 0xc420095c80, 0xc4201ea300)
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc/server.go:685 +0x9f
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt: created by github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
2019-10-31T15:46:45.687-0400 [DEBUG] plugin.terraform-provider-libvirt:     /home/abuild/debbuild/BUILD/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df/_build/src/github.com/dmacvicar/terraform-provider-libvirt/vendor/google.golang.org/grpc/server.go:683 +0xa1
2019/10/31 15:46:45 [DEBUG] libvirt_domain.lun_guest: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalMaybeTainted
2019/10/31 15:46:45 [TRACE] EvalMaybeTainted: libvirt_domain.lun_guest encountered an error during creation, so it is now marked as tainted
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalWriteState
2019/10/31 15:46:45 [TRACE] EvalWriteState: removing state object for libvirt_domain.lun_guest
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalApplyProvisioners
2019/10/31 15:46:45 [TRACE] EvalApplyProvisioners: libvirt_domain.lun_guest has no state, so skipping provisioners
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalMaybeTainted
2019/10/31 15:46:45 [TRACE] EvalMaybeTainted: libvirt_domain.lun_guest encountered an error during creation, so it is now marked as tainted
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalWriteState
2019/10/31 15:46:45 [TRACE] EvalWriteState: removing state object for libvirt_domain.lun_guest
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalIf
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalIf
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalWriteDiff
2019/10/31 15:46:45 [TRACE] <root>: eval: *terraform.EvalApplyPost
2019/10/31 15:46:45 [ERROR] <root>: eval: *terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing
2019/10/31 15:46:45 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2019/10/31 15:46:45 [TRACE] [walkApply] Exiting eval tree: libvirt_domain.lun_guest
2019/10/31 15:46:45 [TRACE] vertex "libvirt_domain.lun_guest": visit complete
2019/10/31 15:46:45 [TRACE] dag/walk: upstream of "provider.libvirt (close)" errored, so skipping
2019/10/31 15:46:45 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2019/10/31 15:46:45 [TRACE] dag/walk: upstream of "root" errored, so skipping
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: read nil snapshot
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: no original state snapshot to back up
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 1
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
2019-10-31T15:46:45.690-0400 [DEBUG] plugin: plugin process exited: path=/home/ibmadmin/.terraform.d/plugins/linux_amd64/terraform-provider-libvirt pid=8749 error="exit status 2"
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2019/10/31 15:46:45 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
2019-10-31T15:46:45.693-0400 [DEBUG] plugin: plugin exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Additional information:

Similar to #418

community enhancement
Source
picture muroj
👍1

Most helpful comment

Crash logs indicate the problem is in resourceLibvirtDomainRead(). I've narrowed it down to the disk loop. There is no clause for diskDef.Source.Block so it falls into the else clause, which assumes the disk is a storage volume. I am working on adding the support for block devices, as the xslt override doesn't work.

picture muroj on 8 Nov 2019
👍2 ❤1

All 7 comments

Crash logs indicate the problem is in resourceLibvirtDomainRead(). I've narrowed it down to the disk loop. There is no clause for diskDef.Source.Block so it falls into the else clause, which assumes the disk is a storage volume. I am working on adding the support for block devices, as the xslt override doesn't work.

muroj picture muroj on 8 Nov 2019
👍2 ❤1

@muroj ok nice. For any info/help feel free to ping me

MalloZup picture MalloZup on 12 Nov 2019

I have hit this same issue adding a raw block device via xslt. Have you been able to make any progress @muroj ? I'd be happy to test a branch.

pcbentz picture pcbentz on 25 Nov 2019

@pcbentz I am working on getting PR #670 accepted. Would greatly appreciate any testing you can do.

https://github.com/muroj/terraform-provider-libvirt/tree/block_device?files=1

muroj picture muroj on 26 Nov 2019
❤1 👍1

Pulled down your branch - and build for ubuntu 16 using go version 1.13.4

Ran terraform with built plugin on an ubuntu 16 VM talking to an Ubuntu 18 kvm host accessing a raw block device on that host.

Worked as expected - nicely done!

pcbentz picture pcbentz on 26 Nov 2019
🎉1 👍1

well done @muroj I will look at your pr soon :+1:

MalloZup picture MalloZup on 26 Nov 2019
👍1

pr merged closing. Try out latest master

MalloZup picture MalloZup on 10 Dec 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

tormath1 picture tormath1  路  3Comments
jdandrea picture jdandrea  路  4Comments
lifeofguenter picture lifeofguenter  路  5Comments
muroj picture muroj  路  3Comments
marcindulak picture marcindulak  路  5Comments