Terraform-provider-libvirt: Not support real PCI Passthrough?

Created on 26 Oct 2018 · 12Comments · Source: dmacvicar/terraform-provider-libvirt

Version Reports:

Distro version of host:

Ubuntu 16.04

Terraform Version Report

terraform -v
Terraform v0.11.10

Provider and libvirt versions

terraform-provider-libvirt -version
.terraform/plugins/linux_amd64/terraform-provider-libvirt was not built correctly
Compiled against library: libvirt 1.3.1
Using library: libvirt 1.3.1
Running hypervisor: QEMU 2.5.0
Running against daemon: 1.3.1

If that gives you "was not built correctly", get the Git commit hash from your local provider repository:

git describe --always --abbrev=40 --dirty
1854aa93f614b0dfc54d5670bb27e9ac6bbda039-dirty

Description of Issue/Question

Setup

(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive info)

diff --git a/examples/ubuntu/cloud_init.cfg b/examples/ubuntu/cloud_init.cfg
index dd29f27..5a243e0 100644
--- a/examples/ubuntu/cloud_init.cfg
+++ b/examples/ubuntu/cloud_init.cfg
@@ -15,5 +15,5 @@
 ssh_pwauth: True
 chpasswd:
   list: |
-     root:terraform-libvirt-linux
+     root:passw0rd
   expire: False
diff --git a/examples/ubuntu/ubuntu-example.tf b/examples/ubuntu/ubuntu-example.tf
index c65603e..91fa3c5 100644
--- a/examples/ubuntu/ubuntu-example.tf
+++ b/examples/ubuntu/ubuntu-example.tf
@@ -7,7 +7,7 @@ provider "libvirt" {
 resource "libvirt_volume" "ubuntu-qcow2" {
   name   = "ubuntu-qcow2"
   pool   = "default"
-  source = "https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
+  source = "/home/fan/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
   format = "qcow2"
 }

@@ -39,7 +39,12 @@ resource "libvirt_domain" "domain-ubuntu" {

   network_interface {
     network_name = "default"
+    hostname = "new"
   }
+  network_interface {
+    passthrough = "enp2s16f6"
+  }
+

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

Additional Infos:

The generated vm configuration has:

    <interface type='direct'>
      <mac address='aa:bb:cc:dd'/>
      <source dev='enp2s16f6' mode='passthrough'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>

It will attach the host interface to vm with passthrough mode. The interface wasn't removed from host and assigned to vm. I think it should be:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x10' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </hostdev>

bug help wanted

Source

fkpwolf

Most helpful comment

Can you try with https://github.com/dmacvicar/terraform-provider-libvirt/pull/431 using a xslt transform?

dmacvicar on 13 Nov 2018

❤1 🎉1 👍1

All 12 comments

@fkpwolf thx for issue

@inercia maybe you have better insight here :smile: :japan:

MalloZup on 26 Oct 2018

Can you try with https://github.com/dmacvicar/terraform-provider-libvirt/pull/431 using a xslt transform?

dmacvicar on 13 Nov 2018

❤1 🎉1 👍1

thanks @dmacvicar. It is nice feature. But looks too advanced for me and for this task. I am not familiar with with xslt.
HCL(.tf file) is a kind of configuration language, and here introduce another one(xslt). I am not sure if this is a good idea.

fkpwolf on 14 Nov 2018

Ok, ignore everything I said.

I just realized we already support the _passthrough_ element, and this is about it not working. Then it is (probably, as we may not support all variations with a single attribute) a valid bug and we should read https://libvirt.org/formatdomain.html#elementsNICSHostdev and understand how it should work.

        } else if devI, ok := d.GetOk(prefix + ".passthrough"); ok {
            netIface.Source = &libvirtxml.DomainInterfaceSource{
                Direct: &libvirtxml.DomainInterfaceSourceDirect{
                    Dev:  devI.(string),
                    Mode: "passthrough",
                },
            }

We are setting the value you pass to the attribute as the device. We generate a _"direct"_ source, which from the documentation:

PCI Passthrough

A PCI network device (specified by the element) is directly assigned to the guest using generic device passthrough, after first optionally setting the device's MAC address to the configured value, and associating the device with an 802.1Qbh capable switch using an optionally specified element (see the examples of virtualport given above for type='direct' network devices). >Note that - due to limitations in standard single-port PCI ethernet card driver design - only SR-IOV (Single Root I/O Virtualization) virtual function (VF) devices can be assigned in this manner; to assign a standard single-port PCI or PCIe ethernet card to a guest, use the traditional device definition and Since 0.9.11

I would need to understand why we used direct (git history), and in case of switching to the _hostdev_ variant, how to not break SR-IOV capable devices, or how to use _VFIO device assignment_ in a way that works with both.

dmacvicar on 14 Nov 2018

👍1

Hello Duncan Mac-Vicar P.,
Is there any plan to address this ? Otherwise I have working patch which supports PCI Passthrough using (using 'hostdev'. element) would like to integrate it.
Thanks,

amitinfo2k on 5 Dec 2018

hi @amitinfo2k thank you for interest!

We don't have any time in short-term to fix so a contribution from your part would be awesome.

For submitting PRs read this one
https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/CONTRIBUTING.md
Feel free to work on this! and ask for infos! :sunflower:

MalloZup on 10 Dec 2018

any update on this? @amitinfo2k

fkpwolf on 12 Feb 2019

just submit a PR to address this issue #552. Looks I can't reuse existing network_interface element and have to create new type in schema.

My test tf script is:

variable "hostdev_list" {
  default = [
    [{
      domain = 0
      bus = 1
      solt = 16
      function = 0
    }],
    [{
      domain = 0
      bus = 1
      solt = 16
      function = 2
    }],
    [{
      domain = 0
      bus = 1
      solt = 16
      function = 4
    }],
  ]
}

....
hostdev = "${var.hostdev_list[count.index]}"

fkpwolf on 13 Feb 2019

@fkpwolf The PR #552 was closed. I understood that you agreed that you can achieve the results you need with XSLT. So I'd suggest to @MalloZup to close this one but please feel free to reopen if that's not the case.