Terraform: azurerm_virtual_machine Not Attaching Data Disks - "InvalidVhd"

[TStraub-rms]

Terraform Version

terraform version 0.7.10

Affected Resource(s)

• azurerm_virtual_machine
• azurerm_storage_blob

Terraform Configuration Files

variable "tenant_id" {}
variable "client_id" {}
variable "client_secret" {}
variable "subscription_id" {}
variable "location" {}
variable "module_name" {}
variable "vnet_address_space" {}
variable "subnet1" {}
variable "admin_username" {}
variable "admin_password" {}
variable "instance_type" {
  default = "standard_a1"
}
variable "instance_count" {
  default = 1
}
variable "data_disk_size_gb" {
  default = 1
}

provider "azurerm" {
  tenant_id       = "${var.tenant_id}"
  subscription_id = "${var.subscription_id}"
  client_id       = "${var.client_id}"
  client_secret   = "${var.client_secret}"
}

resource "azurerm_resource_group" "module" {
  name     = "${var.module_name}"
  location = "${var.location}"
}

resource "azurerm_virtual_network" "module" {
  name                = "${var.module_name}-Vnet1"
  resource_group_name = "${azurerm_resource_group.module.name}"
  address_space       = ["${var.vnet_address_space}"]
  location            = "${var.location}"
}

resource "azurerm_subnet" "subnet1" {
  name                 = "${var.module_name}-SubNet1"
  resource_group_name  = "${azurerm_resource_group.module.name}"
  virtual_network_name = "${azurerm_virtual_network.module.name}"
  address_prefix       = "${var.subnet1}"
}

resource "azurerm_network_interface" "module" {
  count               = "${var.instance_count}"
  name                = "${azurerm_resource_group.module.name}${count.index + 1}"
  resource_group_name = "${azurerm_resource_group.module.name}"
  location            = "${var.location}"

  ip_configuration {
    name                          = "${azurerm_resource_group.module.name}${count.index + 1}"
    subnet_id                     = "${azurerm_subnet.subnet1.id}"
    private_ip_address_allocation = "dynamic"
  }
}

resource "azurerm_storage_container" "module" {
  name                  = "vhds"
  resource_group_name   = "${azurerm_resource_group.module.name}"
  storage_account_name  = "${azurerm_storage_account.module.name}"
  container_access_type = "private"
}

resource "azurerm_storage_account" "module" {
  name                = "${azurerm_resource_group.module.name}random1"
  resource_group_name = "${azurerm_resource_group.module.name}"
  location            = "${var.location}"
  account_type        = "Standard_LRS"
}

resource "azurerm_storage_blob" "module" {
  name                   = "${azurerm_resource_group.module.name}${count.index + 1}data.vhd"
  count                  = "${var.instance_count}"
  resource_group_name    = "${azurerm_resource_group.module.name}"
  storage_account_name   = "${azurerm_storage_account.module.name}"
  storage_container_name = "${azurerm_storage_container.module.name}"
  type                   = "page"
  size                   = "${var.data_disk_size_gb * 1073741824 + 512}"
}

resource "azurerm_virtual_machine" "module" {
  name                  = "${azurerm_resource_group.module.name}${count.index + 1}"
  count                 = "${var.instance_count}"
  location              = "${var.location}"
  resource_group_name   = "${azurerm_resource_group.module.name}"
  network_interface_ids = ["${element(azurerm_network_interface.module.*.id, count.index)}"]
  vm_size               = "${var.instance_type}"

  storage_image_reference {
    publisher = "OpenLogic"
    offer     = "CentOS"
    sku       = "7.2"
    version   = "latest"
  }

  storage_os_disk {
    name          = "${azurerm_resource_group.module.name}${count.index + 1}os"
    vhd_uri       = "${azurerm_storage_account.module.primary_blob_endpoint}${azurerm_storage_container.module.name}/${azurerm_resource_group.module.name}${count.index + 1}os.vhd"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }

  storage_data_disk {
    name          = "${azurerm_resource_group.module.name}${count.index + 1}data"
    vhd_uri       = "${azurerm_storage_account.module.primary_blob_endpoint}${azurerm_storage_container.module.name}/${element(azurerm_storage_blob.module.*.name, count.index)}"
    create_option = "Attach"
    disk_size_gb  = "${var.data_disk_size_gb}"
    lun           = "0"
  }

  os_profile {
    computer_name  = "${azurerm_resource_group.module.name}${count.index + 1}"
    admin_username = "${var.admin_username}"
    admin_password = "${var.admin_password}"
  }
}

Expected Behavior

The Virtual Machine should be created with an empty data disk attached.

Actual Behavior

terraform apply will create all the resources without an issue except for the Virtual Machine.
When creating the VM and attaching the data disk there is an error:

Error applying plan:

1 error(s) occurred:

* azurerm_virtual_machine.module: compute.VirtualMachinesClient#CreateOrUpdate: Failure 
sending request: StatusCode=200 -- Original Error: Long running operation terminated with status 
'Failed': Code="InvalidVhd" Message="The specified cookie value in VHD footer indicates that disk 
'testdata1data' with blob https://<OMITTED>.blob.core.windows.net/vhds/testdata1data.vhd is not a 
supported VHD. Disk is expected to have cookie value 'conectix'."

Steps to Reproduce

1. terraform apply

Important Factoids

This had been working for us in the past without issue and is currently being used in several environments.
The error started occuring in the last few weeks with existing terraform code that has worked for a few months without modifications. I have compiled a small example above to demonstrate the problem.
I have tried several older version of Terraform back to (0.7.4) which lead me to believe this is a change in the Azure API or Terraforms interaction with the API.

It appears that this code is no longer creating a valid empty vhd that can be attached as a data disk to an Azure Virtual Machine:

resource "azurerm_storage_blob" "module" {
  name                   = "${azurerm_resource_group.module.name}${count.index + 1}data.vhd"
  count                  = "${var.instance_count}"
  resource_group_name    = "${azurerm_resource_group.module.name}"
  storage_account_name   = "${azurerm_storage_account.module.name}"
  storage_container_name = "${azurerm_storage_container.module.name}"
  type                   = "page"
  size                   = "${var.data_disk_size_gb * 1073741824 + 512}"
}

[quentez]

Same here. We've had to use custom scripts in the meantime as there is no other way to do this with Terraform that I could find of.

[TStraub-rms]

@quentez you mind sharing your work around? Could be helpful to myself and others.

[quentez]

@TStraub-rms What we ended up doing was to create one single disk with the size that we wanted using the storage_os_disk -> create_option -> Empty resource. After this was done, we reverted back to using azurerm_storage_blob, but with the source_uri pointing to the disk created earlier. Now, instead of creating the blobs from scratch, they'll be copied from the existing one which already has the correct VHD flags.

[zot24]

I have encounter the same issue I just change what you commented storage_os_disk -> create_option -> Empty and that make the trick and remove the azurerm_storage_blob block so the azure_virtual_machine did the job of creating the disk and attach it to it self.

[quentez]

@zot24 Happy to hear that it worked for you! In our case, we needed to be able to detach and re-attach to another instance later, which is why we needed the extra step with azurerm_storage_blob.

[StephenWeatherford]

@TStraub-rms Am I understanding correctly that you said this used to work in 0.7.4? I just tried with 0.7.4 and hit the same error. Thanks.

[StephenWeatherford]

@zot24 @TStraub-rms, @quentez As far as I've been able to determine, you can’t create an empty page blob and then immediately attach it to a VM - it has to be a valid VHD. The Azure guys I've talked to aren't aware of any change in that behavior.

Here are my suggestions:
1) Preferably, move to using managed disks (should be available with the next public release after 0.9.2, or available now in the master branch), I believe this will allow what you want (creating the disk separately and then attaching it)
2) Create a valid VHD first (via terraform or the Azure portal, or copying from an existing VHD) and then attach it, like zot24 says above
3) Using code such as https://github.com/Microsoft/azure-vhd-utils to initiatize the blob to a VHD before attaching

Stephen

[donaldgray]

I came to the same conclusion as @StephenWeatherford.

Just noticed that 0.9.3 has dropped and Managed Disk resource is available. There is also a sample showing creating a VM with an empty data disk so (1) should now be a viable option.

[TStraub-rms]

@StephenWeatherford Yeah this worked in the past, but something changed in Azure and it no longer works. After much investigation, I do not believe this has anything to do with TF, but rather in Azure. At the end of the day we have since rebuilt all the infrastructure that used the "old" method of attaching a newly created data disk, but we were stuck in a bad spot for a while when this issue arose.

Of course moving forward, managed disks are clearly the way to go anyway :)

[tombuildsstuff]

Hi @TStraub-rms

As you've mentioned - this functionality is no longer supported by Azure. Given this, and that there's some alternative options available as outlined in the comments above - I'm going to close this issue :)

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.