Terraform-provider-azurerm: Virtual machine scale set remove the publicIPAddressConfiguration when performing simple plan change
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform (and AzureRM Provider) Version
$ terraform -v
Terraform v0.11.14
+ provider.azurerm v1.30.1
+ provider.random v2.1.2
+ provider.template v2.1.2
Affected Resource(s)
azurerm_virtual_machine_scale_set
Terraform Configuration Files
resource "azurerm_virtual_machine_scale_set" "main" {
name = "${local.resource_name}"
location = "${var.location}"
resource_group_name = "${var.resource_group}"
overprovision = "${var.overprovision}"
upgrade_policy_mode = "Manual"
automatic_os_upgrade = "${var.automatic_os_upgrade}"
single_placement_group = "${var.single_placement_group}"
priority = "${var.priority}"
tags {
environment = "${var.environment}"
}
zones = ["${var.zones}"]
sku {
name = "${var.sku_name}"
tier = "${var.sku_tier}"
capacity = "${var.capacity_count}"
}
os_profile {
computer_name_prefix = "${local.computer_prefix}"
admin_username = "${var.admin_username}"
custom_data = "${var.custom_data}"
}
os_profile_linux_config {
disable_password_authentication = true
ssh_keys = {
path = "/home/ubuntu/.ssh/authorized_keys"
key_data = "${file(var.ssh_key)}"
}
}
network_profile {
name = "${local.network_profile_name}-network"
primary = true
network_security_group_id = "${var.nsg_id}"
accelerated_networking = "${var.enable_accelerated_networking}"
ip_configuration {
name = "${local.network_profile_name}"
primary = true
subnet_id = "${var.subnet_id}"
application_security_group_ids = ["${var.asg_ids}"]
load_balancer_backend_address_pool_ids = ["${var.lb_backend_address_pool_ids}"]
public_ip_address_configuration {
name = "${local.network_profile_name}-public"
domain_name_label = "${var.resource_group}"
idle_timeout = 4
}
}
}
storage_profile_image_reference {
id = "${var.image_id}"
}
storage_profile_os_disk {
name = ""
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Standard_LRS"
}
storage_profile_data_disk {
lun = 0
caching = "ReadWrite"
create_option = "Empty"
disk_size_gb = "${var.data_disk_size}"
}
}
Debug Output
I managed to reproduce by running this test case
diff --git a/azurerm/resource_arm_virtual_machine_scale_set_test.go b/azurerm/resource_arm_virtual_machine_scale_set_test.go
index 6bf155e3..24e2d390 100644
--- a/azurerm/resource_arm_virtual_machine_scale_set_test.go
+++ b/azurerm/resource_arm_virtual_machine_scale_set_test.go
@@ -144,6 +144,37 @@ func TestAccAzureRMVirtualMachineScaleSet_basicPublicIP(t *testing.T) {
})
}
+func TestUnitAzureRMVirtualMachineScaleSet_basicPublicIP_alex(t *testing.T) {
+ resourceName := "azurerm_virtual_machine_scale_set.test"
+ ri := tf.AccRandTimeInt()
+ location := testLocation()
+ config := testAccAzureRMVirtualMachineScaleSet_basicPublicIP(ri, location)
+ updatedConfig := testAccAzureRMVirtualMachineScaleSet_basicPublicIP_update_tags(ri, location)
+
+ resource.ParallelTest(t, resource.TestCase{
+ PreCheck: func() { testAccPreCheck(t) },
+ Providers: testAccProviders,
+ CheckDestroy: testCheckAzureRMVirtualMachineScaleSetDestroy,
+ Steps: []resource.TestStep{
+ {
+ Config: config,
+ Check: resource.ComposeTestCheckFunc(
+ testCheckAzureRMVirtualMachineScaleSetExists(resourceName),
+ testCheckAzureRMVirtualMachineScaleSetIsPrimary(resourceName, true),
+ testCheckAzureRMVirtualMachineScaleSetPublicIPName(resourceName, "TestPublicIPConfiguration"),
+ ),
+ },
+ {
+ Config: updatedConfig,
+ Check: resource.ComposeTestCheckFunc(
+ testCheckAzureRMVirtualMachineScaleSetExists(resourceName),
+ testCheckAzureRMVirtualMachineScaleSetPublicIPName(resourceName, "TestPublicIPConfiguration"),
+ ),
+ },
+ },
+ })
+}
+
func TestAccAzureRMVirtualMachineScaleSet_basicApplicationSecurity(t *testing.T) {
resourceName := "azurerm_virtual_machine_scale_set.test"
ri := tf.AccRandTimeInt()
@@ -1674,11 +1705,110 @@ resource "azurerm_virtual_machine_scale_set" "test" {
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
upgrade_policy_mode = "Manual"
+
+ tags = {
+ test = "test"
+ }
+
+ sku {
+ name = "Standard_D1_v2"
+ tier = "Standard"
+ capacity = 0
+ }
+
+ os_profile {
+ computer_name_prefix = "testvm-%[1]d"
+ admin_username = "myadmin"
+ admin_password = "Passwword1234"
+ }
+
+ network_profile {
+ name = "TestNetworkProfile-%[1]d"
+ primary = true
+
+ ip_configuration {
+ name = "TestIPConfiguration"
+ subnet_id = "${azurerm_subnet.test.id}"
+ primary = true
+
+ public_ip_address_configuration {
+ name = "TestPublicIPConfiguration"
+ domain_name_label = "test-domain-label-%[1]d"
+ idle_timeout = 4
+ }
+ }
+ }
+
+ storage_profile_os_disk {
+ name = "osDiskProfile"
+ caching = "ReadWrite"
+ create_option = "FromImage"
+ vhd_containers = ["${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}"]
+ }
+
+ storage_profile_image_reference {
+ publisher = "Canonical"
+ offer = "UbuntuServer"
+ sku = "16.04-LTS"
+ version = "latest"
+ }
+}
+`, rInt, location)
+}
+func testAccAzureRMVirtualMachineScaleSet_basicPublicIP_update_tags(rInt int, location string ) string {
+ return fmt.Sprintf(`
+resource "azurerm_resource_group" "test" {
+ name = "acctestRG-%[1]d"
+ location = "%[2]s"
+}
+
+resource "azurerm_virtual_network" "test" {
+ name = "acctvn-%[1]d"
+ address_space = ["10.0.0.0/16"]
+ location = "${azurerm_resource_group.test.location}"
+ resource_group_name = "${azurerm_resource_group.test.name}"
+}
+
+resource "azurerm_subnet" "test" {
+ name = "acctsub-%[1]d"
+ resource_group_name = "${azurerm_resource_group.test.name}"
+ virtual_network_name = "${azurerm_virtual_network.test.name}"
+ address_prefix = "10.0.2.0/24"
+}
+
+resource "azurerm_storage_account" "test" {
+ name = "accsa%[1]d"
+ resource_group_name = "${azurerm_resource_group.test.name}"
+ location = "${azurerm_resource_group.test.location}"
+ account_tier = "Standard"
+ account_replication_type = "LRS"
+
+ tags = {
+ environment = "staging"
+ }
+}
+resource "azurerm_storage_container" "test" {
+ name = "vhds"
+ resource_group_name = "${azurerm_resource_group.test.name}"
+ storage_account_name = "${azurerm_storage_account.test.name}"
+ container_access_type = "private"
+}
+
+resource "azurerm_virtual_machine_scale_set" "test" {
+ name = "acctvmss-%[1]d"
+ location = "${azurerm_resource_group.test.location}"
+ resource_group_name = "${azurerm_resource_group.test.name}"
+ upgrade_policy_mode = "Manual"
+
+ tags = {
+ test = "another test"
+ }
+
sku {
name = "Standard_D1_v2"
tier = "Standard"
- capacity = 2
+ capacity = 0
}
os_profile {
With the above test case, I was able to have a constant reproducible case.
Expected Behavior
When we provision a scale-set with public ip and made a simple change such as changing the scale-set tags, we should not alter the network configuration of the scale-set
Actual Behavior
The definition of the scale-set changes and the part for the public ip assignment is removed, leading to new VM's comming online to be missing the public ip assigment.
Steps to Reproduce
create a simple scale-set as (seen with the above tf code), no instance are required.
terraform applyReview the scale-sets' json from Azure example:
az vmss list --subscription=$AZ_SUB -g $AZ_RESOURCE_GROUP |jq '.[].virtualMachineProfile.networkProfile.networkInterfaceConfigurations'
Change a tag value
terraform applyhere you will see that this will show as change only the change of the tag we just did.Review the scale-set json again as was done from step (3)
References
_might_ be related
- #1853
alexsapran
All 6 comments
kwilczynski
on 1 Jul 2019
@alexsapran thanks for opening this issue, I have been able to reproduce the issue on my dev box and am currently investigating the issue.
Initial Provision:
"ipConfigurations": [
{
"name": "TestIPConfiguration",
"properties": {
"publicIPAddressConfiguration": {
"name": "TestPublicIPConfiguration",
"properties": {
"idleTimeoutInMinutes": 4,
"ipTags": [],
"dnsSettings": {
"domainNameLabel": "test-domain-label-1234567890"
}
}
},
"primary": true,
"subnet": {
"id": "/subscriptions/*****/resourceGroups/acctestRG-1234567890/providers/Microsoft.Network/virtualNetworks/acctvn-1234567890/subnets/acctsub-1234567890"
},
"privateIPAddressVersion": "IPv4"
}
}
]
After Tag Update:
"ipConfigurations": [
{
"name": "TestIPConfiguration",
"properties": {
"primary": true,
"subnet": {
"id": "/subscriptions/*****/resourceGroups/acctestRG-1234567890/providers/Microsoft.Network/virtualNetworks/acctvn-1234567890/subnets/acctsub-1234567890"
},
"privateIPAddressVersion": "IPv4"
}
}
]
WodansSon
on 2 Jul 2019
Hi @jeffreyCline and @alexsapran, after reading through the issues description and look at the reproduction case (see above), I believe the following ~might be able to resolve the issue~ might point us in the right direction:
Update: See new comment below and a related Pull Request for a more complete fix.
diff --git i/azurerm/resource_arm_virtual_machine_scale_set.go w/azurerm/resource_arm_virtual_machine_scale_set.go
index 80b07891..1cf29136 100644
--- i/azurerm/resource_arm_virtual_machine_scale_set.go
+++ w/azurerm/resource_arm_virtual_machine_scale_set.go
@@ -1328,12 +1328,11 @@ func flattenAzureRmVirtualMachineScaleSetNetworkProfile(profile *compute.Virtual
if properties.PublicIPAddressConfiguration != nil {
publicIpInfo := properties.PublicIPAddressConfiguration
- publicIpConfigs := make([]map[string]interface{}, 0, 1)
publicIpConfig := make(map[string]interface{})
publicIpConfig["name"] = *publicIpInfo.Name
publicIpConfig["domain_name_label"] = *publicIpInfo.VirtualMachineScaleSetPublicIPAddressConfigurationProperties.DNSSettings
publicIpConfig["idle_timeout"] = *publicIpInfo.VirtualMachineScaleSetPublicIPAddressConfigurationProperties.IdleTimeoutInMinutes
- config["public_ip_address_configuration"] = publicIpConfigs
+ config["public_ip_address_configuration"] = publicIpConfig
}
ipConfigs = append(ipConfigs, config)
The flattenAzureRmVirtualMachineScaleSetNetworkProfile function that is in turn used internally by the resourceArmVirtualMachineScaleSetRead function (which is also set to be the Read function in the resource schema, see: resource_arm_virtual_machine_scale_set.go#L25).
The function resourceArmVirtualMachineScaleSetRead is then in turn used internally by the resourceArmVirtualMachineScaleSetCreateUpdate function which is set to serve as both the Create and Update functions in the resource schema (see: resource_arm_virtual_machine_scale_set.go#L24 and terraform-provider-azurerm/blob/master/azurerm/resource_arm_virtual_machine_scale_set.go#L26).
After looking closely, this might explain why initial creation would work, but an update would cause the public IP properties to be removed. The Create function (as per the resource schema) utilises primarily the following:
Whereas the Update function relies primarily on the following to work correctly:
Looking more closely, we can see that because config variable is defined and declared to hold a map of instance{} types, as per:
Then, simply storing the following:
Would not have caused a compile-time (type checking) issue. But, it would result potentially in an empty values (empty slice, etc.) causing the JSON marshaller to simply skip it, thus the publicIPAddressConfiguration would not be present any more as an attribute in the request, as per the result shown after the tags were updated, see: https://github.com/terraform-providers/terraform-provider-azurerm/issues/3763#issuecomment-507458304
This is my initial observation. I might be wrong as I haven't been able to test this.
What do you think? @jeffreyCline, does it make sense?
kwilczynski
on 2 Jul 2019
Hi @jeffreyCline, I believe we have a fix!
@alexsapran was able to take over in the morning, and took the issue at hand through the finish line.
He looked at my proposed solution and discovered that it wasn't entirely correct, as the API expects the public IP configuration to be a list of key-value pairs.
An amazing team work here!
kwilczynski
on 2 Jul 2019
@kwilczynski That is exactly what I figured out yesterday myself, awesome job guys and thanks for the hard work on this issue!
WodansSon
on 2 Jul 2019
I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.
If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!
![hashibot[bot] picture](https://avatars2.githubusercontent.com/in/8332?v=4&s=40)
hashibot[bot]
on 3 Aug 2019
Related issues
TPPWC
·
55Comments
mjyeaney
·
34Comments
titilambert
·
35Comments
matthid
·
42Comments
tombuildsstuff
·
36Comments
Most helpful comment
@kwilczynski That is exactly what I figured out yesterday myself, awesome job guys and thanks for the hard work on this issue!