Terraform: 0.6.16 - azurerm - unable to obtain public IP

[ghost]

Terraform Version

0.6.16

Affected Resource(s)

azurerm_public_ip

Terraform Configuration Files

resource "azurerm_public_ip" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0-public-ip"
    location = "${var.location}"
    resource_group_name = "${var.resource_group_name}"
    public_ip_address_allocation = "dynamic"
}

resource "azurerm_network_interface" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0-nic"
    location = "${var.location}"
    resource_group_name = "${var.resource_group_name}"
    network_security_group_id = "${azurerm_network_security_group.service_discovery_server.id}"

    ip_configuration {
        name = "${var.node_type}-0-ip-config"
        subnet_id = "${element(split(",", var.subnet_id), 0)}"
        private_ip_address_allocation = "dynamic"
        public_ip_address_id = "${azurerm_public_ip.ap-service-discovery-server-0.id}"
    }
}

resource "azurerm_virtual_machine" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0"
    resource_group_name = "${var.resource_group_name}"
    location = "${var.location}"
    vm_size = "${var.instance_type}"
    network_interface_ids = ["${azurerm_network_interface.ap-service-discovery-server-0.id}"]
    availability_set_id = "${azurerm_availability_set.ap-service-discovery-availability-set.id}"


    storage_os_disk {
        name = "${var.node_type}-0-osdisk"
        #Source VHD as reference
        image_uri = "${var.source_vhd_path}"
        #Destination vhd to create
        vhd_uri = "${var.vhd_storage_base_uri}${var.node_type}-0.vhd"
        create_option = "fromImage"
        os_type = "linux"
    }

    os_profile {
        computer_name = "${var.node_type}-0"
        admin_username = "${var.ssh_username}"
        admin_password = "${var.ssh_password}"
        # Custom data must be base64 and lt 87380 chars
        custom_data = "${base64encode(template_file.user_data_0.rendered)}"
    }

    os_profile_linux_config {
       disable_password_authentication = true
        ssh_keys {
          path = "/home/${var.ssh_username}/.ssh/authorized_keys"
          key_data = "${file("${var.ssh_key_path}")}"
        }
    }
}

output "public_ips" { value = "${azurerm_public_ip.ap-service-discovery-server-0.ip_address}" }

Expected Behavior

The public ip should be retrievable as per the docs.

Actual Behavior

• Resource 'azurerm_public_ip.ap-service-discovery-server-0' does not have attribute 'ip_address' for variable 'azurerm_public_ip.ap-service-discovery-server-0.ip_address'
  

  Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. create a vm, with a public IP, then try and access that public ip from another resource or for module output.

[stack72]

Hi @tasquith

Am looking at this right now - can you post your full config (minus secrets) - I only see partial config above as it's missing RGs and Availability Sets etc

P.

[ghost]

Thanks @stack72,

Our approach makes heavy use of modules and tfvar files - so posting the full config is a bit of a task!

I'll get you a streamlined config that you should just be able to apply.

Cheers,

Tom

[ghost]

I've written a noddy example where the issue still occurs. I couldn't get it as an output, so I've just referenced the field in a tag for another RG.

resource "azurerm_resource_group" "test" {
  name     = "acctestrg"
  location = "North Europe"
}

resource "azurerm_virtual_network" "test" {
  name                = "acctvn"
  address_space       = ["10.0.0.0/16"]
  location            = "North Europe"
  resource_group_name = "${azurerm_resource_group.test.name}"
}

resource "azurerm_subnet" "test" {
  name                 = "acctsub"
  resource_group_name  = "${azurerm_resource_group.test.name}"
  virtual_network_name = "${azurerm_virtual_network.test.name}"
  address_prefix       = "10.0.2.0/24"
}

resource "azurerm_public_ip" "test" {
    name = "public-ip"
    location = "North Europe"
    resource_group_name = "${azurerm_resource_group.test.name}"
    public_ip_address_allocation = "dynamic"
}

resource "azurerm_network_interface" "test" {
  name                = "acctni"
  location            = "North Europe"
  resource_group_name = "${azurerm_resource_group.test.name}"

  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "dynamic"
  }
}

resource "azurerm_storage_account" "test" {
  name                = "hashicorpaccsa"
  resource_group_name = "${azurerm_resource_group.test.name}"
  location            = "northeurope"
  account_type        = "Standard_LRS"

  tags {
    environment = "staging"
  }
}

resource "azurerm_storage_container" "test" {
  name                  = "vhds"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  storage_account_name  = "${azurerm_storage_account.test.name}"
  container_access_type = "private"
}

resource "azurerm_virtual_machine" "test" {
  name                  = "acctvm"
  location              = "North Europe"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  network_interface_ids = ["${azurerm_network_interface.test.id}"]
  vm_size               = "Standard_A0"

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "14.04.2-LTS"
    version   = "latest"
  }

  storage_os_disk {
    name          = "myosdisk1"
    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/myosdisk1.vhd"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }

  os_profile {
    computer_name  = "hostname"
    admin_username = "testadmin"
    admin_password = "Password1234!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }
}


resource "azurerm_resource_group" "somewheretoputtheip" {
  name     = "testResourceGroup1"
  location = "North Europe"

  tags {
    environment = "${azurerm_public_ip.test.ip_address}"
  }
}

[stack72]

Hi @tasquith

So I have been able to run through this and reproduce as you see. The issue, that i can see seems to be the response from the azure api:

{
    "name": "public-ip",
    "id": "/subscriptions/34ca515c-4629-458e-bf7c-738d77e0d0ea/resourceGroups/acctestrg/providers/Microsoft.Network/publicIPAddresses/public-ip",
    "etag": "W/\"e4661937-a339-4590-9c05-740ed985c4ee\"",
    "type": "Microsoft.Network/publicIPAddresses",
    "location": "northeurope",
    "tags": {},
    "properties": {
        "provisioningState": "Succeeded",
        "resourceGuid": "e21ada16-28c8-4113-b648-d4ab99f37b13",
        "publicIPAllocationMethod": "Dynamic",
        "idleTimeoutInMinutes": 4
    }
}

As you can see, the API doesn't return an IP for a Dynamic allocation.

https://azure.microsoft.com/en-gb/documentation/articles/virtual-network-ip-addresses-overview-arm/

This article says as following:

Allocation method

There are two methods in which an IP address is allocated to a public IP resource - dynamic or static. The default allocation method is dynamic, where an IP address is not allocated at the time of its creation. Instead, the public IP address is allocated when you start (or create) the associated resource (like a VM or load balancer). The IP address is released when you stop (or delete) the resource. This causes the IP address to change when you stop and start a resource.

Thoughts?

P.

[ghost]

Thanks @stack72,

That is a pain, but its understandable! I'm trying to use that IP to then create a DNS entry with DNSimple, so thats a blocker. I'll try with assigning them statically and see how I get on!

Cheers,

Tom

[stack72]

sorry @tasquith

I am trying to work out where the public ip is then actually assigned but I cannot find it in the API calls - I have the full trace of the entire tree from charles proxy to try and see what the requests are doing

P.

[ghost]

No worries! Using statically assigned IPs is a workaround, so I can still carry on :)

Thanks for the help!

Tom

[stack72]

@tasquith

I am going to change this from a bug to an enhancement - this is not a bug in the provider as it works as expected. I will have to try and raise this with MSFT

Paul

[ghost]

@stack72

No worries, Cheers Paul

[igable]

@tasquith, @stack72 , There is another reasonable workaround that you can use for this problem, especially when needing more IPs then the max allowed static IPs:

resource "azurerm_public_ip" "client" {
    count = 2
    name = "client${count.index}"
    location = "West US"
    resource_group_name = "${azurerm_resource_group.somegroup.name}"
    # below results in a dns name that looks like:
    #client01-48e35078
    domain_name_label = "${format("client%02d-%.8s",count.index,  uuid())}"
    public_ip_address_allocation = "dynamic"

    lifecycle {
        # ignore because we are generating with uuid()
        ignore_changes = ["domain_name_label"]
    }
}

Then you can access the FQDN which microsoft does return. You can then use that FQDN in a provisioner connection block like this.:

    connection {
        user = "ubuntu"
        host = "${element(azurerm_public_ip.client.*.fqdn, count.index)}"
        timeout ="15m"
    }

[stack72]

@tasquith just a FYI, the new version of the SDK (https://github.com/hashicorp/terraform/pull/7379) will allow this to be fixed

Just waiting on that to be merged and then will add the public IP part

[ghost]

@stack72

Cheers Paul,

Will this be part of 0.7?

[stack72]

yes :)

[ghost]

@stack72 - Did the new version of the SDK fix this?

[jgoldschrafe]

I can confirm this issue still exists on v0.7.0.

[xynova]

I do get an ip address back but it is not the same one displays in the portal and cannot be used to access the vm either :/ (v0.7.4)

[pmcatominey]

I wonder if a data source would be a solution for this, it would be able to read the IP after the assigned resource (VM etc.) has started.

[moritzheiber]

Any progress on this? I guess the workaround is to assign a static public IP, yet that'll put you in a corner pretty soon :wink:

[nickithewatt]

Still an issue in 0.8.4 :(

[sozercan]

still an issue in 0.8.8

[githubvick]

Not fixed in v0.9.3 either. This should be really useful if fixed.

I mean is there no way in which we can get the (Dynamic) public IP address of a newly created VM in Azure?

Update:

One workaround is by running a separate provisioner and getting the IP Address

resource "null_resource" "ansible-provision" {


  provisioner "local-exec" {
    command =  "echo \"[masters]\n ansible_ssh_host=${azurerm_public_ip.helloterraformips.ip_address} \" >> /home/osboxes/ansible-kube/ansible/inventory/testinv"
  }

[githubvick]

The above worked only once, I dont know how. The workaround I'm proceeding with is running the provisioner separately after the VM is created as it is able to detect on the second run.

[longmuir]

I'm still seeing this in 0.9.5 as well.

Public IP addresses aren't returned as outputs (but are if you run 'terraform apply' a second time, which seems to be a current work around for outputs). It also causes problems with a file provisioner, though I'm using the workaround @igable described above.

[rcarun]

Hi Longmuir, Just wanted to clarify that this is not a blocker for you? How painful is the workaround in your mind? We were considering making IP a Data Source and dynamically returning the address as an output. Will this solve your problem? Please let us know.

[abhijeetgaiha]

A dynamic public IP is allocated by Azure once the VM that is assigned the public IP has been created and has booted. You can now use the public IP data source to get the IP address for dynamic IPs. To ensure that the data source is processed by terraform after the VM has been created, use the resource_group_name property of the VM as an input to the public IP data source:

data "azurerm_public_ip" "datasourceip" { name = "myPublicIP" resource_group_name = "${azurerm_virtual_machine.myVirtualMachine.resource_group_name}" }

The IP address can then be accessed using the ip_address property:
${data.azurerm_public_ip.datasourceip.ip_address}

[aleksap]

Was anyone able to get this work?

I still have issues and can't get provisioner to connect to the VM, neither via ip or fqdn.. can't get correct output with any of those two

[abhijeetgaiha]

@aleksap please see my last comment. If you have any further queries, I'd need happy to look into it.

[aleksap]

@abhijeetgaiha

Thanks for the reply.

I have tried your solution but I had some issues.
I am using modules so I have add your data resource in my network module.
Then I have created output:

output "datasourceip" {
 value = "${data.azurerm_public_ip.datasourceip.ip_address}"
  }

I would call that output from my template via

public_ip = "${module.network.datasourceip}

But it fails..
Am I doing this right or am I missing something?

Thanks,
Alex

[abhijeetgaiha]

@aleksap By fails, do you mean you get an error message or an empty value for the IP address?
Also, please keep in mind that the dynamic IP for a VM isn't allocated until the VM is booted. To ensure that the VM is booted, I suggest using the resource group name property of the VM when configuring the IP data source.

[aleksap]

@abhijeetgaiha ,

Error that I am getting is:

• output 'datasource': unknown resource 'data.azurerm_public_ip.datasourceip' referenced in variable data.azurerm_public_ip.datasourceip.ip_address

As I have mentioned above, I have resource data in network module, I have correctly configured output.. I don't see a reason why it would throw me this error.

Thanks,
Alex

[abhijeetgaiha]

@aleksap I'm looking into this. It would help if you could provide your config files.

[aleksap]

@abhijeetgaiha ,

Here is my network module:

# create virtual network
resource "azurerm_virtual_network" "network" {
  name                = "${format("%s-vnet", var.name)}"
  address_space       = ["${var.address_space}"]
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"
}

# create subnets
resource "azurerm_subnet" "subnets" {
  count                = "${length(var.subnet_cidrs)}"
  name                 = "${format("%s-subnet-%02d", var.name, count.index+1)}"
  resource_group_name  = "${var.resource_group_name}"
  virtual_network_name = "${azurerm_virtual_network.network.name}"
  address_prefix       = "${var.subnet_cidrs[count.index]}"
}

#create public ip (ip_type: static or dynamic)
resource "azurerm_public_ip" "public_ip" {
  name                         = "${format("%s-public_ip-%02d", var.name, count.index+1)}"
  location                     = "${var.location}"
  resource_group_name          = "${var.resource_group_name}"
  #count                        = "${var.vm_count}"
  domain_name_label            = "${var.domain_name}"
  public_ip_address_allocation = "${var.public_ip_type}"
}


# create nic
resource "azurerm_network_interface" "network" {
  #count               = "${length(var.subnet_cidrs)}"
  name                = "${format("%s-network-interface%02d", var.name, count.index+1)}"
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"

  ip_configuration {
    name                          = "${format("%s-public_ip", var.name)}"
    subnet_id                     = "${element(azurerm_subnet.subnets.*.id, count.index)}"
    private_ip_address_allocation = "${var.private_ip_type}"
    public_ip_address_id          = "${azurerm_public_ip.public_ip.id}"
  }

data "azurerm_public_ip" "datasourceip" {
  name                          = "testing"
  resource_group_name           = "${var.resource_group_name}"
}
 }

Here are all the outputs that I have tried:

output "id" {
value = ["${azurerm_network_interface.network.*.id}"]
}

output "pip" {
value = "${element(azurerm_public_ip.public_ip.fqdn)}"
 }

 output "pubip" {
 value = "${element(azurerm_public_ip.public_ip.ip_address)}"
  }

 output "datasource" {
 value = "${data.azurerm_public_ip.datasourceip.ip_address}"
  }

Output "id" works fine when it's called from template file.
Output pip or pubip don't give me either ip_address or fqdn.. they give me random value, for ex:

module.vm.azurerm_virtual_machine.vm (chef): Connecting to remote host via SSH... module.vm.azurerm_virtual_machine.vm (chef): Host: 74D93920-ED26-11E3-AC10-0800200C9A66 module.vm.azurerm_virtual_machine.vm (chef): User: cq module.vm.azurerm_virtual_machine.vm (chef): Password: true module.vm.azurerm_virtual_machine.vm (chef): Private key: false module.vm.azurerm_virtual_machine.vm (chef): SSH Agent: false

When output "datasource" is called I get error:
output 'datasource': unknown resource 'data.azurerm_public_ip.datasourceip' referenced in variable data.azurerm_public_ip.datasourceip.ip_address

[jeromeza]

I'm also seeing this issue in Terraform v0.10.0.

Same issue:

Error applying plan:

1 error(s) occurred:

• azurerm_virtual_machine.prod_vm1: 1 error(s) occurred:

• Resource 'azurerm_public_ip.prod_pubip' does not have attribute 'ip_address' for variable 'azurerm_public_ip.prod_pubip.ip_address'

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

[mikenorgate]

I have managed to work around this issue using the approach from @abhijeetgaiha with one extra addition

Make sure that where you use the data source has a dependency on the virtual machine

Here is my working config:

resource "azurerm_public_ip" "test_host_public_ip" {
    name                              = "test-ip"
    public_ip_address_allocation      = "dynamic"
    ...
}

resource "azurerm_virtual_machine" "test_host" {
    name                              = "test-vm"
    ...
}

data "azurerm_public_ip" "datasourceip" { 
    name = "${azurerm_public_ip.test_host_public_ip.name}" 
    resource_group_name = "${azurerm_virtual_machine.test_host.resource_group_name}" 
}

output "test_ip_addresses" {
    value = "${data.azurerm_public_ip.datasourceip.ip_address}"

    depends_on = ["azurerm_virtual_machine.test_host"]
}

[m4h3]

@mikenorgate I've tried your solution and it doesn't work for me.
After a terraform refresh i'm getting the public ip, but the dependency on the vm isn't strong enough apparently for the apply part.
I'm not sure how depend works, does it wait for resource to be fully created ? Does azure provide a dynamic IP address immediately after the VM creation? It seems to race from my point of view.

Deploying on West-Europe.

[rahulkp220]

@mikenorgate depends_on = ["azurerm_virtual_machine.test_host"] this value should be inside data , not output.

[timmyreilly]

Here's a snippet of how I was able to get it to work:

main.tf:

...
resource "azurerm_public_ip" "public_ip" {
  name              = "${var.machine_name}-public-ip"
  resource_group_name = var.resource_group_name 
  location = data.azurerm_resource_group.resource_group.location
  allocation_method = "Dynamic" 
}

resource "azurerm_network_interface" "main" {
  name                = "${var.machine_name}-core-nic"
  location            =  data.azurerm_resource_group.resource_group.location
  resource_group_name =  var.resource_group_name

  ip_configuration {
    name                          = "${var.machine_name}-private-ip"
    subnet_id                     = data.azurerm_subnet.target_subnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.public_ip.id
  }

}

resource "azurerm_virtual_machine" "main" {
  name                  = var.machine_name
  location              = data.azurerm_resource_group.resource_group.location
  resource_group_name   = var.resource_group_name
  network_interface_ids = [azurerm_network_interface.main.id]
  vm_size               = var.machine_size
...
}

data "azurerm_public_ip" "waited_public_ip" { 
  name = azurerm_public_ip.public_ip.name
  resource_group_name = var.resource_group_name
  depends_on = ["azurerm_virtual_machine.main"]
}

outputs.tf

...
output "network_interface_public_ip" {
  description = "public ip address of the vm nic"
  value         = data.azurerm_public_ip.waited_public_ip.ip_address
}