Terraform: Elastic IP assignment

[vacquah]

I noticed that eips are not assigned when I add the "remote-exec" provisioner to my instance code. when I comment out the provisioner portion of the code, the eip is assgined correctly. Have been able to reproduce this several times. Here is the code:

https://gist.github.com/e481ea8b37d478296396

[CpuID]

Yep this seems to cause some issues. I think it's a resource ordering issue.

Pretty sure Terraform is trying to complete the provisioners within the aws_instance resources before moving on to add the aws_eip resources. This means that instances are stuck with a private only IP during the provisioning phase. If your provisioner does anything that requires external network access it will hang during. This then causes Terraform to never get to the aws_eip resources.

Ideally what needs to be done, is the aws_eip resources should be executed right after the instances come online but before the aws_instance provisioners commence.

[brycekahle]

I know it is backwards, but have you tried putting the provisioner on the EIP resource?

[mdulghier]

Putting the remote-exec provisioner on the aws_eip resource actually works. You just have to define the connection, e.g.

resource "aws_eip" "example-ip" {
    instance = "${aws_instance.example.id}"
    vpc = true
    connection {
        host = "${aws_eip.example-ip.public_ip}"
        user = "ubuntu"
        key_file = "${var.ssh_key}"
    }
    provisioner "remote-exec" {
        script = "setup.sh"
    }
}

[duggan]

When updating security groups (or other instance-destructive action), this means any provisioning steps will have to be run manually, as they are not invoked under the instance resource as they otherwise would be.

[mitchellh]

I agree with @brycekahle that it sounds backwards but it actually sounds the best for me that in this case you should run the provisioner on the EIP and setup the connection to talk to the EIP. It isn't ideal but this would be a pretty hard problem to fix at the moment.

[duggan]

There's also a side effect here that the public DNS of the instance is incorrect for provisioning, meaning one can't make use of the (very convenient) split horizon DNS resolution of AWS (where externally it resolves to the elastic IP, but resolves to the private interface within the network).

Edit:

For what it's worth, a workaround for this is to query the EC2 metadata API on the provisioner, like:

provisioner "remote-exec" {
                inline = [
                        "sudo sed -i s/ADVERTISED_HOST/$(curl http://169.254.169.254/latest/meta-data/public-hostname)/ /etc/foo.conf",
                ]
        }

[phinze]

Checking in on some old issues here. The fact that provisioners for a given resource before its dependencies are processed is pretty inherent to the way the dependency graph works. Today we'd recommend the same solution of dropping the provisioners later in the graph, either by adding them to the aws_eip resource directly or via a null_resource placed afterwards.

These options can be considered first-class strategies for accomplishing this, and I can't picture any way for us to implicitly make the assignment occur before the instance's provisioners run, so I'm going to close this issue. Feel free to follow up if you think this should still be open. :+1:

[rcoh]

FYI: if you key the null_resource off of the instance.id you run into the same issue :-/
You need to set the connection to the aws_eip.xxx.public_ip_address

[hfuops]

@rcoh , unfortunately I ran into the issue as you mentioned :(, and a morning of time was lost due to this.
Here is the final sample I've tested successfully. FYI, hope this could ease the pain of someone who cloud meet the same problem in future.

resource "aws_eip" "demo" {
  instance = "${aws_instance.TS-demo.id}"
  depends_on = ["aws_instance.TS-demo"]
  vpc      = true
}

resource "aws_instance" "TS-demo" {
  ami                    = "${lookup(var.aws_opsman_ami,var.region)}"
  instance_type          = "m3.medium"
  key_name               = "${var.key_name}"
  subnet_id              = "${aws_subnet.public.id}"
  vpc_security_group_ids = ["${aws_security_group.all_pass.id}"]

  tags {
    Name = "demo"
  }

  provisioner "local-exec" {
    command = "echo ${aws_instance.TS-demo.private_ip} > private_ips"
  }

}
resource "null_resource" "preparation" {
    triggers {
        instance = "${aws_instance.TS-demo.id}"
    }
  connection {
    host        ="${aws_eip.demo.public_ip}"   # don't forget  this option.
    user        = "ubuntu"
    timeout     = "30s"
    private_key = "${file("./skyfree.pem")}"
    agent = false
  }

  provisioner "file" {
    source      = "./tfvars"
    destination = "/tmp/"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get -y update",
    ]
  }

}

[afeld]

/cc #2585

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.