Terraform: Existing security groups are not recognized on an aws_instance in TF 0.6.15

[MattiasGees]

Terraform Version

Terraform v0.6.15

Affected Resource(s)

Please list the resources as a list, for example:

• aws_instance

Terraform Configuration Files

-/+ module.vpc.aws_instance.nat
    ami:                                       "ami-ef76e898" => "ami-ef76e898"
    availability_zone:                         "eu-west-1a" => "<computed>"
    ebs_block_device.#:                        "0" => "<computed>"
    ephemeral_block_device.#:                  "0" => "<computed>"
    instance_state:                            "running" => "<computed>"
    instance_type:                             "t2.micro" => "t2.micro"
    key_name:                                  "mattias" => "mattias"
    placement_group:                           "" => "<computed>"
    private_dns:                               "ip-10-0-0-244.eu-west-1.compute.internal" => "<computed>"
    private_ip:                                "10.0.0.244" => "<computed>"
    public_dns:                                "ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.com" => "<computed>"
    public_ip:                                 "xx.xx.xx.xx" => "<computed>"
    root_block_device.#:                       "1" => "1"
    root_block_device.0.delete_on_termination: "false" => "0"
    root_block_device.0.iops:                  "0" => "<computed>"
    root_block_device.0.volume_size:           "8" => "8"
    root_block_device.0.volume_type:           "standard" => "standard"
    security_groups.#:                         "0" => "1" (forces new resource)
    security_groups.200636373:                 "" => "sg-39363a5c" (forces new resource)
    source_dest_check:                         "false" => "0"
    subnet_id:                                 "subnet-3232aa45" => "subnet-3232aa45"
    tenancy:                                   "default" => "<computed>"
    vpc_security_group_ids.#:                  "1" => "<computed>"

Code inside module

# Create NAT security group
resource "aws_security_group" "sg_nat" {
  name = "sg_nat"
  description = "Security group that is needed for the nat servers"
  vpc_id = "${aws_vpc.main.id}"

  # Allow incoming HTTP connections
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow outgoing HTTP connections
  egress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# Create nat server
resource "aws_instance" "nat" {
  ami = "${var.nat_ami}"
  instance_type = "t2.micro"
  subnet_id = "${element(aws_subnet.bastion_subnets.*.id, 0)}"
  key_name  = "${var.key_name}"
  security_groups  = ["${aws_security_group.sg_nat.id}"]
  source_dest_check = "false"

  root_block_device {
    volume_type = "standard"
    volume_size = "8"
    delete_on_termination = "false"
  }
}

Expected Behavior

After updating to the latest terraform version, terraform doesn't recognise the current added security groups anymore. In the state file before I do a plan I see the security group. After doing the plan, the security groups are removed from the state file and terraform want to recreate the instances.

State file before

                "aws_instance.nat": {
                    "type": "aws_instance",
                    "depends_on": [
                        "aws_security_group.sg_nat",
                        "aws_subnet.bastion_subnets"
                    ],
                    "primary": {
                        "id": "i-0a888ba0",
                        "attributes": {
                            "ami": "ami-ef76e898",
                            "availability_zone": "eu-west-1a",
                            "ebs_block_device.#": "0",
                            "ebs_optimized": "false",
                            "ephemeral_block_device.#": "0",
                            "iam_instance_profile": "",
                            "id": "i-0a888ba0",
                            "instance_state": "running",
                            "instance_type": "t2.micro",
                            "key_name": "mattias",
                            "monitoring": "false",
                            "private_dns": "ip-10-0-0-244.eu-west-1.compute.internal",
                            "private_ip": "10.0.0.244",
                            "public_dns": "ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.com",
                            "public_ip": "xx.xx.xx.xx",
                            "root_block_device.#": "1",
                            "root_block_device.0.delete_on_termination": "false",
                            "root_block_device.0.iops": "0",
                            "root_block_device.0.volume_size": "8",
                            "root_block_device.0.volume_type": "standard",
                            "security_groups.#": "1",
                            "security_groups.200636373": "sg-39363a5c",
                            "source_dest_check": "false",
                            "subnet_id": "subnet-3232aa45",
                            "tags.#": "3",
                            "tags.Environment": "staging",
                            "tags.Name": "auth-staging-nat01",
                            "tags.Project": "auth",
                            "tenancy": "default",
                            "vpc_security_group_ids.#": "1",
                            "vpc_security_group_ids.200636373": "sg-39363a5c"
                        },
                        "meta": {
                            "schema_version": "1"
                        }
                    }
                },

State file after plan in 0.6.15:

                "aws_instance.nat": {
                    "type": "aws_instance",
                    "depends_on": [
                        "aws_security_group.sg_nat",
                        "aws_subnet.bastion_subnets"
                    ],
                    "primary": {
                        "id": "i-0a888ba0",
                        "attributes": {
                            "ami": "ami-ef76e898",
                            "availability_zone": "eu-west-1a",
                            "disable_api_termination": "false",
                            "ebs_block_device.#": "0",
                            "ebs_optimized": "false",
                            "ephemeral_block_device.#": "0",
                            "iam_instance_profile": "",
                            "id": "i-0a888ba0",
                            "instance_state": "running",
                            "instance_type": "t2.micro",
                            "key_name": "mattias",
                            "monitoring": "false",
                            "private_dns": "ip-10-0-0-244.eu-west-1.compute.internal",
                            "private_ip": "10.0.0.244",
                            "public_dns": "ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.com",
                            "public_ip": "xx.xx.xx.xx",
                            "root_block_device.#": "1",
                            "root_block_device.0.delete_on_termination": "false",
                            "root_block_device.0.iops": "0",
                            "root_block_device.0.volume_size": "8",
                            "root_block_device.0.volume_type": "standard",
                            "security_groups.#": "0",
                            "source_dest_check": "false",
                            "subnet_id": "subnet-3232aa45",
                            "tags.#": "3",
                            "tags.Environment": "staging",
                            "tags.Name": "auth-staging-nat01",
                            "tags.Project": "auth",
                            "tenancy": "default",
                            "vpc_security_group_ids.#": "1",
                            "vpc_security_group_ids.200636373": "sg-39363a5c"
                        },
                        "meta": {
                            "schema_version": "1"
                        }
                    }
                },

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. On older terraform version create a simple project with an instance and sg
2. Upgrade to new TerraForm version
3. Run terraform plan

[kristjanelias]

Seeing the same behaviour. It is causing real problems...

[choppedpork]

I'm seeing the same. Currently working around by using 0.6.14 and adding the security groups to the tfstate file manually (or reverting changes to the tfstate file made by 0.6.15). Running a diff against the 0.6.14 version, I can see that all by aws_instances now have an additional "disable_api_termination": "false", but also:

-                            "security_groups.#": "1",
-                            "security_groups.2753xxxx19": "sg-87xxxxe2",
+                            "security_groups.#": "0",

[catsby]

Hello all –

I'm not immediately sure of the changes there, but I'll investigate. In the meantime, if you are on a VPC (original post seems to be), can you use the vpc_security_group_ids attribute for your security groups in your instance? security_groups are meant for EC2 Classic users, as EC2 Classic has the limitation that you cannot update security groups without re-creating the instance, where as VPC's do not have this limitation.

Ex:

# Create nat server
resource "aws_instance" "nat" {
  ami = "${var.nat_ami}"
  [...]
  vpc_security_group_ids  = ["${aws_security_group.sg_nat.id}"]
  [...]
}

For some time it was "okay" to use security_groups for VPCs and we would simply log a warning, but perhaps that's changed (this is what I'll be looking into).

• https://www.terraform.io/docs/providers/aws/r/instance.html

Let me know if that helps in the meantime

[choppedpork]

Thanks @catsby. That's something I've already discovered when creating a brand new environment - using vpc_security_group_ids from the start does work however trying to simply change security_groups to vpc_security_group_ids in my config still results in the same change plan as before. vpc_security_group_ids is already populated in my 0.6.14 state which used security_groups. Hope this helps.

[catsby]

Hey @choppedpork does your statefile have an entries for vpc_security_group_ids? Can you try making a backup copy of your statefile, and modifying security_groups to just be "security_groups.#": "0",, does that resolve it?

[choppedpork]

Ack, that was my bad - turns out I was editing the wrong .tf file. Replacing security_groups with vpc_security_group_ids does work in 0.6.15! The resulting plan applies no changes and security_groups.# gets changed to 0 in the state file automatically. Thanks for your help @catsby!

[catsby]

Awesome, I hope @MattiasGees and @kristjanelias report similar success!

[bgutierrez]

I was also bitten by this today, and switching to vpc_security_group_ids fixed it.

[kristjanelias]

@catsby I got is sorted by using vpc_security_groups aswell! Thanks!

[geekifier]

Thanks @catsby, your workaround, or rather, correction on proper usage; prevented my ec2 instance from being destroyed over and over.

[MattiasGees]

@catsby thanks for the explanation, this fixes it.

[catsby]

Hey all, glad to hear the suggestion is working out. I'm going to close this, but for what it's worth, another issue was opened https://github.com/hashicorp/terraform/issues/6416 for the same thing, so I'm going to try to dig into what happened and fix something. If nothing else, re-word/update the docs.

Sorry for the trouble, and thanks!

[notjames]

FYI still an issue in 0.8.8

[scottf]

Was he using the security group name or id, because sg-39363a5c looks like an id. But the doc says use name A list of security group names to associate with

[jfromaniello]

@choppedpork I am currently having this issue with v0.10.7 despite that I'm using vpc_security_group_ids = [ "${aws_security_group.foo.id}" ]

The same terraform works fine in some vpcs/region but it fails in some others. It stores the security group in the tfstate as follows:

"security_groups.#": "1",
"security_groups.2372130387": 

even if I use vpc_security_group_ids and it wants to change that on every plan.

Your comment caught my attention:

I'm not immediately sure of the changes there, but I'll investigate. In the meantime, if you are on a VPC (original post seems to be), can you use the vpc_security_group_ids attribute for your security groups in your instance? security_groups are meant for EC2 Classic users, as EC2 Classic has the limitation that you cannot update security groups without re-creating the instance, where as VPC's do not have this limitation.

The environment where I am having this issue is the only one that has Default VPC = true. Could be related to that? Maybe terraform thinks this is an EC2 Classic security group because is on a default vpc?

Thank you in advance

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.