Terraform: aws_security_group doesn't appear to detect changes

[autodesk-azepeda]

Let's say I define a security group like so:

resource "aws_security_group" "sg-app" {
  name        = "app-sg"
  description = "Application SG"
  vpc_id      = "${var.aws_vpc}"

  ingress {
    from_port       = 22
    to_port         = 22
    protocol        = "tcp"
    security_groups = ["${aws_security_group.sg-admin.id}"]
  }

If I go in and change that ingress rule behind Terraform's back (say change that rule via the console to allow SSH in only from 127.0.0.1/32), then run terraform plan, I'm greeted with:

$ AWS_PROFILE=eu-prod terraform plan
Refreshing Terraform state prior to plan...

module.security_groups.aws_security_group.sg-app: Refreshing state... (ID: sg-NNNNNNNN)

No changes. Infrastructure is up-to-date. This means that Terraform
could not detect any differences between your configuration and
the real physical resources that exist. As a result, Terraform
doesn't need to do anything.

I would expect to see diffs indicating that the rule should be changed. Unsure what the intended behavior is.

[phinze]

Ah yeah at first glance it looks like we only consider protocol, to_port, and from_port so we'll just need to get the other fields added there as well so we can detect this drift.

[autodesk-azepeda]

I'm not entirely sure that's the case. In that function the key is only used locally, and a list of objects with all of the rule attributes is returned.

In support of this, when changing things behind the back of Terraform:

• If I take a rule and change its security group filter no change is detected.
• If I take a rule and change its IP address filter, a change is detected but at least with a starting value of 0.0.0.0/0, the printed diff is wrong.

~ module.security_groups.aws_security_group.sg-dmz
    ingress.2214680975.cidr_blocks.#:     "0" => "1"
    ingress.2214680975.cidr_blocks.0:     "" => "0.0.0.0/0"
    ingress.2214680975.from_port:         "" => "80"
    ingress.2214680975.protocol:          "" => "tcp"
    ingress.2214680975.security_groups.#: "0" => "0"
    ingress.2214680975.self:              "" => "0"
    ingress.2214680975.to_port:           "" => "80"
    ingress.2617001939.cidr_blocks.#:     "1" => "1"
    ingress.2617001939.cidr_blocks.0:     "0.0.0.0/0" => "0.0.0.0/0"
    ingress.2617001939.from_port:         "443" => "443"
    ingress.2617001939.protocol:          "tcp" => "tcp"
    ingress.2617001939.security_groups.#: "0" => "0"
    ingress.2617001939.self:              "0" => "0"
    ingress.2617001939.to_port:           "443" => "443"

In the above case I went into the AWS console and changed the IP filter from anything (0.0.0.0/0) to a specific IP. I would expect (incorrectly?) that the refresh action would change the CIDR blocks in the current state representation to match what's in the rule.

[dwradcliffe]

I've also noticed that refresh doesn't always update the ingress/egress information in the state correctly.

[paulcdejean]

Possibly related to #3341

[catsby]

Hey friends – Sorry for the silence here. I'm reviewing issues that haven't seen any update in a while and found this, and I'm happy to report that this issue has been resolved. I've created security groups and changed their attributes in the console and Terraform is detecting the change and offering to change them back as expected.

If you feel you're still hitting this on the latest Terraform (v0.6.7 at time of writing) please let me know and I can take another look. Thanks!

[romenyrr]

Hi @catsby, I'm running the latest version (v0.6.8 as of this post) and I'm still seeing this bug. The only way it detects drift is if I delete _all_ the rules that were set by TF initially. If I add a rule it doesn't detect it, if I remove 1 of the 2 rules it doesn't detect it.

[catsby]

Folding https://github.com/hashicorp/terraform/issues/2047 into this, e.g. removing the last of any ingress or egress rule will result in an empty plan. Ex. config:

resource "aws_security_group" "buggy" {
  name        = "Buggy"
  description = "Buggy"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags {
    Name = "tf-buggy"
  }
}

Removing a single ingress rules works as expected. Remove both results in no plan

[catsby]

Same as https://github.com/hashicorp/terraform/issues/6029

[bandesz]

This is quite a serious issue and not even an update for half a year?

I'm using Terraform 0.7.7. I removed an egress rule from a security group and terraform plan shows no change. Quite scary that in a security related matter I can't trust Terraform :( Any known workarounds? (e.g. adding a dummy egress rule?)

resource "aws_security_group" "bastion" {
  name = "bastion"
  vpc_id = "${var.vpc_id}"

  ingress {
    from_port = 22
    to_port = 22
    protocol = "tcp"
    cidr_blocks = [
      "1.2.3.4/32",
      "1.2.3.4/32",
      "1.2.3.4/32",
      "1.2.3.4/32"
    ]
  }
}

related block in the tfstate file:

"aws_security_group.bastion": {
    "type": "aws_security_group",
    "depends_on": [],
    "primary": {
        "id": "sg-xxxxxxx",
        "attributes": {
            "description": "Managed by Terraform",
            "egress.#": "2",
            "egress.1615060862.cidr_blocks.#": "0",
            "egress.1615060862.from_port": "8080",
            "egress.1615060862.prefix_list_ids.#": "0",
            "egress.1615060862.protocol": "tcp",
            "egress.1615060862.security_groups.#": "1",
            "egress.1615060862.security_groups.3403577584": "sg-xxxxxx",
            "egress.1615060862.self": "false",
            "egress.1615060862.to_port": "8080",
            "egress.2990234175.cidr_blocks.#": "1",
            "egress.2990234175.cidr_blocks.0": "172.18.0.0/16",
            "egress.2990234175.from_port": "22",
            "egress.2990234175.prefix_list_ids.#": "0",
            "egress.2990234175.protocol": "tcp",
            "egress.2990234175.security_groups.#": "0",
            "egress.2990234175.self": "false",
            "egress.2990234175.to_port": "22",
            "id": "sg-xxxxxx",
            "ingress.#": "1",
            "ingress.3415520183.cidr_blocks.#": "4",
            "ingress.3415520183.cidr_blocks.0": "1.2.3.4/32",
            "ingress.3415520183.cidr_blocks.1": "1.2.3.4/32",
            "ingress.3415520183.cidr_blocks.2": "1.2.3.4/32",
            "ingress.3415520183.cidr_blocks.3": "1.2.3.4/32",
            "ingress.3415520183.from_port": "22",
            "ingress.3415520183.protocol": "tcp",
            "ingress.3415520183.security_groups.#": "0",
            "ingress.3415520183.self": "false",
            "ingress.3415520183.to_port": "22",
            "name": "bastion",
            "owner_id": "xxxxxxx",
            "tags.%": "0",
            "vpc_id": "vpc-xxxxxxx"
        },
        "meta": {},
        "tainted": false
    },
    "deposed": [],
    "provider": ""
}

[apparentlymart]

This has the same root cause as #5290. Although that one is newer, it contains some details on the problem and the likely fix so I'm going to close this one to consolidate discussion over there.

(Sorry for the noise for anyone following this old issue; we're currently trying to prune the issue queue so we can better keep track of things.)

[mikemac42]

This issue concerns aws_security_group ingress and egress rule changes not taking effect. This seems different from #5290 since that is lists of security groups in other resources not taking effect. What makes you think these have the same root cause?

[mohneeshdamade]

So after a year , is this issue still open ?
I am stuck because i have to remove an egress rule via terraform.
Please provide any workaround , ofcourse not by console because we dont have prod environment access directly.

[2rs2ts]

It might be a good idea to re-file this issue in the provider repo now that providers are plugins and the code that controls them exists in other repos.

[igozali]

Just wanted to follow up on this issue. I tried searching the terraform-aws-provider repo for issues related to "security group", and found this one, but doesn't have a lot of discussion and hasn't been linked to this one yet.

Does anyone know the current state of this issue? As someone mentioned earlier, this issue seems quite important since it involves AWS security groups, and having undetected configuration drift on that area could open up pretty bad security issues...

[igozali]

So I did some tests based on the summary of the issue. Say I have the following main.tf file (with the relevant versions belows), and apply the changes.

provider "aws" {
  version = "~> 2.19"
  region  = "us-west-1"
}

terraform {
  required_version = "~> 0.12.0"
}

resource aws_security_group "test" {
  name   = "please_delete_me"
  vpc_id = "vpc-xxxxxxxx"

  egress {
    from_port   = 2222
    to_port     = 2222
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Then, in the AWS console, manually edit that egress rule (e.g. change the destination to "My IP", or change the protocol from "TCP" to "UDP"), Terraform seems to be able to detect the changes.

However, if I now add an ingress rule manually via AWS console, but still use the same main.tf file above, Terraform will not detect any changes on the ingress rules, despite the resource not specifying any ingress rule. If I understand correctly, looks like this has something to do with this comment and this issue.

One reliable workaround (or maybe this is the proper way to do it?) is to set ingress = [] so that Terraform knows that the ingress rule list should be empty, like so:

provider "aws" {
  version = "~> 2.19"
  region  = "us-west-1"
}

terraform {
  required_version = "~> 0.12.0"
}

resource aws_security_group "test" {
  name   = "please_delete_me"
  vpc_id = "vpc-xxxxxxxx"

  ingress = []

  egress {
    from_port   = 2222
    to_port     = 2222
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

[teamterraform]

Hi all,

This is a very old issue that relates to code that is no longer maintained in this codebase. The AWS provider now has its own repository and we'd encourage you to open an issue there if you are having problems with the aws_security_group resource type other than what #5290 is representing.

The other issue #5290 is representing the fact that there is currently no straightforward way for a provider to represent the difference in user intent between "There should be no objects of this type" and "I wish to ignore all existing objects of this type", and so currently in some cases providers have elected to prefer the second interpretation. That's particularly true for security group rules where otherwise having no blocks would conflict with separate aws_security_group_rule resource blocks.

You can read more about this special case in the documentation page Attributes as Blocks, which is the special parsing mode that applies only to these existing ambiguous cases, including the ingress and egress blocks in aws_security_group.

As noted on that page, using syntax like ingress = [] is the documented way to explicitly say "zero of these" for the arguments that this special behavior applies to; the arguments that use this are documented as such on their own documentation pages, including ingress and egress.