Terraform-provider-aws: Terraform EC2 security group ingress rule flapping

[hashibot]

_This issue was originally opened by @anosulchik as hashicorp/terraform#6135. It was migrated here as part of the provider split. The original body of the issue is below._

---

Hi,

When I create security group ingress rule that opens access from SG in another aws account, terraform apply always recreates this rule even if no changes in terraform config files were made:

-/+ aws_security_group_rule.apigw_access
    from_port:                "443" => "443"
    protocol:                 "tcp" => "tcp"
    security_group_id:        "sg-ba999ac2" => "sg-ba999ac2"
    self:                     "false" => "0"
    source_security_group_id: "sg-8c777ceb" => "373682980000/sg-8c777ceb" (forces new resource)
    to_port:                  "443" => "443"
    type:                     "ingress" => "ingress"

SG ingress rule config:

resource "aws_security_group_rule" "apigw_access" {
  type = "ingress"
  from_port = 443
  to_port = 443
  protocol = "tcp"
  security_group_id = "${aws_security_group.ecs_security_group.id}"
  source_security_group_id = "373682980000/sg-8c777ceb"
}

Please advise. Thank you.

P.S. terraform 0.6.14

[dblooman]

I believe this is due to the fact the state file only looks at id, which would not include account ID. The EC2 API seems a little loose here, accepting a single account ID and SG ID as one string with a forward slash between them. It then splits into GroupId and UserId, so the plan will always show a difference.

I think we can add an optional input of aws_account_id and check for its existence, then handle the verification of the correct UserId from the API.

resource "aws_security_group_rule" "apigw_access" {
  type = "ingress"
  from_port = 443
  to_port = 443
  protocol = "tcp"
  security_group_id = "${aws_security_group.ecs_security_group.id}"
  source_security_group_id = "sg-8c777ceb"
  aws_account_id = "373682980000"
}

[scott2449]

This still exists as of 0.9.x. It is concerning in production environments because it is removing and reapplying the rule which could result is a short loss of connectivity.

[philipwigg]

Might help someone - I have noticed I don't seem to have this problem if I use separate aws_security_group_rule resources instead of inline ingress blocks.

[cwiggs]

@philipwigg I seem to still have this issue when using a separate aws_security_group_rule and I put the account number before the security group, if i leave the account number off the issue goes away.

using terraform 0.9.6

[martinhoefling]

We have the same issues as @cwiggs pointed out (0.10.8). Is the suggestion from @DaveBlooman acceptable? If so, I can also go ahead on implementing this if nobody else prefers to.

[tatellos]

I am also experiencing this problem in terraform 0.11. It would be great if this could be fixed, as my setup is using security groups heavily and it is necessitating a lot of manual fine tuning now.

[cwiggs]

I was looking into this issue again today and noticed that if i leave the account # out of source_security_group_id and just include the security group id the rules gets created and shows up with / in the AWS console. It grabs the correct account_id even when the SG is in another account.

This same thing happens when I manually create the same rule via the AWS console. I'm not sure if AWS changed something, but this issue seems to have gone away, at least for now.

Using terraform 0.9.6

[arturas-d]

Same issues when applying cross account security group

[raam86]

@radeksimko any pointers on picking this issue up?

[tomdev]

Curious to know when this will be fixed or if there are any blockers on the attached pull request? I'd be happy to perform testing in case that's needed.

Same issues when applying cross account security group

Indeed, this is the issue I'm facing here as well.

[anosulchik]

I can confirm that it happens on recent versions of aws provider. Since we're in VPC, as a workaround we switched to CIDR based rules instead of cross account SG rules.

[thatarchguy]

This is still an issue - Terraform v0.11.11

[maartenschalekamp]

This is still an issue - Terraform v0.11.11

Yes it is.

[rquadling]

Something that I found. If I do NOT put in the external account number, both in the AWS Console, and in my tf file, it works correctly. I am using the longer, 17 character identifiers. Maybe that's a key difference here for those using the older 8 character identifiers.

If I enter just the sg-12345678901234567 code in the AWS Console, the result will include the account number.

[asahu569]

@rquadling Your way works for me too!

[thatarchguy]

The workaround @rquadling proposed works fine for me.

[mbevc1]

This workaround doesn't seems to work here, AWS console is automatically adding account # .

[thatarchguy]

This workaround doesn't seems to work here, AWS console is automatically adding account # .

That is the expected outcome described in the workaround. You use the full 17 char identifier and AWS will add the account ID in the console. However, it will stop the rule from trying to update when re-applying.

[mbevc1]

Hm, not sure it solves it here. After using long SG ID I'm seeing TF trying to re-apply it on each run :thinking:

[jonathanhle]

Just specifying the SG_ID and not the ACCOUNT_ID/SG_ID worked for me. Checking TF state, it has the ACCOUNT_ID/SG_ID in there. Using TF 11.14. Thanks @rquadling !!!

[bkedryna]

The important thing is the provider version. We had an old provider defined in the code so it didn't work even with a new terraform version - but when we upgraded the provider to 2.x, just using the SG ID did it.

[ghost]

It is still an issue in the latest version of terraform (0.12.24) and aws provider (2.56.0):

TF apply:
module.app_security_group_rule_https.aws_security_group_rule.security_group_rule_sg will be created

• resource "aws_security_group_rule" "security_group_rule_sg" {
  
  
  
  • description = "Allow SSH from Fundaments Jump Server"
  
  
  • from_port = 22
  
  
  • id = (known after apply)
  
  
  • protocol = "tcp"
  
  
  • security_group_id = "sg-08718b9f4af597811"
  
  
  • self = false
  
  
  • source_security_group_id = "/sg-0554962333b483ed4"
  
  
  • to_port = 22
  
  
  • type = "ingress"
    
    }
  
  

TF output:
Error: Error authorizing security group rule type ingress: InvalidGroup.NotFound: The security group 'sg-0554962333b483ed4' does not exist
status code: 400, request id: 6d4efd7c-a4d6-4b8e-b4b7-627f291ef25a

on ../../../../../modules/services/securitygroup-rule-sg/main.tf line 1, in resource "aws_security_group_rule" "security_group_rule_sg":
1: resource "aws_security_group_rule" "security_group_rule_sg" {