Terraform: Potential "Forces new resource" issue(s) when using `terraform import` command with aws_security_group_rule?

[PaulCapestany]

Terraform Version

Terraform v0.7.0-dev (8c3ff933951a5ab0111c0ef6b3553e42e4ce14af)

Affected Resource(s)

• aws_security_group_rule
  

  Expected Behavior

1. If I create aws_security_group_rules in a Terraform config and then do terraform plan, I should see "No changes. Infrastructure is up-to-date."

2. Then, if I delete the _.tfstate_ and _.tfstatebackup_ files, and terraform import all the same resources, and then do terraform plan, I should see "No changes. Infrastructure is up-to-date."

   Actual Behavior

3. If I create aws_security_group_rules in a Terraform config and then do terraform plan, I see "No changes. Infrastructure is up-to-date."

4. _If I delete the_ _.tfstate_ _and_ _.tfstatebackup_ _files, and terraform import all the same resources, and then do terraform plan, I see the following:_

Plan: X to add, 0 to change, X to destroy.

Steps to Reproduce

1. See above ↑
   

   Important Factoids

It appears that there are two issues here:

1. the terraform import command for _aws_security_group_rule_ seems to unnecessarily force a new security group rule resource due to self: "" => "false" (forces new resource)
2. the order of IPs in _cidr_blocks_ (when >10) looks like it may not be properly parsed when brought in via terraform import, also forcing a new resource

References

Perhaps relates to https://github.com/hashicorp/terraform/issues/6914?

[PaulCapestany]

To be more specific, what worries me about this potential issue is that when a new aws_security_group_rule resource is forced, it looks like RevokeSecurityGroup* is called (based on what I see with TF_LOG=DEBUG set).

Amazon states that "Rule changes are propagated to instances within the security group as quickly as possible. However, _a small delay might occur_.", which when terraform import-ing lots of complex pre-existing infrastructure it seems like the kind of thing one might want to avoid...

[PaulCapestany]

If anyone else has the same problem, a quick/hacky workaround is to manually edit the _.tfstate_ file to reflect the format that it _should_ have in order to avoid all the (forces new resource) issues.

[dlsniper]

@catsby I too have the same problem as @PaulCapestany with 0.7-RC1, appears to be related to the security groups.

security_groups.#:           "0" => "1" (forces new resource)
security_groups.1270133655:  "" => "sg-b0e662d7" (forces new resource)

The problem is that I haven't changed anything these resources / EC2 and for example if I apply what Terraform wants I would bring down the bastion host, nat instance and basically every other instance which is statically bound by security groups (not in an auto-scaling group).

While I'm not sure I can share the config I would definitely be happy to test out any patch that could solve this.

I think this is important enough not to launch 0.7 without it being fixed.

Thank you.

[dlsniper]

I've looked more into this, it seems that in my particular case the security group is not saved with the instance in the state file. Please let me know if I should open a separate ticket or not.

[catsby]

@dlsniper I'm not sure you have the same issue as @PaulCapestany – it sounds like you're using security_groups with an EC2 Instance inside a VPC, when you should be using vpc_security_group_ids? If so, please see the backwards incompatibility noted in the CHANGELOG for v0.7.0:

• https://github.com/hashicorp/terraform/blob/master/CHANGELOG.md#07-unreleased

See also:

• https://github.com/hashicorp/terraform/issues/6416

@PaulCapestany thanks for the report, I'll investigate!

[dlsniper]

@catsby thank you for the quick reply. You were right, it was from that. Could maybe Terraform abort operations when this situation is encountered? This seems like something that would be better handled by breaking the run rather than breaking the environment of the user.

[PaulCapestany]

@catsby so yeah, it seems that items in ordered arrays (such as the IPs in cidr_blocks) are stored in lexicographical order, i.e. an ordered array [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] is then represented as being [0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9]... which might be what causes conflicts between actual _.tf_ config file and _.tfstate_ files when doing terraform import?

Perhaps that points to part of the problem being located in all the calls to sort in state.go?

[catsby]

Hey @PaulCapestany – think this is mostly up to the self => false thing you points out. I submitted #7164 as a patch and from my testing (following your steps) I was able to reproduce, and verify the patch. Are you able to check it out?

[PaulCapestany]

@catsby good to hear—will test out your patch first thing tomorrow morning! :)

[PaulCapestany]

Hey @catsby, your patch seems to have improved things a bit (as in, less new resources are forced), but didn't appear to be a full fix. I'll send you some more info privately that I hope might help...

[catsby]

https://github.com/hashicorp/terraform/pull/7706 should address the additional thing @PaulCapestany mentioned

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.