Terraform-provider-aws: Error tagging resources

[hashibot[bot]]

_This issue was originally opened by @vmorkunas as hashicorp/terraform#24395. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Terraform Version

Terraform v0.12.23
provider.aws v2.53.0

Terraform Configuration Files

Root module

module "peering_ldap_intapp" {
    source = "../../modules/Stack/Peering"
    stackCommon = var.stackCommon
    providers = {
        aws.src = aws.ldap
        aws.dst = aws.stack
    }
    peering = {
        peering_connection_name = "Ldap-IntApp", 
        different_account = true,
        account_id = var.account_id,
        src_vpc_id = var.ldap_ops_vpc_id,
        dst_vpc_id = module.intapp_vpc.vpc.id,
    }
}

Peering module

resource "aws_vpc_peering_connection" "src_peering" {
    provider = aws.src
    peer_owner_id = var.peering.different_account ? var.peering.account_id : null
    vpc_id = var.peering.src_vpc_id
    peer_vpc_id = var.peering.dst_vpc_id
    peer_region   = var.stackCommon.stack_region
    auto_accept   = false

    tags = merge(
        map(
            "Name", "${var.stackCommon.stack_name}-${var.peering.peering_connection_name}"
        ),
        var.stackCommon.common_tags
    )

    lifecycle {
        create_before_destroy = true
    }
}

Debug Output

Crash Output

error updating EC2 VPC Peering Connection (pcx-0c090a18f48d63647) tags: error tagging resource (pcx-0c090a18f48d63647): InvalidVpcPeeringConnectionID.NotFound: The vpcPeeringConnection ID 'pcx-0c090a18f48d63647' does not exist

Expected Behavior

Resource should be tagged

Actual Behavior

No taggs added and execution stops with error message above

Steps to Reproduce

Issue doesn't occur all the time.

1. terraform init
2. terraform apply

Additional Context

Many resources, started failing on Tagging step in the resources with the same error - resource not found

[faucherb94]

We've started seeing the same thing within the past few days. We tried reverting the aws provider version back a couple versions, no help.

[dshkyra]

I'm also seeing this issue after switching to terraform AWS provider v2.54 for security groups and KMS keys:

Error: error adding EC2 Security Group (sg-123) tags: error tagging resource (sg-123): InvalidGroup.NotFound: The security group 'sg-123' does not exist

Error: error updating KMS Key (key-123) tags: error tagging resource (key-123): NotFoundException: Key 'arn:aws:kms:us-east-1:1234567890:key/key-123' does not exist

[mgusiew-guide]

My team has also observed tagging errors after switching to terraform AWS provider v2.52 (we also switched from terraform 0.12.18 to 0.12.23)

We often get errors when tagging internet gateways or security groups. See samples below:

1) Tagging internet gateway:
Error: error adding EC2 Internet Gateway (igw-013f22d7f3ebe56e9) tags: error tagging resource (igw-013f22d7f3ebe56e9): InvalidInternetGatewayID.NotFound: The internetGateway ID 'igw-013f22d7f3ebe56e9' does not exist status code: 400, request id: 58d2d7a0-538f-4948-a198-55f552a5aef5 on ../../../commons/modules/vpc/main.tf line 12, in resource "aws_internet_gateway" "test_igw": resource "aws_internet_gateway" "test_igw" [command.go:158: command.go:158: retry.go:80: Returning due to fatal error: FatalError{Underlying: exit status 1}

2) Tagging security group:
error adding EC2 Security Group (sg-0847237555c0220b9) tags: error tagging resource (sg-0847237555c0220b9): InvalidGroup.NotFound: The security group 'sg-0847237555c0220b9' does not exist status code: 400, request id: 002fe92a-e58e-4f91-9ac7-2ed4cbd733d1 on ../../../commons/modules/ec2/main.tf line 43, in resource "aws_security_group" "test_sg": resource "aws_security_group" "test_sg" [command.go:158: [command.go:158: retry.go:80: Returning due to fatal error: FatalError{Underlying: exit status 1}

We drilled into security group scenario, searched CloudTrail and found that security group and tags are created at exactly the same time (seconds precision) . So it may be that create tag is started before create security group is completed with success (race condition), according to AWS docs it takes some time for security group to propagate.

Unfortunately this results in flaky test so it would be great to have it fixed

[hashibot[bot]]

This has been released in version 2.57.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!