Terraform-provider-aws: New aws provider slow aws_ssm_parameter calls when running terraform plan

[Supakadoo]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform v0.11.3

Affected Resource(s)

• aws_ssm_parameter

Terraform Configuration Files

Add upwards of 100 parameter store values.

resource "aws_ssm_parameter" "service-s3-bucket" {
  name      = "/${var.service}/${var.environment}/S3_BUCKET"
  type      = "String"
  value     = "${aws_s3_bucket.service-messages.id}"
  overwrite = true
}

Expected Behavior

Better performance on newer versions of the provider

Actual Behavior

A terraform plan is taking significantly longer on aws_ssm_parameter state refresh

Steps to Reproduce

1. Set provider to:

provider "aws" {
  region  = "${var.region}"
  version = "1.9.0"
}

1. Run terraform plan -> takes about 2 minutes to run in our case
2. Set provider to:

provider "aws" {
  region  = "${var.region}"
  version = "1.28.0"
}

1. Run terraform init
2. Run terraform plan -> takes 10+ minutes now

Important Factoids

Abnormally large amount of param store variables in our aws account(s)

[julienduchesne]

Fixed in https://github.com/terraform-providers/terraform-provider-aws/pull/5325

[r-kok]

The speed difference seems to occur betweem AWS provider versions <= 1.10.0 (fast) and >= 1.11.0 (slow)

[bflad]

Hi everyone 👋 Sorry for the strange behavior here. It turns out that the SSM API can return results in an unexpected manner (e.g. empty DescribeParameters pages for _all_ SSM parameters even though we asked specifically for one name) depending on which type of request filtering is used.

We just merged in #5325 which changes the filtering type to only make a single API request. It should be released later today in version 1.29.0 of the AWS provider. Huge shout out to @julienduchesne who spent a lot of time doing investigative work here and coming up with (we expect) the correct solution to reduce the number of API calls.

[bflad]

This has been released in version 1.29.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!