Terraform-provider-aws: Unable to Change RDS Parameter Group Values

[bauteb-sci]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.26

• provider.archive v1.3.0
• provider.aws v2.58.0
• provider.null v2.1.2
• provider.random v2.2.1
• provider.template v2.1.2

Affected Resource(s)

aws_db_parameter_group

Terraform Configuration Files

locals {
  db_parameters = [   
    {
      "name"         = "db_cache_size"
      "value"        = 5
      "apply_method" = "immediate"
    },
       {
      "name"         = "db_files"
      "value"        = 200
      "apply_method" = "pending-reboot"
    }
  ]
}

resource "aws_db_parameter_group" "parm_group_test" {
  name        = "parm-grp-tst"
  description = "test parms"
  family      = "oracle-se2-19"

  dynamic "parameter" {
    for_each = local.db_parameters
    content {
      name         = parameter.value.name
      value        = parameter.value.value
      apply_method = lookup(parameter.value, "apply_method", null)
    }
  }

  lifecycle {
    create_before_destroy = true
  }
}

Steps to Reproduce

1. terraform apply
2. change db_cache_size value from 5 to 15
3. terraform apply

Expected Behavior

The end result should be a parameter group with the following values:

Name Values Allowed values Modifiable Source Apply Type

---

db_cache_size 15 0-2199023255552 true user dynamic
db_files 200 200-20000 true user static

Actual Behavior

Here are the actual values:

Name Values Allowed values Modifiable Source Apply Type

---

db_cache_size 0-2199023255552 true engine-default dynamic
db_files 200 200-20000 true user static

Anytime there is a change to a parameter value, Terraform is setting that value to engine-default instead of the requested value. All future applys will attempt to set the value requested, but are unable to override the default setting.

Please correct the logic to properly apply updates to parameter values.

[sravankurra]

Hey We are also facing the same issue we are not able to modify PostgreSQL parameter group values. Any Expected ETA for this fix cc @breathingdust

[breathingdust]

Thanks for getting in touch @sravankurra. 👋 At this time this issue doesn’t have enough community support for the internal team to address it. We will continue to monitor the community interest in this issue, and once enough support is demonstrated, we will consider either doing the work ourselves or a merging a community contribution.

[wgorski]

We're facing the same problem in the Postgres12 parameter group family.

Terraform version: 0.12.28
AWS provider version: 2.70

This is a critical issue as parameters are getting lost and there is no workaround for setting the parameters aside from not using Terraform. Please reconsider the decision of not addressing this issue at this time. Thank you.

Example:

resource "aws_db_parameter_group" "temp-pg" {
  name        = "temp-pg"
  family      = "postgres12"
  description = "Postgres 12 parameter group"

  parameter {
    name  = "max_standby_archive_delay"
    value = "1200"
  }

  parameter {
    name  = "max_standby_streaming_delay"
    value = "1100"
  }
}

Changing the parameters through API works correctly:
aws rds modify-db-parameter-group --db-parameter-group-name temp-pg --parameters "ApplyMethod=immediate,ParameterName=max_standby_archive_delay,ParameterValue=360000"

Possible interesting terraform debugging output:

2020-07-28T14:40:14.805+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/28 14:40:14 [DEBUG] Not persisting max_standby_streaming_delay to state, as its source is "engine-default" and it isn't in the config

2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Date: ...
2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Accept-Encoding: gzip
2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Action=ModifyDBParameterGroup&DBParameterGroupName=temp-pg&Parameters.Parameter.1.ApplyMethod=immediate&Parameters.Parameter.1.ParameterName=max_standby_archive_delay&Parameters.Parameter.1.ParameterValue=360000&Parameters.Parameter.2.ApplyMethod=immediate&Parameters.Parameter.2.ParameterName=max_standby_streaming_delay&Parameters.Parameter.2.ParameterValue=360000&Version=2014-10-31
2020-07-28T14:40:09.108+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/28 14:40:10 [DEBUG] [aws-sdk-go] DEBUG: Response rds/ModifyDBParameterGroup Details:
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ RESPONSE ]--------------------------------------
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: HTTP/1.1 200 OK
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Connection: close
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Length: 375
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Type: text/xml
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Date: Tue, 28 Jul 2020 12:40:19 GMT
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amzn-Requestid: ...
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/28 14:40:10 [DEBUG] [aws-sdk-go] <ModifyDBParameterGroupResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   <ModifyDBParameterGroupResult>
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     <DBParameterGroupName>temp-pg</DBParameterGroupName>
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   </ModifyDBParameterGroupResult>
2020-07-28T14:40:10.007+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   <ResponseMetadata>
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     <RequestId>...</RequestId>
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   </ResponseMetadata>
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: </ModifyDBParameterGroupResponse>
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/28 14:40:10 [DEBUG] Reset DB Parameter Group: {
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   DBParameterGroupName: "temp-pg",
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   Parameters: [{
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ApplyMethod: "immediate",
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ParameterName: "max_standby_archive_delay",
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ParameterValue: ""
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     },{
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ApplyMethod: "immediate",
2020-07-28T14:40:10.008+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ParameterName: "max_standby_streaming_delay",
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:       ParameterValue: ""
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:     }],
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:   ResetAllParameters: false
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: }
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/07/28 14:40:10 [DEBUG] [aws-sdk-go] DEBUG: Request rds/ResetDBParameterGroup Details:
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: POST / HTTP/1.1
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Host: rds.ap-southeast-1.amazonaws.com
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: User-Agent: aws-sdk-go/1.32.12 (go1.13.7; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.28 (+https://www.terraform.io)
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Length: 425
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ...
2020-07-28T14:40:10.009+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2020-07-28T14:40:10.010+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Date: 20200728T124010Z
2020-07-28T14:40:10.010+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Security-Token: ...
2020-07-28T14:40:10.010+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Accept-Encoding: gzip
2020-07-28T14:40:10.010+0200 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
:

[wgorski]

If you are not able to define an ETA, is there a workaround for this?

This is really a critical issue if you are having 20+ prod environments managed by terraform and the only way to have the parameter groups set correctly is to manually changing them after every apply.

[forkinme]

This appears to be a duplicate of #11846 which has an associated PR #12112 under review

[anGie44]

Closing this as it duplicates ##13551. Nevertheless, thanks for taking the time to open up this issue @bauteb-sc.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!