Terraform-provider-aws: aws_elasticsearch_domain removing log_publishing_options continuously shows update

[nodesocket]

With the resource aws_elasticsearch_domain originally I had defined log_publishing_options:

    log_publishing_options {
        log_type = "INDEX_SLOW_LOGS"
        cloudwatch_log_group_arn = "${aws_cloudwatch_log_group.es_index_slow_logs.arn}"
    }

    log_publishing_options {
        log_type = "SEARCH_SLOW_LOGS"
        cloudwatch_log_group_arn = "${aws_cloudwatch_log_group.es_search_slow_logs.arn}"
    }

Howerver I remove those blocks completely, but now everytime I run plan or apply it is showing the follow in-place update:

  ~ module.elasticsearch.aws_elasticsearch_domain.elasticsearch
      log_publishing_options.#:                                  "2" => "0"
      log_publishing_options.502943241.cloudwatch_log_group_arn: "" => ""
      log_publishing_options.502943241.enabled:                  "false" => "false"
      log_publishing_options.502943241.log_type:                 "SEARCH_SLOW_LOGS" => ""
      log_publishing_options.810384934.cloudwatch_log_group_arn: "" => ""
      log_publishing_options.810384934.enabled:                  "false" => "false"
      log_publishing_options.810384934.log_type:                 "INDEX_SLOW_LOGS" => ""

I've run apply multiple times, but the change persists. I believe this may be a bug.

[nodesocket]

Any ideas on this?

[alkar]

I see the same behaviour:

$ terraform version
Terraform v0.11.10
+ provider.aws v1.46.0
+ provider.template v1.0.0

[marcelsavelkoul]

Same behaviour here.

Terraform v0.11.10
provider.aws v1.28.0

[forestjohnsonpeoplenet]

AWS' api for these log configuration settings is kind of bizarre.

As far as I can tell, it is actually impossible to remove them. Once they get created, you cant remove them.

But you can turn them off with enabled = false

the AWS management console (web site) displays them the same regardless of whether they exist but are turned off or don't exist (never were turned on).

I ran into this issue and it looks like Terraform is trying to remove the log configurations rather than turning them off.

You can work around it like this:

  // There is a bug/issue in the AWS API related to elasticsearch domain log publishing options.
  // They cannot be deleted, once created... In order for terraform to get rid of the log publishing option,
  // it must be specified as enabled=false (with an arn, because the arn is a required field ).  
  // Otherwise terraform will continue to attempt to remove the log publishing option eternally
  log_publishing_options {
    log_type                 = "ES_APPLICATION_LOGS"
    enabled                  = false
    cloudwatch_log_group_arn = "${aws_cloudwatch_log_group.es_log.arn}"
  }

But I think that terraform should probably just set them to disabled instead of trying to delete them, OR work with AWS to fix the api so they can be deleted.

[mbolek]

Terraform v0.11.7
+ provider.aws v1.35.0
+ provider.random v2.0.0

Same issue. I think there was a similar thing some time ago when disabling masters - AFAIR it was resolved by ignoring the values when enabled = false.
This is the info from aws-cli:

"LogPublishingOptions": {
                "ES_APPLICATION_LOGS": {
                    "Enabled": false
                }, 
                "INDEX_SLOW_LOGS": {
                    "Enabled": false
                }, 
                "SEARCH_SLOW_LOGS": {
                    "Enabled": false
                }
            }

and terraform:

      log_publishing_options.#:                                                                            "3" => "0"
      log_publishing_options.174383655.cloudwatch_log_group_arn:                                           "" => ""
      log_publishing_options.174383655.enabled:                                                            "false" => "false"
      log_publishing_options.174383655.log_type:                                                           "ES_APPLICATION_LOGS" => ""
      log_publishing_options.502943241.cloudwatch_log_group_arn:                                           "" => ""
      log_publishing_options.502943241.enabled:                                                            "false" => "false"
      log_publishing_options.502943241.log_type:                                                           "SEARCH_SLOW_LOGS" => ""
      log_publishing_options.810384934.cloudwatch_log_group_arn:                                           "" => ""
      log_publishing_options.810384934.enabled:                                                            "false" => "false"
      log_publishing_options.810384934.log_type:                                                           "INDEX_SLOW_LOGS" => ""

Maybe TF should ignore other logs-related fields when enabled=false similarly to the master=false situation?

[mbolek]

The solution proposed here: https://github.com/terraform-providers/terraform-provider-aws/issues/5752#issuecomment-459486355 does seem to work but it's only a client side workaround which is not ideal.
Could we maybe get a fix as in https://github.com/terraform-providers/terraform-provider-aws/issues/252 -> https://github.com/terraform-providers/terraform-provider-aws/pull/5423/files

[sshishov]

Hi all, we are using this approach meanwhile to overcome the issue:

resource "aws_elasticsearch_domain" "aws-elasticsearch" {
  ...

  # TODO: Remove this when the issue is fixed on AWS Provider: https://github.com/terraform-providers/terraform-provider-aws/issues/5752
  lifecycle {
    ignore_changes = [log_publishing_options]
  }
}

It brings us ability to not add additional useless code on client side at least.