Terraform-provider-aws: aws_route53_record.test_cert_validation issue with aws provider

[vlinx]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14

• provider.aws v2.21.1
• provider.null v1.0.0

Affected Resource(s)

• aws_route53_record, aws_acm_certificate

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

Debug Output

terraform apply output below....

data.aws_route53_zone.cluster_zone: Refreshing state...
aws_acm_certificate.test_cert: Creating...
arn: "" => ""
domain_name: "" => ""
domain_validation_options.#: "" => ""
subject_alternative_names.#: "" => ""
tags.%: "" => "5"
.
.
.
validation_emails.#: "" => ""
validation_method: "" => "DNS"
aws_acm_certificate.test_cert: Creation complete after 3s (ID: )

Error: Error applying plan:

1 error occurred:

• aws_route53_record.test_cert_validation: Resource 'aws_acm_certificate.test_cert' does not have attribute 'domain_validation_options.0.resource_record_value' for variable 'aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_value'

Panic Output

Expected Behavior

should create the aws_route53_record

Actual Behavior

1 error occurred:

• aws_route53_record.test_cert_validation: Resource 'aws_acm_certificate.test_cert' does not have attribute 'domain_validation_options.0.resource_record_value' for variable 'aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_value'

Steps to Reproduce

1. terraform apply

on the following code replacing redacted values with suitable values.

provider "aws" {
region = "ap-southeast-2"
version = "~> 2.21.1"
assume_role {
role_arn = "arn:aws:iam:::role/"
session_name = "SESSION_NAME"
external_id = "EXTERNAL_ID"
}
}

resource "aws_acm_certificate" "test_cert" {
domain_name = " "
validation_method = "DNS"
tags {

}
}

data "aws_route53_zone" "cluster_zone" {
name = ""
private_zone = false
}

resource "aws_route53_record" "test_cert_validation" {
name = "${aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_name}"
type = "${aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_type}"
zone_id = "${data.aws_route53_zone.cluster_zone.zone_id}"
records = ["${aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_value}"]
ttl = 60
}

resource "aws_acm_certificate_validation" "test_cert" {
certificate_arn = "${aws_acm_certificate.test_cert.arn}"
validation_record_fqdns = ["${aws_route53_record.test_cert_validation.fqdn}"]
}

Important Factoids

References

• #0000

[vlinx]

The same code has been used regularly until about a week ago when it started failing.

[jdenly]

This appears to be a region specific problem. If I run the acceptance test TestAccAWSAcmCertificate_dnsValidation in the default region (us-west-2) it works fine. If I then set the region to ap-southeast-2 by exporting AWS_DEFAULT_REGION, I get the following error:

--- FAIL: TestAccAWSAcmCertificate_dnsValidation (19.56s)
    testing.go:568: Step 0 error: Check failed: Check 2/10 error: aws_acm_certificate.cert: Attribute 'domain_name' expected "tf-acc-6668933416344597644.<REDACTED>", got ""
FAIL

[zygimantas]

I've started getting the same error on us-east-1 and other regions and that made my whole infrastructure undeployable. Is that a breaking change from AWS?

[jdenly]

@zygimantas Definitely seems that way, this started happening for us about a week ago in ap-southeast-2. Investigating with the AWS CLI, it looks like describe certificate is now returning an empty validation options array to start with - it resolves later. This change is breaking the terraform code as it doesn't retry in that scenario.

[bflad]

Hi everyone 👋 Thank you so much for reporting this issue and sorry its being so problematic. 😖 The additional information being provided is super helpful. Indeed this does like an API change that is being problematic with the aws_acm_certificate resource.

The error messaging shown in this issue, is relevant for Terraform 0.11 and earlier, e.g.

Resource 'aws_acm_certificate.test_cert' does not have attribute 'domain_validation_options.0.resource_record_value' for variable 'aws_acm_certificate.test_cert.domain_validation_options.0.resource_record_value'

While the error messaging shown in #9616, is the equivalent for Terraform 0.12 and later, e.g.

aws_acm_certificate.cert.domain_validation_options is empty list of object

We are opting to consolidate similar reports of the same underlying issue here in this issue just to keep discussions and efforts in one location.

It looks like a potential fix has been submitted in #9598 so we will be taking a look at that shortly. I'll provide an additional update here once we have verified that pull request, hopefully later today.

Thanks everyone for your patience and hoping this can get addressed quickly.

[obourdon]

See my latest comment in #9345 under this section

[bflad]

The fix for this has been merged and will release with version 2.23.0 of the Terraform AWS Provider. Our releases generally occur on Thursdays, but given the nature of this bug, we may release sooner.

[angiedev]

Is there a workaround for this? Our deployment scripts keep failing because of this error.

Thanks,
Angela

[obourdon]

@angiedev on my side besides fixing the code of the provider myself, relaunching the deployment a 2nd time right after it fails just works

[angiedev]

Thanks @obourdon. Unfortunately this doesn't work for me. I am attempting to change our deployment scripts to get around this issue. I hope the fix will be available soon. I greatly appreciate everyone's efforts in getting this fixed.

[angiedev]

I am trying to find a workaround since when I create my cert using:
resource "aws_acm_certificate" "arb_certificate" {
domain_name = "${var.domain_name}"
validation_method = "DNS"
lifecycle {
create_before_destroy = true
}
tags = "${merge(
local.common_tags,
map(
"Name", "${local.hostname}-acm-cert"
),
local.fe_common_cloud_tags
)}"
}

My output vars are coming out blank:
output "arb_cert_record_name" {
value = "${aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_name}"
}
output "arb_cert_record_type" {
value = "${aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_type}"
}
output "arb_cert_record_value" {
value = "${aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_value}"
}

As a workaround I thought I could try looking up the cert after the fact and then get the values but that did not work either:

data "aws_acm_certificate" "arb_certificate_recall" {
domain = "${var.domain_name}",
statuses = ["ISSUED", "PENDING_VALIDATION"],
depends_on = ["aws_acm_certificate.arb_certificate"]
}

output "arb_cert_record_name" {
value = "${data.aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_name}"
}

output "arb_cert_record_type" {
value = "${data.aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_type}"
}

output "arb_cert_record_value" {
value = "${data.aws_acm_certificate.arb_certificate_recall.domain_validation_options.0.resource_record_value}"
}

The values are still blank. Is there another approach you could recommend? I am unable to deploy. I would appreciate any tips!

[obourdon]

@angiedev on my side before changing the AWS provider code I was thinking about using some kind of "delay method" which could probably be combined with that you tried above but this is more like a wild guess ...
More specifically something like was is explained in this link

[jltuts]

When will they release v2.23.0?

[obourdon]

@jltuts: from what @bflad told above

The fix for this has been merged and will release with version 2.23.0 of the Terraform AWS Provider. Our releases generally occur on Thursdays, but given the nature of this bug, we may release sooner.

We are Wednesday so I guess that the plan is still to make this release happen tomorrow.

[angiedev]

@obourdon,
Thanks for the tip! I can give it a quick try to hold us over until the release.

[hashibot[bot]]

This has been released in version 2.23.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[jltuts]

I just tested it and it works well!

Thanks guys...

[obourdon]

me too, confirming that 2.23.0 solves the issues. Many thanks to all

[angiedev]

Works great!! Thanks so much!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!