Aws-cdk: CloudFormation Error using Certificate in some regions: "ValidationDomain cannot be used with DNS validation"

Created on 12 May 2020  ·  15Comments  ·  Source: aws/aws-cdk

As of 2020-05-11 I'm seeing a CloudFormation Error "ValidationDomain cannot be used with DNS validation" when trying to create a Certificate using DNS validation.

I have currently deployed stacks that no longer deploy. As far as I can tell this is a change on the CloudFormation side of things. It now breaks in us-west-2 and eu-central-1 but there is no error and things work fine in ap-northeast-1.

Reproduction Steps

My certificate code looks something like this (Scala): 

Certificate.Builder.create(scope, "my-ssl-cert")
  .domainName("www.example.com")
  .validationMethod(ValidationMethod.DNS)
  .build()

The resulting CloudFormation template looks like this:

Resources:
  mysslcertF404C81B:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: www.example.com
      DomainValidationOptions:
        - DomainName: www.example.com
          ValidationDomain: example.com
      ValidationMethod: DNS
    Metadata:
      aws:cdk:path: cert-failure-us-west-2/my-ssl-cert/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.38.0,@aws-cdk/assets=1.38.0,@aws-cdk/aws-apigateway=1.38.0,@aws-cdk/aws-applicationautoscaling=1.38.0,@aws-cdk/aws-autoscaling=1.38.0,@aws-cdk/aws-autoscaling-common=1.38.0,@aws-cdk/aws-autoscaling-hooktargets=1.38.0,@aws-cdk/aws-certificatemanager=1.38.0,@aws-cdk/aws-cloudformation=1.38.0,@aws-cdk/aws-cloudfront=1.38.0,@aws-cdk/aws-cloudwatch=1.38.0,@aws-cdk/aws-cognito=1.38.0,@aws-cdk/aws-ec2=1.38.0,@aws-cdk/aws-ecr=1.38.0,@aws-cdk/aws-ecr-assets=1.38.0,@aws-cdk/aws-ecs=1.38.0,@aws-cdk/aws-elasticloadbalancing=1.38.0,@aws-cdk/aws-elasticloadbalancingv2=1.38.0,@aws-cdk/aws-elasticloadbalancingv2-targets=1.38.0,@aws-cdk/aws-events=1.38.0,@aws-cdk/aws-iam=1.38.0,@aws-cdk/aws-kms=1.38.0,@aws-cdk/aws-lambda=1.38.0,@aws-cdk/aws-logs=1.38.0,@aws-cdk/aws-route53=1.38.0,@aws-cdk/aws-route53-targets=1.38.0,@aws-cdk/aws-s3=1.38.0,@aws-cdk/aws-s3-assets=1.38.0,@aws-cdk/aws-sam=1.38.0,@aws-cdk/aws-secretsmanager=1.38.0,@aws-cdk/aws-servicediscovery=1.38.0,@aws-cdk/aws-sns=1.38.0,@aws-cdk/aws-sns-subscriptions=1.38.0,@aws-cdk/aws-sqs=1.38.0,@aws-cdk/aws-ssm=1.38.0,@aws-cdk/cloud-assembly-schema=1.38.0,@aws-cdk/core=1.38.0,@aws-cdk/custom-resources=1.38.0,@aws-cdk/cx-api=1.38.0,@aws-cdk/region-info=1.38.0,jsii-runtime=Java/14

Error Log

I get errors in both us-west-2 and eu-central-1 that look like this:

$ cdk deploy cert-failure-us-west-2
cert-failure-us-west-2: deploying...
cert-failure-us-west-2: creating CloudFormation changeset...
 0/3 | 7:45:10 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 0/3 | 7:45:11 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 1/3 | 7:45:11 AM | CREATE_FAILED        | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) ValidationDomain cannot be used with DNS validation
    new Certificate (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-kernel-jKhSVJ/node_modules/@aws-cdk/aws-certificatemanager/lib/certificate.js:27:22)
    \_ /private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7853:49
    \_ Kernel._wrapSandboxCode (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:8313:20)
    \_ Kernel._create (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7853:26)
    \_ Kernel.create (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7600:21)
    \_ KernelHost.processRequest (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7388:28)
    \_ KernelHost.run (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7328:14)
    \_ Immediate._onImmediate (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7331:37)
    \_ processImmediate (internal/timers.js:456:21)
 2/3 | 7:45:12 AM | CREATE_FAILED        | AWS::CDK::Metadata                   | CDKMetadata Resource creation cancelled
 2/3 | 7:45:13 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack           | cert-failure-us-west-2 The following resource(s) failed to create: [mysslcertF404C81B, CDKMetadata]. . Rollback requested by user.
 3/3 | 7:45:17 AM | DELETE_COMPLETE      | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 3/3 | 7:45:17 AM | DELETE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 4/3 | 7:45:18 AM | DELETE_COMPLETE      | AWS::CDK::Metadata                   | CDKMetadata
 5/3 | 7:45:19 AM | ROLLBACK_COMPLETE    | AWS::CloudFormation::Stack           | cert-failure-us-west-2

 ❌  cert-failure-us-west-2 failed: Error: The stack named cert-failure-us-west-2 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE
    at /usr/local/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:245:13
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at waitFor (/usr/local/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:157:20)
    at Object.deployStack (/usr/local/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:233:26)
    at CdkToolkit.deploy (/usr/local/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:181:24)
    at main (/usr/local/lib/node_modules/aws-cdk/bin/cdk.ts:235:16)
    at initCommandLine (/usr/local/lib/node_modules/aws-cdk/bin/cdk.ts:169:9)
The stack named cert-failure-us-west-2 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE

However this works fine in ap-northeast-1:

Here is the template:

Resources:
  mysslcertF404C81B:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: www.example.com
      DomainValidationOptions:
        - DomainName: www.example.com
          ValidationDomain: example.com
      ValidationMethod: DNS
    Metadata:
      aws:cdk:path: cert-failure-ap-northeast-1/my-ssl-cert/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.38.0,@aws-cdk/assets=1.38.0,@aws-cdk/aws-apigateway=1.38.0,@aws-cdk/aws-applicationautoscaling=1.38.0,@aws-cdk/aws-autoscaling=1.38.0,@aws-cdk/aws-autoscaling-common=1.38.0,@aws-cdk/aws-autoscaling-hooktargets=1.38.0,@aws-cdk/aws-certificatemanager=1.38.0,@aws-cdk/aws-cloudformation=1.38.0,@aws-cdk/aws-cloudfront=1.38.0,@aws-cdk/aws-cloudwatch=1.38.0,@aws-cdk/aws-cognito=1.38.0,@aws-cdk/aws-ec2=1.38.0,@aws-cdk/aws-ecr=1.38.0,@aws-cdk/aws-ecr-assets=1.38.0,@aws-cdk/aws-ecs=1.38.0,@aws-cdk/aws-elasticloadbalancing=1.38.0,@aws-cdk/aws-elasticloadbalancingv2=1.38.0,@aws-cdk/aws-elasticloadbalancingv2-targets=1.38.0,@aws-cdk/aws-events=1.38.0,@aws-cdk/aws-iam=1.38.0,@aws-cdk/aws-kms=1.38.0,@aws-cdk/aws-lambda=1.38.0,@aws-cdk/aws-logs=1.38.0,@aws-cdk/aws-route53=1.38.0,@aws-cdk/aws-route53-targets=1.38.0,@aws-cdk/aws-s3=1.38.0,@aws-cdk/aws-s3-assets=1.38.0,@aws-cdk/aws-sam=1.38.0,@aws-cdk/aws-secretsmanager=1.38.0,@aws-cdk/aws-servicediscovery=1.38.0,@aws-cdk/aws-sns=1.38.0,@aws-cdk/aws-sns-subscriptions=1.38.0,@aws-cdk/aws-sqs=1.38.0,@aws-cdk/aws-ssm=1.38.0,@aws-cdk/cloud-assembly-schema=1.38.0,@aws-cdk/core=1.38.0,@aws-cdk/custom-resources=1.38.0,@aws-cdk/cx-api=1.38.0,@aws-cdk/region-info=1.38.0,jsii-runtime=Java/14

$ cdk deploy cert-failure-ap-northeast-1
cert-failure-ap-northeast-1: deploying...
cert-failure-ap-northeast-1: creating CloudFormation changeset...
 0/3 | 7:48:46 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 0/3 | 7:48:46 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 0/3 | 7:48:47 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) Resource creation Initiated
 0/3 | 7:48:48 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata Resource creation Initiated
 1/3 | 7:48:48 AM | CREATE_COMPLETE      | AWS::CDK::Metadata                   | CDKMetadata
 1/3 | 7:49:19 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) Content of DNS Record is: {Name: _XXXXXXX.www.example.com.,Type: CNAME,Value: _XXXXXXXXXXXXXX.acm-validations.aws.}
1/3 Currently in progress: mysslcertF404C81B
 2/3 | 7:51:20 AM | CREATE_COMPLETE      | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 3/3 | 7:51:22 AM | CREATE_COMPLETE      | AWS::CloudFormation::Stack           | cert-failure-ap-northeast-1

 ✅  cert-failure-ap-northeast-1

Stack ARN:
arn:aws:cloudformation:ap-northeast-1:001234567890:stack/cert-failure-ap-northeast-1/e31b3e34-ffff-ffff-ffff-fffffffffffff

Environment

  • CLI Version: 1.38.0 (and 1.32.2)
  • Framework Version: 1.38.0 (and 1.32.2)
  • OS: Mac
  • Language: Scala

Other

I also posted this to the CloudFormation forums: https://forums.aws.amazon.com/thread.jspa?threadID=321860

This is :bug: Bug Report

@aws-cdaws-certificatemanager bug in-progress
Source
picture tpunder
😕3 👍2

Most helpful comment

@citosid the hack is no longer needed. My stacks that were running into problems earlier are working now without any changes.

picture cal5barton on 20 May 2020
🎉2 👍1

All 15 comments

@skinny85 I have a possible fix here: https://github.com/tpunder/aws-cdk/commit/d94aa22ef0478a70557f3d275bb5c5be7af5c048

aws-certificatemanager builds fine and the updated tests pass but I'm having some issues building the rest of CDK so I can test the fixes in my project.

tpunder picture tpunder on 12 May 2020

Also running into this issue.

cal5barton picture cal5barton on 12 May 2020
👍2

Thanks for the report everyone. It seems like this is some additional validation that the CloudFormation team added here. We're discussing with them whether it's the correct behavior.

To unblock yourself in the meantime, you can use our escape hatch mechanism:

const cfnCertificate = certificate.node.defaultChild as cm.CfnCertificate;
cfnCertificate.domainValidationOptions = undefned;
skinny85 picture skinny85 on 13 May 2020
👍2

@skinny85 Thanks for the note about the escape hatch. It looks like that should also solve my immediate problem.

tpunder picture tpunder on 13 May 2020

@skinny85 :

Hi Team!

For me it is the same by just using cfn! Last time the cfn script worked, now I got this error at the events tab of the stack: "ValidationDomain cannot be used with DNS validation"

CFN ressource block: 

Parameters:
  DomainName:
    Type: String
    Default: "sample.example.com"
    Description: The name of the DNS name where the Load Balancer should be reached on.

Resources:
  CMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref 'DomainName'
      DomainValidationOptions:
      - DomainName: !Ref 'DomainName'
        ValidationDomain: !Ref 'DomainName'
      ValidationMethod: DNS

Outputs:
  CertArn:
    Description: ARN of certificate
    Value: !Ref 'CMCertificate'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'CertArn' ] ]

Do you know the resson of the error, or how to continue correctly with provisioning!
I am not able to post it on the AWS Forum somehow, therefore I post it here ;)

Thanks for help in advance, because this blocks my work!!

ChristophEckerle picture ChristophEckerle on 13 May 2020

@ChristophEckerle

Just remove this from your template (it is not needed for DNS validation) and it should work:

DomainValidationOptions:
  - DomainName: !Ref 'DomainName'
    ValidationDomain: !Ref 'DomainName'
tpunder picture tpunder on 13 May 2020

@tpunder Thank you for quick answer! With this change I get "CMCertificate" >> "Value cannot be null or empty!"

Resources:
  CMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref 'DomainName'
      DomainValidationOptions:
      - DomainName: !Ref 'DomainName'
        ValidationDomain: !Ref 'DomainName'

According to the docu, this option is not requiered:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-certificatemanager-certificate.html#cfn-certificatemanager-certificate-validationmethod

ChristophEckerle picture ChristophEckerle on 14 May 2020

@ChristophEckerle What does your template look like that is giving you that error?

tpunder picture tpunder on 14 May 2020

@tpunder This is the complete Template runing in eu-central-1:


##########################################################################                                                                           #                                                                                                                                                            #
#                    TLS CERTIFICATION MANAGEMENT RESSOURCES                                                        #
#                                                                                                                                                           #
##########################################################################

AWSTemplateFormatVersion: '2010-09-09'

Description: A stack for deploying on the top of the base stack the TLS management by Certifaction Manager. Needs as input the DNS Name. Certificate provisioning requires manual intervention on the first run! The CNAME that is listed in the CloudFormation events needs to be set up manually.
Certificates need to be created in the us-east-1 region.

Parameters:
  DomainName:
    Type: String
    Default: "dev.example.de"
    Description: The name of the DNS name where the Load Balancer should be reached on.

Resources:
  CMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref 'DomainName'
      DomainValidationOptions:
      - DomainName: !Ref 'DomainName'
        ValidationDomain: !Ref 'DomainName'

Outputs:
  CertArn:
    Description: ARN of certificate
    Value: !Ref 'CMCertificate'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'CertArn' ] ]
ChristophEckerle picture ChristophEckerle on 14 May 2020

@ChristophEckerle It's hard to read your copy/paste, can you triple quote it? Also, now you seem to be missing the ValidationMethod field.

This works fine for me:

Resources:
  mysslcertF404C81B:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: www.example.com
      ValidationMethod: DNS
tpunder picture tpunder on 14 May 2020

@tpunder: Sure, I did.

If I put ValidationMethod: DNS I get a different error: ""ValidationDomain cannot be used with DNS validation""

If I leave it, I get: "Value cannot be null or empty!"

The script above worked till yesterday in eu-central-1 fine by using ValidationMethod: DNS, for then it breaks without any change on the script.

ChristophEckerle picture ChristophEckerle on 14 May 2020

@ChristophEckerle The short term fix for the "ValidationDomain cannot be used with DNS validation" error is to remove the DomainValidationOptions field when using DNS validation.

This works fine for me in eu-central-1 without any errors:

##########################################################################
#                    TLS CERTIFICATION MANAGEMENT RESSOURCES
#
##########################################################################

AWSTemplateFormatVersion: '2010-09-09'

Description: A stack for deploying on the top of the base stack the TLS management by Certifaction Manager. Needs as input the DNS Name. Certificate provisioning requires manual intervention on the first run! The CNAME that is listed in the CloudFormation events needs to be set up manually. Certificates need to be created in the us-east-1 region.

Parameters:
  DomainName:
    Type: String
    Default: "example.com"
    Description: The name of the DNS name where the Load Balancer should be reached on.

Resources:
  CMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref 'DomainName'
      ValidationMethod: DNS

Outputs:
  CertArn:
    Description: ARN of certificate
    Value: !Ref 'CMCertificate'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'CertArn' ] ]
tpunder picture tpunder on 14 May 2020

An update here.

Cloudformation was experienceing validation errors when performing CloudFormation stack operations for the AWS::CertificateManager resource type. The issue has been resolved and the service is operating normally.

You should be able to remove any escape hatch code for now. Still considering whether we want to remove DomainValidationOptions when ValidationMethod is DNS. Feel free to reopen if this is still occurring for anyone.

MrArnoldPalmer picture MrArnoldPalmer on 19 May 2020
👍3

@MrArnoldPalmer , just to be clear, this "hack" is not needed anymore?

citosid picture citosid on 19 May 2020

@citosid the hack is no longer needed. My stacks that were running into problems earlier are working now without any changes.

cal5barton picture cal5barton on 20 May 2020
🎉2 👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

eladb picture eladb  ·  3Comments
nzspambot picture nzspambot  ·  3Comments
eladb picture eladb  ·  3Comments
ababra picture ababra  ·  3Comments
kawamoto picture kawamoto  ·  3Comments