Aws-cdk: Cloudwatch eventbus policy failed to create stack

[Black742]

Have tried to create a eventbuspolicy through the cdk module but the stack is failed with the "Internal Failure" message while creating changeset for the first time.

When the cdk deploy the stack its trying to create a changeset however there is no stack exist on the same stack name. Because of this, the stack creation is failed and throwing the "Internal Failure" error.
I have confirmed this issue with AWS Cloudformation team.

Can you have a look and let me know how to fix this issue.

import cdk = require('@aws-cdk/cdk');
import events = require('@aws-cdk/aws-events');
export class EventBusPolicyStack extends cdk.Stack {
    constructor(parent: cdk.App, name: string , organizationId: string,props?: cdk.StackProps){
        super(parent,name,props)


        new events.CfnEventBusPolicy(this,"test",{
            action: 'events:PutEvents',
            principal: '*',
            statementId: "default",
            condition: {
                type: "StringEquals",
                key: "aws:PrincipalOrgID",
                value: organizationId
            }

        })

    }

}

Error:

EventBusPolicyStack
EventBusPolicyStack: deploying...
EventBusPolicyStack: creating CloudFormation changeset...

 ❌  EventBusPolicyStack failed: Error: Failed to create ChangeSet CDK-7ea91d54-95db-473e-bc89 on EventBusPolicyStack: FAILED, Internal Failure
Failed to create ChangeSet CDK-7ea91d54-95db-473e-bc89 on EventBusPolicyStack: FAILED, Internal Failure
ERROR Tue, Feb 19, 2019 11:42:59 AM -- Failed to execute bootstrap.js, so exiting the app

[rix0rrr]

The tool should be creating the stack just before creating the change set.

Can you run cdk deploy -v and paste the output to ? (Add the -v switch)

[Black742]

@rix0rrr Have attached the verbose logs.
```
CDK toolkit version: 0.22.0 (build 644ebf5)
Command line arguments: { _: [ 'deploy' ],
trace: false,
strict: false,
'ignore-errors': false,
ignoreErrors: false,
json: false,
j: false,
verbose: true,
v: true,
ec2creds: undefined,
i: undefined,
'version-reporting': undefined,
versionReporting: undefined,
'path-metadata': true,
pathMetadata: true,
'asset-metadata': true,
assetMetadata: true,
version: false,
help: false,
h: false,
exclusively: false,
e: false,
app: 'node bin/aws-master-bootstrap.js',
a: 'node bin/aws-master-bootstrap.js',
'role-arn': undefined,
r: undefined,
roleArn: undefined,
'$0': '..\..\AppData\Roaming\npm\node_modules\aws-cdk\bin\cdk',
context: undefined,
plugin: undefined,
rename: undefined,
profile: undefined,
proxy: undefined,
'toolkit-stack-name': undefined,
'require-approval': undefined,
STACKS: [] }
Determining whether we're on an EC2 instance.
Using proxy server: http://localhost:3128
cdk.json: {
"app": "node bin/aws-cdk-bootstrap.js",
"requireApproval": "never"
}
Setting "aws:cdk:toolkit:default-region" context to ap-southeast-2
Resolving default credentials
Does not look like EC2 instance.
Retrieved account ID *** from disk cache
Setting "aws:cdk:toolkit:default-account" context to ***
context: { 'aws:cdk:toolkit:default-region': 'ap-southeast-2',
'aws:cdk:toolkit:default-account': '***',
'aws:cdk:enable-path-metadata': true,
'aws:cdk:enable-asset-metadata': true }
outdir: C:\Users\kmuxts\AppData\Local\Temp\cdkFLPzFi
outfile: C:\Users\kmuxts\AppData\Local\Temp\cdkFLPzFi\cdk.out
{ version: '0.19.0',
stacks:
[ { name: 'EventBusPolicyStack',
environment: [Object],
template: [Object],
metadata: [Object] } ],
runtime:
{ libraries:
{ '@aws-cdk/cdk': '0.22.0',
'@aws-cdk/cx-api': '0.22.0',
'@aws-cdk/aws-events': '0.22.0',
'jsii-runtime': 'node.js/v8.11.4' } } }
Removing outdir C:\Users\kmuxts\AppData\Local\Temp\cdkFLPzFi
Stack name not specified, so defaulting to all available stacks: EventBusPolicyStack
Using default AWS SDK credentials for account ***
Waiting for stack CDKToolkit to finish creating or updating...
Stack CDKToolkit does not exist
The environment ***/us-east-1 doesn't have the CDK toolkit stack (CDKToolkit) installed. Use cdk bootstrap "***/us-east-1" to setup your environment for use with the toolkit
.
EventBusPolicyStack: deploying...
Using default AWS SDK credentials for account ***
Attempting to create ChangeSet CDK-265c1a4c-e865-4155-905f-6a949688d8ac to create stack EventBusPolicyStack
EventBusPolicyStack: creating CloudFormation changeset...
Initiated creation of changeset: arn:aws:cloudformation:us-east-1:***:changeSet/CDK-265c1a4c-e865-4155-905f-6a949688d8ac/ab525656-d49d-4f19-8138-1735e1e9af7e; waiting for it to finis
h creating...
Waiting for changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack to finish creating...
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating
Changeset CDK-265c1a4c-e865-4155-905f-6a949688d8ac on stack EventBusPolicyStack is still creating

❌ EventBusPolicyStack failed: Error: Failed to create ChangeSet CDK-265c1a4c-e865-4155-905f-6a949688d8ac on EventBusPolicyStack: FAILED, Internal Failure
Failed to create ChangeSet CDK-265c1a4c-e865-4155-905f-6a949688d8ac on EventBusPolicyStack: FAILED, Internal Failure
Error: Failed to create ChangeSet CDK-265c1a4c-e865-4155-905f-6a949688d8ac on EventBusPolicyStack: FAILED, Internal Failure
at waitFor (C:\Users\kmuxts\AppData\Roaming\npm\node_modules\aws-cdk\lib\api\util\cloudformation.ts:113:11)
at
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
```

[rix0rrr]

I'm sorry, I have no idea what's wrong here, but the problem is more likely with CloudFormation than with CDK.

Can you try the following:

cdk synth > EventBusPolicyStack.yaml

Then deploy EventBusPolicyStack.yaml the way you would normally do it?

aws cloudformation create-stack --stack-name EventBusPolicyStack --template-body file://./EventBusPolicyStack.yml

Or more like what CDK actually does:

aws cloudformation create-change-set --stack-name EventBusPolicyStack --change-set-name CreateChangeSet --template-body file://./EventBusPolicyStack.yml --change-set-type CREATE

[Black742]

@rix0rrr I tried manually deploying the stack through the console and its created successfully. I used the template which created from cdk synth.

[rix0rrr]

I can reproduce the failure by creating a stack set using the template above. The stack set ends up in the same failure state. This behavior happens in both us-east-1 and eu-west-1.

I am a 100% sure this is a CloudFormation bug. I will report it internally as well.

[eladb]

@rix0rrr do we have an internal reference for this issue so we can follow up?

[rix0rrr]

Yes. Where do you want me to put it?

[0xdabbad00]

Any updates on this? I've run into the same problem.

[rix0rrr]

Internal reference: CFN-16877

[rix0rrr]

Public reference: https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/156

[rix0rrr]

Going to remove our "bug" label as this is not actionable for us.

[lazharichir]

Any idea how to fix this temporarily? I need to allow access to and from the event bus for all AWS Accounts in my AWS Organization.

{
    "companyeventbuspolicy": {
        "Type": "AWS::Events::EventBusPolicy",
        "Properties": {
            "Action": "events:PutEvents",
            "Principal": "*",
            "StatementId": "company-event-bus-policy-statement",
            "Condition": {
                "Key": "aws:PrincipalOrgID",
                "Type": "StringEquals",
                "Value": "o-123456789"
            },
            "EventBusName": {
                "Ref": "companyeventbus9ABF23AA"
            }
        },
        "Metadata": {
            "aws:cdk:path": "company-events-stack/company-event-bus-policy"
        }
    }
}

And that part causes the internal failure.