Cloudformation-coverage-roadmap: AWS::Events::Rule->EventPattern - Ability to supply a integer value

[Guslington]

1. Title

AWS::Events::Rule->EventPattern - Ability to supply a integer value

2. Scope of request

Ability to supply a integer value in a cloudwatch event pattern

3. Expected behavior

On create or update supply a integer value in a cloudwatch event pattern, however cloudformation converts that value to a string. Integer values are required for guardduty severity.

4. Suggest specific test cases

Example event pattern for guardduty

{
      "detail-type": [
        "GuardDuty Finding"
      ],
      "source": [
        "aws.guardduty"
      ],
      "detail": {
        "severity": [
          5,8
        ]
    }
}

5. Helpful Links to speed up research and evaluation

https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html#guardduty_cloudwatch_example

[michaelwittig]

This bug get's more relevant with the new filter capabilities:

  ECRIMageScanFindingEvent:
    Type: 'AWS::Events::Rule'
    Properties:
      EventPattern:
        source:
        - 'aws.ecr'
        'detail-type':
        - 'ECR Image Scan'
        detail:
          finding-severity-counts:
            CRITICAL: [{exists: false}, {numeric: ['>', 0]}]
            HIGH: [{exists: false}, {numeric: ['>', 0]}]
            MEDIUM: [{exists: false}, {numeric: ['>', 0]}]
            UNDEFINED: [{exists: false}, {numeric: ['>', 0]}]

is turned into:

{
  "detail-type": [
    "ECR Image Scan"
  ],
  "source": [
    "aws.ecr"
  ],
  "detail": {
    "finding-severity-counts": {
      "HIGH": [
        {
          "exists": "false"
        },
        {
          "numeric": [
            ">",
            "0"
          ]
        }
      ],
      "MEDIUM": [
        {
          "exists": "false"
        },
        {
          "numeric": [
            ">",
            "0"
          ]
        }
      ],
      "UNDEFINED": [
        {
          "exists": "false"
        },
        {
          "numeric": [
            ">",
            "0"
          ]
        }
      ],
      "CRITICAL": [
        {
          "exists": "false"
        },
        {
          "numeric": [
            ">",
            "0"
          ]
        }
      ]
    }
  }
}

which results in all kinds of validation errors.

instead of:

{
  "detail-type": [
    "ECR Image Scan"
  ],
  "source": [
    "aws.ecr"
  ],
  "detail": {
    "finding-severity-counts": {
      "HIGH": [
        {
          "exists": false
        },
        {
          "numeric": [
            ">",
            0
          ]
        }
      ],
      "MEDIUM": [
        {
          "exists": false
        },
        {
          "numeric": [
            ">",
            0
          ]
        }
      ],
      "UNDEFINED": [
        {
          "exists": false
        },
        {
          "numeric": [
            ">",
            0
          ]
        }
      ],
      "CRITICAL": [
        {
          "exists": false
        },
        {
          "numeric": [
            ">",
            0
          ]
        }
      ]
    }
  }
}

[jogold]

@TheDanBlanco any update on this? Really frustrating to see that the new advanced event patterns cannot be deployed via CloudFormation.

[TheDanBlanco]

nothing yet. there are other items with higher +1's that we're working on right now. i'll keep an eye on this though

[jogold]

Thanks for the feedback @TheDanBlanco.

Just to be clear, it's not really a coverage issue here but rather a bug. CloudFormation stringifies everything in AWS::Events::Rule.EventPattern (numbers, booleans) and this leads to validation errors from CloudWatch Events.

[jmnarloch]

I had created initial pull request for dedicated builder: https://github.com/aws/aws-cdk/pull/6414

[stevecaldwell77]

@TheDanBlanco: the example by @michaelwittig above shows that this issue is not just integers, but booleans too. Just wanted to explicitly call that out so it didn't go unnoticed! Not being able to use an exists pattern is really limiiting.

[jackcbrown89]

@TheDanBlanco Any status updates on a fix for this? This blocks us from writing many of our event patterns in CF templates.

[ljacobsson]

Also interested in an update on this. EventBridge is a very potent service, but this bug makes it very unattractive to adopt for any team. We're ending up writing ugly workarounds which is getting painful.

Imho this is more of a bug than a feature request and there must be internal interest from the EventBridge team to get this fixed. Is it solely prioritised based on upvotes or do we have a chance to see this fixed in the near future?

[stevecaldwell77]

@ljacobsson - a fix is supposedly rolling out: https://twitter.com/nickste/status/1237036312571437059

[ljacobsson]

@stevecaldwell77 yes I was following the rollout of that one, but it got pulled back.

I'm tracking this through various threads and just learnt from here that a fix is rolling out very soon 👍

[ritchieleeann]

Any updates on this? That Twitter thread said there'd be another update on ETA last week.

[sukdeb-gitlab]

I can confirm the above pattern works as expected now, probably the fix has been applied:

{
      "detail-type": [
        "GuardDuty Finding"
      ],
      "source": [
        "aws.guardduty"
      ],
      "detail": {
        "severity": [
          5,8
        ]
    }
}

[michaelwittig]

Looks like it still doesn't work with booleans?

Pattern:

EventPattern:
  source:
  - 'aws.ecr'
  'detail-type':
  - 'ECR Image Scan'
  detail:
    finding-severity-counts:
      CRITICAL: [{exists: false}, {numeric: ['>', 0]}]
      HIGH: [{exists: false}, {numeric: ['>', 0]}]
      MEDIUM: [{exists: false}, {numeric: ['>', 0]}]
      UNDEFINED: [{exists: false}, {numeric: ['>', 0]}]

Error:

Event pattern is not valid. Reason: exists match pattern must be either true or false. at [Source: (String)"{"detail-type":["ECR Image Scan"],"source":["aws.ecr"],"detail":{"finding-severity-counts":{"HIGH":[{"exists":"false"},{"numeric":[">","0"]}],"MEDIUM":[{"exists":"false"},{"numeric":[">","0"]}],"UNDEFINED":[{"exists":"false"},{"numeric":[">","0"]}],"CRITICAL":[{"exists":"false"},{"numeric":[">","0"]}]}}}"; line: 1, column: 112] (Service: AmazonCloudWatchEvents; Status Code: 400; Error Code: InvalidEventPatternException; Request ID: d4f20cd8-1edc-4e63-b55b-ab7339111615)

[belangovan]

still its not fixed

"containers": {
      "exitCode": [
        "1",
        "2"
      ]
    },
we need this to be 
`"containers": {
      "exitCode": [ 1  ]
    },`

kindly confirm fix, we are blocked with this issue

[codesmithtech]

Confirmed as not working with booleans when using the latest version of the AWS CLI

[shakyaabiral]

Using cdk, the cloudformation json template is generated well

"containers": {
                          "exitCode": [
                            1,
                            2
]
}

but the is converted to strings on deployment.
Waiting for updates...

[wintersky]

This may be fixed at least for integers. On August 6th I observed a stack with detail: numeric: ["<", 3] produce a rule which looks correct: "numeric": ["<", 3] and which works properly in testing.

[michaelwittig]

I can confirm the fix

[KIVagant]

Inability to use boolean values for boolean parameters for years is ridiculous.

[shihrer]

Are boolean values still not working? I haven't had a chance to check since we haven't had to deploy any new rules with boolean values yet. CloudFormation bugs with event patterns have been a bit of a pain for us.

[fitzplb]

Still not working for me with integers, my template looks like this:

"cloudwatchGuardDutyMediumHighRule":{"Type":"AWS::Events::Rule","Properties":{"Name":"GuardDutyMediumHighRule","Description":"A GuardDuty finding of Medium to High severity has been detected","EventPattern":{"source":["aws.guardduty"],"detail-type":["GuardDuty Finding"],"detail":{"severity":[4,4.1,4.2,4.3,4.4,4.5,4.6,4.7,4.8,4.9,5,5,5.1,5.2,5.3,5.4,5.5,5.6,5.7,5.8,5.9,6,6,6.1,6.2,6.3,6.4,6.5,6.6,6.7,6.8,6.9,7,7,7.1,7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,8,8,8.1,8.2,8.3,8.4,8.5,8.6,8.7,8.8,8.9]}}

but my rule turns out like this:

{ "detail-type": [ "GuardDuty Finding" ], "source": [ "aws.guardduty" ], "detail": { "severity": [ "4", "4.1", "4.2", "4.3", "4.4", "4.5", "4.6", "4.7", "4.8", "4.9", "5", "5", "5.1", "5.2", "5.3", "5.4", "5.5", "5.6", "5.7", "5.8", "5.9", "6", "6", "6.1", "6.2", "6.3", "6.4", "6.5", "6.6", "6.7", "6.8", "6.9", "7", "7", "7.1", "7.2", "7.3", "7.4", "7.5", "7.6", "7.7", "7.8", "7.9", "8", "8", "8.1", "8.2", "8.3", "8.4", "8.5", "8.6", "8.7", "8.8", "8.9" ] } }

[ljacobsson]

@fitzplb

I just tried your pattern and it was created correctly:

{
  "detail-type": [
    "GuardDuty Finding"
  ],
  "source": [
    "aws.guardduty"
  ],
  "detail": {
    "severity": [
      4,
      4.1,
      4.2,
      4.3,
      4.4,
      4.5,
      4.6,
      4.7,
      4.8,
      4.9,
      5,
      5,
      5.1,
      5.2,
      5.3,
      5.4,
      5.5,
      5.6,
      5.7,
      5.8,
      5.9,
      6,
      6,
      6.1,
      6.2,
      6.3,
      6.4,
      6.5,
      6.6,
      6.7,
      6.8,
      6.9,
      7,
      7,
      7.1,
      7.2,
      7.3,
      7.4,
      7.5,
      7.6,
      7.7,
      7.8,
      7.9,
      8,
      8,
      8.1,
      8.2,
      8.3,
      8.4,
      8.5,
      8.6,
      8.7,
      8.8,
      8.9
    ]
  }
}

Could it be that the fix isn't rolled out in all regions. I'm in eu-west-1 and for me it's been fixed for a couple of weeks now.

FWIW, your event pattern can be simplified as

{
  "source": ["aws.guardduty"],
  "detail-type": ["GuardDuty Finding"],
  "detail": { "severity": [ { "numeric": [ ">=", 4, "<", 9] } ] },
}

[fitzplb]

@ljacobsson I think you're right I've just tried it in eu-west-1 and it works fine. Thanks for the tip about simplifying the event pattern, the docs don't have that (https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html)

[fitzplb]

@ljacobsson actually if I use your event pattern Cloudformation renders the integers correctly without quotes, even in eu-west-2

[serverlesspolska]

It does not work for me in Oregon and N.Virginia regions.

I have following CF excerpt (it's a part of huge CF Template generated by Serverless Framework)

"ControlNotificationsCustomEventBridge1": {
      "Type": "Custom::EventBridge",
        .....
      "Properties": {
         ....
        "EventBridgeConfig": {
          "RuleName": "test-event-bridge-lambda-dev-myFunction-rule-1",
          "EventBus": "ServiceBusArn...",
          "Pattern": {
            "source": [
              "my-source"
            ],
            "detail-type": [
              "EVENT_TYPE"
            ],
            "detail": {
              "entityId": [
                {
                  "numeric": [
                    {
                      "=": 159
                    }
                  ]
                }
              ]
            }
          }
   }
}

And CF is failing with

Failed to create resource. Event pattern is not valid. 
Reason: Invalid member in numeric match: {
   at [Source: (String)"{
               "detail-type":["EVENT_TYPE"],
               "source":["my-source"],
               "detail":{ 
                     "entityId":[{"numeric":[{"=":"159"}]}]
  }
}
"; line: 1, column: 126]

[ljacobsson]

@serverlesspolska what does Custom::EventBridge do? I guess that's stringifying all values.

Tried using AWS::Events::Rule?

[serverlesspolska]

@ljacobsson I'm caught between you and Serverless Framework
There is an issue: https://github.com/serverless/serverless/issues/7506#issuecomment-738100855 and they blame CloudFormation.

To be honest I don't know what Custom::EventBridgedoes - it's generated by the framework - but it's part of CloudFormation. And for sure it has a number not string. So it looks like they are right.

Yes, I could use AWS::Events::Rule but the whole point of using Serverless Framework is that it generates a lot of IaC for us. In fact, we are using now AWS::Events::Rule as a workaround solution, but it leads to a lot of additional code instead of several lines in our Yaml template.