Cloudformation-coverage-roadmap: AWS::Events::Rule targetting Cloudwatch logs

[ristkari]

1. Delivering events to Cloudwatch logs with Cloudformation

Interesting discovery with Cloudformation and Eventbridge. As a very simple way of reproducing the problem here is a template snippet. Consider the following:

 CFNLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      RetentionInDays: 3
      LogGroupName: '/aws/events/CFN'

  CFNRule:
    Type: AWS::Events::Rule
    Properties:
      EventPattern:
        source:
          - "aws.states"
      Targets:
        - Id: 'CloudwatchLogsTarget'
          Arn: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CFNLogGroup}"
        - Id: 'LambdaTarget'
          Arn: !GetAtt EventsFunction.Arn

  EventbridgePermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt EventsFunction.Arn
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt CFNRule.Arn

This is very straight forward way to tell that you would like to route all events from Step Functions to both Cloudwatch logs and also to custom lambda-function. Note the "AWS::Lambda::Permission" that is needed in order to invoke Lambda function. In other words the target needs to have resource policy that allows Eventbridge service to deliver the events.

This works partially. Lambda will get triggered but nothing is delivered to Cloudwatch logs. If you create this via UI it works because console does some magic behind the scenes. The magic in this case is the resource based policy for Cloudwatch.
This is told also on documentation: https://docs.aws.amazon.com/eventbridge/latest/userguide/resource-based-policies-eventbridge.html

"If you use the AWS Management Console to add CloudWatch Logs as the target of a rule, this policy is created automatically. If you use the AWS CLI to add the target, you must create this policy if it doesn't exist."

There is no way to add resource based policies for cloudwatch via cloudformation, you are forced to create custom resource if you want to do it. For Lambda it works because you can create AWS::Lambda::Permission via Cloudformation. Cloudwatch resource policy you cannot. Only way of creating those is via CLI, API or Consoles 'behind the scenes' magic.

So the question is whether there is upcoming support to natively doing this? If you are trying to automate this it's either custom resource for Cloudformation which introduces additional complexity since you have to create your custom resource in different stack. Other option is to use CLi but then your pipeline/automation process is littered with CLI here, cloudformation there - not very hygienic solution.

[benkehoe]

The resource policy is part of CloudWatch Logs, so this request is essentially the same as #249. As a workaround, I think you should be able to create an IAM role with permission to deliver to CloudWatch Logs, and give that role to the target.

[ristkari]

The problem is that even if I create such a role and give it as "RoleArn" it won't get used. Only way to make it work is with resource policy on Cloudwatch.

Tried with:

  CFNTargetRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: 'sts:AssumeRole'
      Policies:
      -   PolicyName: "AllowLogging"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: "Allow"
                Action:
                  - 'logs:*'
                Resource: "*"
      -   PolicyName: "AllowLambdaInvoke"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: "Allow"
                Action:
                  - 'lambda:InvokeFunction'
                Resource: !GetAtt EventsFunction.Arn

And the Lambda wouldn't fire either without the permission:

  EventbridgePermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt EventsFunction.Arn
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt CFNRule.Arn

[benkehoe]

When you set that up, can you verify with aws events list-targets-by-rule --rule <rule-name> that the role is correctly set on the target?

[ristkari]

It doesn't look it is set on the target:

❯ aws events list-targets-by-rule --rule cfn-status-consumer-CFNRule-W6YPCCYF0LQA                                                                                           op-cloudfoundation-tools [gke_angrybeardproject_europe-north1_angrybeard|default]
{
    "Targets": [
        {
            "Id": "CloudwatchLogsTarget",
            "Arn": "arn:aws:logs:eu-central-1:REDACTEC_ACCOUNT_ID:log-group:/aws/events/CFN"
        },
        {
            "Id": "LambdaTarget",
            "Arn": "arn:aws:lambda:eu-central-1:REDACTEC_ACCOUNT_ID:function:cfn-event-lambda"
        }
    ]
}

I used the following template snippet:

  CFNRule:
    Type: AWS::Events::Rule
    Properties:
      RoleArn: !GetAtt CFNTargetRole.Arn
      EventPattern:
        source:
          - "aws.states"
      Targets:
        - Id: 'CloudwatchLogsTarget'
          Arn: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CFNLogGroup}"
        - Id: 'LambdaTarget'
          Arn: !GetAtt EventsFunction.Arn

[benkehoe]

Ah, if you put it on the role it should show up in aws events describe-rule --name <rule-name>. What happens if you put the role in the target like this?

CFNRule:
    Type: AWS::Events::Rule
    Properties:
      EventPattern:
        source:
          - "aws.states"
      Targets:
        - Id: 'CloudwatchLogsTarget'
          RoleArn: !GetAtt CFNTargetRole.Arn
          Arn: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CFNLogGroup}"
        - Id: 'LambdaTarget'
          Arn: !GetAtt EventsFunction.Arn

[ristkari]

You cannot put it there. I tried - cloudformation complains that RoleArn is not supported for Cloudwatch (nor for Lambda). However it is required if targetting eventbus on another account.

So this is why it's pretty confusing when it is possible to also provide it in properties and also on individual targets.

[benkehoe]

Wow, that's kind of a mess.

[TLaue]

I am having the same issue. Any workaround available? Thank you very much for any advice.

[possebom]

Same issue here. Does anyone have an AWS CLI script as workaround?

[amirbadnjevic]

Same here. Would really prefer having the support for it in CloudFormation.
any ETA?

[shotty1]

Not sure if this is exactly the same issue, but I have played around with this for a bit and the whole thing only seems to work when the LogGroupName starts with /aws/events/
I still can not set a RoleArn for the individual target though

This definition just works in my tests:

  RuleSecurityScans:
    Type: AWS::Events::Rule
    Properties:
      Description: ""
      EventPattern:
        {
          "detail-type": [
            "ECR Image Scan"
          ],
          "source": [
            "aws.ecr"
          ]
          }
      State: ENABLED
      RoleArn: !GetAtt RoleCWLogsDelivery.Arn
      Targets:
      -
        Arn: !GetAtt LogGroupScanFindings.Arn
        Id: LogGroup


  RoleCWLogsDelivery:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: 'sts:AssumeRole'
      Policies:
      -   PolicyName: "AllowLogging"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: "Allow"
                Action:
                  - 'logs:*
                Resource: "*"

  LogGroupScanFindings:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: /aws/events/SamImageScanFindings
      RetentionInDays: 30

[rnhurt]

To add a bit of context to @shotty1's comment, you can't even use the CloudWatch Events console to send events to a "custom" log group. I have several log groups in my account and the only one the console will allow me to configure is the "TESTING" one (/aws/events/TESTING).

My CFN template tried to configure it to put events to the "/lh/securityhub/events" log group; that's where the "hub/events" snipped comes from in the text box.

tl;dr I think this might be a CloudWatch Event Rules problem and not really a CFN problem.

[ErkoKnoll]

I created L2 CDK construct for using CW as a target for EventBridge. I used a custom resource as a workaround so my Log Group would accept logs sent from EventBridge. This can be translated to CF:

export class LogGroupTarget implements IRuleTarget {
    constructor(private readonly stack: Stack, private readonly logGroup: LogGroup) {
    }

    public bind(rule: IRule, id: string): RuleTargetConfig {
        // Ugly hack to grant a permission for allowing EventBridge to store logs in CloudWatch
        const policyName = `${rule.ruleName}-CloudWatchPolicy`
        new AwsCustomResource(this.stack, "CloudwatchLogResourcePolicy", {
            resourceType: "Custom::CloudwatchLogResourcePolicy",
            onUpdate: {
                service: "CloudWatchLogs",
                action: "putResourcePolicy",
                parameters: {
                    policyName,
                    policyDocument: JSON.stringify({
                        Version: "2012-10-17",
                        Statement: [
                            {
                                Sid: policyName,
                                Effect: "Allow",
                                Principal: {
                                    Service: ["events.amazonaws.com"]
                                },
                                Action: ["logs:CreateLogStream", "logs:PutLogEvents"],
                                Resource: this.logGroup.logGroupArn
                            }
                        ]
                    })
                },
                physicalResourceId: PhysicalResourceId.of(policyName),
            },
            onDelete: {
                service: "CloudWatchLogs",
                action: "deleteResourcePolicy",
                parameters: {
                    policyName
                }
            },
            policy: AwsCustomResourcePolicy.fromStatements([
                new PolicyStatement({
                    actions: ["logs:PutResourcePolicy", "logs:DeleteResourcePolicy"],
                    resources: ["*"]
                })
            ])
        });

        return {
            id: '',
            arn: this.logGroup.logGroupArn,
            targetResource: this.logGroup
        }
    }
}

[josjaf]

Do we need an iam role if we are using /aws/events as the prefix?

[aditya81070]

@josjaf No, we don't need any IAM role. I just changed my log group name from my-events to /aws/events/my-events and things started to work fine.