Aws-cdk: Cloudwatch SQS Event Target

[dbettin]

Is it possible to have a SQS queue as an event target via a EventRule?

[rix0rrr]

It's not built in to Queue right now, but you can write a new class that implements IEventRuleTarget and put a method like this on it:

https://github.com/awslabs/aws-cdk/blob/master/packages/@aws-cdk/aws-sns/lib/topic-base.ts#L282

(Reference your queue instead).

[wangchunyang]

Unfortunately, software.amazon.awscdk.services.events.EventRuleTargetProps doesn't support to set message group id which is required for sending message to FIFO queue. To fix the gap, we can use AWS CloudFormation Library CfnRule and CfnRule.TargetProperty which has an property named CfnRule.SqsParametersProperty.

[Visorgood]

Very important feature for us! We want to send messages to an SQS as soon as StepFunction execution is complete. Currently we have to have a custom Lambda inside SF to do that.

[made2591]

I would like to work on this if it is ok for everyone: I'm currently into the topic for a personal project, and I think I can make some experiment on this

[eladb]

That would be great but sync up with @rix0rrr since he plans a bit of a refactor in this area in the coming days.

[made2591]

I think I drafted a solution, the Event Rule was generated correctly with the SQS Target. I didn't write the test yet...

[rix0rrr]

Sounds great! Feel free to create a draft PR for it, I will have a look.

[rix0rrr]

I'm thinking the model should be a mix between this: https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-events-targets/lib/ecs-ec2-task.ts

And this: https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-events-targets/lib/sns.ts

Specifically, I'm thinking the SQS target takes an

  readonly message?: events.RuleTargetInput;

As well as fields that go into SqsParameters: https://docs.aws.amazon.com/AmazonCloudWatchEvents/latest/APIReference/API_SqsParameters.html

(Seems to be only messageGroupId)

[made2591]

@rix0rrr thank you, it would really be appreciated! I just wrote the test using SNS as model. I'm building right now and I will run them ^^

For everyone reading about this issue, if your use case is around triggering events from S3 event creation directly to SQS you can use successfully addObjectCreatedNotification in the @aws-cdk/aws-s3

Don't forget to provide to SQS the right policy to let S3 publish correctly: I'm working on a working sample here

[made2591]

@rix0rrr I implemented the SqsQueueProps exactly like this. I didn't focus on parameters yet, just build the skeleton for the binding: I used the grantSendMessages inside. What would you like to get from the ecs-task definition?

[rix0rrr]

What I wanted to get from that example is: we have multiple props on SqsQueueProps, some of them go into the Input and others go into SqsParameters, and the distinction is invisible to consumers.

If you've already got that, great!

Also, it seems to me it's not possible to get messageGroupId from the event contents, is that right? Or am I misunderstanding something...?

[made2591]

Ok, regarding parameters I got your point from 10k feets, but I'm confused 🤔I had a look at https://docs.aws.amazon.com/AmazonCloudWatchEvents/latest/APIReference/API_EcsParameters.html and some of them seem to be not present in https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-events-targets/lib/ecs-ec2-task.ts - like the containerOverrides.

Then in the bind, I saw that the input takes from the props, together with networkConfiguration...maybe this is what you were meaning by mixing Input and SqsParameters. Still, by trying to emulate the logic in ECS, I don't understand why some properties like LaunchType are not present.

Also, it seems to me it's not possible to get messageGroupId from the event contents, is that right? Or am I misunderstanding something...?

Regarding this, I still have to investigate. However, I packaged also the tests with just the message - by following almost the SNS topic and I run the tests and they seem to be fine. Do you think I can already make a pull request even if parameters are not present yet?