Serverless-application-model: Proposal: Events property for AWS::SQS::Queue to implicitly define an API Gateway

Created on 2 Nov 2018  路  3Comments  路  Source: aws/serverless-application-model

Description:

It would be really handy the possibility to define an Event as a property when declaring an SQS resource, in order to implicitly define an API Gateway that sends the payload as a message into the queue.

This is a simple SAM Template example of what I'm talking about:

MyProjectQueue:
    Type: AWS::SQS::Queue
    Properties:
        Events:
            MyProjectApi:
                Type: Api
                Properties:
                    Path: /myproject/push
                    Method: post

Current Behavior
Of course, being Events not supported by SQS Queue, the deploy fails and I can see the following error using the command aws cloudformation describe-stack-events --stack-name myproject-stack

MyProjectQueue-CREATE_FAILED-2018-10-30T16:33:29.764Z       
MyProjectQueue                      
CREATE_FAILED   
Encountered unsupported property Events AWS::SQS::Queue arn:aws:cloudformation:eu-west-1:<MY_AWS>:stack/myproject-stack/<GIUD>     
myproject-stack  2018-10-30T16:33:29.764Z

Expected result:
The expected result would be

sqsevent

Including the relevant policies. Exactly like it happens for a Lambda function

typfeature

Most helpful comment

@zeroastro I have an example of how to do this in the Swagger doc on my GitHub:

https://github.com/brysontyrrell/Serverless-Examples/blob/master/API-Gateway-Integrated-SQS-Queue/template.yaml

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: >-
  Create an API Gateway integrated with an SQS queue that is processed by a
  Lambda function as messages are populated.

Resources:

  ApiQueue:
    Type: AWS::SQS::Queue

  QueueApiGatewayRole:
    Type: AWS::IAM::Role
    Properties:
      Path: "/"
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - apigateway.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
      Policies:
        - PolicyName: ApiQueuePolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - sqs:ReceiveMessage
                  - sqs:SendMessage
                Resource: !GetAtt ApiQueue.Arn

  QueueApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      DefinitionBody:
        swagger: "2.0"
        info:
          title: !Ref AWS::StackName
        paths:
          "/":
            post:
              consumes:
              - "application/json"
              produces:
              - "application/json"
              responses:
                "200":
                  description: "200 response"
                  schema:
                    $ref: "#/definitions/Empty"
              x-amazon-apigateway-integration:
                credentials: !GetAtt QueueApiGatewayRole.Arn
                uri: !Sub "arn:aws:apigateway:${AWS::Region}:sqs:path//"
                responses:
                  default:
                    statusCode: "200"
                requestParameters:
                  integration.request.header.Content-Type: "'application/x-www-form-urlencoded'"
                requestTemplates:
                  application/json: !Sub "Action=SendMessage##\n&QueueUrl=$util.urlEncode('${ApiQueue}')##\n\
                    &MessageBody=$util.urlEncode($input.body)##\n"
                passthroughBehavior: "never"
                httpMethod: "POST"
                type: "aws"
        definitions:
          Empty:
            type: "object"
            title: "Empty Schema"

  QueueProcessor:
    Type: AWS::Serverless::Function
    Description: Processes messages from the SQS Queue.
    Properties:
      CodeUri: ./src/queue_processor
      Handler: queue_processor.lambda_handler
      Runtime: python3.6
      Timeout: 30
      ReservedConcurrentExecutions: 10
      Events:
        QueueMessages:
          Type: SQS
          Properties:
            Queue: !GetAtt ApiQueue.Arn
            BatchSize: 10

All 3 comments

@zeroastro I have an example of how to do this in the Swagger doc on my GitHub:

https://github.com/brysontyrrell/Serverless-Examples/blob/master/API-Gateway-Integrated-SQS-Queue/template.yaml

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: >-
  Create an API Gateway integrated with an SQS queue that is processed by a
  Lambda function as messages are populated.

Resources:

  ApiQueue:
    Type: AWS::SQS::Queue

  QueueApiGatewayRole:
    Type: AWS::IAM::Role
    Properties:
      Path: "/"
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - apigateway.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
      Policies:
        - PolicyName: ApiQueuePolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - sqs:ReceiveMessage
                  - sqs:SendMessage
                Resource: !GetAtt ApiQueue.Arn

  QueueApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      DefinitionBody:
        swagger: "2.0"
        info:
          title: !Ref AWS::StackName
        paths:
          "/":
            post:
              consumes:
              - "application/json"
              produces:
              - "application/json"
              responses:
                "200":
                  description: "200 response"
                  schema:
                    $ref: "#/definitions/Empty"
              x-amazon-apigateway-integration:
                credentials: !GetAtt QueueApiGatewayRole.Arn
                uri: !Sub "arn:aws:apigateway:${AWS::Region}:sqs:path//"
                responses:
                  default:
                    statusCode: "200"
                requestParameters:
                  integration.request.header.Content-Type: "'application/x-www-form-urlencoded'"
                requestTemplates:
                  application/json: !Sub "Action=SendMessage##\n&QueueUrl=$util.urlEncode('${ApiQueue}')##\n\
                    &MessageBody=$util.urlEncode($input.body)##\n"
                passthroughBehavior: "never"
                httpMethod: "POST"
                type: "aws"
        definitions:
          Empty:
            type: "object"
            title: "Empty Schema"

  QueueProcessor:
    Type: AWS::Serverless::Function
    Description: Processes messages from the SQS Queue.
    Properties:
      CodeUri: ./src/queue_processor
      Handler: queue_processor.lambda_handler
      Runtime: python3.6
      Timeout: 30
      ReservedConcurrentExecutions: 10
      Events:
        QueueMessages:
          Type: SQS
          Properties:
            Queue: !GetAtt ApiQueue.Arn
            BatchSize: 10

@brysontyrrell thank you for the solution posted, I've done something similar, that's why I came up with this proposal which I believe could be very handy

@brysontyrrell thank you so much

Was this page helpful?
0 / 5 - 0 ratings