Localstack: AWS SQS Lambda event source does not work because the Lambda ARN and the event source ARN are different (when using Serverless and Cloud Formation)

[ashlux]

The ARN between the SQS ARN and the event source ARN are different. This causes the Lambda to not get called when for SQS events.

Steps to reproduce

1. Use my example project: https://github.com/ashlux/serverless-localstack-sqs-lambda-event.git
2. Using my fix to Moto so the Serverless deploy will work: https://github.com/spulec/moto/issues/2045#issuecomment-489759139. (This fixes https://github.com/localstack/localstack/issues/1288).
3. Run the example project: npm i && serverless deploy --stage local
4. Send a message to SQS: awslocal sqs send-message --queue-url http://localhost:4576/queue/my-sqs-dev --message-body rawr

Expected result

The SQS message causes the Lambda to execute and output SQS Event along with the event. The SQS queue ARN should match the event source ARN from the event source mapping.

What actually happens

The Lambda is not called after the SQS message is sent.

The SQS queue ARN does not match the event source ARN from the event source mapping so the SQS event does not cause the Lambda to be executed.

The queue ARN has the region elasticmq and the region 000000000000 whereas the event source ARN has the region us-east-1 and the region 123456789012.

╰─➤  awslocal sqs get-queue-attributes --queue-url http://localhost:4576/queue/my-sqs-dev --attribute-name QueueArn                            
{
    "Attributes": {
        "QueueArn": "arn:aws:sqs:elasticmq:000000000000:my-sqs-dev"
    }
}

╰─➤  awslocal lambda list-event-source-mappings                                                                                                                              

{
    "EventSourceMappings": [
        {
            "UUID": "f31b9c96-6601-4e2e-aa59-c950a75acd24",
            "BatchSize": 100,
            "EventSourceArn": "arn:aws:sqs:us-east-1:123456789012:my-sqs-dev",
            "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:my-lambda-dev",
            "LastModified": 1557254699.0,
            "LastProcessingResult": "OK",
            "State": "Enabled",
            "StateTransitionReason": "User action"
        }
    ]
}

I've tracked down the problem in two places, one in Moto and the other in LocalStack.

1. https://github.com/spulec/moto/blob/master/moto/sqs/models.py#L188
2. https://github.com/localstack/localstack/blob/b51c2c4414a1a412cf57533a2b7c1b984647d2b5/localstack/utils/aws/aws_stack.py#L353

[thetumper]

Can this be bumped to higher priority?

[iamkoch]

Can you add "and Terraform" to the title, as I'm suffering from this issue having created the infra using terraform. I've tried configuring the event source mapping using the lambda's function name _and_ arn properties, but both resolve to the same eu-west-based arn, yet the mapping points to a us-east-1 one for some reason.

$ awslocal sqs get-queue-attributes --queue-url http://localhost:4576/queue/iv-verify_identity --attribute-names QueueArn
{
    "Attributes": {
        "QueueArn": "arn:aws:sqs:us-east-1:000000000000:iv-verify_identity"
    }
}

$ awslocal lambda list-event-source-mappings
{
    "EventSourceMappings": [
        {
            "UUID": "f738853b-b221-4e81-963e-addcf03e8636",
            "BatchSize": 100,
            "EventSourceArn": "arn:aws:sqs:eu-west-1:000000000000:iv-verify_identity",
            "FunctionArn": "arn:aws:lambda:eu-west-1:000000000000:function:lambda_verify_identity_command_handler",
            "LastModified": 1566564979.0,
            "LastProcessingResult": "OK",
            "State": "Enabled",
            "StateTransitionReason": "User action"
        }
    ]
}

Update
OK this made it obvious that if I used us-east-1 it'd probably work - which it did, however there's an underlying issue that needs fixing

[whummer]

Thanks for the detailed bug report @ashlux @thetumper @iamkoch . I was able to run your sample project - can you please give it a try with the latest version of the Docker image?

I applied a few minor patches to your project configuration, see the diff below:

$ git diff
diff --git a/package.json b/package.json
index 05324ef..fb85f4f 100644
--- a/package.json
+++ b/package.json
@@ -10,7 +10,8 @@
   "license": "ISC",
   "dependencies": {
     "aws-sdk": "^2.440.0",
-    "aws-xray-sdk-core": "^2.3.2"
+    "aws-xray-sdk-core": "^2.3.2",
+    "serverless-deployment-bucket": "^1.1.0"
   },
   "devDependencies": {
     "serverless-localstack": "^0.4.12"
diff --git a/serverless.yml b/serverless.yml
index 7bddfc2..a393a0d 100644
--- a/serverless.yml
+++ b/serverless.yml
@@ -12,7 +12,7 @@ provider:

 functions:
   my-lambda:
-    handler: src/index.handler
+    handler: index.handler
     name: my-lambda-${self:provider.stage}
     runtime: nodejs8.10
     timeout: 120
@@ -35,8 +35,10 @@ custom:
   localstack:
     debug: true
     host: http://localhost
+    autostart: true
     stages:
       - local

 plugins:
   - serverless-localstack
+  - serverless-deployment-bucket

Also, please make sure to configure the default region to us-east-1 (AWS_DEFAULT_REGION=us-east-1), then it should work without any mismatches between the ARNs.

Closing this issue - please re-open if the problem persists. Thanks!