Aws-sam-cli: AWS::Serverless::SimpleTable not Created in DynamoDB Local

[mrrobby]

Based on the spaces vs. tabs example https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/

It appears the SAM local YAML configuration with AWS::Serverless::SimpleTable should automatically create the table in DynamoDB Local, which in my case it is not. The voting lambda works as intended if I manually create the table with AWS CLI or the Dynamo Shell.

Should this create our table?

The specific output with error stack returned is:

2017/08/31 16:35:47 Invoking lambda.lambda_handler (python3.6)
START RequestId: 7477bb46-6a7f-42bb-89bd-8819de055637 Version: $LATEST
dynamodb.Table(name='spaces-tabs-votes')
An error occurred (ResourceNotFoundException) when calling the UpdateItem operation: Cannot do operations on a non-existent table: ResourceNotFoundException
Traceback (most recent call last):
  File "/var/task/lambda.py", line 29, in lambda_handler
    ReturnValues='ALL_NEW'
  File "/var/runtime/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/var/runtime/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/var/runtime/botocore/client.py", line 310, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 599, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the UpdateItem operation: Cannot do operations on a non-existent table

END RequestId: 7477bb46-6a7f-42bb-89bd-8819de055637
REPORT RequestId: 7477bb46-6a7f-42bb-89bd-8819de055637

Note: the table name is printed out for verification of correct Environment Variable usage

Here's my configurations:

__template.yaml__

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  VotesTable:
    Type: "AWS::Serverless::SimpleTable"
  VoteSpacesTabs:
    Type: "AWS::Serverless::Function"
    Properties:
      Runtime: python3.6
      Handler: lambda.lambda_handler
      Policies: AmazonDynamoDBFullAccess
      Environment:
        Variables:
          TABLE_NAME: !Ref VotesTable
      Events:
        Vote:
          Type: Api
          Properties:
            Path: /
            Method: post
        Query:
          Type: Api
          Properties:
            Path: /
            Method: get

I tried running DynamoDB local both as:

docker run -p 8000:8000 dwmkerr/dynamb  -sharedDb

and using

java -Djava.library.path=./DynamoDBLocal_lib/ -jar DynamoDBLocal.jar -sharedDb

Same result for both.

Initializing DynamoDB Local with the following configuration:
Port:   8000
InMemory:   false
DbPath: null
SharedDb:   true
shouldDelayTransientStatuses:   false
CorsParams: *

I am connecting to dynamo okay though, as seen by:

$ aws dynamodb list-tables --endpoint-url http://127.0.0.1:8000
{
    "TableNames": []
}

The votes_table assignment appears:

#lambda.py

import os
import json
import boto3

if os.getenv('AWS_SAM_LOCAL'):
  votes_table = boto3.resource('dynamodb', endpoint_url='http://docker.for.mac.localhost:8000/').Table(os.getenv('TABLE_NAME'))
else:
  votes_table = boto3.resource('dynamodb').Table(os.getenv('TABLE_NAME'))

def lambda_handler(event, context):
    print(votes_table)
.
.
.

And this is invoked by

TABLE_NAME=spaces-tabs-votes sam local start-api

then

curl -d '{"vote": "spaces"}' http://127.0.0.1:3000/

on

macOS Sierra 10.12.6
aws-cli/1.11.142 Python/3.6.2 Darwin/16.7.0 botocore/1.7.0
sam version 0.2.0
Docker version 17.06.1-ce, build 874a737

Thank you!

[PaulMaddox]

Sorry for the confusion. SAM Local currently only does anything with AWS::Serverless::Function resources, other resources are ignored. You will need to create the table.

[mrrobby]

Sounds good. I'll make a post to see if he can update the tutorial to alleviate the confusion. Thanks for the quick reply!

[gbassan-br]

@mrrobby

In the mean time, you can use this CLI command to create the local Table:

aws dynamodb create-table --endpoint-url http://localhost:8000 --table-name spaces-tabs-votes --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

You can also use this same command to create the table in DynamoDB service, just remove the --endpoint parameter and configure the AWS credentials on the aws configure

Regards

[heitorlessa]

@mrrobby if that helps, I simply user DynamoDB Local in a Docker Container and use both SAM Local and DynamoDB Local in the same Docker Network.

I also use a Bootstrap script to create my table with same basic data.

DynamoDB Local in a docker container

docker run -d -v "$PWD":/dynamodb_local_db -p 8000:8000 --name dynamodb cnadiminti/dynamodb-local

Inside my code I have a simple check for SAM_LOCAL env variable and initialize my DynamoDB differently:

# Quick sanity checks and predefined local dev
if os.getenv("AWS_SAM_LOCAL", ""):
    ddb = boto3.resource('dynamodb',
                         endpoint_url="http://dynamodb:8000")
    ddb_table = "users"
    # ret = requests.get("https://google.com")
    # print("Status --> ", ret.text)
else:
    ddb = boto3.resource('dynamodb')
    ddb_table = os.getenv("TABLE_NAME", None)

Source code: https://github.com/heitorlessa/sam-local-python-hot-reloading

[luebken]

@PaulMaddox wrote:
Sorry for the confusion. SAM Local currently only does anything with AWS::Serverless::Functionresources, other resources are ignored. You will need to create the table.

Seems to be an important limitation that should be noted in the docs.

[mrrobby]

Thanks guys. I like @heitorlessa's approach. Good call. That actually seems like a worthwhile future implementation for SAM Local.

[revmischa]

I'm okay with having to create the table, however SimpleTable names are mangled by CF. In my template.yml:

      Environment:
        Variables:
          MESSAGE_TABLE_NAME: !Ref qa_message
          FOO: bar
...
  # waiting on: https://github.com/awslabs/serverless-application-model/issues/252
  # TABLES:
  qa_message:
    Type: AWS::Serverless::SimpleTable
    PrimaryKey:
      Name: id
      Type: String

Which I think is what I need to get the SimpleTable's name to use in my lambda. But of course this doesn't work with sam local.
What's the correct way to handle the generated table names with sam local?
(I'm using CodeStar)

[phstc]

Sorry for the confusion. SAM Local currently only does anything with AWS::Serverless::Function resources, other resources are ignored. You will need to create the table.

@PaulMaddox is dynamodb-local support on the ROADMAP? If so, is there any ETA?

[revmischa]

I'm okay without having dynamodb support locally, I'm fine with using it via boto3. My problem is that cloudformation mangles the table name and I need to reference it in my lambda somehow to use it. How should I test a lambda that uses a SimpleTable using sam local?

[phstc]

@revmischa IMO dynamodb-local support would be very welcome, specially for reading the DynamoDB schema from the template YAML, otherwise, we need to manually provision the DynamoDB tables with custom scripts, duplicating in somehow the schema definition already in place in the template YAML. Besides that, we need also to manually configure it to run within the same network as the lambda Docker container.

The more SAM local can provision locally, the less we need to configure.

I don't want compare one with the other, but sls, does all of that, you define your DynamoDB schema, and it can provision DDB locally or on AWS, that's very handy. I would love to have that in SAM local.

[endymion]

I'm working on my first project using Dynamo DB using SAM, rather than the Serverless Framework. And yes, this is a big weakness. It makes me wish that I had used Serverless instead of SAM. I'll stick with it, but this is disappointing. Especially to see that this issue has been closed. If I need to move quickly in the future and SAM still can't manage local Dynamo DB resources then I'll start the next project with Serverless instead.

[jamescosford]

There are big issues with the tutorial. Very frustrating for me, and potentially other people new to the tech.

Actually getting this to work requires

• getting dynamodb-local to work with sam, using a solution like that suggested by @heitorlessa
• manual creation of the dynamodb table

Figuring out those steps helped me learn, but the tutorial does not even hint that they will be necessary. If you just follow the tutorial, it won't work, and that makes it a bad tutorial.

[michaelj-smith]

@revmischa Looks like the feature to be able to have a custom name on the SimpleTable is in progress. If we upvote this issue, we might see a resolution sooner: https://github.com/awslabs/serverless-application-model/issues/252

I don't know of any way to reference the SimpleTable's generated table name. This was a big reason I didn't adopt SAM immediately after it was announced last year. This is my #1 feature I'd love to see.

If we can set a static name on the table, then you can create a local table with the same name going through the suggestions of using dynamodb-local, as stated above.

[jessekoconnor]

I second that. My team is getting into sam and we love the local feature. If SAM prioritizes this feature I will also be quite happy. This is my #1 feature i'd love to see.

[rtalexk]

I just read Tabs or Spaces tutorial and I was wondering if the table should be created or not. And YES! That's kind of confusing. I had to google a lot to figure out that that's not currently supported.

As @phstc pointed The more SAM local can provision locally, the less we need to configure, and the better.

Maybe an integration with LocalStack to provision all services we require would be great.

[Yocker95k]

@rtalexk Yea same happened to me. Please if there is still no solution then at least rework the tutorial.