Moto: DynamoDB2 update does not work with multiple/complex expression attribute values

[austinbyers]

DynamoDB updates do not work with all possible expression attribute values.

Code Snippet

This code snippet works exactly as written against the real DynamoDB (as long as you sleep a few seconds between table creation and item insertion). However, it breaks in moto:

import boto3
import moto

@moto.mock_dynamodb2()
def update_does_not_work():
    dynamo_client = boto3.client('dynamodb')

    # Create table - works fine.
    dynamo_client.create_table(
        AttributeDefinitions=[{'AttributeName': 'SHA256', 'AttributeType': 'S'}],
        TableName='TestTable',
        KeySchema=[{'AttributeName': 'SHA256', 'KeyType': 'HASH'}],
        ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
    )

    # Update item - does not work. "ValueError: too many values to unpack"
    dynamo_client.update_item(
        TableName='TestTable',
        Key={'SHA256': {'S': 'sha-of-file'}},
        UpdateExpression=(
            'SET MD5 = :md5,'
            'MyStringSet = :string_set,'
            'MyMap = :map'  # Maps string to StringSet of strings.
        ),
        ExpressionAttributeValues={
            ':md5': {'S': 'md5-of-file'},
            ':string_set': {'SS': ['string1', 'string2']},
            ':map': {'M': {'EntryKey': {'SS': ['thing1', 'thing2']}}}
        }
    )

I suspect that there is an error in the update parsing logic. Dynamo item updates can be really difficult to wrangle.

Stack Trace

Traceback (most recent call last):
  File "poc.py", line 35, in <module>
    update_does_not_work()
  File "/usr/local/lib/python2.7/site-packages/moto/core/models.py", line 71, in wrapper
    result = func(*args, **kwargs)
  File "poc.py", line 30, in update_does_not_work
    ':map': {'M': {'EntryKey': {'SS': ['thing1', 'thing2']}}}
  File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 530, in _make_api_call
    operation_model, request_dict)
  File "/usr/local/lib/python2.7/site-packages/botocore/endpoint.py", line 141, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python2.7/site-packages/botocore/endpoint.py", line 170, in _send_request
    success_response, exception):
  File "/usr/local/lib/python2.7/site-packages/botocore/endpoint.py", line 249, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/local/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python2.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
ValueError: too many values to unpack

Environment

I'm running Python 2.7 on Mac OS X. Moto is installed with pip:

$ pip show moto
Version: 0.4.31
Location: /usr/local/lib/python2.7/site-packages

[Juberstine]

Same issue FWIW. Only expression i'm using is:

UpdateExpression="SET crontab = list_append(crontab, :i)",
        ExpressionAttributeValues={
            ':i': crontab,
        },

[todor-kolev]

I had quite an awkward one:

table.update_item(
          Key={'entity': 'cycle', 'id': 'cycle-id-1'},
          UpdateExpression="SET nominations = :nominationIds",
          ExpressionAttributeValues={':nominationIds': nominationIds},
          ReturnValues="UPDATED_NEW"
        )

... and I see the response from DynamoDB which looks OK:

{
    "Attributes": {
        "nominations": {
            "SS": [
                "nomination-id-1",
                "nomination-id-2"
            ]
        }
    },
    "ResponseMetadata": {
        "RetryAttempts": 0,
        "HTTPStatusCode": 200,
        "RequestId": "3db8cae9-47e8-447b-89e4-7e35426e1a68",
        "HTTPHeaders": {
            "server": "Jetty(8.1.12.v20130726)",
            "content-length": "681",
            "x-amz-crc32": "2586038436",
            "x-amzn-requestid": "3db8cae9-47e8-447b-89e4-7e35426e1a68",
            "content-type": "application/x-amz-json-1.0"
        }
    }
}

but then when I query the table the type is completely different:

"nominations": {
                "M": {
                    "SS": {
                        "L": [
                            {
                                "S": "nomination-id-1"
                            }, 
                            {
                                "S": "nomination-id-2"
                            }
                        ]
                    }
                }
            }

print(nominations) # nominations ['nomination-id-1', 'nomination-id-2']