Aws-sdk-go: dynamodbattribute.MarshalMap marshalls NULL in a way that doesn't work with indexes

[jcoyne]

Version of AWS SDK for Go?

v1.12.70

Version of Go (go version)?

go1.9.3

What issue did you see?

When I use dynamodbattribute.MarshalMap to create a PutItemInput, then the NULL values of the struct are serialized as "source_id":{"NULL":true}. This NULL is not allowed for fields that are part of a secondary index. Instead the field should be omitted from the output map.

Steps to reproduce

Create a table with a seconday index:

aws dynamodb create-table --table-name resources  
        --attribute-definitions AttributeName=id,AttributeType=S 
           AttributeName=source_id,AttributeType=S
        --key-schema AttributeName=id,KeyType=HASH
        --provisioned-throughput=ReadCapacityUnits=5,WriteCapacityUnits=5
        --global-secondary-indexes "IndexName=ResourceBySourceID,\
  KeySchema=[{AttributeName=source_id,KeyType=HASH}],\
  Projection={ProjectionType=KEYS_ONLY},\
  ProvisionedThroughput={ReadCapacityUnits=5,WriteCapacityUnits=5}"

Create and serialize a struct:

type Resource struct {
    ID        string     `json:"id"`
    SourceID  string     `json:"source_id"`
}

resource := &Resource{ID: "123"}
row, _ := dynamodbattribute.MarshalMap(resource)
log.Printf("ROW: %s", row)

Output:

2018/02/26 09:18:10 ROW: map[source_id:{
  NULL: true
} id:{
  S: "123"
}]

When you attempt to persist this map, it errors out:

input := &dynamodb.PutItemInput{
    Item:      row,
    TableName: tableName,
}

_, err = h.db.PutItem(input)
if err != nil {
  panic(err)
}

ValidationException: Invalid attribute value type
    status code: 400, request id: 20f6e9ab-c048-476d-bebc-859cb7cf46e7

I have reproduced this at the AWS CLI level too: https://github.com/aws/aws-cli/issues/3135

But I believe there is a problem in this library in that the documentation says serializing to a map and making a PutItemInput from that is sufficient, when it may not be.

[outcoldman]

@jcoyne have you tried to use omitempty (see from docs)

type Resource struct {
    ID        string     `json:"id"`
    SourceID  string     `json:"source_id,omitempty"`
}

[jcoyne]

Yes, omitempty does fix if for an initial create