Google-cloud-go: datastore: Allow inserts not just upserts

[lc-chrisbarton]

We're processing orders using datastore and pubsub and some times we have an order followed by a cancellation and sometimes we receive duplicate orders from pubsub. We would like to ensure the cancel fails if we perform a put and the order doesn't exist and duplicated orders are rejected. So basically we want create and update not just upsert. The node.js client has this feature. We work around this using a transaction with a get on the order id but I guess it would be faster if the Go client exposed this functionality?

[apstndb]

872

[piotrkowalczuk]

I'm facing serious contention problem because I am not able to do simple insert. Instead of that I have to perform a transaction. I know Key.ID upfront so Put method does Upsert instead of Insert mutation. Being able to control this behavior outside of Put would solve my problem.

Here is small API change proposal that address this issue: https://code-review.googlesource.com/#/c/gocloud/+/22970/

[jba]

@zombiezen

We do want to support this, and it is not a lot of work. The problem with the proposal in the CL is that it changes the signature of Put, and we can't do that. An obvious alternative is separate Insert and Update methods, but we'd need eight new methods (two each for Client.Put, Client.PutMulti, Transaction.Put and Transaction.PutMulti). Is there a nicer way?

[zombiezen]

(Handwavy and untested.) You could introduce something like:

// Update wraps an argument to Put and causes the operation to fail if the entity does not exist.
type Update struct {
  Src interface{}
}

A user would use this like:

myOrder := Order{Canceled: true}
datastore.Put(datastore.IDKey("Order", 42, nil), datastore.Update{Src: myOrder})

Seems like that would work without breaking compatibility.

[lc-chrisbarton]

@zombiezen I like that approach :)

[apstndb]

I'm worried about complicating Put(and PutMulti) method.
I prefer to introduce generic mutation because insert, upsert(a.k.a put), update and delete are type of mutation in Cloud Datastore API.
https://cloud.google.com/datastore/docs/reference/data/rest/v1/projects/commit?hl=en#request-body

So we may be able to batch them in single method.

client.MutateMulti(ctx context.Context, keys, []datastore.Mutation{datastore.Update{...}, datastore.Upsert{...}, datastore.Delete{...}}) ([]*Key, error)

or fluent style(like firestore)

mutations := datastore.Mutations().
    Update(key1, val1).
    Upsert(key2, val2)
client.Batch(mutations)

[lc-chrisbarton]

@apstndb well I would just have create(), read(), update(), delete(), upsert() since that is simple and obvious and the user can choose what they want

[jba]

I went with the mutation idea. It is the most general. Please take a look at https://code-review.googlesource.com/#/c/gocloud/+/23170.

[jba]

CL submitted.