Google-cloud-node: Should the datastore API support retries?

[chadbr]

From a slack conversation:

chadbr [4:40 PM]
I’d be interested to hear how people are handling retries on datastore with nodejs (edited)
Say I have something like this:
``` let datastore = await GCPUtils.getDatastore();

let key = datastore.key([DBConstant.UserKind, email]);
try {
  let data = await datastore.get({
    key: key
  });
  retVal = data as IUser;

} catch (error) {
  console.log(error);
}

```

if I get an error — because datastore is “busy” or whatever — I’m struggling to see a nice clean way to “retry” some number of times...
I could wrap the calling function - or embed the datastore get in a retry — or… I’m not sure :slightly_smiling_face:
any suggestions?

I’ll just talk to myself here :slightly_smiling_face:

my initial thought is to push most of the datastore specifics into utility functions — and allow retries on the utility functions...

justinbeckwith [10:57 PM]
@chadbr Eugh. This feels like something we should encapsulate in the SDK.

chadbr [11:00 PM]
That would be fantastic @justinbeckwith

justinbeckwith [11:00 PM]
Could you file a bug over at github.com/GoogleCloudPlatform/google-cloud-node?

chadbr [11:00 PM]
Absolutely

justinbeckwith [11:00 PM]
Send the link here and I'll + in a few folks.

@JustinBeckwith

[stephenplusplus]

We do exponential back off retries automatically for errors where retrying is recommended. Right now, you can provide { maxRetries: n } to the Datastore constructor to have some control over how many attempts are made before your error handler is executed.

What types of errors are you getting that you want to retry?

[chadbr]

ok -- I had no idea.

That is all I need!

Exponential backoff is gravy --

Thanks much!

[chadbr]

Are there any recommendations on a "max" number of retries?

[stephenplusplus]

The default value is 3. I don't know of a general best practice, I think it's just a combination of:

• how many API requests you're cool with making (assuming each has a cost)
• how long you can wait before it's just not worth it

We're transitioning some of our APIs to use a different transport layer (#1859), and this will open up much more fine-tuned control on the method level. It will also give you an option to not only set maxRetries, which caters to the first bullet point above, but also totalTimeoutMillis, which addresses the second.

[chadbr]

Outstanding. I'll keep an eye on the PR (hopefully we'll get a heads up in Slack or somewhere).

[AlecHFerguson]

Wow, very glad to find that the Node.js client does exponential backoff retries by default. I thought I might hafta implement my own 🚀.

Is this documented anywhere? I don't see any mention of this on the API reference. Would be good to publicize since it's a pretty powerful feature.
https://cloud.google.com/nodejs/docs/reference/datastore/3.0.x/Datastore