Aws-sdk-js: Default delay time for DynamoDB retries inconsistent between code and docs

[kalpitad]

The docs indicate that the DynamoDB default retry time unit is 100ms, but as far as I can tell, the code seems to use 50ms. Is it 50ms or 100ms? I will accordingly set maxRetries and my Lambda function timeouts. Thanks!

Docs:
retryDelayOptions (map) — A set of options to configure the retry delay on retryable errors. Currently supported options are:

• base [Integer] — The base number of milliseconds to use in the exponential backoff for operation retries. _Defaults to 100 ms._
• customBackoff [function] — A custom function that accepts a retry count and returns the amount of time to delay in milliseconds. The base option will be ignored if this option is supplied.

Code:

retryDelays: function retryDelays(retryCount) {
    var delay = retryCount > 0 ? (50 * Math.pow(2, retryCount - 1)) : 0;
    return delay;
  }

[chrisradek]

The docs are unclear, but the main Config has a note that retryDelayOptions doesn't currently apply to DynamoDB. DynamoDB uses its own retry exponential back-off strategy, with a default retry count of 10.

To answer your question here's what the retry delays should be based on the current retry count:
[ 0, 50, 100, 200, 400, 800, 1600, 3200, 6400, 12800 ]

We're currently working on improving our docs, so we'll make a note to call out the retry strategies used by the SDK.

[kalpitad]

Thanks @chrisradek! What does it mean for the first delay to be 0ms? The retry occurs immediately? Or does it refer to the initial request?

[chrisradek]

When a request is sent but fails with a retry-able error, it calculates aretryDelay, then increments the retryCount. So the first retry will actually be 0ms, using the code you linked above. That's just for DynamoDB. For other services, the first retry will use a base value of 100ms (and add an exponential factor and random jitter).

[kalpitad]

Got it, retryCount is initially 0. So you actually get 2 requests back-to-back without a delay, assuming the 1st produces a retryable error, and then assuming that the 2nd request also produces a retryable error, the 3rd request will be delayed by 50ms. Thanks for the clarification!

[jppellerin]

@chrisradek is there a reason why no jitter is added to DynamoDB while it's there for every other service? We have a case of multiple concurrent requests. These all end up on the same retry interval, and thus end up creating a bottleneck.

Why is DynamoDB different when it comes to specifying a customBackoff algorithm (which would allow us to fix this issue)?

[chrisradek]

@jppellerin
Sorry for the delayed response. #1418 should address your concerns by adding jitter and the ability to change the base retry delay, or to specify a custom backoff function.

[jeskew]

Fixed in #1418.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.