Ethers.js: RetryProvider for unreliable back-ends?

[mrwillis]

This is more of a feature request, but I'm wondering if it would be possible to add an optional retry globally on a provider with some strategy if the response is not an HTTP 200?

I find myself wrapping individual calls with try catch and checking the error code to retry again. This could be useful for unreliable nodes and nodes that rate limit you.

What do you think?

[ricmoo]

This can be accomplished by creating a sub-class of BaseProvider. I will include this as a cookbook recipe (although, probably in TypeScript). In v5 it will also be easier to make these as separate packages.

Here is a quick example though:

const inherits = require("util").inherits;

const ethers = require("ethers");

// Poll is a simple utility in ethers which handles exponential back-off for you, so you
// don't accidentally hammer the server; there are plenty of options in it you could adjust
const poll = require("ethers/utils/web").poll;

function RetryProvider(provider, retryCount) {
    ethers.errors.checkNew(this, RetryProvider);
    ethers.providers.BaseProvider.call(this, provider.getNetwork());
    ethers.utils.defineReadOnly(this, 'provider', provider);
    ethers.utils.defineReadOnly(this, 'retryCount', retryCount || 5);
}
inherits(RetryProvider, ethers.providers.BaseProvider);

RetryProvider.prototype.perform = function(method, params) {
    let attempts = 0;
    return poll(() => {
        attempts++;
        return this.provider.perform(method, params).then((result) => {
            return result;
        }, (error) => {
            if (attempts >= this.retryCount) {
                return Promise.reject(error);
            }
            // Returning undefined to poll tells it to try again
            return undefined;
        });
    });
}

Using this provider:

let wrappedProvider = ethers.getDefaultProvider();
let provider = new RetryProvider(wrappedProvider);

provider.getBlockNumber().then((bn) => {
    console.log(bn);
});

Make sense? I need to extend the documentation for how to sub-class providers, and this is a good example to include.

[mrwillis]

Thanks @ricmoo I will give it a shot. Makes sense. I will share a working typescript implementation

[mrwillis]

Here is a TypeScript version that works for me

import { JsonRpcProvider } from "ethers/providers";
import { Networkish } from "ethers/utils";
import { ConnectionInfo, poll } from "ethers/utils/web";

export class RetryProvider extends JsonRpcProvider {
  public attempts: number;

  constructor(
    attempts: number,
    url?: ConnectionInfo | string,
    network?: Networkish
  ) {
    super(url, network);
    this.attempts = attempts;
  }

  public perform(method: string, params: any) {
    let attempts = 0;
    return poll(() => {
      attempts++;
      return super.perform(method, params).then(
        result => {
          return result;
        },
        (error: any) => {
          if (error.statusCode !== 429 || attempts >= this.attempts) {
            return Promise.reject(error);
          } else {
            return Promise.resolve(undefined);
          }
        }
      );
    });
  }
}

[ricmoo]

What backend are you using that is return 429? This indicates you are indeed being rate-limited, in which case, you should probably add an additional delay, perhaps?

[mrwillis]

Yep. Using AlchemyAPI and the rate limits are 1. concurrent and 2. by the second . They are quite high but my application can be burst-y. So far they resolve quite quickly (max of 2 retries) and it's working quite well. I should probably extend this to 5XX as well now that I think of it.

[ricmoo]

Closing this now. If you feel this shouldn't be closed though, please feel free to re-open. I have added an example to the experimental package in v5, which will be in beta soon.

Thanks! :)