Ethers.js: Event listening efficiency [discussion]

[asselstine]

I want to discuss a fire that I had to put out today, because the discussion might be useful.

Our Infura usage hit 5.5 million requests yesterday. Yikes.

I started digging into the problem and discovered it was how we were using Ethers.js event filters. When PoolTogether loads, it sets up a few dozen event listeners so that it can detect changes and refresh data as needed:

provider.on(filter, callback)

However, for each filter Ethers calls eth_getLogs(filter) during each polling interval (4s by default). This struck me for two reasons:

1. Why poll for events more often than blocks? There are only new logs when there is a new block. If the polling interval instead matched the block intervals there would be ~ 4s / 14s = 28% the number of eth_getLogs calls. This might be an easy optimization.
2. It would be useful to provide a means to filter in-browser. Ethers.js could call eth_getLogs once for each block, then trigger any matching event filters as necessary. In fact, this is the custom solution I built and pushed out reduce the number of server calls. Less on the wire, more in memory.

I've switched over our event subscription code in our library Tightbeam, but I think it would be great to have client-side log filter matching code live in Ethers.

Would love to hear any thoughts.

[ricmoo]

I’ll look into this and provide a longer answer shortly (I’m out right now), but the getFilter, etc. Should only be triggering when the blockNumber changes: https://github.com/ethers-io/ethers.js/blob/master/src.ts/providers/base-provider.ts#L581

[ricmoo]

I've just put together a simple test with a lot of extra tracing added to the base-provider and it looks fine to me. The log queries are only sent when there is a block change.

Do you have any example and reproduction steps of the eth_getLogs being called more often than blocks changing?

[asselstine]

Thanks for looking into that! I'll revert to an older commit and see if I can extract the behaviour into a simple test case.

[asselstine]

After extracting the code, I found the issue: we were creating more than one Provider! Embarrassing but true.

Thanks for the quick response Richard! I've got to dig a little deeper next time.