Web3.js: local HttpProvider gets stuck after a few hundred calls (Node.js)

[flockonus]

I'm running:

Node v8.2.1
web3 0.20.1
Geth/v1.7.1-stable-05101641/darwin-amd64/go1.9.1 - with: geth --rinkeby --cache=1024 --datadir ~/Library/Ethereum/rinkeby/ --fast --rpc --rpcapi="db,eth,net,web3,personal,web3"  --ws

The code is rather simple, we've got a contract running in Rinkeby and then i try reading about 2000 values from there, but it gets stuck after about 460 calls (it varies a bit). I notice the process' CPU get stuck at 100% until i kill the process.

const Web3 = require("web3");

const nodeAddress = "http://localhost:8545";

const abi = [
  {
    constant: true,
    inputs: [],
    name: "kittyCount",
    outputs: [
      {
        name: "",
        type: "uint256"
      }
    ],
    payable: false,
    type: "function"
  },
  {
    constant: true,
    inputs: [
      {
        name: "",
        type: "uint256"
      }
    ],
    name: "kittyIndexToOwner",
    outputs: [
      {
        name: "",
        type: "address"
      }
    ],
    payable: false,
    type: "function"
  }
];

console.log("-- creating new instance");
const web3 = new Web3();
// require a local full node to be running
web3.setProvider(new web3.providers.HttpProvider(nodeAddress));
const Contract = web3.eth.contract(abi);
const instance = Contract.at("0xc211d5b50f0d2ea2926b428af58117942a21c724");

async function run() {
  const n = instance.kittyCount();
  console.log("kittens:", n.toNumber());
  // get who ows who
  const kittyToOwner = new Array(n).fill(0);
  for (let i = 1; i <= n; i++) {
    kittyToOwner[i] = await instance.kittyIndexToOwner(i);
    console.log("owner..", i);
  }
  process.exit(0);
}

run().catch(console.error);

[frozeman]

This may have to do with the http processes the os allows. Not sure how to solve this. Can you google around a bit, if you find a solution we can use?

[flockonus]

@frozeman I did google a bit but found no meaningful solution online (not even the same problem)

What solved it for me was to switch using the less documented IPC provider. It is extremely fast relative to synchronous http, but on the other hand it only takes async.

const nodeAddress = process.env.HOME + "/Library/Ethereum/rinkeby/geth.ipc";

const Web3 = require("web3");
const net = require("net");

const web3 = new Web3();
web3.setProvider(new Web3.providers.IpcProvider(nodeAddress, net)); 
// ...

The only "issue" with this is that the method calls then can only be used with callbacks, so i'm using a sort of helper like this:

async function callAsync(method, ...args) {
  return new Promise(function(resolve, reject) {
    instance[method](...args, (err, res) => {
      if (err) reject(err);
      else resolve(res);
    });
  });
}

UPDATE: I switched to http async methods (passing a callback) and that did solve the hanging problem, same as IPC provider.

[ericmaino]

This is still a problem. I'll try to submit a pull request this weekend the fixes it.

[krisnadiputra]

I am currently facing the same problem... and needed to change to IPC provider and async functions also....

[nivida]

Version 0.20.x got his last maintenance release with v0.20.7. Please update your code to the latest 1.0 version of Web3.js. Further details about the current project state are explained in the release announcement of version 1.0.0-beta.38.