Go-ethereum: Very high difficulty in a private chain

[wawrzek]

System information

Geth version: 1.4.11 and 1.4.10
OS & Version: Linux (Ubuntu 16.04 LTS0

I have few private/consortium test network. Recently I noticed that it's extremely hard to mine in them. eth.getBlock('pending', true) returns difficulty: 17171480576, even when network was initialized with "difficulty": "0x0200".

That might be recent regression, because I don't have the same problem in older network (geth 1.4.9). The old network was using --genesis option and the new doesn't.

[joeb000]

In geth the difficulty adjusts itself each block in order to keep the average block time ~13 seconds. The more hashing power of miners in your private network, the higher the difficulty will be. If you inspect the early blocks in your private chain, you'll notice the difficulty gradually increasing from 0x200 to where it is now. See this stackexchange discussion for details: http://ethereum.stackexchange.com/questions/1880/how-is-the-mining-difficulty-calculated-on-ethereum

Here is where the calculation is done: https://github.com/ethereum/go-ethereum/blob/master/core/block_validator.go (line 265)

Using the --genesis flag was deprecated and phased out - instead you should be using the init command to initialize a new blockchain with a custom genesis block. Here's an example:

geth --datadir /path/to/your/datadir --networkid 100 init /path/to/your/custom_genesis.json

[wawrzek]

@joeb000 thanks for comments.
The problem is the the value I posted are for block 1 - somehow difficulty increased from 1024 to 17171480576.
We've been using init instead of genesis file as describe in documentation for some time. However, I not sure if that works as expected. I see small different between your example and my command (we skipped networkid during initialization). Going to check it.

[joeb000]

Yes, I would highly recommend using a different network ID than the default main net (1) and the testnet (2). This way theres no chance that your node attempts to connect with peers on the main chain and sync with that.

Take a look at the difficulty in block 2 of the main ethereum blockchain: https://etherscan.io/block/2 - its very close to the number you gave above...

[wawrzek]

@joeb000 we use network id, just not in init step. I fixed that - no change in difficulty.

[kevinmartinmauro]

Are you connected to any peers outside your private network? Check using admin in the console

[wawrzek]

@kevinmartinmauro - I'm connected only to 'private network members':

net.peerCount 
5

Over the weekend chained moved a bit - now it's on block 4.

eth.getBlock('pending', true)
{
  difficulty: 17146339321,
  extraData: "0xd78301040b844765746887676f312e362e32856c696e7578",
  gasLimit: 5012,
  gasUsed: 0,
  hash: null,
  logsBloom: null,
  miner: null,
  nonce: null,
  number: 4,
  parentHash: "0x71b739956d23687f239e553057085d2f0b4e384f154073c03d496e3934e4b18e",
  receiptRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  sha3Uncles: "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
  size: 536,
  stateRoot: "0xe84ac60849da82feaf4d5d11ddeb0174722f3f0a72c0acf9da3d7b89682ec6cc",
  timestamp: 1474286645,
  totalDifficulty: 0,
  transactions: [],
  transactionsRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  uncles: []
}

So it's 25141255 lower than at beginning.

[bas-vk]

There is no need to supply the network id during initialization. What is odd, block 4 in the main chain has the exact same difficulty as your block 4.

Could you post the exact commands that you used to create this chain and the genesis file that you used?
Also the output of eth.getBlock(n) with n in [0,3].

[wawrzek]

@bas-vk interesting finding. I automated the whole process with Ansible, so let me double check the actual content of the init command (maybe some variables are not properly pass between the playbook and the role).

The init command in the script is:

 geth --datadir /opt/geth/chain/ --networkid {{ network_id }} init /opt/geth/init.json

where _network_id_ is ansible variable.

That's the genesis file I use (at least I hope my script uses ;).

{
  "alloc": { },
  "nonce": "0x0000000000000042",
  "difficulty": "0x00400",
  "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "coinbase": "0x0000000000000000000000000000000000000000",
  "timestamp": "0x00",
  "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "extraData": "0x11bbe8db4e347b4e8c937c1c8370e4b5ed33adb3db69cbdb7a38e1e50b1b82fa",
  "gasLimit": "0x4c4b40"
}

[wawrzek]

@bas-vk Thanks for you comment. It made me suspicious that geth using a default value. I found a bug in my Ansible script quietly preventing init from run.

On the geth side:

• I don't like that it quietly uses some hard-coded values (main net genesis.json).
• Lack of clear documentation what values are default.
• Lack of clear communication about potential problem (et least warning in logs).

[drouillard]

In case anyone else comes across this in a google search -- when you first fire up your console run

admin.nodeInfo

The difficulty should be what you expect (1024) , not a number like 17179869184. What could be happening is that you are using variables in the arguments that are being passed and geth is not finding the genesis block at the specified location. As a result it is creating another instance, but this time using the hardcoded defaults. I was running Ubuntu 16.04 on Digital Ocean and

geth --datadir "$etherum_home/chain" console 2> console.log

after running init in the location specified by $ethereum_home. This command was actually using /chain/ (off the root path) instead of what I expected (where I had set the env variable)

---

Sidenote - if you are on a service like Digital Ocean and are seeing very high block times try upgrading your plan. Blocktimes should be no more than 20 seconds when starting out. If there are taking a few minutes it likely means your configuration is under-powered and something is not working under the hood.