Steem: Negative latencies

[Jolly-Pirate]

HF20 in production reports negative latencies! Also, some latencies are very small, too good to be true.

Got 42 transactions on block 25895869 by ausbitbank -- latency: -4 ms
Got 36 transactions on block 25896388 by ausbitbank -- latency: -264 ms
Got 20 transactions on block 25896389 by drakos -- latency: -35 ms
Got 29 transactions on block 25896477 by ausbitbank -- latency: -11 ms
Got 21 transactions on block 25896483 by drakos -- latency: -9 ms

[mvandeberg]

We made some changes to the block generation schedule logic to reduce the likelihood of missing production during higher latency scenarios. Part of this was expanding the block generation window and increasing the wake frequency of the production loop. Block production has a short look-ahead that allows it to produce early if it is scheduled next (this already existed), however we did not change a constant there that should have been changed.

When all witnesses are running 0.20.0 we will have effectively shifted block production 100-200ms earlier than what it currently is but with the same block interval. The only potential harm is if a 0.19.X witness produces before a 0.20.0 witness they will have 100-200ms less latency buffer to not get skipped. This is no worse than missing a block currently due to a latency spike and there is no danger running the code as is.

[mvandeberg]

We've decided to leave the behavior as is and change the message wording from "latency" to "block time offset". The logic in steemd allows for blocks to be received a little early. "latency" is a bit of a misnomer anyway because you are attempting to measure timing differences between two machines. As we've seen with other "latency" issues, when a node does not have ntp configured properly, "latency" issues pop up. The block is being received early/late, but latency does not actually represent the latency between the nodes.