Bitshares-core: Change replay percentage to total block size processed

[abitmore]

User Story
As a node admin, during replaying, I want to know percentage of total size of blocks that have been processed, rather than count of blocks, so that I can estimate more accurately when it will complete.

CORE TEAM TASK LIST

• [ ] Evaluate / Prioritize Feature Request
• [ ] Refine User Stories / Requirements
• [ ] Define Test Cases
  
  
  
  • [x] Assigned: @cogutvalera
  
  
  • [x] Estimated: 2 hours
  
  
  • [x] Remitted: Week 39
  
  
• [ ] Design / Develop Solution
• [ ] Perform QA/Testing
• [ ] Update Documentation

[oxarbitrage]

i don't think block % to total will allow you to estimate time. count or % of operations will do better.

[abitmore]

We don't have the number of total operations before replay. For this issue I mean we can use total_processes_block_size / total_block_size which should have been obtained from block log already when replaying.

[bangzi1001]

Is that possible to add "Estimate Completion Time" beside the Percentage? If can, then node admin no need to estimate completion time manually.

[pmconrad]

For this issue I mean we can use total_processes_block_size / total_block_size which should have been obtained from block log already when replaying.

I believe using the file size of the block log is an inaccurate measure of the total block size to apply. The reason is that the block log is only ever appended to. For example, when switching between forks, the blocks that are re-applied will be appended to the block log again. Similarly, the last 50 blocks of a replay are appended again.

I believe the error is small for a typical node, but can be significant for a node that is restarted often.

[abitmore]

the last 50 blocks of a replay are appended again.

This doesn't sound ideal.

I believe the error is small for a typical node, but can be significant for a node that is restarted often.

I guess it won't be too significant, since we don't reapply too old blocks anyway. IMHO, even it's not accurate, it's still better for estimation than block counts.

[cogutvalera]

@ryanRfox assign this issue to me please so I will continue my researchers after #946 issue.

Thanks !

[ryanRfox]

Assigned @cogutvalera and awaiting his estimate for @bitshares/core-dev review. Thanks

[cogutvalera]

@ryanRfox my estimation for this issue is approximately 2 hours

[cogutvalera]

PR is ready #1335

Thanks !

[pmconrad]

Resolved in #1335

[ryanRfox]

I understand this is a closed Issue, so perhaps I need to open a separate one. I feel the output could be improved. Currently is looks like this:

1944127ms th_a       db_management.cpp:103         reindex              ]    [by size: 2.37002290685776762%   821428243 of 34659084544]   [by num: 17.34458437871986192%   5430000 of 31306602]

Is it trivial to make it look something like:

1944127ms th_a       db_management.cpp:103         reindex              ]    [by size: 2.37%   821.42M of 34.66G]   [by blocks: 17.345%   5.43M of 31.31M]

Round to two decimal places and convert to K, M, G notation.

[abitmore]

I more like accurate numbers, at least for blocks ;) I agree that it's better if the decimal places of percentages are fewer.