Cardano-node: [QUESTION] - Is ledger-state data inaccurate?

[bobdobs]

External

Summary
I run a pool and dump+parse the ledger-state every 5 minutes to get the current live stake and all delegators to my pool. The strange thing is that the numbers I get from ledger-state don't match what I see on pooltool.io or adapools.org. The number of delegators and live stake is always higher in the data from ledger-state. I think that pooltool.io or adapools.org use db-sync to get this data, but I always assumed that ledger-state would be more accurate since it comes straight from the node. Now I am not so sure: Right after epoch transition the ledger-state data seems to match pooltools data for live stake and total number of delegators better. So I am thinking that the cardano-node might be consolidating the ledger-state after each epoch transition?

Steps to reproduce
Run a pool and dump ledger-state to get:

1. Total number of delegators for a pool
2. Sum delegated ADA for each delegator to get total live stake

System info (please complete the following information):

• OS: Ubuntu server
• Version: 20.04
• Node version: v1.9.0

[nc6]

Ultimately these data all come from the ledger state. However, there may be issues of timing - delegations do not immediately come into effect in terms of a pool's stake, so it could be that the pooltool is only updating this when it comes into effect at the epoch boundary.

[kevinhammond]

The ledger state could also be temporarily on a fork, but then you would see it rolling back, I expect. I've pointed @JaredCorduan at this so that he's aware and can check, but the most likely situation is that your ledger state is correct, as Nick says. Pooltool gets its information from dbsync. This records some chain data as it is produced (e.g. transactions), and periodically scrapes some additional information from the ledger state (e.g. at epoch boundary transitions). That could explain the difference in what you're seeing.

[JaredCorduan]

yes, exactly what Nick said, the ledger state will be the most accurate way to get the delegated stake. I do not know what pooltool is doing, sorry.

[papacarp]

here is what pooltool does for live stake:

1. extract all addresses for each pool from: ledger['esLState']['_delegationState']['_dstate']['_delegations']
   2. extract all owner addresses for each pool from: ledger['esLState']['_delegationState']['_pstate']['_pParams']
   3. extract all future owner addresses for each pool from: ledger['esLState']['_delegationState']['_pstate']['_fPParams']
2. extract all utxo values for each address from: ledger['esLState']['_utxoState']['_utxo']
3. extract all reward values for each address from: ledger['esLState']['_delegationState']['_dstate']

sum up all utxo + reward for each address and each pool

We do this about once a minute

[JaredCorduan]

@papacarp you actually don't need to get any addresses from _pParams or _fPParams, since these address will have to be delegated (ie in _delegations) for it to count as live stake anyway. also, counting both _pParams and _fPParams is a bit strange anyway, since it could be completely removing old reward owner addresses for new ones, etc. but none of this would explain why the ledger dump is giving a bigger number. is pooltools handling pointer addresses?

@bobdobs when you say "parse the ledger-state", is this a script you wrote?

it sounds like @bobdobs and pooltools are both parsing the same data, and just getting different results.

[papacarp]

@papacarp you actually don't need to get any addresses from _pParams or _fPParams, since these address will have to be delegated (ie in _delegations) for it to count as live stake anyway.

Ok, Its been awhile since I coded that in. Owner addresses are only extracted so we can verify pledge. the code doesn't do anything else with them when we sum up the total stake.

is pooltools handling pointer addresses?

The staking address is most conveniently matched up to the latter half of the utxo address. Is there anything else I need to be aware of?

[bobdobs]

@papacarp you actually don't need to get any addresses from _pParams or _fPParams, since these address will have to be delegated (ie in _delegations) for it to count as live stake anyway. also, counting both _pParams and _fPParams is a bit strange anyway, since it could be completely removing old reward owner addresses for new ones, etc. but none of this would explain why the ledger dump is giving a bigger number. is pooltools handling pointer addresses?

@bobdobs when you say "parse the ledger-state", is this a script you wrote?

it sounds like @bobdobs and pooltools are both parsing the same data, and just getting different results.

Yes I do parse ledger-state.json with a script (there really no other way to go about it as far as I can tell). PapaCarp does something similar. We compared results, and it is clear that I have an error in my script. Once I have fixed that I think my numbers will start to make sense. Sorry for the inconvenience.

[JaredCorduan]

The staking address is most conveniently matched up to the latter half of the utxo address. Is there anything else I need to be aware of?

I was just trying to guess at what might be causing trouble. Pointer addresses cannot be compared directly to the delegation mapping, since the stake reference in a pointer address is a pointer to the stake registration certificate and has to be looked up in the pointer mapping.