Lunie: polkadot governance api endpoints

[faboweb]

Part of #4456

• [x] governance overview

  • totalStakedAssets
  • totalVoters (all staked accounts)
  • treasurySize (treasure/communty pool size)
  • recentProposals (make adjustable pagesize, maybe extra endpoint)
  • topVoters (council members in Polkad and highst voting power in Cosmos)
  • links (title, link, type, icon)

• [x] ~add full text search to proposals endpoint~

• [x] ~make title, description, summary overwritable from db~
• [x] add polkadot proposals
  
  
  
  • add text proposals like in Cosmos
  
  
  • add treasury proposals via a templte for the description and summary
  
  
  • add parameter change proposals via a template
  
  
• [ ] add to proposals detailed votes
  
  
  
  • deposits
  
  
  • deposits sum
  
  
  • percentage of needed deposits
  
  
  • votes (add filter to only show for your validators)
  
  
  • votes sum
  
  
  • voting threshold yes
  
  
  • voting threshold no
  
  
  • links (from db, have title, link, icon, type)
  
  
  • timeline [{title, time}] (maybe put in network config)
  
  

[mariopino]

References:

• https://wiki.polkadot.network/docs/en/learn-governance#docsNav
• https://polkadot.network/a-walkthrough-of-polkadots-governance/

[mariopino]

Referendums:

const startTime = Date.now()
const referendums = await api.derive.democracy.referendums()
console.log(JSON.stringify(referendums, null, 2))
const endTime = Date.now()
console.log(`execution time: ${(endTime - startTime) / 1000}s`)

[
  {
    "image": {
      "at": 2973417,
      "balance": 383333180,
      "proposal": {
        "callIndex": "0x1d02",
        "args": {
          "id": "0x2b3325",
          "when": 3200000,
          "maybe_periodic": [
            3600,
            336
          ],
          "priority": 254,
          "call": {
            "callIndex": "0x060a",
            "args": {
              "additional": 2
            }
          }
        }
      },
      "proposer": "H9eSvWe34vQDJAWckeTHWSqSChRat8bgKHG39GC1fjvEm7y"
    },
    "imageHash": "0x6bee0d2c7ac152a72ce4b617a26eba078b19a4b5f54fbcda45f87330e60472f8",
    "index": 71,
    "status": {
      "end": 3225600,
      "proposalHash": "0x6bee0d2c7ac152a72ce4b617a26eba078b19a4b5f54fbcda45f87330e60472f8",
      "threshold": "Supermajorityapproval",
      "delay": 115200,
      "tally": {
        "ayes": "0x000000000000000000f6f6272d994c00",
        "nays": 0,
        "turnout": "0x000000000000000000cd4412db001400"
      }
    },
    "allAye": [
      {
        "accountId": "EGVQCe73TpFyAZx5uKfE1222XfkT3BSKozjgcqzLBnc5eYo",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 3663563000000000
      },
      {
        "accountId": "HHinQSdeqMDGdKjgjSxCiFGPNGNXn8YrXLouyh2GqNGJjha",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 10000000000000
      },
      {
        "accountId": "GTUi6r2LEsf71zEQDnBvBvKskQcWvK66KRqcRbdmcczaadr",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 68572000000000
      },
      {
        "accountId": "DfiSM1qqP11ECaekbA64L2ENcsWEpGk8df8wf1LAfV2sBd4",
        "isDelegating": false,
        "registry": {},
        "vote": "0x83",
        "balance": 4050000000000000
      },
      {
        "accountId": "HRJC7RRDjyPfsHTFyxQbE2DfwHcURSzoK1T92FDeNSv4Z62",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 1003601000000000
      },
      {
        "accountId": "H9eSvWe34vQDJAWckeTHWSqSChRat8bgKHG39GC1fjvEm7y",
        "isDelegating": false,
        "registry": {},
        "vote": "0x82",
        "balance": 1298000000000000
      },
      {
        "accountId": "Cthi7vhWHnVPNApbGAj48sMHuEtepCcpMyJsRzDoNbMYyP4",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 10000000000
      },
      {
        "accountId": "Fu2yt5ycYPQ244xhq5aEXuWCCkyU98bYaYuz3NpmPujr1Cd",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002137d087a64e00"
      },
      {
        "accountId": "DzAprbrJAMZcQLLNvZcnTWg81NnGYQPT28chidNHPiUfwPn",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 1610944000000000
      },
      {
        "accountId": "DmSEVAxkyr6ui6orAyMKMTkG4R6vGXnPv4kVyzSQ3Fnv8cv",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 311402000000000
      },
      {
        "accountId": "HcADkaPsKcnMHwabYFEcLreAJCJH5yDssY78rFvRYRXH3JV",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 136200000000000
      },
      {
        "accountId": "DmAEBK4iCZH4eXiPxDHQTwJcaVoQHBBxERBqwtyPPwnrFAu",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x000000000000000000252c8ece838c00"
      },
      {
        "accountId": "GwG5QFvexhf7vuwRstUzWvVaLHAWTQ9KmMkiGT2mCHHbBkh",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 958851000000000
      },
      {
        "accountId": "DSNBPrfLNw3o6sBJb78e7Qr8tUtiyTt1eUHaMiHBL2Ef6x3",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002533bede058200"
      },
      {
        "accountId": "J9nD3s7zssCX7bion1xctAF6xcVexcpy2uwy4jTm9JL8yuK",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 2150000000000000
      },
      {
        "accountId": "HfBo7ptrcKN53tsLcMEQT6QEU4wfMi2VsbaCGkWQqUJhDNY",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 1000000000
      },
      {
        "accountId": "HpuznVjBNEqddoCk3kwFHUBMeFbGuFuLiaHpRR1FWRz2sGv",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 1000000000
      },
      {
        "accountId": "GLVeryFRbg5hEKvQZcAnLvXZEXhiYaBjzSDwrXBXrfPF7wj",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 100000000000000
      },
      {
        "accountId": "D2ukdU9o9BxoVLcoHivkk5zucz5UYD13Ny2447vQoFLYT9z",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 4299000000000
      },
      {
        "accountId": "DotZjX3yrnoxnt7f9oqpz4Qz2Ww34qracFND3s8eB5o4PsT",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002138d422d41c00"
      },
      {
        "accountId": "DSgv592n6kqjVwVBYFRUUb6SbB8cWtnS1MmB5skS7TnYJES",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 15000000000000
      },
      {
        "accountId": "DdgLwYftffTJxeQ1qvMHTtYYDTDTMKEka1H2rvrdTEM2KVm",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 360538000000000
      },
      {
        "accountId": "GnL41mJJJTp9npr1W9wLMzEfFL4thqyjPt6hwxLirmJt5vQ",
        "balance": 2400000000000000,
        "isDelegating": true,
        "vote": "0x82"
      }
    ],
    "allNay": [],
    "voteCount": 23,
    "voteCountAye": 23,
    "voteCountNay": 0,
    "votedAye": "0x000000000000000000f6f6272d994c00",
    "votedNay": 0,
    "votedTotal": "0x000000000000000000cd4412db001400",
    "isPassing": true,
    "votes": [
      {
        "accountId": "EGVQCe73TpFyAZx5uKfE1222XfkT3BSKozjgcqzLBnc5eYo",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 3663563000000000
      },
      {
        "accountId": "HHinQSdeqMDGdKjgjSxCiFGPNGNXn8YrXLouyh2GqNGJjha",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 10000000000000
      },
      {
        "accountId": "GTUi6r2LEsf71zEQDnBvBvKskQcWvK66KRqcRbdmcczaadr",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 68572000000000
      },
      {
        "accountId": "DfiSM1qqP11ECaekbA64L2ENcsWEpGk8df8wf1LAfV2sBd4",
        "isDelegating": false,
        "registry": {},
        "vote": "0x83",
        "balance": 4050000000000000
      },
      {
        "accountId": "HRJC7RRDjyPfsHTFyxQbE2DfwHcURSzoK1T92FDeNSv4Z62",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 1003601000000000
      },
      {
        "accountId": "H9eSvWe34vQDJAWckeTHWSqSChRat8bgKHG39GC1fjvEm7y",
        "isDelegating": false,
        "registry": {},
        "vote": "0x82",
        "balance": 1298000000000000
      },
      {
        "accountId": "Cthi7vhWHnVPNApbGAj48sMHuEtepCcpMyJsRzDoNbMYyP4",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 10000000000
      },
      {
        "accountId": "Fu2yt5ycYPQ244xhq5aEXuWCCkyU98bYaYuz3NpmPujr1Cd",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002137d087a64e00"
      },
      {
        "accountId": "DzAprbrJAMZcQLLNvZcnTWg81NnGYQPT28chidNHPiUfwPn",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 1610944000000000
      },
      {
        "accountId": "DmSEVAxkyr6ui6orAyMKMTkG4R6vGXnPv4kVyzSQ3Fnv8cv",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 311402000000000
      },
      {
        "accountId": "HcADkaPsKcnMHwabYFEcLreAJCJH5yDssY78rFvRYRXH3JV",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 136200000000000
      },
      {
        "accountId": "DmAEBK4iCZH4eXiPxDHQTwJcaVoQHBBxERBqwtyPPwnrFAu",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x000000000000000000252c8ece838c00"
      },
      {
        "accountId": "GwG5QFvexhf7vuwRstUzWvVaLHAWTQ9KmMkiGT2mCHHbBkh",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 958851000000000
      },
      {
        "accountId": "DSNBPrfLNw3o6sBJb78e7Qr8tUtiyTt1eUHaMiHBL2Ef6x3",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002533bede058200"
      },
      {
        "accountId": "J9nD3s7zssCX7bion1xctAF6xcVexcpy2uwy4jTm9JL8yuK",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 2150000000000000
      },
      {
        "accountId": "HfBo7ptrcKN53tsLcMEQT6QEU4wfMi2VsbaCGkWQqUJhDNY",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 1000000000
      },
      {
        "accountId": "HpuznVjBNEqddoCk3kwFHUBMeFbGuFuLiaHpRR1FWRz2sGv",
        "isDelegating": false,
        "registry": {},
        "vote": "0x80",
        "balance": 1000000000
      },
      {
        "accountId": "GLVeryFRbg5hEKvQZcAnLvXZEXhiYaBjzSDwrXBXrfPF7wj",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 100000000000000
      },
      {
        "accountId": "D2ukdU9o9BxoVLcoHivkk5zucz5UYD13Ny2447vQoFLYT9z",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 4299000000000
      },
      {
        "accountId": "DotZjX3yrnoxnt7f9oqpz4Qz2Ww34qracFND3s8eB5o4PsT",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": "0x0000000000000000002138d422d41c00"
      },
      {
        "accountId": "DSgv592n6kqjVwVBYFRUUb6SbB8cWtnS1MmB5skS7TnYJES",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 15000000000000
      },
      {
        "accountId": "DdgLwYftffTJxeQ1qvMHTtYYDTDTMKEka1H2rvrdTEM2KVm",
        "isDelegating": false,
        "registry": {},
        "vote": "0x81",
        "balance": 360538000000000
      },
      {
        "accountId": "GnL41mJJJTp9npr1W9wLMzEfFL4thqyjPt6hwxLirmJt5vQ",
        "balance": 2400000000000000,
        "isDelegating": true,
        "vote": "0x82"
      }
    ]
  }
]
execution time: 3.779s

[Bitcoinera]

Do we care here about past proposals or only about active ones? It seems only active ones appear using the api.derive.democracy thing.

Guess we would need to store past ones in DB, but it would be cool because right now old ones are not displayed in polkadot.js

[Bitcoinera]

Also it seems Westend is not a software clone of Polkadot. Look what happens when querying the same api.derive.democracy:

Great for testing :facepalm:

[faboweb]

Where does the error actually happen?

[faboweb]

Let's show only active proposals.for now. Storing historic ones seems to much effort.
I would like to have that as well but we need to write a crawler that processes the whole chain for that.

[Bitcoinera]

Where does the error actually happen?

there is not api.derive.democracy in Westend it seems :shrug:

[mariopino]

Where does the error actually happen?

there is not api.derive.democracy in Westend it seems 🤷

Hey @Bitcoinera there's no governance at all in Westend, coool isn't? :-)

[faboweb]

So let's disable it for Westend and let's write them

[Bitcoinera]

Hey @Bitcoinera there's no governance at all in Westend, coool isn't? :-)

Yes, great for testing

I like paying to test opening up proposals in a "canary" net where the token is 10$ worth

[Bitcoinera]

Looking at the Democracy proposal, some questions:

1. Not sure how to get the Endtime from here? I only see a hash in seconds? Is this a hex? (doesn't look like) Guess will have to play with it:

1. What is the actual deposit for this proposal. In polkadot.js they say it is 1 DOT, taking the first balance, bu in Subscan they claim it is 0.0004 DOT, taking it from the image balance. What is the image actually?

Really strange. Should I rely on polkadot.js or on Subscan?

Polkadot.js

Subscan

1. I just realized: what is this mysterious "Noted" status appearing in Subscan?

[faboweb]

seconds are addresses. Those are "I support this proposal".

[faboweb]

I guess you get the endtime applying an onchain parameter you have to query for

[Bitcoinera]

I got a bit stuck getting the description for proposals and started working on govervanceOverview too.

This here, the recentProposals, should there be an optional parameter to select the limit of recentness? Which should be the default? Last week proposals, or more like the 3 most recent proposals?

[Bitcoinera]

to calculate topVoters, is it ok to assume that it is going to be all validators?

Logically they will have more voting power than the rest. So I'd just need a reduce to compare them and calculate the ones with most votingPower

[faboweb]

recentProposals: (make adjustable pagesize, maybe extra endpoint) [in the description on the top of this issue]
-> just use the actual proposals endpoint

topVoters: (council members in Polkadot and highst voting power in Cosmos) [in the description on the top of this issue]

[Bitcoinera]

topVoters: (council members in Polkadot and highst voting power in Cosmos) [in the description on the top of this issue]

OK, that one I only had to read it actually xD

and then the other one just use getAllProposals. Alright

[Bitcoinera]

About this one :point_right: add to proposals detailed votes

Should these details be included to the Proposal type? Like it could be included as a subtype? Like this:

type Proposal {
  title
  description
  ...
  detailedVotes {
    deposits
    depositsSum
...
  }
}

[Bitcoinera]

percentage of needed deposits

I guess this one would only appear when on deposit period, and during voting period would be undefined

Am I right?

[Bitcoinera]

In the case of Cosmos, the voting threshold yes is simply the tallying parameter you get from the /gov/parameters/tallying endpoint, no?

[Bitcoinera]

In that case, I would say we should include other two fields to the detailedVotes response:
votingPercentageYes
votingPercentageNo

[faboweb]

About this one 👉 add to proposals detailed votes
Should these details be included to the Proposal type?

yep

[faboweb]

percentage of needed deposits
I guess this one would only appear when on deposit period, and during voting period would be undefined

Can't we get the deposits afterwards? but yes it is good simplification

[faboweb]

In the case of Cosmos, the voting threshold yes is simply the tallying parameter you get from the /gov/parameters/tallying endpoint, no?

yep

In that case, I would say we should include other two fields to the detailedVotes response:
votingPercentageYes
votingPercentageNo

good idea

[Bitcoinera]

Can't we get the deposits afterwards? but yes it is good simplification

apparently not... or not for all proposals in Cosmos at least. Idk why, just saw it is like that :shrug:

[Bitcoinera]

4507 was actually just one check from this issue :sweat_smile: