Lighthouse: Implement the standard HTTP for BN

[paulhauner]

Description

This issue tracks the implementation of the standard Eth2 API at https://github.com/ethereum/eth2.0-APIs.

Note: this issue is assuming that #1569 has already been merged into master.

Overall status

The standard is mostly implemented in #1569, however there are still some endpoints missing. Most notably, SSE endpoints have not been implemented.

Per-Endpoint

This section documents the progress of each API endpoint. An endpoint should only be marked as complete if it is 100% complete, with nothing else to do.

Beacon

• [x] /eth/v1/beacon/genesis
• [x] /eth/v1/beacon/states/{state_id}/root
• [x] /eth/v1/beacon/states/{state_id}/fork
• [x] /eth/v1/beacon/states/{state_id}/finality_checkpoints
• [ ] /eth/v1/beacon/states/{state_id}/validators
  
  
  
  • Implemented, but returning non-standard status field.
  
  
  • Here is my not-yet-submitted proposal to change the field: https://hackmd.io/bQxMDRt1RbS1TLno8K4NPg
  
  
• [ ] /eth/v1/beacon/states/{state_id}/validators/{validator_id}
  
  
  
  • As above.
  
  
• [ ] /eth/v1/beacon/states/{state_id}/validator_balances
• [x] /eth/v1/beacon/states/{state_id}/committees/{epoch}
• [x] /eth/v1/beacon/headers
  
  
  
  • Only returns canonical blocks
  
  
• [x] /eth/v1/beacon/headers/{block_id}
• [x] /eth/v1/beacon/blocks
• [x] /eth/v1/beacon/blocks/{block_id}
• [x] /eth/v1/beacon/blocks/{block_id}/root
• [x] /eth/v1/beacon/blocks/{block_id}/attestations
• [x] /eth/v1/beacon/pool/attestations
• [x] /eth/v1/beacon/pool/attester_slashings
• [x] /eth/v1/beacon/pool/proposer_slashings
• [x] /eth/v1/beacon/pool/voluntary_exits

Debug

• [x] /eth/v1/debug/beacon/states/{state_id}
• [x] /eth/v1/debug/beacon/heads

Node

• [ ] /eth/v1/node/identity
  
  
  
  • Partially implemented, missing fields.
  
  
  • #1631 adds one of the required fields.
  
  
• [ ] /eth/v1/node/peers
• [ ] /eth/v1/node/peers/{peer_id}
• [x] /eth/v1/node/version
• [x] /eth/v1/node/syncing
• [ ] /eth/v1/node/health

Config

• [x] /eth/v1/config/fork_schedule
• [x] /eth/v1/config/spec
• [x] /eth/v1/config/deposit_contract

Validator

• [x] /eth/v1/validator/duties/attester/{epoch}
• [x] /eth/v1/validator/duties/proposer/{epoch}
  
  
  
  • Only returns duties for the current epoch to make caching easier. It would be nice to add a non-cached path for historical requests, too.
  
  
• [x] eth/v1/validator/blocks/{slot}
• [x] /eth/v1/validator/attestation_data
• [x] /eth/v1/validator/aggregate_attestation
• [x] /eth/v1/validator/aggregate_and_proofs
• [x] /eth/v1/validator/beacon_committee_subscriptions

Events

• [ ] /eth/v1/events

Known Issues

• [ ] https://github.com/sigp/lighthouse/pull/1569#issuecomment-697658749 (thanks @guybrush)
• [ ] #1717 (thanks @mcdee)

[michaelsproul]

It would be nice to have /eth/v1/node/peers soon, as it's used by eth2stats

[realbigsean]

It would be nice to have /eth/v1/node/peers soon, as it's used by eth2stats

Yes, I've started work on it

[michaelsproul]

Something else I noticed while using the API:

The /eth/v1/beacon/states/{state_id}/validators endpoint doesn't support filtering by validator ID or status, and the statuses returned are Lighthouse enum names, not the status strings from the spec. For example, the query /eth/v1/beacon/states/head/validators?status=active_slashed&status=exited_slashed returns:

{
  "data": [
    {
      "index": "0",
      "balance": "31760402067",
      "status": "Active",
      "validator": {
        "pubkey": "0x8fcf28896a85e5e76ee9e508438e23e7253da1a23a6501e3a7d56182520dbcf4cdb44af3267318188f1f4168342146da",
        "withdrawal_credentials": "0x0010361af430aa7ab4a9567eaaca50ec5e02315ca1513d9ee8d73bde96370091",
        "effective_balance": 31000000000,
        "slashed": false,
        "activation_eligibility_epoch": 0,
        "activation_epoch": 0,
        "exit_epoch": 18446744073709552000,
        "withdrawable_epoch": 18446744073709552000
      }
    },
    ...
  ]
}

The relevant bit of code is:

https://github.com/sigp/lighthouse/blob/a0634cc64f5938f0a9ad75894b9e73f595919f70/beacon_node/http_api/src/lib.rs#L405

[guybrush]

We need /eth/v1/validator/duties/proposer/{epoch} to also work for historical epochs for the explorer.

[paulhauner]

statuses returned are Lighthouse enum names, not the status strings from the spec

I'm working to update the spec for this: https://github.com/ethereum/eth2.0-APIs/pull/94#issuecomment-708110821 We will indeed need to switch to strings once that's finalized.

[mcdee]

We need /eth/v1/validator/duties/proposer/{epoch} to also work for historical epochs for the explorer.

Wanted to add my support for this, as well as the same for /eth/v1/beacon/states/{stateid}/committees. An example of where this is useful beyond explorers: ethdo has a block info command that dumps information about a block. The committees are required to unpick the aggregation bits in the block's attestations and show who attested for what.

I'd expand this to suggest that all data under /eth/v1/beacon/states and /eth/v1/beacon/blocks should allow arbitrarily far back state/block IDs. If the node doesn't have the data it can spit back a 404, but for data that can be retrieved or reconstituted it is very handy to be able to do so.

[dankrad]

We need /eth/v1/validator/duties/proposer/{epoch} to also work for historical epochs for the explorer.

Just want to throw my weight behind this -- apparently this endpoint is required for having a block explorer work with lighthouse, and I would very much like us to be able to have lighthouse based block explorers as a backup to prysm on launch.

[realbigsean]

I've just added historical lookups for /eth/v1/validator/duties/proposer/{epoch} to #1831 👍

[roman-blox]

@paulhauner this endpoint https://ethereum.github.io/eth2.0-APIs/#/Validator/getAttesterDuties should accept POST request, but your implementation doesn't do so. I use openapi generate tool to generate API client based on that openapi.

[realbigsean]

@roman-blox this will be updated with #1831

[begmaroman]

@realbigsean do you have any ETA of this PR?

[begmaroman]

@realbigsean one more issue. This generate tool generates multiple structures with the same name. This affects the block already redeclared error. Do you know how to fix that?

[winlin]

hello, we need /eth/v1/beacon/states/{state_id}/validator_balances to realize the pool function, it's not work when I pass the <slot>, <hex encoded stateRoot with 0x prefix>

[realbigsean]

@winlin if you're querying an old slot or state root, the queries can take a while to complete, but they should be working

[paulhauner]

Although we may not be exactly on spec, the spec is not yet stable and is still being modified. As such, I think we've achieved the goal of this issue and we can close it.

We can open new issues as we learn about spec changes or bugs.

Well done @realbigsean for getting this across the line!