Tidb: Support timestamp bounded read-only transactions on data replicas Tasklist

[NingLin-P]

Development Task

Timestamp bounded read-only transactions return stale data within a user-provided staleness bound, and can provide better performance than normal read-only transactions:

• Can perform at replicas that don't have the most up-to-date data, so the ReadIndex request can be avoided and reduce the network latency cost, especially in cross-region deployment.
• With the specified staleness, write-read conflict can be greatly reduced

Issue: https://github.com/pingcap/tidb/issues/19817
Propoasl: Follower to serve local(stale) read (In Chinese)

New feature

Basic work

• [x] Support timestamp bounded syntax https://github.com/pingcap/parser/pull/610
• [x] Get the historical timestamp issues/21522 @Andrewmatilde
• [ ] Choose the closest replicas by labels issues/21540 @Yisaer
• [ ] Add timestamp bounded option to read requests at kvproto

Timestamp bounded read-only transactions

• [x] Support exact staleness read-only transactions
  
  
  
  • [x] Begin a staleness read-only transaction #21821
  
  
  • [x] Allocate start timestamps for exact staleness read-only transactions #21717
  
  
  • [x] Validate statements for read-only transactions #21824
  
  
• [ ] Support bounded staleness read-only transactions
  
  
  
  • [ ] Support syntax SET TRANSACTION READ ONLY WITH ...
  
  
  • [ ] Maintain a zone-level resolved ts
  
  
  • [ ] Get start ts from resolved ts and specified timestamp
  
  
  • [ ] Choose replicas by distance
  
  
• [ ] Compatibility with other features
  
  
  
  • [ ] Keep compatible with DDL #22427
  
  
  • [ ] Keep compatible with plan cache #22470
  
  
  • [ ] Keep compatible with coprocessor cache #22510
  
  
  • [ ] Keep compatible with SQL binding and SPM
  
  
  • [ ] Keep compatible with history read
  
  
• [ ] Errors handling

TiKV

• [ ] Maintain replica's Resolved TS @NingLin-P
  
  
  
  • [ ] Decouple Resolved TS from component/cdc
  
  
  • [ ] Forward replica's Resolved TS when the region is inactive https://github.com/tikv/tikv/issues/9384
  
  
  • [ ] (optional) Forward Resolved TS when there are ongoing large transactions
  
  
  • [ ] (optional) Forward Resolved TS when there are aborted transactions
  
  
• [ ] (optional) Maintain replica's Applied TS
  
  
  
  • [ ] Forward replica's Applied TS by applying transactional raft logs
  
  
  • [ ] Forward replica's Applied TS when the region is inactive
  
  
  • [ ] Keep compatible with Async Commit
  
  
• [ ] Support handling stale read request on replica
  
  
  
  • [ ] Change the replica read process https://github.com/tikv/tikv/issues/9346
  
  
  • [ ] Refine the handling of EpochNotMatch error
  
  
• [ ] (optional) "1PC" optimization

[5kbpers]

For decoupling Resolved TS, may include the following changes:

• Move the CmdObserver implementation of CdcObserver to resolved ts package
• Start a new thread for capturing changes from apply worker, tracking the lock, and redirecting changes to cdc thread
• Provide an interface (through an atomic variable) for getting the current resolved ts

[nolouch]

Hi, @sunxiaoguang , Is anyone interested in #21522 and #21540？