Tikv: VerKV work plan

[fredchenbj]

VerKV work plan

This is the work plan for the development of verkv rfc. It has two implementing processes:

Fundamental function tasks

• [x] Add verkv proto file in repo kvproto https://github.com/pingcap/kvproto @fredchenbj
• [x] Add a new CF versioncf like the defaultcf in tikv-server @fredchenbj
• [ ] Add the Put/BatchPut RPC interface, and implement tikv-server side logic
• [ ] Redefine Version Region split point, here ensure all versions of a key is located in one region
• [ ] Add the Get/BatchGet RPC interface, and implement tikv-server logic. Here decode value in client-go. Here it needs to use the iterator of RocksDB to get all versions of the key, and filter needed numbers of key-value to return
• [ ] Add config about MaxVerNum about VerKV's data, and add GC logic to implement the version recycling
• [ ] Add the GetNumWithSV/BatchGetNumWithSV RPC interface, and implement tikv-server logic. Here it needs to use the iterator of RocksDB to get all versions of the key and filter older versions of key-value to return. This interface also supports TTL function
• [ ] Implement Scan related RPC functionality, like the scan of RawKV. It would return a range of key-value of needed versions
• [ ] Implement Delete related RPC functionality, like the delete of RawKV.
• [ ] Implement all above RPC API in client-go SDK https://github.com/tikv/client-go. Here define the key-value format and need to obtain the ts from PD TSO, then encode on client-go SDK. For an easy test, mock the server logic in SDK.

Advanced function tasks

• [ ] Implement BR related functionality. Here strong consistency must be ensured, and incremental data backup should be implemented like TxnKV.
• [ ] Implement CDC related functionality like TxnKV.

Pick up one task

If you are interested in the above tasks, comment and pick up one. Mentoring available.

[fredchenbj]

I pick task "Add verkv proto file in repo kvproto".

pr: https://github.com/pingcap/kvproto/pull/591

[fredchenbj]

i pick up task "Add a new CF versioncf like the defaultcf in tikv-server"

pr: https://github.com/tikv/tikv/pull/7607

[kennytm]

Can we quickly implement the verkv interface with unimplemented!() or whatever? Currently we can't update kvproto for BR in TiKV because of this.

[fredchenbj]

@kennytm OK, I will do it tomorrow.

[fredchenbj]

@kennytm see pr: https://github.com/tikv/tikv/pull/7449

[hskang9]

@fredchenbj I will work on the task 'Add the Put/BatchPut RPC interface, and implement tikv-server side logic' Please mentor me for following items:

1. What file implements the RPC interface? Does it require certain trait or struct to communicate since it has versions?
2. What essential property should the data include to put in RocksDB? (Discussion required)

[fredchenbj]

@hskang9 OK, I would have a discussion with you later on slack.

[hskang9]

@fredchenbj any review for 3rd item in PR #8282?