Tidb: Question : TIDB as a replacement for AWS RDS

[rohitjoshi]

Hello,

I am evaluating tidb as a replacement for AWS RDS (MySQL). Here are high-level requirements.
Let me know if this can be achieved with TiDB.

1. Strong consistency (unique constraints)
2. Table Scalability (B+ rows in a table)
3. Rolling upgrade (AWS AMI needs to be rehydrated/updated every month)
4. Cross Region Replication: Availability in AWS East and West regions
5. Writes 10K+
6. Reads 50K+
7. Records per table: 1.5B+
8. User Authentication
9. SSL/TLS connectivity between clients to Server and during replications

CREATE TABLE `table_xxx` (
  `hash` VARCHAR(48) NOT NULL,
  `token` VARCHAR(48) NOT NULL,
  `enc_value` VARCHAR(160) NOT NULL,
  `csum_token` VARCHAR(32) NOT NULL,
  `csum_detoken` VARCHAR(32) NOT NULL,
  `status` VARCHAR(1) NOT NULL DEFAULT 'A',
  `created_timestamp` DATETIME DEFAULT CURRENT_TIMESTAMP,
  `updated_timestamp` DATETIME DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  `id BIGINT AUTO_INCREMENT UNIQUE`,
  PRIMARY KEY (`hash`,`token`),
  UNIQUE KEY `unique_hash` (`hash`),
  UNIQUE KEY `unique_token` (`token`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

INSERT: Stored Proc is used to insert 50 records at a time.
SELECT: Stored Proc is used to retrieve 50 records either by token or hash
SELECT: When application starts, it retrieves all the records in memory using id column 5000 records at a time so reads are only done when not found in the cache.

[c4pt0r]

Thanks for your attention and information, we're preparing a detailed answer to your questions.

[rohitjoshi]

Sure, thanks.
support for stored-proc is not necessary as long as we can support either batch operations or high throughput for writes.

Regarding SSL/TLS requirement, data must be transferred over network through a secure channel.

[rohitjoshi]

1. The only reason we are using Relational DB is strong consistency with unique constraints for storing the combination of both hash and token values.

If below operation can be performed atomically, we can just use tikv.

Query `token` by `hash` and return `token` if found 
else  
insert `token` and `hash`

NOTE: token is generated randomly so there is a chance of duplicate token generated by multiple application instances. We need to ensure that we never store two different tokens for same hash value.

If this can be supported in tikv, I would prefer that over tidb as our 95% use-case is read.

1. We are using two layers (application In-memory and Elastic Cache/Redis) of caches. Would love to get rid of Redis if tidb or tikv can support 500K+ read per second with low-latency. Today, we are using six shards of redis which is quite expensive.

[c4pt0r]

We're still testing in our environment with your scenario, we'll update on this issue when we have a test result.
I can answer you following question at a high level.

1. Yes, TiKV supports ACID transaction semantics in key-value level, so your operation can be performed atomically and tikv would guarantee the consistency.

1. TiKV is not a drop-in replacement for Redis, TiKV has to do more things than Redis do to guarantee data durability and consistency (like MVCC / Distributed transaction /...), TiKV uses RocksDB underneath for persisting data. Although RocksDB has a Memtable for caching hot data in memory, it's not designed for the scenarios that require extremely low latency. But we're still trying our best to give you a reasonable result. :)

[ChinaXing]

+1