Timescaledb: A guide on how to setup replication with timescaledb

[tobernguyen]

Are there any how-to guides on setup replication/cluster with timescaledb?

[akulkarni]

Hi there. In general, we support standard PG streaming replication, but should have a more detailed tutorial soon. Nothing published at the moment however.

[WilliamHoang]

Hey @tobernguyen, love to understand your use-case! Curious what are you currently building and a more detailed tutorial is coming soon like @akulkarni mentioned.

[tobernguyen]

@WilliamHoang thanks for asking, we are using TimescaleDB to store cryptocurrency price ticks. Every 5 minutes we do some heavy aggregation (min, max to calculate % price change over time) and this process takes about 2-3 minutes to complete and the CPU usage is about 100% (both cores) during processing time. During that 2-3 minutes, the querying to get tick data is much slower, about 3-5 times slower than normal (I think because of the hardware constraint and there are a lot of pending operations on DB during that time). So I'm planning to set up replication so that I have one instance for doing heavy aggregation and others for real-time querying to reduce response time (which directly impacts user experience).

What do you guys think about this?

Many thanks!

[pantchox]

Also interested to know this!

[djk447]

Also would be interested to know how much replication lag you run into given that pg streaming replication is single-core and you hit your peak write with lots of cores, I would imagine that you can't keep up with sustained 100,000 rows/s copy loading and not end up falling behind significantly, but I might be wrong. Would be interesting to see.

[ancoron]

In this case it might make more sense to front the DB with pgpool2, which can send the write queries to multiple hosts/clusters in parallel. The whole system would then be as slow as the slowest host/cluster. Could also enable load-balanced reads across multiple clusters/hosts.

[balupton]

Anyone make any headway in the last 5 months?

[solugebefola]

We actually have a tutorial in the works, probably coming out next week.

[clourens]

Any news on the documentation / tutorial ?

[LeeHampton]

Hey all, just letting you know that the documentation on streaming replication has been released. Take a look in our official documentation . And of course if you have any questions, don't hesitate to reach out on our support slack group!

[tobernguyen]

That's awesome. Thank you all for your great works. I requested this 6 months ago and now I am still satisfied with TimescaleDB even with a single instance. I will definitely look into the guide and see if I can manage to create a statefulset running on K8S for TimescaleDB. Thanks again!

[RobAtticus]

I'm going to close this out for now, but feel free to re-open or file a new issue if you run into other issues or have feature requests :)