Timescaledb: Downsampling/decaying howto?

[nwmcsween]

Is there a simple way to do drop_chunks but instead of throwing away data it's munged through some sort of downsampler (e.g. 1s intervals to 10s) and is it possible to cascade it (1s to 10s to 100s to ...)?

[mfreed]

Hi @nwmcsween the common approach folks do is use a scheduled UPSERT to aggregate data from one hypertable to a second. https://docs.timescale.com/v1.0/using-timescaledb/writing-data#upsert

With our new support for background tasks, we'll be adding it policy-drive aggregations in the future. For now, you can easily schedule them external to the database similar to with drop_chunks: https://docs.timescale.com/v1.0/using-timescaledb/data-retention

The nice approach using an UPSERT is they you can more easily handle late data. i.e., every 5second, do a pass over the last 10 seconds to make sure you recompute any of the secondly data that arrived late.

[mfreed]

Initial release of continuous aggregations:

Release - https://github.com/timescale/timescaledb/releases/tag/1.3.0
Overview Docs - https://docs.timescale.com/v1.3/using-timescaledb/continuous-aggregates
Tutorial - https://docs.timescale.com/v1.3/tutorials/continuous-aggs-tutorial