Timescaledb: Feature request: Continuous downsampling

[Sytten]

I know this has been asked a few times already over the years (https://github.com/timescale/timescaledb/issues/686, https://github.com/timescale/timescaledb/issues/242), but I wanted to know if there was an update on the priority of this feature. I feel that almost all time series application need this one way or another.
Maybe the integration with pipelinedb (https://github.com/timescale/timescaledb/issues/14) would could solve this issue?

[mfreed]

Hi @Sytten: Thanks for the interest. We'll be releasing support for in-database continuous aggregations in v1.3, which should be released within the next few weeks. Code should hit master branch within a few days.

[Sytten]

Wow, this is amazing! Do you have a public roadmap where we can the planned features?

[mfreed]

We have not published it yet. Can you say more about your particular application needs or use case?

[Sytten]

Mostly sensor sampling that we want to keep for a longer period without the same accuracy. If we can keep a count and a sum that would be perfect. We were considering using PipelineDB for that, but it is not well supported by managed services.

[nickbabcock]

Very neat. I was about to roll my own by having a realtime table that is downsampled (then drop_chunks) every day into 15 minute intervals into another table (another hypertable) and have a view encapsulate both for grafana. But if this will cover that use case, then I'm all for it and can wait! Hopefully this feature is slotted for the community edition. 😄

I'm curious how you handle aggregations. When graphite downsamples data, one can configure if the resulting value is averaged, summed, min value, max value, or last value.

If you want to know more about my use case: I have timescale at the receiving end of hardware sensor data

[mfreed]

@nickbabcock The initial release of continuous aggregations will all be in the community edition. Future extensions may include some enterprise features. You can indeed con figure the result fairly flexibly -- basically any aggregate in PostgreSQL that can be efficiently parallelized (average, sum, min, max, count, etc.).

What's interesting about our implementation is it leverages partials: The aggregation doesn't actually store an average, but instead stores sums & counts, and then will compute the average from those at query time -- all transparent to the user -- which will allow us to do a lot more interesting things in transparently combining pre-computed aggregates (e.g., in a hierarchical fashion).

And here it is: https://github.com/timescale/timescaledb/pull/1184

[mfreed]

Release - https://github.com/timescale/timescaledb/releases/tag/1.3.0
Overview Docs - https://docs.timescale.com/v1.3/using-timescaledb/continuous-aggregates
Tutorial - https://docs.timescale.com/v1.3/tutorials/continuous-aggs-tutorial