Influxdb: Wildcard support for fields in continuous queries.

[steverweber]

when the only value was "value", there was a simple way to create a dynamic CQ when backreferencing :measurement however the new format telegraf uses seems to break this.

https://docs.influxdata.com/influxdb/v0.10/query_language/continuous_queries/#cqs-with-backreferencing

I'm wondering if there is a better way to do this so sub measurements are included.

CREATE DATABASE telegraf WITH DURATION 100d
CREATE RETENTION POLICY "7y" ON telegraf DURATION 2555d REPLICATION 1

CREATE CONTINUOUS QUERY "30m_cpu" ON telegraf BEGIN SELECT mean(usage_idle) as usage_idle INTO telegraf."7y".:measurement FROM telegraf."default".cpu GROUP BY time(30m), host END

CREATE CONTINUOUS QUERY "30m_mem" ON telegraf BEGIN SELECT mean(used_percent) as used_percent INTO telegraf."7y".:measurement FROM telegraf."default".mem GROUP BY time(30m), host END

CREATE CONTINUOUS QUERY "30m_disk" ON telegraf BEGIN SELECT mean(used_percent) as used_percent INTO telegraf."7y".:measurement FROM telegraf."default".disk GROUP BY time(30m), host END

[zstyblik]

As far as I know, there isn't better way and all fields must be listed.

[steverweber]

Would be nice to have.
Can this request get a milestone?

[adrianlzt]

+1

[rubycut]

If I understood correctly, in order to downsample data, I need have create cq query for every single field in every single measurement, that will take like whole week. And every time I add new fields, I will have to add more continuous queries.

I am currently filling up influxdb with new 0.10 version of telegraf, cadvisor and heapster.

And after I do all this, grafana will not recognise this new retention policies unless I change data source.

This is pretty much deal breaker for influxdb,

[nickhuber]

This seems like a really important feature, since the old style (just a single value) is very simple to downsample but when taking advantage of multiple values per measurements you need to make many continuous queries to get the same result, and could easily miss adding new CQs when new data points are added

[arkadiuszbicz]

+1

[TomGudman]

+1

I reckon a blog post on how to set retention policies for many (random) metrics that you do not even know or manage would be good.

For instance in the following examples, you would like influxdb to automagically handle the 'retention rules' of any new metrics :

• Add a new hosts which will send its metrics to the influx
• Add a new metric/check on all hosts which will then end in influx

FWIW, our setup: collectd-client --> collectd-server --> influx

[jsternberg]

This will be fixed by #6529 when that's merged.

[klyngbaek]

+1

[jsternberg]

Wildcard support for only fields is now supported.

[beckettsean]

@jsternberg This seems to be the right issue for field wildcards. I think it was closed by mistake. This functionality definitely does not yet exist.

[beckettsean]

We do now support SELECT *::field FROM foo but that's only for selecting non-aggregated points. Therefore not useful in CQs, which require a function.

The original ask is to support something like:

SELECT MEAN(*::field) INTO bar FROM foo GROUP BY time(5m) so that CQs can be built without perfect knowledge of the field sets in the measurement. It also makes the CQs future-proof as new fields are added to the measurement.