Influxdb: Flux function erroneously returns results from previous function
I have a long flux script. As I add functions, my flux query breaks. My final function is erroneously returning results from a previous function.
I tried this with 2.0, 1.7, and also from the CL, so it's not a browser issue.
1st Part of Flux Script
y = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> yield(name: "y")
x = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> yield(name: "x")
x()
Results as expected
Now I add some more to the script.
2nd Part of Flux Script
y = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> yield(name: "y")
x = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> yield(name: "x")
y_bar = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> mean(columns: ["_value"])
|> keep(columns : ["_value"])
|> yield(name: "y_bar")
x_bar = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> mean(columns: ["index"])
|> keep(columns : ["index"])
|> yield(name: "x_bar")
diff_y = () => y()
|> map(fn: (r) => ({_time: r._time, _value: r._value, y_bar: 2.56}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r._value - r.y_bar)}))
diff_x = () => x()
|> map(fn: (r) => ({_time: r._time, index: r.index, x_bar: 15}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r.index - r.x_bar)}))
joined = () => join(tables: {diff_y: diff_y(), diff_x: diff_x()}, on: ["_time"]])
joined()
Results as expected
Now I add the last function, and it calls
x()
instead of executing
m_num()
y = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> yield(name: "y")
x = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> yield(name: "x")
y_bar = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> mean(columns: ["_value"])
|> keep(columns : ["_value"])
|> yield(name: "y_bar")
x_bar = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> mean(columns: ["index"])
|> keep(columns : ["index"])
|> yield(name: "x_bar")
diff_y = () => y()
|> map(fn: (r) => ({_time: r._time, _value: r._value, y_bar: 2.56}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r._value - r.y_bar)}))
diff_x = () => x()
|> map(fn: (r) => ({_time: r._time, index: r.index, x_bar: 15}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r.index - r.x_bar)}))
joined = () => join(tables: {diff_y: diff_y(), diff_x: diff_x()}, on: ["_time"]])
m_num = () => joined() |> map(fn: (r) => (r.diff_y * r.diff_x))
m_num()
results are from x() instead of from m_num()
All data is system stats.
Anaisdg
All 5 comments
@timhallinflux
Anaisdg
on 19 Mar 2019
This might be related to https://github.com/influxdata/flux/issues/1102
nathanielc
on 1 Apr 2019
@Anaisdg that's because the UI is returning one of the results it could compute before failing.
x() yields the result x and the UI displays it. However, later steps include errors, so they don't produce any result.
Your definition of m_num in the script doesn't use the correct identifiers (for example, r.diff_y should be r._value_diff_y). If I try to run that script from the CL, I get:
Result: x
Table: keys: [_start, _stop, _field, _measurement, cpu, host]
_start:time _stop:time _field:string _measurement:string cpu:string host:string _time:time index:int
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ --------------------------
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:29:21.000000000Z 1
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:29:31.000000000Z 2
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:29:41.000000000Z 3
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:29:51.000000000Z 4
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:01.000000000Z 5
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:15.000000000Z 6
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:25.000000000Z 7
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:35.000000000Z 8
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:45.000000000Z 9
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:55.000000000Z 10
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:05.000000000Z 11
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:15.000000000Z 12
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:25.000000000Z 13
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:35.000000000Z 14
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:45.000000000Z 15
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:55.000000000Z 16
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:05.000000000Z 17
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:15.000000000Z 18
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:25.000000000Z 19
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:35.000000000Z 20
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:45.000000000Z 21
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:55.000000000Z 22
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:05.000000000Z 23
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:15.000000000Z 24
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:25.000000000Z 25
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:35.000000000Z 26
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:45.000000000Z 27
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:55.000000000Z 28
2019-05-03T09:29:20.574642000Z 2019-05-03T09:34:20.574642000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:05.000000000Z 29
Result: y
Table: keys: [_start, _stop, _field, _measurement, cpu, host]
_start:time _stop:time _field:string _measurement:string cpu:string host:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
Error: Failed to execute query: failed to read meta data: function references unknown column "diff_y".
With the correct identifiers (the implementation that you are using in the attached image), I also get:
Result: x
Table: keys: [_start, _stop, _field, _measurement, cpu, host]
_start:time _stop:time _field:string _measurement:string cpu:string host:string _time:time index:int
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ --------------------------
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:15.000000000Z 1
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:25.000000000Z 2
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:35.000000000Z 3
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:45.000000000Z 4
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:30:55.000000000Z 5
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:05.000000000Z 6
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:15.000000000Z 7
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:25.000000000Z 8
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:35.000000000Z 9
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:45.000000000Z 10
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:31:55.000000000Z 11
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:05.000000000Z 12
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:15.000000000Z 13
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:25.000000000Z 14
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:35.000000000Z 15
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:45.000000000Z 16
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:32:55.000000000Z 17
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:05.000000000Z 18
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:15.000000000Z 19
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:25.000000000Z 20
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:35.000000000Z 21
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:45.000000000Z 22
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:33:55.000000000Z 23
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:05.000000000Z 24
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:15.000000000Z 25
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:25.000000000Z 26
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:35.000000000Z 27
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:45.000000000Z 28
2019-05-03T09:30:08.589714000Z 2019-05-03T09:35:08.589714000Z usage_system cpu cpu-total macbook 2019-05-03T09:34:55.000000000Z 29
Result: y
Table: keys: [_start, _stop, _field, _measurement, cpu, host]
_start:time _stop:time _field:string _measurement:string cpu:string host:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
Error: Failed to execute query: failed to read meta data: failed to evaluate map function: parameter "r" has the wrong type, expected {_value_diff_y: float,_value_diff_x: float,} got {_value_diff_y: float,_value_diff_x: int,}.
So, you are mixing types when you are multiplying: r._value_diff_x * r._value_diff_y.
I was able to produce a correct result with this script:
y = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> yield(name: "y")
x = () => from(bucket: "Default")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total")
|> map(fn: (r) => ({_time: r._time, index: 1}))
|> cumulativeSum(columns: ["index"])
|> yield(name: "x")
diff_y = () => y()
|> map(fn: (r) => ({_time: r._time, _value: r._value, y_bar: 2.56}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r._value - r.y_bar)}))
diff_x = () => x()
|> map(fn: (r) => ({_time: r._time, index: r.index, x_bar: 15}))
|> map(fn: (r) => ({"_time": r._time, "_value": (r.index - r.x_bar)}))
|> toFloat()
joined = () => join(tables: {diff_y: diff_y(), diff_x: diff_x()}, on: ["_time"]])
m_num = () => joined() |> map(fn: (r) => ({_value: r._value_diff_x * r._value_diff_y}))
m_num()
As you can see, I added toFloat as a last step to diff_x.
@Anaisdg Didn't the UI show you that that was a partial result and the script also returned an error?
Thanks for the example and the detailed information.
Thanks to it, I was able to discover 2 bugs:
- why does
yis empty when yielded andxnot? - if you try to
float(r._value_diff_x) * r._value_diff_yas opposed to usingtoFloat()you get an error!
I'll file issues for those ones.
I'll run your script from the UI and check for the presence of an error. If there is none I'll file an issue for that too.
Thanks again.
affo
on 3 May 2019
❤1
I ran the script from Chronograf and I could see the error in a separate table:
However, I am filing an issue. It would be nice if Chronograf could be more explicit when an error is encountered (https://github.com/influxdata/chronograf/issues/5172).
affo
on 3 May 2019
❤1
@affo Thank you!
Anaisdg
on 3 May 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
bigKS
路
3Comments
shilpapadgaonkar
路
3Comments
dandv
路
3Comments
MayukhSobo
路
3Comments
756445638
路
3Comments