Chronograf: Faster updates, shorter time period

[mgazic]

I'd like to use Chronograf to display data that is updated at a high rate. I hacked a quick change to allow plots to update at 500ms and 1s, and allow time scales of 10s and 1 minute. This seemed to work great! I do not know enough about Chronograf to say this is a desirable change, but the actual source code changes were very small, and I'd love to hear if this is something you'd find acceptable. What I'd really like is if there was a way for InfluxDB to push data to Chronograf when updates occur, but that seems like a much more complicated change, and I don't know if it's something others would even find desirable.

diff --git a/ui/src/shared/data/autoRefreshes.js b/ui/src/shared/data/autoRefreshes.js
index 5ce7bf6..cf421dd 100644
--- a/ui/src/shared/data/autoRefreshes.js
+++ b/ui/src/shared/data/autoRefreshes.js
@@ -1,6 +1,16 @@
 const autoRefreshItems = [
   {milliseconds: 0, inputValue: 'Paused', menuOption: 'Paused'},
   {
+    milliseconds: 500,
+    inputValue: 'Every 500ms',
+    menuOption: 'Every 500ms',
+  },
+  {
+    milliseconds: 1000,
+    inputValue: 'Every 1 seconds',
+    menuOption: 'Every 1 seconds',
+  },
+  {
     milliseconds: 5000,
     inputValue: 'Every 5 seconds',
     menuOption: 'Every 5 seconds',
diff --git a/ui/src/shared/data/timeRanges.js b/ui/src/shared/data/timeRanges.js
index 914f0dd..fdcb42f 100644
--- a/ui/src/shared/data/timeRanges.js
+++ b/ui/src/shared/data/timeRanges.js
@@ -1,5 +1,21 @@
 export const timeRanges = [
   {
+    defaultGroupBy: '0.1s',
+    seconds: 300,
+    inputValue: 'Past 10 seconds',
+    lower: 'now() - 10s',
+    upper: null,
+    menuOption: 'Past 10 seconds',
+  },
+  {
+    defaultGroupBy: '1s',
+    seconds: 300,
+    inputValue: 'Past 1 minute',
+    lower: 'now() - 1m',
+    upper: null,
+    menuOption: 'Past 1 minute',
+  },
+  {
     defaultGroupBy: '10s',
     seconds: 300,
     inputValue: 'Past 5 minutes',

[russorat]

@mgazic thanks for opening an issue. In the past, we've refrained from adding such small granularity to our dropdowns since most users don't need to update that frequently and it causes a lot of queries to be run on the DB. I'm trying to envision the use case for updating a UI so frequently though.

Could a user actually see and react an issue in < 5 seconds? Or is there some other usecase this would solve for you?

[mgazic]

The system I’m working with has sensor data produced at up to 50Hz, but is used sporadically during engineering development. I’ve got PC based tools that serve the data over TCP and websockets to multiple apps that analyze and plot the data in real-time for a unit connected to a PC. I just added storage to InfluxDB to that system and it seems to work great so far. I’m excited about users being able to scroll back through historical data on the web, but I’m not sure how to integrate the real-time view of data with the web based view of historical data. It seems awkward for users to use my own tools for real-time, and then switch to chronograf for historical data. Adding those faster options to Chronograf seemed to work and it performed well for my use case, but if it leads to performance problems, maybe it shouldn’t be done. I’ve got a video showing the fast updates in chronograf, grafana, and my own real-time tools, I’ll post that later tonight.

[mgazic]

Here's a video showing 4 apps plotting sensor data in real time:
http://miles-engineering.com/FastUpdates.mp4

The sensor is a 3-axis accelerometer on the PCB that's visible being rotated. The 4 apps are:
1) upper left is a custom python app receiving data via TCP sockets
2) lower left is a custom webapp receiving live data via websockets
3) upper right is Grafana pulling data out of InfluxDB
4) lower right is Chronograf pulling data out of InfluxDB

[russorat]

@mgazic that's really cool. thanks for sharing. we will see how we can enable this use case.

[mhfrantz]

We have a similar use case. Our engineers have requested 10 Hz (100ms) refresh rate because they use the UI to intercede in semi-autonomous mode during development of the autonomous mode algorithms, and feel the high frame-rate enables them to use their reflexes to maintain control of our product. (I had similar reservations about the usefulness of a 10 Hz refresh until I understood this use case.)

We have a proprietary UI that was built using websockets to receive the telemetry updates at 10 Hz as they are relayed through the proprietary telemetry stack. I hacked a 10 Hz auto-refresh option into Chronograf, and it performs reasonably well, but it does put significant load on both client and server when compared with our custom solution.

I have considered using Kapacitor to relay the data to a custom client, which would mirror the proprietary architecture.

This feels like it is in the spirit of a "continuous query". Could that be part of the answer?

In the interim, I thought that making the auto-refresh and time-ranges data structures configurable would help. I'll be studying the Chronograf code to see how that could be accomplished.

[mgazic]

I wonder about the possibility of modifying Chronograf to work both with InfluxDB queries (as is currently implemented), plus a new feature of having Chronograf tell Kapacitor to stream incoming data to it. Perhaps the switchover would happen automatically depending on UI options, perhaps it'd be a separate option called something like "streaming" in the UI. I'd much rather this become a part of Chronograf (or Grafana, or some existing tool), rather than having my users switch between two different front ends.

[clancye]

This would be useful for us as well. We have electromechanical systems under test and want the data display to be as close to real-time as possible when debugging/investigating a system on the test bench.

[kangting5]

Hi,
I'm trying to increase the refresh time, I'm limited to a minimum of 5 seconds on chronograf and I didn't really understand how you configured this.
Thank you for your feedback

[mgazic]

I'm trying to increase the refresh time, I'm limited to a minimum of 5 seconds on
chronograf and I didn't really understand how you configured this.

You would need to get Chronograf source code, and make the changes I showed (to autoRefreshes.js and timeRanges.js), and then run your modified version of Chronograf.

If you want this in a released version of Chronograf, you'd have to convince Chronograf developers to implement it and support it (perhaps enabled by a configuration setting).

[rpetteruti]

Thats would be a great feature!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Feel free to reopen if this issue is still important to you. Thank you for your contributions.

[let4be]

Would be nice to have... any plans?

[russorat]

@let4be thanks for the comment. We are planning to implement incremental dashboard query updates (only fetch data since the last refresh) as part of influxdb 2.0, which should allow us to increase the refresh frequency. Stay tuned!

[xufangda]

@let4be thanks for the comment. We are planning to implement incremental dashboard query updates (only fetch data since the last refresh) as part of influxdb 2.0, which should allow us to increase the refresh frequency. Stay tuned!

Any progress for this function yet? Eager to know!

[let4be]

Afaik right now it's already possible via direct JS edit either in browser or by using custom chronograf build - but this would produce way more load on influxdb

[russorat]

@xufangda we plan to add builds for window in 2.0 as we get closer to a GA release towards the end of Q2.

[danczw]

@xufangda we plan to add builds for window in 2.0 as we get closer to a GA release towards the end of Q2.

Has this been updated? Couldn't find anything in the documentation.

[saaib]

Hi @russorat , has this feature been released?
Thanks,

[russorat]

@danczw you can follow the windows process here: https://github.com/influxdata/influxdb/issues/17192

@saaib we are tracking this capability for our latest influxdb release (issue is in a private repo though)