Dash: Best practices for live updates

[crazycodingcat]

Hi,

I'm working on a dashboard that queries the database every 1 minute and updates various charts with the new data. I noticed some strange behavior with the live updates. Most of the time everything works fine, but occasionally some of the charts gets updated with old data (like, data that's 5 or 10 minutes ago), which doesn't make sense to me. Checked the database and the data there are always up to date.

Initially, I have set up callback functions for each of the charts with events=[Event('interval-component', 'interval')] as the trigger. I have a separate callback function which does the database query and updates data in a few global variables, which are used for charts plotting.

I suspect it might be due to some race condition problem - the charts are getting updated with the event trigger, but the data update has not finished yet. So I changed the condition for the callback. Now I have one callback function which gets triggered every one minute and pulls latest data (into global variables) from the database and updates a div on my page with a timestamp. Then, all the other charts watch the timstamp div and update themselves (using the global variables) if there is any changes. I thought this may ensure that my database query finishes first before the charts get updated. However, it seems it still has not solved the problem.

So I would like to know what is the best practice for handling such live updates. e.g.

• Is it a good idea to store the data in global variables? I don't want to have repeated database queries for each chart so I just want to do it in one go.
• If events=[Event('interval-component', 'interval')] are used for updating multiple components, in what sequence does Dash execute the callback functions?

[chriddyp]

It's likely an issue with the global variables. Traditional Python global variables are not safe to use in Dash as they are not shared across processes.

To safely share data across processes (and prevent redundant computation), I recommend using Flask-Caching. Here is a tutorial: https://plot.ly/dash/plugins.

If events=[Event('interval-component', 'interval')] are used for updating multiple components, in what sequence does Dash execute the callback functions?

If the components don't depend on each other, then the updates happen all at once in parallel. Or, more precisely, all of the requests to update the data are made in parallel and the Dash server will process them in parallel across the number of processes that are available.

When you run an app with app.run_server(debug=True) (in development), there is only a single process and so only a single request is processed at a time. You can run Dash across multiple processes with something like gunicorn. With gunicorn, multiple server instances are forked and multiple requests can be processed at once. To use gunicorn, expose the server variable and then run the command line gunicorn name_of_your_file:server. That is:

my_app.py

app = dash.Dash(__name__)
server = app.server

$ gunicorn my_app:server

A more detailed guide can be found here: https://plot.ly/dash/deployment

[crazycodingcat]

Thanks a lot for the quick reply! Yes that makes sense. I'm going to try flask-caching and gunicorn as you suggested.

[RLstat]

I am actually want to do something similar, i.e. run a program every second and update the graph until certain condition is met (time lapsed, number of loops reached, no more new data to fetch, etc. ).

I found it not very easy to do, since I am not familiar with javascript or flask-cache.

I am wondering whether Dash will have callbacks that initiate for loops or while loops rather than just functions, since a function will only update a result or figure once upon a callback input change.

If there is a good way to do it within the current framework, can anyone show me an example or tutorial that I can learn from. Thanks!!

[chriddyp]

@RLstat - check out https://plot.ly/dash/live-updates

[renardeinside]

Hi,
I'm now trying to use Dash with multiple Interval elements (with different update time), using the solution with gunicorn, but seems that it's not working. Is there any solutions for multiple Interval?

[xgdgsc]

It seems dash only supports interval based update or update on reload. Would it be possible to have a feature to trigger the update by some function call or "watch" changes in a data object and redraw the table?

[alexcjohnson]

Server-driven updates is something we're thinking about, but it's got a lot of complications, particularly around multiple clients of the same app and how to handle clients who go offline for a while and then come back online (at which point they may be many updates behind).

The recommended way to handle this right now is to use a relatively fast interval along with some indication of the last time the front end was updated - a timestamp (server-generated!) or hash of the data, something like that - and if that's still up to date on the back end raise a PreventUpdate in the callback.

[xgdgsc]

Thanks!