Dash: Save state in url

[ppwfx]

Is there a way to save the state in the url? So that I can bookmark or share certain states?

Btw, f*cking nice framework! Love it!

[chriddyp]

It's not possible right now, but this would be a great feature. IIRC, this function that I wrote last year gets most of the way there:
https://github.com/plotly/dash-renderer/blob/cef2313478ee2c37b91b9ef773bf6991bc96240e/src/actions/index.js#L583-L613
We'd just need to hook it up to the config (so that you can turn the feature on and off) and test it.

Once the state is in the URL, you could probably restore the state in a callback with the new query paramater support in the dcc.Link component as implemented here: https://github.com/plotly/dash-core-components/pull/131

[ppwfx]

The way I understand it, callbacks are executed exclusively in the browser. Meaning, the page needs to be "reloaded" in order to get back to that state. So that when one first opens the page, the content isn't there yet.

Regarding that it would be nice if the backend could access the query.

If that's not possible, how could we trigger the callback?
Is there a DOM ready, we could attach a callback to?

[mjclawar]

@chriddyp Do you have any thoughts on reloading the page right at the start given a JSON-dumped store?

Client-side

The serialize function in dash-renderer does grab all of the necessary props, but I can't easily find an API in dash-renderer to fire off all of the redux actions at once to update the store on first render on the client -- unless that's completely the wrong idea.

Server-side

Another (perhaps terrible/infeasible) idea is to intercept the default layout and update all of the relevant pieces of state. e.g., if you knew that the 'values' in a slider were [2, 3] you could perhaps force that to be the values in the layout by:

1. Intercepting this route: https://github.com/plotly/dash/blob/master/dash/dash.py#L71
2. Check for a state query parameter (or modify from a GET to a POST request), which would be set by the client
3. If there is state data included with the request:
   
   
   
   • Modify the serve_layout method to patch in all of the desired layout values
   
   

[mjclawar]

Sketch of a working solution that uses a backend database to store the redux store in this gist. We're currently using this as of this morning to allow for PDF printing of the current page on an external puppeteer service based on https://github.com/alvarcarto/url-to-pdf-api. Storing the state purely in the URL does not work for a relatively large app, so we had to set up a database to hold the store and then use an id to retrieve it.

1. Set up a POST endpoint that takes the result of serialize(window.store.getState()) and stores in the database keyed by an externalStoreId (making sure to validate & sanitize the store), e.g., with jquery:

$.post('/my/dash/post/endpoint/', {store: serialize(window.store.getState())});`

This endpoint could either take the externalStoreId from the client, or create a unique one itself.

1. Override the server_layout method in dash.Dash to have some flag (in this case a cookie) that returns the original layout response if the flag does not exist. If the flag does exist (in our case it's a cookie with externalStoreId pointing to the store in a database, then update the layout and return that.

def serve_layout(self):
    external_store_id = request.cookies.get('externalStoreId', None)
    if external_store_id is None:  # We're good, just do the default layout
        return super().serve_layout()
    else:
        def _conn_str_pg() -> str:
            return 'my postgresql connection string here'
        store_data = None

        # SQL injection checks should go here            

        # store_data is a PostgreSQL JSON field
        query_str = "SELECT store_data FROM my_store_table WHERE store_id='{store_id}';".format(store_id=external_store_id)
        with psycopg2.connect(_conn_str_pg()) as conn:
            with conn.cursor() as curs:
                try:
                    curs.execute(query_str)
                except Exception as e:
                    print('ERROR ON', query_str)
                    conn.rollback()
                    raise e

                res = curs.fetchone()
                store_data = res[0]
                assert isinstance(store_data, dict), 'store must be dictionary!'

        if store_data is not None:
            # Update the layout with the values in store_data instead of the initial layout
            layout = self._update_layout_value(
                layout=self._layout_value(),
                store=store_data)
        else:
            raise ValueError('store data not found for {} {}'.format(APP_NAME, external_store_id))

        # Return the updated layout response
        return flask.Response(
            json.dumps(layout, cls=plotly.utils.PlotlyJSONEncoder), mimetype='application/json')

self._update_layout_value takes an iniital layout and a store and returns a new valid layout to use the first time the Dash application is rendered:

def _update_layout_value(layout, store: dict):
    assert isinstance(store, dict)
    for k, v in store.items():
        try:
            component_id, component_prop = k.split('.')
        except ValueError as e:
            raise ValueError(k, e)
        try:
            my_object = layout[component_id]
        except KeyError:
            print(component_id, 'not found')  # For when component ids are not yet in the layout
            continue
        setattr(my_object, component_prop, v)
    return layout

1. Add an after_request to the Flask server that checks if externalStoreId is a query parameter from the request

# Add a cookie when the user hits `?restoreFromExternalStore` that makes `_dash-layout` serve the right layout
@app.server.after_request
def update_store_cookie(response):
    # If we are requesting an external store id
    if request.args.get('restoreFromExternalStore', None) is not None:
        response.set_cookie('externalStoreId', request.args.get('restoreFromExternalStore'))

    # If we have gotten the layout, let's wipe the unnecessary store id
    if request.path == '{}_dash-layout'.format(app.config['routes_pathname_prefix']):
        response.set_cookie('externalStoreId', '', expires=0)

    return response

For the print operation (with additional credential steps excluded), the flow can be:

1. Client hits a print button which posts the current store the the dash server
2. Dash server validates the store, creates a unique externalStoreId and stores them in a database
3. Now a request to the Dash application with ?restoreFromExternalStore={myExternalStoreId} will add the proper cookie before serve_layout is called, forcing the layout to be updated with the store located at externalStoreId = myExternalStoreId in the database.

This feels well beyond what Dash should be doing. It's also probably not great to use the cookies for the externalStoreId between requests, since if you failed before hitting _dash-layout after loading the page the first time, you would get the external store the next time even if you did not request it.

But:

1. It works with relatively little code(!)
2. Perhaps a few helper methods would be nice, like the _update_layout_value?

[ppwfx]

I had another thought, what about:

(frontend) writing the state in the url of all inputs when an input is changed?
(frontend) propagating the state from an url into the inputs on page load. and the change of the inputs trigger a backend callback?

advantages:

• full control over response (e.g. caching) for app dev
• generation of urls that haven't been visited before

cons:

• slight delay in load time, as we have to wait for the callback

I guess the implementation should be fairly simple as only the initialisation and the change of values of an input need to be modified (and some details ofc).

This change is probably also downwards compatible.

[jtpio]

We have the same need (being able to share states via URL between people), and decided to try an implementation while staying in Python land.

Here is a gist that shows how to save the value of the components as query string parameters:
https://gist.github.com/jtpio/1aeb0d850dcd537a5b244bcf5aeaa75b#file-app-py

1. Whenever a component changes its state, recreate the query string parameters and update the URL
2. At page load, parse the URL and apply the state to the components in the layout

You will notice a little trick with the Location component to avoid infinite callback loop:

• For the input, href is used: https://gist.github.com/jtpio/1aeb0d850dcd537a5b244bcf5aeaa75b#file-app-py-L62
• For the output, search is used: https://gist.github.com/jtpio/1aeb0d850dcd537a5b244bcf5aeaa75b#file-app-py-L77

I also noticed that the page_load callback is actually fired off twice at page load, the first time with href=None but this was actually known before.

Only primitive values where taken into account, such as numbers or strings. For more complex values like lists we would need to recreate the corresponding Python object using for example json.loads('[1,2,3]').

Another solution is to encode the state as a json string, base64 encode it, and put the base64 string as a query string parameter, which also has the benefit of keeping the URL shorter:

import json
from base64 import b64encode
encoded = b64encode(json.dumps({'dropdown': ['MTL', 'NYC', 'SF'], 'slider': 2}).encode())

When reading the URL, use b64decode and json.loads.

Also to generate the callback that updates the URL, we can either iterate through app.callback_map to get the list of Inputs and States, or list them manually like in the example:
https://gist.github.com/jtpio/1aeb0d850dcd537a5b244bcf5aeaa75b#file-app-py-L78

Here is a little demo of the flow:

In the end it could work as a simple workaround for small use cases.

Again as said above it would be a great feature to have built-in Dash itself!

[eddy-ojb]

We are also looking for this feature for collaboration purposes. I am eager to push for dash at my firm and this could be a massive selling point if i can get one of these hacks to work, even better if its on the official Dash roadmap!

[manniL]

I'd love to see this on the roadmap!

[sjtrny]

@jtpio Thank you so much, your gist helped me greatly. I have one suggestion related to the following:

Only primitive values where taken into account, such as numbers or strings. For more complex values like lists we would need to recreate the corresponding Python object using for example json.loads('[1,2,3]').

If you have a list of strings as one of your incoming url parameter's then the value from parse_qsl will look something like ['a', 'b', 'c'], which will cause json.loads to fail as JSON requires double quoted strings. Two easy work arounds:

1. Use str.replace to map ' to " and then use json.loads
2. Use ast.literal_eval

[oegesam]

I implemented the base64+json trick, so if someone is also in need of setting arbitrary attributes or assigning lists (such in multi item dropdowns):

(also works nicely with tabs btw)

def apply_default_value(params):
    def wrapper(func):
        def apply_value(*args, **kwargs):
            if 'id' in kwargs and kwargs['id'] in params:
                kwargs[params[kwargs['id']][0]] = params[kwargs['id']][1]
                #kwargs['value'] = params[kwargs['id']][1]
            return func(*args, **kwargs)
        return apply_value
    return wrapper

def parse_state(url):
    parse_result = urlparse(url)
    query_string = parse_qsl(parse_result.query)
    if query_string:
        encoded_state = query_string[0][1]
        state = dict(json.loads(urlsafe_b64decode(encoded_state)))
    else:
        state = dict()
    return state

@app.callback(Output('page-layout', 'children'),
              inputs=[Input('url', 'href')])
def page_load(href):
    if not href:
        return []
    state = parse_state(href)
    return build_layout(state)

component_ids = {
    'tabs' : 'value',
    'tab1_multidropdown' : 'value',
    'tab2_datepicker' : 'start_date'
}

@app.callback(Output('url', 'search'),
              inputs=[Input(id, param) for id, param in component_ids.items()])
def update_url_state(*values):
    state = dict(zip(list(component_ids.keys()), list(zip(component_ids.values(), values))))
    encoded = urlsafe_b64encode(json.dumps(state).encode())
    params = urlencode(dict(params=encoded))
    return f'?{params}'

[lioneltrebuchon]

I am trying to extend @oegesam's implementation for element where we might be interested in multiple Inputs. For example dcc.DatePickerRange, or maybe if you want to access the style of an element to show that is has been clicked...

I am failing at the very stupid step of writing the "update_url_state" function. Indeed, I need to unpack a list inside a list, and this is not accepted as dash.Input. An idea could be to use another mapping, that maps different strings to the same component_id (e.g.: "my-date-picker-range1" and "my-data-picker-range2" both mapping to "my-data-picker-range") but that is dangerous and demands some overhead to the programmer...

Here a working example, where I cheat by just using the first element of the list in "update_url_state", so it does not properly update the dcc.DatePickerRange as would be wished for:

import dash
import dash_core_components as dcc
import dash_html_components as html
from urllib.parse import urlparse, parse_qsl, urlencode
from dash.dependencies import Input, Output
from datetime import datetime as dt
import json
from base64 import urlsafe_b64decode, urlsafe_b64encode

app = dash.Dash()

app.config.suppress_callback_exceptions = True

component_ids = {
    'dropdown': ['value'],
    'input': ['value'],
    'my-date-picker-range': ['start_date', 'end_date'],
    'slider': ['value'],
}

app.layout = html.Div([
    dcc.Location(id='url', refresh=False),
    html.Div(id='page-layout')
])


def build_layout(params):
    layout = [
        html.H2('URL State demo', id='state'),
        apply_default_value(params)(dcc.Dropdown)(
            id='dropdown',
            options=[{'label': i, 'value': i} for i in ['LA', 'NYC', 'MTL']],
            value='LA'
        ),
        apply_default_value(params)(dcc.Input)(
            id='input',
            placeholder='Enter a value...',
            value=''
        ),
        apply_default_value(params)(dcc.Slider)(
            id='slider',
            min=0,
            max=9,
            marks={i: 'Label {}'.format(i) for i in range(10)},
            value=5,
        ),
        html.Br(),
        apply_default_value(params)(dcc.DatePickerRange)(
            id='my-date-picker-range',
            min_date_allowed=dt(1995, 8, 5),
            max_date_allowed=dt(2019, 9, 19),
            initial_visible_month=dt(2017, 8, 5),
            end_date=dt(2017, 8, 25),
            start_date=dt(2017, 3, 25),
        ),
    ]
    return layout


def apply_default_value(params):
    '''
    Fills the default values based on a state dict **kwargs. Beware, if
    :param params:
    :return:
    '''
    def wrapper(func):
        def apply_value(*args, **kwargs):
            if 'id' in kwargs and kwargs['id'] in params:
                the_value_keys = params[kwargs['id']][0]
                the_values =  params[kwargs['id']][1]
                if type(the_values) is list:
                    if len(the_value_keys) == len(the_values):
                        for ii in range(len(the_value_keys)):
                            kwargs[the_value_keys[ii]] = the_values[ii]
                    else:
                        print('We could not properly map keys to values! Please fill in *all* values in component_ids.')
                else:
                    for ii in range(len(the_value_keys)):  # Writing all keys with the same default value.
                        kwargs[the_value_keys[ii]] = the_values
            return func(*args, **kwargs)
        return apply_value
    return wrapper


def parse_state(url):
    parse_result = urlparse(url)
    query_string = parse_qsl(parse_result.query)
    if query_string:
        encoded_state = query_string[0][1]
        state = dict(json.loads(urlsafe_b64decode(encoded_state)))
    else:
        state = dict()
    return state


@app.callback(Output('page-layout', 'children'),
              inputs=[Input('url', 'href')])
def page_load(href):
    if not href:
        return []
    state = parse_state(href)
    return build_layout(state)


@app.callback(Output('url', 'search'),
              inputs=[Input(this_id, param[0]) for this_id, param in component_ids.items()])
def update_url_state(*values):
    state = dict(zip(list(component_ids.keys()), list(zip(component_ids.values(), values))))
    encoded = urlsafe_b64encode(json.dumps(state).encode())
    params = urlencode(dict(params=encoded))
    return f'?{params}'


if __name__ == '__main__':
    app.run_server(debug=True)

That's the end result I'd like to be using:

@app.callback(Output('url', 'search'),
              inputs=[[Input(this_id, i_param) for i_param in param] for this_id, param in component_ids.items()])
def update_url_state(*values):
    state = dict(zip(list(component_ids.keys()), list(zip(component_ids.values(), values))))
    encoded = urlsafe_b64encode(json.dumps(state).encode())
    params = urlencode(dict(params=encoded))
    return f'?{params}'

[Thiebout]

I have slightly updated the update_url_state method to work with the DatePickerRange:

@app.callback(Output('url', 'search'),
              inputs=[Input(id, param[i]) for id, param in component_ids.items() for i in range(len(param))])
def update_url_state(*values):
    l = []
    idx = 0
    for k in component_ids.values():
        amount_to_take = len(k)
        l.append(values[idx:idx + amount_to_take])
        idx = idx + amount_to_take
    print(l)
    state = dict(zip(list(component_ids.keys()), list(zip(component_ids.values(), l))))
    encoded = urlsafe_b64encode(json.dumps(state).encode())
    params = urlencode(dict(params=encoded))
    return f'?{params}'

def apply_default_value(params):
    def wrapper(func):
        def apply_value(*args, **kwargs):
            if 'id' in kwargs and kwargs['id'] in params:
                the_value_keys = params[kwargs['id']][0]
                the_values = params[kwargs['id']][1]
                if type(the_values) is list:
                    print(' is list')
                    if len(the_value_keys) == len(the_values):
                        for ix in range(len(the_value_keys)):
                            kwargs[the_value_keys[ix]] = the_values[ix]
                    else:
                        print('We could not properly map keys to values! Please fill in *all* values in component_ids.')
                # kwargs['value'] = params[kwargs['id']][1]
            return func(*args, **kwargs)
        return apply_value
    return wrapper

component_ids = {'dataset1-dropdown': ['value'],
                 'dataset2-dropdown': ['value'],
                 'date-picker': ['start_date', 'end_date']}

[rdslkfjasdf]

How would this be possible if you had multiple apps?

I have 0.0.0.0:8000/dash1 and 0.0.0.0:8000/dash2. I can get it to work for one dash but not for the other. I tried including that code in both dash1 script and dash2 script but I run into error saying that duplicate callbacks aren't allowed. So I tried including it in index.py and changed component_ids.items() to dash1.component_ids.items()+dash2.component_ids.items() in the inputs for the callback, but this didn't work either.

Does anyone else have any idea?

[eddy-geek]

Here's my take on saving state to the URL, "v4" in this thread: https://gist.github.com/eddy-geek/73c8f73c089b0f998a49541b15a694b1

TL;DR: Encoded URL looks like: ?picker::start_date="2019-03-12"&picker::end_date="2019-03-19"&dropdown="LA"&input="foo"&slider=6

...which is displayed properly by FF and Chrome (Chrome refuses to display single-quotes).
One can remove the quotes completely but code to convert back int, list etc. becomes brittle.

_Background:_ I wanted the benefits of @Thiebout's aproach (support DatePickerRange, lists, etc.) but still wanted the URL as readable as @jtpio. Using literal_eval as @sjtrny mentioned.

Sample URLs to compare the different approaches:

• @jtpio: ?dropdown=LA&input=foo&slider=6
• @lioneltrebuchon: ?param= base64 of {"dropdown": ["value", "LA"], "input": ["value", "foo"], "slider": ["value", 5]}
• @Thiebout: ?param= base64 of {"picker":+[["start_date",+"end_date"],+[null,+null]],+"dropdown":+[["value"],+["LA"]],+"input":+[["value"],+[""]],+"slider":+[["value"],+[5]]}
• mine: ?picker::start_date="2019-03-12"&picker::end_date="2019-03-19"&dropdown="LA"&input="foo"&slider=6

I'm not sure this kind of approach is solid enough for inclusion "as-is" in Dash core:

• the cumbersome "one custom callback per page" could be lifted I guess
• but I would expect the built-in solution to work with multiple pages -- as @rdslkfjasdf pointed out it does not currently

.