Dash: not compatible with zappa for serverless deployments

[richiverse]

cc: @miserlou

So I ran the app locally for the first tutorial example here:
https://plot.ly/dash/getting-started

I tried deploying via zappa and got the following error:

(venv) rich@vbox:~/repos/explore_dash$ zappa deploy test                                                                                             
Calling deploy for stage test..
Creating zappa-permissions policy on ZappaLambdaExecution IAM Role.
Oh no! An error occurred! :(

==============

Traceback (most recent call last):
  File "/home/rich/.pyenv/versions/3.6.1/lib/python3.6/distutils/dir_util.py", line 70, in mkpath
    os.mkdir(head, mode)
FileExistsError: [Errno 17] File exists: '/tmp/user/1000/1498073842/dash'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/cli.py", line 2351, in handle
    sys.exit(cli.handle())
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/cli.py", line 456, in handle
    self.dispatch_command(self.command, stage)
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/cli.py", line 490, in dispatch_command
    self.deploy()
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/cli.py", line 650, in deploy
    self.create_package()
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/cli.py", line 1943, in create_package
    output=output
  File "/home/rich/repos/explore_dash/venv/lib/python3.6/site-packages/zappa/core.py", line 488, in create_lambda_zip
    copy_tree(temp_package_path, temp_project_path, update=True)
  File "/home/rich/.pyenv/versions/3.6.1/lib/python3.6/distutils/dir_util.py", line 159, in copy_tree
    verbose=verbose, dry_run=dry_run))
  File "/home/rich/.pyenv/versions/3.6.1/lib/python3.6/distutils/dir_util.py", line 135, in copy_tree
    mkpath(dst, verbose=verbose)
  File "/home/rich/.pyenv/versions/3.6.1/lib/python3.6/distutils/dir_util.py", line 74, in mkpath
    "could not create '%s': %s" % (head, exc.args[-1]))
distutils.errors.DistutilsFileError: could not create '/tmp/user/1000/1498073842/dash': File exists

==============

I removed the dash and dcc local files and it deployed fine. However, when I go to the AWS URL, I get the following error:

"{'message': 'An uncaught exception happened while servicing this request. You can investigate this with the `zappa tail` command.', 'traceback': ['Traceback (most recent call last):\\n', '  File \"/var/task/handler.py\", line 433, in handler\\n    response = Response.from_app(self.wsgi_app, environ)\\n', '  File \"/var/task/werkzeug/wrappers.py\", line 903, in from_app\\n    return cls(*_run_wsgi_app(app, environ, buffered))\\n', '  File \"/var/task/werkzeug/test.py\", line 884, in run_wsgi_app\\n    app_rv = app(environ, start_response)\\n', '  File \"/var/task/zappa/middleware.py\", line 66, in __call__\\n    response = self.application(environ, encode_response)\\n', \"TypeError: 'Dash' object is not callable\\n\"]}"

I get the following when tailing the logs:

[1498077793326] /var/task/plotly/tools.py:103: UserWarning:
[1498077793326] Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.
'Dash' object is not callable

If the dash and dcc files are necessary, is there a way I can rename them and reference them separately?

Also is there a way I can run the first tutorial in offline mode?

thanks!

[jackluo]

If you're running Plotly online it means that you haven't setup your API keys, you'll need to do so here: https://plot.ly/settings/api

However the default example should be offline by default, so I'm not sure what you did there. Try deploying the app with your credentials setup, otherwise it may be a secret key issue @chriddyp

[Miserlou]

Hi @richiverse, thanks for the ping, I was just thinking about trying this!

[richiverse]

I'm on Python 3.6 in this environment.

```#! /usr/bin/env python

-- coding: utf-8 --

from os import environ as env
import dash
import dash_core_components as dcc
import dash_html_components as html

app = dash.Dash()

app.layout = html.Div(children=[
html.H1(children='Hello Dash'),

html.Div(children='''
    Dash: A web application framework for Python.
'''),

dcc.Graph(
    id='example-graph',
    figure={
        'data': [
            {'x': [1, 2, 3], 'y': [4, 1, 2], 'type': 'bar', 'name': 'SF'},
            {'x': [1, 2, 3], 'y': [2, 4, 5], 'type': 'bar', 'name': u'Montréal'},
        ],
        'layout': {
            'title': 'Dash Data Visualization'
        }
    }
)

])

if __name__ == '__main__':
DEBUG = False if env['STAGE'] == 'prod' else True
app.run_server(debug=DEBUG)
```

I'm taking a look at dash.Dash() in the source now to see if I need to set something up there

[chriddyp]

[1498077793326] /var/task/plotly/tools.py:103: UserWarning:
[1498077793326] Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.

This is just a warning, it's not an error. You can ignore this for now.

[mcrowson]

https://github.com/plotly/plotly.py/blob/master/plotly/plotly/plotly.py#L50

It looks like when plotly initializes it checks local file permissions which will fail in AWS Lambda

https://github.com/plotly/plotly.py/blob/master/plotly/tools.py#L77

[Miserlou]

We would need any conf/install in our /var/task or /tmp.

[mcrowson]

That would get rid of the warning is all I think. There still is some underlying issue of the Dash object not being callable.

[mcrowson]

I believe the Dash object not being callable is simply its lack of a __call__ method. I added the following to the Dash class and it loads in lambda now. It just says "Loading...." however so this might not be the right fix.

```python
def __call__(self, args, *kwargs):
return self.server.__call__(args, *kwargs)
````

[jwkvam]

Does the new version of Dash use socket.io? I remember reading somewhere that socket.io wouldn't work with serverless.

edit: Nevermind, it appears to be restful now.

[mcrowson]

After adding the call method to dash I was able to get it on Lambda. I now see two other things.

1. Dash does not like to be anywhere other than root of a domain. If it is possible I'm not sure how to configure it. AWS API Gateway likes to host things by their stage. So dash is hosted for examples here: https://0xp1dyyij5.execute-api.us-east-1.amazonaws.com/dev. Because it didn't like the /dev I put it at a subdomain and it works partially in loading the dashboard from Lambda: https://dash.struce.com.

2. Some of the calls to /_dash-update-component return with a 403 error. I suspect it is having trouble authenticating or it doesn't like csrf for some reason, but I'm not sure how to address it.

I will leave the https://dash.struce.com up for a while so others can see its console output and network calls. The unminified version of react-dom are being used there for easier troubleshooting.

[Miserlou]

Dope work guys!

[chriddyp]

Dash does not like to be anywhere other than root of a domain.

For now, you can set url_base_pathname like app = dash.Dash(__name__, url_base_pathname='/dev/'). This syntax may change in the future, just a heads up.

Some of the calls to /_dash-update-component return with a 403 error. I suspect it is having trouble authenticating or it doesn't like csrf for some reason, but I'm not sure how to address it.

Try setting the server's secret key with something like app.server.secret_key = os.environ.get('secret_key', 'secret').

[mcrowson]

Hey good call on the secret key! Got Dash working on Lambda now!

Zappa attempts to return a callable Flask object. Would adding the call method (as described above) to the Dash class be something you would merge? @Miserlou may have a better suggestion for a way to handle this on the zappa middleware side, but I'm not seeing it.

[chriddyp]

Hm, weird. Why is it calling the Dash object? Can you just do something like:

app = dash.Dash(...)
server = app.server # the flask instance

and have it call the server instead?

[mcrowson]

So simple. Works like a dream, thanks. The only thing now that I can't seem to get working is the /dev/ part, but it is fine using a subdomain.

[chriddyp]

@mcrowson - did you try the url_base_pathname idea from https://github.com/plotly/dash/issues/22#issuecomment-310505552?

[mcrowson]

I did, no dice.

[fdroessler]

@mcrowson: apologies for the probably basic question but could this be used to run multiple dash(boards) as subdomains? I.e. if I have a few projects for which I want 1+ dash(boards), could they be deployed such that they are available using https://project1.domain.com, https://project2.domain.com ?

[mcrowson]

Yes. Those would each be a separate stage on your zappa deployment. I threw up the POC here: https://github.com/mcrowson/zappa_dash

If you want to get more details check out the Zappa docs: https://github.com/Miserlou/Zappa

[keeganmccallum]

I found a little workaround that might help anyone that doesn't want to use subdomains. This line of code lets you add your prefix into JUST the async requests made from js which solves the problem as long as you aren't using the option to serve locally:

app.config['requests_pathname_prefix'] = '/dev'+app.config['requests_pathname_prefix']