Incubator-superset: Integrate Jupyter model to superset

[vimalgoyal]

Hi,

First of all thanks to everyone who is contributing on Apache Superset, a really beautiful data visualization tool with the rich features.

Currently, I am working on a project where I have a situation where I need some guidance.

I have three systems, one has the database, another has jupyter notebook and third has superset.

First, we create the data model in R/Python using jupyter nb. Now we would like to run these data model on the database via superset so that we can visualize the data. And each time when we run the data model on data it should visualize the latest data from the database.

We know that;

1. We can directly connect superset with database and play with data.
2. or can run data model manually at the backend and generate the output data as csv and can visualize the result on the superset.

We can not go with the 1st approach because we have the separate users for both Jupyter and superset. Jupiter user cannot access superset to create the data model, slices, and dashboard. Also, we do not want to expose raw data with superset user and superset user should able to only see the graphs/charts generated from the output of the data models.

We can also not go with 2nd approach because every time when we want to see the latest data, we have to first run data model manually.

So is there any solution to my problem where I can run data models in the background whenever I reload the dashboard and I can see refreshed data.

Any lead/suggestion will be highly appreciated.

Thanks!

[TryHarder01]

What if at the end of your model running you INSERT back into your DB the info you'd want visualized?

If I'm not understanding exactly, could you give some idea about what you've visualizing in the notebook.

[akrabawi]

Using pandas to_sql() function you can directly sync your df to a database which is connected to superset.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[tooptoop4]

reopen

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[petacube]

this should not be closed. its important feature to have