Incubator-superset: Smaller JS package sizes!

[mistercrunch]

I think we've beaten the world record of biggest JS bundles. Let's contain this.

State of things:

$ du dist/*|sort -n -r | head
14640   dist/dashboard.13e463a9f8b8738180b3.entry.js
14032   dist/explore.248040abcbbaf60a59b9.entry.js
6072    dist/sqllab.ea495dee48ee855009f6.entry.js
2296    dist/addSlice.66d103d9c1275427d857.entry.js
2064    dist/profile.90e39d37579af32c6091.entry.js
2040    dist/welcome.41f4f4057ba02c52f350.entry.js
1072    dist/common.7ece420589d15408bce6.entry.js
520     dist/theme.ed0d61cb90e4574ee3da.css
216     dist/89889688147bd7575d6327160d64e760.svg
160     dist/dashboard.13e463a9f8b8738180b3.css

Ideas:

• break out a superset-visualizations npm package with sub packages for each visualizations sets, one for deckGL-related visualizations, one for nvd3-related, and any other big chunks separately. This will also pave the way towards a superset-embedables package
• the biggest win is to break out the deckgl related packages (luma.gl, mapbox.gl, deck.gl, ...)
• in the main app, visualizations bundles should be lazy loaded
• working viz out of explore and dashboard, should make those much leaner
• keep theme.js we still need that has standalone
• maybe add a common.js with things like React and commonly used utilities that we need across the board
• ...

[mistercrunch]

comments? ideas? @graceguo-supercat @williaster @GabeLoins

[kkalyan]

@mistercrunch unrelated, but is there a way for users to hide certain visualizations like deckGL?
Most common viz_types used by our users are table, line, bar and area.

[williaster]

I think the biggest win here will be with lazy loading, which would also address @kkalyan's comment. I agree there's no reason to force everyone to load every vis (I think another solution to this would be a more structured and vibrant ecosystem of vis plugins).

I will look into this more today/tomorrow (just had a big project land so have a bit more time for this)

[williaster]

I might have a go at a PR to add lazy loading for visualizations, sound okay @mistercrunch ?

[williaster]

think I have a proof of concept (this is just lazy loading the visualizations, could do more with mathjs, etc). dashboard + explore are cut in half on their own.

before

after

[vibinsv09]

@williaster This is great work and would dramatically help in optimizing Superset performance.
I was just wondering why my js file sizes are almost half than yours in the before screenshot

 du dist/*|sort -n -r | head
7664    dist/dashboard.4d5e5eea04a5c0ebdcd4.entry.js
7344    dist/explore.9f1172c4902e89a2db66.entry.js
3560    dist/sqllab.659eaa505425bb31afe8.entry.js
1288    dist/addSlice.70355f12808eb866abef.entry.js
1160    dist/profile.44aab22ce1d1646ac4b3.entry.js
1148    dist/welcome.dffbb04150e98261f5a2.entry.js
564     dist/common.99d345e775ca73f033e1.entry.js
312     dist/theme.e4d50476aef490a57f7f.css
108     dist/89889688147bd7575d6327160d64e760.svg
96      dist/dashboard.4d5e5eea04a5c0ebdcd4.css

cat version_info.json
{"version": "0.23.0dev", "GIT_SHA": ""}%     

[NZHEzreal]

@mistercrunch @williaster hello, bro. I have some questions about smaller JS package. May I have your contact？Thanks a lot.

[mistercrunch]

This is an open community, the conversation should take place here on Github or on the mailing list.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.