Kepler.gl: Enable dynamic data loading & chunking by adding map panned/zoomed events

[ncthbrt]

Is your feature request related to a problem? Please describe.
Kepler seems excellent at working with large files, however leaves it as an exercise to the reader to partition global data sets into chunks by location. I would like to offer analysts the ability to not have to worry about this and instead have lazily data load when a user pans the viewport.

Describe the solution you'd like
Given that data can come from many different places, it might make sense to have some sort of generic event which is emitted when the bounding rectangle of the viewport changes. This would allow users to add/remove data at the appropriate times without increasing the complexity of the core kepler codebase.

[heshan0131]

@ncthbrt are you using the demo app or you are building your own application and connected kepler.gl to your backend?

for the former case, we are unlikely to support it, because the hosted app is a client-side only app without a backend. the only way to load data is by uploading static files.

For the latter case. You can subscribe to kepler.gl actions, and the @@kepler.gl/UPDATE_MAP is fired every time the viewport changes. By listening to that action in your reducer, you can dynamically request data to your backend

[ncthbrt]

Great to hear @heshan0131! Yes am interested in the latter use case. Just was struggling to find docs around that. Thanks!

[ricardoekm]

Hi @heshan0131,
What do you think is the best way to define the data to be lazy-loaded? I've noticed UPDATE_MAP event sends:

width: 1440
height: 408
latitude: 37.264430599659576
longitude: -94.85342784584381
zoom: 6.103121864276854

Somehow it's necessary to translate width, height, lat, long and zoom to a lat-long range that defines the points to be loaded and rendered. Is there a method in Kepler that would help to solve this equation? Or any other better way to know the lat-long range currently visible in the viewport?

Thanks.

[anshuman]

Somehow it's necessary to translate width, height, lat, long and zoom to a lat-long range that defines the points to be loaded and rendered. Is there a method in Kepler that would help to solve this equation? Or any other better way to know the lat-long range currently visible in the viewport?

@heshan0131 kindly answer this question.

[gautham20]

I'm evaluating Kepler gl for a web app I'm building, and the lack of support for this feature is my main concern.

Can someone also clarify this part? Currently, out-of-the-box, If I have a large dataset (GBs) stored in my server, this entire dataset should be transferred to the client-side for Kepler to render, and the feature described here is a fix for this. Am I right to assume this?

[anshuman]

@gautham20 yes you are right to assume this.

A crude workaround to the lack of this feature I used for my project is to use the crude solution provided by mapbox here.

Use {latitude, longitude, zoom} from mapState (provided by kepler.gl state), and calculate geographical distance covered by viewport using zoom (again from kepler.gl state) and crude measurements provided by mapbox. Make a bounding box using these information.

Use your API to serve data for the bounding box calculated above.

@heshan0131 it'd be great if this problem gets solved to provide accurate bounding box covered by viewport in kepler.gl state itself.