Realm-java: Paging or limit/offset feature planned?

[r-cohen]

Hey!
Are you guys planning a paging or limit/offset feature? I have a huge dataset which takes some time to process after fetch, and I really would like to prevent this by getting pages. If not, what workarounds are there for this?

Thanks.

[beeender]

You can:

1. Do the transaction in the background thread instead of the UI thread to avoid blocking UI. You might need Asynchronous Transactions.
2. You can split the big transaction to small pieces by yourself, something like:

for (int i = 0; i<data.size(); i += 500) {
realm.beginTransaction();
// Write 500 records
realm.commitTransaction();
}

[r-cohen]

Thanks for your answer, but that's not at all what I want to achieve. I am reading from realm a huge amount of data, the query doesn't take that much time to return the result though. After that, I need to go through this dataset to populate a view, and all this is done in a background thread. Meanwhile user doesn't have his data.

I could handle the paging on the dataset I got after the query, but the problem now is that I have to maintain and update the dataset and refresh it every time data in the database gets updated, which means adding, updating, deleting items while user is also paging through the data.

In my case, I have a background service which queries an API and updates the database on certain events.

For consistency and avoiding having a copy of the data to maintain in memory, I need to have paging on the realm side. I also welcome any advice to find a workaround for this.

[beeender]

I don't quite get the question... If you are concerning about the memory, Realm is always doing lazy loading. Even you query a huge result from the Realm, the actual data won't be loaded until you access the specific field.

[r-cohen]

Thanks for your reply.
To make it simple, I just have large amount in data in realm which I need to display to the user. How can I achieve paging?

[beeender]

That depends on which paging widget are you using i think? You probably could shared the same RealmResults across pages, but each page has its own startIndex. Then you can do some tricks in your adapter's getData(int index) like realmResults.get(startIndex+index)

Please notice that RealmResults will be automatically updated when your Realm file gets changes. So if everything is dynamically calculated in your adapter, you just need to add a change listener and call notify data changes in the listener.

[r-cohen]

Thanks but that's not going to work so well. Let me give you an example. Let's say you have a RecyclerView and you want to achieve an infinite scroll, so you want to display only the first 20 items, and as the user scrolls down, you load the next 20.

So every time I want to load a page, I query realm, get the results, and only add the items for that page.

But meanwhile, some data in one of the previous pages got added. So now, the page which I am displaying contains data from the previous page, and I have 2 times the same item displayed.

Let's say that some data got removed, I might even

[r-cohen]

... Miss some items and not display them at all.

So this is one of the many problems I have if I don't have paging.

[beeender]

Do you need to query the internet when doing the infinite scroll (not pull to refresh)?

[r-cohen]

No, querying the internet is done in a background service which takes care of updating the data in realm. Updating realm is also done on other events such as push notifications.

[beeender]

Why don't you just query everything you need for the RecyclerView and use the returned RealmResults as the data source? Then you don't have to query again when doing infinite scroll.

Please notice that Realm doesn't support fine grained notification for now, so you might not be easily refresh the specific index. See https://github.com/realm/realm-java/issues/989

Also there is an article about using Realm together with RecyclerView http://gradlewhy.ghost.io/realm-results-with-recyclerview/ which is mentioned in #861 .

[r-cohen]

Thanks for the article, it really helps. I didn't know you could bind RecyclerView directly to realm dataset and get updated when data in realm has changed. Will definitely try this technique.

[beeender]

cheers! :cake:

[squirtle1990]

Related:

The Monarchy library wraps Realm for use with the Paging library from Android Architecture Components.

https://github.com/Zhuinden/realm-monarchy/issues/18#issuecomment-423403021

[Zhuinden]

(although I'm having trouble setting up findAllAsync on a background looper thread for some reason)