Realm-java: Android app freezes with 100K objects dataset

[jpmcosta]

Goal

I'm thinking of building an Android app using Realm.

Over time, its dataset could grow to become huge. So, to make sure I could work with Realm, I have built a simple test project with it.

My goal is to have multiple feeds to filter and group items.

Code (Kotlin)

Here are my test objects:

open class Feed(
        @PrimaryKey
        open var id: Long = 0,

        open var name: String? = "feed$id"
) : RealmObject()

open class Item(
        @PrimaryKey
        open var id: Long = 0,

        open var name: String = "item$id",

        open var feed: Feed? = null
) : RealmObject()

I have a RecyclerView.Adapter that receives a list of items RealmResults<Item> object. Here's the query:

fun Realm.getItems(feedId: Long): RealmResults<Item> =
        where(Item::class.java)
                .equalTo("feed.id", feedId)
                .findAllSorted("id", Sort.DESCENDING)

Test

My test project dataset:

• 1 Feed
• 100K+ Item objects

Issue

Whenever a new Item is created, the UI freezes for 3 to 5 seconds.

If I remove the sorting on the query, the freeze is not so long, but it still breaks. Also, my dataset will be a lot more complex and 100K is just a reference number.

Skipped 159 frames!  The application may be doing too much work on its main thread.
Window 'Window{2850f572 u0 io.test/io.test.activity.MainActivity}' spent 5048.6ms processing the last input event: MotionEvent(…)

Debug

With some method profiling, I noticed that io.realm.internal.TableView.nativeSyncIfNeeded() is the culprit.
It is too slow, and it runs in the UI thread.

Question

Is there any optimization I could do to handle a dataset of this size? Thanks.

Version of Realm

Realm version: 1.2.0

[Zhuinden]

Please try adding a field to Item called feedId, add @Index annotation, and try to query based on "feedId" instead of "feed.id".

[jpmcosta]

Update (objects and query)

open class Item(
        @PrimaryKey
        open var id: Long = 0,

        open var name: String = "item$id",

        @Index
        open var feedId: Long = 0
) : RealmObject()

fun Realm.getItems(feedId: Long): RealmResults<Item> =
        where(Item::class.java)
                .equalTo("feedId", feedId)
                .findAllSorted("id", Sort.DESCENDING)

Results

Thanks for the suggestion, it improves things. However, it still freezes with sorting and has 2 or 3 small breaks without it.

One of the worst breaks (with sorting):

Skipped 144 frames!  The application may be doing too much work on its main thread.
Window 'Window{367281c u0 io.test/io.test.activity.MainActivity}' spent 2462.9ms processing the last input event: MotionEvent(…)

I'm afraid that with other types of objects (that may reference each other) and with more complex queries, the issue could be even worse.

[Zhuinden]

Unfortunately me as a non-Realm person am out of ideas, maybe @cmelchior knows what's up

[Zhuinden]

@jpmcosta if there is one thing you could try, it's RealmRecycleViewAdapter from https://github.com/realm/realm-android-adapters and findAllAsync()?

[nhachicha]

Hi @jpsim
suggestion, have you tried async queries it call https://github.com/realm/realm-java/blob/master/realm/realm-library/src/main/java/io/realm/RealmResults.java#L909 on completion that will not invoke nativeSyncIfNeeded
Cheers

[jpmcosta]

@Zhuinden thank you very much, I think that solves my issue!
@nhachicha I was just updating with that info. :) (thanks!)

First, I tested with RealmRecyclerViewAdapter and findAllSortedAsync(), and it works well.

Then, I tested with my simple RecyclerView.Adapter that adds a RealmChangeListener:

class FeedAdapter(val feedItems: RealmResults<FeedItem>) :
        RecyclerView.Adapter<FeedAdapter.ViewHolder>() {

init {
    items.addChangeListener {
        notifyDataSetChanged()
    }
}

// Adapter methods.

}

It also works without freezing.

So, the trick is in using findAllSortedAsync() or findAllAsync().

I've even tested with more complex datasets, and it works fairly well when creating a single item. When adding 5K items in a single transaction it still freezes, but I don't think that has any impact in my case.

[jpmcosta]

Just a final note:

• When adding 5000K items in a single transition the app freezes due to io.realm.internal.SharedGroup.nativeAdvanceReadToVersion being called in the main thread.

I'm closing the issue, since I don't think that is relevant in my case.

[nhachicha]

cool, glad it worked!