Realm-java: Realm is too slow on Android first call when compared to iOS

[abhi-scenedoc]

Goal

What do you want to achieve?
Faster return for realm.getInstance() call the first time it is executed

Expected Results

I am noticing the iOS Realm is much faster at returning the Realm Instance the first time it is called, significantly faster (with the same schema setup).

Actual Results

I am noticing it takes upwards of 1 second on the Samsung s8. On some devices it takes up to 8-9 seconds.

Steps & Code to Reproduce

Describe your current debugging efforts.
Please run this (https://github.com/abhi-scenedoc/realmTest) sample project that I have put together which displays the results (time taken is ms).

Code Sample

long startTime = System.currentTimeMillis();
Realm realm = Realm.getDefaultInstance(); //first time it's called after application is opened
long result = (System.currentTimeMillis() - startTime);
Log.d(REALM, "total time = " + result);

Version of Realm and tooling

Realm version(s): 3.7.2(testing) and 3.3.1 (in our production app)

Realm sync feature enabled:no

Android Studio version: 2.3.3

Which Android version and device: Samsung S8 - Android 7.0

[Zhuinden]

Any migration code or compact on launch?

[abhi-scenedoc]

None, no migration code or compaction.

[cmelchior]

Can you post your RealmConfiguration?

[abhi-scenedoc]

For the test project or our actual application?

[Zhuinden]

@cmelchior there are many classes, maybe it is schema validation? Although I think that is done by OS now.

I am curious about performance in 4.0.0-RC1

[nhachicha]

Hi @abhi-scenedoc Thanks for the test project I've installed your app locally, and I can confirm your observation.

I've run the app on a (Nexus 6P) and Nexus 5 Genymotion 5.1.0 API 22, and I can notice a 3-4X factor for the time to open the default Realm, between the version 3.7.2 and 3.6.0.

I tested with 3.6.0 since it corresponds to the latest update to Core AFAICT.

We'll try to investigate and see what caused the performance drop.

_Test results_

Realm 3.7.2 (Nexus 6P)
    1439ms
    1540ms
    1379ms
    1383ms
    1376ms

Realm 3.7.2 (Nexus 5 Genymtion 5.1.0 API 22)
    450ms
    477ms
    520ms
    449ms
    581ms

Realm 3.6.0 (Nexus 6P)
    287ms
    147ms
    132ms
    139ms
    128ms

Realm 3.6.0 (Nexus 5 Genymtion 5.1.0 API 22)
    72ms
    32ms
    36ms
    18ms
    44ms

[diegomontoya]

We have tested 3.6.0 vs 3.7.2 on our code and there is no performance difference for db open on several devices (using both migration plus auto-compact). Thankfully this serious regression is not affecting all users.

[abhi-scenedoc]

Hi @nhachicha, I notice you are getting average times much lower than I am getting on my nexus 6p and I was wondering if you are doing anything differently (both on 3.6.0 and 3.7.2).

Thanks,

[cmelchior]

This is a trace from my Nexus 6P.

In a recent release, we changed how we are doing schema validation from Java to instead building up a schema representation we can send to lower layers for validation. This has several advantages when it comes to detecting and reporting errors as it can be done in one go, but it might also have caused some performance regressions judging from the trace. At least there are a lot of array manipulation going on, that I'm kinda surprised as we should be able to efficiently generate those in the annotation processor

Furthermore, this change was in 3.6.0, so the different results between 3.6.0 and 3.7.2 is puzzling. Also, it seems the class loader is very slow which is yet another puzzle.

Bottom line. A few things stand out, but no real smoking gun yet. I'll dig deeper into it.

Thank you very much for the easy to use test project 💯 👏

[cmelchior]

Two observations so far:

• It isn't a surprise that the class loader seemed slow. We are creating the schema information as a static field, which means the class loader is responsible for creating it.

• It seems our OsObjectSchemaInfo holds a couple of internal Array lists that doesn't have a pre-defined capacity. On Android ArrayList starts with an initial size of 0 and increment when hitting 12 elements. This is causing a lot of internal array copies which are quite slow: https://android.googlesource.com/platform/libcore/+/fe39951/luni/src/main/java/java/util/ArrayList.java#82 We should fix that.

[abhi-scenedoc]

@cmelchior Thanks.

[cmelchior]

An interesting observation was that moving the code from the Async task to the UI thread cut the time in half. The threads in the Async Task Pool has pretty low priority, but it is still kinda surprising because there isn't a lot of other work going on which could have taken priority.

[abhi-scenedoc]

That is indeed interesting, however on slower devices the time it takes to getInstance() the first time is longer even on Async task. So moving this to main thread would not be ideal.

[Zhuinden]

Well you can always try Realm.getInstanceAsync() from the UI thread.

[beeender]

Another thing is, with @abhi-scenedoc 's sample project, we can reproduce the slow performance on two Nexus 6p devices, but none of my devices including a low end device made 5 years ago. The first getDefaultInstance() call with the sample project takes around 300 ms on that low end device.

maybe it is related with https://stackoverflow.com/questions/42389287/slow-code-on-nexus-6p

[abhi-scenedoc]

@beeender I have tried it with a samsung s8 which is a blazing fast device and I noticed 1050ms avg wait times.

[diegomontoya]

@cmelchior You have performance data of 3.7.2 plus various stage of your implemented optimizations in your pull request. But missing is the 3.6.0 data points. Is the snapshot currently now on-par with 3.6.0 or still slower?

https://github.com/realm/realm-java/pull/5404

[cmelchior]

There weren't any changes between 3.6.0 and 3.7.2 that should have affected this. So I didn't test that variant.

We tested a variety of devices and it seems that the Nexus 6P is far slower than other devices, especially compared to Xiaomi Mi5 and Huawei Honour 7 (which @beeender tested).

Right now a guess is perhaps device encryption is doing the slowdown, or perhaps the Nexus 6P is aggressively throttling background threads (moving it to the UI thread made a big difference).

But we are still looking into it.

[beeender]

@diegomontoya

From my testing, even without #5404 , schema initialization for 3.7.2 is faster on 3.6.0 on all my devices (Nokia X2, Mi5, Hornor 7). But i didn't save the benchmark data.

Also please notice, there are actually two separated cases for the schema initialization.

1. Init from empty file.
   in this case, with #5065, by pushing all the schema initialization works to the native level, the first time calling Realm.getInstance(), 3.7.2 (which contains #5065) should be around 40% faster than before. See benchmark data here https://github.com/realm/realm-java/pull/5065#issuecomment-320184013 the coldCreateAndClose()

2. The schema has been initialized before, and the realm file exists on the disk.
   in this case, the first time Realm.getInstance() will not do disk IO write anymore, instead, it will just validate the schema. by #5065 , the validation has been done done in the native code as well, which in theory will be faster. But i forgot to save the data.

So the key issue for this one is to figure out why it is slow for Nexus 6P and Samsung S8 especially.

[diegomontoya]

Nexus 6P uses 810 chip which has huge heating issues so the throttling is really very likely. But @abhi-scenedoc s8? That is really a mystery. Most recent phones actually boost power to cpu slightly in the first few seconds of an app launch so that user experience and app launch benchmarks are positively effected.

I don't know about you guys, but I am actually very happy to see this mystery unfold. =) Not everyday you get find an genuine wtf nut to crack. Godspeed everyone!

[abhi-scenedoc]

Hi Guys, any news on this?
Thanks,
Abhishek

[nhachicha]

We did a release to enhance performance a little bit https://github.com/realm/realm-java/pull/5404 have you tested it?
btw I tried to run your app again against different versions of Realm and I couldn't reproduce a perf regression (on a cold start).

4.1.1-SNAPSHOT
                    347
                    358
                    351
                    368

3.6.0
                    391
                    352
                    353
                    379

3.5.0
                    460
                    447
                    417

3.4.0 
                    552
                    402
                    525
                    532    

Also is getInstanceAsync an option for your use case?

[rajscet]

@nhachicha i am using version 3.7.2 and in Samsung s8 it taking 1 sec to launch app.
how much improvement in 4.3.2 ?

[abhi-scenedoc]

Any update on this? I need a more comparative result to Realm for iOS.

[jasper-chan]

+1

This is a very concerning issue for us to deal with. Are there any updates in regards to this? We have a major customer who have indefinitely postponed going live on our production because of this challenge.

[Zhuinden]

@jasper-chan are you sure you don't just have a migration with a transform() call in it executed on the ui thread...?

[abhi-scenedoc]

@Zhuinden the first getDefaultInstance() is on a background thread and that should ensure the migration is being performed in the background correct? I did update to realm 4.3.1 and I am seeing average times of 400ms on pixel 2 to getDefaultInstance() the first time and I am okay with this for the time being.

I have another concern, the Realm.compactRealm() call is proportional to the file size and I notice it takes approx. 1 second per 1mb of realm file to complete. I am thinking of alternatives to this process on app launch in order to make sure the db is compact, however even writeEncryptedCopyTo is taking the same amount of time (give or gave a few 100 milliseconds). Can you give me any suggestions as per how I can speed up login times for our users if compactRealm takes 1second/mb?

Thanks,
Abhishek

[cmelchior]

There isn't really a way to speed that up. Compacting require reading all the data on disk and writing it back down. Instead of compacting on startup, I would look into scheduling it at regular intervals when the app isn't running or when it is about to be closed.

[abhi-scenedoc]

@cmelchior Yea I agree that there isn't a way to speed up compaction. However I want to know if there is a way to perform auto compaction when a certain threshold is reached.

I will try setting up compaction when app is about to be closed or when the app isn't running at all (however the user properties are not available so I don't think I will be able to load the realm config for the user database) so it might not be possible to compact when the app isn't running at all.

[cmelchior]

You can add a custom CompactOnLaunchListener which gives you some more information the Realm file: https://realm.io/docs/java/5.0.0/api/io/realm/RealmConfiguration.Builder.html#compactOnLaunch-io.realm.CompactOnLaunchCallback-

[clementetb]

We have run the benchmark on the latest releases with a Xiaomi 9T Pro. We can see a considerable reduction for the first initialization call:

---------
3.7.2
---------
133ms 

---------
5.15.2
---------
159ms
176ms
151ms

---------
6.1.0
---------
138ms
186ms
110ms
113ms
190ms

---------
7.0.5
---------
74ms
67ms
58ms
52ms
75ms

---------
10.0.0
---------
86ms
103ms
37ms
59ms
55ms
104ms

Closing this issue, please reopen if you experience lag again on the first initialization.