Realm-java: App crashes with FORMAT_UPGRADE_REQUIRED while using Realm 7.0.0

[73sravanthi]

We have been using Realm from the past year, We recently upgraded the application from V6.1.0 to V7.0.0, post this upgrade we are witnessing the crash mentioned, we are using Kotlin for application development with Realm java and lastly, we have enabled DB encryption

Process: com.com.myapp.Application.qa, PID: 31558
java.lang.RuntimeException: Unable to create application com.myapp.Application: io.realm.exceptions.RealmFileException: Opening Realm files of format version 0 is not supported by this version of Realm (Database has an unsupported version and cannot be upgraded
Exception backtrace:
(/data/data/com.myapp.Application:/files/myapp.realm) in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 107 Kind: FORMAT_UPGRADE_REQUIRED.
at io.realm.internal.OsSharedRealm.nativeGetSharedRealm(Native Method)
at io.realm.internal.OsSharedRealm.(OsSharedRealm.java:175)
at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:251)
at io.realm.BaseRealm.(BaseRealm.java:137)
at io.realm.BaseRealm.(BaseRealm.java:104)
at io.realm.Realm.(Realm.java:163)
at io.realm.Realm.createInstance(Realm.java:499)
at io.realm.RealmCache.createInstance(RealmCache.java:507)
at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:473)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:414)
at io.realm.Realm.getDefaultInstance(Realm.java:407)

[cmelchior]

How are you opening the Realm?

[73sravanthi]

DB configuration :
RealmLivedataWrapper.init(context)
val config = RealmConfiguration.Builder()
.name(BuildConfig.REALM_FILE_NAME)
.modules(Realm.getDefaultModule(), DataRealmModules())
.encryptionKey(StorageKeyUtils.getKey(sharedPreferences, context))
.schemaVersion(BuildConfig.REALM_SCHEMA_VERSION)
.migration(DbMigration())
.build()
Realm.setDefaultConfiguration(config)

realmLivedataWrapper = RealmLivedataWrapper.Builder()
  .setRealmConfiguration(config)
  .build()

While navigating between the modules its crashing at
val realm: Realm = Realm.getDefaultInstance()

[cmelchior]

Weird. The error message seems to indicate that you are opening an older already existing file. Is that the case?

[73sravanthi]

It's crashing while navigating from one module to another after some modifications. We have used Realm v6.1.0. This issue is observed when upgraded to v7.0.0 .

[73sravanthi]

The same scenario is working fine with Realm v6.1.0

[73sravanthi]

Will there be any issue if we use following method in 7.0.0

val realm: Realm = Realm.getDefaultInstance()
realm.sharedRealm.refresh()
try {
realm.executeTransaction { realmInstance ->
realmInstance.copyToRealmOrUpdate(dataObjects)
}
} catch (ex: Exception) {
Log.d("DOWNLOAD_ERROR", "storeToLocal DB store EXCEPTION")
Log.d(this::class.java.simpleName, ex.message)
} finally {
realm.refreshAndClose()
}

[73sravanthi]

Hi.Any update on the above issue? We are facing the above crash with above error in multiple places .

[edgar-zigis]

I have noticed this error too. We have only upgraded realm version and did not touch schema itself.
Mostly it occurs on Android 10 devices. Most popular devices which failed to upgrade:

Samsung Galaxy A70
Xiaomi Mi9 SE
Nokia 5

Other devices seemed to work fine, but this is quite critical.

[ganfra]

Exactly the same issue on our side, reproduced on my Pixel 3 on Android 9.

[cmelchior]

@ganfra Can you share how to reproduce? If you want to keep it private you can email me at christian.[email protected]

[ganfra]

If you are brave enough you can test it on our open-source project: https://github.com/vector-im/riotX-android
Updating realm to 7.0.0, then after some initial launches it will crash with this issue.
(Also, not related at all, but we are having massive performance issues when db is getting big)

[ganfra]

So I created a test account where you will be able to reproduce the error:
Login: realm7
Password: realm7.0help

You should use the develop branch.

So here is how to reproduce

1. compile the app with realm 6.1.0 (in the matrix-sdk-android gradle file).
2. launch and log in with given credentials.
3. Compile the app with realm 7.0.0
4. Relaunch the app, use the app a bit.
5. kill the app and relaunch => it should crash. If not, kill and relaunch until it crashes

Tell me if you need more information :)

[ganfra]

@cmelchior : any chance you could look at it?

[cmelchior]

@ganfra Thank you for the project. I am able to reproduce the problem. We are looking into it.

A few things ruled out:

• It isn't a wrong encryption key causing this issue or lack of key completely.

[ganfra]

Did you have any lead on what could be wrong?

[patelnishantk]

@cmelchior Would appreciate any update on this. Unfortunately, we too face this issue after upgrading from v6.1.0 to v7.0.0. It has resulted into hundreds of crashes reported in Firebase in barely few hours.

[cmelchior]

@patelnishantk Unfortunately we still don't have a root cause for this.

[ppamorim]

We are experiencing the same issue with a single client.

[ppamorim]

@cmelchior We are going to extract the user database and analyse it to see if it's the realm running in the device that cannot open it or if the realm file is corrupted.

[ppamorim]

We got the Realm file and when opening it on Realm Studio, we get the very same error:

[edgar-zigis]

@cmelchior

[cmelchior]

@ppamorim Is that a file you are able to share with us? You can send it to christian.[email protected] to keep it confidential

[ppamorim]

@cmelchior For the moment, giving direct access, no. But I will try to find ways to share it with you. One thing I noticed is that the file is 707mb which is completely wrong. My superior told me that we can do remote access of the file.

[cmelchior]

Interesting. How do you know the filesize was wrong? But if we can find a way to see the layout of the file would be great.

[ppamorim]

@cmelchior Based on the realm file size. I opened the encrypted realm in a hex editor and I can see a bunch of zeros in some situations. I am not sure if it's part of the paging or something. I will send the layout of the file shortly.
Isn't the fix https://github.com/realm/realm-core/pull/3838 from realm-core 6.0.15 related to this issue?

[cmelchior]

It could be yes. We will be making a new release with this fix shortly. Probably tomorrow.

[ppamorim]

@cmelchior Is there any way to run tests against the realm file that I have here by using the realm-core tests with this fix in place?

[cmelchior]

Yes, you can build Realm Java against source by adding -PcoreSourcePath=<pathToCore> to Gradle when building it, but if the file is already "broken" that fix is not going to help from my understanding.

[ppamorim]

@cmelchior I would rather try to open on Realm Studio.

[ppamorim]

@cmelchior Isn't it the case of migrating the realm file version + encrypted data migration of a huge file? Is there any test against this situation?

[cmelchior]

We released Realm Java 7.0.2 which we believe should fix this. Please reopen if this still happens after upgrading to that version.

[cmelchior]

@ppamorim If you cannot upgrade a huge file due to size, it should just abort opening the file.

[ganfra]

Hi there, it's still crashing on our side...

[cmelchior]

Hi @ganfra Thanks for info. Do you know if it is "old" Realms already seeing this that keep seeing it or "new" ones that was recently upgraded?

[ganfra]

What do you mean by old and new?
I was using the app with realm 6.1.0 then I compiled with 7.0.2 and when lauching the app it crashes. I guess it's ok when there is no existent db.

[cmelchior]

@ganfra Is it something you can reproduce? We would be very interested in seeing the 6.1.0 Realm before the upgrade so we can figure out what exactly is going on

[ganfra]

I already gave you the info to reproduce, if you scroll up in the comments, you have the github project (riotx-android/element-android). It was crashing for 7.0.0, still there with 7.0.2 with the exact same procedure

[cmelchior]

Ah yes. Sorry :(

[ganfra]

No problem :)

[ganfra]

Not sure, but it looks like it doesn't crash if not using compactOnLaunch, so maybe look in this direction?
My file also went to 700MB but it was only 100Mb before updating to 7.0.2

[rorbech]

@ganfra Thanks for the input. I was not able to reproduce neither going from 6.1.0 to any of the 7.0.x. I also tried from the previous commit where cmelchior was able to reproduce it without luck, so might me data related.

[ganfra]

Did you try with the account I provided?

[rorbech]

Yes. I followed the steps of https://github.com/realm/realm-java/issues/6889#issuecomment-650180906

[ganfra]

Weird, but the issue still exists, I just retried with my personal account

[cmelchior]

I'm no longer able to replicate on the latest develop branch. That said, I do remember when I reproduced it that it seems like the upgrade it self is fine. The problem happens when you relaunch the app after it has been upgraded, which makes me wonder exactly what you do on startup (It is still a bug on our end though).

[ganfra]

So, might be related to compactOnLaunch like I said or not at all?

[jedelbo]

At some point there was a problem with the combination of encryption and compactOnLaunch. It was fixed just recently. Could this be what we are seeing here?

[edualonso]

@ganfra is this still an issue with 7.0.3?

[ganfra]

Tested right now, still crashing (done twice, crashed twice too)

[clementetb]

@ganfra Would you mind trying version 7.0.4, we released it today and it addresses several memory issues.

[finnschiermer]

@ganfra Trying 7.0.5 would be even better.

[ganfra]

Just tried 7.0.5, still no chance with it...

[jedelbo]

@ganfra I you have a file that producers this error when opened, we cannot fix this with a new version. If the file has a file format of '0' it cannot be opened. So what we need to focus on is how we ended up in this situation. Are you able to produce new files where this is a problem?

[jedelbo]

We have found the root cause for this error.

[ganfra]

Any idea when we'll get a new release with this fix? :)

[jedelbo]

Fix has been committed. It will take e few days for that to reach the Java binding.

[ppamorim]

@ganfra Be aware that already corrupted databases won't be restored (in my case). We had to create a special app to export critical data from our apps. Far from ideal and a bit humiliating. :( I would like to see the Realm team speaking more about it. @jedelbo Could you please refer to the PR? Was it in the core?

[jedelbo]

Be aware that the file is not corrupted.We just fail to read the file correctly. The issue was fixed in https://github.com/realm/realm-core/pull/3910.

[ppamorim]

@jedelbo I did export 3 databases + encryption keys and I could not open any of them on Realm Studio. All of them where around 700mb for an unknown reason. That's why I suspect that the file was corrupt.

[ganfra]

Same issue of 700mb size, but I did manage to see this issue before releasing the app, so no problem for me...

[jedelbo]

@ganfra so you see an unexpected increase in size? How big a difference is there compared to earlier?

[ganfra]

Generally the file is 150mb in this usecase. Multiple time after this crash it grows to 700mb as I said.

[ppamorim]

@jedelbo Could you please confirm if the latest version is capable of opening these large databases and if it's able to reduce the file size?

I saw that you recently did a change in the core but this version on this library is not out, is there any way to release this version anytime soon?

[jedelbo]

The Java team is working out getting the release out later today.

We are able to handle realms of any practical size. You can use CompactOnLaunch to reduce the size of a realm. However we are of course interested in keeping the size of the realm down, so we would be interested in investigating why you experience this growth in size. So would you be willing and able to share a 'before' and 'after' realm with us? You can share the files privately by mail ([email protected]).

[ppamorim]

I think @ganfra will be able to provide a database, I could not replicate the issue on my side and the data that I have cannot be shared.

[ppamorim]

@jedelbo Any update?

[ppamorim]

@cmelchior The version 7.0.2 didn't sort the issue. We are going to update it to version 7.0.5. The situation is bad.

[cmelchior]

We will be making a 7.0.6 release in the next few hours that should have a fix for this.

[jedelbo]

You should wait for v7.0.6. It should be just around the corner.

[prettyClouds]

Is there any kind of public pipeline we can monitor to get an idea of when things will release?

[cmelchior]

You can watch releases on Bintray here: https://bintray.com/realm/maven/realm-android-library

[cmelchior]

7.0.6 was just released which should fix this. Please reopen the issue if you still see it after upgrading.

[ppamorim]

@cmelchior The version 7.0.6 is still not available, any timeframe or workaround for the moment?

[cmelchior]

It should be available now.

[ganfra]

Ok so now I can't reproduce, looking good! Thanks for this long-awaited fix!

[ppamorim]

@ganfra Hi, were you able to open that 700mb realm file too?

[ganfra]

Didn't try, but I got a SEGFAULT 11 and a corrupted file just after, so shout victory too quickly...

[ppamorim]

@ganfra Very nice, imagine living in a world where the database of your application gets corrupted due to a bug.

[ppamorim]

@ganfra Some users were able to restore the app. I got one case where the app wasn't able to load the database.

[ganfra]

@ppamorim thanks for letting me know, but I just keep getting native crashes on some other parts