Realm-java: Fatal Exception: java.lang.IllegalArgumentException Illegal Argument: Invalid format of Realm file.

[mihirjoshi21]

Even with the latest version of Realm I am getting this crash in production builds. It is amusing that the rate I am receiving the crash is tripled with the latest version.

Crash link

[stk1m1]

@mihirjoshi21

Even with the latest version of Realm I am getting this crash in production builds. It is amusing that the rate I am receiving the crash is tripled with the latest version.

Crash link

Sorry for the inconvenience you're going through. In order to help you, we need more information on how and what leads to the issue. Could you please refer to our guide and add more information?

Secondly, your case might have been covered in 0.90.1 as it is mentioned in a #2488. Could you please try our latest build?

[mihirjoshi21]

I am pre-shipping the realm file with the application. It is made in v0.91.1 and I suppose there are no changes from that to version 1.0.0

[stk1m1]

@mihirjoshi21
There were more fixes made to 1.0.0. I'd ask you to give it a try. If you're using 0.91.1, there is a good chance that you can simply change the version number to use the latest. If you're in pre-shipping phase as you mentioned, isn't it all the more rational to try new one to see if this issue persists? 😉

And please, share us with a sample project to see if we can reproduce.

[kneth]

@mihirjoshi21 Did you try with latest version?

[kneth]

I'm closing the issue for now. Please reopen or create new if the problem occurs again.

[vagrant1991]

In my Application onCreate method :

RealmConfiguration config = new RealmConfiguration.Builder(this)
.name("myrealm.realm").build();
Realm.setDefaultConfiguration(config);

then in MainActivity oncreate method:

final Realm realm = Realm.getDefaultInstance();

the error occur:

Caused by: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.

at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
at io.realm.internal.SharedGroup.openSharedGroupOrFail(SharedGroup.java:95)

at io.realm.internal.SharedGroup.(SharedGroup.java:74)

at io.realm.internal.SharedGroupManager.(SharedGroupManager.java:49)

at io.realm.BaseRealm.(BaseRealm.java:86)

at io.realm.Realm.(Realm.java:135)

at io.realm.Realm.createAndValidate(Realm.java:233)

at io.realm.Realm.createInstance(Realm.java:214)

at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)

at io.realm.Realm.getDefaultInstance(Realm.java:160)

at com.example.vagrant.mytools.ui.MainActivity.onCreate(MainActivity.java:41)

at android.app.Activity.performCreate(Activity.java:5245)

at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)

at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2164)

at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2249) 

at android.app.ActivityThread.access$800(ActivityThread.java:141) 

at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1212) 

at android.os.Handler.dispatchMessage(Handler.java:102) 

at android.os.Looper.loop(Looper.java:136) 

at android.app.ActivityThread.main(ActivityThread.java:5113) 

at java.lang.reflect.Method.invokeNative(Native Method) 

at java.lang.reflect.Method.invoke(Method.java:515) 

at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793) 

at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:609) 

at dalvik.system.NativeStart.main(Native Method) 

And i'm sorry for my poor english :)

[beeender]

@vagrant1991 Can you reproduce this issue with your device?

[vagrant1991]

@beeender yes, i can reproduce it. i have tested two devices , HM 2LTE-CMCC Android 4.4.4 KTU84P , FRD-AL10 Android 6.0, two chinese popular mobilephone.

[beeender]

@vagrant1991 It could happen if you were using encrypted Realm before, and switch to a non-encrypted same name Realm without deleting the Realm file. Is this the case?

[vagrant1991]

yes, first time :

    byte[] key = new byte[64];
    new SecureRandom().nextBytes(key);
    RealmConfiguration config = new RealmConfiguration.Builder(this).encryptionKey(key)
    .name("myrealm.realm").build();

then i change to:

    RealmConfiguration config = new RealmConfiguration.Builder(this)
    .name("myrealm.realm").build();

this issue occur, now, i change the name:

    RealmConfiguration config = new RealmConfiguration.Builder(this)
    .name("mytool.realm").build();

this issue still occur

[mihirjoshi21]

Sorry for being 3 months late but I tried using the latest version (1.2.1) and this bug is still there. I even removed the encryption because of this and the inter process sharing bug but both are still occurring. Eventually, I am checking the exception so even if my app doesn't work it won't crash.
http://crashes.to/s/8c809755f34

[vagrant1991]

if i want to encrypt the database, then i change my mind, not to encrypt the database, or i want to change the realm file name, it sames i can't acommplish that

[vagrant1991]

@beeender i uninstall my apk, rebuild the project then reinstall my apk, the issue still here.

[beeender]

@vagrant1991 Just send your apk to [email protected] , i will take a look. It would be more helpful if you can just share your project.

[beeender]

@mihirjoshi21 When the problem happens, it mostly means the db file is corrupted. Do you delete the Realm file when this happens?

[mihirjoshi21]

I am pre shipping file with the application so chances of it being corrupt are minimal.

[beeender]

@mihirjoshi21 Are you using https://realm.io/docs/java/1.2.0/api/io/realm/RealmConfiguration.Builder.html#assetFile-android.content.Context-java.lang.String- to do copying of pre-shiping Realm file?

Which version of Realm did you use to generate the pre-shipping Realm file?

[mihirjoshi21]

I made it on v0.91.1 and using OutputStream to transfer file from raw to internal storage.

[beeender]

If there is a Realm instance opened (in one thread or another process) for some reasons, transferring with OutputStream could corrupt the Realm file. Will it be the case?

[vagrant1991]

@beeender I have send you a email with my project. i changed the realm file name, and checked the data/data/...../files directory, it seems it dose not generate the new mytool.realm file

in the dir:

Documents
a
bdpush_modeconfig.json
files
libcuid.so
libso
msclib
myrealm.realm
myrealm.realm.lock
myrealm.realm.management

[mihirjoshi21]

From the crash log this problem is only coming when realm file is accessed from the service.

[beeender]

@mihirjoshi21

• where did you do the copying? in the application or the main activity?
• Is your service in a different process? I guess not.
• Could your service started while the file is being copied?

[mihirjoshi21]

I am copying in the splash activity.
There is no separate process for service
File is copied inside Splash and then main activity is starting where I am calling the service.

[beeender]

According to the error trend, is there a peak than right after user update the apk? I am asking because it seems to be the same root cause with https://github.com/realm/realm-java/issues/2459#issuecomment-226520214

[mihirjoshi21]

Yep it can be the case but I have no analytics data to verify that.

[kneth]

@mihirjoshi21 @vagrant1991 Sorry for not getting back to you earlier - Realm Mobile Platform has kept us busy. It could be interesting to hear if you see the issue if you upgrade to the latest version.

[kneth]

@mihirjoshi21 @vagrant1991 Did you have a chance to upgrade?

[mihirjoshi21]

@kneth Testing the new build will update soon enough.

[kneth]

@mihirjoshi21 Please report back when you have any news.

[paisrikanth]

Fatal Exception: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(SharedGroup.java)
at io.realm.internal.SharedGroup.(SharedGroup.java)
at io.realm.internal.SharedGroupManager.(SharedGroupManager.java)
at io.realm.BaseRealm.(BaseRealm.java)
at io.realm.Realm.(Realm.java)
at io.realm.Realm.createAndValidate(Realm.java:248)
at io.realm.Realm.createInstance(Realm.java:228)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
at io.realm.Realm.getDefaultInstance(Realm.java:174)

[Zhuinden]

Realm corruption. Files do that sometimes, unfortunately.

[kneth]

@paisrikanth Can you provide some context? Do you use encryption? Which Realm version? Can you easily reproduce it?

[paisrikanth]

@kneth Yes I do use encryption, Realm version is 1.0.0. Yes I can easily reproduce this

Scenario

When I first login to my application in the filesdir my realm files are created such as

default.realm.enc
default.realm.enc.lock
default.realm.enc.management
realmKey

during logout I recursively delete the all files from the application filesdir.

Now, When the user is already logged in and if the application updates from the playstore and the user try to logout.. we delete the files from the filesdir and this exception thrown

[mihirjoshi21]

Now this is the error I am getting while reading the realm file in the latest version in some devices.

Bad Realm file header (#3). in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81 Kind: ACCESS_ERROR.

[carlbenson]

One of user is reporting a similar error as well, even with complete app reinstall. Starts happening after updating to Realm 2.0.2.

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.getdreams/com.dreams.start.views.StartActivity}: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.getdreams/files/default.realm': Realm file decryption failed. in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81 Kind: ACCESS_ERROR. at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2464) at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2524) at android.app.ActivityThread.access$900(ActivityThread.java:154) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1391) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:234) at android.app.ActivityThread.main(ActivityThread.java:5526) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) Caused by: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.getdreams/files/default.realm': Realm file decryption failed. in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81 Kind: ACCESS_ERROR. at io.realm.internal.SharedRealm.nativeGetSharedRealm(Native Method) at io.realm.internal.SharedRealm.a(SharedRealm.java:178) at io.realm.internal.SharedRealm.a(SharedRealm.java:155) at io.realm.au.a(RealmCache.java:124) at io.realm.Realm.n(Realm.java:210) at com.dreams.di.module.RealmModuleKt$realmModule$1$2.invoke(RealmModule.kt:39) at com.dreams.di.module.RealmModuleKt$realmModule$1$2.invoke(RealmModule.kt) at com.github.salomonbrys.kodein.f.a(factories.kt:44) at com.github.salomonbrys.kodein.b.a(Factory.kt:87) at com.github.salomonbrys.kodein.b.a(Factory.kt:75) at com.github.salomonbrys.kodein.b.a$c.invoke(KodeinContainerImpl.kt:79) at com.github.salomonbrys.kodein.q$c.invoke(KodeinContainer.kt:58) at com.dreams.start.presenters.StartPresenterImpl.onStart(StartPresenterImpl.kt:61) at com.dreams.start.views.StartActivity.onStart(StartActivity.kt:66) at android.app.Instrumentation.callActivityOnStart(Instrumentation.java:1238) at android.app.Activity.performStart(Activity.java:6302) at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2427) ... 9 more

[mihirjoshi21]

@carlbenson Yes it is occurring even after reinstall
http://crashes.to/s/706cc3e62fc

[cmelchior]

Are you by any chance using Google Play Services backup?

[mihirjoshi21]

No, I am not using Android backup service.

[carlbenson]

This could potentially be an issue with our user since it is automatically enabled for API 23 unless explicitly turned off. And we have android:allowBackup="true" in our manifest currently.

[cmelchior]

@carlbenson I have been reading up on Autobackup ( See #3694 ). After browsing the docs, I no longer think it is responsible for error. Not unless their implementation differ from the docs at least.

[mihirjoshi21]

Should I do android:allowBackup="false" in the next release?

[carlbenson]

We are turning auto-backup off in our next release anyway since we recreate all data in Realm from our backend and it also contains personal financial information that should not exist outside of our systems and the user's device.

[cmelchior]

@mihirjoshi21 No, there shouldn't be any need for that.

[cmelchior]

Folding this into #3064 since this error has been tracked in number of different issues.

[carlbenson]

A user having this problem did a re-install and _then_ removed all app-data which made the problem go away. Sounds like the only explanation is that data from a previous installation was copied from Google auto-backup.

[fobidlim]

In my case, problem solved by android:allowBackup="false"

[FarhanAhmad]

I am getting following error:
Read-only file system in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp

Realm was initialized in provider onCreate since that place was being called even before App->onCreate.
Error came when getting a defaultInstance in a service right after boot completed event.

[mihirjoshi21]

It is not resolved even after android:allowBackup="false". Crashes coming after v2.0.1.

Bad Realm file header (#3). in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81 Kind: ACCESS_ERROR.

http://crashes.to/s/706cc3e62fc

• pre-shipped realm file
• No Encryption
• Copied using assetFile()
• Realm File created in v 2.0.1

[mihirjoshi21]

Any update on this?

[kneth]

@mihirjoshi21 We would be happy to get a copy of your pre-shipped Realm file. You can send it to [email protected].

[mihirjoshi21]

Any update I already sent the Realm file along with the apk

[kneth]

@mihirjoshi21 Thanks, we will take a look.

[kneth]

@mihirjoshi21 I have installed the APK on an (x86) emulator and my phone (OPO), and I have seen a crash. Moreover, I opened the pre-shipped Realm file in the Realm Browser, and nothing really showed up.

So I haven't been able to come up with a case which reproduces your crash. I wonder if the pre-shipped Realm file can be damaged when it is copied the first time the app starts.

[mihirjoshi21]

@kneth You said you have seen the crash. So you reproduced it right?

[kneth]

@mihirjoshi21 Sorry, I didn't see any crashes.

The Google Play store installation process might damaged an asset but we have never been able to reproduce that. It might be useful at add a checksum in RealmConfiguration and verify it.

[morder]

In last build of our app, we have added compact at app start and we got many crashes to fabric

_Fatal Exception: java.lang.RuntimeException
Unable to create application org.iggymedia.periodtracker.PeriodTrackerApplication: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file._

realm version io.realm:realm-gradle-plugin:1.2.0
it is good idea to upgrade to latest version of realm?

[Zhuinden]

@morder I think you probably have multiple processes trying to compact the Realm (for example, a broadcast receiver with android:process=":remote") and they write random stuff into it.

Personally I added an increment/decrement to listen to the number of open activities, and compact when all of them are closed.

Realm 2.x does not support Blackberry devices yet, so I'd opt against it.

[diegomontoya]

@kneth Regarding your comment about google install/update corrupting assset file, was there a previous realm issue for this? We have not seen this happen yet but your comment alarmed us.

[kneth]

@diegomontoya I don't recall any reported issues on it but we have heard about crashes just at the same time as upgrades. It is virtually impossible to get triggered in our lab setup :-( It is more fair to call it a working hypothesis than fact.

[paisrikanth]

Process: com.clinicloud.app, PID: 15452 java.lang.RuntimeException: Unable to create application com.clinicloud.app.MainApp: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.clinicloud.app/files/default.realm.enc': Realm file decryption failed. in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 92 Kind: ACCESS_ERROR. at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5406) at android.app.ActivityThread.-wrap2(ActivityThread.java) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1545) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:154) at android.app.ActivityThread.main(ActivityThread.java:6119) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)

The application crashes on launch, If i clear the app data from the settings the application works however if I uninstall the app and reinstall the app crashes again

[kneth]

@paisrikanth Can you share an apk with us ([email protected]) so we can try?

[EllisShen]

@mihirjoshi21 and @vagrant1991
We got many crashes on production app too. Did your issue get resolved after upgrading to newer version of Realm? Our app is running on Realm 1.2 realm version io.realm:realm-gradle-plugin:1.2.0, the error log is basically same as yours:

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.mycurrentmessenger/com.mycurrentmessenger.MainActivity}: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2318)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2396)
    at android.app.ActivityThread.access$800(ActivityThread.java:139)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1293)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:149)
    at android.app.ActivityThread.main(ActivityThread.java:5257)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:609)
    at dalvik.system.NativeStart.main(Native Method)
Caused by: java.lang.IllegalArgumentException: Illegal Argument: Invalid format of Realm file.
    at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
    at io.realm.internal.SharedGroup.openSharedGroupOrFail(SharedGroup.java:95)
    at io.realm.internal.SharedGroup.<init>(SharedGroup.java:74)
    at io.realm.internal.SharedGroupManager.<init>(SharedGroupManager.java:49)

I originally think the issue was caused by the android:autoBackup behavior, however, crashes happened across multiple versions including Android 4.4, 5.0, 5.1 and 6.0. We have the encryption function turned on and have adapted increment / decrement listening of running activity, but still get this crash happened sometimes. (Fail rate is around 5% among users).

I've spent time on reading through related discussion of this issue but seems like there's no confirmed and proved solution to resolve this critical issue... Anyone can give me some advice?

[Zhuinden]

Do you by chance have android:process=":remote"? Because multiprocess access can cause trouble if both processes can do writes.

[EllisShen]

@Zhuinden, no... there's no android:process define in all manifest files. We have checked through the app, couldn't find multiple processes used in the app.

[kneth]

@EllisShen We have recently fixed a couple of bugs related to encryption, so my best advice is that your upgrade to Realm Java 3.0.0.

[mihirjoshi21]

@EllisShen @kneth

I am still experiencing lots of crashes. http://crashes.to/s/706cc3e62fc. The issues in my case came after Realm.init() was introduced. Now instead of defining Realm.init() in Application class, I am using it separately for Service and Activities. This reduced the crash but still, they keep on coming.

• No Encryption
• Not used android:process

[kneth]

@mihirjoshi21 How do you get your Realm instances? Your link indicates the crashes happen when you get an instance.

[mihirjoshi21]

@kneth Realm.getDefaultInstance();

[kneth]

@mihirjoshi21 Can you elaborate a bit on what kind of service it is? Is the service started from an activity?

[mihirjoshi21]

@kneth It's an unbound service started from an activity.

[kneth]

An unbound service might not close the Realm properly. I wonder if restarting the activity might mess up some internal state.

[mihirjoshi21]

The service will only work if the application is in the background. In some cases, there is an open Realm instance in an activity or fragment and user puts the app in the background so there might be an unclosed instance. In coming release I will check that is it really the case or the issue is also faced by first time user as well.

[kneth]

Thanks - I'll be looking forward to your update.

[EllisShen]

Sounds good. Which version will you plan to add into the service?
On Wed, Mar 15, 2017 at 9:09 AM Kenneth Geisshirt [e-mail]
wrote:

Thanks - I'll be looking forward to your update.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/realm/realm-java/issues/2984#issuecomment-286735909,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHv-B0xIIbCyYJykw9c-1gVW3na2pDJaks5rl-L0gaJpZM4IzpCB
.

[mihirjoshi21]

@kneth As it turns out the app is crashing as soon as it calls the getDefaultInstance().

In changelog of version 3.1.0 under bug fix -

Fixed a potential cause for Realm file corruptions (never reported).

Are you referring to this bug only or is it some other issue?

[kneth]

@mihirjoshi21 It is another issue: it was a bug which can only be triggered when sync is enabled.