Realm-java: "Bad Realm File Header" error after compacting that threw "No space left on device"

[KimiChiu]

Goal

Don't let the realm file crashed

Expected Results

Realm file not crashed

Actual Results

Realm file crashed

Steps & Code to Reproduce

I have a user experienced a "No space left on device" error at first when compactRealm was called.
Then the user can never open my app due to the io.realm.exceptions.RealmFileException: Bad Realm file header (#1) error.

Code Sample

Exception io.realm.exceptions.RealmError: Unrecoverable error. write(): failed: No space left on device in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109

        Realm.init( context );
        RealmConfiguration realmConfiguration = new RealmConfiguration.Builder()
                .schemaVersion( REALM_DATABASE_VERSION ) // Must be bumped when the schema changes
                .migration( this ) // Migration to run instead of throwing an exception
//                .deleteRealmIfMigrationNeeded()
                .build();

        Realm.compactRealm( realmConfiguration );

        Realm.setDefaultConfiguration(realmConfiguration);

Version of Realm and tooling

Realm version(s): 3.0.0

Realm sync feature enabled: no

Android Studio version: IntelliJ IDEA 2017.1

Which Android version and device: API 23, Samsung SM-G532G

[bmeike]

@KimiChiu
Investigating. Will keep you posted.
-blake

[bmeike]

@KimiChiu
Can you reproduce this reliably? That would be helpful.

[KimiChiu]

It can be reproduced every time through these steps.

1. Inserts 50MB data into realm database
2. Fills the internal storage.
3. Open the app again
4. Crashed
5. Can never open the app because of the Bad Realm file header (#1) error.

[cmelchior]

Hi @KimiChiu
Sorry for the late response. I have created an issue with our Core team as this behavior doesn't sound right: https://github.com/realm/realm-core/issues/2796

Just to be clear. You are 100% sure that the No space left error was thrown while compacting the Realm?

[KimiChiu]

I'm pretty sure that the error was thrown while compacting the realm.
But I can't reproduce the No space left error again with Realm 3.5.0.
Now it only throws the Bad Realm file header while the space is not enough.
Not throwing the No space left error.
And I still need to clear App data to make it works again.
Here are the error logs with realm 3.5.0.

Uncaught exception is:
io.realm.exceptions.RealmError: Unrecoverable error. Bad Realm file header (#1) (/data/data/com.ihad.ptt/files/default.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 607
   at io.realm.internal.SharedRealm.nativeCompact(Native Method)
   at io.realm.internal.SharedRealm.compact(SharedRealm.java:367)
   at io.realm.BaseRealm.compactRealm(BaseRealm.java:602)
   at io.realm.Realm.compactRealm(Realm.java:1711)
   at com.ihad.ptt.domain.dao.realm.impl.RealmMigration.init(RealmMigration.java:29)
   at com.ihad.ptt.App.onCreate(App.java:101)
   at com.android.tools.fd.runtime.BootstrapApplication.onCreate(BootstrapApplication.java:370)
   at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1024)
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5403)
   at android.app.ActivityThread.-wrap2(ActivityThread.java)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1545)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:154)
   at android.app.ActivityThread.main(ActivityThread.java:6119)
   at java.lang.reflect.Method.invoke(Native Method)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)

08-17 18:26:01.360 17301-17301/com.ihad.ptt E/UncaughtException: io.realm.exceptions.RealmError: Unrecoverable error. Bad Realm file header (#1) (/data/data/com.ihad.ptt/files/default.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 607
  at io.realm.internal.SharedRealm.nativeCompact(Native Method)
  at io.realm.internal.SharedRealm.compact(SharedRealm.java:367)
  at io.realm.BaseRealm.compactRealm(BaseRealm.java:602)
  at io.realm.Realm.compactRealm(Realm.java:1711)
  at com.ihad.ptt.domain.dao.realm.impl.RealmMigration.init(RealmMigration.java:29)
  at com.ihad.ptt.App.onCreate(App.java:101)
  at com.android.tools.fd.runtime.BootstrapApplication.onCreate(BootstrapApplication.java:370)
  at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1024)
  at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5403)
  at android.app.ActivityThread.-wrap2(ActivityThread.java)
  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1545)
  at android.os.Handler.dispatchMessage(Handler.java:102)
  at android.os.Looper.loop(Looper.java:154)
  at android.app.ActivityThread.main(ActivityThread.java:6119)
  at java.lang.reflect.Method.invoke(Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)

[alessandrojp]

Hi,

I can also confirm that this happens when the device is running with no space left.
All devices reported to Crashlytics had 1% free space.

[cmelchior]

Note that we have a solution for this problem. It will be released as part of Realm Java 4.0. See https://github.com/realm/realm-core/pull/2852

[trr-amsiq]

@cmelchior
It also happens on devices with plenty of disk space have a look here.

[ironage]

@trr-amsiq according to the stack trace of your report, you are getting a different error that is not related to this issue.
Caused by io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1476395008 offset: 0 in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 252
Although the devices are not out of disk space, they cannot map so much contiguous memory (that's 1.4 GB according to your report). You should make sure you are not storing large binary data like sound or video data and also remember to close old versions and not pin them open for a long time. For more information you can check https://realm.io/docs/java/latest/#faq-large-realm-file-size

[trr-amsiq]

@ironage the link is pointing to the wrong stack trace. Here is the correct:

Fatal Exception: java.lang.RuntimeException: An error occured while executing doInBackground()
       at android.os.AsyncTask$3.done(AsyncTask.java:300)
       at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
       at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
       at java.util.concurrent.FutureTask.run(FutureTask.java:242)
       at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:818)
Caused by io.realm.exceptions.RealmFileException: Realm file is currently open in another process which cannot share access with this process. All processes sharing a single file must be the same architecture. (Incompatible lock file. Shared info version doesn't match, 771 10.) (/data/data/com.app/files/default.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 91
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.(SharedRealm.java)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:227)
       at io.realm.BaseRealm.(BaseRealm.java)
       at io.realm.BaseRealm.(BaseRealm.java)
       at io.realm.Realm.(Realm.java)
       at io.realm.Realm.createInstance(Realm.java:417)
       at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:348)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:281)
       at io.realm.Realm.getDefaultInstance(Realm.java:325)
       at com.app.Account.CollectionAccount$1.doInBackground(CollectionAccount.java:112)
       at com.app.Account.CollectionAccount$1.doInBackground(CollectionAccount.java:108)
       at android.os.AsyncTask$2.call(AsyncTask.java:288)
       at java.util.concurrent.FutureTask.run(FutureTask.java:237)
       at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:818)

[ironage]

@trr-amsiq that stack trace is also different than what this issue describes, but it is an unresolved issue we are investigating. I've posted your trace to https://github.com/realm/realm-java/issues/5198, please follow that issue and feel free to post any more information you find there.

[cmelchior]

The original problem have been solved in Realm Java 4.0.0. The other issue is being tracked in https://github.com/realm/realm-java/issues/5198. Closing