Realm-java: Incompatible histories. Expected a Realm with no or in-realm history

[raja-baz]

We get the following exception when trying to open the default realm:

FATAL EXCEPTION: main
Process: com.anghami, PID: 8169
java.lang.RuntimeException: Unable to create application com.anghami.AnghamiApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.anghami/files/default.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.anghami/files/default.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 252 Kind: ACCESS_ERROR.
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5973)
    at android.app.ActivityThread.-wrap3(ActivityThread.java)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1710)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:154)
    at android.app.ActivityThread.main(ActivityThread.java:6776)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1520)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1410)
 Caused by: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.anghami/files/default.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.anghami/files/default.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 252 Kind: ACCESS_ERROR.
    at io.realm.internal.SharedRealm.nativeGetSharedRealm(Native Method)
    at io.realm.internal.SharedRealm.<init>(SharedRealm.java:186)
    at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:239)
    at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:202)
    at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:298)
    at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:284)
    at io.realm.Realm.getDefaultInstance(Realm.java:281)
    at com.anghami.data.local.RealmHelper.call(RealmHelper.java:155)
    at com.anghami.data.local.RealmHelper.call(RealmHelper.java:148)
    at com.anghami.config.RealmConfig.initialize(RealmConfig.java:76)
    at com.anghami.AnghamiApplication.onCreate(AnghamiApplication.java:69)
    at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1032)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5970)
    at android.app.ActivityThread.-wrap3(ActivityThread.java) 
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1710) 
    at android.os.Handler.dispatchMessage(Handler.java:102) 
    at android.os.Looper.loop(Looper.java:154) 
    at android.app.ActivityThread.main(ActivityThread.java:6776) 
    at java.lang.reflect.Method.invoke(Native Method) 
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1520) 
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1410)

This realm doesn't use encryption(and we've never used it on the default realm) but another realm in the same app does use encryption.

I will attach the actual realm file that's causing this blowup in a bit

Version of Realm and tooling

Realm version(s): 3.5.0

Realm sync feature enabled: no

Android Studio version: Android Studio 3.0 beta 2

Which Android version and device: Android 7.0 running on a Samsung S8+

[raja-baz]

Note that trying to open the file using realm browser asks for an encryption key. I tried the one we're using for the other realm in the same app but didn't work

[raja-baz]

default.realm.zip

[cmelchior]

Where does the Realm come from and how is it created? It seems to be rather large, so I assume some data must have been put into it at some point?

[raja-baz]

This is the default realm for our app. Config is as follows:

      sDefaultRealmConfig = new RealmConfiguration.Builder()
          .schemaVersion(MigrationHelper.REALM_VERSION)
          .migration(new RealmMigration() {
            @Override
            public void migrate(DynamicRealm realm, long oldVersion, long newVersion) {
              MigrationHelper.migrate(realm, oldVersion, newVersion);
            }
          })
          .build();

      Realm.setDefaultConfiguration(sDefaultRealmConfig);

And yes, it has quite a bit of data in it. This comes from an app in active usage that started crashing on launch with the above exception at some point today. I installed a debug version of the app using the same signing as our release build and extracted the file that way, but I don't know the exact history. But it's been in active usage for at least 2 weeks and possibly more.

The app is a music streaming app, and we store data about users's playlists, downloads, song cache, etc... in realm so it can grow pretty large

[raja-baz]

Also, this seems relevant because of the reference to encryption and because the change occurred recently. We had a table called "Account" in the default realm which we moved to an encrypted realm. The migration code is as follows:

    final Account oldAccount = RealmHelper.call(new RealmCallable<Account>() {
      @Override
      public Account call(Realm realm) {
        Account result = Account.fetch(realm);
        if (result != null) {
          return realm.copyFromRealm(result);
        }
        return null;
      }
    });
    if (oldAccount != null) {
      RealmHelper.executeTransaction(new Realm.Transaction() {
        @Override
        public void execute(Realm realm) {
          realm.copyToRealmOrUpdate(oldAccount);
        }
      }, sAccountRealmConfig);
      RealmHelper.executeTransaction(new Realm.Transaction() {
        @Override
        public void execute(Realm realm) {
          realm.delete(Account.class);
        }
      });
    }

The code for call() and executeTransaction are as follows:

  public static <T> T call(RealmCallable<T> callable, RealmConfiguration configuration) {
    Realm realm;
    if (configuration == null) {
      realm = Realm.getDefaultInstance();
    } else {
      realm = Realm.getInstance(configuration);
    }
    try {
      if (Looper.myLooper() == null && !realm.isInTransaction()) {
        realm.refresh();
      }
      return callable.call(realm);
    } finally {
      realm.close();
    }
  }

  public static void executeTransaction(Realm.Transaction transaction, RealmConfiguration configuration) {
    Realm realm;
    if (configuration == null) {
      realm = Realm.getDefaultInstance();
    } else {
      realm = Realm.getInstance(configuration);
    }
    try {
      if (realm.isInTransaction()) {
        logTransactionInTransaction();
        transaction.execute(realm);
      } else {
        realm.executeTransaction(transaction);
      }
    } finally {
      realm.close();
    }
  }

Those are just wrappers around the common case of opening a realm, doing something and closing it properly(so we don't leave dangling realms)

sAccountRealmConfig is defined as follows:

      sAccountRealmConfig = new RealmConfiguration.Builder()
          .name("account.realm")
          .modules(new AccountModule())
          .encryptionKey(bytes)
          .build();

[raja-baz]

Note that the above code was tested and worked just fine on at least 10 different installs of the app. Just this one started exhibiting the behaviour shortly after updating

[cmelchior]

Looking into the file, it doesn't look encrypted, but something is definitely not right with it. What version of Realm did you upgrade from?

[cmelchior]

Also, to be clear. The upgrade was fine and the app could be opened, but it then started crashing some time after?

[raja-baz]

The realm version didn't change throughout, 3.5.0. The update was an app update. Also, the user in question informs me that the crash predates the code above. He tried the latest app update in hopes of fixing the crash but it didn't resolve it. He can't remember whether or not there was a crash before that that might have corrupted the realm though

[cmelchior]

Are you compacting the Realm anywhere?

[raja-baz]

Unless it does anything by default: no

[beeender]

@raja-baz Do you access same Realm from multiple-processes? And do you use the API Realm.deleteRealm()?

[raja-baz]

Neither

On Thu, Aug 31, 2017 at 6:39 AM Chen Mulong [e-mail]
wrote:

@raja-baz https://github.com/raja-baz Do you access same Realm in
multiple-processes? And do you use the API Realm.deleteRealm()?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/realm/realm-java/issues/5175#issuecomment-326180577,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABNP7o-6FWnJRQkVeoO3EyJTIxBYEQXoks5sdirRgaJpZM4PHDXY
.

[beeender]

Would it be possible that the user is actually using adb pull and push back the default.realm file? (like pull the Realm file, check with the Realm browser, change something then push it back while the app is still running). changing the file through any other way without Realm involved will damage the Realm file.

[raja-baz]

No not possible. Up until I got my hands on the phone, the version of the
app installed was non debuggable(installed from store beta) and the phone
isn't rooted

On Thu, Aug 31, 2017 at 7:27 AM Chen Mulong [e-mail]
wrote:

Would it be possible that the user is actually using adb pull and push
back the default.realm file? (like pull the Realm file, check with the
Realm browser, change some something then push it back while the app is
still running). change the file through any other way without Realm
involved will damage the Realm file.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/realm/realm-java/issues/5175#issuecomment-326186241,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABNP7uq-Ieu7MRkSfe4BxEBTXlUpPr2bks5sdjYjgaJpZM4PHDXY
.

[finnschiermer]

A first analysis of the realm file unfortunately provides no good leads. The 2 top level refs seem valid, in the sense that they are aligned correctly and point inside the file. The top arrays (metadata for group, free lists, top of history entre etc) that you can get to from the top level refs are however invalid - both have wrong magic values. Most user data appears to be intact.

with regard to the log statement: the message about different histories are not a reflection of the root cause. It is just the first inconsistency that is reported, so we can skip looking for problems specifically related to the histories.

[jollyjoker992]

I have similar issue at version 4.2.0 of Realm so I do not know What is the root cause. Can you show me all the causes may make this Realm issue ? Thank you.

[Zhuinden]

@jollyjoker992 based on a different issue I saw previously, it could be caused by multi-process compact

[jollyjoker992]

@Zhuinden Our app is single-process app, so I think It's a root cause. Are there another reason ?

[pkovalenko]

Also, I saw this crash for one user in Crashlytics console. Realm version 4.3.3

[clementetb]

Hi. Version 7 and upward of Realm had a significant updated underlying database engine that improves stability and performance. We recommend updating to version 10. Closing this issue as outdated please reopen if you experience it again.

[Zhuinden]

Version 10?

EDIT: oh, 10.0.0 (2020-10-15), congrats!