Realm-java: Crash: table index out of range

[diegomontoya]

Realm version: Snapshot 20160218-1450004-77 build.
Devices: Adroid 4.0.4 and 4.2.2 devices

Got 5 users crashing on this exception:

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Table index out of range in io_realm_internal_Group.cpp line 190
at io.realm.internal.Group.nativeGetTableNativePtr(Unknown Source)
at io.realm.internal.Group.getTable(Unknown Source:199)
at io.realm.internal.SharedGroupManager.getTable(Unknown Source:88)

[beeender]

Would you please send the complete exception message to us? Privately through [email protected]. Thanks!

[diegomontoya]

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Table index out of range in io_realm_internal_Group.cpp line 190
       at io.realm.internal.Group.nativeGetTableNativePtr(Group.java)
       at io.realm.internal.Group.getTable(Group.java:199)
       at io.realm.internal.SharedGroupManager.getTable(SharedGroupManager.java:88)
       at io.realm.BaseRealm.getVersion(BaseRealm.java:409)
       at io.realm.Realm.getVersion(Realm.java:107)
       at io.realm.Realm.createAndValidate(Realm.java:250)
       at io.realm.Realm.createInstance(Realm.java:234)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:114)
       at io.realm.Realm.getInstance(Realm.java:199)

We are passing a realmConfig to retrieve a Realm instance.

[diegomontoya]

Of the 5 user. 4 are from Motorola MZ604+MZ601 devices with 4.0.4 os and 1 user from TCT One Touch 4016A with 4.2.2 os. The crashing devices had plenty of free memory and storage space so not appear to be resource related.

[beeender]

Were those device updated from an old version of realm? what is the previous version of Realm on those devices?

[diegomontoya]

@beeender Do not know the version state of the realm file on the effected devices.

This crash point is new in our radar and only appeared in our last release.

The only realm difference in our last app release is that we are performing a realm.compact(confg) before realm.getInstance(conf). We are only only compacting the db the first time realm is first opened (app restart) and not for subsequent opens.

[diegomontoya]

And the crash happened after the compact stage has successfully ran BTW.

[beeender]

Thanks for the information! Let me try if we can reproduce it from our side.

[beeender]

Similar issue reported before.
https://github.com/realm/realm-java/issues/2140#issuecomment-187728631

But I am not sure they are the same.
No, not the same issue.

[beeender]

@diegomontoya

The only realm difference in our last app release is that we are performing a realm.compact(confg) before realm.getInstance(conf).

Does this mean that the Realm.compact() happens when the app starts? If so, i think it is not related with my above comment.

[diegomontoya]

@beeender Realm.compact() happens before the first Realm.getInstance() of RealmConfig. It may happen on a UI thread or a background non-looper thread.

[diegomontoya]

@beeender We tweaked our app release which removed almost all Realm.compact() usage and the Moto 4.0.4 MZ601 MZ604 devices are still crashing at the same code points (just two of them) on a background non-looper thread opening up a Realm instance without any calls to Compact().

[beeender]

Just ordered a MZ601. Let's see if we can reproduce it after its arrival.

[beeender]

@diegomontoya I got the MZ601, but haven't been able to reproduce the issue yet.
If you have some hint about the reproducible steps, please let me know.
Also, can you please share the apk so i can try it on the device? If it might happen after apk updated, please share the old and the new apk as well. Maybe a test account if it is needed.

Thanks!

PS. It is a super slow device ...

[cmelchior]

We have fixed subtle cases in 1.0.0 which could lead to the crash reported here. We will close the issue for now, but If you experience the issue again with our latest release, please open an issue.

[imidhuntv]

This bug still exists. This happened in an old low end device.

CrashLog

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Table index out of range in io_realm_internal_Group.cpp line 190
       at io.realm.internal.Group.nativeGetTableNativePtr(Group.java)
       at io.realm.internal.Group.getTable(Group.java:199)
       at io.realm.internal.SharedGroupManager.getTable(SharedGroupManager.java:90)
       at io.realm.BaseRealm.getVersion(BaseRealm.java:427)
       at io.realm.Realm.getVersion(Realm.java:7120)
       at io.realm.Realm.createInstance(Realm.java:228)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
       at io.realm.Realm.getDefaultInstance(Realm.java:174)
       at com.talkoot.android.pubnub.PubNubObserverDelegateImpl$$Lambda$2.run(Unknown Source:1072)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
       at java.lang.Thread.run(Thread.java:838)

We are using realm 1.0.0.
Device : iris405
OS : 4.2.2
Rooted : yes
Free Space :300.64 MB
Free Ram : 61.21 MB

[diegomontoya]

Re-opening this due to consistent crashes on Realm 1.2.0 release.

1) New crashes on a few devices, All android 4.2.2.
2) All crashes are happening on a same-process android WakefulBroadcastReceiver class invoked by android scheduler/timer.
3) The BroadcastReceiver class receives the alarm event as intent every 15 minutes does some cleanup which requires opening db.

Caused by io.realm.exceptions.RealmError: Unrecoverable error. Table index out of range in io_realm_internal_Group.cpp line 185
       at io.realm.internal.Group.nativeGetTableNativePtr(Group.java)
       at io.realm.internal.Group.getTable(Group.java:199)
       at io.realm.internal.SharedGroupManager.getTable(SharedGroupManager.java:90)
       at io.realm.BaseRealm.getVersion(BaseRealm.java:490)
       at io.realm.Realm.createAndValidate(Realm.java:234)
       at io.realm.Realm.createInstance(Realm.java:214)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
       at io.realm.Realm.getInstance(Realm.java:178)

Updated: Not a long running Service but a WakefulBroadcastReceiver which receive alarm broadcasts from android os.

[diegomontoya]

List of crashing devices regarding my last post.

BLU DASH MUSIC 4.0 (4.2.2)
ONE TOUCH 4016A (4.2.2)
ONE TOUCH 4033A (4.2.2)
S5E (4.2.2)
ME173X (4.2.2)

[diegomontoya]

@cmelchior Not sure why the issue wasn't automatically re-opened. Can you re-open this? Thanks.

[Zhuinden]

Do you close the database when the operation is complete in a finally block?

[kneth]

@diegomontoya Is it reproducible in an emulator running 4.2.2?

[diegomontoya]

@kneth We are trying to reproduce this bug on a real Samsung 4.2.2 device but no luck thus far. @Zhuinden the dbs are closed in the future after open (~15seconds).

[kneth]

@diegomontoya We have seen crashes on one or two models before (completely separate issue) in the part, and it turned out to be vendors modifying Android for their models. So it can be very hard to reproduce :-(

[diegomontoya]

@kneth Did you guys find out what they modified (in the separate issue)? This info can perhaps help me think of workarounds.

[kneth]

@diegomontoya It was definitely unrelated (one vendor changes the OpenSSL library so we ended up building and including libcrypto).

[Zhuinden]

@kneth ah that was the fix in 0.85.0

[diegomontoya]

@kneth Thanks for the info. It doesn't apply here but I will add a multi-process protection via fcntl and see if that fixes this on 4.2.2 for our next app release. If the protection works, then it would mean another case of bad/unexpected behavior of the app launched in a separate process when it shouldn't have.

[kneth]

@diegomontoya We will be very happy if you can report the success - or failure - of your fix.

[diegomontoya]

@kneth

Based on some guesses from issue https://github.com/realm/realm-java/issues/2522 I am tracking a potential problem with android alarm scheduler executing alarm intents in a separate processId. This may cause corruption within realm <= 1.2.0 which does not support multi-process access.

The code is pretty straightforward. Acquire a fcntl file lock via java file channel api on app startup. Based on fcntl api, the lock is released when the process dies.

Non-fatal Exception: java.io.IOException: fcntl failed: EAGAIN (Try again)
       at java.nio.FileChannelImpl.basicLock(FileChannelImpl.java:123)
       at java.nio.FileChannelImpl.tryLock(FileChannelImpl.java:177)
       at java.nio.channels.FileChannel.tryLock(FileChannel.java:587)

Device: Android version
GT-I9060C 4.4.4
LG-D337 4.4.2
LG-D331 4.4.2
GT-N7105 4.4.2
SCH-I535 4.4.2
SPH-L710T 4.4.2
SM-G360HU 4.4.4

The above devices had reports of failure to acquire lock, each device failed multiple times while receiving an Alarm intent from android which is consistent with our repeating-alarm code.

Android doc only mentions that the intent is received by default, on the UI/main thread if a Handler is not provided. However, there is nothing in the doc saying it must be the same process of the app that is already running.

https://developer.android.com/reference/android/content/BroadcastReceiver.html#ReceiverLifecycle

Edit: adding more devices as we we get more log data.

[Zhuinden]

@diegomontoya you are correct, you should not directly write to the Realm from the BroadcastReceiver, because it is in a different process.

[diegomontoya]

@Zhuinden From real device logs, BroadcastReceivers (NetworkChange/Alarm) may be in a different process, but not always in a different process. Looks like the only safe way for any broadcastReceiver to do any real realm work is for it to receive the intent, use IPC/Messaging to send msg to a in-process Service which does the real work.

[Zhuinden]

@diegomontoya well in android:process=":remote" annotated broadcast receiver, it is always a different process

[diegomontoya]

@Zhuinden The problem is that android:process=":remote" is partially true on some android devices API <= 19 even when we don't set the :remote notation for BroadCastReceivers. We haven't seen this problem on API > 19, yet.

[cmelchior]

Interesting, I didn't know that. The error in the OP is consistent with an interprocess error as well.
Note that Realm itself is process safe, but the code we have running on top that manage the Realm schema is not. So one work-around could actually be to use a DynamicRealm. It will always directly access the Realm file.

[diegomontoya]

@cmelchior Closing this bug. We have not seen the crash resurface after implementting process lock check protection. Can safefully assume it is caused by OP, at least for my case with broadcast receivers.

[icoder20150719]

Can you guess what went wrong?