I have some weird errors on deleting object, but I don't understand why it's happened.
Unfortunately, I don't encountered this issue myself, but according to crashlytics data, lot of users have this bug.
Stack Trace by firebase Crashlytics :
Non-fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Key not found in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_Table.cpp line 280
at io.realm.internal.Table.nativeMoveLastOver(Table.java)
at io.realm.internal.Table.moveLastOver(Table.java:315)
at io.realm.RealmObject.deleteFromRealm(RealmObject.java:117)
at io.realm.RealmObject.deleteFromRealm(RealmObject.java:89)
at com.samsic.contrattravail.data.factory.DocumentFactory$Companion.removeDocument(DocumentFactory.java:599)
at com.samsic.contrattravail.data.factory.DocumentFactory$Companion.deleteDocumentsById(DocumentFactory.java:592)
at com.samsic.contrattravail.data.factory.DocumentFactory$Companion.deleteDocuments$app_prodRelease(DocumentFactory.java:585)
at com.samsic.contrattravail.data.factory.DocumentFactory$Companion.deleteCarteVitale(DocumentFactory.java:414)
at com.samsic.contrattravail.data.factory.SalarieFactory$Companion.deleteSalarie(SalarieFactory.java:339)
at com.samsic.contrattravail.data.factory.SalarieFactory$Companion.deleteSalarieIfUnused(SalarieFactory.java:438)
at com.samsic.contrattravail.data.factory.SalarieFactory$Companion.removeUnusedTabletSalarie(SalarieFactory.java:387)
at com.samsic.contrattravail.data.helper.SynchroReferentielHelper$Companion$cleanDataOnLaunch$1.write(SynchroReferentielHelper.java:137)
at com.androidcore.realm.DatabaseWriteHelper$getWriterCompletable$1$1.execute(DatabaseWriteHelper.java:113)
at io.realm.Realm.executeTransaction(Realm.java:1489)
at com.androidcore.realm.DatabaseWriteHelper$getWriterCompletable$1.subscribe(DatabaseWriteHelper.java:112)
at io.reactivex.internal.operators.completable.CompletableCreate.subscribeActual(CompletableCreate.java:39)
at io.reactivex.Completable.subscribe(Completable.java:2309)
at io.reactivex.internal.operators.single.SingleFlatMapCompletable$FlatMapCompletableObserver.onSuccess(SingleFlatMapCompletable.java:91)
at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
at io.reactivex.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:67)
at com.androidcore.realm.AbstractDatabaseManager$getRealmEmitter$1.subscribe(AbstractDatabaseManager.java:161)
at io.reactivex.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:39)
at io.reactivex.Single.subscribe(Single.java:3666)
at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
at io.reactivex.android.schedulers.HandlerScheduler$ScheduledRunnable.run(HandlerScheduler.java:124)
at android.os.Handler.handleCallback(Handler.java:739)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:145)
at android.os.HandlerThread.run(HandlerThread.java:61)
Realm version(s): 7.0.0
Realm Sync feature enabled: No
Android Studio version: 4.0.0
Android Build Tools version: 29.0.3
Gradle version: 6.5
➤ Brian Munkholm commented:
[~finn.schiermer-andersen]Â Have you seen this before?
No.
The method SalarieFactory.deleteSalarie exist in my project since a long time, but it was not called as much.
Before, it was called only when the user ask a log out. Whenth user log out, we remove all "salaries" in application.
Now, it's called at each launch if has obsolete "salarié" object.
The other changes is the upgrade to realm 7.0.0.
@GaelCO How many is "a lot of users"? How big a % of users is it?
Can you possibly try to create tests for this yourself so you can reproduce what a lot of users seems to hit?
It's most likely that you are trying to delete the same object twice. Perhaps check those places to guard against double deletion.
About 30% of users encounter this issue. This users could have a lot elements to remove.
I don't think I try to delete a same object twice. Moreover, I get the object before delete if.
`
private fun removeDocument(bgRealm: Realm, documentId: Long) {
get(bgRealm, documentId)?.deleteFromRealm()
}
internal operator fun get(bgRealm: Realm, pkDocument: Long): Document? {
return bgRealm.where(Document::class.java).equalTo(Document.Attributes.ID, pkDocument).findFirst()
}`
And you are 100% sure removeDocument() is the only place you delete?
That seems strange...
It would be awesome if you in any way could reproduce this in testing as we haven't seen others have this issue I believe.
I have a similar issue with more than 2000 crashes in my app :(
Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Key not found in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsResults.cpp line 475
at io.realm.internal.OsResults.nativeEvaluateQueryIfNeeded(OsResults.java)
at io.realm.internal.OsResults.load(OsResults.java:664)
at io.realm.RealmResults.load(RealmResults.java:142)
at io.realm.RealmQuery.createRealmResults(RealmQuery.java:2397)
at io.realm.RealmQuery.findAll(RealmQuery.java:1778)
at com.moviebase.ui.home.shard.RealmListValuesHelper.get(RealmListValuesHelper.java:61)
@pstens @chrisbln Do you have any way to reproduce this? Or any hints as to what is happening before this occurs?
This occurred only after upgrading from 6.1.0 to 7.0.1.
The crash itself happened after downloading a bunch of entities from a remote API and then using Realm’s insertOrUpdate().
Unfortunately, I don’t have more information myself :/
This occurred only after upgrading from 6.1.0 to 7.0.1.
The crash itself happened after downloading a bunch of entities from a remote API and then using Realm’s insertOrUpdate().
Unfortunately, I don’t have more information myself :/
Yes, I have a similar case.
The crashes happen in the main thread when the user opens the app. Users mentioned that it crashes when they sync some data in a background task (could be a few hundred items), which starts in a separate thread at the launch time.
I tried to reproduce this without being able. @GaelCO or @chrisbln if any of you have access to a Realm or a demo project that are able to reproduce this, it would be very helpful.
I am in a huge trouble with this issue :( The app is running for many users, but when they delete the app data then the next startup it crashes. Mostly the mentioned issue, but also sometimes this one https://github.com/realm/realm-java/issues/6995.
The app uses Realm without big issues since 2016. Is it possible to downgrade to Realm 6?
Are they deleting the app data when the app is running? Because in that case, I wouldn't be surprised there are weird crashes. But if you close the app, delete the app data and reopen the data I would expect things to work.
I guess the big question is that how your code can get the above stacktrace if the Realm is empty? 🤔
In terms of downgrading, then you cannot downgrade an upgraded file, but if you delete the file, then it should be possible to downgrade. Not sure if that is an option for you?
When I run the app it is working perfectly.
The users are deleting the app data when the app is closed and I think the app crashes when syncing a lot of data in another thread at the launch time. As you already said, it will create a completely new default Realm file and this shouldn't be the problem. I will test the issue a bit more.
Hm, I think downgrading would end up in more problems and it is just running away from the real issue...
Thanks for your help :)
@cmelchior What is the meaning of "Unrecoverable error. Key not found"?
Each Object has a Key in the database. You get this error when you try to delete an Object that doesn't exist. But we can't rule out something fishy is going on, so a repro-case would be awesome and enable to either explain what you are doing wrong or determine there is indeed a bug.
We are seeing a similar issue (we created #6994), but when trying to read an item from the Realm.
We have a RealmProfileAction, which contains a RealmMember, and are getting the key not found error when trying to get the nested RealmMember.
In our codebase, we never delete the RealmMember so either something is happening inside Realm to delete the entry, or Realm is calculating this key differently at different times.
We are also seeing this error when trying to insertOrUpdate a RealmMember, as well as an error that a key already exists when running insertOrUpdate on a RealmMember.
We have not been able to reproduce ourselves but was effecting about 1% of our users over a wide range of devices before we downgraded realm back to 6.1.0, and this is now stopping us from upgrading until this issue can be fixed.
Seems that I could find the issue. The app has executed around 2200 realm transactions.
The transaction was similar like:
realm.executeTransaction {
val managedMovie = realm.copyToRealmOrUpdate(movie)
realm.where(RealmMediaWrapper::class.java).findAll().forEach {
it.setContent(managedMovie)
}
}
I still have not been able to reproduce by iterating a lot of creation/updates with cross-reference updates.
Are there any patterns on the affected devices, e.g. 32/64bit, specific brands, etc.?
According to Google Play, the top devices it is reporting for this crash are mostly Smasnugs, with some Huawei, Oppo, Xaomi, Byeond and OnePlus devices.
Seems that I could find the issue. The app has executed around 2200 realm transactions.
The transaction was similar like:
realm.executeTransaction { val managedMovie = realm.copyToRealmOrUpdate(movie) realm.where(RealmMediaWrapper::class.java).findAll().forEach { it.setContent(managedMovie) } }
Hmm, that's not what I'm doing and I have the crash in findAll()
like @chrisbln. See #7005
It's the top issue and started when updating Realm from 6.1.0 to 7.0.0
Seems that I could find the issue. The app has executed around 2200 realm transactions.
The transaction was similar like:realm.executeTransaction { val managedMovie = realm.copyToRealmOrUpdate(movie) realm.where(RealmMediaWrapper::class.java).findAll().forEach { it.setContent(managedMovie) } }
Hmm, that's not what I'm doing and I have the crash in
findAll()
like @chrisbln. See #7005It's the top issue and started when updating Realm from 6.1.0 to 7.0.0
The common denominator is it all started with 7.0.0 and all of you are deleting objects in some way or form, that's why we are folding all these issues under the same ticket.
Mainly this Issue of key not found is occurring on Google Pixel 3a XL Android 10 & Samsung Galaxy A51.
Seems that I could find the issue. The app has executed around 2200 realm transactions.
The transaction was similar like:realm.executeTransaction { val managedMovie = realm.copyToRealmOrUpdate(movie) realm.where(RealmMediaWrapper::class.java).findAll().forEach { it.setContent(managedMovie) } }
Hmm, that's not what I'm doing and I have the crash in
findAll()
like @chrisbln. See #7005
It's the top issue and started when updating Realm from 6.1.0 to 7.0.0The common denominator is it all started with 7.0.0 and all of you are deleting objects in some way or form, that's why we are folding all these issues under the same ticket.
The bug still exists for a few users even when I reduce the transactions.
We just released 7.0.2 with a number of fixes. We have not yet been able to reproduce this particular bug, we there is indications that this will have been fixed by 7.0.2. So please upgrade. We are definitely interested in hearing if this issue is still around after 7.0.2
@cmelchior Thank you very much! 😃 I will release an updated version and give you some feedback as soon as possible.
I see this with 7.0.2 as well
The same error:
io.realm.exceptions.RealmError: Unrecoverable error. Key not found in io_realm_internal_OsResults.cpp line 144
I get an error when I extract data from the database.
Realm: 7.0.1
Realm Sync feature enabled: No
Android Studio version: 4.0.1
Which Android version and device(s): Android 8.1 Xiaomi
The issue still exists with version 7.0.2
We are soon going to release version 7.0.3 with some fixes. Unfortunately, we still have not been able to reproduce this particular issue but have been fixing a number of other problems that could have caused it. So please upgrade your dependencies and let us know if the problem is still present.
Thanks for the fast update, but the bug still exists.
Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Key not found in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_OsList.cpp line 124
at io.realm.internal.OsList.nativeGetRow(OsList.java)
at io.realm.internal.OsList.getUncheckedRow(OsList.java:56)
at io.realm.RealmModelListOperator.get(RealmModelListOperator.java:1484)
at io.realm.RealmList.get(RealmList.java:443)
at io.realm.RealmList$RealmItr.next(RealmList.java:1157)
at com.moviebase.service.core.model.media.MediaModelKt.toMediaIdSet(MediaModelKt.java:45)
@chrisbln I don't assume this is something you can reproduce? Or do you somehow have access to a Realm that triggers this behavior?
If not, can you share some more information about what your code is doing there. Like what does MediaModelKt
look like. What is the type of the list (It looks like a list of other models, which? and how does the model class look like?
Also, in what context is toMediaIdSet
being called. Is it running from the UI thread, a worker thread or somewhere else?
If you want to keep that information private you can send it to christian.[email protected]
Unfortunately, I don't have access to the user content and I cannot reproduce it. This bug occurs only for a few users.
Here is the realm class:
open class RealmHiddenItem(
override var mediaId: Int = MediaId.INVALID,
var mediaType: Int = MediaType.GLOBAL_ALL,
var title: String? = null,
var releaseDate: String? = null, // LocalDate in UTC
override var posterPath: String? = null,
var addedAt: String? = null // LocalDateTime
) : RealmObject(), RealmMedia {
That is what the class did:
private val hiddenItems: RealmResults<RealmHiddenItem> = hiddenRepository.getAllHiddenItems(mediaType)
init {
hiddenItems.onChange {
hiddenSet = it.toMediaIdSet()
notifyDataSetChanged.postValue(true)
}
}
Util methods:
fun Collection<Media>?.toMediaIdSet(): Set<Int> = if (this == null) emptySet() else map(Media::mediaId).toHashSet()
fun <E> RealmResults<E>.onChange(change: (RealmResults<E>) -> Unit) {
val listener = RealmChangeListener<RealmResults<E>> { change(it) }
addChangeListener(listener)
}
@chrisbln We have just released version 7.0.4 which includes more fixes regarding memory management. Please upgrade your dependencies and let us know if the problem is still present.
We just released 7.0.5 with fixes which could also lead to "Key not found" errors.
Presumably fixed by https://github.com/realm/realm-core/pull/3893
Great news.
Normally, I publish a new version during next week. So, I let you know if it's fixed this problem in my app.
I have never managed to reproduct, so I can't validate the fix without publishing.
This issue still occured with version 7.0.5.
In my case, this problem does not block my users, because I have try/catch the crash and the offending code is used to delete some obsolete data.
For information, the primary id of my "document" object is "Long?".
I have made a project with a similar model object which let create a lot of objects and delete them in similar condition, but I have never reproduct this issue myself, even in release mode.
I can't find the cause of this error.
@GaelCO Thank you for reporting back. Sounds like we still have a bug to find.
I have reproduce this issue in an "test" app.
Step to reproduce :
The original problem seems to have been fixed in realm 7.0.3, because if I have make directly the realm upgrade from 6.1.0 to 7.0.3 and above, I don't encountered the issue.
If you want, I can give you my test project.
I just want to clean it up before to remove some parasitic code related to different tests.
RealmTest.zip
In the zip, you can found a sample project which lets reproduce this issue, if you follow the previous step.
Is this consistent with the bug being fixed, but corruption introduced into the realm files (by pre 7.0.3 versions) still being present?
@GaelCO Thank you very much for the test project. I tried to reproduce using your project but so far it seems to work as expected. But I'll continue trying to see if I can trigger it.
Can you perhaps give an exact list of the steps you did? Also did you use a emulator or a real phone?
Note that 7.0.6 was just released
Hi,
I use an emulator for my test (API Level 30)
My step (same that previous)
-> delete old existing app on device
-> change in gradle realm to 6.1.0
-> sync project and install on device
-> click on create button to create 1000 objects. You can try with more created data. When I have test, I had the impression that the volumetry had an influence.
-> change in gralde to 7.0.2
-> sync project and install on device
-> click on delete
=> As the error is catch, the app not crash and you can see the error in log (error mode)
Assigning to @cmelchior as he is the one trying to make progress at the moment. Assign back to core if a bug gets out in the open.
Hi @GaelCO. Has this issue been fixed in the latest release?
Hi, This issue still exist with 7.0.8.
The data seem partially corrupt. I can get the object, but can't delete it.
@finnschiermer and @jedelbo I think this answers the question asked in https://github.com/realm/realm-java/issues/6992#issuecomment-694192923 :
Hi, This issue still exist with 7.0.8.
The data seem partially corrupt. I can get the object, but can't delete it.
@GaelCO would you be able to share a realm file that exhibits this behavior? You can send it privately to jorgen.[email protected].
Hi,
I send a file with this behavior. This file contains fake data.
I generate with the test project I share previously.
Besides, I am surprised that you do not reproduce the problem with it.
Out of curiosity, I tried a "deleteAll". I have got an error too. It's not exactly the same (see the track).
After that, each deleteAll conclude to this error, but my other delete passed.
But, if I add new objects and clean it, I get the first error again. It looks like an index problem.
io.realm.exceptions.RealmError: Unrecoverable error. Key not found in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_Table.cpp line 190
at io.realm.internal.Table.nativeClear(Native Method)
at io.realm.internal.Table.clear(Table.java:262)
at io.realm.BaseRealm.deleteAll(BaseRealm.java:741)
at io.realm.Realm.deleteAll(Realm.java:145)
at com.gco.realmtest.MainActivity$deleteAllCompletable$1.write(MainActivity.kt:114)
at com.gco.realmtest.realm.DatabaseWriteHelper$getWriterCompletable$1$1.execute(DatabaseWriteHelper.kt:39)
at io.realm.Realm.executeTransaction(Realm.java:1489)
at com.gco.realmtest.realm.DatabaseWriteHelper$getWriterCompletable$1.subscribe(DatabaseWriteHelper.kt:38)
at io.reactivex.internal.operators.completable.CompletableCreate.subscribeActual(CompletableCreate.java:39)
at io.reactivex.Completable.subscribe(Completable.java:2309)
at io.reactivex.internal.operators.single.SingleFlatMapCompletable$FlatMapCompletableObserver.onSuccess(SingleFlatMapCompletable.java:91)
at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
at io.reactivex.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:67)
at com.gco.realmtest.realm.AbstractDatabaseManager$getRealmEmitter$1.subscribe(AbstractDatabaseManager.kt:73)
at io.reactivex.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:39)
at io.reactivex.Single.subscribe(Single.java:3666)
at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
at io.reactivex.android.schedulers.HandlerScheduler$ScheduledRunnable.run(HandlerScheduler.java:124)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.os.HandlerThread.run(HandlerThread.java:67)
I now have a good lead on this.
Same bug here. Is the realm-core fix already affect a Java release ? If not, what/when will be this released out (Sorry I do not have deep knowledge of or release process) ?
Thanks
@afaucogney The fix hasn't been released yet on realm-java. It will be available in the next release.
Any schedule for the next release ? At least an estimation would be nice.
@GaelCO @chrisbln @afaucogney Sorry for not updating the issue. The fix is included from 10.1.0 from November 23th. Any feedback on the fix is most appreciated.
Thanks @rorbech. I had no plan to just upgrade to 10 just because of the issue ?
Is there any plan to also fix in 7.x ?
BTW, is there any issue when upgrading to 10x, I always experience some difficulties when upgrading Realm version, tbh !
I can't find it in the changelog of 10.1.0, @rorbech, @clementetb.
Has someone already updated to 10.1.x and noticed an improvement?
@Jeff11 Sorry. The core fixes seems to have slipped from the change log. The fix (https://github.com/realm/realm-core/pull/4051) was included by the upgrade to Core 10.1.3.
Assume this has been solved
That's not fixed the data corruption. With version 10.2.0, a crash occurs as soon as I touch the database.
2020-12-14 17:22:01.365 5818-7380/com.gco.realmtest E/REALM_JNI: jni: ThrowingException 5, No such object in /tmp/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 106, .
2020-12-14 17:22:01.365 5818-7380/com.gco.realmtest E/REALM_JNI: Exception has been thrown: Unrecoverable error. No such object in /tmp/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 106
2020-12-14 17:22:01.367 5818-7380/com.gco.realmtest E/AndroidRuntime: FATAL EXCEPTION: RealmThread_write
Process: com.gco.realmtest, PID: 5818
io.realm.exceptions.RealmError: Unrecoverable error. No such object in /tmp/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_OsSharedRealm.cpp line 106
at io.realm.internal.OsSharedRealm.nativeGetSharedRealm(Native Method)
at io.realm.internal.OsSharedRealm.
at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:249)
at io.realm.BaseRealm.
at io.realm.BaseRealm.
at io.realm.Realm.
at io.realm.Realm.createInstance(Realm.java:495)
at io.realm.RealmCache.createInstance(RealmCache.java:481)
at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:448)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:412)
at io.realm.Realm.getInstance(Realm.java:424)
at com.gco.realmtest.realm.RealmThread.onLooperPrepared(RealmThread.kt:31)
at android.os.HandlerThread.run(HandlerThread.java:66)
@GaelCO sorry to hear about this. When you say a crash occurs as soon as I touch the database
is it when you obtain a Realm instance or when you run a query or a transaction? I will pass the info to the core team as soon as I hear from you.
According to the Core team the original problem reported in this ticket should be fixed in https://github.com/realm/realm-core/pull/4051 and was included in the latest Realm 10.1.3 release.
However, once a Realm becomes corrupt, which could happen when upgrading from 6.x.x to 10.0.x (before the fix for the original "key not found" issue was released), it will continue to be so and, unfortunately, there is nothing we can do about it - see https://github.com/realm/realm-core/pull/4051#issuecomment-724026161.
With that in mind, it would be interesting to know if there are still instances of the original "key not found" crash after updating to version 10.1.3.
It's when a run a query. (in my test project, add or delete items)
Most helpful comment
I now have a good lead on this.