Realm-java: Failure when computing UTF-16 size error_code = 1

[trr-amsiq]

Running:

• Realm 3.5.0
• Android 5.0.2

No Encryption
All communication with Realm is performed with AsyncTasks.

Fatal Exception: java.lang.RuntimeException: An error occured while executing doInBackground()
       at android.os.AsyncTask$3.done(AsyncTask.java:300)
       at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
       at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
       at java.util.concurrent.FutureTask.run(FutureTask.java:242)
       at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:818)
Caused by io.realm.exceptions.RealmError: Unrecoverable error. Failure when computing UTF-16 size error_code = 5; retcode = 6; StringData.size = 2442; StringData.data =  in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 151
       at io.realm.internal.UncheckedRow.nativeGetString(UncheckedRow.java)
       at io.realm.internal.UncheckedRow.getString(UncheckedRow.java:167)
       at io.realm.KeyValueObjectRealmProxy.realmGet$value(KeyValueObjectRealmProxy.java:116)
       at com.app.Cache.KeyValueObject.getValue(KeyValueObject.java:26)
       at com.app.Account.Account.loadArray(Account.java:192)

[cmelchior]

Hi @trr-amsiq Is it something you can reproduce? Or what kind of phones are you seeing this on?

[trr-amsiq]

No not yet.
Here some more data http://crashes.to/s/44e3c052321

[cmelchior]

Hi @trr-amsiq

Thanks. It seems to affect 2 users repeatedly, which suggests that their data is indeed faulty somehow and isn't caused by a bug when reading the data. This was e.g. the case for similar errors we saw using encryption where we had a race condition that caused wrong data to be read from disk.

If the underlying data is indeed corrupted, it begs the question on how that happened. It could be a bug in Realm, the OS or disk corruption happening after the data was written.

I don't want to be pessimistic, but without any further to go on, we will probably not be able to solve this as historically it is a huge time sink to investigate these kinds of issues, and the chance of actually finding anything is extremely low unless we can reproduce the error.

I'll keep this issue open though as we definitely want to be able to track if others are seeing similar things.

Just to preserve history:

• Error happened to two users repeatedly
• SM-J110F (Android 4.4.4)
• SM-G530H (Android 5.0.2)

[trr-amsiq]

Hi @cmelchior
I will get back to you if I am able to reproduce it myself.
Cheers

[bmeike]

Adding Help and Waiting for User tags to assist tracking

[bmeike]

Hey @trr-amsiq,
Any progress/new info to share on this?

[bmeike]

@trr-amsiq,
We have been unable to reproduce this. I'm going to close the issue. By all means, if you encounter it again, please open a new issue and we will jump on it.

Cheers!
-blake

[trr-amsiq]

@bmeike
Sorry for the late response. I've been in vacation.
Yes it is still occurring with a more detailed error message

Fatal Exception: java.lang.RuntimeException: An error occured while executing doInBackground()
       at android.os.AsyncTask$3.done(AsyncTask.java:300)
       at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
       at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
       at java.util.concurrent.FutureTask.run(FutureTask.java:242)
       at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:841)
Caused by io.realm.exceptions.RealmError: Unrecoverable error. Failure when computing UTF-16 size error_code = 1; retcode = 621; StringData.size = 2077; StringData.data = {"description":{"allow_edit_date":"","flag_moderate":false,"allow_edit":false},"relation_tx":{"interest":false,"interest_new":false,"message":false,"hide":false,"favorite":false,"message_new":false,"visit":false,"block":false,"visit_new":false},"gender":"female","relation_rx":{"interest":false,"interest_new":false,"message":false,"hide":false,"favorite":false,"message_new":false,"visit":false,"block":false,"visit_new":false},"profile":{"age":"29 سنة","want_kids":"يوما ما","appearance":"مملوح","body_type":"قوام متناسب","complexion":"أبيض","country":"المغرب","country_code":"MA","disbilities":"سليم","eating_habits@ in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 151
       at io.realm.internal.UncheckedRow.nativeGetString(UncheckedRow.java)
       at io.realm.internal.UncheckedRow.getString(UncheckedRow.java:167)
       at io.realm.KeyValueObjectRealmProxy.realmGet$value(KeyValueObjectRealmProxy.java:116)
       at app.devUtilities.Cache.KeyValueObject.getValue(KeyValueObject.java:26)
       at app.devUtilities.Cache.Cache.retreive(Cache.java:99)
       at app.devUtilities.Cache.CacheObject.load(CacheObject.java:56)
       at app.Models.Account.Account.loadFromCache(Account.java:181)
       at app.Models.ActivityStream.Cache.TileCache.parseToTile(TileCache.java:44)
       at app.Models.ActivityStream.Cache.ActivityCache.parseToTiles(ActivityCache.java:48)
       at app.Fragments.ActivityStream.Controller.ActivityStreamController$1.doInBackground(ActivityStreamController.java:74)
       at app.Fragments.ActivityStream.Controller.ActivityStreamController$1.doInBackground(ActivityStreamController.java:68)
       at android.os.AsyncTask$2.call(AsyncTask.java:288)
       at java.util.concurrent.FutureTask.run(FutureTask.java:237)
       at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:841)

[vladimirfx]

I have same issue:
Fatal Exception: io.realm.exceptions.RealmError
Unrecoverable error. Failure when computing UTF-16 size error_code = 1; retcode = 24; StringData.size = 104; StringData.data = ОТДЕЛОМ ЁAAAf in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 161

Realm 5.1.0 , encrypted DB. Massive, 90+ crashes per day. I can't invent some workaround... Please help me.

[nhachicha]

@vladimirfx ay chance you can reproduce this (maybe using the data in production?)

[vladimirfx]

Sadly, I don't have DB file for reproducing this issues. Prod data contains personal information and not extractable from device. Attempts to reproduce in QA env are all failed. But we keep trying to get corrupted DB in QA.
Clearing app data or reinstalling is "fixes" the issue... at cost of customer data.
Application itself is single process. Realm configuration looks like this:

return new RealmConfiguration.Builder()
                .name(realmName)
                .schemaVersion(Constants.DB_VERSION)
                .directory(dbDir)
                .migration(MIGRATION)
                .encryptionKey(key)
                .initialData(initTx)
                .build();

May be you have some workaround or something similar?

[vladimirfx]

650 crashes in last 90 days:
http://crashes.to/s/228b8d4c09c

Issue so critical for us that we consider migrate to another database product. May be reopen this issue? Any advice?

[cmelchior]

Hi @vladimirfx
Unfortunately, we have not been able to reproduce this either, so I don't have any advice to give at this time. We probably need to get our hands on a Realm file with faulty data in order to get any insight into why this is happening

[bmunkholm]

Adding the crash log from above crash:

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Failure when computing UTF-16 size error_code = 1; retcode = 12; StringData.size = 55; StringData.data = ЗАО ЁAAAf in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 161
       at io.realm.internal.UncheckedRow.nativeGetString(UncheckedRow.java)
       at io.realm.internal.UncheckedRow.getString(UncheckedRow.java:157)
       at io.realm.ru_fcb_mobilefieldcollection_debt_DebtorRealmProxy.realmGet$organization(ru_fcb_mobilefieldcollection_debt_DebtorRealmProxy.java:411)
       at ru.fcb.mobilefieldcollection.debt.Debtor.hashCode(Debtor.java:219)
       at java.util.Collections.secondaryHash(Collections.java:3427)
       at java.util.HashMap.get(HashMap.java:300)
       at io.realm.ru_fcb_mobilefieldcollection_debt_DebtRealmProxy.update(ru_fcb_mobilefieldcollection_debt_DebtRealmProxy.java:3009)
       at io.realm.ru_fcb_mobilefieldcollection_debt_DebtRealmProxy.copyOrUpdate(ru_fcb_mobilefieldcollection_debt_DebtRealmProxy.java:2027)
       at io.realm.DefaultRealmModuleMediator.copyOrUpdate(DefaultRealmModuleMediator.java:431)
       at io.realm.Realm.copyOrUpdate(Realm.java:1608)
       at io.realm.Realm.copyToRealmOrUpdate(Realm.java:1025)
       at ru.fcb.mobilefieldcollection.debt.RealmDebtsRepo.lambda$saveDebt$41(RealmDebtsRepo.java:541)
       at ru.fcb.mobilefieldcollection.debt.-$$Lambda$RealmDebtsRepo$qRiIiq5iyNozRDFgEb5XTehd64c.apply(lambda)
       at ru.fcb.mobilefieldcollection.realm.Block-CC.lambda$toFunction$0(Block.java:15)
       at ru.fcb.mobilefieldcollection.realm.-$$Lambda$Block$rxUy2qhKx6FrOVMDHmXqTRfzwqk.apply(lambda)
       at ru.fcb.mobilefieldcollection.realm.RealmTemplate.executeInRealm(RealmTemplate.java:90)
       at ru.fcb.mobilefieldcollection.realm.RealmTemplate.executeInRealm(RealmTemplate.java:80)
       at ru.fcb.mobilefieldcollection.debt.RealmDebtsRepo.saveDebt(RealmDebtsRepo.java:540)
       at ru.fcb.mobilefieldcollection.debt.DebtDataReceiver.processDebt(DebtDataReceiver.java:138)
       at ru.fcb.mobilefieldcollection.debt.DebtDataReceiver.process(DebtDataReceiver.java:61)
       at ru.fcb.mobilefieldcollection.server.MfcServerConnection$WebSocketListener.lambda$onMessage$2(MfcServerConnection.java:251)
       at ru.fcb.mobilefieldcollection.server.-$$Lambda$MfcServerConnection$WebSocketListener$9h2sSXgOqPaj6sBRQIA76EP9FIk.call(lambda)
       at ru.fcb.mobilefieldcollection.realm.RealmUnitOfWork.lambda$doWork$1(RealmUnitOfWork.java:43)
       at ru.fcb.mobilefieldcollection.realm.-$$Lambda$RealmUnitOfWork$GFV_q7dLAawcDKekC9SreLyZVCc.get(lambda)
       at ru.fcb.mobilefieldcollection.realm.RealmUnitOfWork.doWork(RealmUnitOfWork.java:61)
       at ru.fcb.mobilefieldcollection.realm.RealmUnitOfWork.doWork(RealmUnitOfWork.java:41)
       at ru.fcb.mobilefieldcollection.server.MfcServerConnection$WebSocketListener.onMessage(MfcServerConnection.java:248)
       at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:322)
       at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219)
       at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105)
       at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:273)
       at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:213)
       at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153)
       at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
       at java.lang.Thread.run(Thread.java:818)

Perhaps for debugging we could try adding a read after each write of a property, to validate it was indeed written. That way we could know that it is indeed some outside corruption occuring?
Or when this is detected we could run some kind of validation of the database?

@vladimirfx We would really love to solve this, but without an ability to test these things out, we have little to go on. Would you be willing to try out a version with some extra validation in (if others find any of above ideas useful)?

[vladimirfx]

@bmunkholm Yes, sure, we can use special version.

[tfatrifork]

Get same crash with realm version 5.4.0.
Hope this can help identifty the issue. We do not use encryption.

Caused by io.realm.exceptions.RealmError: Unrecoverable error. Failure when computing UTF-16 size error_code = 1; retcode = 3; StringData.size = 54; StringData.data = --X͎¢ /p
h젩n /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 161
       at io.realm.internal.UncheckedRow.nativeGetString(UncheckedRow.java)
       at io.realm.internal.UncheckedRow.getString(UncheckedRow.java:157)

[09ae9c]

any progress? I got the same crash with realm 5.8.0

[erikhuizinga]

I have this same (or similar) issue on Realm 6.1.0. I use an encrypted Realm db.

One user reported the app crashing, but after cleaning app data, the issue is solved. This supports the hypothesis that the Realm file(s) are somehow corrupted.

All the 15 crash logs of this single user show the following pattern:

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Failure when computing UTF-16 size error_code = $1; retcode = $2; StringData.size = $3; StringData.data = $4 in /Users/cm/Realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 161

In the above crash line, I've put numbered variables, which have the follwing values:

• $1: all have value 1, except one has value 5.
• $2: varies for some crashes, I'm not sure if there's a pattern.
• $3: in the range 600–617.
• $4: all values start with a URL (https://...somethingsomething) but end with AAAA. One ends with AAAAF. That's possibly where some decoding breaks, maybe? One doesn't end with AAAA or AAAAF, but the URL suddenly stops halfway and I see a bunch of user-written text directly appended to the URL (this is the one with $1 = 5). This looks like either a middle part of the string is missing, or some other string field got appended suddenly.

This particular user uses:

• Realm 6.1.0
• Android 9
• Samsung Galaxy J5 with over 500 MB free RAM and over 4 GB free disk space