Realm-java: SEGV_MAPERR, Failure when converting short string to UTF-16
We're experiencing easily reproducible crashes when using copyFromRealm while simultaneously writing on a separate thread.
May be related to #2492
Observations
- Crashes only occur with at least 1 write thread
- Crashes occur with relatively few iterations
- Crashes appear to only occur when using encryption
- Crashes do not appear to lead to corruption (as in #4128)
Results
Below is a sampling of crashes on 3.0.0. I can generate crash reports on 2.3.2 if necessary.
E/REALM_JNI: jni: ThrowingException 5, Failure when converting short string to UTF-16 error_code = 1; retcode = 0; StringData.size = 13; StringData.data = class_Message; StringData as hex = 0x63 0x6c 0x61 0x73 0x73 0x5f 0x4d 0x65 0x73 0x73 0x61 0x67 0x65; in_begin = class_Message; in_end = ; out_curr = 0x941812e0; out_end = 0x94181340; in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_Table.cpp line 1294, .
E/REALM_JNI: Exception has been thrown: Unrecoverable error. Failure when converting short string to UTF-16 error_code = 1; retcode = 0; StringData.size = 13; StringData.data = class_Message; StringData as hex = 0x63 0x6c 0x61 0x73 0x73 0x5f 0x4d 0x65 0x73 0x73 0x61 0x67 0x65; in_begin = class_Message; in_end = ; out_curr = 0x941812e0; out_end = 0x94181340; in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_Table.cpp line 1294
E/AndroidRuntime: FATAL EXCEPTION: Thread-312
Process: io.binarysolutions.realmmemtest, PID: 12344
io.realm.exceptions.RealmError: Unrecoverable error. Failure when converting short string to UTF-16 error_code = 1; retcode = 0; StringData.size = 13; StringData.data = class_Message; StringData as hex = 0x63 0x6c 0x61 0x73 0x73 0x5f 0x4d 0x65 0x73 0x73 0x61 0x67 0x65; in_begin = class_Message; in_end = ; out_curr = 0x941812e0; out_end = 0x94181340; in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_Table.cpp line 1294
at io.realm.internal.Table.nativeGetName(Native Method)
at io.realm.internal.Table.getName(Table.java:1097)
at io.realm.MessageRealmProxy.hashCode(MessageRealmProxy.java:804)
at java.util.Collections.secondaryHash(Collections.java:3427)
at java.util.HashMap.put(HashMap.java:385)
at io.realm.MessageRealmProxy.createDetachedCopy(MessageRealmProxy.java:749)
at io.realm.ContactRealmProxy.createDetachedCopy(ContactRealmProxy.java:774)
at io.realm.DefaultRealmModuleMediator.createDetachedCopy(DefaultRealmModuleMediator.java:268)
at io.realm.Realm.createDetachedCopy(Realm.java:1520)
at io.realm.Realm.copyFromRealm(Realm.java:1252)
at io.realm.Realm.copyFromRealm(Realm.java:1225)
at io.binarysolutions.realmtest.MainActivity$6.run(MainActivity.java:308)
at java.lang.Thread.run(Thread.java:818)
signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 11192 (Thread-302)
Revision: '0'
ABI: 'arm'
pid: 10959, tid: 11192, name: Thread-302 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 b826dc70 r1 00000018 r2 b826de08 r3 00000000
r4 0a000000 r5 b826dc70 r6 2d353535 r7 00000018
r8 00000000 r9 b8147890 sl 00000000 fp 00000000
ip b3ac813d sp 94245358 lr b3b9c6ad pc b3affa72 cpsr 600f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
0008ea72 realm::SlabAlloc::do_translate(unsigned int) const+206 libgcc2.c:?
0012b6a9 realm::StringColumn::get(unsigned int) const+200 libgcc2.c:?
-----------------------------------------------------
signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 12135 (Thread-308)
Revision: '0'
ABI: 'arm'
pid: 11985, tid: 12135, name: Thread-308 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 b823b948 r1 00000000 r2 b823c434 r3 00000000
r4 0a000000 r5 b823b948 r6 e281e559 r7 000000df
r8 00000000 r9 b7f07dc0 sl 00000000 fp 00000000
ip b3ac813d sp 94184318 lr b3b9a0ef pc b3affa72 cpsr 600f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
0008ea72 realm::SlabAlloc::do_translate(unsigned int) const+206 libgcc2.c:?
001290eb realm::ArrayStringLong::get(char const*, unsigned int, realm::Allocator&, bool)+36 libgcc2.c:?
-----------------------------------------------------
signal 11 (SIGSEGV), code 1, fault addr 0x41 in tid 12811 (Thread-316)
Revision: '0'
ABI: 'arm'
pid: 12677, tid: 12811, name: Thread-316 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x41
r0 b7e9bf1c r1 0000011a r2 0000011a r3 00000000
r4 941813d8 r5 b7e9bf10 r6 0000011a r7 b7f20ba8
r8 b80812e4 r9 b8081258 sl 6fbf6508 fp 941816bc
ip b3ac813d sp 94181390 lr b3b8ae7f pc b3a958c8 cpsr 200f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
000248c8 realm::BpTree<long long>::get(unsigned int) const+8 libgcc2.c:?
00119e7b realm::StringEnumColumn::get(unsigned int) const+12 libgcc2.c:?
-----------------------------------------------------
signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 13354 (Thread-320)
Revision: '0'
ABI: 'arm'
pid: 12963, tid: 13354, name: Thread-320 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 b8202a30 r1 00000000 r2 b820366c r3 00000000
r4 0a000000 r5 b8202a30 r6 212a4ebe r7 000000fb
r8 00000000 r9 00000379 sl 00000000 fp 00000000
ip b3ac813d sp 941812f8 lr b3b4ddad pc b3affa72 cpsr 600f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
0008ea72 realm::SlabAlloc::do_translate(unsigned int) const+206 libgcc2.c:?
000dcda9 (anonymous namespace)::find_bptree_child(long long, unsigned int, realm::Allocator const&)+60 libgcc2.c:?
-----------------------------------------------------
signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 13632 (Thread-324)
Revision: '0'
ABI: 'arm'
pid: 13498, tid: 13632, name: Thread-324 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 b80eabe8 r1 00002cc8 r2 b80eb710 r3 00000000
r4 0a000000 r5 b80eabe8 r6 f4e842ba r7 000000e4
r8 00000000 r9 b7f24580 sl 00000000 fp 00000000
ip b3ac813d sp 9417e318 lr b3b9a119 pc b3affa72 cpsr 600f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
0008ea72 realm::SlabAlloc::do_translate(unsigned int) const+206 libgcc2.c:?
00129115 realm::ArrayStringLong::get(char const*, unsigned int, realm::Allocator&, bool)+78 libgcc2.c:?
-----------------------------------------------------
signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 14178 (Thread-329)
Revision: '0'
ABI: 'arm'
pid: 14062, tid: 14178, name: Thread-329 >>> io.binarysolutions.realmmemtest <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 b81dc3b8 r1 0001cbf8 r2 b81dc688 r3 00000000
r4 0a000000 r5 b81dc3b8 r6 38f331c8 r7 00000032
r8 00000000 r9 0000010f sl 00000000 fp 00000000
ip b3ac813d sp 940822f8 lr b3b4ddad pc b3affa72 cpsr 600f0030
Stack Trace:
RELADDR FUNCTION FILE:LINE
0008ea72 realm::SlabAlloc::do_translate(unsigned int) const+206 libgcc2.c:?
000dcda9 (anonymous namespace)::find_bptree_child(long long, unsigned int, realm::Allocator const&)+60 libgcc2.c:?
Code
https://github.com/bios-seiji/realm-crash/tree/300
Version of Realm and tooling
Realm version(s): <=3.0.0
Sync: no
Encryption: yes
Android Studio version: 2.2.2
Which Android version and device: 6.0.1
bios-seiji
All 11 comments
I have tried your test app on my OnePlus One (Android 6.0.1 - 32 bit ARM). I see the UTF-16 error message too but it takes a while. The last writer log entry prior to the UTF-16 message is I/Writer: Global: 4, Local: 1, Count: 8985, Duration: 67879.
The error message I see is Failure when computing UTF-16 size. The size of the string is 23134. The update in the writer threads is setting the string to "User" + i.incrementAndGet(), and i can only have 9 digits (is an int) so the string should be shorter than 13.
Next step is to examine the produced Realm file to see if the stored strings are correct or not. This can give me a hint of whether the bug is in copy to or from Realm.
kneth
on 20 Mar 2017
I also tried the project and got the same crash almost instantly on my Nexus 5X running API 24.
I then put synchronized object locks on all transactions and copyFromRealm and ran the test again. This time it took a very long time but it did occur eventually. With a 10ms delay before copyFromRealm -> no crash after 100k iterations.
EdwardvanRaak
on 20 Mar 2017
Realm Browser showed both classes/tables as empty which is likely wrong.
As an experiment, I have tried not to use copyFromRealm() but copy the string by using getUsername(). After 140000 + 225000 iterations, no UTF-16 message. Next step is to investigate copyFromRealm().
kneth
on 20 Mar 2017
@kneth I think the following crash log can help you:
io.realm.exceptions.RealmError: Unrecoverable error. Failure when converting short string to UTF-16 error_code = 1;
retcode = 0;
StringData.size = 44;
StringData.data = cockpit:f55e04ad-4c34-41d5-b90d-117ffb2aecf0;
StringData as hex = 0x63 0x6f 0x63 0x6b 0x70 0x69 0x74 0x3a 0x66 0x35 0x35 0x65 0x30 0x34 0x61 0x64 0x2d 0x34 0x63 0x33 0x34 0x2d 0x34 0x31 0x64 0x35 0x2d 0x62 0x39 0x30 0x64 0x2d 0x31 0x31 0x37 0x66 0x66 0x62 0x32 0x61 0x65 0x63 0x66 0x30;
in_begin = 0d-117ffb2aecf0;
in_end = ;
out_curr = 0x7f976ad94a;
out_end = 0x7f976ad970;
in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 138
at io.realm.internal.UncheckedRow.nativeGetString(Native Method)
at io.realm.internal.UncheckedRow.getString(UncheckedRow.java:166)
at io.realm.TripRealmProxy.realmGet$endAddressId(TripRealmProxy.java:1210)
at io.realm.TripRealmProxy.createDetachedCopy(TripRealmProxy.java:4495)
at io.realm.MyApplicationModuleMediator.createDetachedCopy(MyApplicationModuleMediator.java:431)
at io.realm.Realm.createDetachedCopy(Realm.java:1520)
at io.realm.Realm.copyFromRealm(Realm.java:1201)
at io.realm.Realm.copyFromRealm(Realm.java:1167)
at com.example.app.DataBase.fetchTrips(DataBase.java:897)
at com.example.app.MyService$1$16.run(MyService.java:573)
at android.os.Handler.handleCallback(Handler.java:739)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:158)
at android.os.HandlerThread.run(HandlerThread.java:61)
The concerned String is: cockpit:f55e04ad-4c34-41d5-b90d-117ffb2aecf0 which is:
String addressId = "cockpit:" + java.util.UUID.randomUUID().toString()
mariusboepple
on 8 May 2017
@MariusBoepple Thanks!
@ironage You might be able to use the above in your bug hunt.
kneth
on 9 May 2017
It looks like we might have a fix for this: https://github.com/realm/realm-core/pull/2616
cmelchior
on 17 May 2017
This is fixed by v3.2.1 馃帀
beeender
on 19 May 2017
@beeender , I have updated to 3.2.1 but still getting error
Fatal Exception: io.realm.exceptions.RealmError
Unrecoverable error. Failure when converting short string to UTF-16 error_code = 1; retcode = 0; StringData.size = 30; StringData.data = in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_UncheckedRow.cpp line 136
io.realm.internal.UncheckedRow.nativeGetString
Raw Text
lucasleongit
on 22 May 2017
@lucasleongit are you using encrypted Realm?
beeender
on 22 May 2017
@beeender : yes, I am using encrypted Realm.
lucasleongit
on 22 May 2017
@lucasleongit How does your code differ from the sample app, you find in the description of this issue? I am asking to get a better idea of you are hitting another use case.
kneth
on 22 May 2017
Related issues
AAChartModel
路
3Comments
aschrijver
路
3Comments
AlbertVilaCalvo
路
3Comments
tloshi
路
3Comments
pawlo2102
路
3Comments
Most helpful comment
This is fixed by
v3.2.1馃帀