Realm-java: Error: Operation not permitted in io_realm_internal_SharedGroup.cpp

[snakeice]

Goal

What do you want to achieve?

Actual Results

io.realm.exceptions.RealmError: Unrecoverable error. Operation not permitted in io_realm_internal_SharedGroup.cpp line 113
at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
at io.realm.internal.SharedGroup.openSharedGroupOrFail(SharedGroup.java:95)
at io.realm.internal.SharedGroup.<init>(SharedGroup.java:74)
at io.realm.internal.SharedGroupManager.<init>(SharedGroupManager.java:49)
at io.realm.BaseRealm.<init>(BaseRealm.java:81)
at io.realm.Realm.<init>(Realm.java:140)
at io.realm.Realm.createAndValidate(Realm.java:240)
at io.realm.Realm.createInstance(Realm.java:220)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
at io.realm.Realm.getDefaultInstance(Realm.java:166)
at br.com.jbsoft.mobile.di.modules.ProviderModule.provideRealm(ProviderModule.java:28)
at br.com.jbsoft.mobile.di.modules.ProviderModule_ProvideRealmFactory.get(ProviderModule_ProvideRealmFactory.java:19)
at br.com.jbsoft.mobile.di.modules.ProviderModule_ProvideRealmFactory.get(ProviderModule_ProvideRealmFactory.java:8)
at br.com.jbsoft.mobile.models.ControleUsuario_MembersInjector.injectMembers(ControleUsuario_MembersInjector.java:33)
at br.com.jbsoft.mobile.models.ControleUsuario_MembersInjector.injectMembers(ControleUsuario_MembersInjector.java:9)
at br.com.jbsoft.mobile.di.components.DaggerProviderComponent.inject(DaggerProviderComponent.java:243)
at br.com.jbsoft.mobile.models.ControleUsuario.inject(ControleUsuario.java:31)
at br.com.jbsoft.mobile.models.ControleUsuario.getUsuario(ControleUsuario.java:35)
at br.com.jbsoft.mobile.views.Main.onCreate(Main.java:59)
at br.com.jbsoft.mobile.componentes.BaseActivity.onCreate(BaseActivity.java:86)
at android.app.Activity.performCreate(Activity.java:6500)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1120)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3072)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3218)
at android.app.ActivityThread.access$1000(ActivityThread.java:198)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1676)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:145)
at android.app.ActivityThread.main(ActivityThread.java:6837)
at java.lang.reflect.Method.invoke(Native Method)
at java.lang.reflect.Method.invoke(Method.java:372)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1404)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1199)

Steps & Code to Reproduce

Just start the application and get the exception :[

Code Sample

My config

        RealmConfiguration configuration = new RealmConfiguration
                .Builder(this, new File(getCaminhoDB(Boolean.FALSE)))
                .name(Constantes.arquivos.DB_REALM_FINAL)
               .encryptionKey(Preferencias.getInstance().getDbk())
                .schemaVersion(BuildConfig.DB_VERSION)
                .modules(new InitialVersion())
                .migration(new MigrationDB())
                .build();

        Realm.setDefaultConfiguration(configuration);
        RealmLog.add(new DebugAndroidLogger());

and getCaminhoDB:

  private static String getCaminhoDB(@NonNull Context context) {
        if (isExternalStorageWritable()) {
            return context.getExternalFilesDir("database").toString();
        } else {
            return context.getFilesDir().getAbsolutePath();
        }
    }

getKeyDef():

    public static byte[] getKeyDef() {
        byte[] key = new byte[64];
        String s = "***********************************************************************";
        try {
            new SecureRandom(s.getBytes("us-ascii")).nextBytes(key);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return key;
    }

Version of Realm and tooling

Realm version(s): 1.1.0

Android Studio version: 2.1.2

Which Android version and device: I test with Emulator (Android 6.0) and Xiaomi redmi 2 prime (Android 4.4)

[zaki50]

Hi @Snakeice

The argument in getCaminhoDB(Boolean.FALSE) is Boolean, but defined method accepts Context.

Can you provide the path that getCaminhoDB() returned?

[snakeice]

Hello, the boolean passed to the method is to return the file name only ..

It returns /mnt/sdcard/android/data/br.com.jbsoft.mobile/files/database/
Tomorrow I step the other device

[Zhuinden]

Can you share the code for isExternalStorageWritable?

On newer devices you'd need the runtime permission to write to an external directory, but you must also consider that the SD card isn't mountable at the time.

[kneth]

Yes, on Android 6 the permission policies are a bit more rigid. Maybe https://webrtchacks.com/android-m-app-permissions/ can help you.

[zaki50]

Does /mnt/sdcard/android/data/br.com.jbsoft.mobile/files/database/ directory exist on your disk?

[snakeice]

@Zhuinden the code is:

   public static boolean isExternalStorageWritable() {
        String state = Environment.getExternalStorageState();
        return Environment.MEDIA_MOUNTED.equals(state) && Environment.getExternalStorageDirectory()
                .canWrite();
    }

@kneth I ask all permissions during application startup for Android 6.0.

@zaki50 Yes, he's created, tested to manually create..

[zaki50]

@Snakeice I confirmed this issue and found the root cause.

Realm creates named pipe file, but Android's security system(SELinux) prohibits to create it on external storage.
Current work-around is to use internal storage for Realm files.

To solve this issue, we need to fix Realm core.

[zaki50]

memo:

To set SELinux to permissive mode, execute following commands to rooted device.

$ adb root
adbd is already running as root
$ adb shell setenforce 0

07-11 16:36:47.854 22800 22800 W .realm_template: type=1400 audit(0.0:126): avc: denied { create } for name="access_control.new_commit.cv" scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:fuse:s0:c512,c768 tclass=fifo_file permissive=0
07-11 16:36:47.858 22800 22800 D REALM   : jni: ThrowingException 11, Permission denied in io_realm_internal_SharedGroup.cpp line 113, .
07-11 16:36:47.858 22800 22800 D REALM   : Exception has been throw: Unrecoverable error. Permission denied in io_realm_internal_SharedGroup.cpp line 113
07-11 16:36:47.859 22800 22800 D AndroidRuntime: Shutting down VM
07-11 16:36:47.860 22800 22800 E AndroidRuntime: FATAL EXCEPTION: main

[ankitbabbar89]

Got Similar bug while trying to create Realm on external storage directory.
Stacktrace:

09-04 21:01:15.121: D/REALM(32175): jni: ThrowingException 11, Operation not permitted in io_realm_internal_SharedGroup.cpp line 113,
09-04 21:01:15.121: D/REALM(32175): Exception has been throw: Unrecoverable error. Operation not permitted in io_realm_internal_SharedGroup.cpp line 113
09-04 21:01:15.121: E/AndroidRuntime(32175): FATAL EXCEPTION: main
09-04 21:01:15.121: E/AndroidRuntime(32175): io.realm.exceptions.RealmError: Unrecoverable error. Operation not permitted in io_realm_internal_SharedGroup.cpp line 113
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.internal.SharedGroup.openSharedGroupOrFail(SharedGroup.java:95)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.internal.SharedGroup.(SharedGroup.java:74)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.internal.SharedGroupManager.(SharedGroupManager.java:49)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.BaseRealm.(BaseRealm.java:86)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.Realm.(Realm.java:135)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.Realm.createAndValidate(Realm.java:233)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.Realm.createInstance(Realm.java:214)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
09-04 21:01:15.121: E/AndroidRuntime(32175): at io.realm.Realm.getDefaultInstance(Realm.java:160)

Any timelines for the fix ? For now rolling back to 1.0.0 :(

[Zhuinden]

@ankitbabbar89 considering you're trying to write the Realm on external storage directory which the SELinux of the Android Kernel will block, it's not really something that Realm can fix too quickly.

[nishkarsh]

I got a crash related to similar issue, the only difference is that I had not specified any directory explicitly, which means that it should have used internal storage to store the file. It has been working on all the devices, it's just one time that I have faced this issue. Is it because of something else? Also, would the behaviour differ based on the type of context being used (ApplicationContext or ActivityContext)?

Stacktrace:

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in io_realm_internal_SharedGroup.cpp line 113
at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Unknown Source)
at io.realm.internal.SharedGroup.openSharedGroupOrFail(Unknown Source)
at io.realm.internal.SharedGroup.(Unknown Source)
at io.realm.internal.SharedGroupManager.(Unknown Source)
at io.realm.BaseRealm.(Unknown Source)
at io.realm.Realm.(Unknown Source)
at io.realm.Realm.getInstance(Unknown Source)
at io.realm.Realm.createInstance(Unknown Source)
at io.realm.RealmCache.createRealmOrGetFromCache(Unknown Source)
at io.realm.Realm.getInstance(Unknown Source)

[beeender]

@nishkarsh Your issue is something else. Can you reproduce it with any device you have?

[nishkarsh]

@beeender I couldn't produce the issue on any device without doing any changes. However, I tried using the app on a phone that had SD card and had to explicitly specify to use external storage to store realm and that threw exactly the same exception. Was using realm 1.1.1 if that helps. The crash occurred on Lenovo P1a42 with Android 5.1.1

[beeender]

@nishkarsh Thanks for the information. I am not quite what kind of tricks does the Lenovo phone made to the storage management. We will fix the external issue first then try if we can find a phone like that to test.

[beeender]

More info from #3413 which should be the same issue.

Thank you for making Realm bug free but there is still one issue.

Caused by: io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in io_realm_internal_SharedGroup.cpp line 113
    at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
    at io.realm.internal.SharedGroup.a(SharedGroup.java:95)
    at io.realm.internal.SharedGroup.<init>(SharedGroup.java:74)
    at io.realm.internal.o.<init>(SharedGroupManager.java:49)
    at io.realm.a.<init>(BaseRealm.java:86)
    at io.realm.j.<init>(Realm.java:135)
    at io.realm.j.b(Realm.java:233)
    at io.realm.j.a(Realm.java:214)
    at io.realm.l.a(RealmCache.java:126)
    at io.realm.j.m(Realm.java:160)
    at com.conem.app.pocketthesaurus.display.b.a(ActivitySplash.java:144)
    at com.conem.app.pocketthesaurus.display.b.doInBackground(ActivitySplash.java:99)
    at android.os.AsyncTask$2.call(AsyncTask.java:295)
    at java.util.concurrent.FutureTask.run(FutureTask.java:237)
    ... 4 more

The above log is from playstore this is from Crashlytics. http://crashes.to/s/305f76efbc2
It is linked to #2762

I have been using Realm before 0.70.0 and this bug is there since the beginning. Almost 5 out of 400 daily users who download my app are experiencing it.

Version of Realm and tooling

Realm version(s): ? 1.2.1

Android Studio version: ? 2.1.1

Which Android version and device: ?

[drjunior]

Hi @zaki50 is this issue going to be addressed in the 2.0 milestone?
This is quite important because I have an app with thousand users with low internal storage space and want to have the realm database with ~100MB on the external storage.

Thanks for your support.

[zaki50]

@drjunior sorry for my late reply.

I heard that core team had finished implementing required features. I'll take a look.

[drjunior]

@zaki50 thanks for the update. I will wait for your final answer.
Thanks.

[rimidalv]

I have same problem after update from realm version 0.77.0

 Process: com.rimidalv.dictaphone, PID: 1455
io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in io_realm_internal_SharedGroup.cpp line 113
at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
at io.realm.internal.SharedGroup.openSharedGroupOrFail(SharedGroup.java:95)
at io.realm.internal.SharedGroup.<init>(SharedGroup.java:74)
at io.realm.internal.SharedGroupManager.<init>(SharedGroupManager.java:49)
at io.realm.BaseRealm.<init>(BaseRealm.java:87)
at io.realm.Realm.<init>(Realm.java:139)
at io.realm.Realm.createAndValidate(Realm.java:239)
at io.realm.Realm.createInstance(Realm.java:219)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:126)
at io.realm.Realm.getDefaultInstance(Realm.java:165)

Version of Realm and tooling:
Realm version(s): 1.2.0
Android Studio version: 2.2
Android version and device: Nexus 6p with Android 6/7

[Zhuinden]

@rimidalv is the Realm opened in internal storage, or external storage?

[rimidalv]

Environment.getExternalStorageDirectory() - and it doesn't work. I tried versions 1.0.0 and upper.
Now I return realm version to 0.77.0 and it work fine.

[Zhuinden]

Huh. Can you try version 0.88.3? It works with the Gradle plugin solution, just like 1.2.0.

I found that to be most stable (with minimum changes) before 1.0.0 (which already had this external write problem if I recall right, although you can try that too if you want)

[rimidalv]

0.88.3 - works fine

[rimidalv]

0.89.0 - doesn't work

[Zhuinden]

@rimidalv in that case, use 0.88.3 in the meantime, it's actually fairly stable in my experience :)

Just keep in mind that you still need to reverse iterate the RealmResults, do not use foreach loop yet, and I'd recommend sticking to the synchronous query API.

[drjunior]

Hi @zaki50 and @Zhuinden, I've just updated to 2.0 and the crash continues occurring.
Exception -> io.realm.exceptions.RealmError: Unrecoverable error. Operation not permitted in /Users/emanuelez/repos/realm/realm-java-private/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81

Was the issue addressed in 2.0 ? Sorry for being asking again but this is quite important.

[zaki50]

I've confirmed that realm-core has the option parameter to avoid this.

https://github.com/realm/realm-core/blob/v2.1.1/src/realm/group_shared_options.hpp#L40

TODO:

• [x] add a temp_directory parameter to object-store layer.
• [x] fix external_commit_helper (https://github.com/realm/realm-object-store/blob/master/src/impl/android/external_commit_helper.cpp#L98) to respect temp_directory parameter.
• [x] pass that parameter from Realm-Java.
• [x] test

[zaki50]

I made a PR #3591 and realm/realm-object-store#191.
Working on my Nexus 6P.

final File externalFilesDir = context.getExternalFilesDir(null);

final RealmConfiguration config = new RealmConfiguration.Builder()
        .directory(externalFilesDir)
        .name("ext.realm")
        .build();

final Realm realm = Realm.getInstance(config);
realm.close();

[KimiChiu]

Got similar exceptions here, and we are using the internal storage.

Exception io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in /Users/zaki/fromgit/realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 92
io.realm.internal.SharedRealm.nativeGetSharedRealm (SharedRealm.java)
io.realm.internal.SharedRealm.getInstance (SharedRealm.java:205)
io.realm.internal.SharedRealm.getInstance (SharedRealm.java:182)
io.realm.BaseRealm.compactRealm (BaseRealm.java:586)
io.realm.Realm.compactRealm (Realm.java:1564)
com.ihad.ptt.domain.dao.realm.impl.RealmMigration.init (RealmMigration.java:29)
com.ihad.ptt.App.onCreate (App.java:99)
android.app.Instrumentation.callApplicationOnCreate (Instrumentation.java:1015)
android.app.ActivityThread.handleBindApplication (ActivityThread.java:4786)
android.app.ActivityThread.access$1600 (ActivityThread.java:165)
android.app.ActivityThread$H.handleMessage (ActivityThread.java:1430)
android.os.Handler.dispatchMessage (Handler.java:102)
android.os.Looper.loop (Looper.java:150)
android.app.ActivityThread.main (ActivityThread.java:5546)
java.lang.reflect.Method.invoke (Method.java)
com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:792)
com.android.internal.os.ZygoteInit.main (ZygoteInit.java:682)

And it comes with another,

Exception io.realm.exceptions.RealmError: Unrecoverable error. write(): failed: No space left on device in /Users/zaki/fromgit/realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 92
io.realm.internal.SharedRealm.nativeGetSharedRealm (SharedRealm.java)
io.realm.internal.SharedRealm.getInstance (SharedRealm.java:205)
io.realm.internal.SharedRealm.getInstance (SharedRealm.java:182)
io.realm.BaseRealm.compactRealm (BaseRealm.java:586)
io.realm.Realm.compactRealm (Realm.java:1564)
com.ihad.ptt.domain.dao.realm.impl.RealmMigration.init (RealmMigration.java:29)
com.ihad.ptt.App.onCreate (App.java:99)
android.app.Instrumentation.callApplicationOnCreate (Instrumentation.java:1017)
android.app.ActivityThread.handleBindApplication (ActivityThread.java:4852)
android.app.ActivityThread.access$1500 (ActivityThread.java:178)
android.app.ActivityThread$H.handleMessage (ActivityThread.java:1531)
android.os.Handler.dispatchMessage (Handler.java:111)
android.os.Looper.loop (Looper.java:194)
android.app.ActivityThread.main (ActivityThread.java:5631)
java.lang.reflect.Method.invoke (Method.java)
java.lang.reflect.Method.invoke (Method.java:372)
com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:959)
com.android.internal.os.ZygoteInit.main (ZygoteInit.java:754)

So these two exceptions are the same(on different device)?

[GaborPeto]

I have the exact same issue as the commenter above:

Caused by io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in /Users/zaki/fromgit/realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 92
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:205)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:182)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124)
       at io.realm.Realm.getInstance(Realm.java:229)

This is on Realm 2.1.1