Realm-java: Cannot get Realm instance: io.realm.exceptions.RealmFileException: Unable to open a realm at path

[rajuashok]

Goal

Call Realm realm = Realm.getInstance(mRealmConfig); where

mRealmConfig = RealmConfiguration.Builder() .schemaVersion(2) .deleteRealmIfMigrationNeeded() .build();

Expected Results

Get a Realm without exception.

Actual Results

The following exception is thrown:
io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.dropbox.paper/files/default.realm.management': make_dir() failed: No such file or directory. (make_dir() failed: No such file or directory) (/data/data/com.dropbox.paper/files/default.realm.management) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217 Kind: ACCESS_ERROR.

More here:
realm-exception.txt

Steps & Code to Reproduce

This happens on first launch of the app.

Version of Realm and tooling

Realm version(s): 2.2.1

Tried upgrading to 3.1.2 but still seeing the same issue.

Realm sync feature enabled: unsure. Would this affect this? How can I turn this on/off?

Android Studio version: 2.3

Which Android version and device:

• Nexus 5 on 6.0.1
• Google Pixel 7.1.1

[zaki50]

Hi, @rajuashok

Where and how do you call Realm.init(Context);?

[rajuashok]

I call Realm.init(this); in my Application.onCreate(). Near the very top, before I call Realm.getInstance(mRealmConfig)`.

[zaki50]

Are you using a service with separate process?

[rajuashok]

Not sure I follow. I do run Realm.getInstance(mRealmConfig) on background thread using RxJava's Schedulers.io():

    Observable.defer(() => {
        ...
        Realm realm = Realm.getInstance(mRealmConfig);
        ...
    }).subscribeOn(Schedulers.io());

[rajuashok]

I'm trying a few different things to see what might cause this. I just tried doing this in Application.onCreate():

    Realm.init(this);
    Realm realm = Realm.getInstance(mRealmConfig);

This results in a pretty bad crash:

04-13 15:30:26.661 14021-14021/com.dropbox.paper:crashes A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x24 in tid 14021 (x.paper:crashes)

                   [ 04-13 15:30:26.662   555:  555 W/         ]
                   debuggerd: handling request: pid=14021 uid=10196 gid=10196 tid=14021

Is this expected? Why can't I call these two one after the other? There's nothing in the documentation indicating this.

[rajuashok]

Hmm, looking into Realm.init(Context c) I see that you're loading native library. In our Application we're using ReLinker to load our own native library. Could this be causing an issue?

[Zhuinden]

realm brings in Relinker 1.2.2 https://github.com/realm/realm-java/blob/5ff6f659fbd4388b3109066f6403b5a8bfa8e7ec/realm/realm-library/build.gradle#L150

[rajuashok]

So how would that cause this to break? I'm not able to get any useful errors here. And at some point the Realm instance is created successfully.

[zaki50]

Do you have any services that have android:process or android:isolatedProcess attribute in your AndroidManifest.xml?

[rajuashok]

Yes we have a third party library that we include that has the following line in their manifest:

android:process=":crashes"

[Zhuinden]

Then you might want to move Realm initialization into a BaseActivity instead of application

[zaki50]

@rajuashok Does the service has android:isolatedProcess attribute as well?
And is this error happennig on that servicr?

[rajuashok]

The service does not have android:isolatedProcess. Also we're not initializing Realm on that service's process.

We do the following in our custom Application class onCreate() method:

if ((new ProcessInfo(this)).findNameForCurrentProcess().endsWith(":crashes")) {
            return;
}

...

Realm.init(this);

[beeender]

@rajuashok Where can I get your apk file? Or could you please send it to [email protected] ? Maybe I can try to reproduce it on our side.

[rajuashok]

I used the debugger to step through the code and find that it's crashing in nativeGetSharedRealm(nativeConfigPtr, realmNotifier), which is called to construct a SharedRealm (SharedRealm.getInstance()).

Basically looking at the error it seems like there is a delay in the data/data/com.mypackage/files directory being created. I'm not sure why there is a delay, but Realm seems to assume that the files directory will be there and hence the make_dir() method fails.

Would you be able to fix this to check if the files directory exists first, and create it if it doesn't?

[rajuashok]

Also, I've confirmed with adb that the files directory is in fact not created on first launch of our app.

Really not sure why this is happening. I think we'll try to create some workaround for this, but I think it makes sense for Realm to actually check this in its initialization too. Thoughts?

[beeender]

/data/data/com.dropbox.paper/files/default.realm.management will be created in the c++ code, see https://github.com/realm/realm-core/blob/master/src/realm/group_shared.cpp#L763

It is very strange the calling mkdir doesn't work. Does the happen to all the devices or it only happen to some specific device?

[beeender]

Is that because of at that time, files dir is not created?

[rajuashok]

Yes it's because the files dir was not created. This is happening on all devices as well. I believe your code should check to see if files dir exists first before trying to create default.realm.management. If it doesn't exist it should create files. Right?

[rajuashok]

FYI this is some unfinished code I've added to my app initialization that seems to fix the issue:

File filesDir = mContext.getFilesDir();
if (filesDir == null || !filesDir.exists()) {
    if (filesDir == null) {
        Logger.debug(TAG, "Couldn't even get a file for getFilesDir (null)");
    } else {
        try {
            filesDir.mkdirs();
        } catch (SecurityException e) {
            Logger.error(TAG, "Failed to create the files directory.");
        }
    }
} else {
    Logger.debug(TAG, "Successfully created files directory.");
}

[nhachicha]

Hmm it's strange Android is already doing this check to make sure the directory exist when calling getFilesDir http://androidxref.com/7.0.0_r1/xref/frameworks/base/core/java/android/app/ContextImpl.java#520

http://androidxref.com/7.0.0_r1/xref/frameworks/base/core/java/android/app/ContextImpl.java#537

[rajuashok]

Wow. You're right.. No idea why this is happening in our app. Will try to step through that code and see what happens.

[Zhuinden]

I've seen getFilesDir() return null on some Samsung devices immediately after installation on the very first start-up in Application.onCreate(), maybe they thought this kind of synchronization and initialization is not necessary! ¬¬

[rajuashok]

FYI, simply calling Context.getFilesDir() before Realm.getInstance() also seems to fix the issue. Does the Realm native code call Context.getFilesDir()? Or does it assume the path is there and try to make a directory within it?

[cmelchior]

We do call that method: https://github.com/realm/realm-java/blob/master/realm/realm-library/src/main/java/io/realm/RealmConfiguration.java#L412

I guess we can add more error checking there, so we can fail sooner, but I'm a bit out of ideas on fixing it, because I would assume that calling mkdirs() would throw security exceptions? I would be very surprised if an app process would be allowed to create those base dirs?

[rajuashok]

Yeah I agree, it would be nice if your code had some extra checking, but yes this doesn't make sense according to the Android APIs.

Especially since it's happening on several devices (Pixel, Nexus 5, emulators, etc). We'll just have to fix this with the workaround mentioned.

[kneth]

@rajuashok Did you have a chance to implement the workaround?

@Zhuinden Which Samsung devices?

[Zhuinden]

I think it might have been the tablet, or it was the ace 3. I'm not sure...

[rajuashok]

Hi @kneth , so I tried the workaround and I'm seeing some very weird behavior.

I've tried two things:

1. Add a call to getFilesDir() in the DataStore initialization code. The DataStore is a singleton class that deals with Realm.
   This workaround fixes the issue.

2. Add a call to getFilesDir() to the top of Application.onCreate(). This workaround does NOT fix the issue. When I call getFilesDir() here I actually check File.exists() which is returning TRUE. But even still I get the same error:

io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/<package>/files/default.realm.management': make_dir() failed

How could this still be failing? The call to getInstance() is definitely happening after the Application.onCreate() method.

[beeender]

@rajuashok When does the getFilesDir in DataStore get called in your 1 workaround?

[kneth]

@Zhuinden You might have guessed, I suffer from a Samsung allergic reaction ;-)

[Zhuinden]

Everybody does. Especially @beeender after having to fix a broken memcpy call

[rajuashok]

Hey @beeender this is a general order of operations:

Application.onCreate() {
    ....
    Realm.init(this);
    ....
}
// install native lib for external module that runs in a separate process.
...
DataStore.initialize();
File f = getFilesDir();
f.exists(); // Returns true.
...
// Eventually Realm.getInstance() is called which successfully returns a Realm instance.

If I do the following it does NOT work:

Application.onCreate() {
    ....
    Realm.init(this);
    ....
}
File f = getFilesDir();
f.exists(); // Returns true.
// install native lib for external module that runs in a separate process.
...
DataStore.initialize();
...
// Eventually Realm.getInstance() is called which successfully returns a Realm instance.

[rajuashok]

Another point I forgot to mention is that in the first workaround attempt, where the getFilesDir() method is called after DataStore.initialize() it's called in a background thread: Schedulers.io() using RxJava.

Of course in the second case the call is done on the main thread in Application.onCreate(). I don't see how this could make a difference access wise as both live in the same process. However, maybe there is a race condition? The logs however clearly show that getFilesDir().exists() returns true BEFORE Realm.getInstance() call.

[rajuashok]

I think we might go with the first workaround, but this is unfortunate because there's no guarantee that the DataStore.initialize() code will run before all calls to Realm.getInstance()...

If you have any suggestions on best practices regarding this that would be helpful.

Btw, is it best practice to call Realm.init() in the Application.onCreate() method?

[kneth]

Yes, calling Realm.init() in Application.onCreate() sounds as a good idea (we do that in our examples).

Interestingly, getFilesDir() did have a bug (https://issuetracker.google.com/issues/36918154) but it is supposed to be fixed.

[cmelchior]

Nice find @kneth. Apparently that bug wasn't fixed until 4.4, which makes me think we want to strengthen our code in this area:

I can see us doing 2 things:

1) Implement the code suggested in https://github.com/realm/realm-java/issues/4493#issuecomment-295349044

2) If the above fail, we can implement a small loop that check for this every X (~16ms) for 100-200 ms? Values open for debate of course.

Neither of this is ideal, but since this will only be triggered in those rare cases where the app is started the first time, I believe we can accept some small UI lag if hitting this case.

[cmelchior]

@rajuashok I took your suggestion and added it to our startup logic in #4559

[rajuashok]

Thanks for looking into this @cmelchior ! How would I be able to test this fix? What version will this fix appear in?

[cmelchior]

Once merged it will be part of the next patch release and immediately in our SNAPSHOT. If you want to test it rght now, you can just copy the method into your own code and call it just before Realm.init().

[rajuashok]

Awesome thank you so much. Will give that a shot.

[cmelchior]

4559 has been merged. It will be part of our SNAPSHOT within the next hour or so.