Realm-java: Realm.copyToRealm() should set PrimaryKey values directly

[RobertoArtiles]

Hi guys,

It's known that null is not supported. But it seems like it could lead to the DB corruption if you try to write something with a null @PrimaryKey field. The best would be to just crash before putting something into the DB.
So what happened with me. I by mistake executed something like this:

        Realm realm = Realm.getInstance(this);
        realm.beginTransaction();
        UserRealm userRealm = realm.createObject(UserRealm.class);
        realm.commitTransaction();
        realm.close();

UserRealm class:

public class UserRealm extends RealmObject {
    @PrimaryKey private String token;

    // getter and setter
}

So the first time I executed it nothing crashed. But starting from the second execution it was throwing Caused by: io.realm.exceptions.RealmException: Primary key constraint broken. Value already exists:. And even when I tried to write something with a set primary key it still failed with the same exception:

        Realm realm = Realm.getInstance(this);
        realm.beginTransaction();
        UserRealm userRealm = realm.createObject(UserRealm.class);
        userRealm.setToken("some_token");
        realm.commitTransaction();
        realm.close();

And if you do realm.allObjects(UserRealm.class) it returns one user with a null token. So basically DB is corrupted after that. And you can't write anything in this "table".

Looks like a great way to shoot yourself in both legs in production.

Full Stacktrace:

io.realm.exceptions.RealmException: Primary key constraint broken. Value already exists:
            at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2184)
            at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2233)
            at android.app.ActivityThread.access$800(ActivityThread.java:135)
            at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1196)
            at android.os.Handler.dispatchMessage(Handler.java:102)
            at android.os.Looper.loop(Looper.java:136)
            at android.app.ActivityThread.main(ActivityThread.java:5001)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:515)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
            at dalvik.system.NativeStart.main(Native Method)
     Caused by: io.realm.exceptions.RealmException: Primary key constraint broken. Value already exists:
            at io.realm.internal.Table.throwDuplicatePrimaryKeyException(Table.java:687)
            at io.realm.internal.Table.addEmptyRow(Table.java:376)
            at io.realm.Realm.createObject(Realm.java:1086)
            at chocodile.witdot.com.playground.MainActivity.onCreate(MainActivity.java:31)
            at android.app.Activity.performCreate(Activity.java:5231)
            at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
            at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2148)
            at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2233)
            at android.app.ActivityThread.access$800(ActivityThread.java:135)
            at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1196)
            at android.os.Handler.dispatchMessage(Handler.java:102)
            at android.os.Looper.loop(Looper.java:136)
            at android.app.ActivityThread.main(ActivityThread.java:5001)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:515)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
            at dalvik.system.NativeStart.main(Native Method)

[cmelchior]

Actually, the DB is not corrupted. All types are assigned their default value when new objects are created, in this case "" (empty string). So the error message is actually correct. You are trying to create another object with the same primary key.

Eg. this will also fail, no matter if the primary key is on a int or a String.

Realm realm = Realm.getInstance(this);
realm.beginTransaction();
realm.createObject(UserRealm.class);
realm.createObject(UserRealm.class);
realm.commitTransaction();
realm.close();

Disallowing the empty string as a value is not really great either, as you might actually want to use that as a primary key value. Adding support for Null will not make this easier, as primary key fields will not be allowed to be nullable.

The work-around would be to use Realm.copyToRealm() with stand-alone objects, which should set the primary key directly.

Realm realm = Realm.getInstance(this);
realm.beginTransaction();
UserRealm user = new UserRealm();
user.setToken("xxx");
realm.copyToRealm(user);
realm.commitTransaction();
realm.close();

Except I just checked the code, and currently we create an empty object before setting values, so this will also break, but that is definitely a bug that should be fixed.

We have considered adding a method like:

Realm.createObject(Object primaryKeyValue);

But so far hasn't as it reads really wonky from a user perspective.

[RobertoArtiles]

Ok, I see. Thanks. But just to clarify. Currently if I will do this:

Realm realm = Realm.getInstance(this);
realm.beginTransaction();
realm.createObject(UserRealm.class);
realm.commitTransaction();
realm.close();

or this:

Realm realm = Realm.getInstance(this);
realm.beginTransaction();
UserRealm user = new UserRealm();
realm.copyToRealm(user);
realm.commitTransaction();
realm.close();

I won't be able to create a new UserRealm object after. Because it will always first create an object with an empty primary key which conflicts with the one which is in the database.
Is this correct?

[cmelchior]

Yes, that is correct.

[RobertoArtiles]

And just a general remark.

realm.beginTransaction();
UserRealm user = realm.createObject(UserRealm.class);
user.setToken("xxx");
realm.commitTransaction();

For me as a user of this library it seems very confusing that even though it's a transaction, it can fail in between of beginTransaction and commitTransaction because of the conflict.
Before you explained the reason of this bug I thought that it works more like a builder pattern, so that there is no difference between:

realm.beginTransaction();
UserRealm user = realm.createObject(UserRealm.class);
user.setToken("first");
user.setToken("second");
user.setToken("third");
user.setToken("first");
realm.commitTransaction();

and this

realm.beginTransaction();
UserRealm user = realm.createObject(UserRealm.class);
user.setToken("first");
realm.commitTransaction();

But apparently there is. And I'm just saying it from the user perspective. Totally understand there could be some technical limitations, but I would really like to see it more in a builder style than this implicit approach.

[emanuelez]

Hello Roberto,
that is a fair point, but it goes against Realm's principle of never copying data to and from memory.
As soon as you call setToken("something") the data will be visible in the current thread immediately and in the other threads after committing the transaction.

But you raise a fair point, namely that we need to improve our documentation in order to avoid such incorrect assumption in our users.

[RobertoArtiles]

Hi Emanuele,

Thanks for the explanation. You are right about the documentation. Would be great to see some section which explains in details how Realm works internally and it's philosophy. Currently it's kind of a black box, and users' decisions could be made only based on some examples and method descriptions.

[cmelchior]

Fix has been merged to master.

[bananabastard]

But what to do with createObjectFromJson method?
This isn't working. The primary key is 0 after first object created(no matter that the primaryKey variable is 1 in first time) and on the next time it will be 0 too and this will crash

realm.beginTransaction();

    try {
        long primaryKey = realm.where(Category.class).maximumInt(Constants.MODEL_FIELD_NAME_ID) + 1;
        if (primaryKey < 1){
            primaryKey = 1;
        }
        Category category = realm.createObjectFromJson(Category.class, stream);
        category.setId(primaryKey);
    } catch (IOException e) {
        stream.close();
        realm.cancelTransaction();
    } finally {
        stream.close();
        realm.commitTransaction();
    }

[cmelchior]

@bananabastard
Setting the PrimaryKey in it's own statement like this is not safe, as you point out, between you creating the object and setting the primary key the value will be 0 which can conflict with an existing object.

If your model has primary keys you should really use createOrUpdateFromJson(stream) as that does the correct thing.

You are right that createFromJson doesn't support the use case you have outlined, but that is by design. The reason is that a streaming API with primary keys will have to parse the entire object before inserting it in order to validate the PrimaryKey contract. createOrUpdateFromJson does this, while createFromJson does not.

Doing the below should work:

realm.beginTransaction();
realm.createOrUpdateObjectFromJson(Category.class, stream);
realm.commitTransaction();

[tanjc78]

@cmelchior
how do you target an object which already has a primary key and update the data within? According to Realm documentation, update has to be done within Writes, and any form of writes which involves an existing primary key will result in exceptions. The quote which you provided @bananabastard ends up having the value exists error..

quote:
realm.beginTransaction();
realm.createOrUpdateObjectFromJson(Category.class, stream);
realm.commitTransaction();

error message:
io.realm.exceptions.RealmPrimaryKeyConstraintException: Value already exists: 0

do advise how can we update existing data with primary keys? Cheerios

[jadel42]

So from this discussion it sounds like realm.createObject(...) should never be used because one incorrect insert could result not subsequently being able to perform any inserts.

If this is the case then, the documentation should be updated to reflect this.

[jjhesk]

i have done this with

  instance.beginTransaction();
            LocalData dat = instance.createObject(LocalData.class);
            dat.setBasemap_uuid(uuid_basemap(map));
            update(dat, map);
            instance.commitTransaction();

Looks like i got an Caused by: io.realm.exceptions.RealmPrimaryKeyConstraintException: Value already exists:

[Zhuinden]

You should use either realm.createObject(clazz, primaryKeyValue); or set the fields of an unmanaged object and use insert() or insertOrUpdate()

[vyshnavkr]

@Zhuinden
Hi,
I'm fetching a Json data. To store it in Realm, I used insertOrUpdate(). But instead of updation the data was inserted each time I loaded. I found out there wasn't primary key in my model. How do I set primary key without violating the Json model class?

[Zhuinden]

@VyshnavKR this entirely depends on your JSON model, and whether it has a id field or something similar.

If it does, you should add @PrimaryKey annotation for that field in your Realm object, and a migration that sets addPrimaryKey(fieldName) on the object's schema.

[vyshnavkr]

@ Zhuinden
My Json model does not contain id field or something similar. How do I solve?

[Zhuinden]

What identifies your object? How would you know that you are looking at the same object?

[vyshnavkr]

From the json response, no field is unique. Its actually train schedule data. So should I set PK manually? Using another class?

[vyshnavkr]

@ Zhuinden
From the json response, no field is unique. Its actually train schedule data. So should I set PK manually? Using another class?

[vyshnavkr]

@ Zhuinden
This is the api
www.railwayapi.com/api/#live-train-status

[Zhuinden]

You could combine the schedule date formatted to string along with the id of the train or the line or something

[Zhuinden]

Train number, schedule arrival date and time and station code should be concatenated into one, and used as primary key.

[vyshnavkr]

@Zhuinden
Thankyou...
Will defining this PK in POJO prevent json from getting parsed? Or should I add the JSON response and this PK(as a seperate class) to a new class?

[Zhuinden]

I tend to have a separate RealmObject and separate JsonObject and a mapping layer in between, but even if you define the new field, it will just be null when parsed.

[vyshnavkr]

@Zhuinden
Sorry I am getting confused. I think I should post this in Stack rather than continuing here. I will post the StackOverflow link here soon Zhuinden..

[vyshnavkr]

@Zhuinden
Hi, this is the link
http://stackoverflow.com/questions/40372712/realm-android-insertorupdate-not-updating