Realm-java: Primary key not defined for field 'id' in existing Realm file. Add @PrimaryKey.

[edgars-supe]

Hello! I upgraded from 0.88.2 to 0.90.1 and refactored my RealmObject classes and renamed them. Everything works fine if I clear the app data and it's a new, clean Realm, but migrating from the old version doesn't work.

Stacktrace:

io.realm.exceptions.RealmMigrationNeededException: Primary key not defined for field 'id' in existing Realm file. Add @PrimaryKey.
at io.realm.ChannelRealmProxy.validateTable(ChannelRealmProxy.java:154)
at io.realm.DefaultRealmModuleMediator.validateTable(DefaultRealmModuleMediator.java:71)
at io.realm.Realm.initializeRealm(Realm.java:316)
at io.realm.Realm.createAndValidate(Realm.java:284)
at io.realm.Realm.createInstance(Realm.java:263)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:118)
at io.realm.Realm.getDefaultInstance(Realm.java:195)
...

Channel.java:

public class Channel extends RealmObject {
    @PrimaryKey
    private long id;
    private String name;
    //...

DbMigration.java:

//Don't want to create a Channel schema, if the previously named one exists
if(schema.get("TvObject") == null && schema.get("Channel") == null) {
    schema.create("Channel")
        .addField("id", long.class, FieldAttribute.PRIMARY_KEY)
        //redacted, just a bunch of String, int and boolean fields
}
//...
if(oldVersion == 7) { 
    schema.rename("TvObject", "Channel");
    schema.get("Channel")
        //redacted the changes, it's a bunch of field renaming and removing
    oldVersion++;
}

It also happens with two other classes that I renamed, and it's not always the same one that causes the crash. It changes when I redo the upgrade from the old app version to the new one.

I've tried using Long with @Required instead of long, adding the primary key in the migration (which causes an exception), nothing helps. Am I doing something wrong?

P.S. Is there a better way to structure migrations with regards to creating the objects? If I rename the objects again, I'll have three == null checks in the if()s.

[stk1m1]

Hi @edgars-supe

The exception you are experiencing RealmMigrationNeededException: Primary key not defined for field 'id' in existing Realm file. Add @PrimaryKey. could happen when 1) your old schema TvObject doesn't have @PrimaryKey whileas your new schema Channel does and 2) you don't add FieldAttribute.PRIMARY_KEY attribute to the old one.

Could you add FieldAttribute.PRIMARY_KEY to older version ( <= 7 ) of TvObject to see if this is the case?

It would put us in much better position to help you if you could share your code. You can send a small sample to [email protected] if you prefer.

[edgars-supe]

The @PrimaryKey has always been there. I didn't even rename the field.

I seem to have solved the issue by adding another migration:

if(oldVersion == 9) {
    RealmObjectSchema channel = schema.get("Channel");
    if(!channel.hasPrimaryKey()) channel.addPrimaryKey("id");
    RealmObjectSchema event = schema.get("Event");
    if(!event.hasPrimaryKey()) event.addPrimaryKey("id");
    RealmObjectSchema myChannel = schema.get("MyChannel");
    if(!myChannel.hasPrimaryKey()) myChannel.addPrimaryKey("xprsId");
    oldVersion++;
}

It's weird that this works, because when I try to add the migration right after renaming the schema, before changing other fields, it throws an exception, saying the Primary Key is already defined. If I do it after changing the fields, same thing. But if I separate it like that, it works. Going from an older app version to the new version works fine, too.

[stk1m1]

@edgars-supe

When you migrate to a newer version, you want to cover _all_ older versions appropriately, not just version 7 or 9. The reason we specify a version number is to allow per-version specific migration logic. It seems the renaming has been introduced from version 7.

Could you look into your migration and check if every version is propery covered? You can share your code with us for further discussion.

[edgars-supe]

I've sent the migration class file to [email protected], as you suggested earlier.

I'm not sure I understand what you mean by fully covering all past versions. From my understanding of migrations, I should not change the past version migrations, because if someone's upgrading from v3 to v7, he still needs the incremental changes before a big rename or whatever. In the case of a new Realm file (e.g., fresh app install), I'm guessing, it creates the schema as specified in the migration, but since it starts out with the latest version, it doesn't go through the incremental version changes.

Please correct me if I'm wrong. Or maybe I've misunderstood you.

[Zhuinden]

The incremental migration would mean that instead of

if(oldVersion == 9) {

You would have

if(oldVersion <= 9) {

If I understand right.

[beeender]

@edgars-supe What is the old definition of TvObject? Would you please share both old definition of TvObject and new Channel with us?

[stk1m1]

@edgars-supe

I've checked your migration code with @beeender and here's what we've found.

From my understanding of migrations, I should not change the past version migrations, because if someone's upgrading from v3 to v7, he still needs the incremental changes before a big rename or whatever.

Yes. you are precisely correct about this. I'm pointing you need to incrementally cover _every_ past version so that no version migration would catch an user in surprise.

In the case of a new Realm file (e.g., fresh app install), I'm guessing, it creates the schema as specified in the migration, but since it starts out with the latest version, it doesn't go through the incremental version changes.

In case of a fresh install, migration simply won't happen and it does not go through incremental migration steps to _get the schema_. Realm takes care of creating the schema, not your migration. Hope you see the difference.

If you can send TvObject and Channel to us, we will return you a revised migration logic. We'll look forward.

[edgars-supe]

@beeender I've sent an email to [email protected] with the code.

@stk1m1 That makes sense, it's just that once I added migrations to my project, it threw an exception about needing a migration _on a clean install_. Of course, it's likely I messed something up, it wasn't actually clean or whatever. Anyway, as I said, I've e-mailed the code.

[beeender]

@edgars-supe Thanks for sharing the source code! it seems to be no problem with it.
We used to have a similar bug in migration long time ago, but it has been fixed by https://github.com/realm/realm-java/pull/2663

Is it possible to share your realm db file at version 6 ? If you can share your project as well, it would be perfect. Otherwise we can check the db file first.

[edgars-supe]

@beeender How can I get the .realm file? I tried doing adb pull /data/data/..., but it wouldn't let me.

[beeender]

try copy it to some place without access restriction.

adb shell run-as <your.package.name> cp /data/data/... /storage/sdcard0/download`

then pull it from there.
This might not work with Android M.
But if you are using Genymotion, you will always have root permission.

And check this link http://stackoverflow.com/questions/18471780/android-adb-retrieve-database-using-run-as

[edgars-supe]

@beeender Thanks, that got it working! I've sent the .realm file to [email protected]. Unfortunately, no, I can't share the project.

[beeender]

@edgars-supe Got your db file. And it seems i can reproduce your issue. Please give me some time to check and get back to you.

[edgars-supe]

@beeender No problem, thank you very much for helping! :)

[stk1m1]

@edgars-supe

The root of the cause isn't as straight-forward as it was predicted. I'm escalating this to a bug, and we'll take a thorough look into it.

[stk1m1]

@edgars-supe

The issue has been identified and fixed. Once our next release is deployed, there won't be the issues you've experienced. Hope this helps your development cycle. I'm closing the issue for now.