Realm-java: Android Migration like Cocoa/Swift

[bsmn99]

Hi,

I just have a request,

why migration code is different from ios to android? in ios they just increasing the schema version and the altered table structure is updated automatically, but in android we have to write the migrate code for altered table as well as the schema version to be increased. Can u make in android too just increasing the schema number is able to alter the table? now its horrible to write migrate code for changes, and that too we have to make a track for what columns added, deleted and updated. Pls consider this..

[cmelchior]

Hi @bsmn99

It is also an discussion we had internally and there are good arguments for both types of migrations.

Be aware that the iOS migrations are not automatic in all cases and those corner cases are non-trivial to solve and will have to be maintained even for future migrations.

The Android migrations require more boilerplate code for sure, but they are write-once. After that you can forget about them and they will continue to work. Also we have plans for creating an IntelliJ plugin that can help with creating the migration code, making it less painful.

Note that if you do not want to keep your data across schema changes, you can always use RealmConfiguration.deleteRealmIfMigrationRequired(). For some use cases like offline caches, that is a lot easier as you don't have to write any migration code.

For the current future we plan to let iOS and Android behave differently in this area. That also enable us to gather more realworld input about both approaches.

[bsmn99]

Hi @cmelchior

Thanks for the clarification, expecting that IntelliJ plugin!..

[bsmn99]

@cmelchior

_Be aware that the iOS migrations are not automatic in all cases and those corner cases are non-trivial to solve and will have to be maintained even for future migrations._

Can u explain what kind of corner cases for iOS?

[cmelchior]

A couple of cases:

a)
v1: Add new field name
v2: Remove field name.
v3: Add field name again. name should be empty here, not contain all the v1 data.

b)
v1: Add field email
v2: Rename email to contact
v3: Add field email again. This should now be empty while contact contains all the data

A good way to think about is that iOS always go from currentVersion -> latestVersion skipping any intermediate calculations while Android always go v1 -> v2 -> latestVersion, which means that all steps are applied. So if you are changing the same field across multiple migrations on iOS you usually have to be a bit careful. This is also described in their docs btw: https://realm.io/docs/objc/latest/#linear-migrations

[vonox7]

For the current future we plan to let iOS and Android behave differently in this area.

@cmelchior: As an app development company, we have to support of course both Android and iOS. Every difference between those two platforms makes our work much harder. Realm should make lives as a developer easier. Easier means less things to think about. So please decide on one way, but be consistent for all platforms.

[cmelchior]

One way to get around this could perhaps be to add two different RealmMigration interfaces:

public interface RealmMigration {
   public void migrate(DynamicRealm realm, long fromVersion, long toVersion);
}

public interface RealmAutomaticMigration {
  public void migrate(Migration migration, long fromVersion);
}

public class Migration {
  // Mimic Swift class: https://realm.io/docs/swift/2.4.0/api/Classes/Migration.html
}

That would allow us to support both ways of doing it without breaking existing support.

[Zhuinden]

A lot of people seem to claim that "automatic schema migration is something they miss in Realm", so this is a good idea.

[BoD]

As a new Realm user, I can indeed confirm I was very surprised (and disappointed) to see this is necessary. I was expecting some code to be needed for cases like adding/removing fields in a model. But the "new models" case should be automatic - after all, Realm knows everything that needs to be known about them!
At least, could we need schema.create("NewModel"), but not having to do all the addField? :)

[davidgarciaanton]

Is there any hint if this will be included in development for the near future versions?

[Zhuinden]

You know, after having worked with SQLite as well this summer, in which case you need to bump the schema version and add the new columns and new tables as part of a migration; this behavior is pretty much exactly the same.

[cmelchior]

@davidgarciaanton We will probably add it at some point as a complementary way of doing migrations, but it does not have a very high priority so I cannot give you any timeframe.

[Zhuinden]

Honestly, the real deviant here is the cocoa migration.

I think .NET also has the DynamicRealm thing going on.

[davidgarciaanton]

Ok @cmelchior good to know.

@Zhuinden not sure what do you mean, do you suggest this is not a needed feature? Actually, the only thing I'm missing is auto-creation of brand new entities (for possibly big new additions to the Persisted data model). For any change on existing model classes current behaviour seems nice to me already.

I'd only like to avoid some _boilerplate_ code creating a migration for a new entity (_from scratch_). See my StackOverflow question here

[Zhuinden]

@davidgarciaanton I used to expect automatic migrations too, but once you start migrating SQLite databases you realize this is a necessary evil in any database.

Otherwise, how will you define a migration from version 3 to version 5 in a reliable manner? Even for "just new classes", you end up with having to define additional fields between 4->5 and stuff.

That's why personally I'm not sure how one would simplify the needed boilerplate.

---

Technically Realm has a schema mode called "additive" when connected with a ROS, which is automatic, but that doesn't allow destructive operations on the schema. I think that is trickier.

---

What I'm saying is that this is not as much of a "dealbreaker" as some people tend to suggest: the other alternative requires the exact same steps to change the database, just with different syntax.

[davidgarciaanton]

@Zhuinden I totally understand what you say, I'm only requesting some entry point that allows to create the whole entity with a single call within the migration itself. That's what I was using the family of createRealmObjectSchema methods for.

I knew that I had some weird code within the migration, but in an ongoing project that required several new model classes to be added it payed the difference.

I'm not the kind of person that likes magic behind the scenes for an _auto_ migration, but I'd like to have some public accessor to force entity creation.

[Zhuinden]

@davidgarciaanton the problem with that is that later in the project it won't be fool proof. Let's say you add to schema in version 4, and you remove a field in version 5.

Although I guess you could just put a "if(has field)" check before removing, and "if(!has field)" before adding.

[AamirAbro]

I just spent a day fighting with realm to migrate db and hitting issues in an app which used to use deleteRealmIfMigrationRequired before.
Now after moving to migrate we had implement migration for users all the way to back to first release to support it.

addField looked like a generic method that can work for every object; but turns out we have to use addRealmObjectField for Realm objects ;(

Best part is there is was no auto update for a schema change which only added new fields.

I guess we at-least need an option to to auto migrate on purely additive changes.

[Zhuinden]

@davidgarciaanton and @AamirAbro

I've put together this heavily experimental thing called AutoMigration, which removes the fields no longer in your models, and adds the fields that are in the models but not in the schema.

Please note the use of annotations that you need to apply, otherwise it won't work. Also it is experimental so not a library yet, because of how you need to add things like @AutoMigration.MigratedField essentially duplicating attributes (follow in https://github.com/realm/realm-java/issues/5344 )

Also worth noting that rename == remove + add, so you might want to apply your own migration logic first?

EDIT from 2020: Also, I don't even know if it handles all edge cases. I don't actually support this thing anymore. Beware.

[davidgarciaanton]

Thanks @Zhuinden I'll give it a try as soon as I have the opportunity.

[jgavris]

@cmelchior Checking out this discussion, did the IntelliJ migration generator plugin ever materialize? I use Realm on iOS and indeed we find the auto-migrations there are the typical case (with well-designed data model evolution), and the lack of auto-migrations on Android are a lot less developer friendly.

Also we have plans for creating an IntelliJ plugin that can help with creating the migration code, making it less painful.