Realm-cocoa: Synced Realms cannot have transactions larger than 16MB

[molind]

Goals

Copy data from local realm to synced realm.

Expected Results

Data is copied, transaction is commited.

Actual Results

Something underneath commitTransaction throws exception exception 'RLMException', reason: 'Changeset too large'

Same code works perfectly on two local realms.

Steps to Reproduce

1. Create realm with 60k objects in one table and 20k objects in another table. (realm size is around 10Mb)
2. Try to move data into synchronized realm in one transaction.

Code Sample

- (void) enableSync {
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
        RLMRealm *curRealm = [RLMRealm defaultRealm];

        RLMRealmConfiguration *config = [[RLMRealmConfiguration alloc] init];
        config.syncConfiguration = [[RLMSyncConfiguration alloc] initWithUser:self.user realmURL:self.realmURL];
        NSError *error = nil;
        RLMRealm *newRealm = [RLMRealm realmWithConfiguration:config error:&error];
        if(newRealm != nil && error == nil) {
            [newRealm beginWriteTransaction];
            [self copyAllObjectFromRealm:curRealm toRealm:newRealm];
            [newRealm commitWriteTransaction];

            dispatch_async(dispatch_get_main_queue(), ^{
                [RLMRealmConfiguration setDefaultConfiguration:config];
                [[NSNotificationCenter defaultCenter] postNotificationName:kNFRealmChanged object:nil];
            });
        }
    });
}

-(void) copyAllObjectFromRealm:(RLMRealm *)fromRealm toRealm:(RLMRealm *)toRealm {
    for(ModelFolder *item in [ModelFolder allObjectsInRealm:fromRealm]) {
        [ModelFolder createOrUpdateInRealm:toRealm withValue:item];
    }
    for(ModelBookmark *item in [ModelBookmark allObjectsInRealm:fromRealm]) {
        [ModelBookmark createOrUpdateInRealm:toRealm withValue:item];
    }
    for(ModelTrack *item in [ModelTrack allObjectsInRealm:fromRealm]) {
        [ModelTrack createOrUpdateInRealm:toRealm withValue:item];
    }
}

Version of Realm and Tooling

Realm version: 2.3.0

Xcode version: 8.2.1

iOS/OSX version: 10.2

Dependency manager + version: static framework copied manually

[luna-kx]

Thank you for reaching out, @molind! I'll have someone review what you've provided and respond with a solution or follow-up questions as soon as possible. 🚀

[stel]

@molind try to split copying to separate transactions.

You get this exception because Realm has a limitation for holding data in properties (16 MB) and your sync changeset exceeds this limit. @morten-krogh please correct me if I'm wrong.

[morten-krogh]

Yes, @stel is right.

[jpsim]

I think @stel is _almost_ right. It's true that Realm properties can only store up to 16MB, but the limitation for a single transaction to be under 16MB is only for synchronized Realms and is a temporary limitation that we intend to fix at some point. That constraint doesn't apply to non-synchronized Realms.

[molind]

Thanks for you answers @stel, @jpsim! It would be helpful to see notes about this limitation in documentation. May be as list of functions that could throw an exception. By now I'll add exception handler for commitWriteTransaction and try to sync data in multiple transactions in case of exception.

Is there a way to check current size of transaction? It would be useful to know when you should stop inserting data into it and commit, and start new one.
Could you provide an estimate when you'll remove this limitation on synchronized realm?

[luna-kx]

Thanks for the follow-up questions, @molind. We'll try to respond to these asap.

[jpsim]

Is there a way to check current size of transaction? It would be useful to know when you should stop inserting data into it and commit, and start new one.

This isn't a limitation we ever intended to ship with, so we don't exactly have a great way to avoid it. Short of lifting the restriction altogether, which would be ideal.

Could you provide an estimate when you'll remove this limitation on synchronized realm?

I'll defer to @morten-krogh if he's comfortable sharing that. Meanwhile I've created an internal-use issue to track this at the sync client level here: realm/realm-sync#1124.

[morten-krogh]

I will look into what it takes to lift the restriction.

[morten-krogh]

I have basically lifted this restriction in a working branch. There are a few issues left.

[mokkun]

This issue seems to happen with realm-java as well.

[morten-krogh]

The removal of the 16MB limit has now been merged into the develop branch.

[jpsim]

Great work @morten-krogh! We'll keep this issue open until the improvement is available in this repo.

[jpsim]

Fixed in #4915.