Realm-cocoa: Support grouping in RLMResults

[jpsim]

API suggestions welcome. Could be done by passing a property string and maybe support nested properties. Open questions:

1. Should this be done as a separate type (e.g. RLMGroupedResults) or as an addition to RLMResults.
2. How would this work with fine-grain notifications?
3. How would this work with RLMCollection/RealmCollectionType?
4. How does this affect further querying/filtering?

[beloso]

As a developer using realm.

I would see it either returning a RLMGroupedResults that would resemble a dictionary structure. Where the keys would be the various values for the key path used to group results. The values of the array could possibly be normal RLMResults instances.

For fine grained notifications I would expect to know that the result set for a given key on the structure above was updated, possible knowing which were inserted/updated/deleted.

I am not sure if this is the kind of feedback you were expecting.

[jpsim]

Yes, this kind of feedback is useful, thanks!

[kunalsood]

+1
This is the only use case that I have come across, that current workarounds to lack of distinct query don't solve in a good way.

To add to the feedback provided by @TiagoVeloso..

I would expect the proposed RLMGroupedResults not to contain any empty group, i.e. each of the array values in the key-value pairs of the proposed dictionary structure should contain at least one object.

Further, each key should be unique, i.e. no two keys in the group should be considered equal (using isEqual: or the like).

Since it has not already been stated.. I would expect it to resemble an _ordered_ dictionary.
Ability to sort the keys would be essential, so would the ability to filter.

Fine-grain notifications can take an approach similar to Apple's NSFetchedResutlsController. Provide information about which sections were inserted/moved/updated/deleted with the section's index, and only provide indexPaths for inserted/updated/deleted objects contained within array values.

As for how this would work with RLMCollection/RealmCollectionType.. Just an idea/thought: Apple's NSFetchedResutlsController has a fetchedObjects property which returns an NSArray, similarly RLMGroupedResults (or RLMResultsGroup) can have a containedResults property which could return an RLMResults (which already conforms to said protocols).

[jpsim]

All great ideas @kunalsood! Although I'm not convinced of the constraint to never have empty groups, but it's pretty early for me to consider at this point, so who knows where this will go.

[LeffelMania]

I wouldn't be opposed to an API that simply accepted an array of RLMResults objects to manage in-aggregate.

RLMResults *section1 = ...;
RLMResults *section2 = ...;

RLMResultsGroup *sections = [RLMResultsGroup resultsGroupWithResults:@[ section1, section2 ]];

Fine-grained notifications would then have to work with index paths everywhere (instead of indexes) I think to make APIs consistent. You'd then be able to register a notification block on the RLMResultsGroup object that would aggregate changes from all of the sections into one change set.

This is mostly motivated by how we've implemented sectioned table view results in our application - the main obstacle has been having separate change notifications for each RLMResults section, which can lead to NSInternalInconsistencyExceptions if you try to process updates for one section when the data in more than one section has changed. Being able to register multiple RLMResults sets with one notification block would solve that problem, so that's where I'm currently coming from.

Having this kind of container approach would also allow RLMCollection classes to remain the same; you could simply expose index-path access for RLMResultsGroup, which would drill into the appropriate collection and use existing index-based access.

[rex-remind101]

+1000000

[jpmcglone]

How do I currently get two tokens powering two different sections of my UITableView? Am I doing something wrong, because right now deleting from one section adds it to the other section, but i'm getting really weird change results.

[sxua]

@jpmcglone Just use one token for your main query and do all sectioning logic by yourself.

[alexeychirkov]

Whats the status if this issue?
I have seen examples of the "sectioned UITableViewController" only with static (constant) sections array.
Anybody has an idea how to solve this problem in elegant way?

[jpsim]

@alexeychirkov have you seen our GroupedTableView example? That example has static sections, but there's this StackOverflow example showing dynamic sections: http://stackoverflow.com/a/38797693/373262

[jpmcglone]

My problem is when, say, something moves from section 0 to section 1. Will be looking through the examples, thanks!

[jpmcglone]

So in the example, the notificationToken just calls reloadData. Was hoping to get animations working in a grouped tableview

[jpsim]

Yeah, animations for grouped results are certainly harder, and one of the use cases I hope addressing this feature request would drastically simplify.

It's difficult, but not impossible, to implement yourself right now because you can only have notification blocks for individual Results, but if you know how many notification blocks you have, you can "buffer up" changes until all notification blocks have been invoked, then you can merge and apply them. I understand that's a bit complex, which is why this ticket is still open 😉.

[LeffelMania]

@jpmcglone What we've been doing is keeping a snapshot of visible results per-section to use as our actual data source, and only updating that snapshot per-section when we get a notification for that section.

Roughly:

handleChange(sectionIndex, results, change) {
    ...
    self.tableView.beginUpdates()

    self.visibleResults[sectionIndex] = results.allObjects() // allObjects() captures current set of objects in an array

    self.tableView.insertRows(...)
    self.tableView.deleteRows(...)
    self.tableView.reloadRows(...)

    self.tableView.endUpdates()
}

numberOfSections() {
    return self.visibleResults.count
}

numberOfRowsInSection(section) {
    return self.visibleResults[section].count
}

This keeps the UITableViewDataSource in sync with your results sets as they're updated, because even though all of the RLMResults have live-updated at the same time, the snapshot still reflects the data that hasn't been updated yet.

Full disclosure: this approach hasn't been all smooth sailing for us. Since the visible results aren't live-updating like RLMResults, they can contain invalidated objects, which we can potentially try to dereference and end up with 💥 .

[jpsim]

Yup, that's certainly one way to do it. Like I said, building sectioned results fine-grained collection notifications yourself is _possible_, but a bit complex and likely prone to unsupported edge cases.

All the more reason to build the feature tracked by this ticket.

By the way, we'd love to see the Realm community step in and build features like these that are important to them, but maybe not super high priority for Realm core team members. All the pieces involved are open source (mostly realm-core, realm-object-store and this repo). Though I admit, this feature might be a bit too much of an investment for what outside contributors are willing to commit to, since it does span API design, query syntax, multiple repos/projects/languages, etc.

[stephenkopylov]

Any news about this feature?

Now I am using RBQFetchedResultsController for some tableViews for sections, but it's not such a smooth option and I'm wait for native IsRealm sections so much!
In other tv's Realm's notifications with notification block look and work so beautiful!

[konradzdunczyk]

RBQFetchedResultsController has one big issue - eager load of result objects. If your result has many objects then "unplug" from Realm take a while.

[FreudGit]

Any status on this?

[inPhilly]

Is this on the roadmap for the near future? I am just struggling so much with how Realm can be a replacement for Core Data when it seems that there is not a Realm Native solution that sufficiently emulates the very-important work of NSFetchedResultsController.

[francoishebert]

Hi. My monthly follow up (as no news)

May i ask if you have any status?

I have a realm DB of 300000 objects. I need to filter unique for a specific property, and populate listview with sections grouped with unique. It took 4-5 seconds to get that on older devices. :(

[austinzheng]

If we have any news about this feature we will comment on this thread. We know this feature is important to our users and will build it out as soon as it becomes feasible for us to do so.

[FreudGit]

@austinzheng Will it be implement in your new version? :)

[austinzheng]

We will post when we have something to share. If we don't post, we have nothing to share.

[bmunkholm]

Please see https://github.com/realm/realm-cocoa/issues/6152 for a possible workaround.