Hive: Query support

[leisim]

[AndryHTC]

@leisim that would be awesome... Just the right feature to confidently replace all the other Flutter DBs.

Can't wait to try this out!

[leisim]

I'm working very hard on this feature. Queries on Hive will not support indices and will be computed when they are executed. But they support watching changes which is very efficient.

Here you can find the HiveQuery interface. What do you think?

[AndryHTC]

Great! Very intuitive!

Seems sufficiently comprehensive at a first look, for the use cases that a Key-Value database should give.

Should we expect some types of indexing/improved query speed in the future?

[leisim]

I'm quite unsure about this. Dart is not the fastest language and in some cases extremely slow for example for IO.

And since I believe a mobile database should not contain a huge number of entries, I think the additional cost of calculating and storing indices will not be worth the performance benefits. Especially since Dart's multi threading capabilities are very limited and indices would have to be created in the main thread.

I'd love to hear other opinions or ideas on this topic.

[mdrideout]

Most of my apps that need complex queries are using cloud databases (like firestore, mongo, cloudsql, etc.)... However, I have a few offline-first scenarios where I'll need to be able to query data. But my offline query needs are not that complex. Like maybe...
box.values.where({city: 'Portland'}).toList(); or box.values.where({clientID: '12365476'}).toList();

But maybe I can already do this type of simple query? https://riptutorial.com/dart/example/31185/filter-a-list

[leisim]

But maybe I can already do this type of simple query?

You can using Iterable.filter() the problem is that you cannot subscribe the result and update the UI accordingly. This is what the Hive queries will provide.

Edit: Sorry, accidentally closed the issue.

[winwisely99]

hey @leisim

I saw the query branch a while back and thought that it was merged into master. But since this issue is open i guess it was never achieved ?

This pull seems to refer to query support ?
https://github.com/hivedb/hive/pull/52

Regarding techncial aspects its hard to comment as the branch has gone ?

I saw the https://github.com/leisim/dartx package. It seems that queries can be hand coded using this and iteration ?

Would like to know your thoughts going forward so i can help and get involved.

[leisim]

Hi @winwisely99

Yes, I removed the query branch because it was not compatible with the changes I made in other branches.

I will create a new query branch soon, after v1.2.0 is stable and released.

[Reprevise]

I know you mentioned in #180 that Dart is single-threaded and that's true but there's Isolates and the compute method. What would be your take on using them for large boxes or just any querying whatsoever? @leisim

[leisim]

What would be your take on using them for large boxes or just any querying whatsoever

The problem is that copying data between isolates is quite expensive (no shared memory). I don't think it currently makes sense to provide an API that runs operations in an isolate.

To really make a decision, some benchmarks are required which measure the actual cost of using isolates. I don't have much time currently but I'll keep it in mind.

[winwisely99]

Maybe don't query but instead make materialised views as s result of the mutations. Essentially it's CQRS.

The materialised views don't have to be fast or respect linearizability in database serialisation semantics.

I may be temping fate here though as this is almost like the reflux pattern.

??

[jamesdixon]

@leisim is this still something planned for Hive 1.x? or are you deferring to Hive 2.x?

[leisim]

@jamesdixon @nickreynke

Kind of: I'll publish the Rust part of the new version of Hive very soon. It'll support queries and indices.

Queries in Dart do not make much sense unfortunately. I did not find a way to make them faster than a simple .filter().