Hive: DateTime objects can't be used as key

Created on 11 Nov 2019  路  4Comments  路  Source: hivedb/hive

DateTime objects can't be used as keys. The reason seems to be that DateTime is serialised as an int timestamp, which then gets rejected for being larger than 4294967295 (32-bit unsigned int).

Where does this limitation come from? Does it still make sense?
Dart 2.0 uses 64-bit ints: https://stackoverflow.com/a/31691112

My use-case involves a database where DateTime is being used as unique id, since the entry should be updated if a value already exists for that exact timestamp.
Currently I managed to get around this by using DateTime.toString() as key, but I'm sure I'm paying a hefty performance penalty by doing that.

Steps to Reproduce

  1. Try to put a value with a DateTime as key: _box.put(entry.dateTime, entry);
  2. Exception is thrown: 'package:hive/src/binary/frame.dart': Failed assertion: line 20 pos 13: '(key is int && key >= 0 && key < 4294967295) || (key is String && key.length <= 255)': Unsupported key

Version

  • Platform: Linux
  • Flutter version: 1.10.16-pre.71
  • Hive version: 1.1.1
bug

Most helpful comment

Where does this limitation come from? Does it still make sense?

I added this limitation on purpose. There are two reasons:

  1. 32-bit keys save space. In the vast majority of cases, not even the 32 bits are needed so I thought it would be a good compromise.
  2. For some reason, converting integers to bytes and vice versa is super slow in Dart compared to other operations.

Currently I managed to get around this by using DateTime.toString() as key, but I'm sure I'm paying a hefty performance penalty by doing that.

This is what I would recommend in your case. The performance penalty is really neglectable since storing ASCII strings is very fast.

I would assume the keys get hashed anyways.

They are not. It would not allow sorting the entries by their string key (which is a very powerful tool if used correctly) and it would not be possible to get the keys stored in a box.

But now that you mention it, using DateTime.hashCode may be a more efficient way of doing what I need. It breaks sorting but that's not a requirement for me.

You can provide a custom sorting comparator.

All 4 comments

Hmm is there really a performance penalty? I would assume the keys get hashed anyways...

Just quickly opening the database file in a text editor, it seems string keys are stored as plain text.
Other than that I guess there's a bit of a penalty in the DateTime.toString() method, as well as the amount of data that needs to be written to, and read from the file. In my case that simple change almost duplicates the file size.

But now that you mention it, using DateTime.hashCode may be a more efficient way of doing what I need. It breaks sorting but that's not a requirement for me.

Still, I'm curious about the UINT32 limitation. If it were removed then the problem could be solved in a much more elegant way.

Where does this limitation come from? Does it still make sense?

I added this limitation on purpose. There are two reasons:

  1. 32-bit keys save space. In the vast majority of cases, not even the 32 bits are needed so I thought it would be a good compromise.
  2. For some reason, converting integers to bytes and vice versa is super slow in Dart compared to other operations.

Currently I managed to get around this by using DateTime.toString() as key, but I'm sure I'm paying a hefty performance penalty by doing that.

This is what I would recommend in your case. The performance penalty is really neglectable since storing ASCII strings is very fast.

I would assume the keys get hashed anyways.

They are not. It would not allow sorting the entries by their string key (which is a very powerful tool if used correctly) and it would not be possible to get the keys stored in a box.

But now that you mention it, using DateTime.hashCode may be a more efficient way of doing what I need. It breaks sorting but that's not a requirement for me.

You can provide a custom sorting comparator.

Thanks for the clarification. I'm closing the issue since the limitation is by design.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ProfileID picture ProfileID  路  4Comments

maxim-saplin picture maxim-saplin  路  3Comments

SergeShkurko picture SergeShkurko  路  4Comments

yannickvg picture yannickvg  路  4Comments

MyoLinOo picture MyoLinOo  路  3Comments