Hive: Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.

[daibao520]

hive version: "1.4.0+1"
This happen when my app crash and restart immediately by android system

E/flutter (32001): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
E/flutter (32001): #0      BinaryReaderImpl.readFrame 
package:hive/…/binary/binary_reader_impl.dart:244
E/flutter (32001): #1      FrameHelper.framesFromBytes 
package:hive/…/binary/frame_helper.dart:15
E/flutter (32001): #2      FrameIoHelper.framesFromFile 
package:hive/…/io/frame_io_helper.dart:36
E/flutter (32001): <asynchronous suspension>
E/flutter (32001): #3      StorageBackendVm.initialize 
package:hive/…/vm/storage_backend_vm.dart:72
E/flutter (32001): <asynchronous suspension>
E/flutter (32001): #4      BoxBaseImpl.initialize 
package:hive/…/box/box_base_impl.dart:82
E/flutter (32001): #5      HiveImpl._openBox 
package:hive/src/hive_impl.dart:85
E/flutter (32001): <asynchronous suspension>
E/flutter (32001): #6      HiveImpl.openBox 
package:hive/src/hive_impl.dart:106
E/flutter (32001): #7      HiveConfigBox.open 
package:liaobei/hive_liaobei/hive_config_box.dart:28
E/flutter (32001): #8      HiveLiaobeiManager.openUserBox 
package:liaobei/hive_liaobei/hive_liaobei_manager.dart:80
E/flutter (32001): <asynchronous suspension>
E/flutter (32001): #9      main 
package:liaobei/main.dart:75
E/flutter (32001): <asynchronous suspension>
E/flutter (32001): #10     _runMainZoned.<anonymous closure>.<anonymous closure>  (dart:ui/hooks.dart:239:25)
E/flutter (32001): #11     _rootRun  (dart:async/zone.dart:1126:13)
E/flutter (32001): #12     _CustomZone.run  (dart:async/zone.dart:1023:19)
E/flutter (32001): #13     _runZoned  (dart:async/zone.dart:1518:10)
E/flutter (32001): #14     runZoned  (dart:async/zone.dart:1502:12)
E/flutter (32001): #15     _runMainZoned.<anonymous closure>  (dart:ui/hooks.dart:231:5)
E/flutter (32001): #16     _startIsolate.<anonymous closure>  (dart:isolate-patch/isolate_patch.dart:307:19)
E/flutter (32001): #17     _RawReceivePortImpl._handleMessage  (dart:isolate-patch/isolate_patch.dart:174:12)

[daibao520]

When this error happend even i restart app the error still exist

[HeikoMueller]

I run into the same error, also with hive: ^1.4.1+1
Flutter 1.17.0 • channel stable • https://github.com/flutter/flutter.git
Framework • revision e6b34c2b5c (3 weeks ago) • 2020-05-02 11:39:18 -0700
Engine • revision 540786dd51
Tools • Dart 2.8.1

2020-05-20 10:36:22.133 20482-20654/com.example.v4 E/flutter: [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
    #0      BinaryReaderImpl.readFrame (package:hive/src/binary/binary_reader_impl.dart:249:7)
    #1      FrameHelper.framesFromBytes (package:hive/src/binary/frame_helper.dart:17:26)
    #2      FrameIoHelper.framesFromFile (package:hive/src/io/frame_io_helper.dart:41:12)
    <asynchronous suspension>
    #3      StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:82:30)
    <asynchronous suspension>
    #4      BoxBaseImpl.initialize (package:hive/src/box/box_base_impl.dart:90:20)
    #5      HiveImpl._openBox (package:hive/src/hive_impl.dart:90:17)
    <asynchronous suspension>
    #6      HiveImpl.openBox (package:hive/src/hive_impl.dart:111:18)

[HeikoMueller]

For the moment, I do this as a temporary workaround.

try {
   _box = await Hive.openBox(_boxName);
    } catch(err) {
      /// box could not be opened.
      bool success = false;
      int index = 0;
      while(success == false) {
        _boxName = _boxName + index.toString(); 
        Error _err;   
        try {
          _box = await Hive.openBox(_boxName);
        } catch(err) {
          _err = err;
        }
        if(_err == null) {
          print("Hive box opened with name " + _boxName);
          success = true;
        } else if(index > 100) {
          throw("Hive box could not be opened.");
        }
        index++; 
      }
}

In this provisional way, subfollowing initializations shall find the same new working box name.

However, what is the recommended way to unlock and re-open – or delete – the former box file (in the snippet above "_boxName" without added index) ?

[TheMisir]

Thanks to @Mr-Martini I can now reproduce the issue.

The issue is caused after using key on unencrypted box or not using key on encrypted box.

To fix the issue we might limit encryptionKey usage on boxes. For example if the box is already encrypted you will not able to pass encryptionKey: null on same box. Or if the box is unencrypted you will not able to use encryptionKey. But I'm not sure where to store box is encrypted or not without changing file structure for boxes.

I just wonder is there any other cases that might cause this issue rather than not using encryption key or using invalid key?

import 'dart:io';
import 'dart:typed_data';

import 'package:hive/hive.dart';

/// Runs some tests
///
/// 1. Opens the box
/// 2. Reads some series of values
/// 3. Writes new values
/// 4. Closes
Future<void> test(int index, [Uint8List key]) async {
  var boxName = 'box_$index';
  var hasKey = key != null;
  var info = '(box = $boxName, hasKey = $hasKey)';

  // open
  Box box;
  try {
    box = await Hive.openBox(boxName, encryptionKey: key);
  } catch (error) {
    return print('Crashed! $info');
  }

  // read
  for (int i = 0; i < 100; i++) {
    var value = box.get('item_$i');
    if (value != null && value != i) {
      print('Item: item_$i Value: $value Expected: $i $info');
    }
  }

  // write
  for (int i = 0; i < 100; i++) {
    await box.put('item_$i', i);
  }

  // close
  await box.close();
  print('OK. $info');
}

void main() async {
  final path = Directory.current.path + '/db';

  Directory(path)
    ..deleteSync(recursive: true)
    ..createSync();

  Hive.init(path);

  var key = Hive.generateSecureKey();

  await test(1, null); // OK
  await test(1, null); // OK
  await test(1, key); // OK, Recovering corrupted box.
  await test(1, null); // Crashes!
}

[Maadhav]

I just started using hive for the first time and started getting this issue. Is there any solution to this. I just reinstalled my app and that fixed.
The error started to come after I started using deleteBoxFromDisk

[knaeckeKami]

I just wonder is there any other cases that might cause this issue rather than not using encryption key or using invalid key?

I did get this exception a few times from users in my crash reporting and I don't use encryption at all.
I could not reproduce this issue myself yet, though.

[nemoryoliver]

I'm getting the same error. I'm using the background_fetch plugin and I'm re-initializing my Hive boxes on it's BackgroundFetch.registerHeadlessTask() function. So basically, 2 processes (headless and background/foreground) uses 2 Hive instances that open boxes and CRUDing stuffs.

I/flutter (12177): Recovering corrupted box.
W/FlutterEngine(13591): Tried to automatically register plugins with FlutterEngine (io.flutter.embedding.engine.FlutterEngine@738c3e0) but could not find and invoke the GeneratedPluginRegistrant.
I/flutter (13591): Activity (Initial Load) completed in 5002ms
E/flutter (13592): [ERROR:flutter/lib/ui/ui_dart_state.cc(171)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
E/flutter (13592): #0      BinaryReaderImpl.readFrame (package:hive/src/binary/binary_reader_impl.dart:249)
E/flutter (13592): #1      FrameHelper.framesFromBytes (package:hive/src/binary/frame_helper.dart:17)
E/flutter (13592): #2      FrameIoHelper.framesFromFile (package:hive/src/io/frame_io_helper.dart:41)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #3      StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:82)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #4      HiveImpl._openBox (package:hive/src/hive_impl.dart:106)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #5      HiveImpl.openBox (package:hive/src/hive_impl.dart:135)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #6      HiveManager.init (package:app/core/managers/hive.manager.dart:142)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #7      init (package:app/core/main/main.dart:43)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592):

[hlongngo]

I have the same error without use encryption to open box.

[marcospsviana]

I have the same error, but only run the emulator in android 7. When running the emulator in android 9.0 he works very well. Is this anything with the android version?

[Gadda27]

I also have the same error, without using encrypted boxes. Happens when I open a box on start of the app.
We have huge issues because of that because our customers cannot just reinstall the app on the designated devices, or delete the content.
Is there any progress?

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[MilosKarakas]

Hi, I am facing the same issue. It works only for a few initial calls. I am using regular boxes without any encryption. I create a box called 'x' and put a few custom objects for which i have registered the adapters generated by Hive. After saving the data I navigate to another screen, then open the same box, retrieve the data, and it works. But then I navigate back to the initial screen and try to open the box again, and get this:
'Range Error: Not enough bytes available.'.
It happens every single time. I would have though that adapters are making issues, but they are automatically generated and also it does work at the first call.

Do you have any idea what might be happening or provide a possible workaround? I am using Hive in a production ready app, and we are adding offline caching for some features.

[leisim]

@MilosKarakas

It happens every single time. I would have though that adapters are making issues, but they are automatically generated and also it does work at the first call.

It would be super helpful if you could provide a sample which reproduces the issue.

[vixez]

I have/had the same issue.
I think it this is caused by either

• Not awaiting a box closing before opening a new one
• Opening the same box in rapid succession
  I forgot an await for one close, so I added it. And when hot reloading the app, my setup function would get called twice in a FutureBuilder causing a box to be open twice very quickly. So I've used AsyncMemoizer now to see if it helps by running setup only once.

I'll see if this fixes it.

[pakivika]

I'm getting the same error. I'm using the background_fetch plugin and I'm re-initializing my Hive boxes on it's BackgroundFetch.registerHeadlessTask() function. So basically, 2 processes (headless and background/foreground) uses 2 Hive instances that open boxes and CRUDing stuffs.

I'm getting similar exception when using workmanager plugin, which I think is similar to background_fetch, and uses Hive in a background task in another isolate while main app uses another Hive in another isolate. No encryption is used.

Error is unrecoverable and requires uninstallation of app i.e. removing all data.

I'm using:
hive: ^1.4.4+1
hydrated_bloc: ^6.0.3
workmanager: ^0.2.3

EGL_emulation( 1398): eglMakeCurrent: 0xa870b7a0: ver 3 0 (tinfo 0xa8783f00)
E/flutter ( 1398): [ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
E/flutter ( 1398): #0      BinaryReaderImpl.readFrame package:hive/…/binary/binary_reader_impl.dart:249
E/flutter ( 1398): #1      FrameHelper.framesFromBytes package:hive/…/binary/frame_helper.dart:17
E/flutter ( 1398): #2      FrameIoHelper.framesFromFile package:hive/…/io/frame_io_helper.dart:41
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #3      StorageBackendVm.initialize package:hive/…/vm/storage_backend_vm.dart:82
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #4      HiveImpl._openBox package:hive/src/hive_impl.dart:106
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #5      HiveImpl.openBox package:hive/src/hive_impl.dart:135
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #6      HydratedStorage.build.<anonymous closure> package:hydrated_bloc/src/hydrated_storage.dart:69
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #7      BasicLock.synchronized package:synchronized/src/basic_lock.dart:34
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398): #8      main
E/flutter ( 1398): <asynchronous suspension>
E/flutter ( 1398):

[royston-c]

I have just seen this issue on a users device (Galaxy S20+ 5G). I'm hoping the below might help narrow down the problem because I have a strict access policy to a Hive box. Annoyingly I put them in place hoping it would prevent this very issue.

1. The only way to open a box is via _checkAndOpenBox.
2. Any class extending BaseRepository is a singleton.
3. The app does NOT access any .hive file or box outside the main Flutter process (for example, an background service).
4. The app has never used a cipher
5. This code has been running in the app since its first release. (no box was created using any other code)
6. the box is never programmatically closed by the application.

The Lock used below is from the lib: https://pub.dev/packages/synchronized

class BaseRepository<TKey, TValue> {
  final String name;
  HiveBoxProvider _hiveBoxProvider;
  Box<TValue> _box;
  Lock _lock = new Lock();

  BaseRepository(this.name, {HiveBoxProvider hiveBoxProvider}) {
    _hiveBoxProvider = hiveBoxProvider ?? iocContainer.get<HiveBoxProvider>();
  }

  Future<TValue> get(TKey id) async {
    await _checkAndOpenBox();
    return _box.get(id);
  }
  //... other CRUD operations


  Future _checkAndOpenBox() async {
    if (_box == null || !_box.isOpen) {
      await _lock.synchronized(() async {
        if (_box == null || !_box.isOpen) {
          _box = await this._hiveBoxProvider.openBox<TValue>(this.name);
        }
      });
    }
  }
}

class HiveBoxProvider {
  static bool _initComplete = false;
  static Lock _lock = new Lock();

  Future<Box<TValue>> openBox<TValue>(String name) async {
    if (!_initComplete) {
      await _lock.synchronized(() async {
        if (!_initComplete) {
          await Hive.initFlutter();
          registerAdapters();
        }
        _initComplete = true;
      });
    }
    return await Hive.openBox<TValue>(name);
  }

  void registerAdapters() {
    //... additional Adapters are registered here.
  }
}

Logs from Crashlytics:

HiveError: This should not happen. Please open an issue on GitHub.
BinaryReaderImpl.readFrame (BinaryReaderImpl.java:249)
FrameHelper.framesFromBytes (FrameHelper.java:17)
FrameIoHelper.framesFromFile (FrameIoHelper.java:41)
StorageBackendVm.initialize (StorageBackendVm.java:82)
HiveImpl._openBox (HiveImpl.java:106)
HiveImpl.openBox (HiveImpl.java:135)

I don't fully understand the inner workings but these are just some thoughts to spark some ideas:

1. could the data store file(s) (.hive/.hivec) become corrupt due to backups android:allowBackup="true" while a box is open? I know the flutter_secure_storage package has had issues with this.
2. are writes always flushed to disk when the app is force closed?
3. it appears this error only happens when the app starts up? Can we confirm that no one experiences it while the box is opened and closed during one session? (It looks like @MilosKarakas error is unrelated)
4. can we add some logging to check the bytes on disk are the same as those in memory when closing the box or the app is force closed? (it's my understanding that everything is in memory while the box is open, if this is incorrect then this might not be possible)

[pakivika]

I'm getting the same error. I'm using the background_fetch plugin and I'm re-initializing my Hive boxes on it's BackgroundFetch.registerHeadlessTask() function. So basically, 2 processes (headless and background/foreground) uses 2 Hive instances that open boxes and CRUDing stuffs.

I'm getting similar exception when using workmanager plugin, which I think is similar to background_fetch, and uses Hive in a background task in another isolate while main app uses another Hive in another isolate. No encryption is used.

Error is unrecoverable and requires uninstallation of app i.e. removing all data.

I'm using:
hive: ^1.4.4+1
hydrated_bloc: ^6.0.3
workmanager: ^0.2.3

The issue could be related to multiple isolates accessing same boxes simultaneously. I made an experiment and disabled background job i.e. accessing the box from another isolate. I haven't seen the issue after that (knocking on wood).

[royston-c]

Hi, after some testing this morning I can consistently hit this error.

I am not 100% sure how this particular example could happen in a real-world app. My thoughts were that an unhandled error could cause the process to fail to await all the puts before the box is accessed again? My original example awaited some put's before throwing a custom Error. However, this is the simplest example I have that reproduces the issue.

Hope this helps! @leisim @TheMisir

import 'package:flutter/material.dart';
import 'package:hive/hive.dart';
import 'package:hive_flutter/hive_flutter.dart';
import 'package:uuid/uuid.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        primarySwatch: Colors.blue,
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String _output = "No Test run";
  String _boxId = Uuid().v4();

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(child: Text(_output)),
      floatingActionButton: FloatingActionButton.extended(
        onPressed: () async {
          for (var i = 0; i < 10; i++) {
            var isBroken = await _simpleWayToBreak();
            if (isBroken) {
              break;
            }
          }
        },
        icon: Icon(Icons.bug_report),
        label: Text('Break Me'),
      ),
    );
  }

  Future<bool> _simpleWayToBreak() async {
    await Hive.initFlutter();
    Box<String> box;

    try {
      box = await Hive.openBox<String>(_boxId);
      _setOutputState("Items in box - " + box.toMap().length.toString());
    } catch (e) {
      // print error and reset to new box
      _setOutputState(e.toString());
      _boxId = Uuid().v4();
      return true;
    }
    for (var i = 0; i < 10000; i++) {
      // normally we would await this
      box.put("id-" + i.toString(), Uuid().v4());
    }
    return false;
  }

  void _setOutputState(String output) {
    setState(() {
      _output = output;
    });
  }
}

name: hive_test
description: A new Flutter project.

version: 1.0.0+1

environment:
  sdk: ">=2.7.0 <3.0.0"

dependencies:
  hive: ^1.4.4
  hive_flutter: ^0.3.1
  uuid: ^2.2.2
  flutter:
    sdk: flutter
  cupertino_icons: ^1.0.0

dev_dependencies:
  flutter_test:
    sdk: flutter

flutter:
  uses-material-design: true

I/flutter (26100): Recovering corrupted box.
E/flutter (26100): [ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
E/flutter (26100): #0      BinaryReaderImpl.readFrame (package:hive/src/binary/binary_reader_impl.dart:249)
E/flutter (26100): #1      FrameHelper.framesFromBytes (package:hive/src/binary/frame_helper.dart:17)
E/flutter (26100): #2      FrameIoHelper.framesFromFile (package:hive/src/io/frame_io_helper.dart:41)
E/flutter (26100): <asynchronous suspension>
E/flutter (26100): #3      StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:82)      
E/flutter (26100): <asynchronous suspension>
E/flutter (26100): #4      HiveImpl._openBox (package:hive/src/hive_impl.dart:106)
E/flutter (26100): <asynchronous suspension>
E/flutter (26100): #5      HiveImpl.openBox (package:hive/src/hive_impl.dart:135)
E/flutter (26100): <asynchronous suspension>
E/flutter (26100): #6      _MyHomePageState._simpleWayToBreak (package:hive_test/main.dart:56)
E/flutter (26100): <asynchronous suspension>
E/flutter (26100): #7      _MyHomePageState.build.<anonymous closure> (package:hive_test/main.dart:39)
E/flutter (26100): <asynchronous suspension>
E/flutter (26100):

[Ahmadre]

I could reproduce this error with IntegrationTests in macOS: Just cancel the running Hive write operation and...boom! You app isn't working anymore

[Ahmadre]

@leisim Is there any workaround you'd suggest? I also deleted my XCode DerivedData and deleted PODs and reinstall my macOS app, but nothing works....

[Ahmadre]

Workaround: Rename your box, don't use special characters and you're safe!

[TheMisir]

@leisim Is there any workaround you'd suggest? I also deleted my XCode DerivedData and deleted PODs and reinstall my macOS app, but nothing works....

Currently we are investigating what exactly causes this issue. Probably caused by misconfiguration (invalid type adapters) or invalid usage (not awaiting write calls).

[leisim]

@royston-c Thanks for your work towards solving this issue. Unfortunately after investigating the root of the issue, I have bad news: The reason for the exception in your code is that Hive.initFlutter() is called multiple times which is not allowed.
I will add a better exception but I don't think this really solves the issue of this thread. Still hoping for someone to provide a sample how to reproduce it.

[royston-c]

@leisim You are correct!

In order to try and help out, I've tried to replicate the same access pattern using Isolates and the same error is thrown (in debug).

I'm pretty sure this isn't the same issue as Hive.initFlutter() but instead with Hive.init(config.path). If it is, this could potentially be the issue? Initialising Hive multiple times from different isolates. It's required by the library otherwise you get a HiveError: You need to initialize Hive or provide a path to store the box. error.

import 'dart:isolate';

import 'package:flutter/material.dart';
import 'package:hive/hive.dart';
import 'package:hive_flutter/hive_flutter.dart';
import 'package:path_provider/path_provider.dart';
import 'package:uuid/uuid.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        primarySwatch: Colors.blue,
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String _output = "No Test run";
  String _boxId;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(child: Text(_output)),
      floatingActionButton: FloatingActionButton.extended(
        onPressed: () async {
          _boxId = Uuid().v4();
          var dir = await getApplicationDocumentsDirectory();
          for (var i = 0; i < 10; i++) {
            var config = BreakHiveConfig(_boxId, dir.path, true);
            await Isolate.spawn(_breakHive, config);
          }
        },
        icon: Icon(Icons.bug_report),
        label: Text('Break Me'),
      ),
    );
  }
}

Future<void> _breakHive(BreakHiveConfig config) async {
  if (config.shouldInit) {
    Hive.init(config.path);
  }
  var box = await Hive.openBox<String>(config.boxId);

  for (var i = 0; i < 10000; i++) {
    await box.put("id-" + i.toString(), Uuid().v4());
  }

  return false;
}

class BreakHiveConfig {
  final String boxId;
  final String path;
  final bool shouldInit;

  BreakHiveConfig(this.boxId, this.path, this.shouldInit);
}

name: hive_test
description: A new Flutter project.

version: 1.0.0+1

environment:
  sdk: ">=2.7.0 <3.0.0"

dependencies:
  hive: ^1.4.4
  hive_flutter: ^0.3.1
  uuid: ^2.2.2
  path_provider: ^1.6.24
  flutter:
    sdk: flutter
  cupertino_icons: ^1.0.0

dev_dependencies:
  flutter_test:
    sdk: flutter

flutter:
  uses-material-design: true

[firatcetiner]

I'm getting the same error. I'm using the background_fetch plugin and I'm re-initializing my Hive boxes on it's BackgroundFetch.registerHeadlessTask() function. So basically, 2 processes (headless and background/foreground) uses 2 Hive instances that open boxes and CRUDing stuffs.

I/flutter (12177): Recovering corrupted box.
W/FlutterEngine(13591): Tried to automatically register plugins with FlutterEngine (io.flutter.embedding.engine.FlutterEngine@738c3e0) but could not find and invoke the GeneratedPluginRegistrant.
I/flutter (13591): Activity (Initial Load) completed in 5002ms
E/flutter (13592): [ERROR:flutter/lib/ui/ui_dart_state.cc(171)] Unhandled Exception: HiveError: This should not happen. Please open an issue on GitHub.
E/flutter (13592): #0      BinaryReaderImpl.readFrame (package:hive/src/binary/binary_reader_impl.dart:249)
E/flutter (13592): #1      FrameHelper.framesFromBytes (package:hive/src/binary/frame_helper.dart:17)
E/flutter (13592): #2      FrameIoHelper.framesFromFile (package:hive/src/io/frame_io_helper.dart:41)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #3      StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:82)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #4      HiveImpl._openBox (package:hive/src/hive_impl.dart:106)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #5      HiveImpl.openBox (package:hive/src/hive_impl.dart:135)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #6      HiveManager.init (package:app/core/managers/hive.manager.dart:142)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592): #7      init (package:app/core/main/main.dart:43)
E/flutter (13592): <asynchronous suspension>
E/flutter (13592):

Does disabling headless tasks solve the issue? Have you tried that?

[Dimoshka]

The same error - on Web, Android, and Windows
After hot reload (application rebuild)

Latest dev flutter and latest package.

At windows also:

message:"lock failed"
osError:OSError (OS Error: Процесс не может получить доступ к файлу, так как часть этого файла заблокирована другим процессом.
, errno = 33)
path:"C:\Users\dmytr\Documents\vom_settings.lock"

Future<Box<Settings>> _getSettingsBox() async {
    if (!Hive.isAdapterRegistered(userRoleTypeId))
      Hive.registerAdapter(UserRoleAdapter());
    if (!Hive.isAdapterRegistered(sessionTypeId))
      Hive.registerAdapter(SessionAdapter());
    if (!Hive.isAdapterRegistered(settingsTypeId))
      Hive.registerAdapter(SettingsAdapter());
    if (Hive.isBoxOpen(Settings.dbName)) {
      return Hive.box<Settings>(Settings.dbName);
    } else {
      await Hive.close();
      return Hive.openBox<Settings>(Settings.dbName)
          .catchError((onError) => print(onError));
    }
  }

  Future<Settings> getSettings() async {
    final settingsBox =
        await _getSettingsBox().catchError((onError) => print(onError));
    final settings = settingsBox?.get('default');
    if (settings == null && settingsBox != null) {
      await settingsBox.put('default', Settings());
    }
    return settings ?? Settings();
  }

[jorgegaticav]

I had the same problem when calling Hive in the background using the AndroidAlarmManager lib, every call requires to initialize Hive before doing transactions so it seems that because of that the calls overlap in certain point and the data gets corrupted. I had to remove the background features in my app to avoid the problem.
If the Hive team could add a built-in feature for running hive safe calls in the background, maybe for example keeping Hive constantly initialized, would be great.

[TheMisir]

I had the same problem when calling Hive in the background using the AndroidAlarmManager lib, every call requires to initialize Hive before doing transactions so it seems that because of that the calls overlap in certain point and the data gets corrupted. I had to remove the background features in my app to avoid the problem.
If the Hive team could add a built-in feature for running hive safe calls in the background, maybe for example keeping Hive constantly initialized, would be great.

Messaging between isolates in dart performs very poorly. Due to that and other issues @leisim (creator of hive) works on a new database called isar which uses native core written in rust.

[britannio]

Non-fatal Exception: io.flutter.plugins.firebase.crashlytics.FlutterError: HiveError: This should not happen. Please open an issue on GitHub.
       at BinaryReaderImpl.readFrame(BinaryReaderImpl.java:249)
       at FrameHelper.framesFromBytes(FrameHelper.java:17)
       at FrameIoHelper.framesFromFile(FrameIoHelper.java:41)
       at StorageBackendVm.initialize(StorageBackendVm.java:82)
       at HiveImpl._openBox(HiveImpl.java:106)
       at HiveImpl.openBox(HiveImpl.java:135)

According to the stack trace, the if (frameLength < 8) { condition isn't being met
https://github.com/hivedb/hive/blob/1c62ad55e71463c29283f797132c6cb5ce7061a4/hive/lib/src/binary/binary_reader_impl.dart#L244-L251

I haven't made any changes recently and I'm not using encrypted boxes, this just happened out of the blue in my production app and Firebase Crashlytics picked it up. I don't have a way to reproduce it although my assumption is that something corrupted one of my boxes.

Since the following code throws, the recoveryOffset is never set so it seems like the recovery related code is never executed.
https://github.com/hivedb/hive/blob/1c62ad55e71463c29283f797132c6cb5ce7061a4/hive/lib/src/backend/vm/storage_backend_vm.dart#L81-L82

[tobiaswittekindt]

For everyone who wants to fix the issue "on the fly" without deleting the app. This code will remove the corrupted hive db file from the disc when an exception is thrown during Hive.openBox:
```
Future initDB() async {
try {
await this.registerAdapters();
return await Hive.openBox(this._dbName);
} catch (_) {
print('Attention corrupted DB detected. Try to delete the DB file');
if(!tryFixDB) {
this.tryFixDB = true;
await deleteDatabaseFiles();
return await initDB();
}
}

}

Future deleteDatabaseFiles() async {
var dir = await getApplicationDocumentsDirectory();
String dirPath = dir.path;

String boxName = this._dbName.toLowerCase();

File dbFile = File('$dirPath/$boxName.hive');
File lockFile = File('$dirPath/$boxName.lock');

await dbFile.delete();
await lockFile.delete();

}`

[britannio]

I'd actually recommend Hive.deleteBoxFromDisk() which uses the following code if your box is corrupted:
https://github.com/hivedb/hive/blob/1c62ad55e71463c29283f797132c6cb5ce7061a4/hive/lib/src/backend/vm/backend_manager.dart#L52-L57

[kaptnkoala]

We are also trying to get rid of the problem for some time already and were not able to fully tackle the problem yet. But maybe we can provide some useful insights that will help to narrow down the problem.

First this error occurred much more often, when we were doing parallel write operations on the boxes in Future.wait calls. We drastically reduced the amount of this error by wrapping the Boxes in a BoxAdapter that queues all modifying operations on the boxes and executes them in sequence. The approach is heavily inspired by the FileSync class of @leisim. In this way we do not have to mind when writing the code, that any modifying Hive operations might actually run in parallel. Here are the important parts of the AsyncQueue, BoxAdapter and HiveBucket classes:

class AsyncQueue {
  Future _task = Future.value();

  Future<T> add<T>(Future<T> Function() task) {
    final completer = async.Completer();
    final previousTask = _task;
    _task = completer.future;

    return previousTask //
        .then((_) => task())
        .whenComplete(completer.complete);
  }
}

class BoxAdapter<B extends BoxBase> {
  final AsyncQueue _tasks = AsyncQueue();
  final Future<B> Function() _open;
  Future<B> _box;

  BoxAdapter(this._open);

  Future<B> get _unboxed => _box ??= _open();

  Future<T> run<T>(FutureOr<T> Function(B) task) {
    return _unboxed.then((box) => task(box)), onError;
  }

  Future<T> queue<T>(Future<T> Function(B) task) {
    return _tasks.add(() => _unboxed.then((box) => task(box)));
  }
}

class HiveBucket extend Bucket {
  final String _name;
  BoxAdapter<BoxBase> get _adapter;

  HiveBucket(this._name): _adapter = BoxAdapter(() => Hive.openLazyBox(_name));

  Future<T> get<T>(String key) {
    return _adapter.run<T>((box) => box.get(key).then((value) => value as T));
  }

  Future<void> set<T>(String key, T value) {
    return _adapter.queue((box) => box.put(key, value));
  }

  Future<void> delete(String key) {
    return _adapter.queue((box) => box.delete(key));
  }
}

With this wrapping classes around Hive boxes we were able to fully eliminate the problem on iOS, but this problem still occurs on Android devices. At the moment we have about 80 occurrences (affects 60 users) per day on a total user base of 1.6M DAU. So it only affects a small portion of our user base, but for those the error can be quite frustrating because they sometimes loose their complete progress. When a box is broken, the only thing we can do is deleting and reopening the box as other users already suggested.

At the moment we think that maybe terminating the app while running write operations on Hive boxes might be the remaining cause for the issue. But we have not figured out a good way to verify this yet.

[chengboonrong]

1614733237005|E|X-LOG|2021-03-03 09:00:37.001170 -- update ticket exception : HiveError: This should not happen. Please open an issue on GitHub.

I run into the same error, also with hive: ^1.4.1+1
Many of my customers mentioned this issue to me, and it seems like a random issue.
Any idea of how this still happened and what are the solutions?
Thanks.

Future<bool> updateHiveLocalTicket(
      String tableName, int key, List<String> value) async {
 try {
      final box = await getHiveBox(tableName);
      if (key != null) {
        await box.put(key, value);
        return true;
      } else {
        return false;
      }
    } catch (e) {
      print( '${DateTime.now()} -- update ticket exception : ${e.toString()}');
      return false;
    }
}`

eg for tableName : TICKET_LIST_8

[royston-c]

This example breaks Hive consistently, using isolates (via compute). Tested on a Pixel 2 in debug.

This example highlights the possible access patterns of apps that run in multiple processes, such as in the background. Although 20 Isolates accessing the same box is unrealistic it does throw the unexpected issue, so might help with the investigation.

I also tried to write an integration test similar to put_many_strings_test.dart but I couldn't replicate the issue. So either 1) Flutter has an effect 2) the device it runs on has an effect 3) the implementation of openBox() in integration.dart is different (although this doesn't appear to be the case from what I can see).

@leisim @TheMisir would one of the Hive team be able to investigate this? I understand that you are hard at work on Isar but I hope you can spare an hour or 2 to investigate? I would investigate further myself but the passing integration test has left me a little stumped on where to go next.

import 'dart:io';

import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:hive/hive.dart';
import 'package:hive_flutter/hive_flutter.dart';
import 'package:path_provider/path_provider.dart';
import 'package:uuid/uuid.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        primarySwatch: Colors.blue,
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String _outputField = "No Test run";
  set _output(String value) => setState(() => _outputField = value);
  get _output => _outputField;

  bool _runningField = false;
  set _running(bool value) => setState(() => _runningField = value);
  get _running => _runningField;

  String _boxId;
  List<String> _errors = List<String>();

  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addPostFrameCallback((_) {
      Hive.initFlutter();
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: SingleChildScrollView(child: Center(child: Text(_output))),
      floatingActionButton: FloatingActionButton.extended(
        onPressed: _runTest,
        icon: _running
            ? CircularProgressIndicator(backgroundColor: Colors.white70)
            : Icon(Icons.bug_report),
        label: Text('Break Me'),
      ),
    );
  }

  Future<Directory> _startTest() async {
    _running = true;
    _errors.clear();
    _boxId = Uuid().v4().toString();
    _output = "Test Started";
    return await getApplicationDocumentsDirectory();
  }

  Future<void> _runTest() async {
    var dir = await _startTest();
    var futures = List<Future>();
    var isBroken = false;
    while (!isBroken) {
      for (var i = 0; i < 20; i++) {
        futures.add(
          compute(_breakHive, BreakHiveConfig(_boxId, dir.path)).then(
            (_) {},
            onError: (e) {
              var error = e.toString();
              _errors.add(error);
              _output = _errors.join("\n\n");
              if (error.contains("This should not happen")) {
                isBroken = true;
              }
            },
          ),
        );
      }
      await Future.wait(futures);
    }
    _endTest();
  }

  void _endTest() async {
    _running = false;
    if (_errors.isEmpty) {
      _output = "Test Ended";
    }
  }
}

Future<void> _breakHive(BreakHiveConfig config) async {
  Hive.init(config.path);

  var box = await Hive.openBox<String>(config.boxId);

  for (var i = 0; i < 100; i++) {
    await box.put("id-" + i.toString(), Uuid().v4());
  }
}

class BreakHiveResponse {
  final bool isBroken;
  final String error;

  BreakHiveResponse(this.isBroken, this.error);
}

class BreakHiveConfig {
  final String boxId;
  final String path;

  BreakHiveConfig(this.boxId, this.path);
}

name: hive_test
description: A new Flutter project.

version: 1.0.0+1

environment:
  sdk: ">=2.7.0 <3.0.0"

dependencies:
  hive: ^1.4.4+1
  hive_flutter: ^0.3.1
  uuid: ^2.2.2
  path_provider: ^1.6.24
  flutter:
    sdk: flutter
  cupertino_icons: ^1.0.0

dev_dependencies:
  flutter_test:
    sdk: flutter

flutter:
  uses-material-design: true

[leonardomteixeira]

Thanks @Ahmadre, works fine for me!!!!!

[nemoryoliver]

Workaround: Rename your box, don't use special characters and you're safe!

I have 5 boxes and never had a special character. Still get the issue for me

[anandsubbu007]

Still facing this issue.
not using any special character.