Electrs: Would it be possibe to swap rocksdb for somethig else?

Created on 28 Nov 2019  Â·  3Comments  Â·  Source: romanz/electrs

I found that it takes enormous time to build and strangely, it's now taking me several hours.

It'd be nice if it could be swapped for something else somehow. Even if its slightly slower.

question

Most helpful comment

RocksDB build time is indeed an issue - maybe we can try LMDB?
It builds significantly faster :)

$ time cargo build --release
   Compiling cc v1.0.47
   Compiling pkg-config v0.3.17
   Compiling libc v0.2.66
   Compiling bitflags v1.2.1
   Compiling lmdb-sys v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys)
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_env_get_maxkeysize’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:10033:33: warning: unused parameter ‘env’ [-Wunused-parameter]
warning:  mdb_env_get_maxkeysize(MDB_env *env)
warning:                         ~~~~~~~~~^~~
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_cursor_put’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6737:9: warning: this statement may fall through [-Wimplicit-fallthrough=]
warning:       if (SIZELEFT(fp) < offset) {
warning:          ^
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6742:5: note: here
warning:      case MDB_CURRENT:
warning:      ^~~~
   Compiling lmdb v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs)
    Finished release [optimized] target(s) in 4.94s

real    0m4.952s
user    0m12.722s
sys 0m0.429s

All 3 comments

RocksDB build time is indeed an issue - maybe we can try LMDB?
It builds significantly faster :)

$ time cargo build --release
   Compiling cc v1.0.47
   Compiling pkg-config v0.3.17
   Compiling libc v0.2.66
   Compiling bitflags v1.2.1
   Compiling lmdb-sys v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys)
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_env_get_maxkeysize’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:10033:33: warning: unused parameter ‘env’ [-Wunused-parameter]
warning:  mdb_env_get_maxkeysize(MDB_env *env)
warning:                         ~~~~~~~~~^~~
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_cursor_put’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6737:9: warning: this statement may fall through [-Wimplicit-fallthrough=]
warning:       if (SIZELEFT(fp) < offset) {
warning:          ^
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6742:5: note: here
warning:      case MDB_CURRENT:
warning:      ^~~~
   Compiling lmdb v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs)
    Finished release [optimized] target(s) in 4.94s

real    0m4.952s
user    0m12.722s
sys 0m0.429s

OTOH, there is https://github.com/spacejam/sled which is pure Rust :)

LMDB sounds good. Sled is beta, so I think we should wait with that.

Hopefully I will find some time to contribute a PR until the end of January.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

burguesia picture burguesia  Â·  4Comments

reynico picture reynico  Â·  5Comments

romanz picture romanz  Â·  5Comments

Overtorment picture Overtorment  Â·  3Comments

Emanresueht picture Emanresueht  Â·  8Comments