Electrs: Indexing fails with OversizedVectorAllocation

[andrewtoth]

Here's the full stacktrace:

DEBUG - writing 1257558 rows to RocksDB { path: "./db/mainnet/newindex/txstore" }, flush=Disable
TRACE - parsing 133409003 bytes
TRACE - fetched 136 blocks
TRACE - reading "/home/user/.bitcoin/blocks/blk00978.dat"
thread 'parse-blocks-9' panicked at 'failed to parse Block: OversizedVectorAllocation { requested: 90466760178240, max: 33554432 }', src/libcore/result.rs:997:5
stack backtrace:
   0:     0x5577243017a3 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::h50ebfb8734a81144
                               at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:     0x5577242ffc0b - std::panicking::default_hook::{{closure}}::hc55d0892611a29ff
                               at src/libstd/sys_common/backtrace.rs:71
                               at src/libstd/sys_common/backtrace.rs:59
                               at src/libstd/panicking.rs:197
   2:     0x5577242fec30 - std::panicking::rust_panic_with_hook::h24c9a1c35b1f49cc
                               at src/libstd/panicking.rs:211
                               at src/libstd/panicking.rs:474
   3:     0x5577242fe81d - std::panicking::continue_panic_fmt::h8ed9632bdd4b9299
                               at src/libstd/panicking.rs:381
   4:     0x55772430f145 - rust_begin_unwind
                               at src/libstd/panicking.rs:308
   5:     0x557724313d7b - core::panicking::panic_fmt::h0d6d5c8b201e3246
                               at src/libcore/panicking.rs:85
   6:     0x55772404d026 - core::result::unwrap_failed::h3ebf7163694dfeff
   7:     0x55772414de2f - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
   8:     0x55772414e0e6 - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
   9:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  10:     0x55772414df6c - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  11:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  12:     0x55772414e5bb - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hd88c5e5586054d01
  13:     0x55772407cce0 - rayon_core::registry::WorkerThread::wait_until_cold::hc51d52410ab9f6a5
  14:     0x55772414e01f - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  15:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  16:     0x55772414e0e6 - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  17:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  18:     0x55772414df6c - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  19:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  20:     0x55772414e5bb - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hd88c5e5586054d01
  21:     0x55772407cce0 - rayon_core::registry::WorkerThread::wait_until_cold::hc51d52410ab9f6a5
  22:     0x55772414e01f - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  23:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  24:     0x55772414e0e6 - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  25:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  26:     0x55772414df6c - rayon_core::join::join_context::{{closure}}::h7c2357ec41061411
  27:     0x55772414dc2c - rayon::iter::plumbing::bridge_producer_consumer::helper::h29134a41eb48cd14
  28:     0x55772414e5bb - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hd88c5e5586054d01
  29:     0x5577241d6395 - rayon_core::registry::WorkerThread::wait_until_cold::h5e2de67a7cd9b2c3
  30:     0x5577241d6007 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4dafa15514cc26c0
  31:     0x5577241d5b00 - std::thread::Builder::spawn_unchecked::{{closure}}::he0b91ba59dc3f7b1
  32:     0x55772430ea7d - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h4cac16ae2114a837
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702
  33:     0x557724310377 - std::sys::unix::thread::Thread::new::thread_start::h2adc1b80820f790e
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702
                               at src/libstd/sys_common/thread.rs:14
                               at src/libstd/sys/unix/thread.rs:80
  34:     0x7f7d7a9f16da - start_thread
  35:     0x7f7d7a50288e - __clone
  36:                0x0 - <unknown>
Aborted (core dumped)

[romanz]

Thanks for reporting this issue!
It's possible that the blk*.dat parser logic fails to handle /home/user/.bitcoin/blocks/blk00978.dat.
Could you please upload this file, so I could try to debug its parsing on my machine?

[andrewtoth]

@romanz It's not easy finding a place to upload it. I've uploaded it here, but it will only be active for 1 download or 1 day so please download ASAP. https://send.firefox.com/download/dd49a2ed66767dc9/#Ah8BELEqYlEoKtzouqeA-A

[romanz]

Got it, thanks!

$ sha256sum blk00978.dat
a84c5cc61348b7517247c98b2405acd00ff2d5a9f902b9910e0d2f9e1b707e10  blk00978.dat

[romanz]

Tested on https://github.com/romanz/electrs/blob/tmp/parse_blk/examples/parse_blk.rs (at https://github.com/romanz/electrs/commit/ac6da9481c74da78d5bdef11267f96969f56b760) - seems to work correctly:

$ cargo run --release --example parse_blk -- _bugs/blk00978.dat
     Running `target/release/examples/parse_blk _bugs/blk00978.dat`
[examples/parse_blk.rs:11] &f = "_bugs/blk00978.dat"
2019-07-05T16:20:08.857+03:00 - TRACE - parsing block [8,999208) = 999200 bytes
2019-07-05T16:20:08.863+03:00 - TRACE - parsing block [999216,1997758) = 998542 bytes
2019-07-05T16:20:08.869+03:00 - TRACE - parsing block [1997766,2997050) = 999284 bytes
2019-07-05T16:20:08.875+03:00 - TRACE - parsing block [2997058,3995394) = 998336 bytes
2019-07-05T16:20:08.882+03:00 - TRACE - parsing block [3995402,4994670) = 999268 bytes
2019-07-05T16:20:08.888+03:00 - TRACE - parsing block [4994678,5993968) = 999290 bytes
2019-07-05T16:20:08.894+03:00 - TRACE - parsing block [5993976,6992438) = 998462 bytes
2019-07-05T16:20:08.901+03:00 - TRACE - parsing block [6992446,7991706) = 999260 bytes
2019-07-05T16:20:08.907+03:00 - TRACE - parsing block [7991714,8990089) = 998375 bytes
2019-07-05T16:20:08.913+03:00 - TRACE - parsing block [8990097,9988596) = 998499 bytes
2019-07-05T16:20:08.919+03:00 - TRACE - parsing block [9988604,10987906) = 999302 bytes
2019-07-05T16:20:08.926+03:00 - TRACE - parsing block [10987914,11987163) = 999249 bytes
2019-07-05T16:20:08.932+03:00 - TRACE - parsing block [11987171,12986451) = 999280 bytes
2019-07-05T16:20:08.938+03:00 - TRACE - parsing block [12986459,13985666) = 999207 bytes
2019-07-05T16:20:08.945+03:00 - TRACE - parsing block [13985674,14984825) = 999151 bytes
2019-07-05T16:20:08.951+03:00 - TRACE - parsing block [14984833,15983406) = 998573 bytes
2019-07-05T16:20:08.957+03:00 - TRACE - parsing block [15983414,16982608) = 999194 bytes
2019-07-05T16:20:08.964+03:00 - TRACE - parsing block [16982616,17981208) = 998592 bytes
2019-07-05T16:20:08.970+03:00 - TRACE - parsing block [17981216,18979499) = 998283 bytes
2019-07-05T16:20:08.976+03:00 - TRACE - parsing block [18979507,19978669) = 999162 bytes
2019-07-05T16:20:08.982+03:00 - TRACE - parsing block [19978677,20976882) = 998205 bytes
2019-07-05T16:20:08.991+03:00 - TRACE - parsing block [20976890,21976203) = 999313 bytes
2019-07-05T16:20:08.997+03:00 - TRACE - parsing block [21976211,22974748) = 998537 bytes
2019-07-05T16:20:09.003+03:00 - TRACE - parsing block [22974756,23973302) = 998546 bytes
2019-07-05T16:20:09.010+03:00 - TRACE - parsing block [23973310,24962911) = 989601 bytes
2019-07-05T16:20:09.016+03:00 - TRACE - parsing block [24962919,25952116) = 989197 bytes
2019-07-05T16:20:09.023+03:00 - TRACE - parsing block [25952124,26951145) = 999021 bytes
2019-07-05T16:20:09.030+03:00 - TRACE - parsing block [26951153,27950500) = 999347 bytes
2019-07-05T16:20:09.037+03:00 - TRACE - parsing block [27950508,28949812) = 999304 bytes
2019-07-05T16:20:09.043+03:00 - TRACE - parsing block [28949820,29948155) = 998335 bytes
2019-07-05T16:20:09.050+03:00 - TRACE - parsing block [29948163,30946644) = 998481 bytes
2019-07-05T16:20:09.056+03:00 - TRACE - parsing block [30946652,31945091) = 998439 bytes
2019-07-05T16:20:09.062+03:00 - TRACE - parsing block [31945099,32943668) = 998569 bytes
2019-07-05T16:20:09.069+03:00 - TRACE - parsing block [32943676,33942922) = 999246 bytes
2019-07-05T16:20:09.075+03:00 - TRACE - parsing block [33942930,34942216) = 999286 bytes
2019-07-05T16:20:09.081+03:00 - TRACE - parsing block [34942224,35941469) = 999245 bytes
2019-07-05T16:20:09.088+03:00 - TRACE - parsing block [35941477,36940778) = 999301 bytes
2019-07-05T16:20:09.094+03:00 - TRACE - parsing block [36940786,37940108) = 999322 bytes
2019-07-05T16:20:09.101+03:00 - TRACE - parsing block [37940116,38939410) = 999294 bytes
2019-07-05T16:20:09.107+03:00 - TRACE - parsing block [38939418,39938566) = 999148 bytes
2019-07-05T16:20:09.113+03:00 - TRACE - parsing block [39938574,40937175) = 998601 bytes
2019-07-05T16:20:09.120+03:00 - TRACE - parsing block [40937183,41936456) = 999273 bytes
2019-07-05T16:20:09.126+03:00 - TRACE - parsing block [41936464,42935696) = 999232 bytes
2019-07-05T16:20:09.133+03:00 - TRACE - parsing block [42935704,43934945) = 999241 bytes
2019-07-05T16:20:09.139+03:00 - TRACE - parsing block [43934953,44934289) = 999336 bytes
2019-07-05T16:20:09.145+03:00 - TRACE - parsing block [44934297,45932806) = 998509 bytes
2019-07-05T16:20:09.152+03:00 - TRACE - parsing block [45932814,46932113) = 999299 bytes
2019-07-05T16:20:09.158+03:00 - TRACE - parsing block [46932121,47931352) = 999231 bytes
2019-07-05T16:20:09.164+03:00 - TRACE - parsing block [47931360,48930601) = 999241 bytes
2019-07-05T16:20:09.170+03:00 - TRACE - parsing block [48930609,49929806) = 999197 bytes
2019-07-05T16:20:09.177+03:00 - TRACE - parsing block [49929814,50928309) = 998495 bytes
2019-07-05T16:20:09.183+03:00 - TRACE - parsing block [50928317,51927557) = 999240 bytes
2019-07-05T16:20:09.190+03:00 - TRACE - parsing block [51927565,52926622) = 999057 bytes
2019-07-05T16:20:09.196+03:00 - TRACE - parsing block [52926630,53925135) = 998505 bytes
2019-07-05T16:20:09.202+03:00 - TRACE - parsing block [53925143,54924395) = 999252 bytes
2019-07-05T16:20:09.208+03:00 - TRACE - parsing block [54924403,55922930) = 998527 bytes
2019-07-05T16:20:09.215+03:00 - TRACE - parsing block [55922938,56922278) = 999340 bytes
2019-07-05T16:20:09.221+03:00 - TRACE - parsing block [56922286,57921095) = 998809 bytes
2019-07-05T16:20:09.227+03:00 - TRACE - parsing block [57921103,58920367) = 999264 bytes
2019-07-05T16:20:09.234+03:00 - TRACE - parsing block [58920375,59918625) = 998250 bytes
2019-07-05T16:20:09.240+03:00 - TRACE - parsing block [59918633,60917785) = 999152 bytes
2019-07-05T16:20:09.246+03:00 - TRACE - parsing block [60917793,61916972) = 999179 bytes
2019-07-05T16:20:09.252+03:00 - TRACE - parsing block [61916980,62916208) = 999228 bytes
2019-07-05T16:20:09.259+03:00 - TRACE - parsing block [62916216,63915435) = 999219 bytes
2019-07-05T16:20:09.265+03:00 - TRACE - parsing block [63915443,64914588) = 999145 bytes
2019-07-05T16:20:09.272+03:00 - TRACE - parsing block [64914596,65913063) = 998467 bytes
2019-07-05T16:20:09.278+03:00 - TRACE - parsing block [65913071,66902290) = 989219 bytes
2019-07-05T16:20:09.285+03:00 - TRACE - parsing block [66902298,67901031) = 998733 bytes
2019-07-05T16:20:09.292+03:00 - TRACE - parsing block [67901039,68899806) = 998767 bytes
2019-07-05T16:20:09.298+03:00 - TRACE - parsing block [68899814,69889005) = 989191 bytes
2019-07-05T16:20:09.305+03:00 - TRACE - parsing block [69889013,70887483) = 998470 bytes
2019-07-05T16:20:09.311+03:00 - TRACE - parsing block [70887491,71886794) = 999303 bytes
2019-07-05T16:20:09.318+03:00 - TRACE - parsing block [71886802,72885994) = 999192 bytes
2019-07-05T16:20:09.324+03:00 - TRACE - parsing block [72886002,73884590) = 998588 bytes
2019-07-05T16:20:09.331+03:00 - TRACE - parsing block [73884598,74883045) = 998447 bytes
2019-07-05T16:20:09.337+03:00 - TRACE - parsing block [74883053,75882307) = 999254 bytes
2019-07-05T16:20:09.344+03:00 - TRACE - parsing block [75882315,76881621) = 999306 bytes
2019-07-05T16:20:09.350+03:00 - TRACE - parsing block [76881629,77880861) = 999232 bytes
2019-07-05T16:20:09.356+03:00 - TRACE - parsing block [77880869,78879842) = 998973 bytes
2019-07-05T16:20:09.363+03:00 - TRACE - parsing block [78879850,79879100) = 999250 bytes
2019-07-05T16:20:09.369+03:00 - TRACE - parsing block [79879108,80878200) = 999092 bytes
2019-07-05T16:20:09.375+03:00 - TRACE - parsing block [80878208,81878176) = 999968 bytes
2019-07-05T16:20:09.382+03:00 - TRACE - parsing block [81878184,82867694) = 989510 bytes
2019-07-05T16:20:09.388+03:00 - TRACE - parsing block [82867702,83865886) = 998184 bytes
2019-07-05T16:20:09.394+03:00 - TRACE - parsing block [83865894,84864767) = 998873 bytes
2019-07-05T16:20:09.400+03:00 - TRACE - parsing block [84864775,85863346) = 998571 bytes
2019-07-05T16:20:09.407+03:00 - TRACE - parsing block [85863354,86862658) = 999304 bytes
2019-07-05T16:20:09.413+03:00 - TRACE - parsing block [86862666,87861975) = 999309 bytes
2019-07-05T16:20:09.419+03:00 - TRACE - parsing block [87861983,88860212) = 998229 bytes
2019-07-05T16:20:09.426+03:00 - TRACE - parsing block [88860220,89860082) = 999862 bytes
2019-07-05T16:20:09.432+03:00 - TRACE - parsing block [89860090,90858855) = 998765 bytes
2019-07-05T16:20:09.438+03:00 - TRACE - parsing block [90858863,91857994) = 999131 bytes
2019-07-05T16:20:09.444+03:00 - TRACE - parsing block [91858002,92857168) = 999166 bytes
2019-07-05T16:20:09.451+03:00 - TRACE - parsing block [92857176,93856028) = 998852 bytes
2019-07-05T16:20:09.457+03:00 - TRACE - parsing block [93856036,94854574) = 998538 bytes
2019-07-05T16:20:09.463+03:00 - TRACE - parsing block [94854582,95852874) = 998292 bytes
2019-07-05T16:20:09.470+03:00 - TRACE - parsing block [95852882,96852197) = 999315 bytes
2019-07-05T16:20:09.476+03:00 - TRACE - parsing block [96852205,97850741) = 998536 bytes
2019-07-05T16:20:09.482+03:00 - TRACE - parsing block [97850749,98850094) = 999345 bytes
2019-07-05T16:20:09.489+03:00 - TRACE - parsing block [98850102,99849300) = 999198 bytes
2019-07-05T16:20:09.496+03:00 - TRACE - parsing block [99849308,100848625) = 999317 bytes
2019-07-05T16:20:09.503+03:00 - TRACE - parsing block [100848633,101847961) = 999328 bytes
2019-07-05T16:20:09.509+03:00 - TRACE - parsing block [101847969,102837139) = 989170 bytes
2019-07-05T16:20:09.516+03:00 - TRACE - parsing block [102837147,103837176) = 1000029 bytes
2019-07-05T16:20:09.524+03:00 - TRACE - parsing block [103837184,104836478) = 999294 bytes
2019-07-05T16:20:09.530+03:00 - TRACE - parsing block [104836486,105835221) = 998735 bytes
2019-07-05T16:20:09.537+03:00 - TRACE - parsing block [105835229,106833676) = 998447 bytes
2019-07-05T16:20:09.544+03:00 - TRACE - parsing block [106833684,107832604) = 998920 bytes
2019-07-05T16:20:09.551+03:00 - TRACE - parsing block [107832612,108831798) = 999186 bytes
2019-07-05T16:20:09.557+03:00 - TRACE - parsing block [108831806,109830291) = 998485 bytes
2019-07-05T16:20:09.563+03:00 - TRACE - parsing block [109830299,110829584) = 999285 bytes
2019-07-05T16:20:09.570+03:00 - TRACE - parsing block [110829592,111818886) = 989294 bytes
2019-07-05T16:20:09.576+03:00 - TRACE - parsing block [111818894,112817493) = 998599 bytes
2019-07-05T16:20:09.582+03:00 - TRACE - parsing block [112817501,113816830) = 999329 bytes
2019-07-05T16:20:09.589+03:00 - TRACE - parsing block [113816838,114816008) = 999170 bytes
2019-07-05T16:20:09.595+03:00 - TRACE - parsing block [114816016,115814562) = 998546 bytes
2019-07-05T16:20:09.601+03:00 - TRACE - parsing block [115814570,116813883) = 999313 bytes
2019-07-05T16:20:09.608+03:00 - TRACE - parsing block [116813891,117813211) = 999320 bytes
2019-07-05T16:20:09.614+03:00 - TRACE - parsing block [117813219,118812538) = 999319 bytes
2019-07-05T16:20:09.620+03:00 - TRACE - parsing block [118812546,119810952) = 998406 bytes
2019-07-05T16:20:09.627+03:00 - TRACE - parsing block [119810960,120810883) = 999923 bytes
2019-07-05T16:20:09.633+03:00 - TRACE - parsing block [120810891,121810146) = 999255 bytes
2019-07-05T16:20:09.639+03:00 - TRACE - parsing block [121810154,122808787) = 998633 bytes
2019-07-05T16:20:09.646+03:00 - TRACE - parsing block [122808795,123808053) = 999258 bytes
2019-07-05T16:20:09.652+03:00 - TRACE - parsing block [123808061,124807299) = 999238 bytes
2019-07-05T16:20:09.658+03:00 - TRACE - parsing block [124807307,125806589) = 999282 bytes
2019-07-05T16:20:09.664+03:00 - TRACE - parsing block [125806597,126805846) = 999249 bytes
2019-07-05T16:20:09.671+03:00 - TRACE - parsing block [126805854,127805083) = 999229 bytes
2019-07-05T16:20:09.677+03:00 - TRACE - parsing block [127805091,128804375) = 999284 bytes
2019-07-05T16:20:09.684+03:00 - TRACE - parsing block [128804383,129802606) = 998223 bytes
2019-07-05T16:20:09.690+03:00 - TRACE - parsing block [129802614,130801905) = 999291 bytes
2019-07-05T16:20:09.696+03:00 - TRACE - parsing block [130801913,131799933) = 998020 bytes
2019-07-05T16:20:09.703+03:00 - TRACE - parsing block [131799941,132798191) = 998250 bytes
2019-07-05T16:20:09.709+03:00 - TRACE - parsing block [132798199,133798079) = 999880 bytes
[examples/parse_blk.rs:14] blocks.len() = 134

[romanz]

Which version of electrs are you running?

[andrewtoth]

https://github.com/blockstream/electrs. I will try with master on this repo and report back.

[romanz]

Note that https://github.com/blockstream/electrs exposes REST API for Esplora, while https://github.com/romanz/electrs/ exposes Electrum RPC for Electrum wallets.

[andrewtoth]

Yeah my goal was to try and run esplora locally. However, there isn't a way to submit issues to a forked repo so I had to report here.

[romanz]

Got it, thanks!
I think that you can report the issue here:
https://github.com/Blockstream/esplora/issues

[romanz]

Someday, we should merge the repos together...

[andrewtoth]

I think that you can report the issue here:
https://github.com/Blockstream/esplora/issues

~Unfortunately that link doesn't work, and it redirects to /pulls.~ Whoops, nevermind, I thought you were linking to https://github.com/blockstream/electrs/issues. I can report there I guess, but let me just make sure I can sync this repo locally.

[andrewtoth]

Confirmed this works fine for me. Closing in favour of https://github.com/Blockstream/esplora/issues/109.