Solana: Snapshots generated by v0.23.7 are not loadable by v1.0

[mvines]

Using the v1.0 solana-ledger-tool to load a snapshot generated by SLP (running v0.23.7) causes a panic when rebuilding the status cache. Find the ABI change and mitigate.

$ solana-ledger-tool -l slp verify`
9Mk3XtM1aqw5WbfF2BXide7zWP88Jb8AbSt3ciVvEZAR
[2020-02-27T21:11:13.209239000Z INFO  solana_ledger::blockstore] Maximum open file descriptors: 65000
[2020-02-27T21:11:13.263849000Z INFO  solana_ledger::blockstore] "/Users/mvines/ws/solana/ledger-tool/slp/rocksdb" open took 54ms
[2020-02-27T21:11:13.264027000Z INFO  solana_ledger::bank_forks_utils] Initializing snapshot path: "/Users/mvines/ws/solana/ledger-tool/slp/snapshot"
[2020-02-27T21:11:13.269242000Z INFO  solana_ledger::bank_forks_utils] Loading snapshot package: "/Users/mvines/ws/solana/ledger-tool/slp/snapshot-247120-GPYUk6xrxikt5UBzGT6hVpKZEhWRMocyrtY17iX1MUF9.tar.bz2"
[2020-02-27T21:12:02.570889000Z INFO  solana_ledger::snapshot_utils] snapshot untar took 49.3s
[2020-02-27T21:12:02.571029000Z INFO  solana_ledger::snapshot_utils] snapshot version: 0.23.2
[2020-02-27T21:12:02.571175000Z INFO  solana_ledger::snapshot_utils] Loading bank from "/Users/mvines/ws/solana/ledger-tool/slp/snapshot/.tmpBw3nlr/snapshots/247120/247120"
[2020-02-27T21:12:02.581151000Z INFO  solana_ledger::snapshot_utils] Rebuilding accounts...
[2020-02-27T21:12:05.575925000Z INFO  solana_ledger::snapshot_utils] Rebuilding status cache...
thread 'main' panicked at 'Load from snapshot failed: Serialize(Custom("invalid value: integer `2879970114`, expected `Ok` or `Err`"))', src/libcore/result.rs:1188:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1025
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:471
  11: rust_begin_unwind
             at src/libstd/panicking.rs:375
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
  13: core::result::unwrap_failed
             at src/libcore/result.rs:1188
  14: core::result::Result<T,E>::expect
             at /rustc/5e1a799842ba6ed4a57e91f7ab9435947482f7d8/src/libcore/result.rs:983
  15: solana_ledger::bank_forks_utils::load
             at ledger/src/bank_forks_utils.rs:67
  16: solana_ledger_tool::load_bank_forks
             at ledger-tool/src/main.rs:547
  17: solana_ledger_tool::main
             at ledger-tool/src/main.rs:830

[mvines]

Gotcha! 0fd795a676 added an out-of-order InstructionError enum value

[mvines]

https://github.com/solana-labs/solana/blob/43af91ff4b0769c2ad8bf461b586dbdbc1fd2fca/sdk/src/instruction.rs#L91 was added before https://github.com/solana-labs/solana/blob/f8add53a7131b74a5b13d7618b32bf195e3fce87/sdk/src/instruction.rs#L97, breaking deserialization.

cc: @ryoqun to confirm that the ABI lockdown work would have been able to catch this

[ryoqun]

cc: @ryoqun to confirm that the ABI lockdown work would have been able to catch this

@mvines I've confirmed this would be caught by the ABI lockdown work: https://github.com/solana-labs/solana/pull/8512/files#r385597341 Really sad, I couldn't land it on time...

For that, I've rebased that MEGA PR. Fortunately, I've seen no other ABI breakage for the HEAD of master under the limited ABI coverage of the PR.