Raspiblitz: Lnd panic failure on start

[adolfosrs]

My node is shutting down when I unlock it and I'm not being able to close my open channels.

Have made some research and thought it could be database corruption so I've restored my node to a previous state i had backedup but the problem is still happening.

Full logs: https://termbin.com/728t

May 12 12:01:35 adolfosrs systemd[1]: Starting LND Lightning Daemon...
May 12 12:01:35 adolfosrs systemd[1]: Started LND Lightning Daemon.
May 12 12:10:57 adolfosrs lnd[22410]: panic: runtime error: index out of range
May 12 12:10:57 adolfosrs lnd[22410]: goroutine 1 [running]:
May 12 12:10:57 adolfosrs lnd[22410]: github.com/coreos/bbolt.(*DB).page(...)
May 12 12:10:57 adolfosrs systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 12 12:10:57 adolfosrs systemd[1]: lnd.service: Unit entered failed state.
May 12 12:10:57 adolfosrs systemd[1]: lnd.service: Failed with result 'exit-code'.
May 12 12:11:57 adolfosrs systemd[1]: lnd.service: Service hold-off time over, scheduling restart.
May 12 12:11:57 adolfosrs systemd[1]: Stopped LND Lightning Daemon.
May 12 12:11:57 adolfosrs systemd[1]: Starting LND Lightning Daemon...
May 12 12:11:57 adolfosrs systemd[1]: Started LND Lightning Daemon.

[fluidvoice]

please run this and post the resulting URL/link:
sudo cat /var/log/syslog | nc termbin.com 9999

[adolfosrs]

https://termbin.com/epj7

[fluidvoice]

Most likely data corruption (saw this in past LN issues) and caused by under-voltage (your log is full of this). You should get a better power supply and possibly a separate power supply for your hard drive or plugged into a powered USB hub.

[adolfosrs]

Yeah. Thought it was that. Already bought a new power supplier so I'm waiting the shipping.

What would be my options in this case? Have a few channels that have some liquidity there so would like to recover it somehow.
I have a backup from when it was not corrupted yet but it seems like it get to the same error every time I try to restore it. :/

[fluidvoice]

Have a look at the doc's and FAQ. Backup and recovery is documented.

[adolfosrs]

I did. What I'm saying is that I get the backup running but after re filterning all the blocks the error just showup again on the recovered node.

[fluidvoice]

I did. What I'm saying is that I get the backup running

It would help if you explain what exactly you did.

[fluidvoice]

I've restored my node to a previous state i had backedup

this doesnt' tell me what exactly you did to restore your node.

[fluidvoice]

did you do this? https://github.com/rootzoll/raspiblitz/blob/master/FAQ.md#how-can-i-recover-my-coins-from-a-failing-raspiblitz

[adolfosrs]

Oh, sorry.

I have a lnd-rescue file so I'm doing whats described here: https://github.com/rootzoll/raspiblitz/blob/master/FAQ.md#how-can-i-recover-my-coins-from-a-failing-raspiblitz

[adolfosrs]

Hey @fluidvoice, what if the nodes I have channels opened with force closed on their side? Would the balance be available on my on chain wallet?

I've reported the problem on Lnd repo as well https://github.com/lightningnetwork/lnd/issues/3071 and roasbeef suggested to dig the data manually. Do you have any reference on how this could be done?
Thanks!

[fluidvoice]

Do you have any reference on how this could be done?

@roasbeef said ... "It's possible to attempt to retroactively create SCBs"
The only thing I know about creating SCB's is via lncli commands which you can read about, as I did, by running lncli --help and look for "channel" backup and restore commands. But I'm not sure if he means this, or something more involved.

[adolfosrs]

Ok! Figured it out.

On the corrupted node I have run lncli exportchanbackup --chan_point xxx for each channel and saved each hash. Then I have deleted my ./lnd/data and created a new wallet using my seed words with the passphrase. Then, with the lnd node synced I have run lncli restorechanbackup --single_backup for each one of the backed up channels hashes.

Not sure if thats by design but in my case it closed the channels right away. So I could check the channels listed as waiting_close_channels when running lncli pendingchannels.

@fluidvoice Thanks for pointing me in the right direction. Really appreciate it. Have just tipped you on tippin.me.