Raspiblitz: After recovery with seed and channel.backup only offchain funds recovered not onchain

[lebowski36]

Is there another way to recover the onchain funds beside resync the raspiblitz node from scratch again? I got around 1million sat from the channels back but I am mising 2 million sat from onchain wallet.

[rootzoll]

Now that you closed all channels with the channel.backup file. Shutdown RaspiBlitz and install the LND Desktop App on your laptop: https://github.com/lightninglabs/lightning-app/releases and use the seed words to recover on-chain funds again.

Somehow the LND Desktop App has more success in recovering the on-chain funds by seed words. Not sure why yet and what we do different with LND on the Blitz compared to the LND Desktop App - because recovery window is set to 250 (what should be the same the Desktop App is using). We tested already with very big recovery windows (--recovery_window=250000), but it just made recovery very very slow and had not always success.

See details on issue https://github.com/rootzoll/raspiblitz/issues/278 also #536 #500 .. this a long an painfull story so far.

In the RaspiBlitz the important code is to find in:
raspiblitz/home.admin/config.scripts/lnd.initwallet.py
see lines 153/183

@halseth from lightning labs will be at the ligthning hackathon in munich - hope he can take a look at this and help out

[lebowski36]

thanks for your respone rootzoll

I actually read this part but could not proceed further because I saw no way to add the cipher passphrase extra word into the lightning app. Do I miss something?

Edit:
I do get the error invalid passphrase, the seed is entered correct tried 3 times

[rootzoll]

Best to get feedback on the LND Slack channel for the LND Dektop App: https://dev.lightning.community

[halseth]

@lebowski36 There's no way around rescanning the chain to get on-chain funds back.

@rootzoll How are you getting new addresses from lnd? If you don't use the UNUSED_ address types, then extensive polling of the API can cause address inflation and make it harder to find the uses addresses on rescan.

[btc21cluib]

"I do get the error invalid passphrase, the seed is entered correct tried 3 times"
any solution about the passphrase error on LND desktop wallet?
i have the same problem with the Lightning app 0.5.1-alpha , can't recover

[openoms]

I helped a user to restore funds recently. Managed to get off-chain funds back using the seed and SCB on the RaspiBlitz, but on-chain funds were not restored.

Once cashed out the channel balances proceeded to restore with the seed in the desktop Lightning app.
Funds appeared, but unfortunately the autopilot is on by default in the app, so done some unnecessary channel opening and closing to be able to get all the funds into cold storage eventually.

Do we still think that the restore process on the RaspiBlitz is missing to scan the addresses with the balances?
It is interestingly quite consistent at that.
It is the python LND GRPC API RaspiBlitz uses to communicate with LND on restore. The difference in the restore process should be in the way grpc calls LND compared to the desktop app.

I will try to reproduce the problem with a couple of sats myself.

[rootzoll]

@halseth the raspiblitz gets new addresses for funding in mostly two ways:

BBfundWallet.sh (used from SSH main menu)

lncli --chain=${network} --network=${chain}net newaddress np2wkh

and thru the RTL WebGUI (when activated) thru the /newaddress API endpoint .. the user can choose the type. I cannot find any documentation on a UNUSED_ parameter. Can you point us into the right direction here?

@halseth also see the procedure described above how RaspiBlitz users at the moment handle fund recovery: https://github.com/rootzoll/raspiblitz/issues/632#issuecomment-500121764 - within this process users that set a seed-encryption password seem to fail - but users that created a seed without a password can recover on-chain funds with the lnd-desktop app, but not with the RaspiBlitz. What is the RaspiBlitz doing different here? This is how the raspiblitz is trying to recover a wallet from seed at the moment:

/config.scripts/lnd.iniwallet.py (python protobuff interface)

    request = ln.InitWalletRequest(
        wallet_password=walletpassword,
        cipher_seed_mnemonic=seedwords,
        recovery_window=250,
        aezeed_passphrase=seedpassword
    )

    try:
        response = stub.InitWallet(request)

before v1.3 we used simply lncli call with initwallet and seed info, but also was not able to recover on-chain funds durign scan - even with a very very big recoverywindow.

[mikelmore]

I could recover the onchain wallet funds by generating the private keys of the wallet using a couple of web utilities.
https://guggero.github.io/cryptography-toolkit/#!/aezeed
In this web pasting the seed and the passphrase in the decode mnemonic section and obtaining the HD node root key.
Then pasting the root key on Ian Coleman's BIP39 mnemonic code converter
https://iancoleman.io/bip39/
(Bip 32 root key) and generating as many adresses/private keys pairs as wished using the Bip84 derivation path.

[RED29]

@mikelmore thanks for sharing your knowledge!
I am also trying to recover funds without channel.backup. There are funds that are sent back to my wallet (because I requested the peer to force close the payment channel).

With only my seed I was trying to recover the funds that where sent back.

1. The first step, putting the seed on an offline html:
   https://guggero.github.io/cryptography-toolkit/#!/aezeed
   Works nice, but I have to say that I don't have a passphrase (rapiblitz).

2. Putting the HD node root key into the BIP 32 root key with the html tool https://iancoleman.io/bip39/ gives me the ability to generate different keys.

3. I tried the following things;

For each Derivation Path (BIP32, BIP44, BIP49, BIP84 and BIP141);
I have tried the results from "account extended private key" en "BIP32 Extended Pirivate Key" into the Electrum master key recover.

Results; no funds

1. Than I generate 50k addresses from the BIP84, and try to recover it from private key,
   In Electrum recovery with private key.

Results: no funds

What am I doing wrong? I know the seed I have is the good one, because I already used to recover the onchain funds. No I want to recover the funds that where sent back from forced closed channel.

I hope you can help me out here.

[mikelmore]

Hello, I only tried with wallets created from seed + passphrase and it worked like a charm. You only need the first address to wich you sent your initial funds to find out if it works.
Once you get your HD derivation key use the Bip84 derivation path on Ian Coleman' s Bip39 web utility. I only needed like 40 adresses and 40 change adresses to get all the funds.

From the aezeed spec:

Init quote

aezeed enciphering/deciperhing

Next, we’ll take the plaintext seed described above and encipher it to procure a final cipher text. We’ll then take this cipher text (the CipherSeed) and encode that using a 24-word mnemonic. The enciphering process takes a user-defined passphrase. If no passphrase is provided, then the string “aezeed” will be used.

End quote

I would try using "aezeed" but I have not tried myself.
I hope it helps.
Mikel

El 8 de julio de 2019 0:45:16 CEST, RED29 [e-mail] escribió:

@mikelmore thanks for sharing your knowledge!
I am also trying to recover funds without channel.backup. There are
funds that are sent back to my wallet (because I requested the peer to
force close the payment channel).

With only my seed I was trying to recover the funds that where sent
back.

1. The first step, putting the seed on an offline html:
   https://guggero.github.io/cryptography-toolkit/#!/aezeed
   Works nice, but I have to say that I don't have a passphrase
   (rapiblitz).

2. Putting the HD node root key into the BIP 32 root key with the html
   tool https://iancoleman.io/bip39/ gives me the ability to generate
   different keys.

3. I tried the following things;

For each Derivation Path (BIP32, BIP44, BIP49, BIP84 and BIP141);
I have tried the results from "account extended private key" en
"BIP32 Extended Pirivate Key" into the Electrum master key recover.

Results; no funds

1. Than I generate 50k addresses from the BIP84, and try to recover it
   from private key,
   In Electrum recovery with private key.

Results: no funds

What am I doing wrong? I know the seed I have is the good one, because
I already used to recover the onchain funds. No I want to recover the
funds that where sent back from forced closed channel.

I hope you can help me out here.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/rootzoll/raspiblitz/issues/632#issuecomment-509036661

--
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

[halseth]

@rootzoll Looks like the Lightning App is also using 250: https://github.com/lightninglabs/lightning-app/blob/7b055a2704631c344bc0add526d86da35a92be20/src/config.js#L31

Are you saying that recovery _never_ works for you, or just sometimes fail? If it never works my guess is that maybe something is wrong with how the seed/password bytes are handled on the python side. I would try logging the raw bytes of the InitWalletRequest both from your python script and from the Lightning App to see where they differ.

[halseth]

You can find the definition of the address types here, doesn't look like they are picked up by the API doc generator for some reason: https://github.com/lightningnetwork/lnd/blob/93aad89f4c132089d77181a7aec5d717bf1c621a/lnrpc/rpc.proto#L1014

[rootzoll]

Are you saying that recovery never works for you, or just sometimes fail?

@halseth recovering the on-chain balance is not working out of the box. As you can see in this issue people were able to recover thru other workarounds ... like using the app or manual scripts. But when doing it with RaspiBlitz it almost always failed. Older lnd versions sometime showed parts of the on-funds - when people made very big recovery windows ... but that was also just a lucky shot and not working all the time.

my guess is that maybe something is wrong with how the seed/password bytes are handled on the python side. I would try logging the raw bytes of the InitWalletRequest both from your python script and from the Lightning App to see where they differ.

@halseth We had the problems with not recovering the on-chain funds also in ealier versions of RaspiBlitz were we used the original lncli dialog to init/recover from seed words. No we use the python/gRPC call like this

    request = ln.InitWalletRequest(
        wallet_password=walletpassword,
        cipher_seed_mnemonic=seedwords,
        recovery_window=250,
        aezeed_passphrase=seedpassword,
        channel_backups=scbHexString
    )

So not sure were we can do someting wrong with the byte handling of the seed words -. because they are just a basic string and they seem to produce the same node id after recovery .. so its the same/correct seed (and password). Seems this so far correct?

You can find the definition of the address types here, doesn't look like they are picked up by the API doc generator for some reason

@halseth how can the address type help? I dont get the context within recovery process - please explain. Thanks.

[openoms]

I could recover the onchain wallet funds by generating the private keys of the wallet using a couple of web utilities.
https://guggero.github.io/cryptography-toolkit/#!/aezeed
In this web pasting the seed and the passphrase in the decode mnemonic section and obtaining the HD node root key.
Then pasting the root key on Ian Coleman's BIP39 mnemonic code converter
https://iancoleman.io/bip39/
(Bip 32 root key) and generating as many adresses/private keys pairs as wished using the Bip84 derivation path.

For completeness and to aid recovering funds pasting the recent article from @guggero about restoring only from the seed and using the information from 1ml.com to generate a "fake" channel backup file: https://medium.com/@guggero/did-you-lose-funds-on-the-lightning-network-because-of-a-disk-crash-8971b6a92494

[escapedcat]

Not sure if this is the underlying issue regarding the LND desktop app but somethign happend on LND side: Issue: #3168 Wallet recovery with passphrase is not working

I think it's not yet released though.

[rootzoll]

@escapedcat thanks for the link ... what they discuss mostly passing parameters correctly to the LND REST API - in RaspiBlitz we use the gRPC thru python. I dont see this related.

[escapedcat]

Ah sorry, I was hoping it's somehow relevant to the workaround recovery using the LND desktop app as discussed above. Because I also get the invalid passphrase error message when trying to recover with the 24 words seed as @lebowski36 mentioned. The app isn't even asking for a passphrase.

[rootzoll]

Yes that the LND Desktop is not asking for a password is a bummer and makes it impossible to recover old RaspiBlitz Wallets with it. Thats why the newer RaspiBlitz versions, dont ask for a seedpassword anymore.

[escapedcat]

Hm, I guess I'll wait till this works someho, will create a new wallet when 1.4 is coming out and start fresh withour trying a recovery. Eventually this will work in a easy way I guess. The suggested approach from @mikelmore seems a bit over head to be honest.

[theotherside]

This is only a FYI and I am going to restore a different LND wallet now so there is no more issue for me at this time.

On a fresh v1.4 I first tried to restore my LND wallet with seed and SCB. When I ssh'd into pi it said there was a problem with channel.backup because it was for a different wallet. So I use repair option for LND and I restore with the correct seed and channel.backup. When the blitz got up and running my on chain balance was not there. I log in with ssh it says channel.backup was successful must wait like an hour. Later I see the funds I would expect to see if my channels were closed, this does not include all previous onchain funds. At this time I still only see that channel closed balance. If I restore seed with lightning it shows my entire onchan balance witch now includes the btc from channel closing. Raspiblitz only shows onchain funds from channel closing.

[openoms]

@rootzoll an idea here:

The RaspiBlitz menu defaults to fund the LND wallet on a nested witness key hash address (3...)
The channel closures will return to Bech32/witness key hash (bc1.. ) addresses.

It seems that on restore from the seed (+/- SCB) the RaspiBlitz only shows the funds on the Bech32 addresses.

I think it is time to default to fund the wallets on Bech32 addresses as they are the cheaper option and those are supported by majority of CoinJoin clients too (Wasabi and Whirpool - JoinMarket is working with nested witness key hash (3...).

It could help to solve this issue too.

In the meantime I was funding my LND wallet with Bech32 addresses anyway manually:
lncli newaddress p2wkh

• p2wkh: Pay to witness key hash = Bech32 (bc1...)
• np2wkh: Pay to nested witness key hash = P2SH (3..)

[rootzoll]

@openoms added that idea to the final v1.4 release: #1018