Raspiblitz: Helper Script for LND Data Backup and Recovery

[rootzoll]

If people wanna move LND data dir for backup and replay, that should be more supported:

• create ZIP (with size/hash in name)
• use SCP commands
• check ZIP size/hash on replay

[openoms]

I have tested this one with success: https://github.com/vindard/lnd-backup/blob/master/do-lndbackup.sh
Trying to make @vindard to make a PR :)

[vindard]

Thanks for the push @openoms 😂 ... PR in by end of this weekend

[fluidvoice]

@openoms @vindard backing up anything Bitcoin related to dropbox or any 3rd party cloud provider, especially by default, is a bad idea IMO. Just make a local backup and let users copy it to wherever they want.

[vindard]

@fluidvoice the script by default only makes a local backup and archives it into a .tar file (exactly the thing you want? 🤷‍♂️)

I've included a gpg encryption option and logic (next commit) that'll only upload to Dropbox if the archive is encrypted. I think that's a fair enough trade-off giving the user the flexibility to do off-site encrypted backups but having the default as a local snapshot.

Other planned remote-backups include an rsync to a remote server and an encrypt-and-then-push-to-Google-Cloud option for folks who would also want that. Haven't gotten the time to code those up as yet though.

[rootzoll]

My goal minimum here is to make it easier for people to backup the LND data dir in emergency cases - when the RaspiBlitz is failing etc. An easy way to get the data off the RaspiBlitz to the laptop of the user. And then have an easy way to play that data back onto a fresh RaspiBlitz.

[openoms]

I think it would be difficult to make regular backups to the users laptop. Having an outdated backup is possibly even worse than having only the cypher seed. I like the idea to make the script run regularly, checking a change in channel states and make an encrypted backup to an always online cloud.
On the first setup from the service menu the user should be asked for

• their gpg key (or strong backup password - maybe password D?)
• dropbox / google drive (or else - mega.nz?) API key with instructions on the screen about how to get it

I presume it would be the best to run as a service with systemd in the background for regular checks.

[fluidvoice]

GPG key's is needless complication. 99% of users have never used them. So we need to think about the target users for this project. Is it for the gen-public or techies? I think the focus should be on simplicity. Don't expose the users to what is not necessary. Having "hidden features" for the techies is fine. Just some things to think about. The more simple (and cheap) it is, the more users there will be.

[vindard]

@rootzoll the default option for the script is to setup a data-backups folder at /home/admin and to maintain an rsync-ed (pretty much same as scp) copy of the entire .lnd dir there. I chose to do the entire dir because the actual data you'd need (channel.db & wallet.db) make up over 90% of the size of the folder, and backing up the whole folder makes restoring easier as well.

When to backup

I'm not sure if doing the backup "when the RPi is failing" is the best option since as @openoms said, if even 1 bit is off the entire state could be corrupted leading to loss of all in-channel funds. That's why I chose to set it up to be compatible with periodic backups (user's choosing) that can also be run after each channel state change if the user likes.

Where to backup

The "upload to laptop" functionality is where I've included the different remote-upload options. The most convenient for me right now is encrypted upload to Dropbox (shows up immediately on my laptop in synced folder) but there's also the option to do something scp-like if your laptop is on and it's network-available (I haven't coded anything but the "encrypted Dropbox upload" functionality in as yet). Open to other ideas for this "upload" part as well.

Automating backups

@openoms a system service may not be necessary. I currently have it running as a cron job on my RPi (cron's ridiculously easy to setup, & can also be automated via another setup script). The script can be set to run as short as every few mins since the script first checks for channel state changes and then terminates if none are found. I've also included a number of flags and argument-parsing options to facilitate setting up the cron jobs in different ways.

---

User-friendliness

@fluidvoice that's a good point about simplicity and the script is pretty simple from a user's perspective right now. Once the cron job is set (can be an automated process) the user wouldn't even have to know this script is running. The default option places backups right into the home directory on the RPi. This isn't ideal though if the RPi fails since what's best is to have some sort of off-Pi backup option.

Unfortunately, it's probably impossible to do remote backups without some sort of input from the user (api keys, or at the very least an IP address to a remote backup site).

If the user chooses a 3rd-party cloud service specifically the script will not continue without some form of encryption as the default (script has an override flag for this as well).
Gpg setup is actually really easy if you need to do it (keybase.io will do it in a few clicks for you, or a setup script can setup everything for you on the RPi and spit out a string of characters for you to copy and keep safe). Also, there are maybe other symmetric encryption options where we could use the lncli unlock password as the encryption password as well or something.

Given the current state of Lightning Network though (state channel backups etc. are still being worked on and the devs still say it's #reckless to run on mainnet) and the fact that we _are_ running on a Raspberry Pi here, I think it's safe to assume potential users have at least some level of technical competence. For everyone else, those folks maybe shouldn't be running "mainnet LN" quite yet so I'm not sure how much sense it makes trying to completely optimise the script for the broadest audience of people possible as you're suggesting (<- I'm very open to pushback and constructive suggestions on my perspective on this from others).

---

... happy to contribute!

So anyway folks, the script's available at this repo if anyone wants to try it out. @rootzoll maybe you can take a preliminary pass at it and at what I've written here and let me know if it looks like the sort of thing that would make sense for this _Issue_. I'd be happy to make tweaks, write up some documentation and open a PR if it fits with what folks are looking for 😉

[rootzoll]

@vindard thanks for the details :)

I decided to go two ways with this. I added a small rescue and restore data script for the v1.1 release. Its goal is to for those "rescue scenarios" and making support easier when people try to move data in a failing blitz scenario. It works just like this:

To backup your lnd data to your laptop you just use:

/home/admin/config.scripts/lnd.rescue.sh backup

and follow the instructions of the script.

To restore the backup data from your laptop back to a RaspiBlitz you use:

/home/admin/config.scripts/lnd.rescue.sh restore

and follow the instructions of the script.

But I agree that making this rescue just as there are already problems with the blitz it might be a bit late. So after v1.1 I would like the background backup script of @vindard - BTW there is a background process already of RaspiBlitz that can take care of executing it in regular periods.

[rootzoll]

OK testing the lnd.rescue script looks good so far. I will close this issue for v1.1 release - which I want to wrapp up for tomorrow.

To add the script of @vindard I opend a new issue for a future release.
https://github.com/rootzoll/raspiblitz/issues/437