Homebridge-config-ui-x: Automated backups

[sschuste]

Describe Your Problem:
I like your idea how you implemented the backup and restore of the Homebridge with all the installed plugins. I would like it even more if I could automate the download. I had in mind a curl command that I would call via cron. Unfortunately, I failed to get authorization.

I tried this GET request:

curl 'https://homepi.lan:8088/api/backup/download' \
-XGET \
-H 'Cookie: io=ilH9DmN8GIQAg-JjjAAAU' \
-H 'Accept: application/json, text/plain, */*' \
-H 'Accept-Encoding: br, gzip, deflate' \
-H 'Host: homepi.lan:8088' \
-H 'user agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15' \
-H 'Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9. eyJ1c2VybmFtZSI6ImFkbWluIiwibmFtZSI6IkFkbWluaXN0cmF0b3IiLCJhZG1pbiI6dHJ1ZSwiaW5zdGFuY2VJZCI6IjYwMjk2MTYyyN2RhMDk0NjNjOTk3NzExNjI5MzAzMTNhMWU4MjMyNmQ2NDgyMTNkMzU5YjAwOWJmY2RmZjEzZmEiLCJpYXQiOjE1ODg0MDI4NTAsImV4cCI6MTU4ODQzMTY1MH0. vPDYU2aSRncXm_IvJApYQ1wd_uT_PB0mHV3yF2ZoDWE' \
-H 'Accept-Language: de-de' \
-H 'Connection: keep-alive'

which works great, but unfortunately stops working as soon as the cookie and auth header become invalid. I have also tried to use homebridge-config-ui-x without login authentication, but it does not work at all.

Is there any way to automate the backup function in this way?

Logs:

n/a

Homebridge Config:

n/a

Screenshots:
n/a

Environment:

• Node.js Version: v12.16.3
• NPM Version: 6.14.4
• Homebridge Version: 1.0.4
• Homebridge Config UI X Version: 4.17.1
• Operating System: Raspbian Buster
• Process Supervisor: systemd

[oznu]

You should be able to request an auth token using curl as well.

I'll take a look and write you up a script tomorrow.

[oznu]

@sschuste here you go:

#!/bin/bash

#
# This script requires the "jq" package to be installed.
# Linux: apt-get install jq
# macOS: brew install jq
#

set -e

# Uncomment to debug
# set -x

USERNAME=admin
PASSWORD=admin
UI_HOST=http://localhost:8581
BACKUP_FILE_PATH="$(pwd)"

BACKUP_FILE_NAME="backup-$(date '+%Y-%m-%d').tar.gz"

# Get Access Token
ACCESS_TOKEN=$(curl -fs "$UI_HOST/api/auth/login" \
    -H 'Content-Type: application/json' \
    --data-binary "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD\"}" \
  | jq -r .access_token)

# Download Backup
curl -fs "$UI_HOST/api/backup/download" \
  -H "Authorization: bearer $ACCESS_TOKEN" \
  -o "$BACKUP_FILE_PATH/$BACKUP_FILE_NAME"

echo "Backup saved to $BACKUP_FILE_PATH/$BACKUP_FILE_NAME"

[sschuste]

Yeah, how cool is that? That's awesome. I just had to make a few minor adjustments and another problem solved. Thanks for doing extra work for me. Wunderbar. Dankeschön.

[oznu]

Happy to help. I've been thinking about making the UI do backups every day automatically (they are very small so disk space won't be a concern) - but still need to work out the implications of doing it.

[agruen]

Any additional thoughts on this? Would love a way to do auto backups every day...!

[oznu]

@sschuste I'm looking at adding this into the UI service directly.

My current idea is as follows:

• automated daily backups at 1am
• backups are stored in the homebridge storage folder by default under backups/instance-backups/
• backups will be kept for a rolling 7 days
• the instance backup path will be customisable (so you can set it to a shared folder or network drive for example)

In the following release I will add the ability to download and restore the backups made automatically directly from the UI.

Thoughts?

[rpolenthon]

@oznu Great ideas. Maybe you could also add an option to mark a backup to be stored even after the seven days, in case it is an important one.

On top of that it would be nice to have an API endpoint which allows you to get an info about all stored backup files with the date the backup was created, if it was manually or automated, if it should stay stored after the 7 days and maybe also notes (as requested in #910 )

[oznu]

API endpoints to list and download the scheduled backups will be available as soon as this feature lands - I'm just thinking of delaying the UI features by a release (~one week) to ensure there are no issues in the server side component.

The initial release will only include scheduled backups, not user generated backups, as they are downloaded to the users local machine - which also kind of solves the "I want to keep it more than 7 days" problem 😄

This is currently in @test - though you won't get a backup saved until 01:15am local time.

https://github.com/oznu/homebridge-config-ui-x/wiki/How-To-Test-Upcoming-Changes

[oznu]

Will land, along with the UI features, in v4.31.0.

[sschuste]

Thank you very much. That's a good one. I scripted a similar mechanism that does a little bit more:

• daily backup on the Raspi triggered by a cron job (will be replaced by your solution)
• rsync copy of the backup to my Mac's iCloud directory

Copying to the iCloud increases the security of my backup. For me the Raspi is something that can break at any time. I do not want to leave a complicated homebridge configuration to the capabilities of the SD card.

But the much more elegant solution would be an upload to GitHub with a simple git push/pull mechanism. I am fully aware that GitHub is not really the right place for backups. But an automated solution that stores backups somewhere other than on the Raspi itself would be awesome.

I have to admit that I've been too lazy to really deal with this. I have a solution that works, even if it could be nicer. Maybe a manual on how to synchronize backups/instance-backups/ with a dropbox or other service is enough. This is not to say that this is your job, but maybe you have a good idea.

[ahwman]

I’ve noticed that even though automated backups are only supposed to be stored for the last 7 days, old backups are not being purged.

[montanaishome]

Any way to build an automation to auto copy backups offsite/off sd card? Maybe to an iCloud or google drive link? The SD card is the main point of failure for the device.

[oznu]

@montanaishome you can set this option to tell the UI where to save the backups:

https://github.com/oznu/homebridge-config-ui-x/wiki/Config-Options#scheduledbackuppath