Timeshift: [Feature request] Add some sort of remote location to save snapshots

[q2dg]

Nowadays snapshots can only be saved on local partitions/disks.
Would it be a crazy idea if there was some other options like NFS/Samba/SSHFS/WebDAV/whatever-mounted folders (like Deja-Dup offers, for instance)?
Thanks!

[teejee2008]

Timeshift snapshots use hard-links which is not supported by cloud storage file systems. Hence there is no way to move the snapshots to cloud storage. You can copy the snapshots folders manually, but the hard-links will break and snapshots will take up a huge amount of space. There's no workaround for this and therefore it is not supported.

[mauromol]

But Samba with Unix extensions and NFS do support hard linking, so why not supporting this kind of network file systems? Not to say about SSH.

I'm using a Samba share with BackInTime (which uses hardlinks & rsync) with no problems. It would be very useful to have remote location support when using a laptop.

[VA1DER]

Is this possible now by making /timeshift an nfs mount?

Is there any movement on this? Remotely storing a backup would be ideal.

[Schroedingers-Cat]

@teejee2008 it is my understanding that SSHFS has great support for hard-links. So that should not hold you back, should it?

I tried to mount an ext4 partition via SSHFS into /timeshift in order to trick timeshift into remote backup. But when starting the timeshift job, I get a bizzare error message, that timeshift needs roughly 744GB to backup the 30GB system partition. It is my guess that timeshift also looks into the /timeshift directory and takes all the files in there into account when calculating the necessary free disk space. I also tried to list the /timeshift folder in the /etc/timeshift.json exlusion list, but that didn't help. Once I select a local target drive, timeshift successfully creates a 10GB snapshot (so the 744GB estimation is really far off). This seems like a bug with the exclusion list being ignored during the calculation of the required free disk space. If it isn't, is there a way to ignore that error?

[teejee2008]

The estimation is broken because you are trying to force the program to do something that it is not meant to do. Even if you manage to somehow create a backup on a remote drive, you will not be able to restore it without doing a lot of additional work.

As mentioned previously, there are no plans to add this feature as there is nobody to work on it. If someone can take up this work please go ahead. I won't be merging any PRs for this though, because of the complexity it will introduce.

[Schroedingers-Cat]

From this conversation I cannot see any mentioning that "there is nobody to work on it" - sorry if that has been brought up in another Issue.

Right now, this thread only names "hard-links" as the problem preventing network backup. I wonder what else?

[2martens]

I am doing daily manual backups of the home directory with rsync to my local server in my home. It is a remote location that runs CentOS and has zero problems with hardlinks. Since I am doing essentially what timeshift does but without a nice GUI, I won't necessarily need timeshift to support these use cases but it would be nice if it did.

#!/bin/sh
HOST=homeserver # SSH host
SOURCE=$HOME # source directory for backup
PATHTOBACKUP=/backup/thinkpad/home # path of backup on remote device

date=`date "+%Y-%m-%d"`

rsync -avz --info=progress2 --link-dest=$PATHTOBACKUP/current --exclude="VirtualBox VMs" --exclude=".thunderbird/**" --exclude=".vagrant.d/**"  --exclude="Downloads/**"  --exclude=".wine/**" --exclude=".cache/**" --exclude=".local/share/Trash/**" --exclude="Nextcloud/**" --exclude="git-repos/**" --exclude="**/vendor/**" --exclude=".meteor/**" --exclude=".mozilla/**" --exclude=".npm/**"  $SOURCE $HOST:/backup/thinkpad/home/back-$date

ssh $HOST "rm $PATHTOBACKUP/current && ln -s back-$date $PATHTOBACKUP/current"

[Schroedingers-Cat]

This program uses rsync at its core ("Select snapshot type" -> rsync). Rsync can be used to directly store the snapshot on a remote storage via sftp with a command like rsync [OPTION...] [USER@]HOST::SRC... [DEST]. @teejee2008 What exactly is this program doing that it cannot use this already built-in feature of rsync?

[teejee2008]

It's not impossible. It's just a lot of work. Somebody has to do it.

Since this is open-source, you can download the source code and add the feature if you have the time and interest. I won't be working on this.

[bruceleerabbit]

To be clear, this is not just a convenience feature that is nice to have -- offsite backups are essential for security. A backup tool that forces a local drive destination is not ready for prime-time. It's a big vulnerability. When there is a local disaster, you want your data to be somewhere else. I'm not sure what the core developers are working on that's more important than this.

I'm a bit frustrated because normally I would just walk away from Timeshift and use Duplicity. But linux Mint has been designed so that admins are forced to use Timeshift before doing a dist-upgrade.

I'm tempted to try this workaround:

$ rm -rf /timeshift
$ mkdir /timeshift
$ sshfs user@remotehost:/home/user/backups /timeshift
$ timeshift

If I choose the local disk in the setup, will Timeshift use the remote mountpoint without error?

[Schroedingers-Cat]

@bruceleerabbit I tried the same trick and this is what happened: https://github.com/teejee2008/timeshift/issues/52#issuecomment-449354311

[zer0lean]

Did someone find a solution for this?

[igelkotten]

Disclaimer: I have not yet tried timeshift, so these are assumptions but I think my theory holds(I would love to hear if it doesn't)

Couldn't you just mount a sshfs/nfs/smb share and create a single file that you emulate as a drive (https://superuser.com/questions/518554/how-do-you-create-and-partition-a-raw-disk-image) you then simply point timeshift to run backups on the raw diskimage 'dev' location. This would then emulate as a drive that you might be able to point to using timeshift?

Looking at the 'gui' https://itsfoss.com/backup-restore-linux-timeshift/ timeshift might be looking for 'pure physical disks' based on drivers or other means to differentiate emualte disks so you might need to point to it using the configuration files or some cli commands.

Then in theory that should work? But I have some concerns having it 'remote' as if you f-up your network drivers or something you will ofcourse not be able to simply locally 'timeshift' back. Maybe the raw file can be exported to a drive or some other ways of getting timeshift to create a a 'point in time' that you could export to another file that you then import?

Like: timeshift get-exact-files-at-date xx-yy-zz --export-to-dummy-file drivestate.raw
and then simply dd if=drivestate.raw of=/dev/sd{X}

However for most happy penguincampers I think another backup solution is preferred for 'media' and then use timeshift as it's intended purposes. Perhaps try to separate all your media to other drives or(I assume timeshift support lvms or even exclusions of folders) so you back these up using some 'copy-on-write' and validate these are running and then you just timeshift the drive/lv that contains boot and root applications.

[JeRoNZ]

It is possible to set up an rsync service on another linux box, and to use hard links for unchanged files. I've been doing this for several years with a cloud storage service as the backup device. The options to make this work are --link-dest=DIR --inplace. e.g.
rsync -vrlptDh --modify-window=1 --inplace --link-dest=YESTERDAY DIR rsync://user@host:/path/to/backup/TODAY/

where YESTERDAY is the most recent backup, TODAY is the new backup to make. File unchanged in YESTERDAY are hardlnked into TODAY.

The first backup transfers every file, lock stock and barrel. Thereafter backups are incremental due to the hard links.

Surely this is exactly the same was a local rsync?

[357up]

Disclaimer: I have not yet tried timeshift, so these are assumptions but I think my theory holds(I would love to hear if it doesn't)

Couldn't you just mount a sshfs/nfs/smb share and create a single file that you emulate as a drive (https://superuser.com/questions/518554/how-do-you-create-and-partition-a-raw-disk-image) you then simply point timeshift to run backups on the raw diskimage 'dev' location. This would then emulate as a drive that you might be able to point to using timeshift?

Looking at the 'gui' https://itsfoss.com/backup-restore-linux-timeshift/ timeshift might be looking for 'pure physical disks' based on drivers or other means to differentiate emualte disks so you might need to point to it using the configuration files or some cli commands.

Then in theory that should work? But I have some concerns having it 'remote' as if you f-up your network drivers or something you will ofcourse not be able to simply locally 'timeshift' back. Maybe the raw file can be exported to a drive or some other ways of getting timeshift to create a a 'point in time' that you could export to another file that you then import?

Like: timeshift get-exact-files-at-date xx-yy-zz --export-to-dummy-file drivestate.raw
and then simply dd if=drivestate.raw of=/dev/sd{X}

However for most happy penguincampers I think another backup solution is preferred for 'media' and then use timeshift as it's intended purposes. Perhaps try to separate all your media to other drives or(I assume timeshift support lvms or even exclusions of folders) so you back these up using some 'copy-on-write' and validate these are running and then you just timeshift the drive/lv that contains boot and root applications.

This does work (at least with NFS). After the virtual drive in mounted, use blkid command to determine UUID of the loop device. Then edit /etc/timeshift.json and replace backup_device_uuid with the loop device' UUID.