Panel: Backups System

[DaneEveritt]

Idea of Backups System:

• Per-server backup space allocation, or per-server total backup quantity.
• Ability to store backups offsite or onsite; support for Backblaze B2, AWS, and Google Cloud at minimum.
• In-panel management interface to specify which files and folders should be backed up.
• Integration with command sending system to send commands to server before starting the backup (i.e. save-off, or telling users a backup is in progress)
• Ability to schedule backups through the task system.
• View, download, delete backups though panel.
• Ability to restore backup through the panel.

[xyeLz]

This is a great thought since not many panels currently offer this. As usual, I have my own thought of what this would look like:

First off, I do agree that per-server space allocation is a great way to limit backups from getting out of control. It would also be great to have the option to specify "backup storage size" on a per-client basis.

I'm not so keen on keeping the backups on the storage that their particular game server is being run on since if the host goes down, bye bye server and bye bye backups. This is another reason it would be nice to specify the amount of size to allow for backups on a per-client basis. I really like the thought you had of doing offsite backups as well. If you do this, though, it might be helpful to have some sort of way to limit the bandwidth used for backups.

I also had a thought that may or may not work: You guys have the option to add "database servers". What about adding something like "backup servers" as well? Just chunk storage to store backups on a routine basis. Users can setup scheduled tasks to backup as often as they'd like (or perhaps this can be limited to a certain timeframe based on the admin's preferences?) until their disk space allocation is at capacity. I haven't given this much thought in terms of security, since obviously an admin won't want to deploy a different server for each client just for backups, so the files will need to be separated and secured somehow, but just a thought. When the user wants to restore data, they can pick a snapshot from a drop down list that they want to restore from.

Just thinking out loud, as usual. You guys always come up with a very pretty way to do things, so I'm sure this will be just the same if/once it's implemented.

[IlPicasso]

Maybe a way to define a NODE as a Backup NODE so it can save there the backups

[Derkades]

How about something like this?

• Every hour a backup which is kept for a day
• Every day a backup which is kept for a week
• Every week a backup which is kept for a month
• Every month a backup (kept forever)

[xyeLz]

I'd say just give the users the option to select the frequency and files which should be backed up instead of limiting it, just like with the tasks.

[Derkades]

We could add an option to the existing tasks system:

• Power change
• Server command
• Make backup

[ghost]

I'm currently working on a backup system with bash script via tar and ran automatically via crontab. I will update when I have it setup so that it is working and you can select restore to a date and time on the panel.

[inklit]

It might be worth using a differential backup method so you don't have to store files you already have twice.

[schrej]

That would be a later iteration as it is harder to implement. The first approach will be full backups.

[gaigelama]

Why not just base the backup system off rsync?

[xtechgamer735]

@xion87 ive since decided to use a tool called Duplicati. Check it out

[Chazotu]

Suggestion on Backup Features

• Ability to auto encrypt backups

• Ability to have more backends

• Ability to to select the Backend Note

• Ability to mirror the backups to different Backend Notes

[gaigelama]

Restic is another great option and supports a lot backup services. However it might be too complex.

[bloggy]

Is backup function already included?

[Malachiel87]

I don't think so, but you can do it yourself with a sh script and cron

[lancepioch]

Encrypting backups natively with PHP 7.2: https://twitter.com/skollro/status/1145041226682503169?s=20

For encrypting, a zip password is sufficient.

[104938-sevenwolden]

just throwing this in here:
export all MySQL databases for a server to sql dump files in the root of the server. Name the sql backup of database "s2_test" s2_test.sql and if there's an option planned to restore backups, restore the sql dumps to their respective databases

EDIT: Would be nice to see support for stuff like SFTP and WebDAV (davfs2) too

[DaneEveritt]

This functionality was added to the panel in ad9194a65cb82a7a624930b379ba119d616f952e, 15b436d26e8e0219c672c741242816cbfbf85ab8, be05d2df81cf85c2bd9444a86bd685294bae649c, and 7b69b4b82409833c5238156d7e7400cf5586e275. Functionality was added to the daemon in https://github.com/pterodactyl/wings/commit/4ce2b734907448ed669b71dbc5c50816689de4b2. A basic backup functionality is now present in the panel core, and we will be continuing to iterate on it before release.

To answer a few common questions here:
1.) Yes, you will be able to generate backups on a schedule.
2.) Yes, you can limit the number of backups that any given server is allowed to have at a single time.
3.) Backups are generated as compressed tar archives.
4.) At this time the only two supported modes will be storing them on the local machine in a specific directory and storing them in AWS S3. We do have plans to support plenty more, we're just starting with a very narrow scope to ensure it works well.

Thank you for your patience with this feature, I know it has long been requested and I dragged my feet getting it implemented.