Docker-mailserver: Data Backup / Migrate / Preserve

[Lewiscowles1986]

Hey, this is great, but how would someone migrate e-mails (~100GB across 10 accounts) into this, how would a person export emails from this for backup & migration?

[tomav]

Hi @Lewiscowles1986, thanks for your support.
I'm currently thinking about backups.
It's not only a docker-mailserver question, but a more large postfix question.

I'm thinking using docker data volume container, and perform a backup using a cronjob.
Ideas are welcome. We can work together on this feature.

Regarding migration, this will be specific to your current mail server / deamons / paths...
But have a look to /var/mail, folder follow this pattern: /var/mail/${domain}/${user} which is the user maildir.

[Lewiscowles1986]

I'd love to help if I can be of assistance, I have to say, I use cronjob backups quite a lot sans-docker, with rsync, so I'd love to help in any way I can. My workflow tends to be a lot more vagrant than docker, which I am currently learning.

I'm thinking

total backup

cd /var/mail; tar -zvcf /tmp/mail-`date +"%Y-%m-%dT%H%M.tar.gz"` ./* >> /dev/null

domain backup

cd /var/mail/${DOMAIN}; tar -zvcf /tmp/mail-`date +"%Y-%m-%dT%H%M.tar.gz"` ./* >> /dev/null

domain specific-user backup

cd /var/mail/${DOMAIN}/${USER}; tar -zvcf /tmp/mail-`date +"%Y-%m-%dT%H%M.tar.gz"` ./* >> /dev/null

then just ssh in and dl from /tmp, but maybe ideal world just pipe to a file?

the above works in a cron script (vars -> constants), or shell script for other folders, and can be generated

[gegere]

This process could occur directly via a docker data container copying data directly to a new host.

On Nov 23, 2015, at 18:43, Lewis Cowles [email protected] wrote:

I'd love to help if I can be of assistance, I have to say, I use cronjob backups quite a lot sans-docker, with rsync, so I'd love to help in any way I can. My workflow tends to be a lot more vagrant than docker, which I am currently learning.

I'm thinking

total backup

cd /var/mail; tar -zvcf /tmp/mail-date +"%Y-%m-%dT%H%M.tar.gz"./* >> /dev/null

domain backup

cd /var/mail/${DOMAIN}; tar -zvcf /tmp/mail-date +"%Y-%m-%dT%H%M.tar.gz"./* >> /dev/null

domain specific-user backup

cd /var/mail/${DOMAIN}/${USER}; tar -zvcf /tmp/mail-date +"%Y-%m-%dT%H%M.tar.gz"./* >> /dev/null

then just ssh in and dl from /tmp, but maybe ideal world just pipe to a file?

the above works in a cron script (vars -> constants), or shell script for other folders, and can be generated

—
Reply to this email directly or view it on GitHub.

[Lewiscowles1986]

@gegere do you mean another docker container launched with environment args for DOMAIN, USER, DESTHOST etc? would this really need to be done just to back up?

Don't get me wrong it will be amazing if it works (my current email host delivered me an 8MB backup of emails because their script does not work with 30+GB of emails), so whatever we do I think we need to make sure it is robust. For that reason I am also not sure that tarball is the best option, especially for SSD VPS users who do not think it is reasonable to keep n% of (often limited) space aside for backups.

I am also wondering about piping the output, or simply adding the option for a user to supply an ssh public key for a backup user (lock off all other user connections via ssh and publish over another port, this would provide an outlet for users to use a better solution for large folder syncing)

[Sabmit]

Hello,

Thanks for this project, it looks great.
What about managing backups the same way you would manage backups for data-container ?
I mean, you could start your container with a volume for /var/mail.

Then, to make a backup, you could :

• Run a new container with the --volumes-from option
• Make a script that will tar, rdiff, send the backup to s3/store it locally)

Here's everything you need to know to implement that : https://docs.docker.com/engine/userguide/dockervolumes/#backup-restore-or-migrate-data-volumes

[Lewiscowles1986]

:+1: That is actually amazing

[tomav]

As proposed by @Sabmit, this is a more general docker related topic.
I don't think we have to build something specific inside the image because a lot of things exist in the open source world.
Probably more a README.md update.

@Sabmit would you have few minutes to do that?

[Sabmit]

Yep @tomav, I'll try to find that time to write something understandable ASAP.
Here's another link explaining what really are data containers (volumes) : http://container-solutions.com/understanding-volumes-docker/

[tomav]

And I add https://github.com/5pi/docker-backup to the discussion :-)

[rubytastic]

Im still investigating the options.
It seems the most simple way for now ( if wanted to use CoreOS ) is making a volume on the CoreOS itself with read and write permissions. This would still be inside a digitalocean droplet for example but the data could be easily exported to external backup.
Then upon recreating a droplet or restoring a backup one would manually need to trigger a restore of this data. It is not ideal but I think best solution?

Using EC2 and other solutions make it utterly complicating.
Or using not CoreOS but since CoreOS is lean and secure I still would prefer it above something like Ubuntu because it has more issues of being hacked into.

[rubytastic]

This

http://www.formatccolon.com/mounting-storage-to-a-coreos-server/

Backup could be a very simple script exporting data to external data source
would love to hear your opinions regarding this

[tomav]

@rubytastic I think that the backup should be simple as possible to let everybody use it.
People using CoreOS, AWS or other particular have skills to plug their own backup solution.
Moreover, it probably has nothing to do with docker-mailserver and should be a separated software.

The short term way is to document how to backup the /var/mail folder (assuming that you have your configuration in your git repository).

Assuming that you /var/mail is a docker data volume container named maildata, you can backup it like that:

docker run --rm \
--volumes-from maildata_1 \
-v "$(pwd)":/backups \
-ti tvial/docker-mailserver \
tar cvzf /backups/docker-mailserver-`date +%y%m%d-%H%M%S`.tgz /var/mail

[tomav]

Added to README.

[Lewiscowles1986]

:+1: Thx

[xyb]

Added to README.

It's no longer in the README. It might be a good idea to put the backup command in the wiki.