Mailcow-dockerized: Migrate from Non-Mailcow installation to Mailcow-Dockerized?

[SoftCreatR]

Hey,

i'm currently working on a server, which is going to be taken off in the near future. On this server, a "standard mail setup" can be found (postfix, dovecot, ...).

On another server, i run mailcow-dockerized and the current plan is to move all mails from the old server to the new one. There's no need to take over any configuration or something. All that is required are the emails which are currently located in /var/vmail/{domain}/{mailbox}/Maildir/ on the old server.

Would that be possible after manually setting-up the mailboxes in mailcow? If yes, how?

[andryyy]

Hi,
We have this covered in the docs. :-)

Though I'm not a fan of messing with the volumes this way: You can copy those mails to /var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/.

[chaosbunker]

Have you taken a look at this guide?

[SoftCreatR]

@chaosbunker

Yes but I wasn't sure if it's useable in my case.

@andryyy

Danke. Ich schaue mal.

[SoftCreatR]

Das scheint funktioniert zu haben. Nachdem ich die Mails portiert habe, habe ich noch ausgeführt:

docker-compose exec dovecot-mailcow doveadm quota recalc -A
docker-compose restart

und alle Mails waren in SOGo sichtbar.

Danke :)

[zeigerpuppy]

The link https://mailcow.github.io/mailcow-dockerized-docs/migrate-mc014/ is no longer available.
I am also looking to migrate about 400GB of emails from a previous maildir format install (dovecot).

I have tried sync jobs and they work well but are quite slow compared to the ZFS replication setup I have between the two servers. What would be the best method to copy the maildirs to the new mailcow server, hopefully with encryption support?

I was thinking something like this:

1. create new accounts in mailcow UI (or perhaps via bulk creation: https://github.com/mailcow/mailcow-dockerized/issues/789 )
2. make sure that maildir format matches (domain/user formatting of folders)
   
   
   
   • write bash script if needed to reformat
   
   
3. copy maildirs into vmail-vol-1
4. trigger encryption script to encrypt the mail files in the new maildirs
5. force an update of the dovecot cache

The main parts I'm not sure how to do are 4 ad 5
p.s. once I have it working, I'd be happy to write a doc on how to do this

[zeigerpuppy]

follow-up:

I think I found stage 4 in the docs

from dovecot container:

# Encrypt /var/vmail
find /var/vmail/ -type f -regextype egrep -regex '/.*[0-9]{10}.+,.+' | while read file; do
if [[ $(head -c7 "$file") != "CRYPTED" ]]; then
doveadm fs put crypt private_key_path=/mail_crypt/ecprivkey.pem:public_key_path=/mail_crypt/ecpubkey.pem:posix:prefix=/ \
  "$file" "$file"
  chmod 600 "$file"
  chown 5000:5000 "$file"
fi
done

[zeigerpuppy]

Here's a working migration script for anyone who is interested....

You WILL need to adjust for your needs but it does most of the heavy lifting as far as getting maildirs into the new install. Please read carefully, it's almost certain that your setup will be a bit different!

#!/bin/bash

# make sure to mount old mail folder first
# after running this script, you need to enter the  `dovecot` container  and
# make sure that your files are owned by vmail, 5000:5000
# if you want encrypted files, then run the encryption script after this one

# mark to dry_run=1 to execute a test but not move any files
dry_run=1

src_folder=/mnt/backups/old-mail-data/
src_maildir=Maildir  # our old server had a folder called Maildir in each user's dir
dest_folder=/dockerdata/mail/vmail-vol-1/
owner=5000

# a list of domains to exclude (for instance if you have migrated them with the sync script)
exclude_domains="test.domain1/,test.domain2/"

echo -e "excluding $exclude_domains\n"

cd ${src_folder}

# iterate over all the folders (domains)
for domain in */; do
  echo "$domain";

  # check if folder (domain) exists on destination AND is not in exclude list
  if [ -d ${dest_folder}${domain} ] && [[ $exclude_domains != *"$domain"* ]]
  then
    echo -e "\t\t domain matches"

    # iterate over users
    cd ${domain}
    for user in */; do
      if [ -d ${dest_folder}${domain}${user} ]
      then
        echo -e "\t\t will copy: ${src_folder}${domain}${user}${src_maildir}/ \n\t\t to ${dest_folder}${domain}${user}"

          if [ $dry_run -eq 1 ]
          then
            echo -e "\t\t dry run"
          else
            echo -e "\t\t do the deed"
            rsync -av -P ${src_folder}${domain}${user}${src_maildir}/ ${dest_folder}${domain}${user}
            chown -R ${owner}:${owner} ${dest_folder}${domain}${user}
            chmod -R a-x ${dest_folder}${domain}${user}
            chmod -R u+rwX ${dest_folder}${domain}${user}
          fi
      fi
    done
    cd ..

  fi
done


echo -e "\n after running this script, you need to enter the docker container and run the encryption script"

[zeigerpuppy]

I have noted an issue with the migration of messages. The number of messages shown in the Nailbox summary in the Mailcow UI appears incorrect after manually copying over messages.
In my case it shoes 1 message instead of 512. The messages appear in SoGo properly, so I guess there must be a message count somewhere that is not getting updated for the UI.

[andryyy]

Yep, that is fixed in 24h or by running docker-compose exec dovecot-mailcow doveadm quota recalc -A. :)

[zeigerpuppy]

great, thanks that works well. Now the last thing to do is to speed up the encrypting, maybe I'll try multi-threading the encrypt script... seems to be the rate limiting step!

[MAGICCC]

I will reopen the issue, since @zeigerpuppy seems so work on this and may document it then :p

[andryyy]

It is different from setup to setup. :/

[zeigerpuppy]

It will vary a bit from one setup to another but will try to keep the scripts very generic and hopefully well described. I agree that it may not necessarily end up in the main docs but just wanted to properly document the steps. I think it will be really useful for people like me who are migrating fairly large previous installs.

[andryyy]

A script will not work for all setups. Some may use maildir, others mdbox etc., some may have a completly different folder structure and so much more.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dawoodjee]

Looking to trying imapCopy

https://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapcopy

[Braintelligence]

@dawoodje What's the benefit over the already available imapsync in Mailcow?

[dawoodjee]

@Braintelligence

I've been looking for a solution. An already integrated (imapsync) solution did not come up in my search. You're the first one to point this out, is there any mailcow documentation for it?

[vooxo]

Hello,

I'm in the same situation - migration from an older cPanel server to the new mailcow-dockerized - but can't make it to work.

I've done the following:

1. copied emails from the old server; directory structure is as described here: https://www.thecpaneladmin.com/understanding-courier-imap-and-maildir/
2. put files to:
3. /var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/
   so that the structure is /var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/example.com/username
4. chown'd everything to 5000:5000
5. ran docker-compose exec dovecot-mailcow doveadm quota recalc -A
6. restarted docker

Question 1:
How do I import password data from /home/username/etc/shadow ?

Question 2:
Where do I get private and public*.pem keys which are required in the encrypt script ?