Mailcow-dockerized: Docker Swarm Compatible docker-compose file

[jesse-osiecki-zz]

I am looking into running this on swarm, which should be straightforward. Mailcow has the init tag all over the compose file and that has been removed in compose v3 causing errors.
Does the mailcow team have plans to make the project deployable on a docker swarm?

[mkuron]

See #619, init is currently being removed.

[jesse-osiecki-zz]

I see that, there may be more steps involved to make the MTA work on swarm. Is there any effort underway to do so? I'd be willing to help if that's not the case

[nightah]

Don't mean to necro this issue but @jesse-osiecki did you have much luck with this?
Would also be great to have an official answer from the devs regarding a swarm compatible docker-compose file.

[mkuron]

There are multiple issues open about supporting different container management platforms (Kubernetes, Rancher etc.). Supporting them all would require major maintenance efforts, so I don’t expect we’ll do it.

Docker Swarm however is an official Docker product and should in theory work with an unchanged docker-compose file. So I think we can support it easily and any issues you find running Mailcow on Swarm should be reported and fixed.

[Dragnucs]

@mkuron Swarm would be supperted indded, but I am curious if setting up some load-balancing or replicas would also work out of the box.

[mkuron]

Load-balancing only works with those services that have no persistent state (Nginx, PHP-FPM, Unbound, ClamAV, Memcached, SOGo, rspamd). For Redis and MySQL you would need to use their built-in replication mechanism. For Postfix and Dovecot, the problem will be that you cannot have multiple instances working on the same queue/vmail storage.

[Dragnucs]

you cannot have multiple instances working on the same queue/vmail storage.

Wouldn't the whole point of distribution not to work on the same queue? Each node should have its own sending, etc. queue separate and provisioned with separate data. But I don't know.

[mkuron]

Wouldn't the whole point of distribution not to work on the same queue?

You'd still need to be able to merge these multiple Postfix queues when removing an instance.

For the mailbox storage with Dovecot, it seems to be possible to have multiple instances operating on the same storage, see https://wiki2.dovecot.org/NFS . It only requires minor config modifications.

[unclesamwk]

Hey guys,

storageos may be an solution for shared persistant swarm storage.

greetings
sam

[mesudip]

I have listed out the options that will be either ignored or not supported when deploying the compose file using docker stack deploy

services:
    dovecot-mailcow:
      cap_add:
        - NET_BIND_SERVICE
    postfix-mailcow:
      cap_add:
        - NET_BIND_SERVICE
    netfilter-mailcow:
      privileged: true
      network_mode: "host"
    ipv6nat-mailcow:
      privileged: true
      network_mode: "host"

I would be nice if you guys can come with a workaround for those options.

I was also wondering if it's important to assign each service a predefined ip, What are the side-effects of allowing docker to chose ip for the containers and netowrk?

[mkuron]

NET_BIND_SERVICE

That allows binding to port numbers < 1024. If things work without it, you don't need it.

privileged: true
network_mode: "host"

That is needed to allow the container to modify host firewall rules. Without it, you lose IPv6, fail2ban and the capability to set a source IP. Fail2ban needs access to your Docker logs and firewall configuration, so it could probably run on your swarm master with some modifications. I have no idea however how networking in Swarm works. Does it somehow NAT the containers or does all incoming traffic go through the userland proxy? In the latter case, you would have an open relay... Similarly, all incoming IPv6 traffic goes through the userland proxy if you don't have the ipv6nat container.

I was also wondering if it's important to assign each service a predefined ip

I think the only place we use a static IP is to connect to the DNS server in the unbound container.

[bd8392]

This would be really nice if we can achieve this =) I can help with testing if you want =). I suggest ceph-storage as you can also store database on it...but first I would test it on a single-node swarm =). And it would be very nice if it is not neccesserry to modify iptables of host, as some protect their vps with custom rules that may get overriden then...look at the approach of mailu.io -> they configured a single container that connects all others and is the only one exposed to the outside...it can contain iptables inside, and you can apply any reverseproxy on it (like traefik).

[mkuron]

it would be very nice if it is not neccesserry to modify iptables of host, as some protect their vps with custom rules that may get overriden then...look at the approach of mailu.io -> they configured a single container that connects all others and is the only one exposed to the outside...it can contain iptables inside

You cannot have iptables rules inside a Docker container. There is no separate kernel running inside and Linux does not support namespacing for iptables. iptables rules are necessary to work around Docker's incomplete IPv6 support and for fail2ban.

[andryyy]

There are other and much better ways to archive a cluster without swarm (which was sold, too, iirc).

You will lose a lot of features. Everything in a single container is, well, what do you call it ... why use Docker then anyway? We _will also not remove very useful security features_ just so swarm is satisfied. Is swarm even able to work with IPv6? No way we drop IPv6 in 2020. :(

Why not build 3 kvm hosts based on your ceph storage to begin with? You keep all functionality and have a redundancy. You can use it for any service, Docker or non-Docker. You can even have redundant Windows machines.

There are other, MUCH better ways to cluster mailcow. Keep Redis in sync, create a SQL cluster (Galera, master/master), replicate Dovecot. You will not need a ceph storage. :)

[mtcaddy]

what's the current status on the docker swarm support as I wish to lunch mailcow on our swarm cluster too?

[bd8392]

Why not build 3 kvm hosts based on your ceph storage to begin with? You keep all functionality and have a redundancy. You can use it for any service, Docker or non-Docker. You can even have redundant Windows machines.

There are other, MUCH better ways to cluster mailcow. Keep Redis in sync, create a SQL cluster (Galera, master/master), replicate Dovecot. You will not need a ceph storage. :)

Cool Ideas thanks :-)

[sebastiansterk]

I would also really like to see a Docker Swarm compatible version of Mailcow.

[timonmasberg]

+1 this would be awesome

[ktechmidas]

There are other and much better ways to archive a cluster without swarm (which was sold, too, iirc).

It was sold, however support is still there and it was sold with the whole "Docker" project, so it's a bit silly to support Docker but not Swarm....

Anyway, the reason people want Swarm support is because the rest of their system usually runs in swarm as well, having a mail server that runs outside sorta defeats the point and makes it difficult to maintain (as then there are two completely separate clusters to maintain!) which just adds technical debt.

There's a lot of people tinkering at home with self-hosted systems who need to learn swarm/k8s/etc and it's good to have friendly software that may not support these things out of the box, but may give a basic "Here's a swarm compose, we can't promise it'll work, and will likely need some readjusting"