Mailcow-dockerized: Single Container Distribution

Created on 28 Jul 2017 · 7Comments · Source: mailcow/mailcow-dockerized

In the past months that I have been running mailcow-dockerized in production in a cattle environment, it has become increasingly appearant for our ops that deploying mailcow through anything but the vanilla docker-compose configuration is a pain. Because of the the strongly interdependant multi-container architecture that mailcow-dockerized is currently taking, any change to the upstream code basically demands for a full manual reconfiguration of all the docker containers. Additionally the interdependance of the containers of each other regularly leads to deadlocks upon restart of individual containers and thus service outages.

I would like to start a discussion on engineering a single-container mailcow-dockerized distribution inspired by the gitlab-omnibus distribution. It is a single docker image containing all services needed to deploy the service and thus much less simpler to upgrade and operate. I am fully aware of the drawbacks coming with a single container distribution but am convinced, that the benefits clearly outweight the drawbacks for small and medium-size deployments.

Looking forward to your feedback!

Source

lgleim

👎3

Most helpful comment

Like I said in my initial post I agree with the pricipal superiority of a multi-container distribution. The one app per container paradigm is preferrable to a single-container distribution, as it allows

to scale each application independently
to upgrade and build containers independently
for much more granular setting of container limits and permissions
for easier monitoring of the individual services.

Nevertheless:

Docker compose is not really well suited for deployment of applications in cloud environments, as it (to my knowledge) does not play well with any of the relevant container orchestration tools (Kubernetes, Mesos, Cattle).
The current compose file defines it entire own network with its own dns service etc. which does not fit in with common software defined network architectures used in the cloud
From what I have seen from skimming over the code, some of the interdependancies of the containers are only ensured by sleep times in the startup scripts, which has lead to deadlocks on our systems when restarting individual containers.
Scaling the individual services is not actually possible as is as they are not configured for sharding/clustering, so that this potential benefit of "one app per container" becomes irrelevant.

So the current docker-compose works great if you want to start the service on an an individual machine but not so well in the cloud. That being said, a single container distribution would not fix this issue. However it would make it possible to easily deploy Mailcow in a Kubernetes, Mesos or Cattle environment.

An alternative would be to provide up to date templates/definitions for kubernetes helm, rancher catalog and whatever the equivalent for Mesos is.

Please do not take this as criticism of the project but as an experience report from trying to operate mailcow-dockerized in production using orchestration tools.

lgleim on 28 Jul 2017

👍4

All 7 comments

That's not what Docker is made for. One app per container (more or less) is much much more preferable. There is no advantage of a single container installation vs multi container. It is also a totally different concept.

I also don't see a problem with Docker Compose. If anything it helps to configure multiple containers with ease. It takes me 5 minutes including the installation of Docker and Compose to start a mailcow.
You can define resources, switch app versions etc.

For a single container installation LXC or systemd-nspawn should be used. Docker is not meant for this.

Let us know how we can help to deploy mailcow in your environment. :-)

Am 28.07.2017 um 13:36 schrieb Lars Gleim notifications@github.com:

In the past months that I have been running mailcow-dockerized in production in a cattle environment, it has become increasingly appearant for our ops that deploying mailcow through anything but the vanilla docker-compose configuration is a pain. Because of the the strongly interdependant multi-container architecture that mailcow-dockerized is currently taking, any change to the upstream code basically demands for a full manual reconfiguration of all the docker containers. Additionally the interdependance of the containers of each other regularly leads to deadlocks upon restart of individual containers and thus service outages.

I would like to start a discussion on engineering a single-container mailcow-dockerized distribution inspired by the gitlab-omnibus distribution. It is a single docker image containing all services needed to deploy the service and thus much less simpler to upgrade and operate. I am fully aware of the drawbacks coming with a single container distribution but am convinced, that the benefits clearly outweight the drawbacks for small and medium-size deployments.

Looking forward to your feedback!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

andryyy on 28 Jul 2017

👍4

to scale each application independently
to upgrade and build containers independently
for much more granular setting of container limits and permissions
for easier monitoring of the individual services.

Nevertheless:

Docker compose is not really well suited for deployment of applications in cloud environments, as it (to my knowledge) does not play well with any of the relevant container orchestration tools (Kubernetes, Mesos, Cattle).
The current compose file defines it entire own network with its own dns service etc. which does not fit in with common software defined network architectures used in the cloud
From what I have seen from skimming over the code, some of the interdependancies of the containers are only ensured by sleep times in the startup scripts, which has lead to deadlocks on our systems when restarting individual containers.
Scaling the individual services is not actually possible as is as they are not configured for sharding/clustering, so that this potential benefit of "one app per container" becomes irrelevant.

An alternative would be to provide up to date templates/definitions for kubernetes helm, rancher catalog and whatever the equivalent for Mesos is.

Please do not take this as criticism of the project but as an experience report from trying to operate mailcow-dockerized in production using orchestration tools.

lgleim on 28 Jul 2017

👍4

This is related to #196.
A specific rancher branch (or even better: rancher template) of mailcow would ease this up. With rancher you would need to work with rancher-compose to make the network work like you want it to, I guess.

Braintelligence on 30 Jul 2017

From what I have seen from skimming over the code, some of the interdependancies of the containers are only ensured by sleep times in the startup scripts, which has lead to deadlocks on our systems when restarting individual containers.

They are pretty much useless now. Rspamd had trouble re-discovering disappeared maps. That was fixed.

I try to make it swarm-enabled in the future. Configuration files were a show-stopper, but Docker just recently added a feature to distribute them over swarm! See https://docs.docker.com/engine/swarm/configs/#about-configs

A dedicated mailcow network helps to not interfere with existing containers.

I also prefer to code mailcow for personal use. If I find time for some cloud-enabled features, I will gladly implement them. But I don't know _if_ and _when_, as this is an open-source project I don't benefit from.

andryyy on 30 Jul 2017

👍1

https://github.com/phusion/baseimage-docker could be used to adopt the strategy @lgleim has proposed. I'm inclined to agree, there is a much higher level of complexity when deploying applications using an orchestration platform. Projects such as this do not adapt very well to n number of nodes. By no means is this a criticism, projects like gitlab-omnibus exist precisely because once you start trying to coordinate redis, postgres, workers, nginx, registry, etc. things get complex _really_ fast.

@lgleim give https://github.com/tomav/docker-mailserver a look. It takes the single-container approach though I don't think it has some of the user-friendly features of mailcow. Also, I couldn't figure out how to deal with SPF in Swarm (connections arriving via Ingress had the internal network IP rather than the senders, resulting in mail being rejected).

notsureifkevin on 22 Aug 2017

I know that is an old thread but I would also like a single-container distribution with env variables to configure. It would take out much complexity when deploying to swarm and you can ensure that you don't run an open-relay easy...