Docker-mailserver: [WIP] Upgrade to Debian 10 Buster

[tomav]

This will allow to get updated software.
I just pushed a WIP branch here: https://github.com/tomav/docker-mailserver/tree/test-debian-buster

A couple of things broke but:

• Build is fixed.
• Spam detection is fixed

There are still error on fail2ban, mail redirects...
Output can be seen here: https://travis-ci.org/tomav/docker-mailserver/builds/562805061

Will try to work on it in the coming days.

[erik-wramner]

@tomav if you want help with some sub-task here please let me know and I can look into it.

[everflux]

Attention: There is no security update in -backports while stretch and buster received updated packages for a possible unauthenticated remote code execution.
Any progress on updating to buster?

[erik-wramner]

@tomav is working on that, but he has been silent for a while so I think he is on vacation.

[tomav]

And I'm back but no time for this currently.
If someone wants to continue, the branch is pushed.

[erik-wramner]

Good news, I finally have a buster release where all tests are working! It is in my repo, I'll merge it back here as soon as we have decided where it should go.

@tomav ideally I would like to release it as a new docker image from another branch (naming proposal next) but we can't edit the docker build. Can you help, please? If not I'll ask people to use my buster image until we are ready to get it into master.

@fbartels and others, we need to test it in the wild before we put it in master. Any volunteers? Ideally with mail servers that are not mission critical as there are bound to be bugs.

[501st-alpha1]

@erik-wramner I volunteer ~as tribute~ to test the new image.

[erik-wramner]

@501st-alpha1 excellent. If you want to build it yourself I have created the new branch "next" in this repo. I have also configured a build in https://hub.docker.com/repository/docker/erikwramner/docker-mailserver tagged buster if you want to use that. Hopefully we will get a next or buster build for our own docker image as well.

[michaeljensen]

What are the remaining tasks for Debian 10 compatibility?

I just switched my mailserver over to use your next branch and might be able to put some time into working on this.

[erik-wramner]

I really don't know, I think the new version works. The problem is that I can't create a separate docker image for it in this repo (@tomav needs to help) so when we switch latest/master it will affect many existing installations at once.

There are changes. In particular filebeat is gone from the container, there is a wiki entry for how to run it in another docker image. Perhaps we should add such a container in the repo as an example to make the migration easier? Ideally I would like someone who was using filebeat test that as well.

Still, I think it is more about deciding on a deadline, documenting the change and then doing it.

[casperklein]

I noticed that the Dovecot repository was commented out from the Dockerfile in the next branch: https://github.com/tomav/docker-mailserver/blob/d03f9c8b8d55e9634b900716073385c1102793e6/Dockerfile#L96-L101

When you build docker-mailserver, depending on the branch you get this versions:

master branch: Dovecot 2.3.10 (from Dovecot repository) - released 2020-03-06.
next branch: Dovecot 2.3.4.1 (from Debian repository) - released 2019-02-05.

Switching from master to the next branch implies a downgrade of Dovecot at the moment. Is that intentional?

[erik-wramner]

Well, yes and no. The move back to the official Debian version was intentional as I noticed we were on the same major version, but the gap from 2.3.10 to 2.3.4 is a bit long. Perhaps we should use the latest Dovecot release after all to avoid downgrade issues. Debian is a bit slow incorporating new versions.

[radicand]

I maintain an ARM fork of this project, for folks using raspberry pi hardware or similar. Unfortunately the dovecot repo does not have support for ARM architecture, but the Debian one does. I was looking forward to the removal of filebeat and migration to Debian for the dovecot component, as it would mean I should be able to build directly against the sources here (or work on a new pipeline here for multiarch builds) and not maintain a separate fork with those changes. It's worth understanding what's actually different between those patch versions before making that change back.

[erik-wramner]

Good point. I don't think there are huge differences, it is certainly worth checking. It is the same major release after all.

[gmasse]

There are changes. In particular filebeat is gone from the container, there is a wiki entry for how to run it in another docker image. Perhaps we should add such a container in the repo as an example to make the migration easier? Ideally I would like someone who was using filebeat test that as well.

@erik-wramner, I can provide some details like Docker-compose file and filebeat basic configuration. Should I create a new wiki page or add a 'beta' chapter in the current ELK section?

I cannot guarantee seamless migration since I am not using the same ELK stack. But I'am eager to help anybody that is currently using it.

[erik-wramner]

Thanks @gmasse. The part about seamless migration is why I would like someone using it to test, perhaps someone steps up? I would suggest putting it in the same section tagged with v7/buster. We can link from the README as well, making it easier to see.

[erik-wramner]

@radicand version 2.3.4 is from October 2018 and there are many versions since then, some with security fixes. We probably want if not the very latest version at least a more recent one. A pity that it takes so long time for updates to get into the official Debian repo.

[radicand]

Even though Debian is slow to update versions, they are decent with backporting security patches. Info here: https://security-tracker.debian.org/tracker/source-package/dovecot it looks like recent vulnerabilities are covered.

[erik-wramner]

Decisions, decisions.... The hard part here is that it is difficult to reach most of our users, otherwise I would suggest a poll. If the security fixes are in I would personally vote for the supported Debian version. @fbartels? @tomav?

[michaeljensen]

I concur that Debian does a good job with backporting security fixes. My vote would be to use the Debian supported version.
(my opinion is also based on many years of sysadmin experience using both compiled and distro-based packages, learning that distro-based packages make my workload much lighter.)

[erik-wramner]

I added an announcement to the readme. I suggest that we plan for releasing next by the end of this month. At that point the current latest becomes the new stable. I will add the current changes from master into next. A pity I can't rebase it, but I guess that will break the forks.

[tomav]

@erik-wramner what's you docker hub username?

[tomav]

My vote would go as well to Debian stable/supported version + security backport.

[tomav]

Just found your username in another message. You can now configure Docker hub.

[gmasse]

Thanks @gmasse. The part about seamless migration is why I would like someone using it to test, perhaps someone steps up? I would suggest putting it in the same section tagged with v7/buster. We can link from the README as well, making it easier to see.

PR #1434 to review. I will now work on the wiki part.

[erik-wramner]

Thanks @tomav. I still can't select "manage the repository" to configure auto-builds, but perhaps I can push manually? I'll give it a try.

[gmasse]

PR #1434 to review. I will now work on the wiki part.

Wiki done
https://github.com/tomav/docker-mailserver/wiki/Configure-ELK

[tomav]

Just created next tag in Docker Hub. Currently building.
https://hub.docker.com/repository/registry-1.docker.io/tvial/docker-mailserver/builds/90705309-b8dd-41a6-b402-d40a857d6978

[tomav]

SUCCESS. 🎉