Mailu: Properly configure Fail2Ban

[iohenkies]

Hi,

Maybe this is not the correct place and should I instead ask this at Fail2Ban, but I would like to try and explain anyway.

After a fresh installation the last couple of days, I was getting the occasional authentication problem, where I had to put in my password for all accounts. Finally I was able to pinpoint it, at least for the webmail, to an

"Authentication rate limit from one source exceeded" while in http auth state

error. In the mailu issues I found several topics with the same problems and in one of them the tip to up the AUTH_RATELIMIT in the mailu.env. This seems to solve the problem, but opens me up to another issue: increased brute force attempts.

I already had investigated this as well, and found information on Fail2Ban in your FAQ and several Github issues. Eventually I installed and configured it on the host as described here: https://github.com/Mailu/Mailu/issues/584#issuecomment-433702594

This blacklists the IPs on the host, as seen with

fail2ban-client status bad-auth

but the attempts keep happening in the containers. This is on a single host, single (virtual) NIC and IP. What am I doing wrong?

[micw]

Just a guess: fail2ban creates rules to prevent from hiting packets at the host. Docker uses iptable's DNAT, so the packets don't go to the host but to the container. So it might be an iptables issue.

[kaiyou]

Also, please consider that we plan on solving the authentication issue this year, pretty much when any of us has the time to code it :)

[iohenkies]

@micw Yes that is what I guessed as well, but the big question is: how to solve it? :) Any ideas? The steps from the comment I posted seemed rock solid (IPs immediately got banned) but elas...

@kaiyou With the larger AUTH_RATELIMIT, for me the authentication problems, seems solved.

[kaiyou]

It is not really solved if it creates other issues, like allowing some form of brute force because of high limits.

[iohenkies]

Yes, you are absolutely correct. But it got from unworkable to absolutely usable for me so I'm happy so far ;)

I will look into the Fail2Ban config more but am still secretly hoping other Mailu users have come across this as well.

[iohenkies]

Very glad to have found the solution (scattered around the interwebs).

1. In the mailu docker-compose set the logging driver of the front container to journald

    logging:
      driver: journald

1. Create the file /etc/fail2ban/filter.d/bad-auth.conf

# Fail2Ban configuration file
[Definition]
failregex = .* client login failed: .+ client:\ <HOST>
ignoreregex =

1. Add the /etc/fail2ban/jail.d/bad-auth.conf

[bad-auth]
enabled = true
filter = bad-auth
logpath = /var/log/messages
bantime = 604800
findtime = 300
maxretry = 5
action = docker-action

1. Add the /etc/fail2ban/action.d/docker-action.conf

[Definition]

actionstart = iptables -N f2b-bad-auth
              iptables -A f2b-bad-auth -j RETURN
              iptables -I FORWARD -p tcp -m multiport --dports 1:1024 -j f2b-bad-auth

actionstop = iptables -D FORWARD -p tcp -m multiport --dports 1:1024 -j f2b-bad-auth
             iptables -F f2b-bad-auth
             iptables -X f2b-bad-auth

actioncheck = iptables -n -L FORWARD | grep -q 'f2b-bad-auth[ \t]'

actionban = iptables -I f2b-bad-auth 1 -s <ip> -j DROP

actionunban = iptables -D f2b-bad-auth -s <ip> -j DROP

Restart fail2ban. Hope this helps.

[hoellen]

I'm happy, that you could figure it out. Maybe you want contribute it to our documentation?

[iohenkies]

I'm not too familiar with pull requests etc. but I tried to edit it, hope it comes through. Also, the editor is quite getting used to, I couldn't figure out how to add an 'Issue reference' to this ticket, so maybe you can add it.

[sholl]

thanks @iohenkies for the description, I had to set the /var/log/syslog as logpath in /etc/fail2ban/jail.d/bad-auth-conf though.

[micw]

Closing this in favor to #1269

[micw]

Oops, #1269 was a PR, thought it was an issue ;)

[Tin-Bui]

Thanks @iohenkies for the comment: https://github.com/Mailu/Mailu/issues/1263#issuecomment-557227298
and @sholl at: https://github.com/Mailu/Mailu/issues/1263#issuecomment-578069864
I looked up a fail2ban issues in general and substitute the "logpath" config entry with:
backend = systemd
That worked for me.

[Simonmicro]

@iohenkies

1. Add the /etc/fail2ban/action.d/docker-action.conf

[Definition]

actionstart = iptables -N f2b-bad-auth
              iptables -A f2b-bad-auth -j RETURN
              iptables -I FORWARD -p tcp -m multiport --dports 1:1024 -j f2b-bad-auth

actionstop = iptables -D FORWARD -p tcp -m multiport --dports 1:1024 -j f2b-bad-auth
             iptables -F f2b-bad-auth
             iptables -X f2b-bad-auth

actioncheck = iptables -n -L FORWARD | grep -q 'f2b-bad-auth[ \t]'

actionban = iptables -I f2b-bad-auth 1 -s <ip> -j DROP

actionunban = iptables -D f2b-bad-auth -s <ip> -j DROP

Restart fail2ban. Hope this helps.

Well, you've got an problem in your solution there - as I am lazy, I'll just cite myself from my own website:

When you want to use fail2ban with Docker you MUST use an other approach: Because the default way for the docker-action.conf is to create an own chain and insert the blocking rules there. This is fine when you are using normal
applications - but Docker will may or may not recreate its rules on its own. This causes your fail2bain rules to shift down in the hierarchy and therefore will get ignored. You can test that by simply restarting your container and then
watching the efficiency of your newly installed fail2ban rules: They'll get ignored. To make that all work you have to use an other docker-action.conf like this:

[Definition]
actioncheck = iptables -n -L DOCKER-USER | grep -q 'DOCKER-USER[ \t]'

actionban = iptables -I DOCKER-USER -s <ip> -j DROP

actionunban = iptables -D DOCKER-USER -s <ip> -j DROP

As you may note there are no actionstart and actionstop - as we now use the iptables chain DOCKER-USER, which is specially designed to be used for stuff like this! This chain is guaranteed to be evaluated _before_ any further internal
networking for Docker. When you fail to do this you will may note that your rukes are ignored randomly or when you restart your containers!

[ofthesun9]

@Simonmicro thanks for your comment. I am currently using your docker-action.conf and this is working ok (here ubuntu 20.04)

[Simonmicro]

@ofthesun9 Interesting. I am currently using Debian 10 and noticed today that the iptables does not seem to work anymore. Then I dived into that topic and discovered that the config file from here is not correctly setup. Are you able to disengage the rules by docker-compose down and docker-compose up -d your containers? What Docker version / Kernel version are you running?

[ofthesun9]

docker version: 19.03.8
kernel: 5.4.0-52-generic (x86_64 ubuntu)

I checked that the connection attempts were detected and iptables -L DOCKER-USER was filled accordingly
The rules survived after docker-compose down & docker-compose up -d, which I assume is the normal behaviour.
I have not checked whether the rules would survive after a docker restart. (systemctl restart docker)

[Simonmicro]

docker version: 19.03.8
kernel: 5.4.0-52-generic (x86_64 ubuntu)

I checked that the connection attempts were detected and iptables -L DOCKER-USER was filled accordingly
The rules survived after docker-compose down & docker-compose up -d, which I assume is the normal behaviour.
I have not checked whether the rules would survive after a docker restart. (systemctl restart docker)

Strange! When I checked my iptables after using the config from above a new chain f2b-... was created and DOCKER-USER only contained the default rule - therefore triggering my mentioned problem. Strange!

_Maybe Ubuntu has already workaround-configs for that case added?_

[ofthesun9]

@Simonmicro I think there is a misunderstanding :-) I am using the configuration you have provided:

[Definition]
actioncheck = iptables -n -L DOCKER-USER | grep -q 'DOCKER-USER[ \t]'
actionban = iptables -I DOCKER-USER -s <ip> -j DROP
actionunban = iptables -D DOCKER-USER -s <ip> -j DROP

[Simonmicro]

Well... Somehow I mixed you with the OP. What a day...

Sry 😆