Fail2ban: Having trouble setting up fail2ban for dovecot login/auth

[eNTi]

• Fail2Ban version fail2ban/bionic,bionic,now 0.10.2-2 all [installed]
• Linux divzero.at 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Ubuntu 18.04 LTS
• [x] Fail2Ban installed via OS/distribution mechanisms
• [x] You have not applied any additional foreign patches to the codebase
• [x] Some customizations were done to the configuration (provide details below is so)

The issue:

Hi,
I'm trying to ban unsuccessful auth / login tries. There's some tutorials/howtos but it seems they are using some kind of outdated regex?

/etc/fail2ban # fail2ban-regex /var/log/mail.log /etc/fail2ban/filter.d/dovecot-pop3imap.conf

Running tests
=============
Use   failregex filter file : dovecot-pop3imap, basedir: /etc/fail2ban
ERROR: No failure-id group in '(?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*'

I was looking at https://wiki.dovecot.org/HowTo/Fail2Ban and also https://www.fail2ban.org/wiki/index.php/Dovecot.

The example in the 2nd page doesn't throw an error but I'm also not sure if it does what I actually want it to do.

Steps to reproduce

Create dovecot-pop3imap.conf as discribed in the dovecot wiki

Expected behavior

Banning failed login/auth tries.

Observed behavior

Regex not working.

Any additional information

I've upgraded ubuntu server yesterday and it installed a newer version of fail2ban. Since then the custom configuration no longer works.

Configuration, dump and another helpful excerpts

Any customizations done to /etc/fail2ban/ configuration

Relevant parts of /var/log/fail2ban.log file:

_preferably obtained while running fail2ban with loglevel = 4_

2018-05-16 14:20:09,136 fail2ban.filter         [31796]: ERROR   No failure-id group in '(?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*'
2018-05-16 14:20:09,136 fail2ban.transmitter    [31796]: WARNING Command ['set', 'dovecot-pop3imap', 'addfailregex', '(?: pop3-login|imap-login): (?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed).*rip=(?P<host>\\S*),.*'] has failed. Received Rege
xException("No failure-id group in '(?: pop3-login|imap-login): (?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed).*rip=(?P<host>\\S*),.*'",)
2018-05-16 14:20:09,137 fail2ban                [31796]: ERROR   NOK: ("No failure-id group in '(?: pop3-login|imap-login): (?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed).*rip=(?P<host>\\S*),.*'",)

Relevant lines from monitored log files in question:

[sebres]

Value of (?P<host>\S*) is not valid anymore (since IPv6-support in 0.10, newest version of fail2ban has other handling and regexp for DNS/IPv4/IPv6, because differentiate between this groups during matching).
So you should use <HOST> (cumulate all forms), <ADDR> (ips only) or <DNS> (dns-hosts only) instead of.

BTW. Interpolation <HOST> is an "alias" for following regex:

(?:(?:::f{4,6}:)?(?P<ip4>(?:\\d{1,3}\\.){3}\\d{1,3})|\\[?(?P<ip6>(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}|(?<=:):))\\]?|(?P<dns>[\\w\\-.^_]*\\w))

[bgcomrade]

Where is DISLIKE button. After update ubuntu it comment same issue:

fail2ban.filter         [8151]: ERROR   No failure-id group in '(?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed|Aborted login \(\d+ authentication attempts).*rip=(?P<host>\S*),.*'
2018-08-15 14:19:18,200 fail2ban.transmitter    [8151]: WARNING Command ['multi-set', 'dovecot-pop3imap', 'addfailregex', ['(?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed|Aborted login \\(\\d+ authentication attempts).*rip=(?P<host>\\S*),.*', '.*(?:pop3-login|imap-login):.*(?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed|Aborted login \\(tried to use disallowed plaintext auth).*\\s+rip=(?P<host>\\S*),.*', 'pam.*dovecot.*(?:authentication failure).*\\s+rhost=<HOST>(?:\\s+user=.*)?\\s*$']] has failed. Received RegexException("No failure-id group in '(?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed|Aborted login \\(\\d+ authentication attempts).*rip=(?P<host>\\S*),.*'",)
2018-08-15 14:19:18,200 fail2ban                [8151]: ERROR   NOK: ("No failure-id group in '(?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \\(auth failed|Aborted login \\(tried to use disabled|Disconnected \\(auth failed|Aborted login \\(\\d+ authentication attempts).*rip=(?P<host>\\S*),.*'",)

And developer sebres comment this;

(?:(?:::f{4,6}:)?(?P<ip4>(?:\\d{1,3}\\.){3}\\d{1,3})|\\[?(?P<ip6>(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}|(?<=:):))\\]?|(?P<dns>[\\w\\-.^_]*\\w))

So is anybody understand how to repair it, becouse we didn't understand developer sebres. When bad developers WILL STOP do things like that " (?:(?:::f{4,6}:)?(?P(?:\d{1,3}\.){3}\d{1,3})|\[?(?P(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}|(?<=:):))\]?|(?P[\w\-.^_]*\w)) "

And give SOLUTIONS. 100 % decents manuals that works. sebres tomorrow you have to go to police, someone has see you has smoke marijuana and you go to jail. Question is why sebres is going in jail, becouse it is the same: No 100 % decent explanaition.

But difference is here that sebres will complaint in police and PEOPLES here will not complaint BAD developers like sebres. I vote peoples make database for BAD developers, everybody will see everyday bad developes is high. So there is no difference sebres is going in police jail or peoples are complaining. So sebres has time to put:
So you should use (cumulate all forms), (ips only) or (dns-hosts only) instead of. and after that misguided: BTW. Interpolation is an "alias" for following regex:

Examples: how much time it will take sebres to write EXAMPLES:

1 min. compared to sebres has write fail2ban code. And remember if you fail to solve dovecot-pop3imap it will flood your syslog.

[sebres]

do you see the "DISLIKE" now...

And developer sebres comment this...

Nope, "developer sebres" commented - use token <HOST> instead of (?P<host>\S*).

The regex (?P<host>\S*) was anyway an undocumented feature (and from the times as you have been still not born)... Nevertheless, if some undocumented feature had operated previously, it does not meant, that the same will work with newest version, especially if changelog says it heavy bold at the top:
https://github.com/fail2ban/fail2ban/blob/e2a255d104f947f149cc34b17e778c05175e9f78/ChangeLog#L9-L22

Still again:

• (?P<host>\S*) - NO.
• <HOST>, <ADDR>, <DNS> - YES.

Just because fail2ban since 0.10 should differentiate between IPv4 and IPv6, that's why it is rewritten.
Therefore the proper regex-based replacement for (?P<host>\S*) is more complex.

Additionally to the common tag <HOST> there are many other tags possible now - <ADDR>, <IP4>, <IP6> as well as DNS.

If you want to use something other as IP for the failure-ID see wiki :: How to ban something other as host (IP address), like user or mail, etc.

how much time it will take sebres to write EXAMPLES:

Too many (because for free). But he (and many others) already wrote enough examples (see wiki above), docu and changelog entries (that nobody reads).

And last but not least - try to learn english, the you'll be able to understand the answer in the first comment:

• you should use <HOST> (cumulate all forms), <ADDR> (ips only) or <DNS> (dns-hosts only) instead of.

Comprendo?

[sebres]

sebres some day human person will come and will rewrite your program

1. it is not mine
2. you can begin right now...
3. discussion closed.