Fail2ban: Nextcloud Talk - COTURN log - no timestamp

[xtrmbuster]

Environment:

Here is the snippet of log file generated by latest version of COTURN application.

Running on Ubuntu 20.04LTS and also on Debian 10 x64

fail2ban-regex --version => 0.10.2

The log example :

I'm testing on Debian now but my goal is to manage to match this failing authentication logs on 4-5 lines :

27307: YTUTUyt tyututut
27313: azeaeaeae
27369: handle_udp_packet: New UDP endpoint: local addr 1.2.3.4:3478, remote addr 9.9.9.9:51622
27369: handle_turn_command: STUN method 0x1 ignored
27369: session 001000000000000011: realm <hostname.online> user <>: incoming packet message processed, error 401: Unauthorized
27369: check_stun_auth: Cannot find credentials of user <1595978236:test_user>
27369: session 001000000000000011: realm <hostname.online> user <1595978236:test_user>: incoming packet message processed, error 401: Unauthorized
28402: ZAEAE

The issue / Steps to reproduce :

So I used this command to test :

fail2ban-regex logfile.txt regex.txt -l debug

Got this error message

W: Found a match for '27369: handle_udp_packet: New UDP endpoint: local addr 1.2.3.4:78, remote addr 9.9.9.9:51622\n27369: handle_turn_command: STUN method 0x1 ignored\n27369: session 001000000000000011: realm <hostname.online> user <>: incoming packet message processed, error 401: Unauthorized\n27369: check_stun_auth: Cannot find credentials of user <1595978236:test_user>\n27369: session 001000000000000011: realm <hostname.online> user <1595978236:test_user>: incoming packet message processed, error 401: Unauthorized'
but no valid date/time found for '27369: session 001000000000000011: realm <hostname.online> user <1595978236:test_user>: incoming packet message processed, error 401: Unauthorized'.
Please try setting a custom date pattern (see man page jail.conf(5)).
If format is complex, please file a detailed issue on https://github.com/fail2ban/fail2ban/issues in order to get support for this format.

After several days of research I concluded that it was because there is no timestamp recognized by fail2ban in each line of the log.

The first numbers are the "timestamp" but it's not a real timestamp.

Is there any trick to bypass this condition or a workaround ?

My regex file :

[Init]
maxlines = 4

[Definition]

failregex = "[0-9]+: handle_udp_packet: New UDP endpoint: local addr (?:[0-9]{1,3}\.){3}[0-9]{1,3}:[0-9]+, remote addr <HOST>:[0-9]+\n[0-9]+: handle_turn_command: STUN method 0x1 ignored\n[0-9]+: session [0-9]+: realm <.*> user <.*>: incoming packet message processed, error 401: Unauthorized\n[0-9]+: check_stun_auth: Cannot find credentials of user <.*>\n[0-9]+: session [0-9]+: realm <.*> user <.*>: incoming packet message processed, error 401: Unauthorized"

Expected behavior

The goal is the succeed to match the log lines when it fails
The final goal is to ban after X time failure - I know it will be set in another file, but just giving a context.

Thank you so much for your help

[sebres]

Unfortunately at the moment it is impossible to monitor log-file (not a journal) without any datepattern.
I have some experimental branch introducing this possibility for file-related backends, but it is neither public yet, nor released.
I'll try to provide it in the near future.

Right now only one case where it would work is to log to systemd-journal (and monitor it with backend systemd).

maxlines = 4

4 lines buffer can be too small - just imagine some parasite log-entries (an intruder may also produce) occur between line with IP and 401: Unauthorized.
Let alone this multi-line handling is obsolete, it is not recommended at all.

If you use fail2ban >= 0.10 and a number 27369 (that you called "timestamp") is some connection ID or can be used as it (unique for some long time interval), you could use new handling using tag <F-MLFID> (see #1698).
It is fewer vulnerable by timing issues and additionally much more faster.

[xtrmbuster]

Thanks for your reply so fast !
Hope it will be a feature request

[sebres]

at the moment it is impossible to monitor log-file (not a journal) without any datepattern.
I have some experimental branch introducing this possibility for file-related backends, but it is neither public yet, nor released.
I'll try to provide it in the near future.

2814 implements this as part of PR. special datepattern {NONE} would allow to find failures without date-time in log messages (filter uses now as timestamp):

datepattern = {NONE}