Fail2ban: Whitelist outside jail.conf

[binarykitchen]

Hi there

Would be great if I can maintain a whitelist outside of jail.conf. For two reasons: a) I have a very long list of IP addresses, the ignoreip entry would become too long. b) I want to share the same whitelist for other programs like spamassassin (less redundancy).

Thanks,
Michael

[yarikoptic]

See https://github.com/fail2ban/fail2ban/issues/114 for the resolution -- you would be able to maintain jail.d/whitelist.conf which you are welcome to generate or actually use to generate one for sa

I think this would be the only resolution for this, so I should not forget to close this one when I close gh-114

[binarykitchen]

cool thanks. i'll wait a little until these changes are available until the next apt-get update release ...

[yarikoptic]

might take a bit -- waiting on someone to contribute a manpage
describing config files ;-)

On Tue, 19 Feb 2013, Michael Heuberger wrote:

cool thanks. i'll wait a little until these changes are available until
the next apt-get update release ...

Yaroslav O. Halchenko
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: [phone] Fax: [phone]
WWW: http://www.linkedin.com/in/yarik

[buanzo]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Why not just adding those whitelists to a firewall rule with the appropiate precedence?

On 02/19/2013 06:48 PM, Michael Heuberger wrote:

Hi there

Would be great if I can maintain a whitelist outside of jail.conf. For two reasons: a) I have a
very long list of IP addresses, the |ignoreip| entry would become too long. b) I want to share
the same whitelist for other programs like spamassassin (less redundancy).

Thanks, Michael

— Reply to this email directly or view it on GitHub
https://github.com/fail2ban/fail2ban/issues/123.

---

⁂ Arturo "Buanzo" Busleiman ⁂ - MUSICA: soundcloud.com/no-carrier
Independent Linux and Security Consultant - 16+y of IT exp. at your service .
OWASPer - http://www.buanzo.com.ar/pro/eng.html ..:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEAREKAAYFAlEsBgUACgkQAlpOsGhXcE3urgCfcEy2/d+1a/fPm1P+Ox9hQywi
D4sAn19hBgUbR43kh4nxrgd4JMrKFbqe
=MlAI
-----END PGP SIGNATURE-----

[yarikoptic]

On Mon, 25 Feb 2013, buanzo wrote:

Why not just adding those whitelists to a firewall rule with the
appropiate precedence?

might be more cumbersome:

• what is whitelisted in fail2ban might not necessarily mean 'let it
  through unconditionally', so it would still be nice to keep firewall
  rules following 'generic' setup
• fail2ban would still then see those failures and keep banning
  while keeping whining on subsecutive failures since bans were not in
  effect -- fail2ban.log would become polluted, etc

imho it does make sense to have f2b's whitelist, thus sensible to have
it easy to configure for large deployments

Yaroslav O. Halchenko
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: [phone] Fax: [phone]
WWW: http://www.linkedin.com/in/yarik

[buanzo]

Agreed.

f2b external source whitelisting tables +1

On Tue, Feb 26, 2013 at 11:20 AM, Yaroslav Halchenko <
[email protected]> wrote:

On Mon, 25 Feb 2013, buanzo wrote:

Why not just adding those whitelists to a firewall rule with the
appropiate precedence?

might be more cumbersome:

• what is whitelisted in fail2ban might not necessarily mean 'let it
  through unconditionally', so it would still be nice to keep firewall
  rules following 'generic' setup
• fail2ban would still then see those failures and keep banning
  while keeping whining on subsecutive failures since bans were not in
  effect -- fail2ban.log would become polluted, etc

imho it does make sense to have f2b's whitelist, thus sensible to have
it easy to configure for large deployments

Yaroslav O. Halchenko
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: [phone] Fax: [phone]
WWW: http://www.linkedin.com/in/yarik

—

Reply to this email directly or view it on GitHubhttps://github.com/fail2ban/fail2ban/issues/123#issuecomment-14116766
.

[yarikoptic]

ok -- since #114 is now closed, you pretty much should have your "external" way to manage the whitelist, thus closing this issue. Thanks!

[spacecwoboy]

I'm interested in seeing a simple, working example of how to incorporate an external file to the ignoreip whitelist. I've read the howto/READ.me, but didnt quite get the best grip of it.

Let's say I start with this:

• [x] The whitelist of IP Addresses /etc/fail2ban/jail.d/list_of_ipaddresses_to_Whitelist.txt
• 127.0.0.1/32
• some.domainname.us
• 192.168.0.0/16

I need to understand:

• [x] Other configs required to reference "/etc/fail2ban/jail.d/list_of_ipaddresses_to_Whitelist.txt"

[jult]

Yes, me too. How does this work? I have a global list, already in use for other machines and services, it's on the server where fail2ban resides, so it can be referenced using something like
allow-file=/path/dir/ip_ignoredallowed_list
As you seem to have closed this, it should be in config somewhere, but it totally is not.

Perhaps you can copy the code for this from CSF/LFD, who do this exact thing very well. You can even have it autoreload a global remote file for it from a URL.

[sebres]

It is currently not possible via "list_of_ipaddresses_to_whitelist.txt" in plain text form.
But via separated python-config in form of ignoreip = comma-, or space-separated list of IPs resp. DNS.

file jail.local

[INCLUDES]

before = %(known/before)s
              list_of_ipaddresses_to_whitelist.conf

file list_of_ipaddresses_to_whitelist.conf

[DEFAULT]
ignoreip = 127.0.0.1/32 some.domainname.us 192.168.0.0/16

[sshd]
ignoreip = %(known/ignoreip)s another.domainname.us

Corresponding of the source code (function splitwords used for this), you can use newline+space as separator (Note at least one space is expected after new-line, because of the standard syntax of python config-file):

file list_of_ipaddresses_to_whitelist.conf (using newline+space)

[DEFAULT]
ignoreip = 127.0.0.1/32
 some.domainname.us
 192.168.0.0/16
 ...

Also note: extending of this "list" does not affect until fail2ban will be reloaded (resp. restarted in v.0.9).

[jult]

How is this (a python config syntax for a whitelist) progress in any way or form? This puts the user back ~20 years, reminiscent of sendmail files and similar.
I think I'll move to CSF/LFD completely, as far as I can tell LFD can do all fail2ban can, and more. Plus it handles plaintext IP-lists, as mentioned above, with a wide range of accepted syntaxes.

[sebres]

I think I'll move to CSF/LFD completely...

You can use the software you want...

Back to the issue, I don't see it currently as task with heighest prio in the priority-list, sorry.

[mikeg-de]

Searching for a solution which was requested here. The scenario is simple:

1. CDN makes a lot of legit requests which are banned
2. CDN IP's do frequently change
3. Using i.e. the AWS CloudFront an SNS topic could be subscribed to to generate a file

Integrating fail2ban with AWS WAF or EC2 Security Groups may be others options. Though, the first can come with a heavy price tag due to too many rules and the later is quite sophisticated to been implemented.

Having a file which could be referenced is much more straight forward and efficient.

[sebres]

Using i.e. the AWS CloudFront an SNS topic could be subscribed to to generate a file

And which profit it could theoretically make?
Shortly: the IP produces a failure, you would verify the IP against SNS-list, and find 3 related IPs that are known by fail2ban as bad (e. g. were already banned previously).
Then what should happen? The current IP will be banned longer? But as you said "IP's do frequently change"...

One option would be to begin a ban earlier (e. g. by first failed attempt), so the IP does not reach maxretry (normally 3) and banned quasi immediately.
Another would be to ban preventive (for example periodical pull this SNS-list and reban all the IPs from the list, like "switch" old IP to new IP).
Just it depends (and is a bit ugly in my opinion).

First option would be more conceivable and can be relatively simple implemented within 0.11th observer as a kind of "plugin" affected on failure found event to check and increase attempts for such "bad" IPs (to do the ban earlier up to immediately).
For example in my own fail2ban build (not yet public 0.11th continuation) I have similar "plugin" doing that based on GeoIP data.

Integrating fail2ban with AWS WAF or EC2 Security Groups may be others options.

Well one could already now implement many scenarios (e. g. using own actions) to do such integration.
Especially if something like #2304 gets implemented (or if I'll provide my "plugin-API" enhancements for 0.11).

[sebres]

Oh... may be I was confused as regards the searching solution (at least I could read the subject :smile:).
So the comment above describes more the blacklisting of some "bad" intruders often changing IPs.

For the whitelisting of own IPs (in order to ignore them) it could be implemented pretty simply: write your own script validating this SBS-list (or whatever AWS could provide). And add this into jail.local for default or jails sections you need:

[DEFAULT]
ignorecommand = /path/to/command <ip>

The IP is whitelisted if command exits with code 0.
Also note #2176 (since 0.10.4) which allows you to implement caching of ignorecommand for some time (reduces the load and improves performance of fail2ban).

[srulikuk]

Hi @sebres I tried the following but get an error on reload

[INCLUDES]

before = %(known/before)s
              list_of_ipaddresses_to_whitelist.conf

fail2ban-client reload
2020-03-27 08:40:36,907 fail2ban [12337]: ERROR Failed during configuration: Bad value substitution:
section: [INCLUDES]
option : before
key : known/before
rawval :
whitelist.local

Fail2Ban v0.10.5
any ideas?

[srulikuk]

Resolved. in jail.local i put

[INCLUDES]
before = whitelist.local
[DEFAULT]
ignoreip = %(ignoreip)s

and the whitelist.local is as described above

[DEFAULT]
ignoreip = 127.0.0.1/32
 192.168.1.1/32
 10.8.0.0/32