Icinga2: restrict icinga daemon/port access by src-ip via industry standard mechanisms

[devZer0]

i am missing some feature like "hosts_allow" to restrict which host can open a tcp session to the icinga2 daemons listener port.

since icinga is implementing openssl/libssl shared lib, we should have learned from the past that this was vulnerable to different exploits, most notably "heartbleed", see:
https://www.cvedetails.com/product/383/Openssl-Openssl.html?vendor_id=217

with tcp-wrappers/librwap ( https://en.wikipedia.org/wiki/TCP_Wrappers ) or some internal "hosts_allow" mechanism, you could have a much more "secure out of the box" design. so, with icinga agent installation/rollout to hundreds of hosts , you could restrict access to the daemon for !=$monitoring_server without touching firewall configuration at all (which is way less complex)

It's way better to keep the evil people away at the tcp level instead of allowing them to "knock the door" at the ssl level (i.e. disallowing them to check the "door" for weakness)

yes, we can do that at the firewall/iptalbes level, but you need to make sure the firewall is active and firewall configuration is much more comlicated and error-prone than having a simple "deny_all, allow_from" directive inside the daemons config file(s).

also see:

https://github.com/NagiosEnterprises/nrpe/commit/381beecbaf94f4a8d56eea9a3862aacfffb3a512

• Applied Kaspersky Labs supplied patch for extending allowed_hosts (Konstantin Malov)

https://github.com/NagiosEnterprises/nrpe/commit/319ebd6d9d010d8cefd4f936f52640abc7db5c0d

there is a reason why kaspersky labs provided those patches to nrpe in 2006

[dnsmichi]

I don't think that re-inventing the wheel adds another layer of security here. Instead, it adds an additional burden for developers and always will introduce bugs and problems.

Looking at your proposal with an allowed hosts list, this doesn't prevent someone using an IP spoofing attack, which I consider very common these days. Firewalls and additional layers upfront the application provide detection and mitigation possibilities.

The linked patches from NRPE may sound reasonable, but they just hide the problem that the TLS implementation in NRPE is vulnerable to many more attacks, i.e. with not checking the CN during TLS handshake. You can find a discussion on that topic in our community forums. Btw, security is not a first class citizen in NRPE, you need enable this on your own based on a sparse documentation.

Also, not having a firewall or DMZ for any sorts of service exposed other than localhost is something I cannot recommend in modern times.

That being said, security is taken serious with Icinga, and possible exploits are treated responsibly. You can learn more about this here. In case you are interested in the technical details for securing the cluster messages, check the technical concepts chapter in the docs.

Cheers,
Michael

[devZer0]

i can understand your point, but please rethink that it is quite common practise to have ip acl at daemon level. yes, you can spoof ip, but you most likely you need to be on the local lan for it.

we have development machines here inside our companies network where we cannot run local firewall for different reason.

we do not feel comfortable to rollout icinga agent on every machine without being able to restrict who can connect to that, as it would reduce security.

we know that libssl had multiple severe security issues in the past, and now we are rolling out icinga2 to 100+ hosts where multiple of them have no local firewall.

it does not feel good to rollout a service which may introduce new security issues.

the guy who's rolling out the icinga agent told that it be safe and that he had restricted access at the firewall level. the first servers i checked had no access restriction at all because the install script did wrong... 1st hurdle for security: fail

if you had your own house in a really dangerous neighborhood - what wold you do when you cannot built a fence (=firewall) around your house or if you do not trust your fence ? i'm sure you would introduce steel lattice (= ip acl) to your doors and windows.

keeping someone away from trying tls handshake at all will prevent any attack at the ssl layer, which is a very complex one. checking source ip for validity is simple and very effective.

there is no "sufficient security", security is always a product of the individual security measures and which measure is the possible or optimal one may heavily depend on the environment

we want to keep our systems "safe by design" , at least we do not want to add additional risk when adding monitoring to them

the networked world is a dangerous place to be....you can't be paranoid enough...

[devZer0]

https://bugzilla.redhat.com/show_bug.cgi?id=1495181

 R P Herrold 2017-11-29 20:01:11 UTC

I see:
 Jan Kurik 2017-09-25 08:26:39 EDT
Whiteboard: ChangeAcceptedF28, SystemWideChange

At what meeting was this change 'Accepted'?  a pointer to a Fedora entity issue tracker of mailing list archive, or minutes is requested

Certainly in the mailing lists, this question seems unanswered


Date: Wed, 13 Sep 2017 06:15:39
From: Neal Gompa <[email protected]>
Reply-To: Development discussions related to Fedora <[email protected]>
To: Development discussions related to Fedora <[email protected]>
Subject: f-dev] Re: F28 System Wide Change: Deprecate TCP wrappers
...


So, I'm a comaintainer of a package that uses libwrap and such
(stunnel), and I don't particularly want to lose the tcp_wrappers
support in it, because I use stunnel in containers to set up secure
tunnels across a number of systems. Unlike firewall rules (which apply
globally to the host), the hosts.deny rules apply only within the
container, which is the behavior I want.

Also, your recommended alternative of using tcpd doesn't work if the
package containing it is gone (tcp_wrappers).

======================

similarly, we use hostname based denials (against a DNS we control) to restrict access to some servers [every interior host is allocated by MAC address, and switch and VLAN filtering, and has a PTR and forward A record]

There is no way to attain this in the firewalld, nor in a tcpd without wrappers, of which I know

Thank you

-- Russ herrold

[devZer0]

https://hyperhci.com/2019/07/27/integrate-tcp-wrapper-into-nutanix-security/

[devZer0]

http://rogerprice.org/hosts.allow/#CONCLUSION

[dnsmichi]

the guy who's rolling out the icinga agent told that it be safe and that he had restricted access at the firewall level. the first servers i checked had no access restriction at all because the install script did wrong... 1st hurdle for security: fail

Should that justify that an application should take care of human failure? If so, I truly recommend to rethink your deployment strategy for servers. Modern tools like Puppet, Ansible, etc. ensure that specific config and services are running, even if the user fails to properly setup this.

we have development machines here inside our companies network where we cannot run local firewall for different reason.

Interesting. Why?

we know that libssl had multiple severe security issues in the past

All of them have been patched by upstream vendors, and updated systems are safe and secure. Question remains, what layer of security could you implement without using OpenSSL/TLS (and variant implementations)?

quite common practise to have ip acl at daemon level.

Not sure who told you that, or where you did read that, but I can tell you - no it is not.

there is no "sufficient security", security is always a product of the individual security measures and which measure is the possible or optimal one may heavily depend on the environment

Right, and there are many ways you can ensure that this happens.

we do not feel comfortable to rollout icinga agent on every machine without being able to restrict who can connect to that, as it would reduce security.

Considering that you may do a TLS handshake and try to exploit that, keep in mind that OpenSSL are typically patched by upstream vendors in zero time. If you as an administrator do not take care about applying these security updates, this is another problem layer.

we want to keep our systems "safe by design" , at least we do not want to add additional risk when adding monitoring to them

So you're saying that your "safe by design" strategy does not involve any sorts of firewalls and additional security layers? Well, then you really don't understand the zero trust principle.

Besides, you're exposing that your infrastructure is open for possible attack vectors. If someone now does some social engineering and gathers insights about your employer, who knows what is supposed to happen. I can delete this issue if you prefer this information not to be public.

[devZer0]

ok, i see we will not get to an agreement, as we have very different point of view.

but let me explain a last time and then please stop this thread:

i simply think it's much much easier and safer to configure to deny connection on config level by a single "allow this host(s) only" entry instead of rolling out icinga agent including firewall rule to different types of linux boxes, where each of those have a completely different means of firewall configuration (interface).

even centos6 and centos7 are completely different regarding firewall configuration, so to correctly implement the simple securtity rule "allow only $icingaserver connect to this monitored host" is multiple times more complex and error prone when doing that in an automated or manual way with the hosts firewall.

it's not "human failure" which is the problem here. it's complexity, and complexity is the worst enemy of security.

i think layered security is good, and it's good to start with simplicity.

being able to add the icinga servers monitoring server ip to the agents config to avoid connection from other hosts is simplicity for me. it's very unlikely that something goes wrong here. but it is very likely that something goes wrong with hosts firewall configuration, especially in big and heterogenous environments. you cannot even be sure that firewall is always up and running, as it is a service which may fail to start.

https://lists.fedoraproject.org/pipermail/devel/2014-March/196918.html

https://lists.fedoraproject.org/pipermail/devel/2014-March/197019.html

thank you for listening/discussion

[dnsmichi]

If you care about security in your environment, you shouldn't move this into the application/service layer. Instead, no-one and nothing can be trusted.

Here's a good read on Zero Trust: https://about.gitlab.com/blog/2019/04/01/evolution-of-zero-trust/

[devZer0]

will have a look, thanks

[Thomas-Gelf]

@dnsmichi: your arguments are valid. Still, IMHO @devZer0 makes a good point. Let me try to break this down:

• no one feels comfortable with one more open port on ALL systems
• IP spoofing is a theoretical issue. As sequence numbers are no longer predictable in today's operating systems this only works when you're in the very same broadcast domain. It usually also involves flooding your switches CAM table, which is also not that easy as it used to be with modern switches
• I'm with @dnsmichi when it goes to his concerns about IP restrictions. When you do not trust a software to validate SSL certs, why would you trust it to get IP address checks right?
• and once you're delegating socket creation to a third party tool - you could as well apply one single iptables rule, rejecting connections to that port from all but one IP

So, here we are. But there is one option we didn't talk about: couldn't we allow Agents to not listen to a port at all, when they're configured to connect to their parent (and not the other way round)? This would be easy to implement, has no drawback - and would probably make a perfect fit for @devZer0's environment: no listening port on any Agent.

Cheers,
Thomas

[devZer0]

When you do not trust a software to validate SSL certs, why
would you trust it to get IP address checks right?

it's not about trusting software for validating SSL certs. It's about expectation that there are bugs inside this complex piece of code , which perhaps makes it exploitable in the future.

and once you're delegating socket creation to a third party tool -
you could as well apply one single iptables rule, rejecting connections
to that port from all but one IP

applying iptable rules in a complex environment and make sure they are reboot-safe is a non trivial task, as you always need to take distro specific higher-level firewall management stuff into account.

on centos6 there is system-config-firewall-*/lokkit, on centos7 there is firewalld/firewall-cmd.

our freshly installed debian (exotic here) does not even seem to have any firewall "higher level management" installed, at least i don't see any firewall/iptables related stuff in /etc or in systemd (i'm not that much into debian) where i can add iptable rules. need to dig into it to make it work...

suse was using SuSEfirewall2 until SLES15 and it is now also using firewalld

so - there is a colorful zoo of firewall management out there, so even when using ansible for deploying icinga agent, it isn't a trivial task do add firewall rules correctly on every system and to make it sure it's reboot save and make sure it is really active on each reboot.

if distro update is changing something in firewall management, deploy mechanism for firewall configuration may break and you need to fix it. you need permanent maintenance to keep this safe and error free in a larger environment.

adding a single configuration entry to a config file is trivial and will always be immune to every system change. it's providing inherent basic safety.

you can always add the firewall proection layer if you don't trust it, but you get a large amount of basic security for zero work with some config entry which makes the agent deny connection at the application level

[dnsmichi]

couldn't we allow Agents to not listen to a port at all, when they're configured to connect to their parent

Set bind_host to the local loop back address then.
https://icinga.com/docs/icinga2/latest/doc/09-object-types/#apilistener

if distro update is changing something in firewall management, deploy mechanism for firewall configuration may break and you need to fix it. you need permanent maintenance to keep this safe and error free in a larger environment.

That sounds like as if you don't like to do that and want to move the responsibility to application developers than to secure your environment. It is your job to keep your systems safe, no matter which application is installed on these hosts.

adding a single configuration entry to a config file is trivial and will always be immune to every system change. it's providing inherent basic safety.

No. Especially in the cloud environment where IP addresses change often, this complicates setups in favor of "adding security". Do a little research on how often people ask that they cannot reach nrpe/nsclient++, and it turns out that allowed_hosts is wrongly configured. I'm not willing to add this burden to our support. We are still an open source project with limited resources. And we do a lot more for security than even enterprise paid products do.

[devZer0]

https://unix.stackexchange.com/questions/383013/why-does-debian-come-without-a-firewall-enabled-by-default
-snipp
I'm using Debian 9.1 with KDE and I'm wondering why it comes without a firewall installed and enabled by default? gufw is not even in DVD1's packages.
-snipp
Very good question. Ubuntu server doesn't even come with iptables pre-installed! I guess people just trying to take the end-to-end principle to the layer 7 extreme.
-snipp
And you might prefer to use iptables's successor, nftables, anyway.