Node_exporter: Binding to Unix Socket

[Omeryl]

Best I can tell, it's not possible to bind node_exporter to a unix socket, is this something that we can consider adding? (I couldn't find any issues about this after a few moments of searching, apologies of this is a duplicate issue).

[SuperQ]

Prometheus only supports polling targets over http/tcp. Adding unix socket support to the node_exporter wouldn't be very useful.

What is the use case you are trying to solve?

[Omeryl]

@SuperQ Having some potentially-sensitive metrics in a shared environment, I'd like to reverse proxy the daemon behind nginx adding basic auth support there. It's a nice to have, but not a complete deal breaker I guess.

[mdlayher]

Bind to localhost only and then go through nginx?

[Omeryl]

@mdlayher The problem is I don't trust every user on localhost, and there's metrics I want to feed through node_exporter I don't necessarily want exposed.

[mdlayher]

And nobody else on localhost can use sudo/root and view your metrics?

[Omeryl]

@mdlayher correct.

[SuperQ]

If you have sensitive metrics, I recommend not sending them through the node exporter, and adding them directly to your application, or writing your own exporter.

Also note that basic auth is not secure. It is sent in plain text in the http headers.

[Omeryl]

@SuperQ Why .. this is such a simple thing to add in the long run (only a few lines of change to the node_exporter).

[Omeryl]

"Basic auth is not secure" is also a bit of a cop out here, it's a mechanism (username and password over headers) that can be verified in a number of ways, and can be rolled easily per-platform just like API keys are.

[mdlayher]

I think the problem is that this is a pretty unusual request when the rest of the prometheus ecosystem uses and expects HTTP. I would also agree that you're probably best off doing a custom exporter as I suspect the data you're concerned about would be sourced from a text file script anyway.

[Omeryl]

@mdlayher Just because it's unusual, doesn't make it irrelevant. The alternative here would be going crazy with iptables, I suppose, but that seems bizarre when it's relatively easy to just patch node_exporter to allow unix sockets. If the Prometheus organization doesn't want to do it in repo, I guess I get to maintain my own builds instead.

[Omeryl]

Instead, a feature request from someone really wanting to use Prometheus in our infrastructure just got shut down as meaningless in under 24 hours, which rather saddens me.

[SuperQ]

The node_exporter is only one exporter of hundreds. We only scrape over TCP, so adding UNIX sockets to one exporter is now a pattern we would need to follow in all exporters. This is not something we can support.

I'm sorry if you feel this was shot down quickly, but we've had this discussion before. It's not our intention to shoot down ideas for no reason.

Our real end goal is to develop our security guides such that we can eliminate the need for reverse proxies altogether. We _think_ the story will be TLS + client certs. But we're not security experts, so we need time to develop a proper security story.

[Omeryl]

Some thoughts after discussion in IRC:

I've been told that the node_exporter main file is basically copy and pasted between the exporters, so issuing patches would be relatively simple (this was my understanding, not something they said). The fear here is that they don't want to have to potentially support something like this on everything, when they feel like TLS auth is the way to go in the future. It has been requested that instead of this issue, I go email the prometheus-developers mailing list (which I intend to do when I get the time). Just wanted to plop this down for anybody else that stumbles upon this issue.

[dwfreed]

What's to support? Unix sockets don't behave any differently in the normal case where you don't care about the peer's address and aren't sending any fancy messages across. As far as net/http is concerned, it's the same thing as a TCP socket.