Beats: [filebeat] Expose the number of file open in the UI

[ph]

When back pressure is happening on the system, Filebeat can continuously open file descriptors for every file discovered. It would be useful to display that information in the monitoring UI, if the number keeps growing it will help to detect if the system is falling behind.

Displaying File descriptor in general could be useful for.

• Logs input
• UDP input
• TCP input
• Syslog input

[ph]

cc @ppf2

[ruflin]

@ph Do we already expose it in our stats API?

[ph]

@ruflin Yes, we have some code to fetch it so it should be exposed in the stats API, https://github.com/elastic/beats/blob/f6343865bf1d7f772443df02fba0370d993e75ec/libbeat/metric/system/process/process.go#L157-L182 Making a graph for it should not be too complicated.

[ruflin]

I think then we already report the data to monitoring.

[kvch]

Unfortunately, I haven't found any trace of reporting this value. Now I am implementing it.

EDIT: I have found the metric for log input as harvester.open_files.

[andrewkroh]

FYI: We have this data for Linux in go-sysinfo. The library can provide both the FD count and the names of things that the FDs point to (list of files, sockets, pipes, etc).

• Code: https://github.com/elastic/go-sysinfo/blob/b603f0b7f2050dbc69b053eb5c796be0f3ac9fd2/types/process.go#L42-L45

• Example data: https://travis-ci.org/elastic/go-sysinfo/jobs/408147162#L642-L649