Telegraf: Automatic discovery of services in the server

[gianarb]

Feature Request

Proposal:

I like the idea to install and run telegraf without any kind of configuration.

1. Download binary.
2. Start it.

And all plugins that support the automaticDiscovery capabilities are trying to start themselves with a default configuration.
Obviously, not all the plugins can support this capability but I am using RabbitMQ as an example. We know that by default we can resolve RabbitMQ via localhost on some port and it exposes an api to grab metrics. We can make the RabbitMQ smart enough to get this info by itself without any kind of configuration.

The same can work for Docker. We can look in /var/run/docker/sock to understand if it's up and running and we can start to listen.

Use case: [Why is this important (helps with prioritising requests)]

At the moment my use case is just to make my life easy when I am running a demo. But I think that it can be super useful also in other use cases. In practice, we are going to have a super easy and smart agent!

[jaredscheib]

This seems like a great idea

[danielnelson]

Do you think we should try to autostart all plugins with automatic discovery or only ones explicitly enabled? For instance in the agent section of the config we could have:

auto_discover_inputs = ["rabbitmq", "docker"]

This would cause only these plugins to run their automatic configuration.

We should also consider how it would operate if the plugin is manually configured. Should it override? Should it create a second plugin?

[gianarb]

The purpose of this is to avoid any kind of configuration file. It means that imho we can cross on all the plugins and that's it. No list or anything. The plugins that don't detect anything will stay disabled

[nhproject]

+1

Like Sysdig works, for instance.

[gianarb]

Another good use case can be related to docker, right now swarm (orchestration manager) supports global strategy. to deploy in every server 1 container. It's perfect the deploy agents but right now it's useless for telegraf because you also need to manage different configuration file, depends about where you are and which service are running in that node.

[danielnelson]

That's pretty nice.

I think the best way to add this would be as a config plugin, once the work for #272 is completed. We should be able to introduce an interface for local discovery and write a config plugin to handle it.

[miguelbernadi]

It needs to have a method to disable this behaviour. Otherwise some complex schemes would end with duplicated data (some from custom setup in one agent and some from automatic configuration in other nodes).

[sciurus]

This seems to be related to discovering processes that are running directly on the host.

I've be interested in discovering services that are running in containers in the host as well. For example, let's say I run nginx as a sidecar for my apps in kubernetes. Each host may have zero to many copies of nginx running accessible at different IPs.

Datadog's approach to this is described here.