Beats: Filebeat should error out if no modules/prospectors are configured
Filebeat should return an error if no modules or prospectors are defined at all AND no dynamic configurations are in place (like modules reload or autodiscover). Current output:
2018-02-23T18:08:01.962+0100 INFO instance/beat.go:475 Home path: [/home/exekias/go/src/github.com/elastic/beats/filebeat] Config path: [/home/exekias/go/src/github.com/elastic/beats/filebeat] Data path: [/home/exekias/go/src/github.com/elastic/beats/filebeat/data] Logs path: [/home/exekias/go/src/github.com/elastic/beats/filebeat/logs]
2018-02-23T18:08:01.962+0100 INFO instance/beat.go:482 Beat UUID: 302670b3-66d2-4d28-a04d-55437d44530b
2018-02-23T18:08:01.962+0100 INFO instance/beat.go:220 Setup Beat: filebeat; Version: 7.0.0-alpha1
2018-02-23T18:08:01.963+0100 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-02-23T18:08:01.963+0100 INFO pipeline/module.go:76 Beat name: X1
2018-02-23T18:08:01.963+0100 INFO instance/beat.go:309 filebeat start running.
2018-02-23T18:08:01.964+0100 INFO registrar/registrar.go:108 Loading registrar data from /home/exekias/go/src/github.com/elastic/beats/filebeat/data/registry
2018-02-23T18:08:01.964+0100 INFO registrar/registrar.go:119 States Loaded from registrar: 0
2018-02-23T18:08:01.964+0100 INFO crawler/crawler.go:48 Loading Inputs: 0
2018-02-23T18:08:01.964+0100 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-02-23T18:08:01.964+0100 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 0
2018-02-23T18:08:01.964+0100 INFO cfgfile/reload.go:127 Config reloader started
2018-02-23T18:08:01.964+0100 INFO cfgfile/reload.go:219 Loading of config files completed.
and stays there forever. It should print an error and die instead
exekias
All 5 comments
Case 1: Modules: Not configured, Inputs: Configured
➜ filebeat git:(filebeat-nomod-fail) ✗ ./filebeat -c filebeat.yml -e -d "*"
2018-03-05T19:27:00.496+0530 INFO instance/beat.go:475 Home path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Config path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Data path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/data] Logs path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/logs]
2018-03-05T19:27:00.496+0530 DEBUG [beat] instance/beat.go:502 Beat metadata path: /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/meta.json
2018-03-05T19:27:00.496+0530 INFO instance/beat.go:482 Beat UUID: 1af5e290-ed9d-48c6-b55f-5254bdd04394
2018-03-05T19:27:00.496+0530 INFO instance/beat.go:220 Setup Beat: filebeat; Version: 7.0.0-alpha1
2018-03-05T19:27:00.496+0530 DEBUG [beat] instance/beat.go:237 Initializing output plugins
2018-03-05T19:27:00.496+0530 DEBUG [processors] processors/processor.go:49 Processors:
2018-03-05T19:27:00.497+0530 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-03-05T19:27:00.497+0530 INFO pipeline/module.go:76 Beat name: lake
2018-03-05T19:27:00.497+0530 INFO instance/beat.go:270 filebeat stopped.
2018-03-05T19:27:00.497+0530 ERROR instance/beat.go:674 Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
Case 2: Modules: Not configured, Inputs: Not configured
➜ filebeat git:(filebeat-nomod-fail) ✗ ./filebeat -c filebeat.yml -e -d "*"
2018-03-05T19:21:20.626+0530 INFO instance/beat.go:475 Home path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Config path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Data path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/data] Logs path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/logs]
2018-03-05T19:21:20.626+0530 DEBUG [beat] instance/beat.go:502 Beat metadata path: /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/meta.json
2018-03-05T19:21:20.670+0530 INFO instance/beat.go:482 Beat UUID: 1af5e290-ed9d-48c6-b55f-5254bdd04394
2018-03-05T19:21:20.670+0530 INFO instance/beat.go:220 Setup Beat: filebeat; Version: 7.0.0-alpha1
2018-03-05T19:21:20.670+0530 DEBUG [beat] instance/beat.go:237 Initializing output plugins
2018-03-05T19:21:20.670+0530 DEBUG [processors] processors/processor.go:49 Processors:
2018-03-05T19:21:20.670+0530 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-03-05T19:21:20.670+0530 INFO pipeline/module.go:76 Beat name: lake
2018-03-05T19:21:20.670+0530 INFO instance/beat.go:270 filebeat stopped.
2018-03-05T19:21:20.670+0530 ERROR instance/beat.go:674 Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
Case 3: Modules : Configured, Inputs: Not configured
➜ filebeat git:(filebeat-nomod-fail) ✗ ./filebeat -c filebeat.yml -e -d "*"
2018-03-05T19:29:37.405+0530 INFO instance/beat.go:475 Home path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Config path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Data path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/data] Logs path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/logs]
2018-03-05T19:29:37.405+0530 DEBUG [beat] instance/beat.go:502 Beat metadata path: /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/meta.json
2018-03-05T19:29:37.405+0530 INFO instance/beat.go:482 Beat UUID: 1af5e290-ed9d-48c6-b55f-5254bdd04394
2018-03-05T19:29:37.405+0530 INFO instance/beat.go:220 Setup Beat: filebeat; Version: 7.0.0-alpha1
2018-03-05T19:29:37.405+0530 DEBUG [beat] instance/beat.go:237 Initializing output plugins
2018-03-05T19:29:37.405+0530 DEBUG [processors] processors/processor.go:49 Processors:
2018-03-05T19:29:37.405+0530 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-03-05T19:29:37.405+0530 INFO pipeline/module.go:76 Beat name: lake
2018-03-05T19:29:37.406+0530 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-03-05T19:29:37.406+0530 INFO instance/beat.go:309 filebeat start running.
2018-03-05T19:29:37.406+0530 DEBUG [registrar] registrar/registrar.go:90 Registry file set to: /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/registry
2018-03-05T19:29:37.406+0530 INFO registrar/registrar.go:110 Loading registrar data from /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/registry
2018-03-05T19:29:37.406+0530 INFO registrar/registrar.go:121 States Loaded from registrar: 0
2018-03-05T19:29:37.406+0530 INFO crawler/crawler.go:48 Loading Inputs: 0
2018-03-05T19:29:37.406+0530 DEBUG [cfgfile] cfgfile/reload.go:95 Checking module configs from: /home/dolftax/code/src/github.com/elastic/beats/filebeat/modules.d/*.yml
2018-03-05T19:29:37.406+0530 DEBUG [cfgfile] cfgfile/reload.go:109 Number of module configs found: 0
2018-03-05T19:29:37.407+0530 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 0
2018-03-05T19:29:37.407+0530 DEBUG [registrar] registrar/registrar.go:152 Starting Registrar
2018-03-05T19:29:37.407+0530 INFO cfgfile/reload.go:127 Config reloader started
2018-03-05T19:29:37.407+0530 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-03-05T19:29:37.407+0530 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-03-05T19:29:37.407+0530 INFO cfgfile/reload.go:219 Loading of config files completed.
Case 4: Prospectors: Not configured, Modules: Not configured
➜ filebeat git:(filebeat-nomod-fail) ✗ ./filebeat -c filebeat.yml -e -d "*"
2018-03-05T19:41:37.732+0530 INFO instance/beat.go:475 Home path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Config path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat] Data path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/data] Logs path: [/home/dolftax/code/src/github.com/elastic/beats/filebeat/logs]
2018-03-05T19:41:37.732+0530 DEBUG [beat] instance/beat.go:502 Beat metadata path: /home/dolftax/code/src/github.com/elastic/beats/filebeat/data/meta.json
2018-03-05T19:41:37.732+0530 INFO instance/beat.go:482 Beat UUID: 1af5e290-ed9d-48c6-b55f-5254bdd04394
2018-03-05T19:41:37.732+0530 INFO instance/beat.go:220 Setup Beat: filebeat; Version: 7.0.0-alpha1
2018-03-05T19:41:37.732+0530 DEBUG [beat] instance/beat.go:237 Initializing output plugins
2018-03-05T19:41:37.732+0530 DEBUG [processors] processors/processor.go:49 Processors:
2018-03-05T19:41:37.732+0530 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-03-05T19:41:37.733+0530 INFO pipeline/module.go:76 Beat name: lake
2018-03-05T19:41:37.733+0530 INFO instance/beat.go:270 filebeat stopped.
2018-03-05T19:41:37.733+0530 ERROR instance/beat.go:674 Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?
@exekias Is this issue about case 3? Could you please post your filebeat configuration as well? :)
jaipradeesh
on 5 Mar 2018
This issue is about case 3 which is the default setup when you download and run Filebeat (I tested with filebeat-6.2.2-darwin-x86_64.tar.gz). I would add to your characterization of case 3 that while the modules features is configured, there are no modules are enabled.
$ ./filebeat export config
filebeat:
config:
modules:
path: /Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64/modules.d/*.yml
reload:
enabled: false
prospectors:
- enabled: false
paths:
- /var/log/*.log
type: log
./filebeat modules list
Enabled:
Disabled:
apache2
auditd
icinga
kafka
logstash
mysql
nginx
osquery
postgresql
redis
system
traefik
````
$ ./filebeat -e
2018-03-07T23:00:25.253-0500 INFO instance/beat.go:468 Home path: [/Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64] Config path: [/Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64] Data path: [/Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64/data] Logs path: [/Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64/logs]
2018-03-07T23:00:25.253-0500 INFO instance/beat.go:475 Beat UUID: 68cd145c-dc36-44da-84cd-c51eef887ecf
2018-03-07T23:00:25.253-0500 INFO instance/beat.go:213 Setup Beat: filebeat; Version: 6.2.2
2018-03-07T23:00:25.253-0500 INFO elasticsearch/client.go:145 Elasticsearch url: http://localhost:9200
2018-03-07T23:00:25.254-0500 INFO pipeline/module.go:76 Beat name: macbook15.data.sr.local.crowbird.com
2018-03-07T23:00:25.254-0500 INFO instance/beat.go:301 filebeat start running.
2018-03-07T23:00:25.254-0500 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-03-07T23:00:25.254-0500 INFO registrar/registrar.go:108 Loading registrar data from /Users/akroh/Downloads/filebeat-6.2.2-darwin-x86_64/data/registry
2018-03-07T23:00:25.254-0500 INFO registrar/registrar.go:119 States Loaded from registrar: 0
2018-03-07T23:00:25.254-0500 INFO crawler/crawler.go:48 Loading Prospectors: 1
2018-03-07T23:00:25.254-0500 INFO crawler/crawler.go:82 Loading and starting Prospectors completed. Enabled prospectors: 0
2018-03-07T23:00:25.254-0500 INFO cfgfile/reload.go:127 Config reloader started
2018-03-07T23:00:25.258-0500 INFO cfgfile/reload.go:219 Loading of config files completed.
```
andrewkroh
on 8 Mar 2018
Ref: https://github.com/elastic/beats/pull/6538 (WIP. Yet to write system tests)
I'm a bit confused with the following code:
if !config.ConfigInput.Enabled() && !config.ConfigModules.Enabled() && !haveEnabledInputs && config.Autodiscover == nil {
if !b.InSetupCmd {
return nil, errors.New("no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?")
}
// in the `setup` command, log this only as a warning
logp.Warn("Setup called, but no modules enabled.")
}
Error message says "configuration reloading disabled" but condition for the same is not checked in the if statement.
jaipradeesh
on 13 Mar 2018
config.ConfigInput.Enabled checks if input config reloading is enabled and config.ConfigModules.Enabled checks if modules config reloading is enabled. Thus, the message "config reloading disabled" is correct.
kvch
on 13 Mar 2018
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![botelastic[bot] picture](https://avatars0.githubusercontent.com/u/6764390?v=4&s=40)
botelastic[bot]
on 27 Dec 2020
Related issues
exekias
·
3Comments
dedemorton
·
3Comments
geekpete
·
3Comments
feelan03
·
3Comments
ptrlv
·
3Comments