Beats: filebeat creates mapping explosion with field monitoring.metrics.filebeat.harvester.files

[bsturg]

• Version: 7.6.0
• Operating System: all

With PR https://github.com/elastic/beats/pull/13395 for Filebeat 7.6.0, filebeat added a few new metrics to troubleshoot harvesters. This new feaature can cause a mapping a mapping explosion when the data is loaded into a time series index.

To resolve this issue, use the drop processor to drop monitoring.metrics.filebeat.harvester.files

• https://www.elastic.co/guide/en/beats/filebeat/master/drop-fields.html

Example of mapping explosion.

"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.last_event_published_time",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.last_event_timestamp",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.name",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.read_offset",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.size",
"monitoring.metrics.filebeat.harvester.files.00000aac-6e1e-4dd2-9ea1-bf08d4de9c22.start_time",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817.last_event_published_time",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817.last_event_timestamp",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817.name",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817.size",
"monitoring.metrics.filebeat.harvester.files.00007424-866f-42b6-91a6-e0c8151a7817.start_time",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.last_event_published_time",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.last_event_timestamp",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.name",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.read_offset",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.size",
"monitoring.metrics.filebeat.harvester.files.0002ba70-2983-4cb9-a666-f43eccd4be0e.start_time",

Kibana may also experience an issue when trying to refresh the mapping and requires a restart. The error message seen in kibana is:

error: undefined Response
        at https://kibana-url/bundles/commons.bundle.js:3:1373057

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[andrewkroh]

@EthanStrider saw this while using the Metricbeat golang/expvar metricset to monitor Filebeat. It causes a mapping explosion in the metricbeat-* indices.

[LukaszWasko]

I have also the same problem.

[andrewstucki]

Looks like this was fixed in https://github.com/elastic/beats/pull/19977 and should be available in today's 7.10 release