Kibana: [Telemetry] Audit runtime field implications

[TinaHeiligers]

The work on adding runtime fields is in progress and will be available to use in an elasticsearch index mapping soon.
Usage collection and sending data (telemetry) isn't specifically called out as being vulnerable, we need to audit our plugins and make sure of that.

[elasticmachine]

Pinging @elastic/kibana-telemetry (Team:KibanaTelemetry)

[alexfrancoeur]

@TinaHeiligers What is the total scope of this issue? Is this issue meant to understand the implications runtime fields will have on the cluster and collection plugins? Performance, stability, vulnerabilities, etc. Or is it meant to explore how it can / should affect how we think about partitioning the the data (cc: @mindbat). For the latter (or maybe both?) a lot of it will be dependent on when Kibana supports runtime fields. If this isn't the place to track that, I can open a discuss issue to explore implications runtime fields may have on telemetry-next

[TinaHeiligers]

@alexfrancoeur the scope of this issue is to ensure that usage collection and telemetry will continue to work as intended after runtime fields are introduced in Kibana (ensure that Kibana as a whole supports runtime fields).

I'd like to keep the telemetry-next initiative separate.

[TinaHeiligers]

I've finished an initial sweep of our usage collection code and it looks like we'll be ok. We query saved objects (not impacted by runtime fields) and .monitoring-* index patterns.

Monitoring queries the following index patterns using _source:
.monitoring-kibana-6-*,
.monitoring-kibana-7-*,
.monitoring-es-6-*,
.monitoring-es-7-*,
.monitoring-beats-6-*,
.monitoring-beats-7-*,
.monitoring-logstash-6-*,
.monitoring-logstash-7-*

AFAIK, these are internal index patterns that users can't edit.
cc @chrisronline could you please confirm that these index patterns can't be edited by users? Thanks!

[TinaHeiligers]

I've tentatively marked the Telemetry audit as good to go. Runtime fields support has been merged to master and backported to 7.x so we can test their use now.