Kibana: [Observability]Make a client to expose the indices to read data

[cauemarcondes]

The new Observability overview page (https://github.com/elastic/kibana/issues/68176) needs to fetch data on different solutions.
To avoid reading directly from each appropriate index I think each solution should expose a client that returns it.
For APM, we're going to work on a similar solution, you can find the description here: 67397

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[elasticmachine]

Pinging @elastic/uptime (Team:uptime)

[weltenwort]

Should the the solutions only provide a client or possibly a whole set of panels? Depending on the context the pure data might be hard to interpret.

Maybe we need a registry owned by the observability plugin, with which each observability solution can register panels to include?

[cauemarcondes]

Should the the solutions only provide a client or possibly a whole set of panels?

@weltenwort I believe only the client it's enough. I don't know if you've seen what we're building for APM, but we'll expose something like:

apmClient.search({
  apmIndices: ['transaction', 'span'],
  size: 100,
  query: {...},
  aggs: {}
})

[weltenwort]

Right, but how would the overview page know how to interpret the results?

In any case, I think we could benefit from a more precise formulation of the requirements for each observability solution, such as...

• Which data should be returned and how?
• What parameters can and should be given?

[shahzad31]

Why not have an observability client? i don't think so it make sense every team building a separate client, a common client should be created and other teams can contribute in terms of adding queries or interfaces.

[sqren]

Why not have an observability client? i don't think so it make sense every team building a separate client, a common client should be created and other teams can contribute in terms of adding queries or interfaces.

The client we are proposing is simply a lightweight wrapper around the existing elasticsearch client which handles the indicies to query. It will not know anything about the underlying queries.

[sqren]

Right, but how would the overview page know how to interpret the results?

What do you mean with interpret? Since we are making the elasticsearch queries we know how to interpret the response.

[weltenwort]

What do you mean with interpret? Since we are making the elasticsearch queries we know how to interpret the response.

I might be misunderstanding the point of this request completely, sorry.

I thought the purpose of such an abstraction would be to translate the parameters and response from ES terms to domain-specific terms.

In any case, in order to choose a useful presentation and consumer (such as the observability landing page) would still have to understand what the results mean. Is it a rate? Is it a bunch of log lines? Is it a histogram of anomaly scores?

[jasonrhodes]

I'd like to see an interface for how the results should come back so they can populate the charts/graphs, and then have each team expose a ... function? ... that does the query and returns the results. (This basically sounds like how a search strategy works in Kibana data, but I'm not sure if we are able to move completely to that...)

[cauemarcondes]

We had a meeting and agreed on, each solution will provide the data needed to fill the charts.
Some actions:

• @cauemarcondes will look on the registration part.
• @cauemarcondes will provide a contract that each solution must use to return the data.
• @cauemarcondes will provide the function contract.
• Each solution will create a function that returns the corresponding data.

[andrewvc]

Is this issue still relevant? Is there anything to do here?

[cauemarcondes]

No, this issue can be closed, we've tackled the implementation on other issues.