Jaeger: query service "search" api 'limit', 'total' and 'offset' fields are not supplied on response

[jkandasa]

Requirement - what kind of business use case are you trying to solve?

set of fields are not supplied on the query service search response

Service: jaeger query
API: /apt/traces
Current response:

{
  data: "...",
  total: 0,
  limit: 0,
  offset: 0
}

Response struct: https://github.com/jaegertracing/jaeger/blob/886b96574253a005ee7ebe74140098f3fe183606/cmd/query/app/http_handler.go#L56
Response part: https://github.com/jaegertracing/jaeger/blob/886b96574253a005ee7ebe74140098f3fe183606/cmd/query/app/http_handler.go#L238

Fields total, limit and offset are not supplied on the response part.

I am ready yo submit a PR if someone guide me to get these fields.

[yurishkuro]

Thet are not supported by the backend, and pagination is also not supported. There's no benefit in returning them in the current state.

[ohetrifork]

This really makes the API difficult to use. So there is not way to order the search results, and there is no way to use pagination? In effect, you need to query down to eg. second long time intervals if you need to make sure that you pick up all results in an even moderately loaded system.
I would highly recommend introducing time sorting or pagination

[jpkrohling]

Do you want to give it a try? A first step would probably be to create an interface that defines the pagination functions, and make at least one storage plugin to implement it. The Query handler can then check if the current storage plugin implements the interface and will call the appropriate functions to get paginated results.

[sagaranand015]

A first step would probably be to create an interface that defines the pagination functions, and make at least one storage plugin to implement it. The Query handler can then check if the current storage plugin implements the interface and will call the appropriate functions to get paginated results.

@jpkrohling: I was going through this issue and tried a few approaches.
As I understand, the idea is to make the API paginated. To introduce pagination, say in case of api/traces API, can we not use the TraceQueryParams? We already have a limit queryString, adding support for an offset queryString would allow the offset data being passed to the appropriate storage layer implementation and returned paginated (and sorted, if any) from the storage layer itself.

As I understand the approach you mentioned, is it correct to say that the querying method of the storage layer would remain the same? The storage implementation would get the results like it does now and then a pagination function is called to return a paginated response. Is this correct?

I think getting pagination done on the storage query might help the performance, and looks straightforward to implement.
Inputs Please?

Thanks

[yurishkuro]

1. Offsets are a poor way of implementing pagination (there's an unnecessary long blog post that explains why: https://tusharsharma.dev/posts/api-pagination-the-right-way#offset-pagination)
2. Our storage backends (Cassandra & ES) currently do not organize the data in a way that would allow for consistent result ordering to implement pagination