Jaeger: Cannot page queries with both ORDER BY and a IN restriction

[jvans1]

Requirement - what kind of business use case are you trying to solve?

Connecting Jaeger to CosmosDB backend with the Cassandra API

Problem - what in Jaeger blocks you from solving the requirement?

I built binaries from the latest master and have a jaeger collector successfully writing to the cosmosdb cassandra API backend. When I run jaeger-query for a specific operation it works:

but when I have "all" operations selected it 500s:

It looks like a cassandra error. I'm not sure if this is some incompatibility of Azure CosmosDB cassandra API wrapper or if I've misconfigured something. I generated the schema with:

MODE=test DATACENTER=teamlabs  ./create.sh > create_schema.cql

 SSL_VERSION=TLSv1_2 SSL_VALIDATE=false cqlsh <instance> 10350 -u <username> -p <password> --ssl  --cqlversion=3.4.4 -f create_schema.cql

Stackoverflow suggests disabling paging, which I don't think we want here. The actual generation of the query is internal to jaeger and I don't see flags I could modify it with so I assume my setup is wrong somehow. Any ideas what I did wrong?

[yurishkuro]

This sounds more like an incompatibility of CosmosDB features with what Cassandra supports.

[Starblade42]

The Datastax documentation says that this is not a valid way to query Cassandra with both IN and ORDER BY: https://docs.datastax.com/en/archived/cql/3.3/cql/cql_using/useQueryIN.html

Note that using both IN and ORDER BY will require turning off paging with the PAGING OFF command in cqlsh.

(from https://stackoverflow.com/questions/51758089/how-to-use-in-and-order-by-at-the-same-time-in-cassandra/51765097#51765097)

so this is a CASSANDRA thing, and not just a CosmosDB restriction

[forestsword]

For others... I attempted to try and see if I could fix this. I discovered paging is _already_ disabled for this query:

https://github.com/jaegertracing/jaeger/blob/67243904eebe9ac8ef3b9183c8715f55d075644a/plugin/storage/cassandra/spanstore/reader.go#L390-L402

I've tried a few other things like disabling paging on the session itself and changing the query but with no luck. It looks as though CosmosDB is just ignoring any requests to disable paging and always pages. So like @yurishkuro said, this is a cosmos issue. We'll try raising it with them.