Azure-cosmos-dotnet-v3: Azure CosmosDB Unexpected Pagination Behaviour

[Jubast]

We are using CosmosDb C# SDK
We tried both: "Microsoft.Azure.Cosmos 3.4.1", "Microsoft.Azure.DocumentDB.Core 2.9.1 and 2.4.2"
We are getting invalid results and the main problems is the ResponseContinuation

[{"token":null,"range":{"min":"05C1DFFFFFFFFC","max":"FF"}}]

This started showing in one of our smaller service with only 14 documents.

In all queryes we use the folowing headers:

• "x-ms-documentdb-query-enablecrosspartition" = true
• "x-ms-max-item-count" = 100

Query 1:
The query is the folowing SELECT * FROM c.
We get the folowing response:

• 7 items
• ResponseContinuation [{"token":null,"range":{"min":"05C1DFFFFFFFFC","max":"FF"}}]

Then we use the continuation token to get the other 7 items.

Query 2:

If we modify the query to SELECT * FROM c ORDER BY c.property ASC, the order gets messed up! (responses are simplified)

• we get the folowing result ["A", "B", "C", "D", "F"]
• and the second query ["C", "D", "G"]

Query 3:
if we want to find only one item SELECT TOP 1 * FROM c WHERE c.name = @name, and the item is in the "second query result"

• nothing and RequestContionuation {"top":1,"sourceToken":"[{\"token\":null,\"range\":{\"min\":\"05C1DFFFFFFFFC\",\"max\":\"FF\"}}]"}

This is all a really unexpected behaviour.
Why does ORDER BY, TOP even exist if we can't even use it properly..
We can't afford to list all data from cosmos to our server and then do ordering, expecialy on bigger containers.

Stackoverflow link: https://stackoverflow.com/questions/58876612/azure-cosmosdb-unexpected-pagination-behaviour

[Jubast]

Any progress on this?

[j82w]

@bchong95 can you take a look at query 2 and 3?

For Query 1 it looks like you hit a common issue which is people misunderstand the max item count. The max item count is a maximum. There is no minimum. Empty pages are possible. For your scenario it looks like you have at least 2 partitions in Cosmos DB. The SDK went to the first partition and got 7 results and the other 7 results are on the second partition.

[h3ko]

I think we have the same problem. Server-side paging with ordering is not working properly when documents are across multiple partitions. EnableCrossPartitionQuery is enabled, but it seems we get a list of documents from first partition – e.g. first 5 ordered by property X and in the second partition there are documents which should be in response based on ordering by property X). We cannot load all documents and then do sorting and paging on the client side with 100k+ documents. As we have this in production your help would be very appreciated.

[Jubast]

Any progress on this?

[bchong95]

You are not allowed to mix and match continuation tokens with different queries. Please provide a console application code that reproduces the issue if there is no query or continuation token modification across continuations.

[acn-sbuad]

@Jubast you probably figured it out by now, but commenting in case someone else comes across this issue. Regarding query 3.
We have solved this by checking documentQuery.HasMoreResults
If there are more results, we use the continuation token, although doesn't seem to hold any value, and we send the request again. It will then retrieve the matches from the second partition.