Azure-cosmos-dotnet-v2: The continuation token specified in the request is malformed.

[NickBrooks]

I'm having an issue with continuation tokens where it states it is malformed. The token I receive from ResponseContinuation looks something like this (but a serialized string):

{\"token\":\"+RID:qqhuAMREDACTEDAAAAAAA==REDACTED:AQwAAAAAAAAAEwAAAAAAAAA=\",\"range\":{\"min\":\"\",\"max\":\"FF\"}}

But I get the following error:

'Message: {"Errors":["The continuation token specified in the request is malformed. Please re-run the query without it and iterate."]

Then I try and strip the token from the object above, something like this which I've seen other people use in other examples:

+RID:qqhuAMREDACTEDAAAAAAA==REDACTED:AQwAAAAAAAAAEwAAAAAAAAA=

And get this error:

JsonReaderException: Unexpected character encountered while parsing value: +. Path '', line 0, position 0.

My code to query Documents in DocumentDB looks as follows:

```C#
public static async Task GetItemsAsync(Expression> predicate)
{
IDocumentQuery query = client.CreateDocumentQuery(
UriFactory.CreateDocumentCollectionUri(Database, Collection),
new FeedOptions { MaxItemCount = 5 })
.Where(predicate)
.AsDocumentQuery();

        List<T> resultRange = new List<T>();
        var queryResult = await query.ExecuteNextAsync<T>();
        resultRange.AddRange(queryResult);
        var continuationToken = JsonConvert.DeserializeObject<ContinuationToken>(queryResult.ResponseContinuation);

        var result = new DocumentDbQueryResult();
        result.Result = resultRange;
        result.ContinuationToken = (continuationToken.Token != null) ? Base64.Encode(continuationToken.Token) : "";

        return result;
    }

And then my code to process a continuation token is pretty similar, but `RequestContinuation` specified in the `FeedOptions` instead of the `Predicate`.

```C#
public static async Task<DocumentDbQueryResult> GetContinuationTokenItemsAsync(string token)
        {
            if (token == null) return null;

            IDocumentQuery<T> query = client.CreateDocumentQuery<T>(
                UriFactory.CreateDocumentCollectionUri(Database, Collection),
                new FeedOptions { MaxItemCount = 5, RequestContinuation = token })
                .AsDocumentQuery();

            List<T> resultRange = new List<T>();
            var queryResult = await query.ExecuteNextAsync<T>();
            resultRange.AddRange(queryResult);
            var continuationToken = queryResult.ResponseContinuation;

            DocumentDbQueryResult result = new DocumentDbQueryResult();
            result.Result = resultRange;
            result.ContinuationToken = (continuationToken != null) ? Base64.Encode(continuationToken) : "";

            return result;
        }

Is anyone able to point me in the right direction?

[mackep]

I am experiencing the exact same issue as well with Microsoft.Azure.DocumentDB.Core version 1.3.0.

[mackep]

There are numerous examples available that utilize the ResponseContinuation token for paging, but we are unable to get any of them working due to the "The continuation token specified in the request is malformed"-issue.

This issue currently blocks our development progress. Could anyone kindly provide any assistance in this matter?

[sboshra]

Did you try passing in the continuation token as is w/o making any modifications to it?

[mackep]

@sboshra We've tried providing it as-is, i.e.:

{"token":"+RID:yZEwALUEcwAVAAAAAAAAAA==#RT:1#TRC:10#FPC:AgEAAAAKABEA4P9CQP8/BAA=","range":{"min":"","max":"FF"}}

... through the RequestContinuation property on FeedOptions.

[sboshra]

This continuation token looks well-formed:
{"token":"+RID:yZEwALUEcwAVAAAAAAAAAA==#RT:1#TRC:10#FPC:AgEAAAAKABEA4P9CQP8/BAA=","range":{"min":"","max":"FF"}}

But not the one above:
{"token":"+RID:qqhuAMREDACTEDAAAAAAA==REDACTED:AQwAAAAAAAAAEwAAAAAAAAA=","range":{"min":"","max":"FF"}}

Note '#RT:1#TRC:10#FPC' vs. 'REDACTED'.

[NickBrooks]

Yes - I redacted part of my token to protect it, but my token is the same formation as @mackep

And yes, this has halted development process significantly to the point we aren't doing any DocumentDB dev currently.

[NickBrooks]

@ryancrawcour Do you have an answer to this? Thanks 😃

[mackep]

I managed to find a solution to the issue. It seems like the same predicate(s) must be used again when querying items using the continuation token, i.e:

var uri = UriFactory.CreateDocumentCollectionUri(Database, Collection);
var feedOptions = new FeedOptions
{
    MaxItemCount = MaxItemCountPerQuery
};

string responseContinuation;

using (var query = _client.CreateDocumentQuery<MyItem>(uri, feedOptions).Where(a => a.SomeProperty == "bar").AsDocumentQuery())
{
    var result = await query.ExecuteNextAsync<MyItem>();

    responseContinuation = result.ResponseContinuation;

    /* process result */
}

/* ... */

var continuationFeedOptions = new FeedOptions
{
    MaxItemCount = MaxItemCountPerQuery,
    RequestContinuation = responseContinuation
};

using (var query = _client.CreateDocumentQuery<MyItem>(uri, continuationFeedOptions).Where(a => a.SomeProperty == "bar").AsDocumentQuery())
{
    var result = await query.ExecuteNextAsync<MyItem>();

    responseContinuation = result.ResponseContinuation;

    /* process result */
}

@NickBrooks can you confirm?

[aprooks]

@mackep seems like yes, continuation token will only work for the very same query.

spent a couple of hours to figure it out. It's shame this is not documented and no reply from the team yet.

[sboshra]

This is correct. The continuation token is specific to a query request and cannot be used with a different query (you'd either get an error or incorrect results). Basically the continuation token marks where the query execution stopped so that subsequent continuation could resume from this point.

If you're using .Net SDK then you really don't have to deal with the continuation tokens yourself. To iterate through all continuations, you could something like this:

var query = Client.CreateDocumentQuery(
UriFactory.CreateDocumentCollectionUri(databaseId, collectionId),
sqlQuery,
feedOptions).AsDocumentQuery();

while (query.HasMoreResults)
{
var response = await query.ExecuteNextAsync();
results.AddRange(response );
}

Note that query.HasMoreResults will have an initial value of true.

[aprooks]

@sboshra continuation token should be used when iteration cannot be done in back-end in a single loop. For example, you send first 10 results to client and then query next 10 if a user clicks on 'get more results' button click. So using it on .Net is 100% valid

[ursenix]

I've been trying to use this continuation token in query string for WebAPI calls from SPA/mobile app. As @aprooks said, it will only work in very same query. There is no advantage to gain this facility by separate calls.
I see, the only way to paging is to do manually.
Any ideas?

[aprooks]

By very same I meant same structure and same parameters. If you are doing SQL query - you should keep it the same. This way calling next page via separate call worked for me.

[kirankumarkolli]

We will add a sample and also improve the documentation.

[codecadwallader]

@mackep Thank you.. I just spent a couple hours debugging the exact same issue (query needs to be resent on continuation just as it was on the original.. otherwise the SDK silently throws out the continuation without error). I falsely assumed that the continuationToken replaced the need for the query.

[petermarshall]

It is worth noting that the WHOLE token has to be used. Not just the inner part that says token.

[zsybupt]

any update about this ? If I create query with custom query text, it seems that I need to pass the same query and the continuation token, then I can get the next page result, if no query text provided, I see the same exception mentioned above

[zsybupt]

and I am using cross partition enabled