Amplify-cli: nextToken not accepted and gives error "Invalid pagination token given"

[apanagos]

Describe the bug
Query is working in development and build environments but when deployed in the Test environment the nextToken is never accepted.

The same error occurs in the code as well as the AppSync console where we do a query with a limit and get a nextToken. Then we do a query straight away with the same token and it gives the following error:
com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given.

To Reproduce
Do following query in the console (same as in the code) FYI There are 15 items in the table.

{
  listUsers(limit:3){
    items{
      name
    }
    nextToken
  }
}

The response is:

{
  "data": {
    "listUsers": {
      "items": [
        {
          "name": "Joergen Von Klopp"
        },
        {
          "name": "Yolanda McAdams"
        },
        {
          "name": "Info Test"
        }
      ],
      "nextToken": "eyJ2ZX...shortened="
    }
  }
}

Then we do it again with the nextToken:

{
  listUsers(limit:3,nextToken:"eyJ2ZX...shortened="){
    items{
      name
    }
    nextToken
  }
}

The error is:

{
  "data": {
    "listUsers": null
  },
  "errors": [
    {
      "path": [
        "listUsers"
      ],
      "data": null,
      "errorType": "DynamoDB:UserIllegalArgumentException",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
    }
  ]
}

We experience the same behavior with all the tables in this environment. If we increase the limit we of course get all the elements but for some reason nextToken does not work.

Expected behavior
Should get more data returned like on the other environments.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment

npx envinfo --system --binaries --browsers --npmPackages --npmGlobalPackages
npx: installed 1 in 1.301s

  System:
    OS: Windows 10 10.0.18362
    CPU: (12) x64 Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
    Memory: 8.41 GB / 23.94 GB
  Binaries:
    Node: 12.13.1 - C:\Tools\nodejs\node.EXE
    npm: 6.13.2 - C:\Tools\nodejs\npm.CMD
  Browsers:
    Edge: 44.18362.449.0
    Internet Explorer: 11.0.18362.1

Additional context
Amplify --version 4.5.0
"aws-amplify": "^2.2.2",
"aws-amplify-react": "^3.1.3",

[sammartinez]

@apanagos could you by chance provide a code snippet of the React code you are using? Also want to verify, have you gone through the steps in order to connect AppSync to your Amplify app? Please see this link: https://aws-amplify.github.io/docs/js/api

[apanagos]

Hi @sammartinez let me explain my environment, that may help you understand the context.
We have our code in git with 4 different branches, dev, build, test, master.
We have Amplify environments for build, test and master that refresh from GIT branches.
We also have two development environments for the two developers keyed off our branches off dev.
So for the last 5 months we develop in dev and then push to build nightly and every two weeks to test. Once test is approved it is moved to master.

So we have had that flow for months.

This week when I moved code from build to test, test started to fail every where a nextToken was required. Bear in mind the same works in our dev and build environments.

So when it failed we went to the AppSync console to do this manually to see if we got the same response. We did the same steps above for all our environments and it worked but only with our test environment would give us this error on all tables. We could run other queries in the console on test and they worked. We could do the same query but with a limit greater than the number if items and of course it works.

So I can get you the code but it doesn't really matter as it is the same error in console.

I suspect I could delete the environment and create it again and it would most probably start working but that is not a solution and we would like to understand what is happening. We are gearing up to go live in 4 weeks and this would be devastating to do to a live environment. Considering we are using Cognito for users and we cannot backup and restore that.

Anyway if you feel you need a code snippet to debug further then I will get that for you.

[bengsiswantoh]

i also experience this error.

use this query on aws console

query list {
  listUsers {
    items {
      id
    }
    nextToken
  }
}

console return

{
  "data": {
    "listUsers": {
      "items": [
        {
          "id": "692d47ae-ee2e-4e64-90f1-5a9984382f14"
        },
        {
          "id": "9d096d0f-c057-4d29-b30f-de24254a6cf9"
        },
        {
          "id": "a6c937e6-06b3-47be-8306-2d5cde74b0e2"
        },
        {
          "id": "ff5d01b8-a7dd-44d0-8fa2-d0d738233ac1"
        },
        {
          "id": "f7d4960f-bb80-4dc6-a72e-83824afdc0e4"
        },
        {
          "id": "5b2d0722-f38e-40b7-956f-4461f1eae367"
        },
        {
          "id": "2277475b-718c-4582-be91-cc8105ce90ca"
        },
        {
          "id": "71a93b23-742d-4920-84e5-b9f34e77d3c0"
        },
        {
          "id": "15a6bf64-4854-498f-b50e-3d3c5fe26bfe"
        },
        {
          "id": "a6f981a7-50b4-4660-acb1-b297c97503a2"
        }
      ],
      "nextToken": "eyJ2ZXJzaW9uIjoyLCJ0b2tlbiI6IkFRSUNBSGg5OUIvN3BjWU41eE96NDZJMW5GeGM4WUNGeG1acmFOMUpqajZLWkFDQ25BRS9Cc1c1Tks2WTRhVTN3ckZ2Y3N1WUFBQUNIRENDQWhnR0NTcUdTSWIzRFFFSEJxQ0NBZ2t3Z2dJRkFnRUFNSUlCL2dZSktvWklodmNOQVFjQk1CNEdDV0NHU0FGbEF3UUJMakFSQkF4M21KMUYyWTVvMVBTM0x1WUNBUkNBZ2dIUE5rVTdldzdiNFZSVlB2SVkwYW9sTGxXVjJBakRQMlZ5MXBtdW9FQzF0V3F1Rk5YckE3bmcwMFF6bmtuS1JRakZwTnRDeWVkaDZsdFVTUmhZbTRwaXpVVjdPUERJT1RqNTZ6OHZGVzVRVVI5RTVxUGdta3hiNDdjUFJQMjVxdTVpM3RUMFVPaXNBREN1VHRXdTlZYXhqa1NuVzVDWnNoZFdFK3hMd2NBQ3JWa2dzQ2dBZ2hmZ2M1RVp5Mk5EVnB4SWJjc2ZVYW9va2UxWEcwcUt0cGhjN2gzQkVWaC81d1lXVU9iMzVjMEZ4d3lyVGxoRllvSjQzRzZWU1FBbW8rdU5oMEt6Wm9WY0NvZElqMjg0UXM4NXFPZFc4dDhIaHJ6SnJMN0orZ3oxUU95QU1XQ1JXa2V6c1ZkYVFpalRpV0JYaktQS3hUY1Z0NEk4R0F0R1Q4OWI0VnlDdmpnN05RVXZpSTFJWTN6bTl1Vm5ydXNBYXhzT0ZUUEZKZVBaRDVQM3BvWDBueFdxTkkvWDhQNkt6Q1ljdW1jVzUvTUlLYzJXVU9sTXdyRE5TK3F0Wis4cnZFVHBmaVhiQlZEcFYzdmlJdXlnQkwwaURXRE95MzdrTDVFc3A5V2R1TkRHTDZMRklZem43VFdOL0t2cTArQnpZT3RLTFQvakZPZXphcHdRcFM5WXM3a0h3cWVaWFVXRjNQWlZDT0JaQWlMaHI1c3R6RkhwRmtVYlNPYzU2M1g5Tm03dzNsM3JBMVgyUFRkamxFVFN5RjNCS2ltU1YzLzFzSmVhNm93ODNJeGdueG1qeWZqaE5ZeVUrZz09In0="
    }
  }
}

then

query list {
  listUsers(nextToken: "eyJ2ZXJzaW9uIjoyLCJ0b2tlbiI6IkFRSUNBSGg5OUIvN3BjWU41eE96NDZJMW5GeGM4WUNGeG1acmFOMUpqajZLWkFDQ25BRS9Cc1c1Tks2WTRhVTN3ckZ2Y3N1WUFBQUNIRENDQWhnR0NTcUdTSWIzRFFFSEJxQ0NBZ2t3Z2dJRkFnRUFNSUlCL2dZSktvWklodmNOQVFjQk1CNEdDV0NHU0FGbEF3UUJMakFSQkF4M21KMUYyWTVvMVBTM0x1WUNBUkNBZ2dIUE5rVTdldzdiNFZSVlB2SVkwYW9sTGxXVjJBakRQMlZ5MXBtdW9FQzF0V3F1Rk5YckE3bmcwMFF6bmtuS1JRakZwTnRDeWVkaDZsdFVTUmhZbTRwaXpVVjdPUERJT1RqNTZ6OHZGVzVRVVI5RTVxUGdta3hiNDdjUFJQMjVxdTVpM3RUMFVPaXNBREN1VHRXdTlZYXhqa1NuVzVDWnNoZFdFK3hMd2NBQ3JWa2dzQ2dBZ2hmZ2M1RVp5Mk5EVnB4SWJjc2ZVYW9va2UxWEcwcUt0cGhjN2gzQkVWaC81d1lXVU9iMzVjMEZ4d3lyVGxoRllvSjQzRzZWU1FBbW8rdU5oMEt6Wm9WY0NvZElqMjg0UXM4NXFPZFc4dDhIaHJ6SnJMN0orZ3oxUU95QU1XQ1JXa2V6c1ZkYVFpalRpV0JYaktQS3hUY1Z0NEk4R0F0R1Q4OWI0VnlDdmpnN05RVXZpSTFJWTN6bTl1Vm5ydXNBYXhzT0ZUUEZKZVBaRDVQM3BvWDBueFdxTkkvWDhQNkt6Q1ljdW1jVzUvTUlLYzJXVU9sTXdyRE5TK3F0Wis4cnZFVHBmaVhiQlZEcFYzdmlJdXlnQkwwaURXRE95MzdrTDVFc3A5V2R1TkRHTDZMRklZem43VFdOL0t2cTArQnpZT3RLTFQvakZPZXphcHdRcFM5WXM3a0h3cWVaWFVXRjNQWlZDT0JaQWlMaHI1c3R6RkhwRmtVYlNPYzU2M1g5Tm03dzNsM3JBMVgyUFRkamxFVFN5RjNCS2ltU1YzLzFzSmVhNm93ODNJeGdueG1qeWZqaE5ZeVUrZz09In0=") {
    items {
      id
    }
    nextToken
  }
}

console return

{
  "data": {
    "listUsers": null
  },
  "errors": [
    {
      "path": [
        "listUsers"
      ],
      "data": null,
      "errorType": "DynamoDB:UserIllegalArgumentException",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
    }
  ]
}

[AaronHarris]

Thanks for reporting the issue to us. We don't need the React code as much, but we could help debug this better if you can send us the VTL mapping templates for this resolver. If I'm understanding this correctly, youre saying it works in test but not in live? In that case it would be helpful to see both the mapping templates of an API that does work and another that does not.

[nick-secur]

We are having a similar issue. We have a react.js / amplify app with a "master - detail" view where we paginate through records from an aws dynamo db using the 'next token' property returned from a graphQl 'list all' query. The app was working great until we tried to set up a team dev environment with continuous deploys pulling from a git repo. All queries seem to be working well from the new environments with the exception of the 'next token' returned from the list all query ...

[apanagos]

Thanks @AaronHarris , we have 6 environments,

1. 2 DEV environments,
2. a nightly BUILD env,
3. a TEST env
4. and a PROD env.

We merged from DEV -> BUILD -> TEST and this broke in TEST,
It works in DEV and BUILD but not TEST with the token error. FYI TEST worked before the merge and now ALL tables in TEST have the same issue.

Here is Query.listUsers.req.vtl for both DEV and TEST. They are the same.

#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
#set( $ListRequest = {
  "version": "2017-02-28",
  "limit": $limit
} )
#if( $context.args.nextToken )
  #set( $ListRequest.nextToken = "$context.args.nextToken" )
#end
#if( $context.args.filter )
  #set( $ListRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") )
#end
#if( !$util.isNull($modelQueryExpression)
                        && !$util.isNullOrEmpty($modelQueryExpression.expression) )
  $util.qr($ListRequest.put("operation", "Query"))
  $util.qr($ListRequest.put("query", $modelQueryExpression))
  #if( !$util.isNull($ctx.args.sortDirection) && $ctx.args.sortDirection == "DESC" )
    #set( $ListRequest.scanIndexForward = false )
  #else
    #set( $ListRequest.scanIndexForward = true )
  #end
#else
  $util.qr($ListRequest.put("operation", "Scan"))
#end
$util.toJson($ListRequest)

[a-j-olly]

I am also coming across this error when using next token for pagination. This seems to have began after I pushed to my development environment the same day this ticket was raised (The error wasn't spotted until today). This push included a two field change to a table in my graphql schema, where said fields had their type changed from '[String]!' to 'AWSJSON!'. Interestingly, the now failing query that uses next token for pagination reads from a different table than that which was changed.

I am using Cognito user groups for authentication, and I can provide full schema if necessary.

Here is the error that I have captured in the AppSync console:

{
  "data": {
    "articlesByDate": null
  },
  "errors": [
    {
      "path": [
        "articlesByDate"
      ],
      "data": null,
      "errorType": "DynamoDB:UserIllegalArgumentException",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
    }
  ]

And here is the query req vtl in question (not quite as tidy as the gentleman above):

## [Start] Set query expression for @key **
#set( $modelQueryExpression = {} )
## [Start] Validate key arguments. **
#if( !$util.isNull($ctx.args.datePublished) && $util.isNull($ctx.args.dataType) )
  $util.error("When providing argument 'datePublished' you must also provide arguments dataType", "InvalidArgumentsError")
#end
## [End] Validate key arguments. **
#if( !$util.isNull($ctx.args.dataType) )
  #set( $modelQueryExpression.expression = "#dataType = :dataType" )
  #set( $modelQueryExpression.expressionNames = {
  "#dataType": "dataType"
} )
  #set( $modelQueryExpression.expressionValues = {
  ":dataType": {
      "S": "$ctx.args.dataType"
  }
} )
#end
## [Start] Applying Key Condition **
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.beginsWith) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND begins_with(#sortKey, :sortKey)" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.beginsWith" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.between) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey BETWEEN :sortKey0 AND :sortKey1" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey0", { "S": "$ctx.args.datePublished.between[0]" }))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey1", { "S": "$ctx.args.datePublished.between[1]" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.eq) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey = :sortKey" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.eq" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.lt) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey < :sortKey" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.lt" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.le) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey <= :sortKey" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.le" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.gt) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey > :sortKey" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.gt" }))
#end
#if( !$util.isNull($ctx.args.datePublished) && !$util.isNull($ctx.args.datePublished.ge) )
  #set( $modelQueryExpression.expression = "$modelQueryExpression.expression AND #sortKey >= :sortKey" )
  $util.qr($modelQueryExpression.expressionNames.put("#sortKey", "datePublished"))
  $util.qr($modelQueryExpression.expressionValues.put(":sortKey", { "S": "$ctx.args.datePublished.ge" }))
#end
## [End] Applying Key Condition **
## [End] Set query expression for @key **
#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
#set( $QueryRequest = {
  "version": "2017-02-28",
  "operation": "Query",
  "limit": $limit,
  "query": $modelQueryExpression,
  "index": "byDate"
} )
#if( !$util.isNull($ctx.args.sortDirection)
                    && $ctx.args.sortDirection == "DESC" )
  #set( $QueryRequest.scanIndexForward = false )
#else
  #set( $QueryRequest.scanIndexForward = true )
#end
#if( $context.args.nextToken ) #set( $QueryRequest.nextToken = "$context.args.nextToken" ) #end
#if( $context.args.filter ) #set( $QueryRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") ) #end
$util.toJson($QueryRequest)

And the res vtl:
$util.toJson($ctx.result)

Has anyone rebuilt their environment from scratch to see if this fixes it? I would prefer not to have to do this, but when in doubt...

[apanagos]

I am waiting a week or so before I blow my test away and rebuild. I am hoping we can figure out a resolution because go live is coming quick and I cannot be rebuilding a production environment. It has taken a while to stabilize the Amplify environments and make sure everything is done in a certain order so env don't get blown up.

[malcomm]

Just a bit of info, I was running into this as well after I upgraded from amplify-cli 4.13.1 to 4.13.2. Once I downgraded to 4.13.1 and did another amplify push life was better.

[bengsiswantoh]

Thank you @malcomm

[olivoil]

Having the same issue with the latest CLI version (4.13.3) on any @model list query. Forcing version 4.13.1 of the CLI in build settings -> Live package updates does not seem to solve it for me, and I could not test 4.13.0 as there's another issue with this version that was fixed in 4.13.1

It happens with any @model, and is reproducible directly from the AppSync Console.

Generate a simple @model

type Todo @model @auth(rules: [{ allow: owner }]) {
  id: ID!
  owner: ID!
  text: String
  checked: Boolean
}

Create a few objects, and query them, copying the value of nextToken verbatim from a previous query directly in AppSync Console

query ListTodos {
  listTodos(limit: 2, nextToken: "ey...SRiJ9") {
    items {
      id
      text
      checked
    }
    nextToken
  }
}

The error shows up consistently

{
  "data": {
    "listTodos": null
  },
  "errors": [
    {
      "path": [
        "listTodos"
      ],
      "data": null,
      "errorType": "DynamoDB:UserIllegalArgumentException",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
    }
  ]
}

[a-j-olly]

@malcomm For me, this issue was occurring on CLI version 4.13.1, however once I downgraded to 4.13.0 it cleared up. I did try and upgrade to 4.13.2 to see if that might be a fix, but I could still reproduce the error.

@olivoil I'd recommend taking the CLI down to 4.13.0 and then doing a amplify push and see if that works.

Hopefully the Amplify team can get to the bottom of this, but I have a work around for now.

[apanagos]

@a-j-olly I am on Amplify 4.5.0 and I haven't upgraded for a while, so there were no environment changes. Though saying that the problem I am having is with an environment that is built by "AWS Amplify" from GIT. I assume they would be using the latest version of Amplify when building the backend. I will add an echo to the amplify.yml to see what version they are using.
BUT saying that my build environment is also built by "AWS Amplify" so I would expect the same issue if it was a version issue. But BUILD works (and the DEV environments I built by hand work) but TEST does not.

[dtelaroli]

+1

same problem with @key and custom queryField and simple listModel query.
amplify 4.13.1

[SwaySway]

Hi @apanagos
I'm trying to reproduce this though I'm not able to
this is the schema I am trying on version 4.5.0

type Todo @model @auth(rules: [{ allow: owner }]) {
  id: ID!
  owner: ID!
  text: String
  checked: Boolean
} 

I'm running this in us-west-2.

If possible could you provide the schema you are using, here or email [email protected], and provide the region this api exists.

[apanagos]

Hi @SwaySway, problem is that I am experiencing the problem with all the tables in my TEST environment.

Here is a simple one that is causing issues:

type Tags @model @aws_cognito_user_pools @aws_iam{
  id:ID!
  tag:String!
} 

I am running this in us-east-1

Let me go through the process of how I promote in case the problem is there.

git checkout test
git merge build
amplify env checkout testedge
amplify env pull
amplify push

once its is finished i do the following:

git add *
git commit -m "upgrade"
git push

Then the build automatically starts in "AWS Amplify" as I have set it up to build from updating the branch.

Some other things to take into account.

My command shell has amplify at version 4.5.0
BUT when it builds in "AWS Amplify" I guess it uses the latest version of Amplify CLI?

Just a little more info. I follow the same procedure above for my other environments.

e.g.

git checkout build
git pull
git merge dev
amplify env checkout buildaaa
amplify env pull
amplify push
git add *
git commit -m "update"
git push

One last piece of information. The follwoing is the build file I have checked into GIT and is used to build in "AWS Amplify"

file: amplify.yml

version: 0.5
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - echo "User $USER_BRANCH"
        - echo "AWS User $AWS_BRANCH"
        - amplifyPush --environment $USER_BRANCH --simple
frontend:
  phases:
    preBuild:
      commands:
        - echo "Front End Prebuild"
        - npm install --global gulp-cli
        - npm ci
    build:
      commands:
        - gulp
        - npm run build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

If you need anything else, please just reach out and I will help as much as i can.

Thanks

[slatemates]

I have tried deploying a test api through both Amplify Version(4.13.3 and 4.13.2) with the following schema :

type TODO @model {
id: ID!
amount : Int!
}

The resolver generated using amplify CLI

Query.listTODOs.req.vtl

#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
#set( $ListRequest = {
  "version": "2017-02-28",
  "limit": $limit
} )
#if( $context.args.nextToken )
  #set( $ListRequest.nextToken = $util.toJson($context.args.nextToken) )
#end
#if( $context.args.filter )
  #set( $ListRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") )
#end
#if( !$util.isNull($modelQueryExpression)
                        && !$util.isNullOrEmpty($modelQueryExpression.expression) )
  $util.qr($ListRequest.put("operation", "Query"))
  $util.qr($ListRequest.put("query", $modelQueryExpression))
  #if( !$util.isNull($ctx.args.sortDirection) && $ctx.args.sortDirection == "DESC" )
    #set( $ListRequest.scanIndexForward = false )
  #else
    #set( $ListRequest.scanIndexForward = true )
  #end
#else
  $util.qr($ListRequest.put("operation", "Scan"))
#end
$util.toJson($ListRequest)

We keep getting the same error even when running queries with Appsync console directly . This issue needs to be addressed urgently . Proper testing must be ensured before releasing new
versions.

[clodal]

As per @a-j-olly's comment above, can confirm it works when I downgraded the cli from 4.13.2 to 4.13.0 and do amplify push.

[a-j-olly]

Just an update.

Having downgraded to 4.13.0 and finding success, I was annoyed to find out this morning that once again the problem has cropped up. I have not made any alterations whatsoever since I did the downgrade and amplify push.

I'm getting the same error:

{
  "data": {
    "articlesByDate": null
  },
  "errors": [
    {
      "path": [
        "articlesByDate"
      ],
      "data": null,
      "errorType": "DynamoDB:UserIllegalArgumentException",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
    }
  ]
}

Pagination is a critical feature of GraphQL. Fixing this issue is of vital importance.
If you need anymore info see my above posts or PM me.

[grigull]

I ran into the same problem when I went to 4.13.3, downgraded to 4.13.1 which still works for me.

[a-j-olly]

So I just did another 'empty' push, and it has cleared up again. I think I'm just going to take a break...

[kaustavghosh06]

We've root-caused the issue with the latest release and working on a fix for this which will be released soon.
cc: @yuth

[dimonchik-com]

Have the same problem
com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given. https://github.com/aws-amplify/amplify-js/issues/4942

[AshwiniRaysecur]

I attempted to build the application again from scratch, and still getting the pagination error for Amplify version 4.13.3

The resolver generated 'Query.listSecurSamples.req.vtl' file:

#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
#set( $ListRequest = {
  "version": "2017-02-28",
  "limit": $limit
} )
#if( $context.args.nextToken )
  #set( $ListRequest.nextToken = "$context.args.nextToken" )
#end
#if( $context.args.filter )
  #set( $ListRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") )
#end
#if( !$util.isNull($modelQueryExpression)
                        && !$util.isNullOrEmpty($modelQueryExpression.expression) )
  $util.qr($ListRequest.put("operation", "Query"))
  $util.qr($ListRequest.put("query", $modelQueryExpression))
  #if( !$util.isNull($ctx.args.sortDirection) && $ctx.args.sortDirection == "DESC" )
    #set( $ListRequest.scanIndexForward = false )
  #else
    #set( $ListRequest.scanIndexForward = true )
  #end
#else
  $util.qr($ListRequest.put("operation", "Scan"))
#end
$util.toJson($ListRequest)

Worked correctly after downgrading to Amplify version 4.13.0

[kaustavghosh06]

Could you please try this out with 4.13.4? We fixed this in our latest version

[nick-secur]

@kaustavghosh06 Please note we (i.e., myself and @AshwiniRaysecur )have tested publishing our app with version 4.13.4 and the issue seems to be resolved. Smoke tests show our app once again behaves as expected.

[malcomm]

I just upgraded to version 4.13.4 and things appear to be working for me as well. I'll keep folks posted if otherwise. Thanks!

[apanagos]

We tested it today and I can confirm that the fix does work. Thank you @kaustavghosh06 , @SwaySway , @AaronHarris , @sammartinez , @yuth , @attilah for addressing the issue so quickly.
Now I have made some changes to our process. I changed "AWS Amplify" to not use the latest version of Amplify but to use a versioned (4.13.4) so it is in-sync with what we are developing with. As we upgrade and test our version of Amplify then we will change the "AWS Amplify" version so we are not surprised.
Thanks Again.

[slatemates]

Upgraded to 4.13.4 . The issue is resolved .Thanks

On Fri, 21 Feb 2020, 03:03 Andrew Panagos, [e-mail] wrote:

We tested it today and I can confirm that the fix does work. Thank you
@kaustavghosh06 https://github.com/kaustavghosh06 , @SwaySway
https://github.com/SwaySway , @AaronHarris
https://github.com/AaronHarris , @sammartinez
https://github.com/sammartinez , @yuth https://github.com/yuth ,
@attilah https://github.com/attilah for addressing the issue so quickly.
Now I have made some changes to our process. I changed "AWS Amplify" to
not use the latest version of Amplify but to use a versioned (4.13.4) so it
is in-sync with what we are developing with. As we upgrade and test our
version of Amplify then we will change the "AWS Amplify" version so we are
not surprised.
Thanks Again.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/aws-amplify/amplify-cli/issues/3411?email_source=notifications&email_token=AE5UPKLAYER57S6PXF6AYT3RD3ZMJA5CNFSM4KT7CL52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMQHIIY#issuecomment-589329443,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE5UPKMMSR7TTIIIRJG3L4TRD3ZMJANCNFSM4KT7CL5Q
.

[oliverandersencox]

This is still happening for me on 4.13.4

[SwaySway]

@oliverandersencox Are you using @key in your schema?

[oliverandersencox]

@SwaySway yes I am

[SwaySway]

@oliverandersencox Could you try the workaround mentioned in #3493?
If it is related to that please follow that issue for updates.

[rarira]

This is still happening for me on 4.13.4

same here, downgraded to 4.13.1 and it works

[WerewolFox]

I just wanted to confirm that I had this issue with 4.13.3 but upgrading to 4.13.4 resolve it for me. The steps I took were:

• npm install -g @aws-amplify/cli@latest
• make a simple change to the schema (I added a field)
• amplify api gql-compile
• amplify push
  That was it.. works fine now.. thanks guys.

[mithunkalan]

@WerewolFox
Same problem. same fix.
Except that i didnt need amplify api gql-compile

[philateatclub]

I just wanted to confirm that I had this issue with 4.13.3 but upgrading to 4.13.4 resolve it for me. The steps I took were:

• npm install -g @aws-amplify/cli@latest
• make a simple change to the schema (I added a field)
• amplify api gql-compile
• amplify push
  That was it.. works fine now.. thanks guys.

I had the same issue. However I was on Amplify --version 4.13.4. Bumped it to the latest --version 4.17.1.

• amplify api gql-compile
• amplify push

That fixed it for me!

[tomasyaya]

Hello @SwaySway
I'm having this issue even when I try the query from the Appsync console.
My type is NOT using the @key directive.
I'm using the latest Amplify CLI 4.17.2
Could you please address my issue?
Thank you!

UPDATE
Never mind, worked in the next push

[PrometheanGeethika]

Hello,
This issue is back with the latest (4.18.1) Amplify CLI Version. Overriding the version with 4.16.1 (which is the one I happen to know is working) as described at https://docs.aws.amazon.com/amplify/latest/userguide/custom-build-image.html#setup-live-updates no longer throws this error.

[iShavgula-TacTill]

Hello,
I'm having this issue with amplify 4.25.0.

[lqthaiegi]

I have the same issue. I'm using version 4.13.4. Base on some comments above I upgraded to 4.27.3, it works fine for me.

[iShavgula-TacTill]

I'm still having this issue, v: 4.29.7.
It has been months, impossible to to use Amplify in production 👎

[SwaySway]

Hello @iShavgula-TacTill
Could you provide a schema which can reproduce this scenario?

[iShavgula-TacTill]

Thank you for having a look at this.

This happens for every DynamoDB Query (2018-05-29) which goes via Pipeline Resolvers and uses GSI.
I tried to delete and recreate these queries many times, but that does not help, it can be reproduced every time with every query following these patterns. Here is a (pseudo) code :

#set( $QueryRequest = {
  "version": "2018-05-29",
  "operation": "Query",
  "limit": $limit,
  "query": some expression,
  "index": "index name"
} )
#if( !$util.isNull($ctx.args.sortDirection) && $ctx.args.sortDirection == "DESC" )
  #set( $QueryRequest.scanIndexForward = false )
#else
  #set( $QueryRequest.scanIndexForward = true )
#end
#if( $context.args.nextToken ) 
  #set( $QueryRequest.nextToken = $util.toJson($context.args.nextToken) ) 
#end
#if( $context.args.filter ) 
  #set( $QueryRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") ) 
#end
$util.toJson($QueryRequest)

Please let me know if you need more information.

P.S. I did check that nextToken is correctly used in my pipelines, it was working fine until it just stopped working after one of the Amplify cli updates.

[SwaySway]

@iShavgula-TacTill
Currently the CLI does not support pipeline resolvers. Do you have pipeline resolvers setup by overriding the generated resolvers? If so the fix you could apply here is to change the query resolver to remove $util.toJson(). This is what the transformer currently generates for query resolvers that use nextToken.

#if( $context.args.nextToken ) 
  #set( $QueryRequest.nextToken = $context.args.nextToken ) 
#end

[iShavgula-TacTill]

That fix did work, but I find that very strange because of couple of reasons :

• I had some auto-generated queries which used toJSON() for nextToken before commenting on this issue.
• I have not changed DynamoDB Query version

Could that be linked to AppSync utilities' updates ?
Any tips how to avoid this kind of issues ?

Thank you

[yuth]

In an earlier version of the Amplify CLI a bug was introduced which double encoded the payload toJSON. What I suspect is you might have overridden the template when the CLI had the bug, which got you the buggy resolver. This was fixed in the CLI later, but since the template was overriden, that fix not propagate to that template.