Amplify-cli: Auto-generated list resolver templates limits before filtering

[artista7]

It was kinda weird discovering that auto-generated appsync list query resolvers limit the scanned result before it filters. Part of this weirdness could be attributed to ignorance on dynamo usage on my part but part of this is also due to un-intuitive behavior which was not expected.

I ran this particular query :-

query ListRounds{
  listRounds(
    limit:10,
    filter:{
        jobOpeningId:{
            eq:"jobOpening-Company1College1drive1"
      }
    })
  {
    items{
      id
    }
    nextToken
  }
}

It did not return rounds, even though they exist as the filter was applied on the limited result.

The default request resolver is:-

#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
#set( $ListRequest = {
  "version": "2017-02-28",
  "limit": $limit
} )
#if( $context.args.nextToken )
  #set( $ListRequest.nextToken = "$context.args.nextToken" )
#end
#if( $context.args.filter )
  #set( $ListRequest.filter = $util.parseJson("$util.transform.toDynamoDBFilterExpression($ctx.args.filter)") )
#end
#if( !$util.isNull($modelQueryExpression) && !$util.isNullOrEmpty($modelQueryExpression.expression) )
  $util.qr($ListRequest.put("operation", "Query"))
  $util.qr($ListRequest.put("query", $modelQueryExpression))
#else
  $util.qr($ListRequest.put("operation", "Scan"))
#end
$util.toJson($ListRequest)

Q1 - What is $modelQueryExpression in this template?

Q2 - How can this template be modified to use query expression with indexName i.e query instead of scan?

What AWS Services are you utilizing?
aws-amplify version 1.7.8

[RossWilliams]

When you apply a filter it will cause a scan operation in dynamodb. scans will read all items in your table up to the specified limit count (or 1MB scanned), and applies a filter only after the limit is reached.

It looks like you want to get a list of items by an id property jobOpeningId. The way to efficiently do this is as you suggest, with a dynamo global secondary index, with jobOpeningId as the hash key.

Using the key directive, you can specify @ key( fields["jobOpeningId", "XXX"], name: "roundsByJobOpening", queryField: "roundsByJobOpening"). This will give you a new index to efficiently get just the items you need, a new top level query, and I highly recommend a sort key (the "XXX") so that you know the order of the items coming back. Then pagination will occur as you expect as there is no filtering.

[artista7]

Hey @RossWilliams I already had a GSI with following details:-
gsi-JobOpeningRounds | Active | GSI | jobOpeningId (String) | createdAt (String) .

Though this was created using @connection directive.

type round @model @key(name: "nextRoundIdx", fields: ["nextRound", "jobOpeningId"]){
  id: ID!
  jobOpeningId: ID!
  jobOpening: jobOpening @connection(name: "JobOpeningRounds", keyField: "jobOpeningId", sortField: "createdAt")
  applications: [application] @connection(name: "ApplicationInRound", keyField: "roundId", sortField: "createdAt")
  nextRound: ID
  name: String
  canEdit: Boolean
  canDelete: Boolean
  isInterview: Boolean
  startTime: String
  endTime: String
  deadline: String
  url: String
  manager: String
  createdAt: String
  updatedAt: String
}

and then I ran this query:-

query ListRounds{
  listRounds(
    limit:10,
    filter:{
        jobOpeningId:{
            eq:"jobOpening-Company1College1drive1"
      }
    })
  {
    items{
      id
    }
    nextToken
  }
}

Does the query needs to be modified?

Also do you have any clue about $modelQueryExpression in request mapping template?
@kaustavghosh06

[RossWilliams]

It might be an easier option in your case to query by JobOpening and get the rounds as a child of the job opening, since it appears you know the id of the job opening. Behind the scenes this query will use the created gsi. This option has a second added benefit that your JobOpening object is more likely to have authentication properties on it, and you can lock down access.

query JobOpening {
  getJobOpening(id: "jobOpening-Company1College1drive1") {
    rounds {
      items {
        id
      }
  }
}

[RossWilliams]

Regarding Q1, that is an object that holds data only if you are doing a query operation in dynamodb. Its properties are populated here, it is set as the parameter 'queryExprReference' Because you are doing a scan operation, this value should not be set. If you expect your tables to be large, or be accessed frequently you will almost always want to be doing a query rather than a scan.

[attilah]

@RossWilliams thanks for answering @artista7's questions.

@artista7 it seems to me that your question was answered, I'm closing the issue, but feel free to reopen if something new comes up regarding this.