Amplify-cli: GraphQL API @auth list items query give inconsistent and wrong results

[jaxondu]

Describe the bug
Amplify API Add GraphQL with @auth schema, query listing of items give inconsistent and wrong results.

To Reproduce

1. Create a new Amplify project with amplify init
2. amplify add api and select GraphQL
3. Setup API with Cognito User Pool authoriser
4. Use the following schema:
   type Note
   @model
   @auth(rules: [{allow: owner, ownerField: "user"}])
   {
   id: ID!
   content: String
   }

1. amplify push
2. Go to AppSync console > Query. Login as user ABC.
3. Run Mutation createNote to create 3 notes.
4. Login as user XYZ.
5. Run Mutation createNote to create 10 notes.
6. Login back as user ABC.
7. Run the following:
   query ListNotes {
   listNotes(limit: 5) {
   items {
   id
   content
   }
   }
   }
8. This will give empty result:
   {
   "data": {
   "listNotes": {
   "items": []
   }
   }
   }
9. Run the following and will get back the 3 items created under user ABC.
   query ListNotes {
   listNotes(limit: 20) {
   items {
   id
   content
   }
   }
   }

Expected behavior
The reason for the bug is due to Query.listNotes.request resolver not selectively query and use DynamoDB Scan to retrieve all records. And then do the filtering at Query.listNotes.response. The result can be bizarre for example when you login as XYZ and query with a limit of 10, you get back only 8 items instead of 10.

Additional context
amplify --version
0.1.29

[mikeparisstuff]

@jaxondu I believe you may be confused about how DynamoDB scan operations work. If you read here (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html). The filter expression in a scan happens after the data is read from the partition. This means that if I say limit: 10, DynamoDB goes to the partition, reads 10 items, then applies the filter and discards any items that do not apply. This is what is happening. It is not an effect of doing the filtering in the response mapping template.

To get the behaviour you are expecting, we need to solve this feature request https://github.com/aws-amplify/amplify-cli/issues/56. To get all the posts that belong to your "user", you need to create a GSI on the user attribute and then use a top-level query field against that GSI. We are planning on allowing you to define your own key schema and GSIs, but the functionality is not there yet. Once this is in place, you can paginate through all posts with a userId=10, for example, and since the Query will hit a GSI where the userId is the hash key it can efficiently query the desired items.

[mikeparisstuff]

If you look at that same issue #56, the easiest way to move forward would be to solve for option 2 that I mention in my comment. With this solution, you would be able to do this:

type Note
  @model
  @auth(rules: [{allow: owner, ownerField: "user"}])
{
  id: ID!
  content: String
  notes: [Notes] @connection(name: "UserNotes", keyField: "user")
}

type User @model {
  id: ID!

  # Create the connection (which will create the GSI)
  notes: [Note] @connection(name: "UserNotes", keyField: "user")
}

Then you would be able to create a user like this:

mutation CreateUser {
  createUser(input: {id: "[email protected]"}) {
    id
  }
}

then create a note like this:

mutation CreateNote {
  createNote(input: { content: "Note content" }) {
    id
    content
  }
}

which would automatically set the owner field to the logged in users username. You could then efficiently get all a users notes like this:

query GetUserAndNotes {
  getUser(id: "[email protected]") {

    # This hits the GSI so all records will be relevant and pagination
    # will work as you expected.
    notes(limit: 10) {
      items {
        id
        content
      }
    }
  }
}

[jaxondu]

@mikeparisstuff guess I have the wrong understanding of the purpose of @auth.

[jaxondu]

@mikeparisstuff @yuth am I right to say that @auth in Amplify CLI is not suitable to be used for Todo/Note app with login? Those app with user login, list, create, delete own todos/notes. I was trying to create such app while learning AppSync. Guess I have to create resolvers manually then.

[mikeparisstuff]

@auth is responsible for protecting access to objects which it is doing correctly here. You are commenting on DynamoDB query/scan semantics which can, admittedly, be confusing esp if you are coming from a SQL background. With DynamoDB, you need to think about your query patterns when you create tables. You can create GSIs after creating the table but the fact remains that you have to configure the table on the server in order to support your query pattern.

If you don't have a GSI on the user field there is simply no way to guarantee that a scan

fetch 10 objects where the user = "[email protected]"

will return 10 objects. If you need 10 objects that filter on a non-indexed field, the solution is to paginate until you get 10 objects and increase the limit size to increase the rate of doing so.

With this in mind, the solution to this using the transform and CLI is by doing what I describe above. I will work on implementing option 2 from #56 because its evident that this is a pain point. Once that is fixed, you can use @auth alongside @connection to efficiently query GSIs that also adhere to ownership auth during mutations. We are not going to open up top-level query fields quite yet as that is a long-term feature, but we are still working on supporting custom resolvers from the CLI which will solve for any and all use cases we don't think of.

[mikeparisstuff]

Support for what I describe above is added in #356. There is a test case in this file (https://github.com/aws-amplify/amplify-cli/pull/356/files#diff-ba335004cbb001babdb0ce37a4ce2945) that shows how you can use @auth and @connection to easily create relationships between objects and their owners.

[jaxondu]

Thanks will take a look.

[kaustavghosh06]

This issue is fixed and would be a part of the next CLI version (0.1.30).

[jaxondu]

@mikeparisstuff @kaustavghosh06 @yuth @undefobj this article by your colleague in AWS is having the same thought about @auth and same error in listNotes as I encountered in this issue: https://medium.com/open-graphql/create-a-multiuser-graphql-crud-l-app-in-10-minutes-with-the-new-aws-amplify-cli-and-in-a-few-73aef3d49545 I have informed the article author via Twitter. It could be due to some major change in @auth that the author is not aware of. If @auth is not meant to be used this way in a CRUD app, It would be kind of AWS not to spread the wrong usage.