Amplify-cli: AWSJSON update not working correctly

[oliverandersencox]

Describe the bug
I have a model with the attribute 'timelines' set to the type timelines: AWSJSON.

This field is designed to hold an array of objects.

When i create the initial object, everything works as it should do. I first stringify the bject and them push it to the database by updating the parent model table.. in this case 'Event'.

If I then change the object in any way, rather than updating the value in the database, it appends the same object with the new values onto the array.

for example, the first push to timelines with an array with one object in creates the following:
Event:

{ 
   title: 'blah',
    timelines: "[{ id: 1, title: 'hello' }]"
}

If i then run the update:

result = await this.amplify.api().graphql(graphqlOperation(updateEvent, { input }));

with the input being:

{ 
   title: 'blah',
    timelines: "[{ id: 1, title: 'test' }]"
}

The resulting change is that the database displays:

{ 
   title: 'blah',
    timelines: "[{ id: 1, title: 'hello' }, { id: 1, title: 'test' }]"
}

Amplify CLI Version
4.18.0

Expected behavior
It should behave like any other update operation.
if i send an update with the timelines array, the array being send should be the value. I don't understand why it would append to the array!

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
MAC OSX Catalina

Additional context
I have checked the network tab to make sure the stringified value under timelines is correct, and it is, and so the graphql dynamo system must be dong something.

[oliverandersencox]

@SwaySway I take it back, it now seems that any array child of a parent model, when updated will duplicate the current values in the array. Any child on my event Model which is an sarray of types is doing it now. This is with parts of the code I havent even touched.

All network calls show the data being handed over is correct. For example if the current event.info field is ['test'] and I run the update event method with { info: ['testreplace'] } it will append it to the array even though im telling it to put a new array there!

[edwardfoyle]

@oliverandersencox could you share your schema and the update operations you are executing?
With the following schema:

type JsonTest @model {
  id: ID!
  json: AWSJSON!
}

If I execute CreateItem and then UpdateItem

mutation CreateItem {
  createJsonTest(input: {id: "myid", json:"[{\"key1\": \"value1\"}]"}) {
    id
    json
  }
}

mutation UpdateItem {
  updateJsonTest(input: {id: "myid", json: "[{\"key2\": \"value2\"}]"}) {
    id
    json
  }
}

The response after the UpdateItem is

{
  "data": {
    "updateJsonTest": {
      "id": "myid",
      "json": "[{\"key2\":\"value2\"}]"
    }
  }
}

which seems to be as expected

[oliverandersencox]

@edwardfoyle I did exactly that and i ended up with:

{
  "__typename": "JsonTest",
  "_lastChangedAt": 1586985005893,
  "_version": 2,
  "createdAt": "2020-04-15T21:09:36.487Z",
  "id": "myid",
  "json": [
    {
      "key1": "value1"
    },
    {
      "key1": "value2"
    }
  ],
  "updatedAt": "2020-04-15T21:09:36.487Z"
}

Any ideas as to how this is possible?

[oliverandersencox]

My event schema (There are over 15 schema files:

type Event @model(subscriptions: null)@searchable 
  @auth(rules: [
    { allow: owner },
    { allow: owner, ownerField: "owners" },
    { allow: groups, groups: ["users"], operations: [read] },
    { allow: public, provider: iam, operations: [read] },
    { allow: groups, groups: ["admin"] }
  ])
  @key(name: "EntityEventsKey", fields: ["entityId", "dateTime", "status"]) 
  @key(name: "publisherEventsKey", fields: ["publisherId", "dateTime", "status"]) {
  id: ID!
  title: String!
  entityId: ID!
  publisherId: ID!
  subtitle: String
  description: String
  type: String
  score: Float @auth(rules: [
    { allow: owner, operations: [create] }, 
    { allow: groups, groups: ["admin"] },
    { allow: groups, groups: ["users"], operations: [read] }
  ])
  address: Address
  categories: [String]
  dateTime: Float!
  startDate: AWSDateTime!
  endDate: AWSDateTime!
  closingTime: Float!
  rules: String
  buyTicketUrl: AWSURL
  canBook: Boolean
  featured: String
  video: AWSURL
  images: [String]
  artistsList: [String]
  mapImages: [String]
  location: Location
  price: String
  city: String!
  features: [Feature]
  attending: Int
  extraInfo: [AccordionData]
  promoterIds: [String]
  ttl: Float
  views: Int
  whatsappNumber: AWSPhone
  phone: AWSPhone
  status: String
  owner: ID
  owners: [ID]
  ownerType: String
  entity: Entity @connection
  artists: [ArtistEvents] @connection(keyName: "byEvent", fields: ["id"])
  tables: [Table] @connection(keyName: "eventTablesKey", fields: ["id"])
   timelines: [Timeline]
  checklist: Checklist
  tablesAvailable: Int
  createdAt: Float!
  updatedAt: Float!
}

[oliverandersencox]

@edwardfoyle is seems to be any array nesting which is having this issue, and this is with every single model in my schema!

[edwardfoyle]

@oliverandersencox Can you send the schema of the model that includes the AWSJSON datatype and the exact graphql mutation that you are executing?

[oliverandersencox]

as stated above, its not specific to AWSJSON. I am experiencing it on every single array value within a model.

const input {
id: *,
timelines: [{ id: , title: 'title' }],
updatedAt: **
}

await this.amplify.api().graphql(graphqlOperation(updateEvent, { input }));

  export const updateEvent = /* GraphQL */ `
  mutation UpdateEvent(
    $input: UpdateEventInput!
    $condition: ModelEventConditionInput
  ) {
    updateEvent(input: $input, condition: $condition) {
      id
    }
  }
`

The Event schema above, any of those arrays have the same problem when trying to update using this query

[oliverandersencox]

also happening if i try and update the field: extraInfo: [AccordionData]

[edwardfoyle]

@oliverandersencox I'm still unable to reproduce this behavior. I've tried with the following schema and every mutation operation I've tried produces the expected array output

type Test @model {
  id: ID!
  sub: [Another]
}

type Another {
  id: ID!
  values: [String]
}

Was it working as expected before and just started behaving this way? If so, did you change anything about your models between when it was and wasn't working?

[oliverandersencox]

I had to delete my entire api 2 days ago and redeployed using the exact same schema, and yes it was all working fine before this.

[edwardfoyle]

The behavior you're seeing seems to be consistent with the list_append option in DynamoDB but I have no idea how that would be getting into your resolvers. Can you see if the string "list_append" is found anywhere in your amplify folder and if so, attach the file here.

[oliverandersencox]

I thought this as well, and so looked at the build folder in amplify at the mutation update resolvers and can't see anything like this, and running a search for 'list_append' comes up empty.

It has to be a resolve or dyanmo itself due the to fact I've checked countless time, both the input data for the query and the query post within the network tab in chrome and they are always correct.

If i try and set the timelines array, which is now 12 items long after a few updates, to null, the query goes through and succeeds, yet had no impact at all on the array value.

It must be this list append being activated, but where?!?

(Thanks for your help so far)

[oliverandersencox]

I have tried again all morning to figure this out, and no luck.

I Have now send an entire event object to the update method, which should replace every value in the current event, with the new one. It does indeed, however, still, for any arrays it is appending them.

The input object being passed to the update method is completely new
The network call shows this exact object being sent to amplify
Yes the update appends on every array!

If i send a null value to be set as the array value, it simple does nothing, yet the request succeeds with no effect. Th array still has all its values

The resolver file is huge so its very hard to find anything useful that my point towards the issue there. @edwardfoyle do you have any idea what else I should be looking for in the resolver file?

[oliverandersencox]

it seems this issue is effecting everything. Now i have a simple field under 'Event' called 'checklist' which has a title and 'listItems' array of strings:

I updated this with values in the array.. and this is the network call from inspecting in chrome:

query: "mutation UpdateEvent($input: UpdateEventInput!, $condition: ModelEventConditionInput) {↵  updateEvent(input: $input, condition: $condition) {↵    id↵  }↵}↵"
variables: {input: {id: "a0f8bf9e-9833-49d1-bdb4-e1edbb4a8b7f", updatedAt: 1587044032122,…}}
input: {id: "a0f8bf9e-9833-49d1-bdb4-e1edbb4a8b7f", updatedAt: 1587044032122,…}
id: "a0f8bf9e-9833-49d1-bdb4-e1edbb4a8b7f"
updatedAt: 1587044032122
checklist: {title: "Test", listItems: ["Test Item 1", "gbfgdf"]}

Yet the database does not change, the value under listitems is null!

i havent touched the resolvers and so I cant understand this behaviour

[edwardfoyle]

@oliverandersencox this is really strange and if there's no mention of list_append in your resolvers, I don't think there's much the CLI can help you with. I think it may be some issue in AppSync itself. I can cut a ticket to them on your behalf if you send the following information to [email protected]:

1. Your AWS account ID
2. The GraphQL API ID
3. The AWS region of the API
4. Any request IDs or approximate time ranges where you issued queries that resulted in the errors

While you're at it, if you want to zip your project and attach it to the email I can take a look and see if anything looks off.

You can also try posting on the AppSync forum but I'm not sure how likely you are to get a response there.

[oliverandersencox]

@edwardfoyle thanks, email sent!

[edwardfoyle]

Okay I've sent a ticket to AppSync. If you can't zip your whole project, can you send a zip of just your update request resolvers (located in amplify/backend/api/<apiName>/build/resolvers). They'll be named Mutation.update<Entity>.req.vtl

[oliverandersencox]

@edwardfoyle I have just sent a zip with those update resolvers in

[oliverandersencox]

@edwardfoyle is there any update on this? Thanks

[edwardfoyle]

No haven't heard anything back and I didn't see anything suspicious in your resolvers that you sent. I did see there is some usage of the ADD DynamoDB operation in your resolvers but it doesn't look like that codepath would ever be executed and I also have the ADD operation in my resolvers where the issue does not repro.

One other thing you can try is creating a new environment with amplify env add and then push that environment and see if the issue persists.

[oliverandersencox]

@edwardfoyle I deleted the api and recreated it, however this time I chose not to add conflict resolution as the previous working api didn't have it, and hey presto the issue has disappeared.

so I presume the versioning has something to do with the issue

[edwardfoyle]

Okay, that's good information. Will keep digging to figure out the root cause.

[Darapsas]

Are there any updates on this? I am having the same issue and all aws packages are updated to newest versions. I think it is a little bit an overkill to delete the whole api and recreate it without conflict resolution.

[adhikarisandeep]

I'm having the same issue as well. Anyone has any more information on the resolution other than deleting and recreating the api?

[Rafcin]

I'd like to bump this, after editing the api and removing conflict detection like above the issue still persists

[yuth]

@oliverandersencox, @Darapsas , @adhikarisandeep , @Rafcin It looks like this happens when in the data store enabled API.

When data store is enabled for an API, the conflict detection and resolution is set to AUTOMERGE. As documented here

Automerge

Automerge provides developers an easy way to configure a conflict resolution strategy without writing client-side logic to manually merge conflicts that were unable to be handled by other strategies. Automerge adheres to a strict rule set when merging data to resolve conflicts. The tenets of Automerge revolve around the underlying data type of the GraphQL field. They are as follows:

• Conflict on a scalar field: GraphQL scalar or any field that is not a collection (i.e List, Set, Map). Reject the incoming value for the scalar field and select the value existing in the server.
• Conflict on a list: GraphQL type and database type are lists. Concatenate the incoming list with the existing list in the server. The list values in the incoming mutation will be appended to the end of the list in the server. Duplicate values will be retained.
• Conflict on a set: GraphQL type is a list and database type is a Set. Apply a set union using incoming the set and the existing set in the server. This adheres to the properties of a Set, meaning no duplicate entries.
• When an incoming mutation adds a new field to the item, merge that on to the existing item.
• Conflict on a map: When the underlying data type in the database is a Map (i.e. key-value document), apply the above rules as it parses and processes each property of the Map.

Automerge is designed to automatically detect, merge, and retry requests with an updated version, absolving the client from needing to manually merge any conflicting data.

If you want to update the JSON record, you can explictly pass the _version field in the input which ensures that the data is not automerged if the _version field in the input matches to that of one in your database

mutation UpdateItem {
  updateJsonTest(input: {id: "myid", json: "[{\"key6\": \"value6\"}]", _version: 5}) {
    id
    json
    _version
  }
}

[stale[bot]]

This issue has been automatically closed because of inactivity. Please open a new issue if you are still encountering problems.