Amplify-js: DataStore is locally created but AppSync does not sync.

Created on 27 May 2020  ·  135Comments  ·  Source: aws-amplify/amplify-js

Describe the bug
Going through the stages of this article

As a result, data is created, but it is not available in App Sync and as a result, there is no sync of this data between devices.

To Reproduce
Step by step

Expected behavior
Data synchronization between devices and App Sync

Screenshots
Create a record. Locally she is.

Simulator Screen Shot - iPhone 8 - 2020-05-27 at 22 34 16

App Sync doesn't have any
Снимок экрана 2020-05-27 в 22 35 30

What is Configured?
If applicable, please provide what is configured for Amplify CLI:

  • Which steps did you follow via Amplify CLI when configuring your resources.
  • Which resources do you have configured?

    • If applicable, please provide your aws-exports file:

      ```

      const awsmobile = {

      "aws_project_region": "eu-west-2",

      "aws_cognito_identity_pool_id": "eu-west-2:6ff27293-d1c2-47db-a15b-511a2ce5be67",

      "aws_cognito_region": "eu-west-2",

      "aws_user_pools_id": "eu-west-2_tESCC3apu",

      "aws_user_pools_web_client_id": "7j3c9eg6jjoukglkpjjempjbq3",

      "oauth": {},

      "aws_appsync_graphqlEndpoint": "https://xkan4dl5wzej3l4fd5rmtwtr5a.appsync-api.eu-west-2.amazonaws.com/graphql",

      "aws_appsync_region": "eu-west-2",

      "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS"

      };

  • If applicable, provide more configuration data, for example for Amazon Cognito, run aws cognito-idp describe-user-pool --user-pool-id us-west-2_xxxxxx (Be sure to remove any sensitive data)


Environment

 System:
    OS: macOS 10.15.4
    CPU: (8) x64 Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
    Memory: 49.67 MB / 8.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 13.12.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.4 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  Browsers:
    Chrome: 83.0.4103.61
    Firefox: 76.0.1
    Safari: 13.1
  npmPackages:
    @aws-amplify/core: ^3.2.3 => 3.2.3 
    @aws-amplify/datastore: ^2.1.2 => 2.1.2 
    @babel/core: ^7.9.0 => 7.9.0 
    @babel/runtime: ^7.9.2 => 7.9.2 
    @react-native-community/async-storage: ^1.11.0 => 1.11.0 
    @react-native-community/eslint-config: ^1.0.0 => 1.0.0 
    @react-native-community/masked-view: ^0.1.8 => 0.1.8 
    @react-native-community/netinfo: ^5.9.2 => 5.9.2 
    @react-navigation/native: ^5.1.5 => 5.1.5 
    @react-navigation/native-stack: ^5.0.5 => 5.0.5 
    @react-navigation/stack: ^5.2.10 => 5.2.10 
    amazon-cognito-identity-js: ^4.2.1 => 4.2.1 
    aws-amplify: ^3.0.7 => 3.0.7 
    aws-amplify-react-native: ^4.0.3 => 4.0.3 
    babel-core: ^6.26.3 => 6.26.3 
    babel-eslint: ^10.1.0 => 10.1.0 
    babel-jest: ^25.3.0 => 25.3.0 
    babel-preset-airbnb: ^5.0.0 => 5.0.0 
    babel-preset-react-native: ^4.0.1 => 4.0.1 
    eslint: ^6.8.0 => 6.8.0 
    eslint-config-airbnb: ^18.1.0 => 18.1.0 
    eslint-config-prettier: ^6.10.1 => 6.10.1 
    eslint-plugin-flowtype: ^4.7.0 => 4.7.0 
    eslint-plugin-import: ^2.20.2 => 2.20.2 
    eslint-plugin-jsx-a11y: ^6.2.3 => 6.2.3 
    eslint-plugin-prettier: ^3.1.2 => 3.1.2 
    eslint-plugin-react: ^7.19.0 => 7.19.0 
    eslint-plugin-react-hooks: ^3.0.0 => 3.0.0 
    eslint-plugin-react-native: ^3.8.1 => 3.8.1 
    eslint-watch: ^6.0.1 => 6.0.1 
    faker: ^4.1.0 => 4.1.0 
    formik: ^2.1.4 => 2.1.4 
    ini: ^1.3.5 => 1.3.5 
    inquirer: ^6.5.1 => 6.5.2 
    jest: ^25.3.0 => 25.3.0 
    metro-react-native-babel-preset: ^0.59.0 => 0.59.0 
    pre-commit: ^1.2.2 => 1.2.2 
    prettier: ^2.0.4 => 2.0.4 
    prettier-eslint: ^9.0.1 => 9.0.1 
    react: 16.9.0 => 16.9.0 
    react-native: 0.61.5 => 0.61.5 
    react-native-appearance: ^0.3.3 => 0.3.3 
    react-native-gesture-handler: ^1.6.1 => 1.6.1 
    react-native-keychain: ^6.0.0 => 6.0.0 
    react-native-reanimated: ^1.8.0 => 1.8.0 
    react-native-safe-area-context: ^0.7.3 => 0.7.3 
    react-native-screens: ^2.4.0 => 2.4.0 
    react-native-spinkit: ^1.5.0 => 1.5.0 
    react-native-unicorn-uikit: ^0.0.17 => 0.0.17 
    react-native-vector-icons: ^6.6.0 => 6.6.0 
    react-test-renderer: 16.9.0 => 16.9.0 
    yup: ^0.28.3 => 0.28.3 
  npmGlobalPackages:
    @aws-amplify/cli: 4.21.0
    create-react-app: 3.4.1
    nodemon: 2.0.3
    npm: 6.14.4
    patch-package: 6.2.2
    react-native-cli: 2.0.1
DataStore pending-close-response-required

All 135 comments

Do you know if the endpoint is Sync Enabled? The steps in that article seem to use amplify init followed by npx amplify-app@latest which shouldn't be needed, just run the later.

@undefobj in such a blow, the script stops working npm run amplify-modelgen

The documentation says that you need to use the npx amplify-app@latest

Collected without a script npx amplify-app@latest. The result is the same. Repo

Снимок экрана 2020-05-28 в 11 16 28

Could the problem be in my model?

type Job
  @model
  @auth(
    rules: [
      {allow: owner, ownerField: "owner", operations: [create, update, delete]},
    ])
{
  id: ID!
  position: String!
  rate: String!
  description: String!
  owner: String
}

Are you passing the object from aws-exports to Amplify.configure? I.e., https://docs.amplify.aws/lib/datastore/sync/q/platform/js#existing-backend

I didn't see that being performed in the tutorial you linked, so I just wanted to make sure.

@iartemiev Sure. This is done was done

@gHashTag Out of curiosity, what client platform are you on?

@dabit3 Mac OS Catalina 10.15.5

I guess I meant, is this web or React Native?

:) React Native

Got it.

Have you tried providing an error handler to see if there is any error message when the mutation is made?

https://docs.amplify.aws/lib/datastore/conflict/q/platform/js#optional-configurations

No. Something new to me.

@dabit3 Added DISCARD in the project, but did not receive an error in response.

Checked in the region us-east-1 works!
There are no eu-west-1 eu-west-2 regions

But USA is a long way from Russia. How to be?

Hi @gHashTag

Can you verify what your CLI default region is?

You can run amplify configure to see it

Reference: https://docs.amplify.aws/lib/project-setup/prereq/q/platform/ios#option-2-follow-the-instructions

I have the same issue. Information is saved locally but never synced to cloud. I am using the React library.

I tried the following:

  • removing my api config and starting from scratch
  • switching regions
  • Removing @model and @key directives and using a very basic schema containing just one Task object with one string field.

Data is created locally and can see the database in the chrome debugger.

I do see the table created in DynamoDB but never see anything in there. If I use the query tab in the AWS app sync console I can run my mutations and queries and data appears in my DynamoDB tables.

Pretty much at a loss what else I can do now. I went through all of the options in the getting started guide and got nowhere.

I have the same issue. Information is saved locally but never synced to cloud. I am using the React library.

I tried the following:

  • removing my api config and starting from scratch
  • switching regions
  • Removing @model and @key directives and using a very basic schema containing just one Task object with one string field.

Data is created locally and can see the database in the chrome debugger.

I do see the table created in DynamoDB but never see anything in there. If I use the query tab in the AWS app sync console I can run my mutations and queries and data appears in my DynamoDB tables.

Pretty much at a loss what else I can do now. I went through all of the options in the getting started guide and got nowhere.

How did you create your project? Using npx amplify-app@latest or using amplify add api? If the latter did you enable conflict resolution for your whole API? If you look at the data source in the AppSync console is it a versioned enabled data source with a base and delta table?

@manueliglesias
After amplify configure I switched to us-east-1 and that solved my problem. But I am not closing the task yet, as in other regions there are problems.

@undefobj Added DataStore to the working project in the us-east-1 zone and the problem repeated.

If I do not use a script npx amplify-app@latest, do I need to do this step?

@undefobj Thanks for your reply.

I used npx amplify-app@latest to generate basic Task and Note models. I then used npm run amplify-modelgen and finally npm run amplify push This creates the following data source

image

If i then start my React app locally with npm start then I can see in my Application Storage the Task object i am creating getting stored but nothing ever hits the cloud.

Thanks

@baughp when you click on the table and select Edit does it show Enable data source versioning at the bottom with a Delta sync table populated? If not then please run amplify update api to sync enable the endpoint.

Also what region are you working in?

@undefobj Added DataStore to the working project in the us-east-1 zone and the problem repeated.

@gHashTag I'm not sure what you mean by the issue repeated. Can you clarify? You had a working API before and you wanted to add DataStore, or something else?

If I do not use a script npx amplify-app@latest, do I need to do this step?

There are two ways to use DataStore. The "quickest" is navigate into a project (no amplify in it at all) and run npx amplify-app@latest as outlined in the getting started. When you change your schema iteratively locally run npm run amplify-modelgen and this updates the models which DataStore uses for local persistence and syncing. Once happy with your data model you can run npm run amplify-push to push to the cloud.

The second way is longform. If you don't use the npx script then init your Amplify project as normal and do amplify add api. Go through the advanced options to sync enable or after adding the API run amplify update api to configure sync for the whole API. Similarly as with the npx script each time you update your schema you can run amplify codegen models to update the models. Once you're happy run amplify push to deploy this to the cloud.

While there are two methods I recommend using one or the other, not both in a project.

@baughp when you click on the table and select Edit does it show Enable data source versioning at the bottom with a Delta sync table populated? If not then please run amplify update api to sync enable the endpoint.

Also what region are you working in?

@undefobj Yes that is enabled

image

region is US_EAST-1

@baughp Thanks much yes this is sync enabled. Could you tell us if you see any network activity in your dev tools? Perhaps you could place your project in a gist/GitHub repo (or similar project with reproducing the issue) so that we could see what the issue is? I wonder if there's an issue with the app code.

@undefobj I do not see any network activity at all from within my browser debugger

A very simple gist can be found https://gist.github.com/baughp/919f53973f6d79c7c77c24f9cb9a9427

I hope it is something I am doing incorrectly. I removed the API key details from aws-exports.js for the purposes of publishing here.

Thanks

@undefobj ✅ The second way worked for me. Thank you 🙏🏻

Yet something is going wrong.
Started working with another model.
Created a mutation. The model locally is:
Снимок экрана 2020-05-31 в 18 34 49

There is no DinamoDB.
Снимок экрана 2020-05-31 в 18 34 58

@undefobj I do not see any network activity at all from within my browser debugger

A very simple gist can be found https://gist.github.com/baughp/919f53973f6d79c7c77c24f9cb9a9427

I hope it is something I am doing incorrectly. I removed the API key details from aws-exports.js for the purposes of publishing here.

Thanks

@baughp That link doesn't seem to exist

@gHashTag I'm not sure what is happening here in your workflow. From the trace your have _version, _lastChangeedAt, and _deleted are undefined which could be a reason you're seeing an issue. Did you just add a model, run modelgen, push, then do a DataStore.save() on an instance?

@undefobj My bad, please try here

@gHashTag I'm not sure what is happening here in your workflow. From the trace your have _version, _lastChangeedAt, and _deleted are undefined which could be a reason you're seeing an issue. Did you just add a model, run modelgen, push, then do a DataStore.save() on an instance?

I too see undefined for these values? I think the issue @gHashTag and I have is the same issue

I'm not sure what is happening here in your workflow. From the trace your have _version, _lastChangeedAt, and _deleted are undefined which could be a reason you're seeing an issue. Did you just add a model, run modelgen, push, then do a DataStore.save() on an instance?

yes

@gHashTag did you get any further with this? Find a solution?

@baughp No. Everything is in the same place. I'm waiting for the engineers to fix it. And they will write about it here :) I follow the notifications, therefore, the issue.

@gHashTag whats your Appsync conflict resolution set to?

@natedog I did not touch this setting. Means by default.

I have the same problem

Can you post your exact schemas and steps you followed, as well as versions of library and CLI? Link to sample code would help as well. We're not able to reproduce this, I suspect there is some workflow that is causing an issue.

Once again I reproduced the problem on this tutorial. All the same, there is data locally, but App Sync is not.

Code
Model
Mutatiton

Hope the problem is on my part.

@gHashTag are you experiencing the issue in the error_data_store branch? If so, please upgrade to the latest versions of aws-amplify and @aws-amplify/datastore. We released some changes recently that improve syncing performance, amongst other things.

@iartemiev last check 4 days ago in the bug branch

@gHashTag thanks, I'll be attempting to reproduce using that branch

@gHashTag looking at the yarn.lock, there are old versions of DataStore that could be getting picked up. (E.g., here and here)

When I ran the app in the branch as is, it wasn't syncing for me either.

However, I was able to get the app in the bug branch of your repo syncing to the cloud by doing the following:

  • Delete the app in the Simulator
  • Run:
$ rm -rf node_modules yarn.lock
$ yarn
$ yarn ios

Creating new Jobs records via the app in Simulator would result in them getting synced up to DynamoDB.

Give this a try and let me know if it works for you.

@iartemiev Everything is working 🚀 Thank you for the decision 🙏🏻

Here I am also experiencing the same problem.
If I insert through react it updates locally and registers in DynamoDB through DataStore.save ()

If I do this directly through AppSync it does not sync with the device.

I have the same issue. Everything was working fine until half an hour ago when I updated my schema. I followed all of the suggestions here but nothing works and the error persists.

I had the same issue earlier in development and the only way to resolve it was completely destroying my backend and starting again, right now I'm too far into dev for that to be an option.

@iartemiev @undefobj is there anything else that could be causing this issue? How can updating my schema cause such a breakage? All I did was remove a connection, push changes, add a key and new connection, push again, when I tried creating a new object it broke. I've since tried changing conflict resolution types from auto merge, to optimistic concurrency, and back to auto merge.

I'm using Amplify CLI 4.24.1. Project was created using amplify add api.

@Jahans3
I've had the same problem with schema.graphql.
When I edit schema.graphql and do an amplify push, the backend DynamoDB is not updated.

@YasunoriOka I also have a problem all the same. The Data Store is not stable, so I temporarily abandoned it.

Can you give some reproduction steps? There's clearly some edge case/workflow which you're hitting that other customers are not but we're unable to identity. Can you post some explicit schemas and steps in sequence?

I have the same problem, my data works fine locally, I got full CRUD, but it doesn't sync to the cloud at all.

I followed this guide: https://docs.amplify.aws/lib/datastore/getting-started/q/platform/js using the Amplify CLI.

If I go to Amplify Console > All apps > MyApp > dev > Data sources I can see a DynamoDB table created for my API, but the table stays empty. I get no error messages or any hints at anything being wrong, so I don't really know how to troubleshoot this.

So I added the API using amplify add api. Then I did amplify codegen and amplify codegen model and finally amplify push

My config looks like this:

const awsmobile = {
    "aws_project_region": "eu-central-1",
    "aws_cognito_identity_pool_id": "eu-central-1:REDACTED,
    "aws_cognito_region": "eu-central-1",
    "aws_user_pools_id": "eu-central-1_REDACTED",
    "aws_user_pools_web_client_id": "REDACTED",
    "oauth": {},
    "aws_cloud_logic_custom": [
        {
            "name": "apimemorialpages",
            "endpoint": "https://REDACTED.execute-api.eu-central-1.amazonaws.com/dev",
            "region": "eu-central-1"
        }
    ],
    "aws_appsync_graphqlEndpoint": "https://REDACTED.appsync-api.eu-central-1.amazonaws.com/graphql",
    "aws_appsync_region": "eu-central-1",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS"
};


export default awsmobile;

My schema.graphql looks like this:

type SavedSearch @model {
  id: ID!
  searchParams: String!
}

App.tsx

import Amplify from "aws-amplify";
import { DataStore } from "@aws-amplify/datastore";
import awsAmplifyConfig from "./aws-exports";
import { SavedSearch } from "./src/models";

Amplify.configure(awsAmplifyConfig);

export function App() {

...

  DataStore.save(
    new SavedSearch({
      searchParams: 'XYZ',
    })
  );

...

}

If I query the DataStore my item has gotten _deleted, _lastChangedAt and _version added to it.

"_deleted": undefined,
"_lastChangedAt": undefined,
"_version": undefined,
"id": "17537535-0b8f-4212-b218-576d2cbfdf43",
"searchParams": "XYZ"

I have also tried to do DataStore.start() after saving an item.

Any tips for how to troubleshoot this? Or does anyone know what might be wrong?

@bjornarhagen thanks for providing all that info. Is this for a React app or React Native?

Could you please add Amplify.Logger.LOG_LEVEL = 'DEBUG'; to your App.tsx and then share the debug output here?

@iartemiev this is a React Native app. I'm also using Expo.
Debug output: https://gist.github.com/bjornarhagen/4ceb2e2f8a9f86d6efc12073bb9faf4b

I noticed that it seems like only 1 of my 2 APIs is showing up in the debug. I have a REST API called apimemorialpages, the GraphQL API is called SavedSearchApi. The latter is the one I'm having issues with.

If I interact with the REST API I can see debug messages, but when I use the DataStore to interact with the GraphQL API there's no logs. Seems as if the DataStore isn't connected to Amplify?

So I added the API using amplify add api. Then I did amplify codegen and amplify codegen model and finally amplify push

@bjornarhagen when you followed these steps to add an API, did you enable sync? It's possible that your API isn't sync enabled for DataStore.

@bjornarhagen thanks for providing all that info. Is this for a React app or React Native?

Could you please add Amplify.Logger.LOG_LEVEL = 'DEBUG'; to your App.tsx and then share the debug output here?

Great suggestion, I forgot in my Angular app to add Amplify.configure(aws_exports);. I think it would be great to throw a warning when sync is not enabled.

@ajonp Does this mean that you no longer have the issue when enabling sync on your API?

@undefobj I did enable sync yes. If I run amplify update api and select GraphQL I only get the option to Disable DataStore for entire API

I found the cause of my case.

The schema.graphql should look like this

type Restaurant @model {
  id: ID!
  name: String!
  description: String!
  city: String!
}

Use the DataStore to store it.

    DataStore.save(new Restaurant({
      name: "name",
      description: "description",
      city: "city"
    }));

Modify schema.graphql.
Add "town".

type Restaurant @model {
  id: ID!
  name: String!
  description: String!
  city: String!
  town: String!
}

When I try to save again in DataStore, it is not saved.

    DataStore.save(new Restaurant({
      name: "name",
      description: "description",
      city: "city",
      town: "town"
    }));

When I checked the Debug log, I got the following output

"Cannot return null for non-nullable type: 'String' within parent 'Restaurant' (/syncRestaurants/items[0]/town)"

If i modify DynamoDB directly, i will be able to save it.
I'm not familiar with English, so I'm sorry if it's hard to read.

I found the cause of my case.

The schema.graphql should look like this

type Restaurant @model {
  id: ID!
  name: String!
  description: String!
  city: String!
}

Use the DataStore to store it.

    DataStore.save(new Restaurant({
      name: "name",
      description: "description",
      city: "city"
    }));

Modify schema.graphql.
Add "town".

type Restaurant @model {
  id: ID!
  name: String!
  description: String!
  city: String!
  town: String!
}

When I try to save again in DataStore, it is not saved.

    DataStore.save(new Restaurant({
      name: "name",
      description: "description",
      city: "city",
      town: "town"
    }));

When I checked the Debug log, I got the following output

"Cannot return null for non-nullable type: 'String' within parent 'Restaurant' (/syncRestaurants/items[0]/town)"

Did you run modelgen again after modifying your schema and do a push?

@undefobj
Yes, I run "amplify push" and "amplify codegen model" every time I change schema.graphql.

@undefobj
I'm creating on Angular.

@ajonp Does this mean that you no longer have the issue when enabling sync on your API?

@undefobj correct, It is now syncing. But without the debug I would not have even known.

@bjornarhagen I see that you have aws-amplify installed as well as @aws-amplify/datastore. That's likely causing a conflict. Please try the following steps:

yarn remove @aws-amplify/datastore
rm -rf node_modules
yarn

Change your import statement to

import Amplify, { DataStore } from "aws-amplify";

Then run your app again. Let me know if that helps or if you're still having the same issue

@YasunoriOka which version of Angular are you using?

@iartemiev I removed the possible conflict as you suggested and imported { DataStore } from aws-amplify. Then I did a clean expo build, but nothing changed. The sync still doesn't work.

I did however stumble upon API and graphqlOperation, and it works*.

import { API, graphqlOperation } from "aws-amplify";
import { createSavedSearch } from "../../src/graphql/mutations"; // <-- generated from amplify codegen

const res = await API.graphql(
  graphqlOperation(createSavedSearch, {
    input: {
      // input here ...
    }
  })
);

*When I try to delete I just get this error and it doesn't get deleted from dynamodb:
I actually get the same error when I create as well, but for some reason it still works and saves in dynamodb.

Object {
  "data": Object {
    "deleteSavedSearch": null,
  },
  "errors": Array [
    Object {
      "data": null,
      "errorInfo": null,
      "errorType": "InternalFailure",
      "locations": Array [
        Object {
          "column": 3,
          "line": 2,
          "sourceName": null,
        },
      ],
      "message": "An internal error occurred.",
      "path": Array [
        "deleteSavedSearch",
      ],
    },
  ],
}

Can this be related to my DataStore problems maybe?

@bjornarhagen I'm not sure that is preventing the sync from taking place, since the sync performs a graphql query operation. You could try performing some of the operations in src/graphql/queries to see if you encounter errors with those.

In the debug output you shared the other day, I don't see anything being logged from the DataStore category. Could you please start your app, clear the logs, perform a save, and then share those logs?

They should contain something similar to:

[DEBUG] 50:14.730 DataStore - Starting DataStore
[DEBUG] 50:14.731 DataStore - Starting Storage
[DEBUG] 50:14.771 DataStore - GraphQL endpoint available https://**********.appsync-api.us-east-1.amazonaws.com/graphql
[INFO] 50:14.773 DataStore - starting sync engine...
[DEBUG] 50:14.831 DataStore - params ready {predicate: undefined, pagination: undefined, modelConstructor: ƒ}modelConstructor: ƒ Model(init)pagination: undefinedpredicate: undefined__proto__: Object
[INFO] 50:15.58 DataStore - Realtime ready
[DEBUG] 50:15.247 DataStore - Next fullSync in 86297.355 seconds. (Thu Aug 20 2020 08:48:32 GMT-0400 (Eastern Daylight Time))

Also, would it be possible for you to share the entire project in a public repo? If so, I could clone it and run it locally to debug further. But if that's not something you can do - I fully understand.

@iartemiev I just did a npm remove aws-amplify and then installed it again. Which for some reason somewhat fixed it.
However I'm still getting the same error as with the GraphQL operations: An internal error occurred.

And on Android I'm getting

Setting a timer for a long period of time, i.e. multiple minutes, is a performance and correctness issue on Android as it keeps the timer module awake, and timers can only be called when the app is in the foreground. See https://github.com/facebook/react-native/issues/12981 for more info.
(Saw setTimeout with duration 300000ms)

Logs:
https://gist.github.com/bjornarhagen/7bcac6af0ac920e9c428e064407d9b9f

@bjornarhagen most likely there were still conflicting datastore dependencies in your node_modules (which were resolved by reinstalling aws-amplify).

Can you try performing a query via the AppSync Console?
In your terminal execute:

amplify console api
❯ GraphQL

Then log in and run a query like:

query {
  syncSavedSearches {
    items {
      id
      searchParams
    }
  }
}

Let me know if you get an error there as well.

@iartemiev Yup, same error:

{
  "data": {
    "syncSavedSearches": null
  },
  "errors": [
    {
      "path": [
        "syncSavedSearches"
      ],
      "data": null,
      "errorType": "InternalFailure",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "An internal error occurred."
    }
  ]
}

Here's the schema: https://gist.github.com/bjornarhagen/6c04579e2fb1f3ecb40b332696ccacc6

@iartemiev Yup, same error:

{
  "data": {
    "syncSavedSearches": null
  },
  "errors": [
    {
      "path": [
        "syncSavedSearches"
      ],
      "data": null,
      "errorType": "InternalFailure",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "An internal error occurred."
    }
  ]
}

If this is happening from the console then it sounds like there's an issue with the generated resolver code. @bjornarhagen what is the exact schema and process you followed to create your backend? If there's a resolver codegen issue we would like to reproduce this.

cc @renebrandel

@undefobj I just edited my comment and added a link to the schema.
The back-end was automatically setup by the amplify cli. I just did amplify add api.

Can a permission issue for my IAM user cause this kind of error?

@undefobj I just edited my comment and added a link to the schema.
The back-end was automatically setup by the amplify cli. I just did amplify add api.

Can a permission issue for my IAM user cause this kind of error?

This looks like something else than auth. Out of curiosity did you convert your resolver to a pipeline? Also can we see the VTL templates?

@undefobj
All I have done is use these commands:

amplify add api
amplify push
amplify codegen
amplify codegen model

Here's the API that was generated: https://github.com/bjornarhagen/aws-amplify-troubleshooting

@undefobj @iartemiev
I think I found the problem!

In my schema I have notificationType.

type SavedSearch @model {
  ...
  notificationType: [NotificationType]
  ...
}

enum NotificationType {
  PUSH
  SMS
  EMAIL
}

I was using DataStore to save this:

new SavedSearch({
  ...
  notificationType: [],
  ...
})

In DynamoDB this showed up like so:

{
  ...
  notificationType: [

  ],
  ...
}

If I instead save one of my enums or just null it works!
Maybe a more helpful error message for this scenario can be implemented?

Thank you so much for the help, appreciate it! 😄

@bjornarhagen glad to hear that it's working now! ~Technically, you should be able to use [] there, so this might be a bug. We'll confirm and will work on a fix if it is.~ Scratch that, DynamoDB doesn't allow empty arrays. We'll work on adding better validation around that.

Update: in the interest of maintaining parity with the GQL spec, we will be adding the ability for passing in empty arrays as values when interacting with DataStore and the API category

@gHashTag could you please provide more info on the issue you're having? What has changed between now and when sync was last working for you? If you have an updated branch, let me know and I'll dig into it.

@ajonp Does this mean that you no longer have the issue when enabling sync on your API?

@undefobj correct, It is now syncing. But without the debug I would not have even known.

@ajonp we've been thinking about this. How do you think this debugging would work? Since a GraphQL endpont can be used with the API category or DataStore category we would need to give the proper information for you to debug easier depending on the path you choose for your app requirements. Some thoughts:

  • We could throw a console.warn() if DataStore doesn't get a response from the GraphQL server that conforms to the correct signature. Something like "Warning, missing _version from GraphQL response. Are you sure you are using a sync-enabled API? Run 'amplify update api to confirm'"
  • We could add something like syncEnabled:true to the aws-exports.js config file so that the library categories (API & DataStore) can detect this and give a warning message if you're going down the wrong track rather than wondering why sync isn’t working. This would also help the case where someone _Does_ have a sync enabled API and is using the API category and not DataStore but your GraphQL calls are failing because they’re missing a version argument .

Or is there something else that would have helped you set this up easier or found the resolution faster?

@undefobj I like the second option the best as it is more explicit. Firebase will actually throw a console.warn even when the connection is lost to let you know.

Side note:
I have been working on a comparison tutorial and can't get Datastore to sync properly if anyone has a few minutes to jump on Discord LMK.

@ajonp I can give it a shot. Just post a message in #datastore-discussion and tag me, @Ivan. I'll make sure to stick around Discord for a bit this morning.

I was having the same problem. Local DataStore had the data but it wouldn't sync. My workflow was adding DataStore to an existing project that already had several GraphQL models defined. Turned out that enabling DataStore adds a few non-nullable properties behind the scenes that invalidated my existing data. That caused the GraphQL endpoint to return an error when queried by DataStore via the GraphQL endpoint. Turning on Amplify.Logger.LOG_LEVEL = 'DEBUG'; in the App.tsx was the key to tracking down the error. Once I deleted all the invalid data in the DynamoDB tables DataStore sync started working.

I lied, deleting all of the existing data from DynamoDB only solved part of the problem. That change cleared the way for DataStore to send local changes to the backend. DataStore.observe() was still never getting called however.

To get everything working, I ended up doing amplify remove api and then amplify add api followed by amplify update api to enable DataStore. After doing amplify codegen model followed by amplify push DataStore is all happy now.

My Amplify project has evolved over the last week or so as I experimented. I first added api, followed by auth, hosting, pre-signup trigger, admin queries, and user pool groups so I'm not sure what state it was in before attempting to add DataStore support. I can however document the local configuration differences (at least the ones in git) before/after rebuilding my api. Note that when I first attempted to enable DataStore my GraphQL models did not have an owner property like I have seen documented in some blog posts (4-6 months old). Current documentation doesn't mention this.

.graphqlconfig.yml

 projects:
   nxt:
-    schemaPath: amplify/backend/api/nxt/build/schema.graphql
+    schemaPath: src/graphql/schema.json
     includes:

src/graphql/schema.json
Attached before and after versions since the differences are numerous. Hope this helps determine if there is anything that can be done to make enabling DataStore a little less error prone.
before-after-schema.json.zip

@YasunoriOka which version of Angular are you using?

I'm using Angular 9.

I am still having issues with Sync, I am starting to wonder if I am pushing too many calls, but that worries me if it can't handle a call every .5 seconds.

Feel free to try out what I have, maybe someone will have better luck. I am working through the Basics of Amplify tutorial so this should come later anyways but I was hoping I could maybe start with Datastore like I would start with Firestore.

https://github.com/CodingCatDev/amplify-datastore-firebase-firestore

@YasunoriOka I'll attempt to reproduce your issue in Angular 9.

@ajonp Will also be attempting to reproduce the issue by running your project locally.

I will let you both know what I find soon.

@ajonp - I don't think the issue is with too many calls. I've created a sample app where I create and save items sequentially using setInterval, and even when the interval is set as low as 10ms, DataStore handles it fine and keeps syncing. At 200ms, it syncs live with no delay (i.e., it completes a round trip and will have values for _version and _lastChangedAt almost at the same time as it's added to the UI state and rendered). For shorter intervals, it may take a few seconds for the sync to catch up, but the sync remains intact throughout.

Happy to share the sample app or a video of it running if you'd like.

@iartemiev I would love to take a look at the repo, maybe I am just setting something up incorrectly on the Angular side.

@ajonp here it is: https://github.com/iartemiev/datastore-speed-test

It's just an extremely basic POC and it's in React but I don't think the framework plays any role at all with how many calls you can make. The only thing I can think of is making sure to create the DataStore subscriptions in the correct component lifecycles. I don't know Angular particularly well, but most likely you'd want to manage subscriptions in ngOnInit

I'm also having problems in syncing data via DataStore. It reads data from the database assigned to it, but when i create something to send to it, it generates it offline, but doesn't send it to the cloud. It also doens't assign a owner field to it, even when localy.

This is my schema.graphql

type Post @model @auth(rules: [ { allow: owner} ]) { id: ID! name: String! description: String location: String image: String owner: String }

I was trying to replicate this workshop

Data is created, but it is not available in App Sync and as a result, there is no sync of this data between
MyRepository

@novoselovaleksandr please try changing your auth rule to @auth(rules: [{ allow: owner }])
Followed by

amplify push
amplify codegen models

Let me know if that gets your app syncing.

We're currently working through some issues with specifying operations for owner auth.

there's clearly something broken here.

Setup a basic model. Push. Change the schema (you don't even need to change model, you can add or auth, or add another unrelated model). Push. Call listVenue via the AWS console, you get results - as exected. But, calling:

DataStore.query(Venue, Predicates.ALL).then(venues => store.commit('initialiseVenues', venues));

get's me an empty array.

Here's my schema:

type Venue
  @model
  @auth (
    rules: [
      { allow: private, operations: [read, create, update] },
      { allow: public, provider: apiKey, operations: [read, create, update] },
    ]
  )
  {
    id: ID!
    name: String
    pax: Int
    venueType: String
    country: String
    tipples: AWSJSON
  }

The git diff doesn't suggest that anything to do with Venue was changed.

Deleting the IndexDB (either manually or by calling DataStore.clear(); does not help.

Rolling back git does not help.

amplify delete followed by reworking things will get it to work.

But that's starting to become tedious.

I'd appreciate your guidance. Or, if I can help you diagnose what is going on, please let me know.

Ubunty. Vue. Worked through the tutorial, then added API access to allow existing processes to push data and changes to graphql clients. Works a treat until it doesn't.

Personally, I'm suspicious of that @auth stanza....

@renenw there is currently a bug with specifying operations for owner and static group auth. We're actively working on resolving this.

thanks - but i'm not specifying groups and owners. But, point taken.

Do you have an ETA? Or an workaround? Or do I delete and recreate?

I am getting this message when I try to query data, using GraphQL Client works though

image

image

@dlopez12003 have you added a sync-enabled API via the CLI, followed by amplify push?

@iartemiev I did it through that way, I got it to work by uninstalling and installing again my npm packages, weird

I'm also having issues getting DataStore to sync in either direction with AppSync + React Native. Can successfully query AppSync in the console and get data back, but DataStore never seems to interact with it, either after .save() or .query().

See the code & logs from DataStore below. I have it set to query a Charity model when Hub receives the DataStore "ready" event. You can see it returns 2 results, but both are previous local models from calling .save() that never got sent to DynamoDB, and don't have any _version, _lastChangedAt, or _deleted values. There is also an item in my DynamoDB table that should be returned and is not.

Steps tried so far:

  • Disable DataStore for API, amplify push, re-enable DataStore for entire API, codegen models, amplify push
  • Removed @auth directive from Charity model. The model now only has an "id" and "name" field.
  • Removed node_modules and re-installed everything. Only amplify packages specified are "aws-amplify": "^3.3.3" and "@aws-amplify/ui-components": "^0.8.6"

Any ideas how to get it to sync? Appreciate it.

Sample code, copied from another issue's suggestion:

useFocusEffect(
    React.useCallback(() => {
      console.log("home use effect")

      const removeListener = Hub.listen("datastore", async (capsule) => {
        const {
          payload: { event, data },
        } = capsule;

        console.log("DataStore event", event, data);

        if (event === "ready") {
          const charity = await DataStore.query(Charity);

          console.log("Charity: ", charity)
        }
      });

      // Start the DataStore, this kicks-off the sync process.
      DataStore.start();

      return () => {
        removeListener();
      };
    }, [])
);

Console Output:
[DEBUG] 27:34.149 DataStore - Starting DataStore [DEBUG] 27:34.149 DataStore - Starting Storage [DEBUG] 27:34.165 DataStore - GraphQL endpoint available https://xxxxxxxxxxxxxxx.appsync-api.us-east-1.amazonaws.com/graphql [INFO] 27:34.166 DataStore - starting sync engine... [INFO] 27:34.234 Reachability - subscribing to reachability in React Native [DEBUG] 27:34.235 Hub - Dispatching to datastore with {event: "storageSubscribed", data: undefined} DataStore event storageSubscribed undefined [DEBUG] 27:34.238 Hub - Dispatching to datastore with {event: "outboxStatus", data: {…}} DataStore event outboxStatus {isEmpty: false} [INFO] 27:34.240 Reachability - Notifying initial reachability state null [DEBUG] 27:34.240 Hub - Dispatching to datastore with {event: "networkStatus", data: {…}} DataStore event networkStatus {active: null} [DEBUG] 27:34.241 Hub - Dispatching to datastore with {event: "ready", data: undefined}data: undefinedevent: "ready"__proto__: Object DataStore event ready undefined [DEBUG] 27:34.241 DataStore - after createFromExisting - predicate undefined [DEBUG] 27:34.242 DataStore - params ready {predicate: undefined, pagination: {…}, modelConstructor: ƒ} Charity: (2) [Model, Model]0: Model {name: "Test", id: "4a42fa22-582f-4958-8947-b43e633c8808", _version: undefined, _lastChangedAt: undefined, _deleted: undefined}1: Model {name: "Test", id: "b4929c78-e115-4304-b39a-da778f9d2273", _version: undefined, _lastChangedAt: undefined, _deleted: undefined}length: 2__proto__: Array(0) [INFO] 27:36.247 Reachability - Notifying reachability change true [DEBUG] 27:36.247 Hub - Dispatching to datastore with {event: "networkStatus", data: {…}}data: {active: true}event: "networkStatus"__proto__: Object DataStore event networkStatus {active: true}

@rwhitten577 would you mind pushing your app up to a repo and sharing it? If so, I'd be happy to run it locally myself to see if I can reproduce and figure out why it's not syncing.

@iartemiev Thanks, my repo is private but I just invited you. Let me know if you have issues with access.

@rwhitten577, thank you! I was able to clone it successfully. Will let you know what I find

@iartemiev great, thank you! I do not have that useEffect snippet above checked in, but would assume it should work when added to my LandingScreen since no auth is required for that model.

@rwhitten577 just to clarify, are you experiencing issues with syncing when you have just 1 model (Charity) in your schema.graphql? Or are you always trying it with all the models, like in the code you have in your repo?

@iartemiev So far I have only tried with the schema as you see it with the other models defined. I was thinking the query would only look at the one model being queried. Could my other models with @auth directives be causing issues? I did read a few other issues where more complex @auth was not fully supported yet.

@rwhitten577 the other models could be causing issues potentially. Although, if that were the case, you should have seen some errors around sync or subscriptions in your console (particularly since you have Logger set to 'DEBUG'). Also, it's unlikely that DataStore would have reached a 'ready' state like what's showing in the logs.

Regarding @auth known bugs, referencing a relation as your ownerField like you have in one of your models might be problematic with DataStore. Everything else in there should work.

If you have the chance, please try the following:

  1. comment out everything in your schema except for Charity then push and codegen models.
  2. run your app, try saving and querying Charity records

If sync still isn't working, please let me know and I'll dig into it.

If it is working, I would recommend progressively uncommenting more models and starting at 1. until sync stops working. Then once again, just let me know and I'll do my best to help you get it syncing again.

@iartemiev thanks! Following your steps got it to work if I put the query in the listener for the "ready" state. I removed just the ownerField on that one model and now can get the rest to sync too!

I do notice mixed results sometimes where the first query returns an empty array, even if I do the query after the listener receives "ready". Is there a best practice on how to handle the initial query when local storage is empty?

@rwhitten577 glad to hear it's syncing now!

Technically, "ready" should be emitted after syncing has finished, so if you do not see data that should be there, perhaps, we have a bug. I'll see if I can reproduce this.

One more thing, I noticed that you're using multiple Has One connections and wanted to make sure you're aware of this bug. It's actively being worked on, so you can track the progress of it there.

Datastore won't be sincronized... this is the error message before authenticate. Datastore create table in IndexDB database but it won't sync.

image

@EliasCoelho, which versions of Amplify / DataStore are you using? Please share your schema, as well as the actual error message.

@iartemiev
"aws-amplify": "^3.0.24"
"@aws-amplify/ui-react": "^0.8.6"
schema.txt

@EliasCoelho please try the latest version - 3.3.3. Also, enable logs with

Amplify.Logger.LOG_LEVEL = 'DEBUG'

And let me know if you see any errors in the console.

@iartemiev all dependencies updated to
image
and the debugger console is the following

@EliasCoelho I edited your comment and removed the log file because it contained unredacted sensitive information about your cloud environment.

ok, thanks @iartemiev

@EliasCoelho, did syncing work for you at one point and then stop after some change? Or has it never worked?

@iartemiev this enviroment worked for three apps with the same backend, a react-nactive and two react apps until September 30, after it only work for react-nactive project and a react app.

It don't sync actually for a react app, with updated packages dependencies and configured with the same team-provider-info.json file in all projects.

We have been set Amplify.configure(awsExport) and Auth.configure(awsExport), but the problem persists

@EliasCoelho have you tried clearing your local store? You can do that through calling DataStore.clear() in your code or via Chrome Dev Tools by going to the Application tab -> IndexedDB -> amplify-datastore -> Delete database.

For your React apps, is sync not working outbound, as well as inbound? That is, I understand that data is not syncing from your React apps to AppSync, but can you see incoming data when you add records via your React Native app or via the AppSync console?

Also, is the AppSync API Key you're using still valid? By default, it expires after 7 days.

Hi @iartemiev, yes, I delete Datastore data using DataStore.clear() in authentication method and also chrome dev tools.

As comment to you, I could do DataStore.Save() and it work, but this register don't sync to the cloud and the hub.listen('datastore', async callback) not received anything message. The incoming and outbound communication between cloud and project is not working.

After reinstallation and fix in import and use code not async it work for me.
Steps:

  • Install all libraries to the lastest version
  • Use Hub.listen('datastore', callback) and not Hub.listen('datastore', async callback)
  • Don't use import { Datastore } from 'aws-amplify/datastore' as documentation indicate
  • Don't use import { Hub } from '@aws-amplify/core'
  • Use import Amplify, { Hub, DataStore } from 'aws-amplify'

this work for me...
Thanks for your help @iartemiev

@EliasCoelho, I'm happy to hear you were able to get it working.

@rwhitten577 - So far I haven't been able to reproduce the scenario you described where the "ready" event is getting emitted before the models are synced. Are you experiencing this in all expo platforms (ios, android, web)? Is it occurring when querying a particular model or each of them? Or is it seemingly random? Are there any other patterns you might have noticed that could help with reproducing?

@iartemiev I believe it was a transient issue. Shortly after getting sync to work, I did end up replacing DataStore with the standard AppSync API for now due to other DataStore issues including lack of customization for setting object IDs on create and the codegen bug you linked. I love the concept of DS but need to get this prototype done quickly and can live without it until more customization is available.

Appreciate the work of you and the team though and look forward to future updates!

@rwhitten577 thank you for the feedback! We're constantly working on improving DataStore and genuinely hope that it will meet your business needs soon.

Allowing customers to use their own PKs on models is an upcoming feature. You can track its progress via this issue.

I am having hard time with Datastore syncing with my current project, the data is saved locally but not reflecting in my dynamoDB base table and delta sync table, no graphql call is triggered in network tab.

The error in my console
image

However I have already configured correct graphql endpoint. After adding Amplify.Logger.LOG_LEVEL = 'DEBUG' even logs display the endpoint to be configured correctly.
image

The weird thing is same AppSync backend works with web(React) but doesn't work with app(React-native. I am trying to import an existing AppSync backend.

Any idea on what could go wrong?

@pragyajha could you share your app code where you're calling Amplify.configure?

Hey @iartemiev

Here's the sample code, Both where we are calling Amplify.configure as well as config defined in aws-exports.

image

@pragyajha I've seen this type of issue when there are multiple versions of Amplify or Amplify dependencies in node_modules.

Make sure you only have aws-amplify in your package.json (as opposed to, for example, @aws-amplify/datastore) and run:

rm -rf node_modules yarn.lock package-lock.json
npm install (or yarn, depending on which one you're using)

Then try running your app again.

Can I chime in and share my current situation with DataStore, bare in mind I've only 1 year experience. Its very possible I could be making some mistakes but I find the bigger my project gets the trickier it seems to get for Datastore to behave as expected, I notice some models are synching and others are not, multiple graphql requests in my network tab in developer tools in chrome, synching happening sporadically. The fact that the documentation even states that synching with the cloud is difficult scares me!! Is there any medium size DataStore example projects floating around that I could compare with. I have adapted some aws graphql examples like the Chatt app and a photo album app into my project, mainly borrowing from the schemas and tweeking them to my own needs and creating everything using DataStore, so they have been great help and helping me design better schemas. Its all working well, making mistakes, finding solutions, tweeking my schema, it just the synching part that doesnt play nice! Thanks!

The fact that the documentation even states that synching with the cloud is difficult scares me!!

Hey @myfairshare, which part of the docs are you referring to?

Also, would you be able to share your project where you're experiencing these syncing issues? If so, I'd be happy to take a look.

In this part..

At the top you will see heading ..

Set Up cloud sync

Synchronization between offline and online data can be tricky. DataStore goal is to remove that burden from the application code and handle all data consistency.....

@myfairshare that sentence communicates that syncing data between multiple devices and the cloud can be a difficult problem to solve in general _without_ DataStore and that DataStore simplifies it for developers.

Ahh, my apologies, thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This issue has been automatically closed because of inactivity. Please open a new issue if are still encountering problems.

@iartemiev we're seeing a strange variant of this on react native. It appears to be only on iOS devices, and we've only managed to replicate it cross-device (if you resync the datastore on the same device then it appears to load correctly). We're using a user pool auth set-up where on authentication we restart the datastore (clear it and restart), then listen for the hub event to be ready. When the hub ready event fires we sync the data into our local redux store.

I've checked that there's only a single version of aws-amplify (v 3.82) in our yarn.lock. We're using aws-amplify-react-native 4.28.

For reference here's our schema structure:

type A @model @auth(rules: [{ allow: owner }]) {
  id: ID!
  z: String!
}

type B @model @auth(rules: [{allow: owner}]) {
 id: ID!
 c: Int!
 d: Int!
 e: String!
 aId: ID!
 a: A @connection(fields: ["aId"])
}

The connection on the xId field didn't appear to be working, so we've been manually managing the relation between the A and B models in app.

Our configuration is as follows:

const awsmobile = {
    "aws_project_region": "eu-west-1",
    "aws_cognito_identity_pool_id": "xxxx",
    "aws_cognito_region": "eu-west-1",
    "aws_user_pools_id": "xxxx",
    "aws_user_pools_web_client_id": "xxxx",
    "oauth": {},
    "aws_appsync_graphqlEndpoint": xxxx",
    "aws_appsync_region": "eu-west-1",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS"
};

Any advice would be greatly appreciated.

@paulfreund94 could you please open a new issue for this? I would also recommend enabling logs by calling Amplify.Logger.LOG_LEVEL = 'DEBUG' and checking for any errors or warnings around DataStore subscriptions. If you see any, please include them in your issue.

Was this page helpful?
0 / 5 - 0 ratings