Amplify-js: DataStore - Validation error of type MissingFieldArgument: Missing field argument owner for Subscription and Sync

[getglad]

Describe the bug
When using the below schema

type Recipe
  @model
  @auth(rules: [
    { allow: owner, provider: oidc, identityClaim: "sub" }
  ]) {
  id: ID!
  name: String
  description: String
  ingredients: [Ingredient] @connection(keyName: "byRecipe", fields: ["id"])
  sourceType: String
  source: Source @connection
}

I get the below error:

DataStore - subscriptionError Connection failed: {"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onCreateRecipe'"}]}
DataStore - Sync error Connection failed: {"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onCreateRecipe'"}]}

The record is created locally, but does not push. Because it cannot push, also cannot get

To Reproduce

1. Create an API with an OIDC auth + AUTOMERGE conflict resolution
2. Run node amplify/scripts/amplify-modelgen.js + amplify push
3. Launch the application and obverse the console

Expected behavior
No error in console, the records push and pull correctly.

Code Snippet
DataStore.query(Recipe) <- Recipe is imported from the generated index.d.ts

What is Configured?
If applicable, please provide what is configured for Amplify CLI:

• Which steps did you follow via Amplify CLI when configuring your resources.
  
  
  
  • See reproduction steps above
  
  
• Which resources do you have configured?
  
  
  
  • If applicable, please provide your aws-exports file:
    
    const awsmobile = { "aws_project_region": "us-east-1", "aws_appsync_graphqlEndpoint": "https://<address-of-endpoint>.appsync-api.us-east-1.amazonaws.com/graphql", "aws_appsync_region": "us-east-1", "aws_appsync_authenticationType": "OPENID_CONNECT" };
  
  
  • If applicable, please provide your manual configuration example:
    
    { Auth:{ identityPoolId: 'us-east-1:<id-of-pool>', region: 'us-east-1', refreshHandlers: { '<address-of-saml>': <refresh-function> } } }
    
    
    
    
    

    Environment
  
  

npx envinfo --system --binaries --browsers --npmPackages --npmGlobalPackages

  System:
    OS: Linux 4.19 Ubuntu 20.04.1 LTS (Focal Fossa)
    CPU: (8) x64 Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
    Memory: 18.16 GB / 24.92 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 14.15.3 - /usr/bin/node
    Yarn: 1.22.10 - /usr/bin/yarn
    npm: 6.14.10 - /usr/bin/npm
  npmPackages:
    @angular-devkit/build-angular: ^0.1100.5 => 0.1100.5 
    @angular/cli: ^11.0.5 => 11.0.5 
    @angular/common: ^11.0.5 => 11.0.5 
    @angular/compiler: ^11.0.5 => 11.0.5 
    @angular/compiler-cli: ^11.0.5 => 11.0.5 
    @angular/core: ^11.0.5 => 11.0.5 
    @angular/forms: ^11.0.5 => 11.0.5 
    @angular/language-service: ^11.0.5 => 11.0.5 
    @angular/localize: ^11.0.5 => 11.0.5 
    @angular/platform-browser: ^11.0.5 => 11.0.5 
    @angular/platform-browser-dynamic: ^11.0.5 => 11.0.5 
    @angular/router: ^11.0.5 => 11.0.5 
    @auth0/auth0-spa-js: ^1.13.5 => 1.13.5 
    @aws-amplify/analytics: ^3.3.11 => 3.3.11 
    @aws-amplify/cache: ^3.1.41 => 3.1.41 
    @aws-amplify/datastore: ^2.9.2 => 2.9.2 
    @aws-amplify/interactions: ^3.3.16 => 3.3.16 
    @aws-amplify/storage: ^3.3.16 => 3.3.16 
    @aws-amplify/xr: ^2.2.16 => 2.2.16 
    @capacitor/cli: ^2.4.5 => 2.4.5 
    @capacitor/core: ^2.4.5 => 2.4.5 
    @ionic-native/core: ^5.30.0 => 5.30.0 
    @ionic-native/splash-screen: ^5.30.0 => 5.30.0 
    @ionic-native/status-bar: ^5.30.0 => 5.30.0 
    @ionic/angular: ^5.5.2 => 5.5.2 
    @okta/okta-angular: ^3.0.1 => 3.0.1 
    @types/jasmine: ^3.6.2 => 3.6.2 
    @types/jasminewd2: ~2.0.3 => 2.0.8 
    @types/jest: ^26.0.19 => 26.0.19 
    @types/mocha: ^8.2.0 => 8.2.0 
    @types/node: ^14.14.14 => 14.14.14 
    aws-amplify-angular: ^5.0.42 => 5.0.42 
    buffer: ^6.0.3 => 6.0.3 
    codelyzer: ^6.0.1 => 6.0.1 
    core-js: ^3.8.1 => 3.8.1 
    ini: ^1.3.8 => 1.3.8 
    inquirer: ^7.3.3 => 7.3.3 
    jasmine-core: ^3.6.0 => 3.6.0 
    jest: ^26.6.3 => 26.6.3 
    karma: ^5.2.3 => 5.2.3 
    karma-chrome-launcher: ^3.1.0 => 3.1.0 
    karma-coverage: ^2.0.3 => 2.0.3 
    karma-jasmine-html-reporter: ^1.5.4 => 1.5.4 
    protractor: ^7.0.0 => 7.0.0 
    puppeteer: ^5.5.0 => 5.5.0 
    rxjs: ^6.6.3 => 6.6.3 
    tslib: ^2.0.3 => 2.0.3 
    tslint: ~6.1.0 => 6.1.3 
    typescript: ^4.0.5 => 4.0.5 
    zone.js: ^0.10.2 => 0.10.3 
  npmGlobalPackages:
    @angular/cli: 11.0.5
    @aws-amplify/cli: 4.40.1
    @ionic/cli: 6.12.3
    flag: 4.4.0
    ionic: 5.4.16
    native-run: 1.3.0
    npm: 6.14.10
    yarn: 1.22.10

Smartphone (please complete the following information):

• Device: Windows Machine
• OS: Windows
• Browser: Firefox + Chrome
• Version: 84 / 87

Additional context
Cognito authentication looks correct (no auth errors messages) and JWT token can authenticate through the AppSync console, so my best guess is that this is something to do with the schema. Happy to provide any other insights.

[amhinson]

@getglad Just curious, are you authenticated when you try to create the item?

Also, could you share the code for creating the item with DataStore.save(...)?

[getglad]

@amhinson - yes, the user is authenticated - when I use the OIDC JWT in the app sync console (similar to the steps on this blog post under "Testing" https://aws.amazon.com/blogs/mobile/appsync-okta/) I am able to execute queries.

The save code is as follows, and can confirm that it creates a local record:

async createRecipe(recipe: RecipeQL) {

    const savedSource = await this.sourceService.createRecipeSource(recipe)

    const savedRecipe = await DataStore.save(
      new RecipeQL({
        name: recipe.name,
        description: recipe.description,
        sourceType: recipe.sourceType,
        source: savedSource
      })
    );

   return savedRecipe;
}

However want to be clear on the context of the above - it's an angular app, and the view that this error appears on is the "list" view I have created. So only the query call is made - I am assuming this is part of a DataStore.start. Per the docs...

Synchronization starts automatically whenever you run DataStore.query() however you can explicitly begin the process with DataStore.start().

Source: https://docs.amplify.aws/lib/datastore/other-methods/q/platform/js

Also, the raised error is not consistently for the Recipe schema or the onCreate mutation - if I refresh the page, it is almost random which schema or mutation pings first, but it appears to always be some schema with an OIDC owner.

[getglad]

Additionally, as a sanity check - I had the same thought, so I have broken my Auth flow on purpose.

Per the docs, the correct path has Auth.federatedSignIn invoked to connect the OIDC JWT w/ Cognito.

1. I see an appsync unauthenticated error when I skip the invocation all together
2. I see a Cognito unauthenticated error when I use the wrong OIDC configuration
3. I have a .then and a .catch on the federatedSignIn promise. I see it fall into the .then when configured correctly, and the .catch when not configured correctly.

So I am fairly confident I am connected - but the docs are a little weak in this area so happy to do something else as a validation step if you want me to.

[amhinson]

Ok thanks for looking into that! Would you be able to share the logs from Amplify.Logger.LOG_LEVEL = 'DEBUG'? That will give a better idea of what is going on. Be sure to remove any sensitive data as needed.

[getglad]

@amhinson I'm not sure if it should do more than set it as a global(?) but this is what was produced...

Amplify.Logger.LOG_LEVEL = 'DEBUG'
Amplify.configure(awsconfig);
DataStore.configure(awsconfig);

15:03:48.381 [DEBUG] 03:48.382 Amplify - amplify config 
Object { aws_project_region: "us-east-1", aws_appsync_graphqlEndpoint: "https://<endpoint>.appsync-api.us-east-1.amazonaws.com/graphql", aws_appsync_region: "us-east-1", aws_appsync_authenticationType: "OPENID_CONNECT" }
ConsoleLogger.js:107
15:03:48.382 [DEBUG] 03:48.383 I18n - configure I18n ConsoleLogger.js:97
15:03:48.383 [DEBUG] 03:48.383 I18n - create I18n instance ConsoleLogger.js:97
15:03:48.384 [DEBUG] 03:48.384 AnalyticsClass - configure Analytics 
Object { aws_project_region: "us-east-1", aws_appsync_graphqlEndpoint: "https://<endpoint>.appsync-api.us-east-1.amazonaws.com/graphql", aws_appsync_region: "us-east-1", aws_appsync_authenticationType: "OPENID_CONNECT" }
ConsoleLogger.js:107
15:03:48.385 [DEBUG] 03:48.385 Parser - parse config 
Array(3) [ {…}, "to amplifyconfig", {…} ]
ConsoleLogger.js:107
15:03:48.386 [DEBUG] 03:48.387 AWSPinpointProvider - configure Analytics 
Object { disabled: undefined, autoSessionRecord: true, aws_project_region: "us-east-1", aws_appsync_graphqlEndpoint: "https://<endpoint>.appsync-api.us-east-1.amazonaws.com/graphql", aws_appsync_region: "us-east-1", aws_appsync_authenticationType: "OPENID_CONNECT" }
ConsoleLogger.js:107
15:03:48.387 [DEBUG] 03:48.387 Hub - Dispatching to analytics with  
Object { event: "configured", data: null, message: "The Analytics category has been configured successfully" }
ConsoleLogger.js:107
15:03:48.387 [DEBUG] 03:48.387 AnalyticsClass - on hub capsule analytics 
Object { event: "configured", data: null, message: "The Analytics category has been configured successfully" }
ConsoleLogger.js:107
15:03:48.388 [DEBUG] 03:48.388 AnalyticsClass - current configuration 
Object { autoSessionRecord: true, aws_project_region: "us-east-1", aws_appsync_graphqlEndpoint: "https://<endpoint>.appsync-api.us-east-1.amazonaws.com/graphql", aws_appsync_region: "us-east-1", aws_appsync_authenticationType: "OPENID_CONNECT" }
ConsoleLogger.js:107

[amhinson]

Hmm interesting. There should be a considerable amount more showing in the console. I don't think this will fix the problem, but could you remove DataStore.configure(awsconfig)? It isn't needed since you're already using Amplify.configure().

Although, that does make me wonder if the configuration is involved here. I see in the original issue that you provided aws-exports.js, but you also provided a manual configuration object that only contains the Auth category. Typically, you would only use one or the other. What does the object look like that you are passing to Amplify.configure(awsconfig)?

[getglad]

Swapping to

Amplify.Logger.LOG_LEVEL = 'DEBUG'
Amplify.configure(awsconfig);
// DataStore.configure(awsconfig);

Results in the same output, but with the additional warning

 DataStore - Data won't be synchronized. No GraphQL endpoint configured. Did you forget `Amplify.configure(awsconfig)`?

For the life of me, I can't remember where I found the DataStore.configure use case, but it did fix that warning. Doing a quick google search, finding something about it on StackOverflow but don't know if that was my source or not - https://stackoverflow.com/questions/63646190/aws-amplify-datastore-wont-synchronise-with-server

Per the aws-exports question... I was short handing a bit - so not to give too much information at once but:

1. I am using the generated file created by the amplify cli
2. Then configuring using the "Re-use existing authentication resource" method from the docs - https://docs.amplify.aws/lib/auth/start/q/platform/js#re-use-existing-authentication-resource - as part of the Auth flow

   const ampAth = {
      identityPoolId: 'us-east-1:<pool-id>',
      region: 'us-east-1',
      refreshHandlers: {
        <domain>': this.<refresh-method>
      }
    }
    Amplify.configure(awsconfig)
    Amplify.configure({ Auth: ampAth })
    Auth.configure(Amplify.configure())
    DataStore.configure(Amplify.configure())

You may notice that last Auth.configure - that is because I get the below error unless I configure it as well

 Error: Federation requires either a User Pool or Identity Pool in config

Things maybe worth knowing:

1. It does not seem to matter where in the flow I perform the DataStore.configure invocation - schemas with owners throw the message and the ones without do not.
2. If I remove all owners, then DataStore pushes/pulls fine, but still requires the Amplify.configure({ Auth: ampAth }) or else throws that Federation error message

[amhinson]

Ah ok! I've got a few observations and suggestions:

In most cases, you should only need to call Amplify.configure(...) once. Calling it additionally from the other categories can lead to unexpected behavior due to the singleton nature of most of the categories. I would suggest combining the aws-exports with your custom Auth configuration like this:

Amplify.configure({
  ...awsconfig,
  Auth: {
    identityPoolId: 'us-east-1:<pool-id>',
    region: 'us-east-1',
    refreshHandlers: {
      <domain>': this.<refresh-method>
    }
  }
})

Additionally, I see in your package.json you have individual modules imported instead of just aws-amplify. This probably isn't causing any problems now because you're using the latest version of all of them, but our recommendation is to just simply use aws-amplify and let the bundler handle any treeshaking needed.

[getglad]

So going that route, I still end up with the Error: Federation requires either a User Pool or Identity Pool in config and DataStore - Data won't be synchronized. No GraphQL endpoint configured. Did you forgetAmplify.configure(awsconfig)? errors, unless I run like the below.

    Amplify.configure({
      ...awsconfig,
      Auth: ampAth
    })
    Auth.configure(Amplify.configure())
    DataStore.configure(Amplify.configure())

So while this change "works" - it works just as well, still producing the original errors.

To reiterate - the issue appears as soon as a wss connection is created, not when a mutation is attempted. Given the language - "Validation error of type MissingFieldArgument - really wondering if something is missing from what the CLI is generating.

As an extra quirk and dirty check, I have tried calling DataStore.start() and it also produces the "MissingFieldArgument" message.

[amhinson]

I'm still curious on the logging issue, since we'd be able to tell a lot about what might be going on from that. There should be additional DataStore and AWSAppSyncRealTimeProvider logs. I'd still suggest looking more into why this behavior is happening, because it could somehow be related to the underlying issue you are seeing.

Regardless, would you be able to provide the websocket message data for the start query? You can find it in the Network tab in chrome and selecting the graphql?header=... WS request and looking through the messages. I'm particularly interested in the payload.data of one of the start messages.

Also, could you share the Recipe type from the generated file at amplify/backend/api/<API_NAME>/build/schema.graphql?

[getglad]

I'd still suggest looking more into why this behavior is happening, because it could somehow be related to the underlying issue you are seeing.

Agreed - just not sure what to be doing... If you have a troubleshooting tip happy to try it. The below is still just spitting out the lines from above - it's like something is causing a bailout?

    Amplify.Logger.LOG_LEVEL = 'DEBUG'
    Amplify.configure({
      ...awsconfig,
      Auth: ampAth
    })
    Auth.configure(Amplify.configure())
    DataStore.configure(Amplify.configure())

I'm particularly interested in the payload.data of one of the start messages.

I'm not seeing a start message in the appsync WSS. Some additional data points if helpful...

WSS log:

12:56:54.381 GET wss://<endpoint>.appsync-realtime-api.us-east-1.amazonaws.com/graphql?header=<auth>=&payload=e30=
[HTTP/1.1 101 Switching Protocols 687ms]

12:56:55.911 [WARN] 56:55.912 DataStore - subscriptionError Connection failed: {"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onDeleteIngredient'"}]} ConsoleLogger.js:107
12:56:55.914 [WARN] 56:55.915 DataStore - Sync error Connection failed: {"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onDeleteIngredient'"}]}

A couple of back and forth messages...

up: {"id":"f86b52ef-9691-4c3a-b335-58c5574a2a68","payload":{"data":"{\"query\":\"subscription operation {\\n  onCreateRecipe {\\n    id\\n    name\\n    description\\n    sourceType\\n    owner\\n    _version\\n    _lastChangedAt\\n    _deleted\\n    source {\\n      id\\n      _deleted\\n    }\\n  }\\n}\\n\",\"variables\":{}}","extensions":{"authorization":{"Authorization":"<auth>","host":"<endpoiint>.appsync-api.us-east-1.amazonaws.com","x-amz-user-agent":"aws-amplify/3.8.7 js"}}},"type":"start"}
down: {"id":"f86b52ef-9691-4c3a-b335-58c5574a2a68","type":"error","payload":{"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onCreateRecipe'"}]}}
up: {"id":"e4fae4b4-b7c9-44f7-879d-6a0227da8a54","payload":{"data":"{\"query\":\"subscription operation {\\n  onUpdateRecipe {\\n    id\\n    name\\n    description\\n    sourceType\\n    owner\\n    _version\\n    _lastChangedAt\\n    _deleted\\n    source {\\n      id\\n      _deleted\\n    }\\n  }\\n}\\n\",\"variables\":{}}","extensions":{"authorization":{"Authorization":"<auth>","host":"<endpoint>.appsync-api.us-east-1.amazonaws.com","x-amz-user-agent":"aws-amplify/3.8.7 js"}}},"type":"start"}
down: {"id":"e4fae4b4-b7c9-44f7-879d-6a0227da8a54","type":"error","payload":{"errors":[{"message":"Validation error of type MissingFieldArgument: Missing field argument owner @ 'onUpdateRecipe'"}]}}

could you share the Recipe type from the generated file

type Recipe {
  id: ID!
  name: String
  description: String
  ingredients(filter: ModelIngredientFilterInput, sortDirection: ModelSortDirection, limit: Int, nextToken: String): ModelIngredientConnection
  sourceType: String
  source: Source
  _version: Int!
  _deleted: Boolean
  _lastChangedAt: AWSTimestamp!
  createdAt: AWSDateTime!
  updatedAt: AWSDateTime!
  owner: String
}

[getglad]

This may be unrelated - but I am also noticing this message:

Warning: ./node_modules/amazon-cognito-identity-js/es/utils/cryptoSecureRandomInt.js
Module not found: Error: Can't resolve 'crypto' in '<path>node_modules/amazon-cognito-identity-js/es/utils'

I ran into this with DataStore before and it's use of buffer interrupting the owner flow - going to see if I can find a work around...

[amhinson]

Could you be able to reach out to me on Discord? We can have a quicker feedback cycle for debugging this type of issue. My username is amhinson.

[getglad]

@amhinson - happy to ping you - will be later today but we can sync there

[amhinson]

So going that route, I still end up with the Error: Federation requires either a User Pool or Identity Pool in config and DataStore - Data won't be synchronized. No GraphQL endpoint configured. Did you forgetAmplify.configure(awsconfig)? errors

Also, just for a sanity check on this issue when just using one configure() and aws-amplify, be sure to manually delete your node modules and remove any lock files before reinstalling. This actually fixes a lot of issues due to duplicate dependencies.

[getglad]

To TLDR some Discord convo... having refreshed the npm installs, was able to get the Amplify logger to spit out some helpful logs.

That lead me to https://github.com/aws-amplify/amplify-js/blob/main/packages/datastore/src/sync/processors/subscription.ts#L262

In my app, aws_cognito_region returns undefined, so either something is being checked incorrectly or set incorrectly, blowing up the oidcTokenPayload along the way...

If I comment out that check, I end up with clean looking logs and can get records to sync... so not sure what is happening upstream but I believe this to be the issue :fingers_crossed:

[amhinson]

Based on the config that you are passing Amplify.configure(), aws_cognito_region should be undefined since it isn't there. However, Auth.region should exist, so I don't think this is the root cause of the issue you are seeing.

However, based on the logs you provided, I do see this error (AuthError - Error: Amplify has not been configured correctly.), which typically means that something is off either with the Amplify dependency versions in your project or that calling configure() for each category is causing problems. I'm leaning towards the latter, since I think all of your deps are up to date.

Would you be able to try again with only calling Amplify.configure() without any other category configure? I know you tried that before, but lets try to troubleshoot any errors that might come up. Feel free to respond to me on Discord if that is easier.