Amplify-js: apmlify + appsync subscriptions stop getting data after some time

[aaayushsingh]

Do you want to request a feature or report a bug?
bug
What is the current behavior?
subscriptions initially work but timeout after a while and no data is then received on the client.

What is the expected behavior?
subscriptions not timing out automatically. Or if timeout is a feature, then the ability to manually specify timeout/ disable timeout.

Which versions of Amplify, and which browser / OS are affected by this issue? Did this work in previous versions?
same across different versions

CONSOLE LOG [native code]: {
"[INFO] 24:12.893 MqttOverWSProvider": {
"errorCode": 8,
"errorMessage": "AMQJS0008I Socket closed.",
"uri": "wss://<uri>

[goatandsheep]

I believe the timeout is a day by default

[aaayushsingh]

@goatandsheep it works for ~2minutes for me and then I get this socket closed error.

[goatandsheep]

looked at my debug errors and my socket closes right away. I'm still trying to figure it out myself. I believe it's an issue with my credentials

[manueliglesias]

Hi @goatandsheep , @aaayushsingh

Are you establishing multiple concurrent subscriptions? or do you see this behavior even with just one subscription?

[aaayushsingh]

@manueliglesias same behaviour in both the cases

[manueliglesias]

When you establish a graphql subscription to AppSync, a handshake process takes place. The process is roughly as follows:

• A GraphQL subscription is sent to AppSync
• AppSync sends a response with mqtt connection information in the {extensions: ...} field Using the network inspector or similar tool for your platform, make a note of this info, we are using it later
• Amplify looks at the mqtt connection information and extracts the clientId, uri and topics
• Amplify connects to that uri and subscribes to the topic

You can try to manually connect using the connection info :

npx @manueliglesias/mqtt-ws-cli --url <url> --clientId <clientId> --topic <topic>

can you share the output of that?

[goatandsheep]

From the console I get the following error:

Uncaught (in promise) TypeError: Cannot read property 'subscription' of undefined
    at APIClass.eval (API.js?fb0e:559)
    at step (API.js?fb0e:40)
    at Object.eval [as next] (API.js?fb0e:21)
    at fulfilled (API.js?fb0e:12)

I'm getting the following from your suggestion

Connecting...
Offline!
Closed! Error: premature close
    at Socket.onclose (/Users/kemal/.npm/_npx/66123/lib/node_modules/@manueliglesias/mqtt-ws-cli/node_modules/end-of-stream/index.js:48:67)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at TCP._handle.close [as _onclose] (net.js:567:12)

[manueliglesias]

Still investigating this.

For context:
Amplify uses the paho mqtt client for mqtt over websockets connections

Seemingly related issues:

• rh389/react-native-paho-mqtt#4
• eclipse/paho.mqtt.javascript#106

[goatandsheep]

Thanks for the suggestions. That is the error but I haven't been able to figure out a fix. Things that I've tried that have marginally improved it:

• using flat Amplify imports only (instead of through e.g. { Auth } )
• checked my package-lock.json for duplicate instances of amplify. I was using amplify-cli as a dev dependency but removed that and made it global
• ensured aws-exports.js file is flat. The sample i started with already had it flat.

What i have yet to try:

• redo cognito/ auth strategy. I get occasional errors with cognito and i suppose it is related to that

[manueliglesias]

@goatandsheep
In which environment are you running this? (react, angular, vue, react native, nativescript, etc)

We found that he paho library might be behaving differently in nativescript

[goatandsheep]

I'm running it in vue in browser.

[manueliglesias]

@goatandsheep Are you importing/requiring a module that overrides the browser's WebSocket object?

[goatandsheep]

Likely! I am using webpack-dev-server. I kinda messed up my subscriptions within some recent refactoring. I'll get back to you soon to see if it works without that!

[goatandsheep]

A different thing that i also was doing that I changed was that my graphql function, i was using functionName(item: ID) but i read i should only use type ID in my type declarations and not in my functions, so i changed that to String

[theharness]

Any progress made so far? I am getting the exact error message with the same symptoms of closed connections.

Also, for what it is worth, we have an Android developer having the exact same issues with Appsync using the AWS Apollo based integration.

[aaayushsingh]

@theharness What platform are you using this on? There is a connect timeout and you're probably facing the issue because of that. AWS team decided to not go forward with this because I'm using NativeScript and they don't support it. We did however succeed in making some progress before we decided to stop.

[theharness]

@aaayushsingh My issues specifically has been using JavaScript on React. However, we recently observed the same behavior in both the Appsync console, and on Android using the Apollo client. The symptoms point towards the issue being on the back end, but nothing is for sure.

[brubbel]

Subscriptions are also not reliable after a network change. Queries continue working (after some network errors), but subscriptions remain silent. Also after an unsubscribe/subscribe cycle.

I guess that the unsubscribe() does remove the subscription from the observer object, but the underlying paho-mqtt client stays untouched and waiting forever.

This issue makes appsync subscriptions not suited for mobile apps.

Edit:
Yes, definitely the paho-mqtt client is the culprit:

1. change networks (old/new subscriptions hang forever).
2. force the paho-mqtt client to disconnect.
3. resubscribe: all starts working again.

[jordanranz]

@aaayushsingh,

Are you still experiencing this issue or is this safe to close?

[aaayushsingh]

@jordanranz I haven't used amplify in a while. Last time I was using it, the issue still wasn't resolved (but that was a few months ago).

This can be closed in that sense. I'll reopen it if I face the issue again.

[artista7]

Hi @jordanranz @manueliglesias Im using amplify cli version 1.1.6. I have created a react redux app with graphQL API. I am using subscription, and it keeps breaking after 2-3 mins or so. The error it throws is :-

errorMessage: "AMQJS0008I Socket closed."
reconnect: undefined

This issue kind of defeats whole purpose of subscription if we have to keep checking if subscription is active and not closed.

Meanwhile I noticed a reconnect attribute in error object. Can I somehow use this to reconnect? If yes, then how?

[gotripglobal]

@jordanranz and @manueliglesias

We are a team of researcher's from IIT Delhi and we are facing somehow the same error.

Ie when we run the script. The socket gets closed and gives the error as shown in the image.

We're connecting directly to iot endpoint,

we are following the Amplify documentations ( https://aws-amplify.github.io/docs/js/pubsub#working-with-the-api/ )

[goatandsheep]

@gotripglobal where are you storing the reference to the subscription object? Also you subscribe, then unsubscribe right after. Separate it into different functions.

[artista7]

in the error code block, you can re-subscribe

var createCvReviewSubscription;
onCreateCvReviewSubscription() {
console.log('starting onCreateCvReviewSubscription');
var that = this;
createCvReviewSubscription = API.graphql(
graphqlOperation(subscriptions.onCreateCvReview)
).subscribe({
next: (data) => {
//do something with data
}
},
error: err => {
console.log('error in onCreateCvReviewSubscription', err);
this.state.createCvReviewSubscription.unsubscribe(); //just for safe side
that.onCreateCvReviewSubscription(); //reconnect to subscription
}
});
}

On Tue, 19 Mar 2019 at 00:31, Kemal Ahmed [e-mail] wrote:

@gotripglobal https://github.com/gotripglobal where are you storing the
reference to the subscription object?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/aws-amplify/amplify-js/issues/1283#issuecomment-474057015,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AO9fZS6LqAgk7TfIrSuzuvU_SJJ-7-2Qks5vX-KBgaJpZM4VYLGj
.

[artista7]

@aaayushsingh do you have any better solution? This is what has been helping me and is working pretty fine.
Can you describe why you did not like the above solution?

[goatandsheep]

@artista7 good suggestion but it's not feasible to have to re-subscribe every 2 minutes. He's trying to figure out if the initial setup is even correct. Also your suggestion will cause an infinite loop when there's a network error

[aaayushsingh]

@artista7 adding to what @goatandsheep said above.

There's a slight delay between disconnect and reconnect and you might end up not getting some data. Unless you also query for any missed data(which then makes subscriptions useless) each time a reconnect happens, there's no way to know if there were some messages which never got delivered.

If you want, set up two clients and publish some message from one client at 100ms intervals. Then check the messages lost while reconnect happens only the other client.

sorry for closing and reopening. I clicked that by mistake.

[artista7]

@goatandsheep @aaayushsingh both your points are valid and i also faced the same issues. By far i found that as the best approach. Though this is not obviously the ultimate solution. Im also eager to learn about the fix.

[jordanranz]

@gotripglobal The issue you are experiencing with PubSub seems related to this:
https://github.com/aws-amplify/docs/issues/460#issuecomment-481499155

We have a PR in the works to fix the documentation here.

@artista7 There is an example for using reconnect here:
https://github.com/aws-amplify/amplify-js/issues/1844#issuecomment-427193413

This example handles a failed subscription and executes exponential backoff retry logic.

[haverchuck]

Closing due to inactivity. If the responses above are inadequate, please feel free to open another issue with additional detail. Thank you.

[jordimuntada]

I am having this error. And I am sure the code is OK because it comes from a tutorial.
And I think it is related to what is being discussed here.

TypeError: Cannot read property 'items' of undefined

I can't find a solution whatsoever.

[JenilynnB]

I'm also experiencing this error on a React Native app using Appsync and Apollo. I'm using Amplify to manage all the Cognito Auth functions. My subscription is initialized on the subscribeToMore method returned from an Apollo Query HOC, as follows:

<Query query={getMilkByStash} variables={{milkStashID: this.state.user.milkStashID}} errorPolicy='all'>
  {({loading, error, data, refetch, subscribeToMore})=>{
    if(data){...}
    if(error){...}
    <Milk
      stash={data}
       refetch={refetch}
       subscribeToNewMilk={()=>[
         subscribeToMore({
           document: updateMilkSubscription,
           variables: {milkStashID: this.state.user.milkStashID},
           updateQuery: this.updateQuery,
           onError: (error)=>{console.log("subscription error", error)}}),
         subscribeToMore({
           document: deleteMilkSubscription,
           variables: {milkStashID: this.state.user.milkStashID},
           updateQuery: this.updateQuery,
           onError: (error)=>{console.log("subscription error", error)}})
         ]}
    />
</Query>

The subscriptions do get set up and work correctly, but after a few minutes I get a error with AMQJS0008I Socket closed.

Is there any update on this?

[livioso]

@JenilynnB Same setup, same issue. :) Did you find a workaround in the meantime? :)

[MaiconGilton]

I was facing the same problem, I just did a workaround I found here: @https://github.com/aws-amplify/amplify-js/issues/2610#issuecomment-471816736, it seems to be working now

[JenilynnB]

@livioso In onError I resubscribe to the query. It seems to be working so far. Though the data I'm subscribing to isn't coming at the speed of a chat app, so I'm not as concerned about missing any data when reconnecting. Here's the general idea of what I had to do to get it working.

Component function (along with similar function for subscribing to item deletions):

milkUpdateSubscriptionFunction(subscribeToMore, retrying=50){
  subscribeToMore({
    document: updateMilkSubscription,
    variables: {milkStashID: this.state.user.milkStashID},
    updateQuery: this.updateQuery,
    onError: (error)=>{
      if(error.errorMessage.includes('Socket')){
        setTimeout(()=>{this.milkUpdateSubscribeFunction(subscribeToMore, retrying * 2)}, retrying)
      }
    }
  })
}

In component render method:

<Query query={getMilkByStash} variables={{milkStashID: this.state.user.milkStashID}} errorPolicy='all'>
  {({loading, error, data, refetch, subscribeToMore})=>{
    if(data){...}
    if(error){...}
    <Milk
      stash={data}
       refetch={refetch}
       subscribeToNewMilk={this.milkUpdateSubscriptionFunction}
       subscribeToMilkDeletions={this.milkDeletionSubscriptionFunction}
       subscribeToMore={subscribeToMore}
    />
</Query>

[plus-]

Is this still the accepted workaround or has a better option been integrated since then ?