Nodejs-dialogflow: Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded

[developerarcgate]

After few transaction while detecting intent using dialogflow APi's, DEADLINE_EXCEEDED: Deadline Exceeded error is coming and after that always need to restart the service, then again it starts working for a while. Not getting any relevant answers even after trying lots of blogs.

Using node package: dialogflow
and dialogflow standard edition(Free Version)

//Piece of code
 Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded
     at Object.exports.createStatusError (/srv/node_modules/grpc/src/common.js:87:15)
     at Object.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:1188:28)
     at InterceptingListener._callNext (/srv/node_modules/grpc/src/client_interceptors.js:564:42)
     at InterceptingListener.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:614:8)
     at callback (/srv/node_modules/grpc/src/client_interceptors.js:841:24)
   code: 4,
   metadata: Metadata { _internal_repr: {} },
   details: 'Deadline Exceeded' } 

While using dialogflow node package, and here using
const sessionClient = new dialogflow.SessionsClient();
to detect intent
not getting the result as detected intent after hitting query for phrase.

[bcoe]

@archit150290 what version of the module are you installing? we haven't released the library since yesterday, so I was wondering if you're installing directly off of GitHub.

[callmehiphop]

@archit150290 would you be able to provide us with a minimal amount of code that we can use to reproduce your issue?

[developerarcgate]

@bcoe using version 0.8.0
And I have installed it directly via npm

[JayVanderlyn]

Also seeing this same issue. Ours is related to a inefficient and non-scalable SDK method listDocuments.

using v2Beta1 : https://github.com/googleapis/nodejs-dialogflow/blob/master/samples/detect.v2beta1.js

Referencing listDocuments method as an example. This is the function we used to debug:
https://github.com/googleapis/nodejs-dialogflow/blob/dead90b276f73f651fb6eba883b8bd2a3622299d/samples/detect.v2beta1.js#L175

Issue Example:
We have ~6500 documents in 1 Knowledgebase. When we use this method it makes 650 API calls in synchronous (Waiting for each to respond with next page hash). This puts it into the long running operations. According to our Google API stats, each request is taking 2-14 seconds to return. If we go with 2 seconds, that is 21 minutes to return 1 listDocuments method.. Not to mention this 1 listDocuments method often crushes our max operations per minute quota and we have to stop all API calls to google while completing this task or else we risk getting a Quota exceeded error.

Issue Example 2:
The web console is failing with the same error DEADLINE EXCEEDED. When we open the knowledgebase base it says "add your first document" even though there are around 6500 documents in there. After about 10 minutes, in the bottom right corner, a red dialog comes up with DEADLINE EXCEEDED. This verifies its not a code issue on our side, but a scalability, API design, architecture issue on the API server side.

We have 2 hacks around this to prevent our entire infrastructure from being crippled:

1. We used the pageSize param to increase to the max of 100 without getting an error response. However at 100 per call, the API response is around 14seconds to respond. Putting us at a total of 15 minutes of wait time for 1 listDocuments method.
2. We began caching and maintaining manifests of document lists and knowledge base data so that we didn't need the API as much. (Redis).

Als we did the following:

1. We requested higher quotas. However at this time the engineering team does not have the resources to complete the requests. We have been waiting for about 8 days now.
2. Opened a ticket with Dialogflow support. They have verified the issue and are assigning engineers to the ticket.

If you wish to classify this as an addition ticket, please do so. However we are getting the same error.

[callmehiphop]

@JayVanderlyn if you wanted to discuss more, I would suggest opening a new issue for it, but typically we offer a streaming version of list rpcs (in this case listDocumentsStream). I think this would better fit your needs given the large number of documents in your knowledgebase. If at anytime you want to stop fetching results you would just need to call stream.end()

[bcoe]

@developerarcgate could you please try the 1.0.0 version of the library, we recently changed the underlying library being used to interact with the upstream API.

If you are continuing to run into issues, could you please provide a code sample that demonstrates the issue?

[callmehiphop]

Haven't seen any activity here, so I'm going to close this out under the assumption that it has since been resolved. Please let us know if you continue to experience this issue and we'll be happy to re-open and investigate.

[HilalRather]

@callmehiphop, I am getting the same issue code": 4, "message": "Webhook call failed. Error: DEADLINE_EXCEEDED while working with Google.Cloud.Dialogflow.V2 C# library.
Although the webhook call gets executed successfully and returns the expected result.

Any help would be appreciated.

Thanks.

[callmehiphop]

@HilalRather I would suggest making an issue in the C# repo, I'm largely unfamiliar with that code base unfortunately.

[HilalRather]

@callmehiphop already did that.

[wtflove]

I am having exactly the same issue right now, don't know if there is a solution to this. Could you help me? @callmehiphop I rose another issue here

[yakim333]

Also getting Webhook call failed. Error: DEADLINE_EXCEEDED time to time and can't get what are conditions to reproduce (using fulfillment in NODEJS and FACEBOOK integration)

[CarioniStefano]

I'm also getting what it seems like random DEADLINE_EXCEEDED errors, with no clear reproduction steps.

It happens randomly on different intents on different occasion.

Any update?

[NicolaSoccal]

Having the same issue. It usually happens, but not always and not exclusively, the first time an intent with active fulfillment is called. Any update?

[oort7]

I am preparing json response myself and I also get this error from time to time.

[brntsw]

This error happens randomly for me too, even though there is no API call or some heavy background task going on in the webhook when it does and I have the paid plan, so that should not be a limitation issue.

[ankitnumer8]

Hello everyone, we are also facing similar issue. any solution for this?

[JoaquinIMT]

@callmehiphop This issue still is on, it happens in one of my agents but in the side of the webhook calls from dialogflow, have any advance?

[JBX028]

Hi, same issue for me. Half of my calls end with this error :(

[josegpulido]

Any update on this? I can't figure out how these threads are opened since 2 or 3 years ago and no one at the Google Team brings a solution or even an explanation at least. My team and I've spent so much time since weeks but nothing seems to work, literally. We thought about even migrate and leave definitively Firebase since this is a critical error to us.

[awolad]

Same here. Still, now I'm getting this error.

[vicrad]

The problem is in the firebase-admin version, the fulfillment is created with version 5.13.1, this for a reason that I do not understand, it randomly failed when it goes to generate the token, my solution was to edit the cloud function from the google console and edit the package.json to change the version 5.13.1 to 9.11.0 and lauch the new cloud function.

This is mi new package.json

"dependencies": {
"actions-on-google": "^2.2.0",
"firebase-admin": "^9.11.0",
"firebase-functions": "^2.0.2",
"dialogflow": "^0.6.0",
"dialogflow-fulfillment": "^0.5.0"
}