Firebase-functions: Error at Http2CallStream.call.on on .auth.user().onCreate

[brunopalaoro]

Related issues

536

[REQUIRED] Version info

node: 8.16.0

firebase-functions: ^8.2.0

firebase-tools: 7.2.3

firebase-admin: ^8.2.0

[REQUIRED] Test case

import * as admin from "firebase-admin";
import * as functions from "firebase-functions";
import { onCreateUserListener } from "./user-listeners";

admin.initializeApp(functions.config().firebase);

export const createUserListener = functions.auth.user().onCreate(onCreateUserListener);

/* user-listeners.ts */
export async function onCreateUserListener(user: UserRecord) {
    console.log("creating user", user);
}

[REQUIRED] Steps to reproduce

Don't know, happens randomly.

[REQUIRED] Expected behavior

The listener is executed when a user is created

[REQUIRED] Actual behavior

Randomly I get the following stack trace

Error
    at Http2CallStream.call.on (/srv/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
    at emitOne (events.js:121:20)
    at Http2CallStream.emit (events.js:211:7)
    at process.nextTick (/srv/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
    at _combinedTickCallback (internal/process/next_tick.js:132:7)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)"  

It's very similar to the stacktrace mentioned in the issue #536 but with no code, details, metadata and note, wich makes it even harder to debug.
The first thing my function does is to log the user being created, and not even that appeared on the log. Other times the log message appears but it seems as it is interrupted in the middle of execution, with the same stacktrace. I've got it in other firestore triggers as well, but in the createUser seems to happen more frequently.

Cloud Firestore Location: us-east1
Cloud Functions Region: us-central1

Maybe this helps:
insertId: "000000-03c08893-7a26-4f9e-a473-eb968a219339"
labels."execution_id"="728641127513661"
trace: "projects/beer-awards-platform/traces/c7074db5ea3c72b6f9a9700b5e1a21fd"

Were you able to successfully deploy your functions?

Yes

[FrancoBobadilla]

I've had the same issue twice. Both in functions.firestore.document(...).onCreate() event.
This error happens randomly
I was able to succesfully deploy my function

package.json:
"dependencies": {
"@google-cloud/storage": "^2.5.0",
"@sendgrid/mail": "^6.4.0",
"firebase-admin": "^8.3.0",
"firebase-functions": "^3.2.0",
},

region: us-central1

insertId: "000000-bb086cb9-f9d6-481c-baf4-6113e53508c0"
labels: {
execution_id: "733464334228130"
}

textPayload: "Error
at Http2CallStream.call.on (/srv/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
at emitOne (events.js:121:20)
at Http2CallStream.emit (events.js:211:7)
at process.nextTick (/srv/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
at _combinedTickCallback (internal/process/next_tick.js:132:7)
at process._tickDomainCallback (internal/process/next_tick.js:219:9)"

[tomoyuki28jp]

I've had the same issue.

Error
    at Http2CallStream.call.on (/srv/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
    at emitOne (events.js:121:20)
    at Http2CallStream.emit (events.js:211:7)
    at process.nextTick (/srv/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
    at _combinedTickCallback (internal/process/next_tick.js:132:7)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)

import * as functions from "firebase-functions";
import * as admin from "firebase-admin";
import stripe from "./stripe";

exports.default = functions.auth.user().onCreate(async user => {
        const customer = await stripe.customers.create({
            name: user.displayName,
            email: user.email,
            phone: user.phoneNumber,
            preferred_locales: ["ja"],
            metadata: { uid: user.uid }
        });
        return admin
            .firestore()
            .doc(`stripeCustomers/${user.uid}`)
            .set({
                customerId: customer.id,
                createdAt: admin.firestore.FieldValue.serverTimestamp(),
                updatedAt: admin.firestore.FieldValue.serverTimestamp()
            });
    });

[jbrew138]

It's the same error for me as well:

Error
    at Http2CallStream.call.on (/srv/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
    at emitOne (events.js:121:20)
    at Http2CallStream.emit (events.js:211:7)
    at process.nextTick (/srv/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
    at _combinedTickCallback (internal/process/next_tick.js:132:7)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)

I can't share the code publicly, as it's considered proprietary. But similar to @tomoyuki28jp we're creating a Firestore document for the user and creating a Stripe customer as well. Generally, we create a Firestore document for user writable data, check a Firestore document for captcha validation, create a Firestore document for read-only user data, and then create the Stripe customer. It's coded as an asynchronous function, using await for all Firestore calls.

According to the logs, the error occurs in less than 100ms from when the function was started, so it seems unlikely to be a timeout/deadline exceeded error. The first document was never created, so that tells me that it's being thrown before or as soon as the first call to Firestore is being made. Although the original issue shows code without Firestore.

This happened when a new customer signed up, so this could really hurt us in the long run. I attempted to create an account myself and the logs show the same error.

EDIT: This recently just started happening. We had a user last week sign up without any issues in this area. The last time we deployed the function before this was well over a month ago.

[piavgh]

I have the same issue when exporting Firestore to Google Cloud Storage to backup.

It happened randomly for some days, but other days it's ok

[lego651]

I got it fixed by upgrading dependencies.

It looks like the issue you're experiencing might be caused by a recent gRPC bug which has since been fixed.

The bug is fixed once your dependencies are using later versions than these:
Firebase dependency
└─┬ [email protected]
└── @grpc/[email protected]

[SurfingInWavelets]

It seems as we have the same error. If a user registers in the app he will be created correctly in firebase.
Then usually the trigger function auth.user().onCreate(...) is triggered, but sometimes not.

Testing the with just one user it happens more or less 1 of 100 times, that the function is not invoked. Testing the App with 30 persons at the same time it happend in case of 10 of 30 users :-(.

Updating the dependencies did not help. We use functions located in europe-west1
In contrast to previous feedback: We get no error message.
So first question: Where do you get the error message from?
Second question: Anyone with a fix or a nice workaround?

PS: It looks like we will avoid the trigger function and do everything by function calls of the App.

[deremer]

We're observing something like this but with .onDelete().

If we delete from the Firebase console, the function triggers without issue.

But, if we delete from the mobile app (in this case it's a Flutter app using the Firebase packages), the delete seems to happen (app responds as expected, user is no longer in Authentication console), but the function does NOT trigger.

We reverted to an old version of the app where this is known to be working and it still is not working, so we are thinking it's the function.

"firebase-functions": "~3.2.0"

[andieromero]

Thanks for submitting this issue. It looks like this is a known bug with the europe-west1 region. I'll be sure to update the progress on this bug - thanks for being patient!

Internal tracking: 145502073

[tomoyuki28jp]

@andrianabeltran I've been facing this issue with the us-central1 region.

[andieromero]

I've let the team know that some developers are experiencing this issue in us-central1 region as well. I'll be sure to provide updates.

[andieromero]

@tomoyuki28jp @deremer @brunopalaoro
Thanks for your patience everyone. To investigate the issue, we need a recent failure of a function that hasn't been deleted since server logs get deleted after a while. We suspect that it's an issue with Pub/Sub but need to verify.

Can you provide the function name of a function with a recent failure that is experiencing this issue in us-central1?

[thebeingapp]

Hi
I am facing similar issues, trying to run a nodejs client code to push close to ~250 documents in a Firestore collection. This was working the last I tried about couple of months ago. The same code when I tried this morning is throwing a bunch of these error messages:

Error writing document: { Error: 4 DEADLINE_EXCEEDED: Deadline exceeded
at Object.callErrorFromStatus (/Users/abhi/utils/firestore-upload/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
at Http2CallStream.call.on (/Users/abhi/utils/firestore-upload/node_modules/@grpc/grpc-js/build/src/client.js:96:33)
at emitOne (events.js:121:20)
at Http2CallStream.emit (events.js:211:7)
at process.nextTick (/Users/abhi/utils/firestore-upload/node_modules/@grpc/grpc-js/build/src/call-stream.js:100:22)
at _combinedTickCallback (internal/process/next_tick.js:132:7)
at process._tickCallback (internal/process/next_tick.js:181:9)
code: 4,
details: 'Deadline exceeded',
metadata: Metadata { internalRepr: Map {}, options: {} } }

Any suggestions?

[askilondz]

@andrianabeltran I just received this error in us-central1 with a function named: dbUsersOnCreate but it wasn't on auth.user().onCreate() it was on firestore.document('users/${userId}).onCreate()

[mikebcurtis]

@andrianabeltran I also received this error in us-central1. My function is an onUpdate. The name of the function is 'startLivestreamNotifications'.

[bnetter]

same error here!

[StevenH86]

I have a pubsub function that drops a bunch of data into the db that other functions pick up and process... Errors happen randomly. Most of the time no problem, but this pubsub runs every 45 mins so during a 24hr period might get anywhere from 5 - 10 fails.

[andieromero]

@StevenH86 What is your pubsub function name?

[google-oss-bot]

Hey @brunopalaoro. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@brunopalaoro if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.