Firebase-admin-node: Error when deploying to AWS Lambda (Node 6.10.3)

[shauntrennery]

I'm getting a "Unable to import module" error when deploying v5.3.0 or above to AWS Lambda. The error log is cryptic but it does contain reference to "firebase-admin/node_modules/grpc/src/node/src/grpc_extension.js:30:15"

All works as expected when downgrading to v5.2.1.

[srameshr]

@shauntrennery Same here! I did downgrade firebase-admin lib to 5.2.1 but no luck. How did you solve it?
https://github.com/firebase/firebase-admin-node/issues/105

[shauntrennery]

@srameshr I'm sorry, my solution was simply downgrading to 5.2.1. No other changes were needed.

[srameshr]

Have you done any changes to files in .ebextensions?

[hiranya911]

v5.2.1 doesn't depend on Firestore/grpc. @srameshr what is the exact error you get with v5.2.1?

I've pinged the Firestore SDK team about the grpc error. Awaiting their response.

[shauntrennery]

@srameshr I did not. 100% sure.

[wilhuff]

GRPC requires native code. There's some discussion about how to make this work here https://github.com/grpc/grpc/issues/6443

[srameshr]

@hiranya911 Let me deploy v5.2.1 again and check the logs, in the meanwhile I will update the question with my package.json file.

[srameshr]

@hiranya911 It's the same. Even on 5.2.1 it gives me the same error.

[hiranya911]

That doesn't make any sense. grpc wasn't a dependency until v5.4.0. Have you tried deleting your node_modules directory, and installing v5.2.1 on a fresh set up?

[srameshr]

@hiranya911 AWS EB is a nightmare. I removed firebase-admin from my package.json and deployed a new build and it still says gprc is not found.

I will restart my env and check the logs.

[hiranya911]

@shauntrennery can you share any detailed logs you have when you were experiencing this issue? We would like to get a better handle of this problem, and see how AWS users can be impacted.

[shauntrennery]

@hiranya911 Here is the log from CloudWatch:

Unable to import module 'survey': Error
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/var/task/node_modules/firebase-admin/node_modules/grpc/src/node/src/grpc_extension.js:30:15)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)

As mentioned, all I need to do to resolve the issue was to downgrade to version 5.2.1. No other changes were needed.

[hiranya911]

Thanks @shauntrennery. FWIW you should be able to get by with v5.3.0 as well.

[srameshr]

@hiranya911 I ssh into my eb machine and see that, package.json does not include firebase-admin and node-modules does not have firebase admin but still my latest deployment logs show the same error.

[hiranya911]

@srameshr Please share your logs. Perhaps we can try to trace down why it tries to load grpc. But given that Admin SDK is out of the picture, it's probably some other dependency you're using.

[srameshr]

@hiranya911 Yes, I will. Just running my tests against 5.3.0 on production. After that I will get the logs.
My previous comment was a result of some caching fault I guess. But I am sure, grpc error came from including firebase-admin module > 5.3.0

[PutziSan]

same error here, can confirm that it works with firebase 5.3.0 as expected (for me).

[hiranya911]

@PutziSan can you also share your logs? More info we have, easier it will be for us to resolve this.

And just FYI, there is some work in progress to fix this. We are trying to get to a state where the Admin SDK doesn't always try to import Firestore/grpc. This requires:

1. Separating Firestore type definitions into a separate module.
2. Lazy loading the Firestore namespace (admin.firestore).

[jpamorgan]

@hiranya911 do you foresee a fix coming anytime soon to get Firestore running on AWS Lambda with this grpc dependency?

I know this thread is about firebase-admin, but I also would like to get firestore running on Lambda as well.

[hiranya911]

@schmidt-sebastian @wilhuff can you comment on the status of Firestore/grpc support on AWS Lambda environment?

[murgatroid99]

One possible cause of this problem with gRPC is running the installation on a different platform than you are deploying on. The important things to consider are Node version (the major version has to match), operating system (Linux vs Windows vs Mac), and architecture (32 bit vs 64 bit vs ARM). If you can't easily do that, you can force it to install the right binary by passing additional arguments to npm install: --target=<node version> --target_arch=x64 --target_platform=linux

[wilhuff]

@jpamorgan There's no fix for running Firestore on Lambda. Firestore uses gRPC, and gRPC only supports node.js via native code. At this point we recommend dockerizing your deployments. We're working on a write-up on how to do this, but I can't promise a timeline.

[PutziSan]

Just same error like the ones from @shauntrennery (attached below).
Does anyone noticed the blog post from https://aws.amazon.com/de/blogs/compute/nodejs-packages-in-lambda/ , have no time yet for a closer look, but this may give a hint.

Native modules are similarly installed and deployed, but you’ll need to build them against the Amazon Linux libraries. You’ll need to either ensure that the libraries and their transitive dependencies are statically compiled or use rpath-style linking; we’ll do it the static way in this post, and demonstrate use of rpath in a subsequent post. (Note that many, but not all, libraries can be statically linked this way.)

Unable to import module 'firebaseServerAuth': Error
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/var/task/node_modules/grpc/src/node/src/grpc_extension.js:30:15)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)

[samtstern]

I am currently working on a guide for how to do this (as @wilhuff said). However the meat of it is that you need to use docker to install your dependencies.

If you rm -rf node_modules and then use this command:

docker run --rm -it -v "$PWD":/worker -w /worker node:6.10.3 npm i --production --silent

Your dependencies will be installed in a way that makes them compatible with Lambda's node 6.10 runtime. I have successfully written data to Cloud Firestore from AWS Lambda.

[samtstern]

Here's a full writeup:
https://medium.com/@samstern_58566/how-to-use-cloud-firestore-on-aws-lambda-4bf6d3a473d9

[hiranya911]

I've published an RC at https://storage.googleapis.com/firebase-admin-node/firebase-admin-5.4.3-rc0.tgz

@shauntrennery @srameshr can you run some tests with it on AWS and let us know if that resolves your issue? This should allow you to at least use the latest Admin SDK without bumping into any gRPC module loading problems.

[bradennapier]

This is extremely annoying issue - you guys really should have thought about this before deploying this... nothing is more annoying that everythign breaking when doing an update and no notice anywhere that it may occur.. When you start depending on C++ modules in Node.js its a pretty strong indicator that things are going to break for a fairly large percentage of people.

I can understand if I were actually adding the firestore somewhere but I am not...

On that note... @hiranya911 yes your given pacakge does appear to resolve the issue so thank you.

[srameshr]

@hiranya911 I did try it. It broke again.
I cant do it again because I dont have a dev version of EB stalk. My production server runs there. As of now I will stick with 5.3.0 without any updates.

[samtstern]

@bradennapier glad the new package is working for you. Please be respectful when discussing issues on Github, we try to be honest about our mistakes and fix them when we break something.

[hiranya911]

Thanks everybody who helped test this fix. It is now released: https://firebase.google.com/support/release-notes/admin/node#5.4.3

[andresmijares]

I'm running 5.7.3 and this issue still persists... any idea?