Firebase-tools: firebase serve functions + functions.database.ref('/').onWrite stalls without explanation

[anfedorov]

Version info

3.15.0
npm 6.11.1 (anything else gives error)
firebase-admin 5.4.3
firebase-functions 0.7.3

Steps to reproduce

const functions = require("firebase-functions");

const watchWrite = functions.database.ref('/').onWrite(event => {
  console.log("Hello from Firebase!");
});

exports.watchWrite = watchWrite;

then run

$ firebase serve --only functions

=== Serving from '/Users/fedorov/code/thingimbuilding/ce'...

i functions: Preparing to emulate functions.

Expected behavior

the command to not stall and functions to start serving

Actual behavior

icebucket:ce fedorov$ firebase serve --only functions --debug
[2017-11-13T00:09:31.466Z] ----------------------------------------------------------------------
[2017-11-13T00:09:31.473Z] Command: /Users/fedorov/n/bin/node /Users/fedorov/n/bin/firebase serve --only functions --debug
[2017-11-13T00:09:31.474Z] CLI Version: 3.15.0
[2017-11-13T00:09:31.474Z] Platform: darwin
[2017-11-13T00:09:31.474Z] Node Version: v6.11.1
[2017-11-13T00:09:31.475Z] Time: Sun Nov 12 2017 16:09:31 GMT-0800 (PST)
[2017-11-13T00:09:31.475Z] ----------------------------------------------------------------------

[2017-11-13T00:09:31.487Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2017-11-13T00:09:31.487Z] > authorizing via signed-in user
[2017-11-13T00:09:31.489Z] >>> HTTP REQUEST GET https://admin.firebase.com/v1/projects/my-proj-name
Sun Nov 12 2017 16:09:31 GMT-0800 (PST)
[2017-11-13T00:09:32.149Z] <<< HTTP RESPONSE 200 server=nginx, date=Mon, 13 Nov 2017 00:09:32 GMT, content-type=application/json; charset=utf-8, content-length=141, connection=close, x-content-type-options=nosniff, strict-transport-security=max-age=31536000; includeSubdomains, cache-control=no-cache, no-store
[2017-11-13T00:09:32.150Z] >>> HTTP REQUEST GET https://admin.firebase.com/v1/database/my-proj-name/tokens
Sun Nov 12 2017 16:09:32 GMT-0800 (PST)
[2017-11-13T00:09:32.706Z] <<< HTTP RESPONSE 200 server=nginx, date=Mon, 13 Nov 2017 00:09:32 GMT, content-type=application/json; charset=utf-8, content-length=274, connection=close, x-content-type-options=nosniff, strict-transport-security=max-age=31536000; includeSubdomains, cache-control=no-cache, no-store

=== Serving from '/Users/fedorov/code/thingimbuilding/ce'...

i functions: Preparing to emulate functions.
[2017-11-13T00:09:33.184Z] >>> HTTP REQUEST GET https://appengine.googleapis.com/v1/apps/my-proj-name
Sun Nov 12 2017 16:09:33 GMT-0800 (PST)
[2017-11-13T00:09:33.184Z] >>> HTTP REQUEST GET https://apikeys.googleapis.com/v1/projects/my-proj-name/apiKeys
Sun Nov 12 2017 16:09:33 GMT-0800 (PST)
[2017-11-13T00:09:33.707Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Mon, 13 Nov 2017 00:09:33 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="41,39,38,37,35", accept-ranges=none, connection=close
[2017-11-13T00:09:33.799Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Mon, 13 Nov 2017 00:09:33 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="41,39,38,37,35", accept-ranges=none, connection=close

[anfedorov]

if I upgrade I also get a nice warning

Warning: You're using Node.js v9.1.0 but Google Cloud Functions only supports v6.11.1.

[obbe79]

Are you behind a proxy? I have a similar behavior using deploy and I'm behind a proxy. I haven't found any solution so far.

[anfedorov]

Nope. Deploy CLI piece works fine, but is a big pain for development because it's slow and the instances take a minute or so to switch over to new versions of the code (at least based on the logs). Mitigating this by renaming my functions every time I deploy a new one but would much prefer to develop locally, of course.

[laurenzlong]

Hey @anfedorov that's a really interesting behavior. Have you ever been able to emulate the functions successfully in this project? And what trigger types are you using?

[anfedorov]

No, I have not. I have been in other projects, so seems it may be something specific to my code. I am using code that is compiled from TypeScript (targeting ES2015). Will debug more when I get the chance tonight & update.

[Jonovono]

@anfedorov have you solved this? I seem to be getting the same issue.

[anfedorov]

Thanks for the poke, sorry I forgot to follow up. It works when I replace my compiled index.js to

var functions = require('firebase-functions');

exports.helloWorld = functions.https.onRequest((request, response) => {
 response.send("Hello from Firebase!");
});

[Jonovono]

@anfedorov weird, that works for me too if I replace everything with that. I have my functions defined in different files though, but at least this gives me something to go off of. Thanks!

[anfedorov]

The smallest reproducing index.js I can find is

const functions = require("firebase-functions");

const watchWrite = functions.database.ref('/').onWrite(event => {
  console.log("Hello from Firebase!");
});

exports.watchWrite = watchWrite;

and it still breaks I include

const admin = require("firebase-admin");
admin.initializeApp(functions.config().firebase);

, as well.

@laurenzlong let me know if there's anything else I can do to help diagnose. I've updated to firebase-tools 3.15.4 and am on node v8.9.1 (the last LTS).

[inlined]

@anfedorov I've opened a pull request to add some better logging to help find out what is stalling for you. Could you temporarily install that version with

npm install -g git://github.com/firebase/firebase-tools#inlined.verbase-emulator-logging

and attach the end of your new debug output? You can get back to the public version of the CLI with

npm install -g firebase-tools

[krsnaa]

@anfedorov @inlined - I see the exact same behavior.
@inlined , I tried installing your logging version, but get this error:

npm ERR! code 1
npm ERR! Command failed: /usr/bin/git checkout inlined.verbase-emulator-logging
npm ERR! error: pathspec 'inlined.verbase-emulator-logging' did not match any file(s) known to git.

Here is the complete error log:
0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli '/usr/local/bin/npm',
1 verbose cli 'install',
1 verbose cli '-g',
1 verbose cli 'git://github.com/firebase/firebase-tools#inlined.verbase-emulator-logging' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session 183f7f9636c50324
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 silly fetchPackageMetaData error for git://github.com/firebase/firebase-tools.git#inlined.verbase-emulator-logging Command failed: /usr/bin/git checkout inlined.verbase-emulator-logging
7 silly fetchPackageMetaData error: pathspec 'inlined.verbase-emulator-logging' did not match any file(s) known to git.
8 verbose stack Error: Command failed: /usr/bin/git checkout inlined.verbase-emulator-logging
8 verbose stack error: pathspec 'inlined.verbase-emulator-logging' did not match any file(s) known to git.
8 verbose stack
8 verbose stack at ChildProcess.exithandler (child_process.js:275:12)
8 verbose stack at emitTwo (events.js:126:13)
8 verbose stack at ChildProcess.emit (events.js:214:7)
8 verbose stack at maybeClose (internal/child_process.js:925:16)
8 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
9 verbose cwd /Users/kewlking/src/webstorm-projects/Gaa/gaa-carnatic-web/functions
10 verbose Darwin 16.7.0
11 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "git://github.com/firebase/firebase-tools#inlined.verbase-emulator-logging"
12 verbose node v8.9.3
13 verbose npm v5.5.1
14 error code 1
15 error Command failed: /usr/bin/git checkout inlined.verbase-emulator-logging
15 error error: pathspec 'inlined.verbase-emulator-logging' did not match any file(s) known to git.
16 verbose exit [ 1, true ]

[laurenzlong]

@krsnaa Can you try this command instead?

npm install -g https://github.com/firebase/firebase-tools.git#inlined.verbase-emulator-logging

[krsnaa]

@laurenzlong - same error:
npm ERR! code 1
npm ERR! Command failed: /usr/bin/git checkout inlined.verbase-emulator-logging
npm ERR! error: pathspec 'inlined.verbase-emulator-logging' did not match any file(s) known to git.

I thought there was a typo in 'verbase' and tried verbose also. same error.

[anfedorov]

Shouldn't the branch show up here?
https://github.com/firebase/firebase-tools/branches/all

On Mon, Dec 18, 2017 at 9:03 PM, krsnaa [e-mail] wrote:

@laurenzlong https://github.com/laurenzlong - same error:
npm ERR! code 1
npm ERR! Command failed: /usr/bin/git checkout inlined.verbase-emulator-
logging
npm ERR! error: pathspec 'inlined.verbase-emulator-logging' did not match
any file(s) known to git.

I thought there was a typo in 'verbase' and tried verbose also. same error.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/firebase/firebase-tools/issues/535#issuecomment-352640626,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAmXDJa802T4U8qwLrDQxaAOFVrltVDks5tB0OcgaJpZM4QbG2z
.

[krsnaa]

what's weird is that the emulator works interactively with > 'firebase experimental:functions:shell'

[inlined]

@anfedorov @krsnaa Sorry, I cleaned up my branch when it hit master. This should work:

npm install -g https://github.com/firebase/firebase-tools.git

[krsnaa]

@inlined thanks so much for the update. The following is my debug spew. If you need me to downgrade to node 6.11.5, please let me know. But in my previous attempts, apart from the warning about the unsupported node version, there was no difference...

$ firebase serve --only functions --debug

[2017-12-22T05:28:25.741Z] ----------------------------------------------------------------------
[2017-12-22T05:28:25.744Z] Command: /usr/local/bin/node /usr/local/bin/firebase serve --only functions --debug
[2017-12-22T05:28:25.744Z] CLI Version: 3.16.0
[2017-12-22T05:28:25.744Z] Platform: darwin
[2017-12-22T05:28:25.744Z] Node Version: v8.9.3
[2017-12-22T05:28:25.745Z] Time: Fri Dec 22 2017 10:58:25 GMT+0530 (IST)
[2017-12-22T05:28:25.745Z] ----------------------------------------------------------------------

[2017-12-22T05:28:25.755Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2017-12-22T05:28:25.755Z] > authorizing via signed-in user
[2017-12-22T05:28:25.757Z] >>> HTTP REQUEST GET https://admin.firebase.com/v1/projects/fir-fem

Fri Dec 22 2017 10:58:25 GMT+0530 (IST)
[2017-12-22T05:28:27.307Z] <<< HTTP RESPONSE 200 server=nginx, date=Fri, 22 Dec 2017 05:28:27 GMT, content-type=application/json; charset=utf-8, content-length=101, connection=close, x-content-type-options=nosniff, strict-transport-security=max-age=31536000; includeSubdomains, cache-control=no-cache, no-store
[2017-12-22T05:28:27.309Z] >>> HTTP REQUEST GET https://admin.firebase.com/v1/database/fir-fem/tokens

Fri Dec 22 2017 10:58:27 GMT+0530 (IST)
[2017-12-22T05:28:28.586Z] <<< HTTP RESPONSE 200 server=nginx, date=Fri, 22 Dec 2017 05:28:28 GMT, content-type=application/json; charset=utf-8, content-length=254, connection=close, x-content-type-options=nosniff, strict-transport-security=max-age=31536000; includeSubdomains, cache-control=no-cache, no-store

=== Serving from '/Users/kewlking/src/webstorm-projects/Gaa/gaa-carnatic-web'...

i functions: Preparing to emulate functions.
[2017-12-22T05:28:29.053Z] Fetching environmenmt
[2017-12-22T05:28:29.054Z] >>> HTTP REQUEST GET https://appengine.googleapis.com/v1/apps/fir-fem

Fri Dec 22 2017 10:58:29 GMT+0530 (IST)
[2017-12-22T05:28:29.055Z] >>> HTTP REQUEST GET https://apikeys.googleapis.com/v1/projects/fir-fem/apiKeys

Fri Dec 22 2017 10:58:29 GMT+0530 (IST)
[2017-12-22T05:28:30.541Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 22 Dec 2017 05:28:30 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35", accept-ranges=none, connection=close
[2017-12-22T05:28:30.783Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 22 Dec 2017 05:28:30 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35", accept-ranges=none, connection=close
[2017-12-22T05:28:30.783Z] Starting @google-cloud/functions-emulator
Warning: You're using Node.js v8.9.3 but Google Cloud Functions only supports v6.11.5.
[2017-12-22T05:28:31.848Z] Parsing function triggers

[laurenzlong]

@krsnaa Could you downgrade your Node version to v6.11.5, updating your firebase-tools pulled in a new version of the GCF emulator, which has stricter Node version requirements.

[krsnaa]

@inlined @laurenzlong, a quick clarification:

Is 'firebase serve --only functions' supposed to work only with HTTP triggers or is it also supposed to be able to work with Realtime Database triggers?

The reason I ask is that it works great for HTTP triggers and additionally, the documentation at https://firebase.google.com/docs/functions/local-emulator indicates that "This command outputs a URL for your Firebase hosting content, and a URL for each HTTPS function."

The errors that we are seeing above with the emulator failing to start are happening with the Realtime Database triggers.

[inlined]

Firebase serve does not locally serve background functions, only HTTP functions. If you want to interactively play with database functions, try the shell: firebase experimental:functions:shell. The shell will help you create fake events to send to your functions.

[The-Alchemist]

I can also confirm that I get a hang at Parsing function triggers. My index.js is:

'use strict';

const admin = require('firebase-admin');
const functions = require('firebase-functions');


var serviceAccount = require("./47152072d8.json");

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    databaseURL: "https://my-project.firebaseio.com"
});
var db = admin.firestore();


exports.createUserDocument = functions.auth.user().onCreate(event => {
    const user = event.data; // The Firebase user.

    const email = user.email; // The email of the user.
    const displayName = user.displayName; // The display name of the user.
    const emailVerified = user.emailVerified;

    return db
        .collection('users')
        .doc(user.uid)
        .set({
            email,
            displayName,
            emailVerified
        }).then(() => {
            db
                .collection('users')
                .doc(user.uid)
                .collection('roles')
                .doc('FAKE_ROLE')
                .set({});
        });
});

If I change it to:

var functions = require('firebase-functions');

exports.helloWorld = functions.https.onRequest((request, response) => {
 response.send("Hello from Firebase!");
});

The emulator starts fine.

> firebase --version
3.17.1
> node --version
v6.12.3
> npm --version
5.6.0
>

[anfedorov]

@inlined could you make it error & quit or warn & ignore instead of hanging?

[ilearnio]

Phew! This "Parsing function triggers" hanging glitch was driving me insane for like two hours. It appeared that this was because I had a backup copy of "node_modules" folder in my "functions" folder. After removing it all get back to normal! Hope this will help someone

[inlined]

@anfedorov This is a great suggestion; so long as we don't support emulating a background function we could explain that there are no HTTP functions discovered and exit early. @laurenzlong thoughts?

[laurenzlong]

Yes good suggestion, I don't have the bandwidth to make a PR at the moment, but am happy to review if anyone wishes to make a PR.

[hanklidchi]

Seems like my antivirus software was causing the problem. I disabled it just for a minute while I ran firebase deploy and it deployed fine. Then re-enabled it. Using Kaspersky btw.

[phra]

i have a similar problem but my CPU goes to 100% :\
i'm debugging the issue..

EDIT: it seems that after few minutes it works, but it's very slow compared to before.. not sure what changed exactly. (seconds vs minutes)

[anfedorov]

@phra that sounds like a different issue. this might be worth closing to make a feature request instead?

key point is the hearted post above starting "we don't support..."

[anfedorov]

although, tbh, if you could just have a way to tunnel function from a public URL (or datastore!) to my laptop's dev environment, that would be aaamazing. I know there are other services that do it, but they keep coming and going as there doesn't seem to be much of a business model there and having a solid implementation with no extra dev dependencies would make me so happy.

[phra]

@anfedorov i've debugged the issue with a profiler and yes, you are right, the problem is due to the zipping of functions folder. it will loop for every function and compress everytime the directory..

[realtebo]

@anfedorov

I confirm

[debug] [2018-06-16T09:30:40.695Z] Parsing function triggers
[debug] [2018-06-16T09:30:41.352Z] Error while deploying to emulator: RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= -2147483648 and <= 2147483647. Received 2858637923
RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= -2147483648 and <= 2147483647. Received 2858637923
    at checkInt (internal/buffer.js:35:11)
    at writeU_Int32LE (internal/buffer.js:515:3)
    at Buffer.writeInt32LE (internal/buffer.js:684:10)
    at Object.entryHeaderToBinary (C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\adm-zip\headers\entryHeader.js:216:18)
    at Object.packHeader (C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\adm-zip\zipEntry.js:260:39)
    at C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\adm-zip\zipFile.js:198:41
    at Array.forEach (<anonymous>)
    at Object.compressToBuffer (C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\adm-zip\zipFile.js:182:23)
    at Object.writeZip (C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\adm-zip\adm-zip.js:454:32)
    at client.generateUploadUrl.then.then (C:\Users\mirko\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\@google-cloud\functions-emulator\src\cli\controller.js:168:13)
[warn] !  functions: Failed to emulate refresh

[may17]

I`ve got the same problem. The example from @anfedorov works fine:

var functions = require('firebase-functions');

exports.helloWorld = functions.https.onRequest((request, response) => {
 response.send("Hello from Firebase!");
});

If i include

const admin = require("firebase-admin");
admin.initializeApp(functions.config().firebase);

nothing happens.

Very frustrating :( Local development works fine 4 months ago. I didn`t changed something.

[phra]

@may17 functions.config().firebase was removed in 1.0, see https://firebase.google.com/docs/functions/beta-v1-diff#functionsconfigfirebase_removed

[may17]

@phra thanks. I´ve replaced it but the problem still exists. I`m just getting this two informations:

i  functions: Preparing to emulate functions.
i  functions: No HTTPS functions found. Use firebase functions:shell if you would like to emulate other types of functions.

[phra]

try to leave it there for 10 minutes.. if it's working then you have to clean up your directories as i said above: https://github.com/firebase/firebase-tools/issues/535#issuecomment-392487379

[libersolis-dev]

In order for cloud functions to work as expected aren't they supposed to

1. send a response object back to the client for https
   or
2. return a promise if a background function

I had this the issue with the node versions and what helped me was npm rebuild and I have 6.11.5 installed via nvm so I can switch back and forth.

also I only use admin.initializeApp() to initialize my functions.

When I read the code above, the thing that sticks out to me is that most of the issues people are having are happening when there there is no response sent or promise returned.

[abeisgoat]

Hey everyone!

Thanks for all the feedback on this issue. In May '19 we replaced the existing functions emulator with a completely rewritten backend which at the very least will change the nature of this bug and probably makes it entirely obsolete.

As a result, all solutions / information in this thread is no longer relevant (as it's specific to the old backend). So I'm going to close this issue. If you are still seeing some uninteded behavior please file a new bug and we'll start a discussion with the context of the new emulator backend.