Firebase-functions: Receiving exception of stackoverflow by lodash from simple queries on onCall but not onRequest

Created on 28 Jun 2018 · 17Comments · Source: firebase/firebase-functions

from package.json:

        "firebase-admin": "~5.12.1",
        "firebase-functions": "^1.0.3",
        "mysql2": "^1.5.3",
        "sequelize": "^4.37.10"

And using this code:

const models = require('./db/models');

const throw_db_issue = e => {
  throw new functions.https.HttpsError('failed-db-issue', `DB issue: ${e.message}`);
};

exports.all_categories = functions.https.onCall((data, context) => {
  return models.Category.findAll({
    where: { parent_id: null },
    include: ['children', models.CategoryImage],
  }).catch(throw_db_issue);
});

Where models is defined as:

const Sequelize = require('sequelize');
const CategoryImage = sequelize.define(
  'category_images',
  {
    category_id: Sequelize.INTEGER,
    image_path: Sequelize.STRING,
  },
  {
    underscored: true,
  }
);

const Category = sequelize.define(
  'categories',
  {
    name: Sequelize.STRING,
    parent_id: Sequelize.INTEGER,
  },
  {
    underscored: true,
  }
);
CategoryImage.belongsTo(Category);
Category.hasOne(CategoryImage);

And I get this stack trace in firebase console:

Unhandled error RangeError: Maximum call stack size exceeded
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)

But this is fine if I run this code as under plain onRequest.

Please fix/suggest solution as I prefer onCall to onRequest

Source

fxfactorial

Most helpful comment

Hi. This error occurs when your callable function _returns_ an object that has cycle within it.

In @diegogarciar's case, the issue is that the function is returning the result of the call to userGroupsRef.once, which is a DataSnapshot. I suspect that instead of this:

return userGroupsRef.once("value", snapshot =>{

you probably meant to write this:

return userGroupsRef.once("value").then(snapshot =>{

The snapshot is the same either way, but if you want to chain the promises so that everything runs before the function completes (and you do), you have to do the latter.

In @fxfactorial's case, it looks like it's returning an array of sequelize Models. I'm assuming that object has some cycle in it. I'm not sure what the best fix there would be. If you could transform each Model into a plain JavaScript object with just the fields you care about, that would fix it. Maybe instead of this:

return models.Category.findAll({
    where: { parent_id: null },
    include: ['children', models.CategoryImage],
  }).catch(throw_db_issue);

you could do this:

return models.Category.findAll({
    where: { parent_id: null },
    include: ['children', models.CategoryImage],
  }).then(models => {
    for (var i = 0; i < models.length; i++) {
      models[i] = models[i].toJSON();
    }
    return models;
  }).catch(throw_db_issue);

bklimt on 31 Aug 2018

👍2

All 17 comments

Could you make a more minimal repro? I can't really tell what's going on here, especially it seems like the return result of your function depends on the implementation of sequelize (I assume that's what's providing the findAll function). My hunch is that findAll is creating an infinite recursion somehow, which would not be due to the firebase-functions SDK.

laurenzlong on 3 Jul 2018

I gave a lot of code, this is the situation because I use the same data producing code but in onRequest (which works fine) and Sequelize is a very popular and widely used library.

More than this would mean spinning up a db.

Sent from my iPhone

On Jul 3, 2018, at 2:10 PM, Lauren Long notifications@github.com wrote:

Could you make a more minimal repro? I can't really tell what's going on here, especially it seems like the return result of your function depends on the implementation of sequelize (I assume that's what's providing the findAll function). My hunch is that findAll is creating an infinite recursion somehow, which would not be due to the firebase-functions SDK.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

fxfactorial on 4 Jul 2018

I am having the same issue, I am not doing anything complex with my data, it seems it is failing during the call to Firestore, also if I remove the first database call, functions seems to work fine
```
exports.requestJoinGroup = functions.https.onCall((data, context) => {
const tripId = data.tripId;

if (tripId == undefined){
throw new functions.https.HttpsError('invalid-argument', 'The function must be called with ' +
'a group id.');
}
if(!context.auth){
throw new functions.https.HttpsError('failed-precondition', 'The function must be called ' +
'while authenticated');
}

const userId = context.auth.uid;

const userGroupsRef = admin.database().ref(/user_groups/${userId}/${tripId})

return userGroupsRef.once("value", snapshot =>{
const status = snapshot.val()
if(status != null){
throw new functions.https.HttpsError('permission-denied', User has already joined group ${tripId});
}else{
console.log("requesting join");

  return firestore.collection("testgroups").doc(tripId).get().then((doc) =>{
    console.log("trip obtained");
    if (!doc.exists) {
      throw new functions.https.HttpsError('not-found', `Trip doesnt exist: ${tripId}`);
    }
    var trip = doc.data();
    if(trip.privacy == "public"){
      console.log("trip is public");
      trip.user_ids[userId] = false;
      return doc.ref.update({"user_ids": trip.user_ids}, { merge: true })
      .then(()=>{
        console.log("added user");
        return userGroupsRef.set("idle");
      })

    }else{
      throw new functions.https.HttpsError('permission-denied', 'Trip is private: ' + tripId);
    }
  })
}

})
})
```

diegogarciar on 22 Aug 2018

@fxfactorial Taking another look at your logs, I think this function is what's causing the stack overflow: https://github.com/firebase/firebase-functions/blob/master/src/providers/https.ts#L337. In trying to encode your data, there's an infinite recursion caused. So if my theory is correct, you'll still get the issue even if you removed everything inside the function body. So the question is what does your client code look like when you are calling this function? What data are you calling it with?

@diegogarciar If you also see this line in your logs: at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18), then you're running into the same issue. otherwise, it's something different, and please file a new issue.

laurenzlong on 22 Aug 2018

Hello @laurenzlong, it seems the issue is in same file but some lines below
at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:242:18)

Unhandled error RangeError: Maximum call stack size exceeded
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13395:23)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:242:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
    at encode (/user_code/node_modules/firebase-functions/lib/providers/https.js:242:18)
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
    at /user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
    at Function.mapValues (/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)

diegogarciar on 22 Aug 2018

@diegogarciar What data are you calling the function with?

laurenzlong on 22 Aug 2018

Functions.functions().httpsCallable("requestJoinGroup").call(["tripId": "6nbKuzTcPUW6elSMosNhfmn1iXJ3_2"])

From my function logs I see the "requesting join" then the error occurs and the function finalization message appears. After a few seconds the logs show the "trip obtained" and rest of logs.

diegogarciar on 22 Aug 2018

Hi. This error occurs when your callable function _returns_ an object that has cycle within it.

In @diegogarciar's case, the issue is that the function is returning the result of the call to userGroupsRef.once, which is a DataSnapshot. I suspect that instead of this:

return userGroupsRef.once("value", snapshot =>{

you probably meant to write this:

return userGroupsRef.once("value").then(snapshot =>{

The snapshot is the same either way, but if you want to chain the promises so that everything runs before the function completes (and you do), you have to do the latter.

return models.Category.findAll({
    where: { parent_id: null },
    include: ['children', models.CategoryImage],
  }).catch(throw_db_issue);

you could do this:

return models.Category.findAll({
    where: { parent_id: null },
    include: ['children', models.CategoryImage],
  }).then(models => {
    for (var i = 0; i < models.length; i++) {
      models[i] = models[i].toJSON();
    }
    return models;
  }).catch(throw_db_issue);

bklimt on 31 Aug 2018

👍2

@fxfactorial @diegogarciar did @bklimt's suggestions resolve your issues?

thechenky on 29 Nov 2018

as @bklimt suggester using .then() as
return userGroupsRef.once("value").then(snapshot =>{ })
solved the issue perfectly, I adopted the syntax everywhere else. Thanks by the way @bklimt

diegogarciar on 29 Nov 2018

👍1

yes, thank you!

fxfactorial on 29 Nov 2018

👍1

Glad to hear :) thanks for closing. As always, feel free to open another issue if you encounter any other problems!

thechenky on 29 Nov 2018

👍1

I realize the issue is closed, however is there any possibility in making the error a little more intuitive? Or would this be a separate issue that I should open?

Danebrouwer97 on 31 Mar 2020

I'll have to defer to @ryanwilson on that. The fix would be to wrap the serialization in a try-catch and throw a different exception than the one we got back from the json serializer.

bklimt on 31 Mar 2020

Thanks for the speedy response,

Just an additional note:
I see in the encoding source file, that it uses the following code when dealing with an object.

if (_.isObject(data)) {
    // It's not safe to use _.forEach, because the object might be 'array-like'
    // if it has a key called 'length'. Note that this intentionally overrides
    // any toJSON method that an object may have.
    return _.mapValues(data, encode);
  }

However, using this method ignores any custom toJSON methods that are usually called when 'stringifying' the object.

This is the source of the issue in using Sequelize and returning an instance/instances of a model from firebase. An instance of a model has recursion and its custom toJSON method normally removes that, this can be easily tested by returning the 'stringified' instance from a firebase function.

return JSON.parse(JSON.stringify( ... )); // works

The easiest solution that I can see is to just update the error with a more intuitive output, perhaps even mention the fact that firebase's encoding method ignores any custom toJSON method.

Danebrouwer97 on 1 Apr 2020

👍1

Thanks, @Danebrouwer97. That's an interesting point. We should consider whether we can support toJSON on objects that have them without breaking the custom formatting for certain types.

bklimt on 1 Apr 2020

👍1

I changed the node version from 8 to 10 in the functions/package.json, and it worked