Nx: Allow development of Firebase Cloud Functions

[Bielik20]

Cloud Functions are a great way of developing a micro-service based backend. Just like with node backend it is common to share types and some business logic between the two. Could we come up with some way of integrating them with an Nx Workspace? What are the things that people would like to see in that area?

Current Behavior

No _easy_ way of integrating a Cloud Functions within an Nx Workspace.

Expected Behavior

Developers should be able to easily develop a Cloud Functions backend within an Nx Workspace.

[jmarbutt]

I am interested in this but I am not sure what all I would want on this to make it work easier.

[jmarbutt]

I would like to see it easier to build out the functions project while referencing libraries. Here are some other ideas:

• Add Triggers for FireStore, Storage, Http, etc (with tests)
• Emulate Firestore
• Unit Testing Functions
• Add Function to core index so it is exported correctly.

I think that is it off the top of my head.

[Bielik20]

I think we could divide it into milestones:

1. MVP
   
   
   
   1. Scaffold Cloud Functions project that uses the root package.json (does not create a new one)
   
   
   2. Cloud Functions can reference 'libs'
   
   
   3. Triggers
   
   
2. Emulate Firestore / Local development
3. Unit Testing Functions

Can we agree on that? Did I miss something?

[jmarbutt]

Yeah I think that is perfect

[Bielik20]

@FrozenPandaz I think this feature has gained quite a popularity. I do realize that there may be other, more important tasks on hand, but we would like to get an update if there are even plans to take it on.

[Bielik20]

I am not sure it is the right place to post it but... I have made a project with working firebase functions in nx workspace. It uses node-app as a base. You can find it here and here. I think it covers most of the milestones, apart from scaffolding obviously.

[nkwood]

If anyone is using firebase in a nx managed monorepo, you should be aware of this behavior: https://github.com/firebase/firebase-js-sdk/issues/1696#issuecomment-501546596

[jongood01]

@Bielik20 Setting "source": "/" in your firebase.json is not ideal at all as 'firebase deploy' will then try to upload all your source code. I've been banging my head against a wall try to get this to work. Ideally 'source' would be something like 'dist/apps/functions'.

[Bielik20]

@jongood01 Yeah I agree but then firebase would complain that there is no npm package there:

Error: No npm package found in functions source directory. Please run 'npm init' inside dist/apps/functions

It is annoying but I think there is little harm in keeping source code up there. It cannot be accessed by anyone from the outside. Deployment is a little longer but also nothing one should worry about.

Or is there a specific reason against it you have in mind?

[jongood01]

@Bielik20 No other reason but on our team we run the deploy as part of a CI process and the mono repo is going to get huge with a lot of code that has nothing to do with the code on Firebase so builds will get slower and slower. I had a hack around to create a minimal package.json in the dist folder but this was also a hack around and not ideal either. Will pick one or the other option for now.
Not really the place for this but I'm really struggling to get Universal rendering working with this as well. I like your idea of having a lib to handle the express server part that gets imported into the cloud function but requiring the main server bundle throws errors.

[spy4x]

Am I right - this issue affects deployment of Nest.js app as well?
Thanks Nx team for all the efforts! Nx is awesome and I really appreciate your work :)

Btw, any movements on this issue?

[johannesnormannjensen]

Hi @Bielik20 and @jongood01

I had also run into the problem of not wanting to upload all my source code. What i found is that you can configure functions to ignore certain directories in firebase.json like this:

...
"functions": {
      "predeploy": [
        "npm run lint functions",
        "npm run build functions --prod"
      ],
      "ignore": [
        "**/apps/web/**",
        "**/apps/web-e2e/**",
        "firebase.json",
        "**/.*",
        "**/node_modules/**"
      ],
      "source": "/"
    },
...

Documentation here: https://firebase.google.com/docs/hosting/full-config

[george43g]

If nrwl could work natively with firebase cloud functions, that would be ideal.

I'm building an app with Firebase. The amount of hours I've spent fixing configuration errors and trying to get local emulation and testing to work is far higher than time spent doing actual programming. Nothing works, ever. I've already tried Lerna but it seems to be more-so designed for open source projects. Not to mention that its package hoisting breaks everything, meaning I need to npm install each package manually anyway, defeating the purpose.

I was hoping that nrwl would finally help in this regard. A setup that would allow me to start actually programming again rather than adjusting config files all day.

[spy4x]

Guys, I've found a workaround, that allows using single package.json for Nx monorepo with Firebase Functions.

TLDR: https://github.com/spy4x/nx-with-firebase-functions

Idea:

Firebase Functions need only 2 files to work:

1. index.js (you could name it differently, like main.js) with your exported Functions
2. package.json with next minimal template:

{
  "main": 'index.js',
  "dependencies": { ...your production dependencies here } 
}

So we can use @nrwl/node or @nrwl/nest or other node-based application, build it using $ nx build <appName> (bundle will appear in dist/apps/<appName>) and generate a new package.json file inside dist/apps/<appName> with dependencies that only needed for our application (for example, avoiding Angular/React application dependencies).

I created a simple JS script that gets a list of appName-specific dependencies from the package.json section that I called firebase-functions-dependencies.

And it works fine!

You can run $ firebase emulators:start and test app locally.

The only flaw of this workaround that I see now - it's necessary to manually update root package.json file's firebase-functions-dependencies section when you add a new dependency for your Firebase Functions application.

Check my demo repo (link above) for details. I split changes into commits, so you can see that there is no magic behind. The main change is in this commit: https://github.com/spy4x/nx-with-firebase-functions/commit/c95997976df1f985c2fce146708cb26ace3f5208

I hope that helps somebody. And maybe someone will find a way to update firebase-functions-dependencies automatically.

P.S. you can use any triggers, dependencies and frameworks (ie Nest.js) with this approach. Just don't forget to update firebase-functions-dependencies.

[JoelCode]

Google Cloud Functions Generator

Generate a Google Cloud Function within a Nx workspace with dev tools:

• Create : nx generate @joelcode/gcp-function:http functionName
• Serve : nx serve functionName
• Test : nx test functionName
• Deploy : nx deploy functionName

I took Nx development strategy for front-end components & applied it to back-end microservices (Google Cloud Functions). With my plugin, I can create & deploy production-ready microservices in 5 minutes. I then combine my microservices to develop a business automation strategy, business analytics, or data streaming pipeline.

GitHub Repository

[FrozenPandaz]

@spy4x That's amazing! Thank you for sharing!

We have started to provide the ability for the community to provide plugins for Nx and I think this would be a great candidate. If @spy4x or somebody else would be interested in maintaining a plugin for Nx + Firebase functions integrations, we would be happy to help and promote it!

[jaytavares]

My take on @spy4x's solution above. I added the depcheck package as a dev dependency and use it to automate the discovery of unused dependencies within the functions project and then remove those from the generated package.json file.

This means that there's no more need for a separate firebase-functions-dependencies section in your package.json file! 🎉

const packageJson = require('../../package.json') // Take root package.json
const path = require('path')
const fs = require('fs').promises
const depcheck = require('depcheck')

const ROOT_PATH = path.resolve(__dirname + '/../..')
const distProjectPath = `${ROOT_PATH}/dist/apps/functions`

console.log('Creating cloud functions package.json file...')

let packageJsonStub = {
  engines: { node: '10' },
  main: 'main.js',
}

depcheck(
  distProjectPath,
  {
    package: {
      dependencies: packageJson.dependencies,
    },
  },
  unused => {
    let dependencies = packageJson.dependencies
    if (unused.dependencies.length > 0)
      console.log('Deleting unused dependencies:')
    unused.dependencies.reduce((acc, dep, i) => {
      console.log(`${i + 1} - Deleting ${dep}`)
      delete acc[dep]
      return acc
    }, dependencies)

    fs.mkdir(path.dirname(distProjectPath), { recursive: true }).then(() => {
      fs.writeFile(
        `${distProjectPath}/package.json`,
        JSON.stringify({
          ...packageJsonStub,
          dependencies,
        })
      )
        .then(() =>
          console.log(`${distProjectPath}/package.json written successfully.`)
        )
        .catch(e => console.error(e))
    })
  }
)

[vdjurdjevic]

@jaytavares Great job! I have one more improvement for this setup. Instead of custom scripts in package.json, we could add builders with @nrwl/workspace:run-commands. That way it feels more integrated with nx, and also enables usage of affected --target=command. For example we could add command for generating package.json, edit serve command to run emulator, and add deploy command.

{
    "pkgJson": {
          "builder": "@nrwl/workspace:run-commands",
      "options": {
        "command": "ts-node scripts/build-firebase-package-json.ts",
        "cwd": "tools"
      }
    },
    "serve": {
    "builder": "@nrwl/workspace:run-commands",
    "options": {
        "command": "firebase emulators:start --only functions --inspect-functions"
    }
    },
    "shell": {
    "builder": "@nrwl/workspace:run-commands",
    "options": {
        "command": "firebase functions:shell --inspect-functions"
    }
    },
    "deploy": {
    "builder": "@nrwl/workspace:run-commands",
    "options": {
        "command": "firebase deploy"
    }
    }
}

[jaytavares]

@vdjurdjevic 👍 That's the same approach I used to be able to use nx. Though, I renamed and wrapped the node builder in order to just tack the tool script run at the end of nx build.

"architect": {
  "build-node": {
    "builder": "@nrwl/node:build",
    "options": { },
    "configurations": { }
  },
  "build": {
    "builder": "@nrwl/workspace:run-commands",
    "options": {
      "commands": [
        {
          "command": "nx run functions:build-node"
        },
        {
          "command": "node tools/scripts/build-cloud-functions-package-file.js"
        }
      ],
      "parallel": false
    },

[nartc]

@vdjurdjevic @jaytavares Thanks for the snippets. I was able to get Firebase Functions up and running with NestJS within a Nx workspace. However, I need to run build first before I can even serve with firebase emulators and there's no live reload at all (aka watch). Were you guys able to get --watch working for firebase emulators?

PS: Not sure if it's worth mentioning but initially I have source: "/" in firebase.json but that didn't work because firebase emulators keeps saying: Cannot find module /path/to/my_workspace_name

[vdjurdjevic]

Watch works. Just run nx build functions(or your project name) --watch, and in the separate terminal run emulators. You should have live reload. At least works with my setup. I used node project instead of nextjs, but that should not be an issue.

[vdjurdjevic]

I opened an issue in firebase repo to support a more flexible project structure for functions. That would enable better integration with Nx, and I would be willing to contribute full-featured plugin for firebase (not just functions, but firestore config, hosting, storage, etc..). @FrozenPandaz can you please check it out and tell me what you think?

[dalepo]

I created an angular workspace and then I wrapped my functions under a node app but I'm struggling to use imported libraries since I'm getting this error when trying to deploy firebase functions:

Error: Cannot find module '@nrwl/workspace'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object. (/workspace/decorate-angular-cli.js:28:20)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)

Anyone faced this? Any help is appreciated.

[ildemartinez]

Hello

I have the same issue.... any news about that?

[dalepo]

I found out that this is related to the postinstall script with the angular-cli decorator (decorate-angular-cli.js)

One quick way to make it work is removing the postinstall script from package.json before deploying to functions, then add it back again so the nx client doesn't break. Is there a way to skip the postinstall script when deploying to firebase functions?

[beeman]

I found out that this is related to the postinstall script with the angular-cli decorator (decorate-angular-cli.js)

One quick way to make it work is removing the postinstall script from package.json before deploying to functions, then add it back again so the nx client doesn't break. Is there a way to skip the postinstall script when deploying to firebase functions?

You're probably looking for the --ignore-scripts command:

The --ignore-scripts argument will cause npm to not execute any scripts defined in the package.json. Source here.

[dalepo]

@beeman thanks, I tried it but it doesn't work on the firebase cli

[jhlabs]

@spy4x I have been using your solution and it works great for one function (in my case SSR rendered NextJS). However, how are you deploying several functions with the firebase command? As far as I could find, you can only define one source directory in the firebase.json. If I set this to e.g. the dist folder, firebase will complain again that there is no valid package.json.

[spy4x]

@jhlabs

1. I export all my cloud functions in one main.ts file, which is then compiled down to dist/<your_path>/main.js.
   This way all my functions are exported and visible to Firebase.

2. To give firebase a valid package.json I use custom script that is running during predeploy stage.

[jhlabs]

@spy4x thanks for the response. I have two separate apps, a NextJS SSR app and a NestJS API backend. Each of them should be deployed as separate cloud functions. I could not figure out how to do it with your (better) version of having separate package.json files, but am using the one from the main project for now. In order for Firebase to find the two functions I need to:

1. For each predeploy of the two apps I need to copy a main.js file into their directory, so dist/apps/nextjs/main.js and dist/apps/api/main.js.
2. Then I need to also create an index.js file for Firebase to import those two separate directories in the dist/apps directory.

It works for now, but there is definitely room for improvement 😆

[spy4x]

@jhlabs I would recommend to have 2 libs that export NextJS and NestJS apps and 1 app called "cloud functions" that imports those 2 libs and sets up 2 cloud functions for each lib.

Something like this:

import { nextJSApp } from '@yourcompany/next-js-app'; // library #1
import { nestJSApp } from '@yourcompany/nest-js-app'; // library #2

export const nextApi = functions.https.onRequest(async (req, res) => {
  return nextJSApp.handle(req, res);
});

export const nestApi = functions.https.onRequest(async (req, res) => {
  return nestJSApp.handle(req, res);
});

This way you can export both APIs, but keep them separate.

[jhlabs]

@spy4x thanks, this will work well for my use case! The downside that is inherent with this is if you have many cloud functions at some point you cannot deploy them individually right? So the nx affected command would then still pull in that cloud function library and redeploy a function that might not have changed.

[dobromyslov]

@jaytavares your great script does not work with NodeJS 8. And NX by default installs types for Node 8. I rewrote the script in TypeScript and upgraded @types/node to 12.19.0 in the main package.json. And this Node upgrade is required for Google Functions since Google has ditched the Node 8 recently and pushed everyone to migrate to NodeJS 10.

[dobromyslov]

Combined solutions from @spy4x @jaytavares @vdjurdjevic and added my own pipes and tape along with original Firebase config. Works rather well.

workspace.json or angular.json (depends on your initial NX workspace creation options):

Expand

{
...
"projects": {
...
 "functions": {
      "root": "apps/functions",
      "sourceRoot": "apps/functions/src",
      "projectType": "application",
      "prefix": "functions",
      "schematics": {},
      "architect": {
        "build-node": {
          "builder": "@nrwl/node:build",
          "options": {
            "outputPath": "dist/apps/functions",
            "main": "apps/functions/src/main.ts",
            "tsConfig": "apps/functions/tsconfig.app.json",
            "assets": ["apps/functions/src/assets"]
          },
          "configurations": {
            "production": {
              "optimization": true,
              "extractLicenses": true,
              "inspect": false,
              "fileReplacements": [
                {
                  "replace": "apps/functions/src/environments/environment.ts",
                  "with": "apps/functions/src/environments/environment.prod.ts"
                }
              ]
            }
          }
        },
        "build": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "commands": [
              {
                "command": "nx run functions:build-node"
              },
              {
                "command": "ts-node tools/scripts/build-firebase-functions-package-json.ts"
              }
            ],
            "parallel": false
          }
        },
        "serve": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "command": "nx run functions:build && firebase emulators:start --only functions --inspect-functions"
          }
        },
        "shell": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "command": "nx run functions:build && firebase functions:shell --inspect-functions"
          }
        },
        "start": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "command": "nx run functions:shell"
          }
        },
        "deploy": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "command": "firebase deploy --only functions"
          }
        },
        "logs": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "command": "firebase funcions:log"
          }
        },
        "lint": {
          "builder": "@nrwl/linter:eslint",
          "options": {
            "lintFilePatterns": ["apps/functions/**/*.ts"]
          }
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "options": {
            "jestConfig": "apps/functions/jest.config.js",
            "passWithNoTests": true
          }
        }
      }
    }
}
...
}

tools/scripts/build-firebase-functions-package-json.ts:

Expand

import * as path from 'path';
import depcheck = require('depcheck');
import * as fs from 'fs';

const packageJson = require('../../package.json') // Take root NX package.json
const ROOT_PATH = path.resolve(__dirname + '/../..')
const distProjectPath = `${ROOT_PATH}/dist/apps/functions`

console.log('Creating cloud functions package.json file...')

let packageJsonStub = {
  engines: { node: '10' },
  main: 'main.js',
}

depcheck(
  distProjectPath,
  {
    package: {
      dependencies: packageJson.dependencies,
    },
  },
  unused => {
    let dependencies = packageJson.dependencies
    if (unused.dependencies.length > 0)
      console.log('Deleting unused dependencies:')
    unused.dependencies.reduce((acc, dep, i) => {
      console.log(`${i + 1} - Deleting ${dep}`)
      delete acc[dep]
      return acc
    }, dependencies)

    fs.promises.mkdir(path.dirname(distProjectPath), { recursive: true }).then(() => {
      fs.promises.writeFile(
        `${distProjectPath}/package.json`,
        JSON.stringify({
          ...packageJsonStub,
          dependencies,
        })
      )
        .then(() =>
          console.log(`${distProjectPath}/package.json written successfully.`)
        )
        .catch(e => console.error(e))
    })
  }
)

### firebase.json

{
  "hosting": [
    {
      "target": "console",
      "public": "dist/apps/console",
      "ignore": [
        "**/.*"
      ],
      "headers": [
        {
          "source": "*.[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f].+(css|js)",
          "headers": [
            {
              "key": "Cache-Control",
              "value": "public,max-age=31536000,immutable"
            }
          ]
        }
      ],
      "rewrites": [
        {
          "source": "**",
          "destination": "/index.html"
        }
      ]
    }
  ],
  "functions": {
    "source": "dist/apps/functions"
  },
  "firestore": {
    "rules": "firestore.rules",
    "indexes": "firestore.indexes.json"
  },
  "storage": {
    "rules": "storage.rules"
  },
  "emulators": {
    "functions": {
      "port": 5001
    },
    "firestore": {
      "port": 8080
    },
    "hosting": {
      "port": 5000
    },
    "pubsub": {
      "port": 8085
    },
    "ui": {
      "enabled": true
    }
  }
}

package.json dependencies:

Expand

"dependencies": {
  "firebase-admin": "^9.2.0",
  "firebase-functions": "^3.11.0",
},
"devDependencies": {
...
  "@types/node": "~12.19.0",
  "depcheck": "^1.2.0",
  "firebase-functions-test": "^0.2.2",
  "firebase-tools": "^8.0.0",
  "ts-node": "~7.0.0",
...
}

apps/functions/src/main.ts :

Expand

import * as functions from 'firebase-functions';

export const helloWorld = functions.https.onRequest((request, response) => {
  response.send('Hello from Firebase!');
});

[pascalbe-dev]

Has someone already started building an nx plugin?

[KingDarBoja]

I want to add to @dobromyslov answer that his approach works with local emulator as well but needs an extra step at the build commands, as the emulator requires .runtimeconfig.json to load environment variables for cloud functions (AFAIK).

The .runtimeconfig.json file must be copied into the dist folder to make it work, like below:
copyfiles -E .runtimeconfig.json dist/apps/your-app-api-name

So ends up being:

"commands": [
  {
     "command": "nx run functions:build-node && copyfiles -E .runtimeconfig.json dist/apps/functions"
  },
  {
     "command": "ts-node tools/scripts/build-firebase-functions-package-json.ts"
  }
],

[dobromyslov]

@KingDarBoja thank you!

[markwhitfeld]

Just to add an additional tweak to @dobromyslov's awesome comment:

The angular.json "build" configuration needs the configurations section in order to support the --prod flag and to pass it on to the "build-node" configuration.

Note: I was also having an issue where it was saying 'ts-node' is not recognized as an internal or external command (even though it was installed as a dependency), so I also added the npx call to the command that uses this.

        "build": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "commands": [
              {
                "command": "nx run functions:build-node"
              },
              {
                "command": "npx ts-node tools/scripts/build-firebase-functions-package-json.ts"
              }
            ],
            "parallel": false
          },
          "configurations": {
            "production": {
              "prod": true
            }
          }
        }

Specifying the --prod flag will allow for a more optimised build of the function and to support the production environment configuration that was supplied in the example.

[EthanSK]

There is this community plugin, not sure how well it works: https://github.com/JoelCode/gcp-function