Firebase-tools: Firestore emulator: PERMISSION_DENIED

[sgammon]

[REQUIRED] Environment info

firebase-tools: 6.10.0

Platform: macOS

[REQUIRED] Test case

This will take a moment, brb

[REQUIRED] Steps to reproduce

1) Setup a Java project with the Firestore gcloud SDK, or the Firebase Admin SDK. Either way, make sure you're on the newest Firestore release. In Gradle, for us, that's:

  compile group: 'com.google.cloud', name: 'google-cloud-firestore', version: '1.6.0'

2) Install the latest gcloud SDK locally. Gcloud reports the following for gcloud --version:

Google Cloud SDK 248.0.0
alpha 2019.05.17
app-engine-go 
app-engine-java 1.9.74
app-engine-python 1.9.85
beta 2019.05.17
bq 2.0.43
cloud-build-local 
cloud-datastore-emulator 2.1.0
cloud-firestore-emulator 1.4.6
cloud_sql_proxy 
core 2019.05.24
docker-credential-gcr 
emulator-reverse-proxy 
gsutil 4.38
pubsub-emulator 2019.04.26

3) Write up a rules file for local emulator access to Firestore. Try your hardest to make it fully open - i.e. allow all traffic for basic testing.

rules_version = '2';

service cloud.firestore {
  match /databases/{database} {
    allow read, write: if true;

    match /{document=**} {
      allow read, write: if true;
    }
  }
}

4) Try to runlistDocuments() on a query, receive the following exception:

com.google.cloud.firestore.FirestoreException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Metadata operations require admin authentication.
    at com.google.cloud.firestore.FirestoreException.apiException(FirestoreException.java:79)
    at com.google.cloud.firestore.CollectionReference.listDocuments(CollectionReference.java:140)
       [... lots of user code...]

5) Think to yourself, okay, I'll play this dumb credentials game. So you load up your JSON service account key file, and inject the credentials into the Firestore adapter, like you do for production access to Firestore:

    public static @Provides FirestoreOptions getFirestoreOptions() {
      // [... lots of prep ...]
      if (emulator) {
        final CredentialsProvider adminCreds = FixedCredentialsProvider.create(googleCreds);
        opts.setCredentials(googleCreds);
        opts.setCredentialsProvider(credentialsProvider);
        return opts.build();
      }
      return opts.build();
    }

6) Run it again. Observe the following exception, because the emulator isn't running via TLS, we see that the request is rejected again, this time because you cannot use credentials over a plaintext connection:

Caused by: com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Credentials require channel with PRIVACY_AND_INTEGRITY security level. Observed security level: NONE
    at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:73)

[REQUIRED] Expected behavior

It should be possible to run queries on the local emulator for Firestore.

[REQUIRED] Actual behavior

Because of this catch-22 error setup, I don't believe there is a way to run queries on the local emulator for Firestore, at least not queries that involve "metadata."

[sgammon]

It is also worth saying that running the emulator with no rules produces the same exception for listDocuments:

com.google.cloud.firestore.FirestoreException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Metadata operations require admin authentication.

    at com.google.cloud.firestore.FirestoreException.apiException(FirestoreException.java:79)
    at com.google.cloud.firestore.CollectionReference.listDocuments(CollectionReference.java:140)

[abeisgoat]

Hey @sgammon,

Thanks for this report. @ryanpbrewster can you shed some light on if this is expected behavior?

[ryanpbrewster]

It is indeed expected behavior. You will need to be authenticated as an admin. The upside is that it's actually much simpler than authenticating against prod firestore: all you need to do is pass in an "Authorization: Bearer owner" header. The bearer "owner" is accepted by the emulator for all projects.

If you're familiar with @google-cloud/firestore you should be able to use the Admin SDK like so:

const grpc = require("@grpc/grpc-js");
const Firestore = require("@google-cloud/firestore");
const firestore = new Firestore({
  projectId: 'firestore-emulator-sample',
  servicePath: 'localhost',
  port: 8080,
  sslCreds: grpc.credentials.createInsecure(),
  customHeaders: {
    "Authorization": "Bearer owner"
  }
});

[ryanpbrewster]

The REST API looks like

curl "localhost:8080/v1/projects/example-project-name/databases/(default)/documents:listCollectionIds" \
  -X POST \
  -H 'Authorization: Bearer owner'

Out of curiosity, which SDK are you using here? Based on the stack trace it looks like Java?

[sgammon]

I'm using Java (via the Maven dependencies for gcloud, not firebase-admin), but this does look easy and I'll give it a shot. Injecting headers is rather hard in the Java client but I believe there is a facility to do it alongside the code I mentioned for injecting credentials.

[sgammon]

It also wouldn't be too hard to implement this as a CredentialProvider in gax or what not. That would be super helpful, if you could point me in the right direction I'd be happy to file a PR or an issue in the right repo (because that's way closer to general Google APIs Java than it is Firebase, if I'm not mistaken).

Thank you btw for the quick reply Firebase team :)

_edit:_ @ryanpbrewster feel free to close as wontfix once you reply.

[ryanpbrewster]

I think the repo you want is https://github.com/googleapis/google-cloud-java. If you file an issue there, our SDK team can take a closer look.

I implemented this in the Node Admin SDK in https://github.com/googleapis/nodejs-firestore/pull/654, which effectively bypasses the gax credential loop.

[sgammon]

@ryanpbrewster sorry to check back in, but i've tried to inject the specified header and i'm having trouble getting it working with gax.

essentially, i've tried setting the HeaderProvider, and i've tried adding an interceptor manually to the channel in setChannelConfigurator (both on InstantiatingGrpcChannelProvider in gax).

Any ideas about how I might actually get that header in? short of the approach i mentioned earlier, which involves extending gax with a CredentialProvider?

[sgammon]

@ryanpbrewster if you could link me to some java tests somewhere that inject that token, that would be sufficient too, probably. thank you again for your help

[sgammon]

@ryanpbrewster sorry to blow up this thread, lol. but i found some tests, and they use NoCredentials like i was using before :(

https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-clients/google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/FirestoreClientTest.java#L99

[ryanpbrewster]

Got it working, a bit hackily: https://gist.github.com/ryanpbrewster/aef2a5c411a074819c8d7b67be80621c

LMK if that's enough to get you going

[sgammon]

@ryanpbrewster that did the trick! and i like your approach here, because it uses the credentials rather than considering this a custom header (which it isn't, technically). thank you again, i'm all set.

[sgammon]

@ryanpbrewster it might be a good idea to include that sample somewhere, or in the docs, by the way. just saying 😁

[jsilvagluky]

I'm trying to use it on nodejs without luck, I keep getting:

Error: 7 PERMISSION_DENIED: Metadata operations require admin authentication.

Mi code is in typescript for nodejs.

It was working before gcloud components update with this code:

declare global {
  namespace FirebaseFirestore {
    interface Firestore {
      _decorateRequest(request: any): any;
    }
  }
}

class TestFirestore extends Firestore {
  _decorateRequest(request: any) {
    const result = super._decorateRequest(request);
    result.gax.otherArgs.headers['Authorization'] = 'Bearer owner';
    return result;
  }
}

const emulatorHost = process.env.FIRESTORE_EMULATOR_HOST || 'localhost';
const emulatorPort = process.env.FIRESTORE_EMULATOR_PORT || '8080';

const firestore = new TestFirestore({
      projectId: fireProjectName,
      sslCreds: grpc.credentials.createInsecure(),
      servicePath: emulatorHost,
      port: emulatorPort,
      // As with 'loadFirestoreRules', cap how much backoff gRPC will perform.
      'grpc.initial_reconnect_backoff_ms': 100,
      'grpc.max_reconnect_backoff_ms': 100,
    });

But it started throwing that error after last update.

Now, after reading this issue, I'm trying with:

    const emulatorHost = process.env.FIRESTORE_EMULATOR_HOST || 'localhost';
    const emulatorPort = process.env.FIRESTORE_EMULATOR_PORT || '8080';

    const firestore = new Firestore({
      projectId: fireProjectName,
      sslCreds: grpc.credentials.createInsecure(),
      servicePath: emulatorHost,
      port: emulatorPort,
      customHeaders: {
        // tslint:disable-next-line
        "Authorization": "Bearer owner",
      },
      // As with 'loadFirestoreRules', cap how much backoff gRPC will perform.
      'grpc.initial_reconnect_backoff_ms': 100,
      'grpc.max_reconnect_backoff_ms': 100,
    });

But keep getting the same error.

I'm running emulator this way:

gcloud beta emulators firestore start --host-port=127.0.0.1:8080

And this is gcloud components version:

~ gcloud components list   

Your current Cloud SDK version is: 250.0.0
The latest available version is: 250.0.0

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                  Components                                                 │
├───────────────┬──────────────────────────────────────────────────────┬──────────────────────────┬───────────┤
│     Status    │                         Name                         │            ID            │    Size   │
├───────────────┼──────────────────────────────────────────────────────┼──────────────────────────┼───────────┤
│ Not Installed │ App Engine Go Extensions                             │ app-engine-go            │  56.6 MiB │
│ Not Installed │ Cloud Bigtable Command Line Tool                     │ cbt                      │   6.4 MiB │
│ Not Installed │ Cloud Bigtable Emulator                              │ bigtable                 │   5.6 MiB │
│ Not Installed │ Cloud Datalab Command Line Tool                      │ datalab                  │   < 1 MiB │
│ Not Installed │ Cloud Datastore Emulator (Legacy)                    │ gcd-emulator             │  38.1 MiB │
│ Not Installed │ Cloud SQL Proxy                                      │ cloud_sql_proxy          │   3.8 MiB │
│ Not Installed │ Emulator Reverse Proxy                               │ emulator-reverse-proxy   │  14.5 MiB │
│ Not Installed │ Google Cloud Build Local Builder                     │ cloud-build-local        │   6.0 MiB │
│ Not Installed │ Google Container Registry's Docker credential helper │ docker-credential-gcr    │   1.8 MiB │
│ Not Installed │ gcloud Alpha Commands                                │ alpha                    │   < 1 MiB │
│ Not Installed │ gcloud app Java Extensions                           │ app-engine-java          │ 103.3 MiB │
│ Not Installed │ gcloud app Python Extensions (Extra Libraries)       │ app-engine-python-extras │  28.5 MiB │
│ Installed     │ BigQuery Command Line Tool                           │ bq                       │   < 1 MiB │
│ Installed     │ Cloud Datastore Emulator                             │ cloud-datastore-emulator │  18.4 MiB │
│ Installed     │ Cloud Firestore Emulator                             │ cloud-firestore-emulator │  40.8 MiB │
│ Installed     │ Cloud Pub/Sub Emulator                               │ pubsub-emulator          │  34.8 MiB │
│ Installed     │ Cloud SDK Core Libraries                             │ core                     │  10.8 MiB │
│ Installed     │ Cloud Storage Command Line Tool                      │ gsutil                   │   3.8 MiB │
│ Installed     │ gcloud Beta Commands                                 │ beta                     │   < 1 MiB │
│ Installed     │ gcloud app PHP Extensions                            │ app-engine-php           │           │
│ Installed     │ gcloud app Python Extensions                         │ app-engine-python        │   6.0 MiB │
│ Installed     │ kubectl                                              │ kubectl                  │   < 1 MiB │
└───────────────┴──────────────────────────────────────────────────────┴──────────────────────────┴───────────┘
To install or remove components at your current SDK version [250.0.0], run:
  $ gcloud components install COMPONENT_ID
  $ gcloud components remove COMPONENT_ID

To update your SDK installation to the latest version [250.0.0], run:
  $ gcloud components update

The OS is Ubuntu 18.04 and nodejs version is 10.15.1 installed via nvm. Also glcoud SDK was installed from tarball (instead of deb packages repository).

[ryanpbrewster]

What version of @google-cloud/firestore are you using? Support for the customHeaders field was added in the most recently release, IIRC.

[ryanpbrewster]

I'm shifting your comment to a new issue as it's unrelated to this closed one.

[bcspragu]

I'm running into the same issue with the Go Firestore client, where write requests to the local emulator work setting the FIRESTORE_EMULATOR_HOST env variable, but List* calls result in Metadata operations require admin authentication.

I tried a few different schemes to set the Authorization: Bearer owner header, but ran into the same problems as above (can't use creds without TLS, requests are gRPC so using HTTP headers doesn't work, etc). Not sure what the equivalent of ryanpbrewster's hack from above looks like for Go:

.setCredentialsProvider(FixedCredentialsProvider.create(FakeCreds))

But it should probably go here: https://github.com/googleapis/google-cloud-go/blob/master/firestore/client.go#L66-L72

[ryanpbrewster]

Good point, I'll file an issue and follow up with the SDK team.

EDIT: https://github.com/googleapis/google-cloud-go/issues/1537

[emmekappa]

Having the same issue with the dotnet sdk: https://github.com/googleapis/google-cloud-dotnet/issues/3397

[antoniooi]

For my case, I have the following code:

const admin = require("firebase-admin");
const serviceAccount = require("./service-account.json");

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    databaseURL: "http://localhost:8080"
});

const db = admin.firestore();
db.settings({
    host: "localhost:8080",
    ssl: false
});

Error: 7 PERMISSION_DENIED when the local Firestore rules:

service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if false;
    }
  }
}

and

service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if request.auth.uid != null;
    }
  }
}

It only works when I changed the local Firestore rules to:

service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write;
    }
  }
}

I thought the Admin SDK shouldn't bound to Firestore security rules? However, if I remove the local settings and go directly to the production server with allow read, write: if false; , it works too. So does it mean that the local Firestore Emulator is still rejecting Admin SDK from accessing Firestore?

[samtstern]

@antoniooi looks like there are a few things that may be going wrong:

1. databaseURL is a setting for Realtime Database, not Cloud Firestore. Pointing that at your Firestore emulator could cause unknown issues.

2. You are using this method to connect to the emulator:

db.settings({
    host: "localhost:8080",
    ssl: false
});

That's actually how you connect the web JavaScript SDK to the emulator. The best way to connect the Admin SDK to the emulator is by using an environment variable:

export FIRESTORE_EMULATOR_HOST=localhost:8080

If that is set in your environment, when you do admin.initializeApp() we will automatically pick it up and connect to the emulator.

If you're still having issues after trying these fixes, please open a new issue.

[antoniooi]

@samtstern : Thanks for enlightening me. I didn't know that databaseURL is only meant for Real-time Database. No wonder no matter what URL I put in, it still works, LOL. As for the db.settings, I tried before, without it, I can't insert test data in my Firestore Emulator and make it appears in emulator UI -- it will go straight to the Firestore production server despite Firestore emulator is properly running.

Honestly speaking, I'm just a beginner. Hope you don't mind if I seek help from you further, where should I put this line of code:

export FIRESTORE_EMULATOR_HOST=localhost:8080

Is the above code just a pseudo or a fully functional code? Sorry for asking this because many experts here like to give only a _"concept"_, and made me foolishly hit the wall all the time. @_@

As what I know, when I start the emulator, the process.env shows that this has already been set for me, why should I set it again in my code? Thanks in advance!

[samtstern]

@antoniooi that export command is to set an environment variable. So you'd do it in your terminal or .bashrc before running the command.

Although if you see process.env.FIRESTORE_EMULATOR_HOST already set then you shouldn't have to worry about it!

Let's step back for a second:

• Can you show me the full code you're trying to run and where it fails? Ideally the smallest full example that fails.
• Show me how you're starting the emulators and how you are running the code above. Is it running inside the Functions Emulator? Somewhere else?

[antoniooi]

@samtstern :

Can you show me the full code you're trying to run and where it fails? Ideally the smallest full example that fails.

You mean if I go without the db.settings that points to the localhost:8080? No, it won't fail, the data just go straight to the Firestore production server. If you're referring to the PERMISSION DENIED error, then the line that causes the error is the code that writes to the Firestore database: docRef.set(fields);.

Show me how you're starting the emulators and how you are running the code above. Is it running inside the Functions Emulator? Somewhere else?

1. Open my project root folder with my VS Code.
2. Open the terminal window in my VS Code and run firebase emulators:start -- means I run ALL emulators: hosting, functions, firestore, UI.
3. While the emulators are running, I can't issue another command to run my import-test-data.js file. So I open another terminal window in the same instance of VS Code and run node import-test-data.js under my my-project\functions directory (because my import-test-data.js is stored inside my functions folder. The import-test-data.js is the one that contains the code that I showed to you earlier. It is just a JS file that read the JSON test data file and import it to the Firestore Emulator.
4. After successfully running the JS file, I switched to emulator UI, go to Firestore tab, and all test data is there. Without that db.settings, the test data won't be there, it will go straight to the Firestore production server.

Let me know if you still need any further information.

[antoniooi]

@samtstern : Hey, I just remove my db.settings with the following code and it works now! Thanks! :D

const admin = require('firebase-admin');
var db = undefined;

console.log(`process.env = ${JSON.stringify(process.env.FIRESTORE_EMULATOR_HOST)}`);

if (!process.env.FIRESTORE_EMULATOR_HOST) {
    admin.initializeApp();
    db = admin.firestore();
} else {
    const serviceAccount = require("./service-account.json");
    admin.initializeApp({
        credential: admin.credential.cert(serviceAccount),
        // databaseURL: "http://localhost:8080"
    });
    db = admin.firestore();
    /* db.settings({
        host: "localhost:8080",
        ssl: false
    }); */
    setupTestData(db);
};

As for the PERMISSION DENIED error, the Firestore security rules still need to allow all, else the error will appear again. Thanks!

[samtstern]

@antoniooi glad it's working although I am very concerned about the permission denied error. The Admin SDK has no interaction with rules, period. The face that you're seeing that error suggests you somehow have the non-Admin SDK mixed up in there.

[antoniooi]

@samtstern : I think it is not the Admin SDK problem. It could be the Firestore emulator that is still bothering too much about the security rules and forgotten about Admin SDK privilege. If I go directly to the production server without running the emulators, the Admin SDK works as it should be -- even with the rule allow read, write: if false;. I believe it is the emulator that is blocking it based on the rules.

[Kesin11]

I've probably faced the same problem, as the Admin SDK seems to be affected by the rules.

I tried emulators-codelab (https://github.com/firebase/emulators-codelab/tree/master/codelab-final-state) but functions emulator could not work properly.

Cart could not be recalculated. Error: 7 PERMISSION_DENIED: error log appeared from this code.

https://github.com/firebase/emulators-codelab/blob/b6c759ab8c6ba3f9f955cb248ee906021e09cf5f/codelab-final-state/functions/index.js#L28

Then I commented out all existing rules in firestore.rules and rewrote them to allow read, write;, emulator worked correctly.

Using firebase-tools version is here.

$ npx firebase --version
8.4.1

[samtstern]

We found a bug in the Admin SDK where using the db.settings({host, ssl})
can cause the Admin SDK to lose auth and therefore give you permission
denied.

The temporary solution is not to use that setting and instead rely on
FIRESTORE_EMULATOR_HOST to connect to the emulator.

On Sat, May 30, 2020 at 10:44 PM Kenta Kase [e-mail]
wrote:

I've probably faced the same problem, as the Admin SDK seems to be
affected by the rules.

I tried emulators-codelab (
https://github.com/firebase/emulators-codelab/tree/master/codelab-final-state)
but functions emulator could not work properly.

Cart could not be recalculated. Error: 7 PERMISSION_DENIED: error log
appeared from this code.

https://github.com/firebase/emulators-codelab/blob/b6c759ab8c6ba3f9f955cb248ee906021e09cf5f/codelab-final-state/functions/index.js#L28

Then I commented out all existing rules in firestore.rules and rewrote
them to allow read, write;, emulator worked correctly.

Using firebase-tools version is here.

$ npx firebase --version
8.4.1

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/firebase/firebase-tools/issues/1363#issuecomment-636412304,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACATB2TAKTV42RTMILPGI63RUHAATANCNFSM4HSJ4JXA
.

[antoniooi]

@samtstern : With set FIRESTORE_EMULATOR_HOST=localhost:8080, running node import-test-data.js will still have the data go straight to the production server UNLESS the same program is running within the Firebase Function. Below is the example of my index.js Firebase Function:

index.js

"use strict";
const functions = require('firebase-functions');
// When the line below is initialized, data goes to the Firestore Emulator:
const db = require("./import-test-data.js"); 

import-test-data.js

const admin = require('firebase-admin');
var db = undefined;

console.log(`process.env = ${JSON.stringify(process.env.FIRESTORE_EMULATOR_HOST)}`);

if (!process.env.FIRESTORE_EMULATOR_HOST) {
    admin.initializeApp();
    db = admin.firestore();
} else {
    const serviceAccount = require("./service-account.json");
    admin.initializeApp({
        credential: admin.credential.cert(serviceAccount),
        // databaseURL: "http://localhost:8080"
    });
    db = admin.firestore();
    /* db.settings({
        host: "localhost:8080",
        ssl: false
    }); */
    setupTestData(db);
};

If you run the JS file with just the Admin SDK alone using node your-db-import.js without going through the Firebase Function, even with the set FIRESTORE_EMULATOR_HOST=localhost:8080, it won't work -- you still need the db.settings for it to get connected to the Firestore Emulator. But yes, perhaps like what you've discovered, PERMISSION_DENIED error occurs unless security rules allow all.

CONCLUSION: What if only the Functions Emulator setup the connection with Firestore Emulator via FIRESTORE_EMULATOR_HOST environment variable and NOT the Firestore Emulator alone? In order words, could it be that: Your Firestore Emulator is actually depending on your Function Emulator to get the Admin SDK connects to it?

[samtstern]

@antoniooi there have been a few bugs in the Admin SDK which are now all fixed (remove your node_modules and package-lock.json and run npm install again).

If you still have this error, please file a new issue with the simplest possible steps to reproduce and I'll dig into it.

[antoniooi]

@samtstern : Wow, you guys are so efficient! Thanks a lot!

[antoniooi]

@samtstern : Work like a charm with the following steps:

1. Delete functions/node_modules and functions/package-lock.json.
2. Run npm install at the functions directory.
3. Run firebase emulators:start.
4. Run set FIRESTORE_EMULATOR_HOST=localhost:8080.
5. Run node import-test-data.js --> Test data go into the Firestore Emulator! 👍
6. Change the firestore.rules to allow read, write: if false;.
7. Run node import-test-data.js again --> No more PERMISSION_DENIED error. Test data updated at the Firestore Emulator accordingly! 👍

Everything works perfectly! All problems solved! Well done! 👍

[antoniooi]

To prevent from forgotten to set FIRESTORE_EMULATOR_HOST=localhost:8080 at the terminal, add the following in the import-test-data.js, it works:

if (!process.env.FIRESTORE_EMULATOR_HOST) {
    process.env.FIRESTORE_EMULATOR_HOST = 'localhost:8080';
};

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
});

const db = admin.firestore();