Nodejs-pubsub: PubSub Emulator Error With pubsub.v1.SubscriberClient()

[watsab]

Hi !

I'm trying to pull the messages from a PubSub topic using the PubSub Emulator.
At first I used the asynchronous method which worked fine but was not what I really wanted.
So I switched to synchronous pull of messages as explained in the documentation here: https://cloud.google.com/pubsub/docs/pull?hl=FR#synchronous-pull

However I can't get it to work. It keeps throwing the followin error:

node:261) UnhandledPromiseRejectionWarning: Error: Unexpected error while acquiring application default credentials: Could not load the default credentials. Browse to https://developers.google.com/accounts/docs/application-default-credentials for more information.
at GoogleAuth.<anonymous> (/var/code/node_modules/google-gax/node_modules/google-auth-library/build/src/auth/googleauth.js:248:31)
at step (/var/code/node_modules/google-gax/node_modules/google-auth-library/build/src/auth/googleauth.js:47:23)
at Object.next (/var/code/node_modules/google-gax/node_modules/google-auth-library/build/src/auth/googleauth.js:28:53)
at fulfilled (/var/code/node_modules/google-gax/node_modules/google-auth-library/build/src/auth/googleauth.js:19:58)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)

Is it not possible to use the pubsub.v1.SubscriberClient component with PubSub Emulator without specifying credentials?
Thanks!

Below are all the technical details

Environment details

• OS: Linux/Docker
• Node.js version: 8.12
• npm version: 6.4.1
• @google-cloud/pubsub version: 0.20.1

Steps to reproduce

1. Launch PubSub Emulator
2. Call pubsub.v1.SubscriberClient();

[watsab]

Hi !
Update: I have the same issue with the v1.PublisherClient. Are the v1 modules not compatible with Pubsub Emulator ? If not, it would be great to work on this to allow development on local machines. Thanks !

[callmehiphop]

Hi @watsab I believe all the v1 modules are compatible with the PubSub emulator. However IIRC both the Subscriber and Publisher clients will need to be configured to use the emulator like so

const sub = pubsub.v1.SubscriberClient({
  servicePath: 'path.to.your.emulator',
  port: 8080, // port your emulator is running on (default is 443)
});

[watsab]

Hi @callmehiphop !

Thanks for your answer !
I noticed these two options but I still can't make it work with these, as it keeps asking for my credentials. FYI, I am using Docker Compose to run the app in one container and the emulator in another; so I set "servicePath" to my PubSub service name, and "port" to 8085 as it is the one used but maybe I misconfigured these options... I'll keep digging and see if I can find a way to solve this.
What would be great, however, is for the v1 components to use the env variable "PUBSUB_EMULATOR_HOST", so we don't need to specify these options, just as the non-v1 components do ;)

Thanks again !

[watsab]

Hi again @callmehiphop !

I've done some reverse engineering and I noticed that the exception is thrown by the google-auth-library. More specifically in buil/src/auth/googleauth.js in _getApplicationDefaultAsync_ method. So it may well be working with the "servicePath" and "port" options, but the problem is that it keeps trying to get credentials despite using the emulator and I'd prefer not to need credentials when I'm doing local development.
Is there a way I can prevent this?

Thanks !

[watsab]

Update:
I created a service account just to try if it would work with credentials. However I have errors raising in both my PubSub container and my app Container:

pubsub_1 | [pubsub] Dec 04, 2018 3:31:24 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
pubsub_1 | [pubsub] INFO: Adding handler(s) to newly registered Channel.
pubsub_1 | [pubsub] Dec 04, 2018 3:31:24 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
pubsub_1 | [pubsub] INFO: Detected non-HTTP/2 connection.
pubsub_1 | [pubsub] Dec 04, 2018 3:31:24 PM io.gapi.emulators.netty.NotFoundHandler handleRequest
pubsub_1 | [pubsub] INFO: Unknown request URI: /bad-request
app | error: E1204 15:31:24.987311276 134 ssl_transport_security.cc:1229] Handshake failed with fatal error SSL_ERROR_SSL: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number.

From what I get, it seems to me that the emulator does not return a secured connection...

[watsab]

Hi @callmehiphop !

do you have an idea of how I could solve the credentials issue ? I'm wasting a lot of time not being able to have a proper local dev environment :disappointed:
Thanks !

[callmehiphop]

Sorry about that @watsab, I forgot that you also need to provide some credentials as well. I think the following should work

const grpc = require('grpc');

const subscriber = new pubsub.v1.SubscriberClient({
  servicePath: 'path.to.your.emulator',
  port: 8080, // port your emulator is running on (default is 443)
  sslCreds: grpc.credentials.createInsecure()
});

[watsab]

Thanks a million @callmehiphop ! It works fine now ! Thanks so much !!!!!! :smiley:

[andykellr]

The above solution didn't work for me.

It resulted in TypeError: this.credentials._getCallCredentials is not a function

Here is what I used, borrowed from the nodejs-firestore repo:

import { GrpcClient } from 'google-gax';

const grpc = new GrpcClient({} as any).grpc;
const SSL_CREDENTIALS = (grpc.credentials as any).createInsecure();

const subscriber = new pubsub.v1.SubscriberClient({
  servicePath: 'path.to.your.emulator',
  port: 8080, // port your emulator is running on (default is 443)
  sslCreds: SSL_CREDENTIALS,
});

[lucasalexandre-cc]

Sorry about that @watsab, I forgot that you also need to provide some credentials as well. I _think_ the following should work

const grpc = require('grpc');

const subscriber = new pubsub.v1.SubscriberClient({
  servicePath: 'path.to.your.emulator',
  port: 8080, // port your emulator is running on (default is 443)
  sslCreds: grpc.credentials.createInsecure()
});

If you got TypeError: Channel credentials must be a ChannelCredentials object, try this code:

const {v1} = require('@google-cloud/pubsub');
const grpc = require('@grpc/grpc-js');

const subClient = new v1.SubscriberClient({
  servicePath: 'localhost', //path of your emulator
  port: '8085', //port of yout emulator
  sslCreds: grpc.credentials.createInsecure()
});