Nodejs-pubsub: get messages count waiting in subscription

[stephenplusplus]

_From @pierre-b on May 4, 2015 15:34_

Hi,

I need to know how many messages are waiting in the subscription, to pop new workers or monitor the health of my system. This is something AWS SQS does too.

Is it in the roadmap actually ?

Thank you

_Copied from original issue: GoogleCloudPlatform/google-cloud-node#527_

[stephenplusplus]

_From @pierre-b on May 5, 2015 13:1_

FYI: as https://code.google.com/p/cloud-pubsub/issues/detail?id=13 told me, there is a "num_undelivered_messages" endpoint to get that metric.

[stephenplusplus]

_From @ryanseys on May 5, 2015 13:15_

Considering it's "not accurate" I wouldn't use it yet and don't think we should support it in this library until we know it is accurate.

[stephenplusplus]

_From @pierre-b on May 5, 2015 14:21_

This feature could never be accurate because messages are always flowing into the subscription (depending on the publish rate), but it's crucial to know if you need to add more workers to process messages or monitor the health of your workers.

[stephenplusplus]

From https://cloud.google.com/monitoring/api/metrics, they list two interesting ones:

_num_outstanding_messages_: Current number of messages delivered but not yet acknowledged.
_num_undelivered_messages_: Current number of messages pending to be delivered.

Maybe we could open up a generic subscription.getMetrics('num_outstanding_msesages') and anything else from the list: topic.getMetrics('message_sizes'), as these may all be useful.

[stephenplusplus]

Assigned this to @callmehiphop since you had some ideas for how to implement this generically.

[stephenplusplus]

_From @leibale on December 10, 2015 17:54_

what about this one?

[stephenplusplus]

We have a few more things to implement with a higher priority before getting to this one, mostly support for the Prediction and Translate APIs, then promises.

[stephenplusplus]

_From @leibale on December 12, 2015 16:3_

@stephenplusplus if i will create a pull request, you will merge?

[stephenplusplus]

That would be great!

[stephenplusplus]

_From @sahas- on March 31, 2017 0:16_

Great to see you guys working on this already. If its worth, right now I use stackdriver's subscription metrics capabilities to get this information, apparently, stackdriver lags behind and the count is not accurate. Just wanted to add +1 to this request and happy to be a beta tester, if need be. Let me know.

[stephenplusplus]

_From @mpcm on April 6, 2017 2:14_

Adding link to the current related google doc page, for others like me that ended up here looking to learn about msg depth in a sub: https://cloud.google.com/pubsub/docs/ordering#order_of_processed_messages_matters

[stephenplusplus]

@callmehiphop given the large Pub/Sub rewrite that happened since this issue was opened, is this request still relevant / do you have an idea for a reliable implementation?

[stephenplusplus]

_From @callmehiphop on November 27, 2017 17:28_

@stephenplusplus AFAIK the re-write hasn't changed anything in regard to collecting these metrics.

[theacodes]

This functionality unfortunately is not something we can provide without the Pub/Sub API itself adding a feature specifically for it. We've communicated with the product team to express the desire for this feature, but at this point, there's nothing more we can do in this client library.

I'm closing for now, but feel free to continue to comment.

[shinzui]

Why is this closed? How does stackdriver get access to that information if the API does not exist?