Nodejs-pubsub: Monitor and Flush a Batch Publisher Queue

[davidtom]

Are there any methods or strategies that can be used to monitor the size of of a batch publisher's queue and flush it on demand?

I am building a service using the batch publisher and would like to set up some metrics and alerts on the queue size. In addition, I'd like to be able to flush the queue on demand in response to certain lifecycle events (ie SIGTERM).

I’ve read through the docs and examples and didn’t see anything that mentioned these features. I did notice that the Publisher object has a publish__ method to forcefully publish the queue, but that is explicitly called out as a private method not to be used externally, so I’m hesitant to make use of it.

Do these features exist in any form? If not, are there any recommended strategies to implement them? Any plans to add in official support?

Any help or references would be greatly appreciated, thanks!

[walshie4]

You should be able to just peek inside the publisher via publisher.inventory_

https://github.com/googleapis/nodejs-pubsub/blob/222b6e3ad5279e49b92af370ad6fa3933ccb5c15/src/publisher.js#L81

[callmehiphop]

Currently that functionality does not exist, however adding support seems pretty straight forward. We would probably just make the private method publish_ into a public method and name it flush and add some getters to retrieve the bytes/length from the inventory_ object.

@davidtom I'm not sure where this stands on our priority list, but we're always more than happy to accept a pull request!

[davidtom]

@walshie4 That does look like it should work! Not sure how I missed that when reviewing the module, but thank you for pointing it out to me.

@callmehiphop It does indeed. For now I may be able to get what I need by using _inventory, but if/when I need to add a flush method I'll absolutely open a PR. I would love to contribute to this project.

I'll close this issue given I've got no further questions. Thank you both for your help!