Bentoml: YataiService monitoring and prometheus endpoint

[parano]

Currently, there is no easy way to monitor a YataiService deployment.

• Add a '/healthz' HTTP endpoint via the node server, in addition to te gRPC health check endpoint
• Add a '/metrics' HTTP endpoint via the node server, and allow expose metrics from the python gRPC server

[aarnphm]

I would love to take this on

[CallmeMehdi]

Hello @parano, I am currently working on this issue, I just wanted to ask, what does the endpoint /healthz do exactly? Thank you!

[parano]

Hi @CallmeMehdi - I think @aarnphm is currently working on this issue as well, maybe you two can sync up and discuss?

"/healthz" endpoint is a convention in building microservices, sometimes a deployed service can get into a bad state and become incapable of handling requests. "/healthz" is a special endpoint that let the monitoring system to discover such failed service instance and do something about it(usually restarting the instance)

Implementing the healthz endpoint is usually very straightforward, e.g. https://cloud.ibm.com/docs/node?topic=node-node-healthcheck

[yubozhao]

@CallmeMehdi @aarnphm Any update on this issue? Let me know if you are running into any issues, happy to help