Nest: Add metrics module

[weeco]

For a better monitoring of my Nest microservices I'd like to have a metrics endpoint which can be scraped by monitoring systems like Prometheus.

I believe a module written for NestJS could provide quite a couple metrics by default:

• Metrics about the Node process resources (offered by default in the prom-client library)
• Requests per second
  
  
  
  • All endpoints alltogether
  
  
  • Each endpoint
  
  
• Request duration
  
  
  
  • All endpoints alltogether
  
  
  • Each endpoint
  
  
• Number of responses grouped by response status codes for
  
  
  
  • All endpoints alltogether
  
  
  • Each endpoint
  
  
• Response times in a bucket ([0-20ms, 20-50ms, ...]) for
  
  
  
  • All endpoints alltogether
  
  
  • Each endpoint
  
  

Beside these default metrics which could be offered out of the box, it should be easily possible to inject the metrics module into services, so that users can create custom metrics for their business code (e. g. number of registrations).

Using these metrics we can easily attach grafana to it and monitor our services:

[kamilmysliwiec]

Sounds great! Do you have any further ideas regarding this feature request? if so, feel free to share them, I'd love to get familiar with the community expectations. 🔥

[weeco]

Well the list of possible integrations with NestJS is kinda endless. Beside the given integrations which could come out of the box in NestJS, you could also offer metrics about:

• Number of total routes bound in NestJS
• Integration of other NestJS modules (e. g. auth - so that you can see Logins (and registrations) grouped by strategy), or websockets (expose number of actibe websocket connections, ...)
• Offer the possibility to manage these out of the box metrics. Basically let a user disable, rename or manipulate specific metrics
• Default Grafana board for NestJS (I am happy to create this)

You may want to keep in mind that there are other metrics formats beside Prometheus (such as DataDog, Graphite, etc.) which you may want to add at some point.

Spring boot already offers such a metrics module, you can probably get further inspiration there: https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html

[cyril-lakech]

Maybe Trace from risingstack?

[DavisJaunzems]

By the way, swagger-stats (http://swaggerstats.io/) is very easy to plug into nest.js app, and you get an API endpoint (+ nice UI), that can be used to scrape metrics. However, you can't add your own metrics that easily.

[weeco]

@DavisJaunzems That looks interesting. Maybe we can use this as inspiration for our own metrics module. Two questions I wasn't able to figure out at a first glance:

1. How does swaggerstats persist the information it shows in the panel? Does it use prometheus and puts a UI on top of that (similiar to Grafana)?
2. How do you configure dynamic routes (e. g. https://example.com/users/{id}) . How does it know it should aggregate stats on example.com/users instead of collecting stats for each queried userId ?

[DavisJaunzems]

@weeco swagger stats stores the data in memory attached to the main node process (when restarted it starts from scratch), and for some metrics it only keeps last 100 entries. Then these values are exposed through prometheus friendly format that prometheus can pull.
It uses swagger.json information to know which routes to aggregate.

I have a simple monitoring solution on top of nest - you create injectable X and register values/functions to it. Then there is a global injectable that scans the nest.js context for these X injectables, and reads their values. Then there is a route that takes the global injectable values and displays them when route is accessed. Pretty simple solution, but not sure if it would make sense to be part of the main nest.js.

[spike008t]

I've just write a nest module for prometheus (see: https://github.com/digikare/nestjs-prom).
At the moment the module is quite simple... that is my first nestjs module.

[cojack]

@weeco but exactly what do you need swaggerstats.io provide, and this what @spike008t create solve a own metrics. So basically thread to close, or provide an PR for documentation HOW TO.

We're using swaggerstats.io for few our projects, and this is really great.

[weeco]

No @cojack you are wrong. swagger-stats (as the name already says) relies on swagger definitions. It's written a generic JavaScript library and therefore not written for a specific framework.

1. First of all the NestJS way to create swagger docs is using Type annotations using @nestjs/swagger , which is really nice because this way the swagger definition is very close to the corresponding code.

2. Also in NestJS we've got a lot more potential to do way more things automatically, instead of relying on self written swagger definition, which in practice or probably often out of sync with the actual code. NestJS does know or routes and our params. That's why NestJS is capable of creating a swagger documentation without you needing to write any documentation. We should use this potential and automatically create our prometheus stats for the given endpoints too. We can nicely integrate it into the framework which leads to a very nice user experience for developers

3. Using Swagger stats you couldn't easily add your own custom business metrics as proposed here in the issue.

[kamilmysliwiec]

I'm pretty sure that you started to work on such integration already @weeco, correct? :)

[weeco]

@kamilmysliwiec Well not really, but I am working on a proof of concept to show you and/or @BrunnerLivio afterwards :-).

[kamilmysliwiec]

Looking forward to it! If you have something to show already, let me know :)

[Chris2011]

@spike008t thx for the module. I need such integration for prometheus too, will check your module, whether it is enough for us or not.

[iangregsondev]

@weeco @kamilmysliwiec Do you know if any progress has been made on the integration of prometheus into the core of nestjs ?

I need to add this functionality next week :-)

[weeco]

@appsolutegeek To be honest I haven't worked on it for months anymore, as I am barely using NodeJS these days. Back in the days I had my own prometheus module + service for each project and added all desired metrics for each project on my own.

That was some boilerplate which I wanted to avoid (which is why I created this issue), but it doesn't block you from adding metrics on your own. I am pretty sure you won't get metrics built into NestJS until next week. I am not sure if Kamil or anyone else is working on this at all. Still seems to be the most requested feature though.

[iangregsondev]

Ok, yep, I will have a go at doing something quick myself. I just didn't want to reinvent the wheel if there was something available

[iangregsondev]

@spike008t Just saw your module over at https://github.com/digikare/nestjs-prom

Looks great, I will probably try to adapt it for my use. I presume you are not currently updating it ?

@kamilmysliwiec have you thought about pulling this module into nest and offering it out of the box like terminus, typeorm etc.

I must admit, I haven't tested it yet :-) - But it looks pretty much complete although the author does state there were some things left to improve.

[iangregsondev]

I thought I would relay my findings here, I am currently using the module in its current state. The author doesn't have time to maintain it anymore but its a great start and currently it does what I need - so I am quite happy.

So confirmed, tested and using it now in production!

[yevdev-fiscalnote]

https://github.com/digikare/nestjs-prom

Seems to work well. With a small tweak it should be possible to add a middleware or guard to restrict access to /metrics unless you come in via a management port. In the case of people that are coming from SpringBoot using port 8888. Then simply restrict access to that port in your infrastructure for internal use only.

https://docs.nestjs.com/faq/multiple-servers

[ZenSoftware]

It would be really nice if we could get some more attention with respect to this. Nest has a lot of potential to provide amazing telemetry by integrating Prometheus. Being able to visualizing spikes of 400-500 status codes for example would have tremendous value.

[Insidexa]

@kamilmysliwiec maybe create metrics nestjs module with appmetrics to export metrics nodejs app and nestjs some metrics as some standart.
Who need to export to prometheus / influxdb get info from "some" nestjs metrics standart.
Example export metrics to influx db from node app - https://github.com/Insidexa/traefik/blob/master/node-app/index.js
and show metrics on grafana dashboard

[SqueezeToyAliens]

Hi,

My company relies on Nest, and we forked and improved a Github project that integrates with Prometheus. It exports a Nest module and provides metrics for NodeJS and a few standard metrics for HTTP services(both are optional).

I can provide the Nest team access to the project for reviewing it if you think that it might be usable for you.

[kamilmysliwiec]

I can provide the Nest team access to the project for reviewing it if you think that it might be usable for you.

That would be great @SqueezeToyAliens!

[omerxx]

Asking everyone on this thread (and @kamilmysliwiec @weeco @spike008t @SqueezeToyAliens in particular) Any progress with this? It would be awesome to implement something of this kind on our own product.
Thank you, guys!

[iangregsondev]

@omerxx I am using this https://github.com/digikare/nestjs-prom - it works well, i would love it to be merged into nestjs though.

Also would be interesting in knowing how the @SqueezeToyAliens implementation is/

[weeco]

In case it's helpful, that's the approach I use to instrument request durations in Go. I simply create a middleware which gets the "routePattern" (to avoid highly cardinal labels on dynamic routes), measure the request duration and put that along with the response status code into my Histogram:

https://github.com/kafka-owl/kafka-owl/blob/master/backend/pkg/common/middleware/instrument.go

PS: I'd also highly recommend to add metrics in the logger, which should be kinda easy if you can create a hook for the log events, see: https://github.com/kafka-owl/kafka-owl/blob/master/backend/pkg/common/logging/logger.go#L29-L54

[Avejack]

I totally love nestjs - and it already comes with a lot of powerful tools and extensions. But this would be an enormous improvement!

Is there any progress or has this idea been abandoned?

[fieldju]

👋 , Hello I am a metrics nerd and primary maintainer of a few metrics/observability/canary libraries Kayenta, Node Measured, Armory's Spinnaker Observability Plugin

I am also a typescript advocate, I recently have discovered NestJs and +1 @Avejack's comment.

I would like to volunteer to own integrating a vendor-agnostic library and establish patterns for observability for NestJs similar to Springboot and Micrometer.

I think Node-Measured could fit the bill here.
I would want to take care of the following issue first https://github.com/yaorg/node-measured/issues/82
But it would be great to integrate a vendor-agnostic metrics instrumentation library into this framework.

I wouldn't want to invest the time into ramping up in the NestJs internals and make the PR for this unless there was buy-in from the project owners and community.

Should we start and RFC, what would that look like I wonder?

https://discord.com/channels/520622812742811698/527863537708695562/725430518815916052