Puma: Getting the queue time in headers

[vprigent]

Is your feature request related to a problem? Please describe.
I run a small rails project and we are using ScoutAPM for our monitoring. One feature we really miss when under load is to know how much of the response time is effectively spent in the queue. Because we are running standalone puma, we don't rely on nginx for queueing so we can't collect that metric at this level.. however, I would expect to be able to know how long the request waited in the queue when reaching puma and be able to report that to ScoutAPM/any APM tool.

Describe the solution you'd like
An option to enable tracking of the queueing time, or having this enabled by default (I assume there would be a complexity matter as much as a performance matter here?)

Describe alternatives you've considered
We could setup nginx but at this stage, the project would really only use nginx for the purpose of tracking the queueing time.
We are running this project on AWS ECS, the traffic goes through an Application Load Balancer. The immediate benefit of another layer like nginx running on our infrastructure isn't obvious.

I may also be wrong here by assuming that it is something that it is puma's role to provide that since it provides a queueing mechanism?

Many thanks!

[nateberkopec]

Unfortunately, this is not possible.

Puma actually doesn't provide the queueing mechanism: the operating system does. Requests connect to the socket and get a TCP ACK, but Puma doesn't actually touch the request until a thread starts actually processing it.

Queue time tracking works on the principle of something adding a request start time header, and then a Rack middleware checks Time.now - that-time-in-the-header. Puma cannot add that request start time header, because to do so it would have to actually process the request, defeating the purpose and always reporting near-zero queue time.

I wish Amazon would do this with ALB or ELB, but they do not and you cannot configure it to do so. You must use nginx or another reverse proxy to add this header when using AWS.

[vprigent]

Thanks for the reply @nateberkopec and for clarifying how this all works!

[nateberkopec]

It's really a limitation of loadbalancers on AWS more than anything else. Heroku adds this header in their load-balancing layer by default, for example, so you can get queue time reporting w/o NGINX on Heroku.