Thanos: /metrics and probe routes does not respect web.route-prefix

[marcusramberg]

Tested on Thanos 0.8.1, but still applies in master.

Object Storage Provider:

What happened:
Even if you run query as query --web-route-prefix="/thanos" - metrics and probes are served from root.

What you expected to happen:
Metrics should be served from /thanos/metrics

How to reproduce it (as minimally and precisely as possible):

thanos query --web.route-prefix="/thanos"
curl localhost:10902/metrics # gives metrics
curl localhost:10902/thanos/metrics # gives 404

Full logs to relevant components:

Anything else we need to know:

[kakkoyun]

I have introduced this regression, sorry for the inconvenience.
I missed this prefix behaviour for querier and ruler, I'll fix it immediately.

And add necessary tests to prevent it from happening again.

[bwplotka]

BTW is it really regression?

[kakkoyun]

@bwplotka This I want to discuss as well.

I guess intended usage --web.route-prefix="/thanos is only for API and UI. This flag is only exist in rule and query subcommand. For other subcommands, we don't have it and we don't use any prefix for /metrics and /debug/pprof/ paths without prefix.

And it flag description it says,"Prefix for API and UI endpoints. This allows Thanos UI to be served on a sub-path. This option is analogous to --web.route-prefix of Promethus."

I think it's not a regression, at least comparing to intended behaviour, I haven't checked what happens in older versions.

What do you think?

[bwplotka]

It is really mixed on Prometheus side as well. Metric endpoint is local, rest (debug and readiness) are prefixed.

The question now is what is intended here (:

[kakkoyun]

@marcusramberg Assuming this was the behaviour for pre v0.8.1, wasn't it?

[marcusramberg]

I have no idea, I'm setting up a new cluster. But it is definitely the behavior I would expect.

[bwplotka]

I would vote to stick debug and internal endpoints under the root, it is usually something you don't want to expose to outside, but happy to hear more opinions.

[FUSAKLA]

Even though I agree that the debug endpoints should stay at the root it feels like doing some decision for the user.

If he needs the prefix he probably already has some proxy in front of it where he can block those endpoints and on the other hand it can just complicate things for some uncommon use-cases :thinking:

[marcusramberg]

Fwiw Prometheus remaps health check according to web.external-url.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.