Thanos: Add additional log messages around the retries for Thanos sidecar

Created on 20 Mar 2019  路  1Comment  路  Source: thanos-io/thanos

Thanos, Prometheus and Golang version used
Prometheus 2.7.2

# thanos --version
thanos, version 0.3.2 (branch: HEAD, revision: 4b7320c0e45e3f48a437bd19294f569785bafb02)
  build user:       circleci@4ea007a18ffa
  build date:       20190304-17:16:56
  go version:       go1.10.8

What happened
At sidecar boot, there is an error concerning the connection with Prometheus and nothing about a success retry whereas everything works properly. It's a bit confusing

What you expected to happen
A log message about the success of the connection retry

How to reproduce it (as minimally and precisely as possible):
Bootstrap with kube-prometheus with thanos

Full logs to relevant components

level=info ts=2019-03-18T11:25:47.781082802Z caller=flags.go:90 msg="StoreAPI address that will be propagated through gossip" address=10.233.100.185:10901
level=info ts=2019-03-18T11:25:47.788039533Z caller=main.go:256 component=sidecar msg="disabled TLS, key and cert must be set to enable"
level=info ts=2019-03-18T11:25:47.788110894Z caller=factory.go:39 msg="loading bucket configuration"
level=info ts=2019-03-18T11:25:47.788979005Z caller=sidecar.go:293 msg="starting sidecar" peer=
level=info ts=2019-03-18T11:25:47.790440086Z caller=sidecar.go:220 component=sidecar msg="Listening for StoreAPI gRPC" address=[10.233.100.185]:10901
level=info ts=2019-03-18T11:25:47.790605586Z caller=main.go:308 msg="Listening for metrics" address=0.0.0.0:10902
level=warn ts=2019-03-18T11:25:47.791852512Z caller=sidecar.go:132 msg="failed to fetch initial external labels. Is Prometheus running? Retrying" err="request flags against http://localhost:9090/api/v1/status/config: Get http://localhost:9090/api/v1/status/config: dial tcp 127.0.0.1:9090: connect: connection refused"
level=info ts=2019-03-18T11:48:47.797909147Z caller=shipper.go:375 msg="upload new block" id=01D68A37EQQ9947B8E52K7DEAC

Anything else we need to know
A thread has been created on prometheus-operator on this subject with the final analysis: https://github.com/coreos/prometheus-operator/issues/2490

Most helpful comment

IMHO this is reasonable request, I created PR adding the line you requested.

>All comments

IMHO this is reasonable request, I created PR adding the line you requested.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

bwplotka picture bwplotka  路  3Comments

gburek-fastly picture gburek-fastly  路  4Comments

bwplotka picture bwplotka  路  4Comments

hedeesaa picture hedeesaa  路  3Comments

sbocz picture sbocz  路  4Comments