Thanos: Compact: Bucket WebUI doesn't show up on compact

Created on 15 Apr 2020 · 3Comments · Source: thanos-io/thanos

Thanos, Prometheus and Golang version used:
quay.io/thanos/thanos:v0.12.0

Object Storage Provider:
AWS

What happened:
According to the new release notes the bucket web ui should be displayed on / when the compactor is set to wait but when I attempt to use it, I get a 404 page not found

What you expected to happen:
The bucket web ui would populate.

How to reproduce it (as minimally and precisely as possible):

Run a instances of thanos compact in k8s (see config below)
Port-forward to 8080 to 10902 on the pod.
Go to localhost:8080/metrics -- Works
Go to loclahost:8080/ -- fails with 404

Full logs to relevant components:

Logs

level=info ts=2020-04-15T16:56:15.364777436Z caller=factory.go:38 msg="loading tracing configuration" level=info ts=2020-04-15T16:56:15.364869067Z caller=jaeger.go:47 msg="loading Jaeger tracing configuration from ENV" level=info ts=2020-04-15T16:56:15.365605775Z caller=logger.go:18 msg="Initializing logging reporter\n" level=info ts=2020-04-15T16:56:15.36623115Z caller=factory.go:46 msg="loading bucket configuration" level=info ts=2020-04-15T16:56:15.367607594Z caller=compact.go:375 msg="retention policy of raw samples is enabled" duration=9600h0m0s level=info ts=2020-04-15T16:56:15.3682701Z caller=compact.go:522 msg="starting compact node" level=info ts=2020-04-15T16:56:15.368324321Z caller=intrumentation.go:48 msg="changing probe status" status=ready level=info ts=2020-04-15T16:56:15.368716065Z caller=intrumentation.go:60 msg="changing probe status" status=healthy level=info ts=2020-04-15T16:56:15.368754635Z caller=http.go:56 service=http/server component=compact msg="listening for requests and metrics" address=0.0.0.0:10902 level=error ts=2020-04-15T16:56:15.38461867Z caller=runutil.go:98 msg="function failed. Retrying in next tick" err="BaseFetcher: iter bucket: Get REDACTED dial tcp 52.217.43.22:443: connect: connection refused" level=info ts=2020-04-15T16:56:15.980442753Z caller=compact.go:887 msg="start sync of metas" level=error ts=2020-04-15T16:56:15.987637233Z caller=compact.go:471 msg="retriable error" err="compaction: sync: BaseFetcher: iter bucket: Get REDACTED dial tcp 52.217.43.22:443: connect: connection refused" level=info ts=2020-04-15T16:57:26.796102996Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=11.417696956s cached=11030 returned=11030 partial=65 level=info ts=2020-04-15T16:57:33.092622838Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=6.041766576s cached=11030 returned=11030 partial=65 level=info ts=2020-04-15T16:58:21.35007527Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=5.971662477s cached=11030 returned=11030 partial=65 level=info ts=2020-04-15T16:59:20.985702485Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=5.607326165s cached=11030 returned=11030 partial=65 level=info ts=2020-04-15T17:00:20.883543709Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=5.505184687s cached=11032 returned=11032 partial=65 level=info ts=2020-04-15T17:01:15.368788667Z caller=compact.go:887 msg="start sync of metas" level=info ts=2020-04-15T17:01:22.265698156Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=6.887360616s cached=11049 returned=11049 partial=68 level=info ts=2020-04-15T17:01:32.224839643Z caller=fetcher.go:451 component=block.BaseFetcher msg="successfully synchronized block metadata" duration=16.855944635s cached=11049 returned=1183 partial=68 level=info ts=2020-04-15T17:01:32.238297815Z caller=compact.go:892 msg="start of GC" level=info ts=2020-04-15T17:01:32.241564577Z caller=compact.go:904 msg="start of compactions"

Anything else we need to know:
Container configuration:

        - name: thanos-compact
          image: "quay.io/thanos/thanos:v0.12.0"
          imagePullPolicy: IfNotPresent
          args:
            - "compact"
            - "--log.level=debug"
            - "--log.format=logfmt"
            - "--http-address=0.0.0.0:10902"
            - "--objstore.config-file=/etc/config/object-store.yaml"
            - "--data-dir=/var/thanos/compact"
            - "--consistency-delay=120m"
            - "--retention.resolution-raw=400d"
            - "--retention.resolution-5m=0d"
            - "--retention.resolution-1h=0d"
            - "--block-sync-concurrency=125"
            - "--compact.concurrency=30"
            - "--delete-delay=0d"
            - "--wait"
            - "--selector.relabel-config-file=/etc/thanos/filters/filter-k8s.yaml"
            - "--tracing.config=type: jaeger"
          env:
            - name: JAEGER_AGENT_HOST
              value: "127.0.0.1"
            - name: JAEGER_SERVICE_NAME
              value: "thanos-compact-k8s.thanos"
            - name: JAEGER_AGENT_PORT
              value: "5775"
            - name: JAEGER_REPORTER_LOG_SPANS
              value: "true"
            - name: JAEGER_SAMPLER_PARAM
              value: "1"
          ports:
            - name: http
              containerPort: 10902

Source