Loki: invalid chunk time range value

[pandey-adarsh147]

Describe the bug
While executing this query from Grafana
{container="istio-proxy"}

Getting following logs

level=warn ts=2020-04-07T05:09:30.113738531Z caller=logging.go:49 traceID=3f93709803b96f2a msg="GET /loki/api/v1/query_range?direction=BACKWARD&limit=1000&regexp=&query=%7Bcontainer%3D%22istio-proxy%22%7D&start=1586197800000000000&end=1586284200000000000&step=8 (500) 225.049337ms Response: \"invalid chunk time range value: 6b653a6431383335393a6c6f67733a636f6e7461696e657200754f6e43704a666138426d4f47336e5142454e68373876724551734f45676c756c767a3061713376317a340056622f64783776673437456a6171414136584d2b6738\n\" ws: false; Accept: application/json, text/plain, /; Accept-Encoding: gzip, deflate, br; Accept-Language: en-IN,en;q=0.9,hi-IN;q=0.8,hi;q=0.7,en-GB;q=0.6,en-US;q=0.5; Sec-Fetch-Dest: empty; Sec-Fetch-Mode: cors; Sec-Fetch-Site: same-origin; User-Agent: Grafana/6.6.1; X-Forwarded-For: 10.240.0.100, 10.240.0.100; X-Grafana-Org-Id: 1; "
To Reproduce
Steps to reproduce the behavior:

1. Started Loki 1.4.0 (I am new user)
2. Using fluentd to push logs in loki
3. Query: {container="istio-proxy"} not working. In some cases it is working.

Expected behavior
It should give me logs lines

Environment:

• Infrastructure: Local both index and chunk
• Deployment tool: binary

[cyriltovena]

I'll take a look soon, it seems that you have a corrupted index. Did Loki restarted or crash in anyway at some point ?

[cyriltovena]

I decode your error and I think you either use a wrong config or you have a corrupted index.

Can you please share the config file you're using, I'm guessing you have a wrong schema config.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[xmackex]

@cyriltovena Hello, I have look alike issue, but 500 and invalid chunk time range value ... happens randomly or when a lot of messages comes in. Interesting part is that Loki is not OOM killed, etc.

Index size is ~1g.

Loki schema config is

schema_config:
  configs:
    - from: 2020-06-03
      store: boltdb
      object_store: s3
      schema: v11
      index:
        prefix: loki-index-us-west-2-
        period: 168h # 7 days