Thanos: Rule UI fails to load, template expansion failed, can't evaluate field GetEvaluationTimestamp

[sevagh]

I can no longer load the loaded rules page (/rules) in the Thanos Rule UI since upgrading Prometheus to 2.21.0 and Thanos to 0.16.0-rc.0:

Oct 13 13:11:14 thanos[37515]:
level=warn
ts=2020-10-13T20:11:14.835535904Z
caller=ui.go:152
component=rules
msg="template expansion failed"
err="template: :67:48: executing \"content\" at <.GetEvaluationTimestamp.IsZero>: 
can't evaluate field GetEvaluationTimestamp in type rules.Group"

I see the following issue in the Prometheus project that may be related? https://github.com/prometheus/prometheus/pull/7865

[prmsrswt]

I see the following issue in the Prometheus project that may be related? prometheus/prometheus#7865

Yes, that seems to be the case. Help wanted, this should be easy enough.

[allenmqcymp]

Hi, is the issue located here
https://github.com/thanos-io/thanos/blob/c885f7a39ec776e999cf741141abf4d58e598d1c/pkg/ui/templates/rules.html#L13

where

GetEvaluationTimestamp

needs to be changed to

GetLastEvaluation

wherever GetEvaluationTimestamp occurs?

Like how https://github.com/prometheus/prometheus/pull/7865 does it?

[prmsrswt]

Yep, exactly.

[allenmqcymp]

@prmsrswt How would I go about verifying the change works - ie. how do I run the Thanos Rule UI? Thanks!

[sevagh]

If you have a branch, I can build from it to test it. I have a fully working Thanos environment with broken rules UIs.

[allenmqcymp]

Alright, do you want to try build and test from https://github.com/allenmqcymp/thanos/tree/rulefix.

[prmsrswt]

Also for future you can use https://github.com/thanos-community/thanos-docker-compose for a local set-up using docker and docker-compose.

[sevagh]

I'll be able to test this shortly. Do I need to run "make assets" before "make build"? I only did "make build" which didn't fix the issue. Trying "make assets && make build" now.

[sevagh]

With the fixed binary I get this error now:

Oct 19 06:49:08 dc6-thanos2 thanos[3244]: level=warn ts=2020-10-19T13:49:08.115333884Z caller=ui.go:152
component=rules
msg="template expansion failed"
err="template: :93:37: executing \"content\" at <.GetLastEvaluation.IsZero>: 
can't evaluate field GetLastEvaluation in type rules.Rule"

[sevagh]

Actually, that worked. This is the final change I made (+ make assets && make build):

$ git diff --cached pkg/ui/templates/rules.html
diff --git a/pkg/ui/templates/rules.html b/pkg/ui/templates/rules.html
index 41bb9083..01270ff4 100644
--- a/pkg/ui/templates/rules.html
+++ b/pkg/ui/templates/rules.html
@@ -10,8 +10,8 @@
         <thead>
           <tr>
             <td colspan="3"><h2><a href="#{{reReplaceAll "([^a-zA-Z0-9])" "$1" .Name}}" name="{{reReplaceAll "([^a-zA-Z0-9])" "$1" .Name}}">{{.Name}}</h2></td>
-            <td><h2>{{if .GetEvaluationTimestamp.IsZero}}Never{{else}}{{since .GetEvaluationTimestamp}} ago{{end}}</h2></td>
-            <td><h2>{{humanizeDuration .GetEvaluationDuration.Seconds}}</h2></td>
+            <td><h2>{{if .GetLastEvaluation.IsZero}}Never{{else}}{{since .GetLastEvaluation}} ago{{end}}</h2></td>
+            <td><h2>{{humanizeDuration .GetEvaluationTime.Seconds}}</h2></td>
           </tr>
         </thead>
         <tbody>

Can anyone who knows the code better confirm that these lines don't need to be changed? https://github.com/allenmqcymp/thanos/commit/26ef1dca7ce195617333c5345bf5e0c7afd65223#diff-f46df6e62ca9bd8a2374a0b3431590f859f6766e73be74c5e93ab86a42108886L39-L41

[prmsrswt]

Looking at https://github.com/prometheus/prometheus/blob/master/rules/recording.go#L177 I think that we don't need to change anything on line 39 and 41. What you did seems correct, we only need to change those two lines.

[allenmqcymp]

Cool. @sevagh if you want to submit a PR fixing it, that's fine by me since you ended up testing. @prmsrswt thanks for the help. Will try look for another issue

[sevagh]

As soon as I get my other open PR merged, I'll open one for this fix.