Incubator-superset: Jinja filter_values function not working in charts used as annotation in another chart

[oashton]

Expected results

the filter_values function should work in a datasource query used in a chart, even if the chart is used as a annotation chart

Actual results

if a chart is used as a annotation in another chart, the filter_values function used inside the datasource query doesn't work, it doesn't return any value. Important to note: a filter value is selected in the dashboard.

How to reproduce the bug

1. Create a line chart to be used as a annotation in another chart. The datasource query should use the filter_values function
2. Create another line chart, add the previosly created line chart as a annotation
3. Add the line chart created in the previous step in a dashboard, add a filter box with the same field used as a parameter in the filter_values of the Step 1
4. The annotation doesn't show up as expected

Environment

• superset version: superset version
  Superset 0.37.0:
  Python 3.6.9
  Flask 1.1.2
  Werkzeug 1.0.1
• python version: python --version
  Python: 3.6.9
• node.js version: node -v
  node: v12.10.0
• npm version: npm -v
  npm: 6.10.3

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [X] I have reproduced the issue with at least the latest released version of superset.
• [X] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

I changed the following the line if (!isDashboardRequest && fd) { (line 254) in chartAction.js to if (fd !== null) { and it solved the problem but I'm not sure about the impact in other behaviours

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.95. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[lilila]

@oashton , does this change fix the behaviour in explore chart mode for you? Do you know how to change this?
Considering your question on the impact, it seems (#10115) this behaviour was chosen on purpose.

I did not get the point of having no control on annotation.

For me if you want to avoid this problem (mentioned in #10115)

" This will cause chart and annotation layer both have same query, so we can't compare filtered data vs original annotation layer."

You can simply create a new view/table for this annotation using fields name different from the ones you use in your filters.

@graceguo-supercat Any clue on this? Thank you

[oashton]

@lilila , it doesn't work in explore mode, right now I don't know how to change it.
Thanks for the reference to the issue #10115... so the behaviour is by design.

I agree with you, I don't get why lose the possibility to add dynamic behaviours to the annotation. Let's see what @graceguo-supercat says about this.

cc @camilo-uptime, check this thread, this is why we weren't receiving the filter values in the annotation line chart from the dashboard

[lilila]

Dear @oashton ,

I wrote this lite patch to get annotation considering extra_filters in Explore view as well.
I add these lines at line 260 in your screenshot

    //case for explore view (have extra params given to annotation)       
    var new_extra_from_adhoc = [];
    if (!fd.extra_filters && fd.adhoc_filters){
        for(var i=0; i<fd.adhoc_filters.length; i++){
            if (fd.adhoc_filters[i]['isExtra']){
                    var new_extra = {"col":fd.adhoc_filters[i]['subject'],
                                     "op": fd.adhoc_filters[i]['operator'],
                                     "val": fd.adhoc_filters[i]['comparator']};
                    new_extra_from_adhoc.push(new_extra);}
            }
        sliceFormData.extra_filters = new_extra_from_adhoc;
    }

@villebro , I don't know if you remember but we discussed this point.
The problem was that in explore mode, extra_filters are already merge with adhoc filters so they were not passed to the annotations. The code snippet (sorry for the quality I never code in Javascript) simply recreates the extra_filters from the adhoc_filters available in Explore mode

[villebro]

Over the holidays I'll try to take a look at how this could be properly integrated into the annotation requests.