Kibana: Make Singnals usable in queries.

[nyurik]

_From @thatguy7 on February 26, 2018 17:7_

Hi,
My goal is to have two queries, over two separate indices, where the second one depends on the results of the first one.

I can extract the necessary values from the first query and store them in a signal, but I cannot actually use any signals inside the 'url' object.

Is there a chance this feature will be added, or is present but I am using it wrong?

_Copied from original issue: nyurik/kibana-vega-vis#46_

[nyurik]

Hi @thatguy7, this is not yet possible -- it has been requested before, and there is some work being done on it at https://github.com/vega/vega/issues/912

[nyurik]

P.S. On the second thought, it is possible to do a workaround inside Kibana, but I would really like to wait for the dynamic data support first, or else it will be a relatively hacky solution. As a syntax, we could have something like this inside the the data url block. Would that work for you?

data: { url: {
    "aggs" : {
        "xxx" : {
            "terms" : {
                "field" : { "%signal%": "mysignal" }
            }
        }
    }
}}

[nyurik]

_From @thatguy7 on February 27, 2018 8:12_

That could work for me and be used as an interim solution, until the dynamic data support is figured out. My current approach was creating a data object, then a scale from it, then a signal containing the domain of that scale as a list of terms that I want to filter by. Then that signal should have determined the query of a second data object.

I am not sure if adding that syntax would make this possible, since the signal's value would be decided by the first query, and affect the other then - so I would need to be sure the signal has the right value at the time the second query is performed.

I would greatly appreciate it, if this functionality was added. I am currently figuring out the most suiting way to store our data in Elasticsearch for further evaluation. Right now I am resorting to nesting some data, but that structure does not play well at all with Kibana and is not very flexible - but still usable with the Vega plugin.

So ideally I would just split it into multiple indices and do two queries, where I can first determine some terms in one query which depends on the Kibana context and then use those terms as filters in the second query. This would mean I can still preserve the relationship in the data, while being able to work with it in Kibana separately.

[nyurik]

@thatguy7 would the signal be based on a user action, or would it be calculated from the original data?

[nyurik]

_From @thatguy7 on February 27, 2018 17:14_

It would be calculated from the original data.

As I tried to outline it in my previous post, I want to perform a terms aggregation with the context of Kibana applied on index1 - I do this to get the all the IDs of the documents in index1 that match the context.
From this aggregation I define a scale, so I get a nice array as its domain, that contains all these IDs. These are then stored in the signal, which I want to use to perform the second query on index2, where every document has a field that matches it to the ID of the first index.

This way I do two queries, where first I get a list of document IDs from index1 that match certain filters and fall in a certain time range, and then a second one that aggregates statistics of the data in index2, but only for documents that are related to the documents that were matched in index1.

In a nutshell, I want to filter data in index2 by fields that are in index1.

[jonatel]

Any advance on this issue? In a similar way, I'm trying to change the index name using a signal input.

"signals": [
    {
      "name": "level", 
      "value": "host_correlations", 
      "bind": {
        "input": "radio", 
        "options": ["host_correlations", "app_correlations"]
      }
    },

"data": [
    {
      "name": "tree",
      "url": {
        **"index": {"signal": "level"},**
        "body": {
          "size": 10000,
          "query": {
            "match": {
              "data": "nodes"
            }
          }
        }
      },