Elasticsearch: composite aggregation with filtered terms sources

[edovac]

Hi,
I'm working on a project requiring something like an heat map between named entities, starting from most frequent values of these entities.

I.E.: We need to calculate document count for people to places document occurrences, or even people to people document occurrences.

I tried different kind of aggregations, but I think the composite aggregation is fit for the problem, using terms value source.
Scrolling is not an option, since the combinations number could be very high, and I only need an NxM matrix for which the entry points are fixed terms values and the cells are document counts. Usually N and M are usually between 3 and 10.
I know in advance the values needed for each terms value source, so it would be easier to have an include parameter (like in the terms aggregation) to specify which values are to be combined and then counted.

The resulting syntax could be something like this:

{
    "size": 0,
    "aggs": {
        "heat_map": {
            "composite": {
                "sources": [
                    {
                        "people": {
                            "terms": {
                                "field": "people",
                                "include": ["John Smith", "Alice White", "Jay", "Silent Bob"]
                            }
                        }
                    },
                    {
                        "places": {
                            "terms": {
                                "field": "places",
                                "include": ["London", "Rome", "New York", "Berlin"]
                            }
                        }
                    }
                ]
            }
        }
    }
}

Obviuosly, included terms are user generated or previously loaded with a terms aggregation.

Thanks for the support and any insight.

[cbuescher]

@edovac upon a quick glance at your example, it seems to me that you could use the filters aggregation for your purpose. On the first level you would create one e.g. for the locations (using a terms query for each, for example) and then use a filters sub-aggregation with the person names. Would that work for your use case? I might me missing something though.

[elasticmachine]

Pinging @elastic/es-search-aggs

[edovac]

@cbuescher thanks :)

It's working as expected.
At first i didn't fully understand the flexibility of filters aggregation. The syntax is a little verbose, but as long as it works it's ok ;)

[edovac]

Cosing the issue

[iukea1]

can you guys post an example of how you were able to pull off filters in composite?

[khlystov]

@edovac could you show us your final example ?