Kibana: [Maps] Geo_shape aggregation support

[pierrecdn]

Hi,
First of all, thanks, Kibana 4 is really powerful.

I expected to make tile maps work with this kind of schema :

{
    "region" : {
        "properties" : { 
            "name" : {"type" : "string", "store" : true },
            "code" : {"type" : "integer", "store" : true },
            "population" : {"type" : "integer", "store" : true },
            "shape" : { "type" : "geo_shape", "precision" : "1km" }
        }
    }
}

From the "Discover" view, no major problems :

But it's impossible to get the shape field from the "Visualize/Tile Map" view :

Does these type of geo coordinates are not supported yet in kibana ?

[rashidkpc]

There is currently no support for geo_shape aggregations in elasticsearch. Should elasticsearch support geo_shape in some form for aggregation purposes we could look into implementing it.

[pierrecdn]

OK, after some research I found that it is related to https://github.com/elasticsearch/elasticsearch/issues/7574.
Many thanks Rashid

[dev-rke]

I also would like to see this feature realized.
I need to visualize relations between coordinates and it would be great to use a linestring or multilinestring. As far as i understood the elasticsearch docs, there is already support for geo_shape aggregations in elasticsearch, so only kibana needs to support a visualization for geo_shape data type.

[carlosvega]

+1

[snario]

+1

[lababidi]

+1

[jljorgenson18]

+1

Would this just be an aggregate of just polygon/linestring geo_points or would you be able to visualize areas as well? I image in some future version we could have heat maps for polygons.

[alexshires]

+1

[omarps]

+1

[sumit-gupta-sgt]

+1

[michimau]

+1

[yeroc]

+1

[epama]

+1

[chadblair]

+1

[ghost]

+1

[tomei-yo]

+1

[emmanuelmathot]

+1

[aparnavarma92]

+1

[peterneubauer]

+1

[joelstewart]

would be great to have aggregates of doc counts that have geo_shape within / intersect buckets that are realized by a query to another index holding the shape, something like:

{
   "aggs" : {
         "docs_in_shape" :{
               "geo_shape" : {
                    "relation": "within",
                    "field" : "geometry",  (aggregated field)
                    "indexed_shapes" : {
                        "query: { } //query to return shapes to use as buckets
                        "field":  "geometry", //field to get shape
                        "bucket_name" : "statename"

one could also provide inline geometries instead of "indexed_shapes"

[aravindiiitb]

+1

[fogfish]

+1

[MathieuB1]

+1

[thomasneirynck]

We are working on a new implementation of map visualizations and are not planning to add any new features to the existing implementation. We will take this suggestion under consideration for the new app.

[pierrecdn]

@thomasneirynck thanks for the update.

I've too admit that I was a bit naive 3 years ago thinking that this was almost doable :). A ton of hard work has been done in between.

Thanks for that, I hope to be able to use this new impl. soon !

[thomasneirynck]

Documents with geo_shape fields can now be visualized in the Maps app

https://www.elastic.co/guide/en/kibana/current/maps.html

[pierrecdn]

And that's awesome.
Thanks for this long but very well executed journey.

[glegal]

So does the geo_shape aggregation is now supported? will it be available in the next release of ELK?

[nreese]

So does the geo_shape aggregation is now supported

You can visualize geo_shape fields in the Maps application but there is still no geo_shape aggregation support.

Geo_shape aggregation support in Elasticsearch is an active work in progress. You can follow development on this branch https://github.com/elastic/elasticsearch/tree/geoshape-doc-values. Once elasticsearch has geo_shape aggregation support, maps application support will quickly follow.

I will re-open this issue until the Maps application supports geo_shape aggregations

[thomasneirynck]

Adding this to Maps will enable clustering/grid/heatmap layers for index-patterns with a geo_shape field. geo_shape-aggs support on the ES-side is licensed for Gold+ users, which means cluster/grids/heatmap layers for geo_shape-fields will also be only available for Gold+ users of Kibana Maps.

See also https://github.com/elastic/kibana/issues/56876": Kibana will need to nest the view-bounds under the geo_shape-aggregation for improved performance.

[pierrecdn]

I'd really love if some of you, elasticians, write a blogpost to quantify the ton of work achieved here and expose what it now unlocks for the maps.