Elasticsearch: geo_shape queries no longer accept circle in 6.6.0

[hudsonb]

Upgrading from 6.5.1 to 6.6.0 all of our geo_shape queries which use a Circle are broken.

Attempting to use a Circle in a geo_shape query now throws an UnsupportedOperationException with the message: "CIRCLE geometry is not supported".

I tested these same queries on a developer machine that was still running 6.5.1 and they execute without issue.

[elasticmachine]

Pinging @elastic/es-analytics-geo

[iverase]

Hi @hudsonb,

Thanks for you interest in Elasticsearch.

I have tried locally to connect to an index created on version 6.5.1 and everything worked as expected. What do you mean with upgrade?

Note that in version 6.6.0, the default geo_shape strategy was changed and it uses a faster and more precise indexing strategy based on Lucene BKD tree. Unfortunately, this strategy does not support querying by circles yet.

If you are creating a new index and you are interested in the former behaviour, you will need to set up the strategy, for example:

curl -X PUT "localhost:9200/example" -H 'Content-Type: application/json' -d'
{
    "mappings": {
        "doc": {
            "properties": {
                "location": {
                    "type": "geo_shape",
                     "strategy" : "recursive"
                }
            }
        }
    }
}'

If this does not solve your issue, could you please add the steps to reproduce it?

I would like to recommend you to upgrade to version 6.6.1 as it contains some bug fixes.

[hudsonb]

Sorry, by upgrading I meant installing elasticsearch 6.6.0, updating our type mappings to take advantage of the faster and more precise BKD trees for geo_shapes, and reindexing our data.

Note that in version 6.6.0, the default geo_shape strategy was changed and it uses a faster and more precise indexing strategy based on Lucene BKD tree.

This was our primary motivation for upgrading.

Unfortunately, this strategy does not support querying by circles yet.

This was not mentioned in Breaking Changes so it caught us off guard. The geo_shape query docs do say that they work with GeoJSON shapes, which doesn't have the concept of circle so maybe we should have expected this.

If you are creating a new index and you are interested in the former behaviour, you will need to set up the strategy

But then we'd lose out on the most exciting feature of 6.6 =). We've worked around the issue by converting circles to polygons, using the polygons for geo_shape queries.

I would like to recommend you to upgrade to version 6.6.1 as it contains some bug fixes.

Will do

+1 for adding support for circle geo_shape queries to BKD based geo_shapes.

[sumpton]

@hudsonb is there something on the Internet you could point me to to help me learn how to convert circles to polygons? Our input forms take a location (point) and a radius and was working brilliantly in Elasticsearch.

We query both ways for stored circles that contain a point and for points that are contained in the query.

Are you using something like http://turfjs.org/docs/#circle

[hudsonb]

JTS can generate them for you, see GeometricShapeFactory.createCircle.

Replicating what Turf.js circle does is fairly easy. I've been porting Turf.js over to Kotlin (unreleased) and have circle implemented if you want to take a look at how it's implemented: https://github.com/hudsonb/turf.kt

[sumpton]

Thanks for taking the time to help me. I had not looked at Kotlin code before and not think all functions should be "fun".

I am working in Node and Turf is working for me for now.

I noticed you default to 64 steps. Do you happen to know if query performance in Elasticsearch in affected by the size of the coordinates array? I don't require a very accurate circle, but I don't know how many steps I should use?

Do you happen to know if the Elasticsearch implementation of circle was actually a polygon?

I am surprised that the removal of Circle is not a breaking change since it was documented.

[hudsonb]

I am working in Node and Turf is working for me for now.

Sorry I assumed you were on the JVM, yes I'd use turf.js in that case

I noticed you default to 64 steps. Do you happen to know if query performance in Elasticsearch in affected by the size of the coordinates array? I don't require a very accurate circle, but I don't know how many steps I should use?

I can't say I know for sure but I would think at some level it must be. Determining if a polygon contains a point becomes more CPU intensive as the complexity of the polygon increases. I default to 64 simply because that's the default in turf.js. You can use fewer points if you don't need a lot of accuracy, just note that you are decreasing the area of the circle so you may miss some results.

Do you happen to know if the Elasticsearch implementation of circle was actually a polygon?

Not sure, I haven't looked into it.

I am surprised that the removal of Circle is not a breaking change since it was documented.

Caught me off guard for sure!

[sumpton]

Thank you again @hudsonb. I appreciate your help.

[lowwa132]

What's the status of this issue? Is circle back in later version of Elasticsearch, or planned to be?

[bramford]

@lowwa132 raises a valid question. I'm using ES latest (7.8) and still getting circle geometry is not supported on the "new" default geo_shape indexing strategy (said to be BKD tree).

The docs say

For the default Indexing approach circles should be approximated using a POLYGON

@iverase Is this just a workaround while we wait for native Circle support in the default strategy or is the goal to deprecate the Circle type? If we're just waiting, it would be helpful to link the feature request to this issue for transparency.

[iverase]

@bramford we currently only support circles natively at search time. For indexing you need to approximate to a polygon as stated in the docs, we are currently not working in adding it natively. We added a ingest circle processor to help:

https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest-circle-processor.html