Presto: Null pointer exception for PagesSpatialIndexSupplier.getEnvelope

Created on 13 Sep 2019  路  9Comments  路  Source: prestodb/presto

I'm running a spatial join with the rtree index by following this improvement
My query is:
select * from points u, polygons v where st_contains(ST_GeometryFromText(v.wkt), ST_Point(u.longitude, u.latitude))

And it throws the 

Query 20190913_052524_00112_ts9wn failed: Internal error
java.lang.NullPointerException
        at com.facebook.presto.operator.PagesSpatialIndexSupplier.getEnvelope(PagesSpatialIndexSupplier.java:149)
        at com.facebook.presto.operator.PagesSpatialIndexSupplier.buildRTree(PagesSpatialIndexSupplier.java:139)
        at com.facebook.presto.operator.PagesSpatialIndexSupplier.<init>(PagesSpatialIndexSupplier.java:94)
        at com.facebook.presto.operator.PagesIndex.createPagesSpatialIndex(PagesIndex.java:471)
        at com.facebook.presto.operator.SpatialIndexBuilderOperator.finish(SpatialIndexBuilderOperator.java:233)
        at com.facebook.presto.operator.Driver.processInternal(Driver.java:397)
        at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
        at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
        at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
        at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
        at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
        at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)
        at com.facebook.presto.$gen.Presto_0_220____20190910_150639_1.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

All my points and polygons are valid and I'm not sure why it throws null pointer exceptions.

picture liusztc09

All 9 comments

Hi Maria, @mbasmanova

I was following your instructions on this https://github.com/prestodb/presto/pull/9474 and ran into a nullpointer exception issue. I was running a simple st_contains join between point and polygon. Could you please take a look that it when you get a chance?

Thanks,
David

liusztc09 picture liusztc09 on 13 Sep 2019

When I converted it to the bing tile based query it throw such error:

The zoom level is too high or the geometry is too complex to compute a set of covering Bing tiles. Please use a lower zoom level or convert the geometry to its bounding box using the ST_Envelope function.
com.facebook.presto.spi.PrestoException: The zoom level is too high or the geometry is too complex to compute a set of covering Bing tiles. Please use a lower zoom level or convert the geometry to its bounding box using the ST_Envelope function.
        at com.facebook.presto.plugin.geospatial.BingTileFunctions.checkCondition(BingTileFunctions.java:712)
        at com.facebook.presto.plugin.geospatial.BingTileFunctions.checkGeometryToBingTilesLimits(BingTileFunctions.java:448)
        at com.facebook.presto.plugin.geospatial.BingTileFunctions.geometryToBingTiles(BingTileFunctions.java:375)
        at com.facebook.presto.$gen.PageProjectionWork_20190913_123313_169.evaluate(Unknown Source)
        at com.facebook.presto.$gen.PageProjectionWork_20190913_123313_169.process(Unknown Source)
        at com.facebook.presto.operator.project.DictionaryAwarePageProjection$DictionaryAwarePageProjectionWork.process(DictionaryAwarePageProjection.java:175)
        at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.processBatch(PageProcessor.java:295)
        at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.process(PageProcessor.java:183)
        at com.facebook.presto.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:315)
        at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:79)
        at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:65)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
        at com.facebook.presto.operator.project.MergingPageOutput.getOutput(MergingPageOutput.java:111)
        at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:296)
        at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:231)
        at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)
        at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
        at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
        at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
        at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
        at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
        at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)
        at com.facebook.presto.$gen.Presto_0_220____20190910_150638_1.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Query:

SELECT *
FROM
    (
        SELECT *, ST_GeometryFromText(wkt) as geometry
        FROM polygon
        CROSS JOIN UNNEST 
            (geometry_to_bing_tiles(ST_GeometryFromText(wkt), 17)) as t(tile)
     ) a,
    points b
WHERE a.tile = bing_tile_at(b.latitude, b.longitude, 17)
    AND ST_Contains(a.geometry, ST_Point(b.longitude, b.latitude))
liusztc09 picture liusztc09 on 13 Sep 2019

@liusztc09 Which version of Presto are you using? The stacktrace doesn't match latest master.

CC: @jagill

mbasmanova picture mbasmanova on 13 Sep 2019

@mbasmanova Thank you for the quick response! I'm using version 0.220. Is there any fix made to this on the newer versions?

liusztc09 picture liusztc09 on 13 Sep 2019

@liusztc09 There were a few fixes and improvements recently. Most relevant might be #13157 and #13079 . Any chance you could upgrade and run the query on latest?

mbasmanova picture mbasmanova on 13 Sep 2019

Will give it a try and let you know. Thank you!

liusztc09 picture liusztc09 on 13 Sep 2019
👍1

@mbasmanova After moving to 0.225, it throws null pointer error from another method in GeometryUtils

Query 20190915_193641_00014_s8nkw failed: Internal error
java.lang.NullPointerException
        at com.facebook.presto.geospatial.GeometryUtils.getExtent(GeometryUtils.java:107)
        at com.facebook.presto.operator.PagesRTreeIndex$GeometryWithPosition.<init>(PagesRTreeIndex.java:85)
        at com.facebook.presto.operator.PagesSpatialIndexSupplier.buildRTree(PagesSpatialIndexSupplier.java:142)
        at com.facebook.presto.operator.PagesSpatialIndexSupplier.<init>(PagesSpatialIndexSupplier.java:94)
        at com.facebook.presto.operator.PagesIndex.createPagesSpatialIndex(PagesIndex.java:473)
        at com.facebook.presto.operator.SpatialIndexBuilderOperator.finish(SpatialIndexBuilderOperator.java:234)
        at com.facebook.presto.operator.Driver.processInternal(Driver.java:397)
        at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
        at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
        at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
        at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
        at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
        at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)
        at com.facebook.presto.$gen.Presto_0_225_9e57310____20190915_193305_1.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Interestingly, if I call ST_envelope(ST_GeometryFromText(v.wkt)) it runs successfully for both v220 and v225, so I'm pretty sure it's because some of the polygons causing problems.

liusztc09 picture liusztc09 on 15 Sep 2019

@liusztc09 Thank you for this test case. The issue is that we weren't handling GeometryCollections correctly (they require special handling): I'm submitting a fix for it. Until the fix is landed, you can work around it by unnesting your Geometry Collections (or converting them to a homogeneous MultiGeometry), if that's possible.

jagill picture jagill on 16 Sep 2019
👍1

@jagill Thank you for quick fix! Yes, I narrow down my data and it is geometry collections causing that issue. Looking forward to your fix :)

liusztc09 picture liusztc09 on 16 Sep 2019
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

tesseract2048 picture tesseract2048  路  3Comments
dterror-zz picture dterror-zz  路  4Comments
synhershko picture synhershko  路  4Comments
rajeshd3v picture rajeshd3v  路  3Comments
shigechuanqi picture shigechuanqi  路  3Comments