Exist: [BUG] NPE at org.exist.xquery.modules.range.RangeQueryRewriter.rewriteLocationStep

[joewiz]

Describe the bug

In eXist develop HEAD (5.3.0-SNAPSHOT), a new NPE is raised when running a query. The same query did not raise an NPE in the current stable release, 5.2.0.

Expected behavior

The query should not raise an NPE.

To Reproduce

To reproduce, install https://github.com/HistoryAtState/release (here's a prebuilt xar: release-0.5.0.xar.zip - just remove the .zip extension), and open http://localhost:8080/exist/apps/release/ebook-batch.xq. The client sees the following 500 error:

HTTP ERROR 500 javax.servlet.ServletException: javax.servlet.ServletException: An error occurred while processing request to /exist/apps/release/ebook-batch.xq: An error occurred: null

Attached is the corresponding error from exist.log: exist-log-npe-rangequeryrewriter.txt

An excerpt:

2020-11-20 17:13:58,622 [qtp558431044-99] ERROR (EXistServlet.java [doGet]:291) - null 
java.lang.NullPointerException: null
    at org.exist.xquery.modules.range.RangeQueryRewriter.rewriteLocationStep(RangeQueryRewriter.java:109) ~[exist-index-range-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.Optimizer.visitLocationStep(Optimizer.java:83) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.LocationStep.accept(LocationStep.java:1242) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.DynamicCardinalityCheck.accept(DynamicCardinalityCheck.java:139) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.DefaultExpressionVisitor.visitBuiltinFunction(DefaultExpressionVisitor.java:48) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.Function.accept(Function.java:526) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.InternalFunctionCall.accept(InternalFunctionCall.java:271) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]

Two clues:

1. Performing the same steps from eXist 5.2.0 (stable) does not yield the error, but instead shows the expected HTML webpage. This suggests a regression in eXist 5.3.0-SNAPSHOT.

2. Also, commenting out https://github.com/HistoryAtState/release/blob/master/ebook-batch.xq#L198 allows eXist 5.3.0-SNAPSHOT to sidestep the error, so something in that code path must be triggering the NPE. I wasn't able to trace the source in my code, but I don't believe my code should be able to cause an NPE.

Context (please always complete the following information):

• OS: macOS 11.0.1
• eXist-db version: eXist 5.3.0-SNAPSHOT f65967548a9cc5886360cac80bc5cb1959d561be 20201119071106
• Java Version: OpenJDK 1.8.0_275 (Liberica full)

Additional context

• How is eXist-db installed? built from source
• Any custom changes in e.g. conf.xml? none

[adamretter]

That's very interesting! So this happens when trying to rewrite the query to replace a Location Step with a function which performs a Range Index comparison.

So the only way you could have null there is if the Inner Expression is not either:

1. A General Comparison, e.g. = !=, >=, eq, ne, ge, etc...
2. The functions matches.

The code is problematic, as it returns null in those cases, but then the code calling it does not expect/handle null and so you get an NPE.

We could easily fix the NPE itself... But, as you describe not seeing this issue before, this however is possibly not the root cause. We would need to know what expression is causing the NPE to determine the root cause.

The first step would probably be to add some trace level logging into Optimizer#visitLocationStep(LocationStep) so that we can see what it is attempting to optimize.

[joewiz]

@adamretter Thanks for your analysis and suggestion about how to proceed with determining the root cause. I'll let you know what we find.

[dizzzz]

So my PR is not ok @adamretter @wolfgangmm ? shall I withdraw it ? :-(