Exist: Path expression with nested positional predicate fails on database node

Created on 30 May 2018 · 11Comments · Source: eXist-db/exist

What is the problem

A path expression containing a nested positional predicate, which works on an in-memory node, fails on an identical node stored in the database.

What did you expect

I expected a path expression on a structure to work uniformly whether it is in-memory or stored in the database.

Describe how to reproduce or add a test

The following xqsuite test, adapted from @ChristianGruen's https://github.com/BaseXdb/basex/issues/1573, shows how the in-database function fails while the in-memory function works correctly.

xquery version "3.1";

module namespace npt="http://exist-db.org/test/nested-positional-predicate";

declare namespace test="http://exist-db.org/xquery/xqsuite";

declare variable $npt:DATA :=
    document {
        <xml>
            <a>
                <b>A</b>
            </a>
            <a>
                <b>B</b>
                <c>correct</c>
            </a>
            <a>
                <b>B</b>
                <c>wrong</c>
            </a>
        </xml>
    };

declare
    %test:setUp
function npt:setup() {
    xmldb:create-collection("/db", "test"),
    xmldb:store("/db/test", "test.xml", $npt:DATA)
};

declare
    %test:tearDown
function npt:cleanup() {
    xmldb:remove("/db/test")
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-memory() {
    $npt:DATA//c[../preceding-sibling::a[1]/b = 'A']
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-database() {
    doc("/db/test.xml")//c[../preceding-sibling::a[1]/b = 'A']
};

The results:

<testsuites>
    <testsuite package="http://exist-db.org/test/nested-positional-predicate"
        timestamp="2018-05-29T23:01:23.898-05:00" failures="1" pending="0" tests="2" time="PT0.029S">
        <testcase name="in-database" class="npt:in-database">
            <failure
                message="assertEquals failed: wrong number of items returned by function. Expected: 1. Got: 0"
                type="failure-error-code-1">&lt;c&gt;correct&lt;/c&gt;</failure>
            <output/>
        </testcase>
        <testcase name="in-memory" class="npt:in-memory"/>
    </testsuite>
</testsuites>

Context information

eXist-db version + Git Revision hash: eXist-db 4.2.0-SNAPSHOT+201805281153 (e46c45c9d)
Java version: Oracle JDK 1.8.0_172-b11
Operating system: macOS 10.13.4
32 or 64 bit: 64 bit
Any custom changes in e.g. conf.xml: none

bug

Source

joewiz

Most helpful comment

I think I might have fixed this here: https://github.com/adamretter/exist/commit/00ec2d7e48e5cb517349bc7ca69e56144f70d6d5 there will be a Pull Request soon I hope.

adamretter on 12 Aug 2018

❤1 👍1

All 11 comments

@joewiz just for reference in case we do use saxon under the hood here: the first one seems relevant because of its use of [1]
https://saxonica.plan.io/issues/3758
https://saxonica.plan.io/issues/3666

duncdrum on 30 May 2018

I think I might have fixed this here: https://github.com/adamretter/exist/commit/00ec2d7e48e5cb517349bc7ca69e56144f70d6d5 there will be a Pull Request soon I hope.

adamretter on 12 Aug 2018

❤1 👍1

@joewiz Okay my PR which might help with this is here - https://github.com/eXist-db/exist/pull/2113 please can you test?

adamretter on 13 Aug 2018

@adamretter The error still appears to be present. Checking out #2113 and running the xqsuite test above, the result is:

<testsuites>
    <testsuite package="http://exist-db.org/test/nested-positional-predicate"
        timestamp="2018-08-13T09:35:08.599-04:00" tests="2" failures="1" errors="0" pending="0"
        time="PT0.025S">
        <testcase name="in-database" class="npt:in-database">
            <failure
                message="assertEquals failed: wrong number of items returned by function. Expected: 1. Got: 0"
                type="failure-error-code-1">&lt;c&gt;correct&lt;/c&gt;</failure>
            <output/>
        </testcase>
        <testcase name="in-memory" class="npt:in-memory"/>
    </testsuite>
</testsuites>

joewiz on 13 Aug 2018

I was able to reduce your test case further to this:

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database-predicate() {
    doc("/db/test.xml")//c[../preceding-sibling::a[1]]
};

Interestingly, if I use the position() function in the predicate instead of just a numeric position then the test passes, e.g.

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database-position() {
    doc("/db/test.xml")//c[../preceding-sibling::a[position() eq 1]]
};

adamretter on 14 Aug 2018

Actually I think we can reduce it further even to just:

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database-predicate() {
    doc("/db/test.xml")//c[../preceding-sibling::a]
};

adamretter on 14 Aug 2018

I am really struggling to find where the problem is here, I think it is somehow related to predicate evaluation, perhaps @wolfgangmm has some thoughts?

adamretter on 14 Aug 2018

I have these test cases now:

xquery version "3.1";

module namespace npt="http://exist-db.org/test/nested-positional-predicate";

declare namespace test="http://exist-db.org/xquery/xqsuite";

declare variable $npt:DATA :=
    document {
        <xml>
            <a>
                <b>B1</b>
            </a>
            <a>
                <b>B2</b>
                <c>correct</c>
            </a>
            <a>
                <b>B3</b>
                <c>wrong</c>
            </a>
        </xml>
    };

declare
    %test:setUp
function npt:setup() {
    xmldb:create-collection("/db", "test"),
    xmldb:store("/db/test", "test.xml", $npt:DATA)
};

declare
    %test:tearDown
function npt:cleanup() {
    xmldb:remove("/db/test")
};

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-memory() {
    $npt:DATA//c[../preceding-sibling::a]
};

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database() {
    doc("/db/test.xml")//c[../preceding-sibling::a]
};


declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-memory-predicate() {
    $npt:DATA//c[../preceding-sibling::a[1]]
};

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database-predicate() {
    doc("/db/test.xml")//c[../preceding-sibling::a[1]]
};

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-memory-position() {
    $npt:DATA//c[../preceding-sibling::a[position() eq 1]]
};

declare
    %test:assertEquals("<c>correct</c><c>wrong</c>")
function npt:in-database-position() {
    doc("/db/test.xml")//c[../preceding-sibling::a[position() eq 1]]
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-memory-predicate-and-path() {
    $npt:DATA//c[../preceding-sibling::a[1]/b = 'B1']
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-database-predicate-and-path() {
    doc("/db/test.xml")//c[../preceding-sibling::a[1]/b = 'B1']
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-memory-position-and-path() {
    $npt:DATA//c[../preceding-sibling::a[position() eq 1]/b = 'B1']
};

declare
    %test:assertEquals("<c>correct</c>")
function npt:in-database-position-and-path() {
    doc("/db/test.xml")//c[../preceding-sibling::a[position() eq 1]/b = 'B1']
};

adamretter on 14 Aug 2018

see #798

duncdrum on 12 Jan 2019

also note that:
$coll//a[1] is fast, but $coll//b/a[1] is VERY slow (talking about db nodes here, not sure about in-memory).
a[position()=1] always seems to perform acceptably. While this is a feasible workaround, positional predicates seem to need more attention in general.