Exist: [BUG] Guest users can create collections hidden from dba users

[joewiz]

Describe the bug

Guest users can create collections. The collections are “available” but completely hidden from dba users.

Expected behavior

Guest users should not be able to create collections. Created collections should always be visible to dba users.

To Reproduce

1. Log into the Java admin client as guest.
2. Create a collection “test” in “/db” (ensuring that this collection doesn’t already exist). You’ll get an error that guest cannot create a collection.
3. Submit a query, xmldb:collection-available("/db/test"), and the query will return true().
4. Refresh the database listing, and there’s no “test” collection.
5. Even weirder, now log in as admin, and run the query, xmldb:store("/db/test", "test.xml", <test/>). The query completes without error. Now refresh the directory contents, and notice that there’s still no "test" collection.

Context (please always complete the following information):

• OS: macOS 10.15.6
• eXist-db version: 5.3.0-SNAPSHOT dd71ec6d727c6963fb23f67bfd55ae371afcde7a 20200610124401
• Java Version: OpenJDK 1.8.0_265 (Zulu 8.48.0.53-CA-macosx build 1.8.0_265-b11)

Additional context

• How is eXist-db installed? Built from source, running via .app
• Any custom changes in e.g. conf.xml? No

[joewiz]

This hit me again in my development and testing of https://github.com/eXist-db/public-repo/pull/53 under the current 5.3.0-SNAPSHOT. Does anyone have an idea about how to attack this bug or what information I could provide that would make it easier to track down?

[joewiz]

To provide some more information:

The error from step 2 (whose associated command in the admin client's console is mkcol "test"):

Failed to invoke method createCollection in class org.exist.xmlrpc.RpcConnection: Permission to write to Collection denied for /db

The only associated entry in exist.log is:

2021-02-23 11:48:33,640 [qtp36647626-431] WARN (TransactionManager.java [close]:421) - Transaction was not committed or aborted, auto aborting!

I can reproduce the bug when running xmldb:create-collection("/db", "test") in eXide. But in this case, exist.log provides much more information:

2021-02-23 11:49:40,434 [qtp36647626-430] ERROR (XMLDBCreateCollection.java [evalWithCollection]:84) - Unable to create new collection test 
org.xmldb.api.base.XMLDBException: Permission to write to Collection denied for /db
    at org.exist.xmldb.function.LocalXmldbFunction.apply(LocalXmldbFunction.java:50) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xmldb.txn.bridge.InTxnLocalCollection.withDb(InTxnLocalCollection.java:59) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xmldb.txn.bridge.InTxnLocalCollectionManagementService.withDb(InTxnLocalCollectionManagementService.java:67) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xmldb.txn.bridge.InTxnLocalCollectionManagementService.createCollection(InTxnLocalCollectionManagementService.java:49) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xmldb.LocalCollectionManagementService.createCollection(LocalCollectionManagementService.java:75) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xmldb.LocalCollectionManagementService.createCollection(LocalCollectionManagementService.java:64) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.createCollection(XMLDBAbstractCollectionManipulator.java:185) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.createCollectionPath(XMLDBAbstractCollectionManipulator.java:195) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.functions.xmldb.XMLDBCreateCollection.evalWithCollection(XMLDBCreateCollection.java:76) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(XMLDBAbstractCollectionManipulator.java:166) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:73) ~[exist-core-5.3.0-SNAPSHOT.jar:5.3.0-SNAPSHOT]
        ...

But as before, the collection that should have been impossible for guest to create has been created—as demonstrated by:

• xmldb:collection-available("/db/test") returns true()
• sm:get-permissions(xs:anyURI("/db/test")) returns:
  <sm:permission xmlns:sm="http://exist-db.org/xquery/securitymanager" owner="guest" group="guest" mode="rwxr-xr-x"> <sm:acl entries="0"/> </sm:permission>
• xmldb:store("/db/test", "test.xml", <foo/>) returns "/db/test/test.xml"
• doc-available("/db/test/test.xml") returns true()
• doc("/db/test/test.xml") returns <foo/>
• xmldb:get-child-resources("/db/test") returns "test.xml"
• xmldb:remove("/db/test") returns error:
  > Cannot remove collection: Account 'guest' is not allowed to remove collection '/db/test' [at line 3, column 1]
• xmldb:remove("/db/test/", "test.xml") successfully deletes the document.

At the same time, the collection is inaccessible to even admin users, as demonstrated by:

• xmldb:get-child-collections("/db") = "test" returns false()

Basically, it appears guests can create hidden collections that are effectively hidden from administrators. Even worse, guest users can upload arbitrary data to the collections they create. Administrators have no way to know the collections or data exists unless they know the collection name.

[joewiz]

As discussed during today's Community Call, @wolfgangmm guesses that an entry is created in the collections.dbx before the permissions are checked.

Just now, I tried to trace down how collections are created and where a permissions check might be misplaced:

• When xmldb:create-collection() is called, createCollectionPath is called here: https://github.com/eXist-db/exist/blob/6094de69e6dfe5885639bf0edd479d7d9e1ca35f/exist-core/src/main/java/org/exist/xquery/functions/xmldb/XMLDBCreateCollection.java#L76
• createCollectionPath in turn calls createCollection, as defined here: https://github.com/eXist-db/exist/blob/6094de69e6dfe5885639bf0edd479d7d9e1ca35f/exist-core/src/main/java/org/exist/xquery/functions/xmldb/XMLDBAbstractCollectionManipulator.java#L181-L198
• createCollection calls mgtService.createCollection, as defined here: https://github.com/eXist-db/exist/blob/6094de69e6dfe5885639bf0edd479d7d9e1ca35f/exist-core/src/main/java/org/exist/xmldb/EXistCollectionManagementService.java#L98-L111

... but not knowing java, I can't tell where/if permissions are checked in this chain of functions.

[dizzzz]

That would be then: https://github.com/eXist-db/exist/blob/ebe22e7b7881e7a081aa2be740cc38211a495dae/exist-core/src/main/java/org/exist/xmldb/LocalCollectionManagementService.java#L46

[joewiz]

@dizzzz Thanks! I think that leads us to createCollection: https://github.com/eXist-db/exist/blob/ebe22e7b7881e7a081aa2be740cc38211a495dae/exist-core/src/main/java/org/exist/xmldb/LocalCollectionManagementService.java#L81-L98

Does it appear to you that the collection is created before permissions are checked? I see references to brokers, collection locks, transactions, and user, but I can't really tell what's going on.

[dizzzz]

No I don't see it (yet)

[line-o]

@dizzzz this is still open right?

[joewiz]

@line-o @dizzzz Correct - see https://github.com/eXist-db/exist/pull/3870#issuecomment-838443515.

[dizzzz]

The problem is that the collection is created and written to disk [with problems as the collections are not visible even for dba] before any actual permission check is performed.

The code is in Nativenbroker#saveCollection

but check the trace, LocalManagement service, line 87:

this is the order:

first get/create collection [that is basically creating all disk changes] is invoked, and only then the (lock) permissions are checked. When the permissions are not OK, the transaction is aborted BUT the changed data on disk remain.