Server: webDAV PROPFIND

Created on 1 Aug 2017  路  29Comments  路  Source: nextcloud/server

If I send a request PROPFIND with Depth "infinity" e.g.

<D:propfind xmlns:D="DAV:">
    <D:prop>
        <D:resourcetype />
        <D:getlastmodified />
        <size xmlns="http://owncloud.org/ns" />
        <favorite xmlns="http://owncloud.org/ns" />
        <id xmlns="http://owncloud.org/ns" />
        <D:getcontentlength />
        <D:getetag />
        <permissions xmlns="http://owncloud.org/ns" />
        <D:getcontenttype />
    </D:prop>
</D:propfind>

the Response has all record as collection:

<d:resourcetype>
    <d:collection />

Possible ?
@MorrisJobke can you try ?

thx

bug help wanted

All 29 comments

I guess we will not support this, because it means to basically dump all of the users file tree, which can be really big.

@marinofaggiana Why do you need this? What is the use case for this?

If you want to detect changes, have a look at the etag. It is described in here for example: https://dragotin.wordpress.com/2015/03/13/owncloud-etags-and-fileids/

oh no ! I use already the etag for detect the changes but for the first sync or download of all files the "infinity" is more efficient of several recursive request in all branches

mmm can I try with search "infinity" .... but I don't know if this is efficient

Cc @icewind1991 for the server

@tobiasKaminsky @AndyScherzinger how is this done in android?

Initial we do it recursive and later we only check the entered folder.
But we discussed to drop the initial scan and use the search instead to get all favorited items.

Just do it recursive. Infinity is not a good idea.

Even the desktop client (where the clients have significant more processing power) doesn't do this.

The problem here is that since owncloud 9.1 the server behaves wrong. With depth: infinity (Which is an almost all the cases at least 10 times faster then the 1000 requests variant) the properties of the children are wrong (ETags....).
My suggestion: if you do not like to support it then you can return a 403 as discussed in section 9.9.1. in rfc4918 or this should be fixed, since what it now returns is just wrong information.

Just do it recursive. Infinity is not a good idea.

the recursive for request on network is a bad, several useless traffic and several useless cycles of CPU

@marinofaggiana well your choice. But I guess a lot of users also won't be happy if you start parsing a >50mb xml file.

Also, can you give me concrete reproduction steps of what is broken? Here the on files I don't see the collection set.

@julian-r same goes for you. Do you have concrete steps that show the issue? Etag is nicely updated here...

@rullzer e.g.

1) for the first sync the App parse 5+5+5+5...... = 50 mb xml (recursive)
2) infinity 50 mb xml ...

For my app it's the same ... the app is blocked but the traffic it's less and I don't have several task on memory ... and I don't have a cycle of CPU for managed 10 ... 15 request on network for read the subfolder

@rullzer

  1. take owncloud > 9.1 or nextcloud > 11 (I am using docker images with mysql - ask me for the compose file if you need it) complete fresh install
  2. curl --header "Depth: infinity" --user testuser:testpass -X PROPFIND http://nextcloud/remote.php/webdav | xmllint --format -
    (xmllint is from xmllib-utils) returns:
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
  <d:response>
    <d:href>/remote.php/webdav/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>6823068</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99fa1b4e"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>78496</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99fa1b4e"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/About.odt</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>77422</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>78496</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99fa1b4e"</d:getetag>
        <d:getcontenttype>application/vnd.oasis.opendocument.text</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/About.txt</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>1074</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>78496</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99fa1b4e"</d:getetag>
        <d:getcontenttype>text/plain</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Nextcloud%20Manual.pdf</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>3922148</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"c9e79010231abd65224aa021f0f6ca93"</d:getetag>
        <d:getcontenttype>application/pdf</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Nextcloud.mp4</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>462413</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"906b84092187548715f21becbfffe8f1"</d:getetag>
        <d:getcontenttype>video/mp4</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99f497b5"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Coast.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>819766</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99f497b5"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Hummingbird.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>585219</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99f497b5"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Nut.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>955026</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99f497b5"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

  • The E-Tag "598ae99f497b5" repeats over and over again, also the <d:collection/> is wrong for Hummingbird.jpg or Nut.jpg.
  • If you do the same with a subdirectory curl --user testuser:testpass -X PROPFIND http://nextcloud/remote.php/webdav | xmllint --format -, it seems to be fine.

Also a request to the root without the infinity header works, but not recursively in the rfc it states it SHOULD be handled like with recursion. (Yeah not MUST)

rfc4918 9.9. :

Servers SHOULD treat a request without a Depth header as if a "Depth: infinity" header was
included.

@julian-r

  • The Etag repeats because you of the way we propegate etags up on folders. A folders etag is determined by all the etags in the folder. That it is the same for 2 folders is not an issue as long as it changes

The infinity header. Yeah 'should'. But this is disabled by default in sabre (the dav library we use). Since it can very easily bring down servers.

I can't reproduce the resourcetype on files. Did you also check on nc 12?

I can't reproduce the resourcetype on files. Did you also check on nc 12?

@rullzer read my first post, this is a bug

The Etag repeats because you of the way we propegate etags up on folders. A folders etag is determined by all the etags in the folder. That it is the same for 2 folders is not an issue as long as it changes

Also the same for files? Hummingird.jpg and all the other files have the same E-Tag as their corresponding parents.

Nut.jpg from curl --header "Depth: infinity" --user testuser:testpass -X PROPFIND http://nextcloud/remote.php/webdav

  <d:response>
    <d:href>/remote.php/webdav/Photos/Nut.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>955026</d:getcontentlength>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598ae99f497b5"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>

Nut.jpg from curl --user testuser:testpass -X PROPFIND http://nextcloud/remote.php/webdav/Photos

 <d:response>
    <d:href>/remote.php/webdav/Photos/Nut.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 10:53:19 GMT</d:getlastmodified>
        <d:getcontentlength>955026</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"8f5dc29a51ade95b9e3fc3f7f06d9e48"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>

If you compare them and check d:getetag, d:resourcetype. Also the quota stuff probably are not belonging there.

I can't reproduce the resourcetype on files. Did you also check on nc 12?

  • see first post
  • It is working on owncloud 9 and 9.1
  • Broken in oc 10 nc 11 and 12
    (checked with a automated test-suite)

Please get me your docker compose file as this is working as intended on my test setup

ok your webdav response indeed looks wrong... but I just launched your docker compose...

<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
  <d:response>
    <d:href>/remote.php/webdav/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>6958791</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598b6ec42d49c"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>78496</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598b6ec410270"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:getcontentlength/>
        <d:getcontenttype/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/About.odt</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:getcontentlength>77422</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"c28d32a95c7296ee5d256f6b50c595fe"</d:getetag>
        <d:getcontenttype>application/vnd.oasis.opendocument.text</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/About.txt</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:getcontentlength>1074</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"42c314731b3d3c93c0b8a8306633444f"</d:getetag>
        <d:getcontenttype>text/plain</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Nextcloud%20Manual.pdf</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:getcontentlength>4057871</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"fca2181dceca2a1e15113dc3c9e13652"</d:getetag>
        <d:getcontenttype>application/pdf</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Nextcloud.mp4</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:24 GMT</d:getlastmodified>
        <d:getcontentlength>462413</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"f4100645143f5f574cad57c638ce151e"</d:getetag>
        <d:getcontenttype>video/mp4</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:23 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>2360011</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"598b6ec3dfcaf"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:getcontentlength/>
        <d:getcontenttype/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Coast.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:23 GMT</d:getlastmodified>
        <d:getcontentlength>819766</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"a147c0ecebf65ec566bd09ab0f230cfc"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Hummingbird.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:23 GMT</d:getlastmodified>
        <d:getcontentlength>585219</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"6902a39565cedfc24459fc701f850af7"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/Nut.jpg</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Wed, 09 Aug 2017 20:21:23 GMT</d:getlastmodified>
        <d:getcontentlength>955026</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"55dafbce5e283451c330c9544919f629"</d:getetag>
        <d:getcontenttype>image/jpeg</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

That is mine on the nextcloud:12 image

    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>

WebDAV is horrible ...

also the 11 images works properly for me... and return valid stuff...

mmm for me no.... always

<d:resourcetype>
    <d:collection />

@rullzer The response you sent, had a payload in the requests, right? I always sent the requests without payload.

@rullzer Are you the maintainer for webdav in nc?
It seems it has been fixed between 12.0.0 and 12.0.1, is this a fix in nc or in sabre?

@julian-r No payload.

Aaaah I did test with 12.0.1 I must admit.
I think we fixed something then because we did not upgrade sabre 12.0.0 => 12.0.1

@julian-r I have been known to work on the dav part yes ;)

@marinofaggiana please upgrade your test instance as it should be fixed there.

Yes, 12.0.1 works @julian-r

<d:response>
        <d:href>/remote.php/webdav/Photos/2015/09/2015-09-19%2018-38-02%208819.jpg</d:href>
        <d:propstat>
            <d:prop>
                <d:resourcetype />
                <d:getlastmodified>Fri, 18 Dec 2015 13:21:54 GMT</d:getlastmodified>
                <oc:size>2104022</oc:size>
                <oc:favorite>0</oc:favorite>
                <oc:id>00002141ocr6ct4odsi8</oc:id>
                <d:getcontentlength>2104022</d:getcontentlength>
                <d:getetag>&quot;08a093aa7a98de104bc1b360946f3b08&quot;</d:getetag>
                <oc:permissions>RDNVW</oc:permissions>
                <d:getcontenttype>image/jpeg</d:getcontenttype>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>

and (e.g. 1200 records is very speed) ... as I imagined, recursive is very bad ... now the issue is for pervious version .. :(

Hello everybody. How can I to get XML response without 'd:' (or any) prefix?

Was this page helpful?
0 / 5 - 0 ratings