Server: Instant upload from Android app fails

Created on 30 Mar 2020 · 9Comments · Source: nextcloud/server

I took a multiple photos today with my mobile phone and for one photo the instant upload from Android fails repeatingly.

I chose the server repository because the error in the nextcloud.log looks like a server issue:

As you can see I get a file not found exception for /bjoern/files/4e4452a6ee37a7444aaf6649f25293c8/0000000000000000-0000000005494087 which looks strange because the right path would be bjoern/uploads/4e4452a6ee37a7444aaf6649f25293c8/0000000000000000-0000000005494087

I'm running Nextcloud 18.0.3

1. to develop bug dav

Source

schiessle

Most helpful comment

the patch suggested by @kesselb solved the problem for me. After adding the three lines the Android client instantly uploaded the previously failing picture.

schiessle on 31 Mar 2020

🎉3

All 9 comments

The app does a propfind request for https://wolke.xyz.org/remote.php/dav/uploads/bjoern/4e4452a6ee37a7444aaf6649f25293c8 :confused:

Probably not bad to catch that exception ;) Do you know how to reproduce this request?

kesselb on 30 Mar 2020

https://github.com/nextcloud/android-library/blob/d81788b50d32f93d81f458a476edc081df081035/src/main/java/com/owncloud/android/lib/resources/files/ChunkedFileUploadRemoteOperation.java#L90-L93

Make sense. I think the server should ignore {http://nextcloud.org/ns}sharees property for such requests and the android app do not request them for chunks ;)

cc @rullzer @tobiasKaminsky

kesselb on 30 Mar 2020

<?php

define('CHUNK_SIZE', 2000000);

$davUrl = 'https://nextcloud.test/remote.php/dav/';
$data = file_get_contents(__DIR__ . '/4k-wallpaper-background-beautiful-853199.jpg');
$name = 'test-upload-' . random_int(1000, 2000);

$ch = curl_init();

// folder for chunks
curl_setopt($ch, CURLOPT_URL, $davUrl . 'uploads/admin/' . $name);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'MKCOL');
curl_setopt($ch, CURLOPT_USERPWD, 'admin:admin');
curl_exec($ch);

// destination folder we use later
curl_setopt($ch, CURLOPT_URL, $davUrl . 'files/admin/destination-' . $name);
curl_exec($ch);

$chunks = str_split($data, CHUNK_SIZE);
$start = 0;
$end = CHUNK_SIZE;

foreach ($chunks as $key => $chunk) {
    // upload chunks
    curl_setopt($ch, CURLOPT_URL, $davUrl . 'uploads/admin/' . $name . '/' . $start . '-' . $end);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
    curl_setopt($ch, CURLOPT_POSTFIELDS, $chunk);
    curl_exec($ch);
    $start = 1 + $end;
    $end = CHUNK_SIZE + $start;

    curl_setopt($ch, CURLOPT_URL, $davUrl . 'uploads/admin/' . $name . '?XDEBUG_SESSION_START=phpstorm');
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PROPFIND');
    curl_setopt($ch, CURLOPT_POSTFIELDS, '<?xml version="1.0" encoding="utf-8" ?>
  <D:propfind xmlns:D="DAV:">
    <D:prop xmlns:OC="http://owncloud.org/ns" xmlns:NC="http://nextcloud.org/ns">
      <D:displayname/>
      <NC:sharees/>
      <OC:size/>
      <OC:id/>
    </D:prop>
  </D:propfind> ');
    curl_exec($ch);
}

// try to move the chunk and use the folder as destination
curl_setopt($ch, CURLOPT_URL, $davUrl . 'uploads/admin/' . $name . '/.file');
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'MOVE');
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'Destination:' . $davUrl . 'files/admin/' . $name . '.jpg', // that should work
]);
curl_exec($ch);

If someone wants to do more digging. You probably need to adjust XDEBUG_SESSION_START and place a breakpoint apps/dav/lib/Connector/Sabre/SharesPlugin.php:handleGetProperties.


Index: apps/dav/lib/Connector/Sabre/SharesPlugin.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- apps/dav/lib/Connector/Sabre/SharesPlugin.php   (revision f5e81749901ae7a5c6db91bfd61c886b2c644592)
+++ apps/dav/lib/Connector/Sabre/SharesPlugin.php   (date 1585604323570)
@@ -179,6 +179,10 @@
            return;
        }

+       if (strpos($sabreNode->getFileInfo()->getInternalPath(), 'uploads/') === 0) {
+           return;
+       }
+
        // need prefetch ?
        if ($sabreNode instanceof \OCA\DAV\Connector\Sabre\Directory
            && $propFind->getDepth() !== 0

If the chunks are always uploaded to uploads/ that should work. At least i'm not able to reproduce it anymore.

kesselb on 30 Mar 2020

👍3

the patch suggested by @kesselb solved the problem for me. After adding the three lines the Android client instantly uploaded the previously failing picture.

schiessle on 31 Mar 2020

🎉3

Same here.
Maybe a patch should be pushed to android client too.

ymage on 31 Mar 2020

Make sense. I think the server should ignore {http://nextcloud.org/ns}sharees property for such requests and the android app do not request them for chunks ;)

While it is true that Android should not ask for this on chunks, according to dav protocol, it is allowed to ask for any property and it should never fail, but show a proper response:

<d:propstat>
      <d:prop>
        <d:displayname/>
        <d:creationdate/>
        <nc:share-expiration/>
        <nc:rich-workspace/>
        <nc:X-Hash-MD5/>
        <d:getcontenttype/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>

tobiasKaminsky on 2 Apr 2020

I have the same issue but it appears to only happen with large video files when recorded with my camera. Fix suggested above allowed them to upload as well Thanks.

phantomdez on 8 Apr 2020

Thanks for confirmation.
On Android I changed it (https://github.com/nextcloud/android-library/pull/418) and it will be in next version.

@kesselb can you maybe create a PR for server? :heart:

tobiasKaminsky on 9 Apr 2020

Ideally we add that check for every property not available for chunks. I'm not sure yet but it might be easier to add that check a bit earlier and unset the properties not available for chunks.

kesselb on 9 Apr 2020

Was this page helpful?

0 / 5 - 0 ratings