Android: Android fails to upload bigfile when chunk assembly take more than 900 seconds

[nesies]

Actual behaviour

The Android App upload a bigfile, it fails when upload seem to be complete.
When the last chunk is uploaded, Owncloud Server assemble all chunks and the process can take a long time.
If it takes more than 60 seconds, the Android App abort and notify:
"An error occurred whilst waiting for the server, the operation couldn't be done"

Owncloud Server is served by nginx. A HTTP Code 499 means that the http client closed the socket.

Expected behaviour

The Android App should wait for the chunk assembly completion.

Steps to reproduce

1. With Android App, upload a big file, assembling all chunk must take more than 60 seconds
2. wait
3. Android App notify a socket error

Can this problem be reproduced with the official owncloud server?
(url: https://demo.owncloud.org, user: test, password: test)

Environment data

Android version: 6.1.0

Device model: Sony Xperia Z5

Stock or customized system: stock

ownCloud app version: 2.1.0

ownCloud server version: 9.0.4

Logs

Web server error log

1.2.3.4 - user [15/Aug/2016:22:28:47 +0200] "PUT /remote.php/webdav/upload/bigfile-chunking-7954-268-266 HTTP/1.1" 201 1065 "-" "Mozilla/5.0 (Android) ownCloud-android/2.1.0" "-" msec="1471292927.367" request_length="1024400" request_time="**2.365**" "connection_requests=70"
1.2.3.4 - user[15/Aug/2016:22:29:47 +0200] "PUT /remote.php/webdav/upload/bigfile-chunking-7954-268-267 HTTP/1.1" **499** 0 "-" "Mozilla/5.0 (Android) ownCloud-android/2.1.0" "-" msec="1471292987.668" request_length="114668" request_time="**60.221**" "connection_requests=71"

edited by @jabarros

• [x] Create 'upload_bigfiles' branch from master in android library repository @jabarros
• [x] Merge 'upload_bigfiles' branch into master
• [x] Add a high timeout to the last chunk when uploading them ( ChunkedUploadRemoteFileOperation.java file) @jabarros
• [x] Create test plan @jesmrec
• [X] Validate test plan @jesmrec

EDITED by @dvelasco
Let's go further:

• [ ] Make upload tolerant to any timeout while chunks are being put together in the server, considering the success as pending to be confirmed in a future refresh of the parent folder.

For the record: current timeout for recomposing files and fail is 15 minutes.

[davivel]

Oooh, the PUT of the last chunk is taking more than 60 seconds.

@PVince81 , @guruz , need help from some of you. In the "traditional" chunked uploads, is the file put together in the server before sending the response to the last PUT?

[PVince81]

No. In the old chunking, every PUT will simply store the chunk as is in a cache folder.
Then when the last chunk arrives with the last PUT, all chunks get assembled within that PHP request. This is why the last one takes longer.

[davivel]

Thanks, that's what I meant.

So, maybe we can make the trick here increasing the timeout for the last PUT in a chunked upload.

[davivel]

@jesmrec , I'm adding this to the current milestone since it's a really small change that can make a big difference.

[nesies]

I try to upload a 4GB file and my server can write at 5MBps, so i need a timeout geater than 800seconds :)

[mwaeckerlin]

Possibly this is the same problem as I described in #1737.

[jabarros]

Fixed increasing the last chunk timeout uploaded
@jesmrec, it is available on this PR

[davivel]

Seems OK, though we'll need also a PR in the app to sync the reference to the library.

[jesmrec]

Checked with files of several sizes. The higher one, greater to 2GB.

In any case, here we have a performance dependency. The final assembly will depend on how fast works the server's hardware.

Approved.

[davivel]

Hi, @seisen . Could you try the test APK in the link below to see if the fix works for you? It will appear in your device as a separate ownCloudTest app, not conflicting with current ownCloud app.

https://github.com/owncloud/android/raw/upload_bigfiles/ownCloudTest.apk

[nesies]

@davivel i tried with a 800MB file and it worked, thank you

[mwaeckerlin]

It works with small files, it still fails with larges files, 30min of HD-Video, about 3GB.

A timeout, like any other arbitrary limit, is never a solution, there is always a bigger file or slower server.

Rejected. :-1:

[mwaeckerlin]

How can I reopen the ticket?

[davivel]

@mwaeckerlin , reopened.

Added a task in the first comment to go further. Let's make this really robust.

[mwaeckerlin]

@davivel: Thank you. Currently I have 8 Videos that I cannot upload. It is a big impediment for me and my automated backup concept.

Idea 1: Couldn't the assembly of the first two chunks be started, while the upload of the third chunk is running?

Idea 2: Stability through asynchronous communication: E.g. App and server get a transaction number which is kept for a long time, e.g. a month. They can communicate about the status of the upload and the next steps to execute. Whenever the communication breaks (which is often the case on a mobile), they can resume at the exact byte they were left off. Assembly can be done offline. At the end, when assembly is done, the server sends a success notification to the client and the transaction is finally closed. In such a protocol, in case of errors, the server could even be able to ask for some bytes to be sent again. - This way, it would be highly robust.

As a user, it is important to me, to have my recordings backed up on the server as soon as possible.

[mwaeckerlin]

Oh yeah, and the «transaction number» could be the checksum of the file, so the server can verify if the transmission was successful or automatically initiate a re-transfer.

[michaelstingl]

In version 2.9, the ownCloud Android app switched to a new networking library, and chunked upload needs to be implemented from scratch. Too many things will change, we can't proceed in this issue.