Server: Chunked upload not working (Files larger than 10MB)

[grahamsk]

Hi,

I am having trouble with chunked uploads in nextcloud 13. It doesn't seem to work from web ui & sync client.
When trying to upload I get the error message that .file doesn't exist. (See below)

If I change the sync client chunkSize setting on my mac os client then I am able to upload files by effectively disabling chunking when setting chunk size to something larger than the file to be uploaded.

Also, webdav using native OSX Sierra works correctly, the issue is with web ui upload and nextcloud client uploads larger than 10MB.

I also upgraded nextcloud to version 13 from 12 a few weeks ago using the webui.

I should also not that my php settings are correct and in line with https://docs.nextcloud.com/server/13/admin_manual/configuration_files/big_file_upload_configuration.html. I have confirmed this using a phpinfo page on the installation.

Could someone please help me to try and troubleshoot this?

Relevant details:

Nextcloud version _(eg, 12.0.2)_: 13.0.0.14
Operating system and version _(eg, Ubuntu 17.04)_: Ubuntu 16.04
Apache or nginx version _(eg, Apache 2.4.25)_: Apache/2.4.18
PHP version _(eg, 7.1)_: 7.0.25

The issue you are facing:

Chunked uploading fails with the following error message:

file-upload.js:274 MOVE https://{url}/remote.php/dav/uploads/{user}/web-file-upload-41fb9a29f46486d4f8d24f9ff0318bab-1520776283884/.file 404 (Not Found)

I've confirmed that if I configure the client (OSX Sierra) with larger chunkSettings than the file to upload then uploads work correctly.

Is this the first time you've seen this error? _(Y/N)_: Y

Steps to replicate it:

1. Upload file larger than 10Mb
2. Check log or developer tools in browser for error message
3.

The output of your Nextcloud log in Admin > Logging:

rewinddir(): 76 is not a valid Directory resource at /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#31

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'debug' => 'true',
  'instanceid' => {instanceid}
  'passwordsalt' => {salt}
  'secret' => {secret}
  'trusted_domains' =>
  array (
    0 => {domain}
  ),
  'datadirectory' => '/mnt/datadirectory',
  'overwrite.cli.url' => {overwriteurl}
  'dbtype' => 'mysql',
  'version' => '13.0.0.14',
  'dbname' => {dbname}
  'dbhost' => {dbhost}
  'dbport' => '', 
  'dbtableprefix' => {prefix}
  'dbuser' => {user}
  'dbpassword' => {password}
  'logtimezone' => 'UTC',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 0,
);

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

[pvanthony]

I am having the same issue.
Nextcloud version 13.0.2
Nginx
php 7.0
Used firefox and chrome.
My desktop client is on Fedora 27

[pvanthony]

The following work around seems to be working. I hope an official fix will come soon.
https://help.nextcloud.com/t/nextcloud-13-chunking-upload-not-working/28755/17

[nextcloud-bot]

Hey, this issue has been closed because the label stale is set and there were no updates for 14 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

[AndrewErman]

Any news here? I an encountering the same error...

[Claymore90]

Same here

[Olli]

Same here - Workaround doesn't work.

[fizux]

Ditto. Using the docker image.

[AndrewErman]

Hm... Is anybody monitoring this issue except the bot? It seems to be quite critical...

[jospoortvliet]

@icewind1991 happens in your lib so I suppose it's one for you?

[Sapphireb1ue]

I encountered the same problem.

[andreaspollak]

same problem.

[alo-is]

Same issue with 14.0.1.1

[mleu77]

Same problem with 14.0.1.1

[alo-is]

Just found the root cause of my issue, which was related to the Nginx configuration, and not Nextcloud : debops/ansible-owncloud#87 and debops/debops#469.

[timm2k]

Hm, this clause isn't in the recommended example config for running Nextcloud with nginx. But it makes absolutely sense to the initiating issue.

https://docs.nextcloud.com/server/13/admin_manual/installation/nginx.html
https://docs.nextcloud.com/server/14/admin_manual/installation/nginx.html

[andreaspollak]

my problem is gone, since changes in vhost config (using ispconfig 3.1 standard template for nginx)

from this (vhost config)
location ~ /\. { deny all; }

lines from access.log:
xx.xx.xx.xx - USERNAME [19/Oct/2018:16:50:47 +0200] "MKCOL /remote.php/dav/uploads/USERNAME/3784624209 HTTP/1.1" 201 0 "-" "Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)" xx.xx.xx.xx - USERNAME [19/Oct/2018:16:50:51 +0200] "PUT /remote.php/dav/uploads/USERNAME/3784624209/00000000 HTTP/1.1" 201 0 "-" "Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)" xx.xx.xx.xx - USERNAME [19/Oct/2018:16:50:54 +0200] "PUT /remote.php/dav/uploads/USERNAME/3784624209/00000001 HTTP/1.1" 201 0 "-" "Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)" xx.xx.xx.xx - USERNAME [19/Oct/2018:16:50:57 +0200] "PUT /remote.php/dav/uploads/USERNAME/3784624209/00000002 HTTP/1.1" 201 0 "-" "Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)" xx.xx.xx.xx - USERNAME [19/Oct/2018:16:50:57 +0200] "MOVE /remote.php/dav/uploads/USERNAME/3784624209/.file HTTP/1.1" 500 186 "-" "Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)"

to this (vhost config)
location ~ /\.(?!file).* { deny all; }

upload with desktop client is working fine now!

[MorrisJobke]

my problem is gone, since changes

Where are those statements coming from?

[blauark]

Same problem.

[jospoortvliet]

@blauark pls don't just add a me-too, unless you have any info that can help us track down the problem - right now we simply can't reproduce it, things work fine on our test setup so this isn't a valid, useful issue until it can be reliably reproduced. It seems all due to bad configuration on the server, eg NGINX settings and so on. I actually think we should close this as a 'not a Nextcloud issue' unless someone has proof to the contrary. This isn't a support forum for web server configuration issues...

[MorrisJobke]

Closing as @jospoortvliet said.

[Zocker1999NET]

@jospoortvliet You are right that this is not an issue of Nextcloud but I would propose to add this to the installation manual cause it is recommended to block dot files when using nginx on root directories with .git or similar directories. I got the same error today and it required too much time for me to find this article so just a little hint in the manual would have saved a few hours of mine and it could save hours and frustration of others in the future.

[karl-in-office]

@Zocker1999NET
I agree with you.

@jospoortvliet @MorrisJobke
I would appreciate If the installation manual clearly state that front-end server should allow accesses to ".file".
(example NGINX and Apache configuration also)

[jospoortvliet]

@Zocker1999NET @karl-in-office that does make sense. Any chance I can persuade one of you to propose such a note about this in the docs? If you go to the docs about nginx there is a neat edit on github button that should make it easy to propose an improvement ;-)

[karl-in-office]

@jospoortvliet
I would like to contribute the docs to Nextcloud, but it'll be difficult for me to do forking and PR due to company regulations :-(
I apologize for that.

I'll leave related issues which might be the same root cause for future reference.
Related issues that I found are: #7999, #17005
Related PR is: /pull/1283

[jospoortvliet]

Many thanks to @Zocker1999NET for the PR ;-)

[BeWog]

• For information, I had this message in Nginx error log:

2020/05/01 10:50:09 [error] 8676#8676: *2 access forbidden by rule, client: 192.168.1.254, server: nextcloud.mydomain, request: "MOVE /remote.php/dav/uploads/myuser/2254808276/.file HTTP/1.1", host: "nextcloud.mydomain"

• And in Nextcloud application for Windows:

Server replied 403 Forbidden to MOVE ...

The problem came from this tool : https://www.digitalocean.com/community/tools/nginx
/etc/nginx/nginxconfig.io/security.conf was the problem.

Fixed with

location ~ /\.(?!well-known)(?!file) {
        deny all;
}

[Aguay-val]

• For information, I had this message in Nginx error log:

2020/05/01 10:50:09 [error] 8676#8676: *2 access forbidden by rule, client: 192.168.1.254, server: nextcloud.mydomain, request: "MOVE /remote.php/dav/uploads/myuser/2254808276/.file HTTP/1.1", host: "nextcloud.mydomain"

• And in Nextcloud application for Windows:

Server replied 403 Forbidden to MOVE ...

The problem came from this tool : https://www.digitalocean.com/community/tools/nginx
/etc/nginx/nginxconfig.io/security.conf was the problem.

Fixed with

location ~ /\.(?!well-known)(?!file) {
        deny all;
}

Solved my problem.
Thank you :+1: !