Server: Sabre\DAV\Exception\BadRequest on many files error NC19 Client 2.6.4Stable
I've got folder with about 31GB (116 633 files and 29 052 folders). I selected it to sync with folder inside Nextcloud app.
On maaaany files there is Sabre\DAV\Exception\BadRequest error. Files with different size, and different types.
There are also other problems i have that i think are not very problematic so i will give them on the end of this report.
Proxy in client is disabled. Connection between PC and Nextcloud box is 1Gb ethernet on both sides. The PC (win10 with desktop client) got 64GB RAM with i7 9gen CPU, the Nextcloud (CentOS 8) box is 8GB RAM, with J4105 CPU, and two HDD's in softraid (raid 1).
Tried so far without success: Disable SELinux, PHP downgrade to 7.3 (REMI's repo), MariaDB downgrade from 10.4.13 (MariaDB repo) to older (from CentOS official repo).
Change Linux distro into CentOS 7, with PHP7.3, nginx 1.16.1, older database and still no success!
Removing redis from config is some kind of solution but it slowdowns everything a lot.
- All test above under fresh install.
My specific PHP/MariaDB configs that may have something to do.
/etc/php.ini:
memory_limit = 1024M
max_execution_time = 60
max_input_time = 120
post_max_size = 16384M
upload_max_filesize = 16384M
default_socket_timeout = 120
mysqli.cache_size = 2000
/etc/php-fpm.d/www.conf
pm.start_servers = 10
pm.max_children = 100
/etc/my.cnf.d/server.cnf
[server]
skip-name-resolve
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 32M
innodb_max_dirty_pages_pct = 90
query_cache_type = 1
query_cache_limit = 2M
query_cache_min_res_unit = 2k
query_cache_size = 64M
tmp_table_size= 64M
max_heap_table_size= 64M
slow-query-log = 1
slow-query-log-file = /var/log/mysql/slow.log
long_query_time = 1
[mysqld]
innodb_buffer_pool_size=1G
innodb_io_capacity=4000
Steps to reproduce
- Use desktop client for syncing from HDD to Nextcloud.
- Find errors in activity inside desktop (Windows x64 app) and inside Nextcloud thru website inside Settings \ Logging.
Expected behaviour
Find no errors while uploading files.
Actual behaviour
Getting Sabre\DAV\Exception\BadRequest errors.
Sabre\DAV\Exception\BadRequest: Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.
/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 154:OCA\DAV\Connector\Sabre\File->put()
/var/www/nextcloud/apps/dav/lib/Upload/UploadFolder.php - line 46:OCA\DAV\Connector\Sabre\Directory->createFile()
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1104:OCA\DAV\Upload\UploadFolder->createFile()/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 527:Sabre\DAV\Server->createFile()
/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php - line 89:Sabre\DAV\CorePlugin->httpPut()
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 474:Sabre\DAV\Server->emit()
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 251:Sabre\DAV\Server->invokeMethod()
/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 319:Sabre\DAV\Server->start()
/var/www/nextcloud/apps/dav/lib/Server.php - line 320:Sabre\DAV\Server->exec()
/var/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:OCA\DAV\Server->exec()
/var/www/nextcloud/remote.php - line 167:require_once("/var/www/ne ... p")
Screenshots of Nextcloud:
Screenshots of Client:
Server configuration
Operating system:
cat /etc/redhat-release
CentOS Linux release 8.1.1911 (Core)
uname -a
Linux nextcloud 4.18.0-147.8.1.el8_1.x86_64 #1 SMP Thu Apr 9 13:49:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Web server:'
nginx -V
nginx version: nginx/1.14.1
built by gcc 8.2.1 20180905 (Red Hat 8.2.1-3) (GCC)
built with OpenSSL 1.1.1 FIPS 11 Sep 2018 (running with OpenSSL 1.1.1c FIPS 28 May 2019)
TLS SNI support enabled
Database:
mariadb -V
mariadb Ver 15.1 Distrib 10.4.13-MariaDB, for Linux (x86_64) using readline 5.1
PHP version:
PHP-FPM connected with nginx thru socket
Version: 7.4.7
Memory Limit: 1 GB
Max Execution Time: 3600
Upload max size: 16 GB
Nextcloud version: (see Nextcloud admin page)
19.0.0.12
Updated from an older Nextcloud/ownCloud or fresh install:
Fresh install
Where did you install Nextcloud from:
nextcloud.com
Signing status:
No errors have been found.
List of activated apps:
sudo -u nginx php occ app:list
Enabled:
- accessibility: 1.5.0
- activity: 2.12.0
- bruteforcesettings: 1.6.0
- calendar: 2.0.3
- cloud_federation_api: 1.2.0
- comments: 1.9.0
- contacts: 3.3.0
- contactsinteraction: 1.0.0
- dav: 1.15.0
- federatedfilesharing: 1.9.0
- federation: 1.9.0
- files: 1.14.0
- files_external: 1.10.0
- files_pdfviewer: 1.8.0
- files_rightclick: 0.16.0
- files_sharing: 1.11.0
- files_trashbin: 1.9.0
- files_versions: 1.12.0
- files_videoplayer: 1.8.0
- firstrunwizard: 2.8.0
- groupfolders: 6.0.6
- logreader: 2.4.0
- lookup_server_connector: 1.7.0
- nextcloud_announcements: 1.8.0
- notifications: 2.7.0
- oauth2: 1.7.0
- password_policy: 1.9.1
- photos: 1.1.0
- privacy: 1.3.0
- provisioning_api: 1.9.0
- recommendations: 0.7.0
- serverinfo: 1.9.0
- settings: 1.1.0
- sharebymail: 1.9.0
- support: 1.2.1
- survey_client: 1.7.0
- systemtags: 1.9.0
- tasks: 0.13.1
- text: 3.0.1
- theming: 1.10.0
- twofactor_backupcodes: 1.8.0
- viewer: 1.3.0
- workflowengine: 2.1.0
Disabled: - admin_audit
- encryption
- updatenotification
- user_ldap
Nextcloud configuration:
sudo -u nginx php occ config:list system
{
"system": {
"blacklisted_files": [],
"instanceid": "REMOVED SENSITIVE VALUE",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"localhost",
"nextcloud",
"127.0.0.1",
"192.168.50.2",
],
"datadirectory": "REMOVED SENSITIVE VALUE",
"dbtype": "mysql",
"version": "19.0.0.12",
"overwrite.cli.url": "http:\/\/nextcloud",
"dbname": "REMOVED SENSITIVE VALUE",
"dbhost": "REMOVED SENSITIVE VALUE",
"dbport": "",
"dbtableprefix": "1c_",
"mysql.utf8mb4": true,
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"installed": true,
"default_language": "pl",
"default_locale": "pl",
"memcache.local": "\OC\Memcache\Redis",
"filelocking.enabled": true,
"memcache.locking": "\OC\Memcache\Redis",
"memcache.distributed": "\OC\Memcache\Redis",
"redis": {
"host": "REMOVED SENSITIVE VALUE",
"port": 6379,
"dbindex": 0,
"timeout": 600
},
"skeletondirectory": "core\/empty-default",
"maintenance": false
}
}
Are you using external storage, if yes which one: local/smb/sftp/...
There is smb plugin with attached storage, but it is not used for sync.
Are you using encryption:
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no
Client configuration
Browser:
Using Nextcloud desktop client.
Operating system:
Windows 10 Pro x64
Logs
Server respond "423 Locked" on "PUT http://192.168.50.2/remote.php/dav/files/USERNAME/dire...
Web server error log
nginx error.log is empty.
Nextcloud log (data/nextcloud.log)
{"reqId":"aG0WKf6Jk3WBKZzcfG5l","level":3,"time":"2020-06-12T10:10:42+00:00","remoteAddr":"192.168.50.5","user":"USERNAME","app":"no app in context","method":"PUT","url":"/remote.php/dav/uploads/USERNAME/2948563684/00000000","message":{"Exce
ption":"Sabre\DAV\Exception\BadRequest","Message":"Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a proble
m writing to the storage on the server side.","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":154,"function":"put","class":"OCA\DAV\Connector\Sabre\File","type":"->"},{"file":"/var/www
/nextcloud/apps/dav/lib/Upload/UploadFolder.php","line":46,"function":"createFile","class":"OCA\DAV\Connector\Sabre\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1104,"function":"cr
eateFile","class":"OCA\DAV\Upload\UploadFolder","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":527,"function":"createFile","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextclo
ud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\DAV\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"S
abre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Serv
er.php","line":319,"function":"start","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/
dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"F
ile":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":238,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows) mirall/2.6.4stable-Win64 (build 20200303) (Nextcloud)","version":"19.0.0.12"}
{"reqId":"aG0WKf6Jk3WBKZzcfG5l","level":4,"time":"2020-06-12T10:10:42+00:00","remoteAddr":"192.168.50.5","user":"USERNAME","app":"webdav","method":"PUT","url":"/remote.php/dav/uploads/USERNAME/2948563684/00000000","message":{"Exception":"Sab
re\DAV\Exception\BadRequest","Message":"Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a problem writing t
o the storage on the server side.","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":154,"function":"put","class":"OCA\DAV\Connector\Sabre\File","type":"->"},{"file":"/var/www/nextcloud/
apps/dav/lib/Upload/UploadFolder.php","line":46,"function":"createFile","class":"OCA\DAV\Connector\Sabre\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1104,"function":"createFile","
class":"OCA\DAV\Upload\UploadFolder","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":527,"function":"createFile","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty
/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\DAV\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\DAV\
Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","li
ne":319,"function":"start","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo
/v2/remote.php","line":35,"function":"exec","class":"OCA\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/
www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":238,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows) mirall/2.6.4stable-Win64 (build 20200303) (Nextcloud)","version":"19.0.0.12"}
Other Problems. Both are with some GUI elements. The bell on the top right with notifications have always red dot informing thet there are some notifications, but when i click on it there are no (then dot disappears). After refresh, or click on any link inside nextcloud red dot is appearing there.
_Second problem is, when i start Nextcloud for first time, or click on my avatar -> About (About Nextcloud window). The first page is empty blue like on the screenshot._
Update1:
Solution for last problem with first run wizard is here: https://help.nextcloud.com/t/first-run-wizard-video-does-not-play-http-302/75281
Update2:
There is also a lot of errors:
session_start(): A session had already been started - ignoring at /var/www/nextcloud/lib/private/Session/Internal.php#209
But i find this is known problem for N19 and many people already reported that.
Update3:
it looks it may be problem with Redis cache. I'm making more tests,
Update4:
I changed 'memcache.local' => '\OC\Memcache\Redis', into 'memcache.local' => '\OC\Memcache\APCu'.
Deleting two lines in config.php fixed the problem:
'filelocking.enabled' => true,
'memcache.locking' => '\OC\Memcache\Redis',
I'll try to learn what is the problem, but i think for now i will leave it disabled so at least, it will work correctly.
Update5:
Fresh CentOS 7 install. Same problem.
nicrame
All 11 comments
Getting the same right now:
Chrome on Huawei P20 Pro Android 10 fails with below errors.
Chrome on Samsung S9 Plus with Android 10 and Chrome works like a charm
Chrome on Samsung S9 Plus with Android 10 and Firefox uploads one img of a batch at 0bytes
Fatal webdav Sabre\DAV\Exception\BadRequest: 2020-07-06T10:35:07+00:00
Expected filesize of
2957473 bytes but read
(from Nextcloud
client) and wrote (to
Nextcloud storage)
425984 bytes. Could
either be a network
problem on the sending
side or a problem
writing to the storage
on the server side. at
apps/dav/lib/Connector/Sabre/File.php
line 238
0. .../Directory.php line 154
OCA\DAV\Connector\Sabre\File->put(
... ll
)
1. .../Server.php line 1104
OCA\DAV\Connector\Sabre\Directory->createFile(
... g",
... ll
)
2. .../CorePlugin.php line 527
Sabre\DAV\Server->createFile(
"test\/IMG_ ... g",
... ll,
... ll
)
3. .../WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPut(
... {},
S ... {}
)
4. .../Server.php line 474
Sabre\DAV\Server->emit(
... T",
[Sabre\HTTP\Request {},Sabre\HTT ... }]
)
5. .../Server.php line 251
Sabre\DAV\Server->invokeMethod(
... {},
... {}
)
6. .../Server.php line 319
Sabre\DAV\Server->start(
)
7. .../publicwebdav.php line 112
Sabre\DAV\Server->exec(
)
8. public.php line 81
require_once(
"\/var\/www\/html\/apps\/dav\/appinfo\/v1\/publicwebdav. ... p"
)
Together with
Error PHP Cannot modify header 2020-07-06T10:35:07+00:00
information - headers
already sent by
(output started at
/var/www/html/3rdparty/sabre/http/lib/Sapi.php:132)
at
/var/www/html/apps/dav/lib/Connector/Sabre/File.php#680
Setup is on Nextcloud 19.0 on Docker with mysql/redis/nginx/php-fpm behind a Traefik 2.
I'll debug this a bit further and add informations as I find them.
KoffeinKaio
on 6 Jul 2020
I also notice the same problem on only one instance of a shared server (several instances deployed).
- Stack LAMP Debian 10 with mod_php (PHP 7.3)
- all 5 Nextcloud are 18.0.6
- the Nextcloud which have the problem has similar config.php than others
- customers from faulty Nextcloud are all with nc-client 2.6.4 (Windows 10 and Mac)
Error is similar to described above:
[webdav] Fatal: Sabre\DAV\Exception\BadRequest: Expected filesize of 2657747 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 65536 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side. at <<closure>>
0. /home/cloud.my-domain.fr/www/apps/dav/lib/Connector/Sabre/Directory.php line 156
OCA\DAV\Connector\Sabre\File->put(null)
1. /home/cloud.my-domain.fr/www/3rdparty/sabre/dav/lib/DAV/Server.php line 1096
OCA\DAV\Connector\Sabre\Directory->createFile("Support_atelier_participatif_diag.pptx", null)
2. /home/cloud.my-domain.fr/www/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 525
Sabre\DAV\Server->createFile("files/userx/A ... x", null, null)
3. <<closure>>
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
4. /home/cloud.my-domain.fr/www/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
call_user_func_array([Sabre\DAV\CorePlugin {},"httpPut"], [Sabre\HTTP\Requ ... }])
5. /home/cloud.my-domain.fr/www/3rdparty/sabre/dav/lib/DAV/Server.php line 479
Sabre\Event\EventEmitter->emit("method:PUT", [Sabre\HTTP\Requ ... }])
6. /home/cloud.my-domain.fr/www/3rdparty/sabre/dav/lib/DAV/Server.php line 254
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
7. /home/cloud.my-domain.fr/www/apps/dav/lib/Server.php line 319
Sabre\DAV\Server->exec()
8. /home/cloud.my-domain.fr/www/apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec()
9. /home/cloud.my-domain.fr/www/remote.php line 165
require_once("/home/cloud.my- ... p")
PUT /remote.php/dav/files/userx/Commun%20Partage/06%20Production/01%20Dossiers%20clients/R%C3%A9alit%C3%A9s/03%20Missions/0%20Diagnostic/Support_atelier_participatif_diag.pptx
from 90.63.151.226 by userx at 2020-07-16T14:50:57+02:00
esprit-libre
on 16 Jul 2020
Same here:
- Private dockerized server:
- DBMS: yobasystems/alpine-mariadb@latest
- nextcloud@latest
- Version Nextcloud 19.0.1
Error: [webdav] Fatal: Sabre\DAV\Exception\BadRequest: Expected filesize of [...]
PHP
Version: 7.4.8
Memory Limit: 512 MB
Max Execution Time: 3600
Upload max size: 2 MB
DB
Type: mysql
Version: 10.4.13
Size: 24,7 MB
System
Operating System: Linux 5.4.51-1-ARCH armv7l
CPU: ARMv7 Processor rev 3 (v7l) (4 cores)
Memory: 3.75 GB
Server time: Thu Aug 20 00:56:51 UTC 2020
Uptime: 6 days, 7 hours, 53 minutes, 33 seconds
Time Servers:
Client system: MS Windows 10
jaalzateolaya
on 20 Aug 2020
@nicrame @KoffeinKaio @jaalzateolaya @esprit-libre
Try to place the source code and data repository on different disks, most possibly it was caused by IO limitation, even in some cases the website will not respond. Another solution is to turn on OPcache, also help
pupboss
on 20 Aug 2020
On my side, OPCache is on. IO disk limitation... I will check: any idea to have relevant metrics? (iotop ?)
But strange thing is that problem occurs mainly (not 100% but nearly) with one instance, and not with the others on the same server. So I'm not confident that this is IO disk limitation.
esprit-libre
on 20 Aug 2020
On my side, OPCache is on. IO disk limitation... I will check: any idea to have relevant metrics? (
iotop?)But strange thing is that problem occurs mainly (not 100% but nearly) with one instance, and not with the others on the same server. So I'm not confident that this is IO disk limitation.
Aside of WebDAV syncing, I have another similar issue, php-fpm got stuck when I watch videos...But finally all were solved by placing source code and data into different disks.
In terms of disk utilisation, first you can check by top command, seeing the average load, if CPU & memory use is low, but a high load, most possibly it is caused by IO limitation (for example CPU use is 2% but the load is over 40). Secondly, you can use sudo iotop or iostat -x 2 (print the status every 2 seconds).
But if all look good, good luck to you haha, because I have no other ideas.
pupboss
on 20 Aug 2020
@nicrame @KoffeinKaio @jaalzateolaya @esprit-libre
Try to place the source code and data repository on different disks, most possibly it was caused by IO limitation, even in some cases the website will not respond. Another solution is to turn on OPcache, also help
From the beginning i got different HDD for OS, and different for Nextcloud with data directory. Also i got enabled OPCache and APCu. I've got only 4 users, and most of the time Nextcloud do nothing. Sometimes this error occurs, when some file is uploaded, but even then there is only that 1 file transfer :S
nicrame
on 20 Aug 2020
I experience the same problem with NC 18.0.4 on mod-php 7.3 and Client 2.6.4 with SMB/CIFS Share External Storage.
Until now i do not know where the problem/disconnect while upload occurs. A Timeout/Blocker of PHP/Apache2 or something behind towards the External Storage... All the uploaders are using Notebooks with LTE connection, the broken uploads are often only 500kb.
It happens here very often and is going to be a real problem productively.
Example Error Message:
:{"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of 344294 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 114688 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.
Schroeffu
on 21 Aug 2020
I'm having the same issue with the webclient.
Don't know what the hell is causing this, but everytime I try to copy like 22 RAW images, I get this error on at least 10 of them.
Also reported this issue on nextcloud foruns, but so far, no help, https://help.nextcloud.com/t/timeouts-while-uploading-multiple-files/90717
nunong21
on 27 Aug 2020
"me too" ... more information/data :
SAMPLE:
[webdav] Fatal: Sabre\DAV\Exception\BadRequest: Expected filesize of 1927500 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 311296 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side. at <<closure>>
0. /srv/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php line 154
OCA\DAV\Connector\Sabre\File->put(null)
1. /srv/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 1104
OCA\DAV\Connector\Sabre\Directory->createFile("2020-08-29_104237_2422.jpg", null)
2. /srv/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 527
Sabre\DAV\Server->createFile("Family/Photos/0 ... g", null, null)
3. /srv/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
4. /srv/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 474
Sabre\DAV\Server->emit("method:PUT", [Sabre\HTTP\Requ ... }])
5. /srv/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 251
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
6. /srv/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 319
Sabre\DAV\Server->start()
7. /srv/nextcloud/apps/dav/appinfo/v1/webdav.php line 82
Sabre\DAV\Server->exec()
8. /srv/nextcloud/remote.php line 167
require_once("/srv/nextcloud/ ... p")
PUT /cloud/remote.php/webdav/Family/Photos/0%20_%20Incoming%20(Uploads)/foo/2020/08/2020-08-29_104237_2422.jpg
from 10.0.0.52 by fermulator at 2020-11-14T18:41:49+00:00
- Using "Auto Upload" from iOS 14.2 iPhone
- Large files (Camera Videos) >100MB nearly ALWAYS fail (probability)
- Smaller files (Camera Photos) <5MB fail sporadically (even small <1MB files fail)
- Hitting through WiFi strong signal, server is local (so routed internally)
- Backend is an external drive (SMB/CIFS)
Version info:
Nextcloud 19.0.4, in a VM
Operating System: | Linux 4.15.0-99-generic x86_64
-- | --
CPU: | Westmere E56xx/L56xx/X56xx (Nehalem-C) (4 cores)
Memory: | 1.95 GB
apache2 2.4.29-1ubuntu4.14 << nextcloud
apache2 2.4.18-2ubuntu3.14 << proxy
- php:
Version: 7.2.24
Memory Limit: 512 MB
Max Execution Time: 3600
Upload max size: 2 MB
- DB:
Type: mysql
Version: 10.1.47
Size: 63.5 MB
config
root@nextcloud:/srv/nextcloud# cat config/config.php
<?php
$CONFIG = array (
'instanceid' => 'SNIP',
'passwordsalt' => 'SNIP',
'secret' => 'SNIP',
'trusted_domains' =>
array (
0 => 'nextcloud',
1 => 'SNIP',
),
'trusted_proxies' =>
array (
0 => '1.0.0.52',
),
'overwritehost' => 'SNIP',
'overwriteprotocol' => 'https',
'overwritewebroot' => '/cloud',
'overwritecondaddr' => '^1\\.0\\.0\\.52$',
'datadirectory' => '/opt/nextcloud/',
'dbtype' => 'mysql',
'version' => '19.0.4.2',
'overwrite.cli.url' => 'https://nextcloud',
'dbname' => 'nextcloud',
'dbhost' => 'localhost',
'dbport' => '',
'dbtableprefix' => 'oc_',
'dbuser' => 'nextcloud_admin',
'dbpassword' => 'SNIP',
'installed' => true,
'theme' => '',
'loglevel' => 2,
'maintenance' => false,
'updater.release.channel' => 'stable',
'mail_smtpmode' => 'smtp',
'mail_smtphost' => 'SNIP',
'mail_sendmailmode' => 'smtp',
'mail_smtpport' => '465',
'mail_smtpsecure' => 'ssl',
'mail_from_address' => 'nextcloud',
'mail_domain' => 'SNIP',
'mail_smtpauthtype' => 'PLAIN',
'mail_smtpauth' => 1,
'mail_smtpname' => 'SNIP',
'mail_smtppassword' => 'SNIP',
'updater.secret' => 'SNIP',
'app_install_overwrite' =>
array (
0 => 'breezedark',
),
);
Other Observations / Tests
- am going through an Apache proxy : BUT: I monitored the logs on both proxy and nextcloud, and there are no "errors" correlated with these events
- looks like parallel uploads are enabled (4x max)
iotopis decently quite, no major bottleneck here afaict (the WiFi is slower than disk capabilities)- CPU is busy ~75% over 4 cores, but not maxed; (
apache2&smbclientprimarily) - memory is <30% utilized
EDIT/UPDATE:
on further observation, actually, the Apache2 proxy logs DO HAVE these:
[Sat Nov 14 14:25:10.266611 2020] [proxy_http:error] [pid 5859] (104)Connection reset by peer: [client 10.0.0.1:51291] AH02609: read request body failed to 10.0.0.55:443 (nextcloud) from 10.0.0.1 ()
[Sat Nov 14 14:25:10.266792 2020] [proxy_http:error] [pid 5859] [client 10.0.0.1:51291] AH01097: pass request body failed to 10.0.0.55:443 (nextcloud) from 10.0.0.1 ()
this seems to roughly correlate with the errors seen in Nextcloud backend; (they are not 1:1 or perfect correlation, BUT, the timestamps align) :exclamation:
perhaps in my case (and for others) if you too are using Apache2 reverse proxy for Nextcloud, ... if the proxy timeout itself is killing the connection / OR / nextcloud is killing/resetting the connection ... this might be the cause? (TBD on solution)
TESTING: nextcloud:~$ sudo a2dismod reqtimeout ... MONITORING WILL REPORT BACK
I dunno ... at first it seemed to help, but now it is uber slow (~ <500KB/sec), and when I restarted the "send full camera roll" the errors started happening again
fermulator
on 14 Nov 2020
We are moving our production to seafile after weeks of troubleshooting with no result, due to this unresolveable bug, 30% moved already. I'll no longer help here with logs and configs -.-
Edit: fyi, now with seafile we are avoiding webdav+sabredav completely, the sync from linux seafile server to windows share is done by their original client prepared as a windows service.
Schroeffu
on 15 Nov 2020
Related issues
juliushaertl
路
3Comments
dl5rcw
路
3Comments
e-alfred
路
3Comments
arno01
路
3Comments
MariusBluem
路
3Comments
Most helpful comment
I experience the same problem with NC 18.0.4 on mod-php 7.3 and Client 2.6.4 with SMB/CIFS Share External Storage.
Until now i do not know where the problem/disconnect while upload occurs. A Timeout/Blocker of PHP/Apache2 or something behind towards the External Storage... All the uploaders are using Notebooks with LTE connection, the broken uploads are often only 500kb.
It happens here very often and is going to be a real problem productively.
Example Error Message:
:{"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of 344294 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 114688 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.