Server: Upload is broken with S3 object storage

[Damillora]

Steps to reproduce

1. Upload a reasonably large file using the desktop sync client (files more than 10mb gives errors as in the log)

Expected behaviour

The file is uploaded successfully.

Actual behaviour

The file is only partially uploaded. Files more than 10mb complains that "A sha256 checksum could not be calculated" as in the log.

Server configuration

Operating system: Ubuntu 18.04

Web server: nginx/1.14.0 (ubuntu)

Database: 10.1.34-MariaDB-0ubuntu0.18.04.1

PHP version: 7.2.10-0ubuntu0.18.04.1

Nextcloud version: 15.0.0

Updated from an older Nextcloud/ownCloud or fresh install: fresh install

Where did you install Nextcloud from: Installation using the zip file

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - accessibility: 1.1.0
  - activity: 2.8.2
  - bruteforcesettings: 1.2.0
  - cloud_federation_api: 0.1.0
  - comments: 1.5.0
  - dav: 1.8.0
  - federatedfilesharing: 1.5.0
  - federation: 1.5.0
  - files: 1.10.0
  - files_pdfviewer: 1.4.0
  - files_sharing: 1.7.0
  - files_texteditor: 2.7.0
  - files_trashbin: 1.5.0
  - files_versions: 1.8.0
  - files_videoplayer: 1.4.0
  - firstrunwizard: 2.4.0
  - gallery: 18.2.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.3.0
  - nextcloud_announcements: 1.4.0
  - notifications: 2.3.0
  - oauth2: 1.3.0
  - password_policy: 1.5.0
  - provisioning_api: 1.5.0
  - serverinfo: 1.5.0
  - sharebymail: 1.5.0
  - support: 1.0.0
  - survey_client: 1.3.0
  - systemtags: 1.5.0
  - theming: 1.6.0
  - twofactor_backupcodes: 1.4.1
  - updatenotification: 1.5.0
  - workflowengine: 1.5.0
Disabled:
  - admin_audit
  - audioplayer
  - encryption
  - files_external
  - mail
  - notes
  - richdocuments
  - twofactor_totp
  - user_external
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "objectstore": {
            "class": "OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "yurikonanaofiles",
                "autocreate": true,
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "port": 443,
                "use_ssl": true,
                "region": "ap-southeast-1",
                "use_path_style": false
            }
        },
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "files.yurikonanao.cloud"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "15.0.0.10",
        "overwrite.cli.url": "https:\/\/files.yurikonanao.cloud",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "loglevel": 2,
        "updater.release.channel": "stable",
        "updater.secret": "***REMOVED SENSITIVE VALUE***",
        "theme": ""
    }
}

Are you using external storage, if yes which one: S3

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser:

Operating system:

Logs

Web server error log

Web server error log

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"yzvfbJ4BHWoIMB0E6mU5","level":3,"time":"2018-12-08T19:27:42+00:00","remoteAddr":"202.62.19.128","user":"Damillora","app":"objectstore","method":"MOVE","url":"\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","message":{"Exception":"Aws\\Exception\\CouldNotCreateChecksumException","Message":"A sha256 checksum could not be calculated for the provided upload body, because it was not seekable. To prevent this error you can either 1) include the ContentMD5 or ContentSHA256 parameters with your request, 2) use a seekable stream for the body, or 3) wrap the non-seekable stream in a GuzzleHttp\\Psr7\\CachingStream object. You should be careful though and remember that the CachingStream utilizes PHP temp streams. This means that the stream will be temporarily stored on the local disk.","Code":0,"Trace":[{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Signature\/S3SignatureV4.php","line":22,"function":"getPayload","class":"Aws\\Signature\\SignatureV4","type":"->","args":[{"__class__":"GuzzleHttp\\Psr7\\Request"}]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Middleware.php","line":126,"function":"signRequest","class":"Aws\\Signature\\S3SignatureV4","type":"->","args":[{"__class__":"GuzzleHttp\\Psr7\\Request"},"*** sensitive parameter replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/FulfilledPromise.php","line":39,"function":"Aws\\{closure}","class":"Aws\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\FulfilledPromise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[true]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/S3\/S3ClientTrait.php","line":33,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php","line":77,"function":"upload","class":"Aws\\S3\\S3Client","type":"->","args":["files-yurikonanaocloud-second","urn:oid:24147",null,"private",{"mup_threshold":524288000,"part_size":524288000}]},{"file":"\/var\/www\/files\/lib\/private\/Files\/ObjectStore\/ObjectStoreStorage.php","line":449,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:24147",null]},{"file":"\/var\/www\/files\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null,null]},{"file":"\/var\/www\/files\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null,null]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Connector\/Sabre\/File.php","line":169,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[null]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":316,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":130,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->","args":[{"__class__":"OCA\\DAV\\Upload\\FutureFile"},{"__class__":"OCA\\DAV\\Connector\\Sabre\\Directory"},"15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":161,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Damillora\/1702689971\/.file","files\/Damillora\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":642,"function":"move","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Damillora\/1702689971\/.file","files\/Damillora\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMove"],[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MOVE",[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Server.php","line":298,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/files\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/files\/remote.php","line":163,"args":["\/var\/www\/files\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Signature\/SignatureV4.php","Line":166,"Previous":{"Exception":"Exception","Message":"Stream from assembly node shorter than expected, got 5041837 bytes, expected 10284717","Code":0,"Trace":[{"function":"stream_read","class":"OCA\\DAV\\Upload\\AssemblyStream","type":"->","args":[8192]},{"file":"\/var\/www\/files\/3rdparty\/icewind\/streams\/src\/Wrapper.php","line":91,"function":"fread","args":[null,8192]},{"file":"\/var\/www\/files\/lib\/private\/Files\/Stream\/CountReadStream.php","line":55,"function":"stream_read","class":"Icewind\\Streams\\Wrapper","type":"->","args":[8192]},{"function":"stream_read","class":"OC\\Files\\Stream\\CountReadStream","type":"->","args":[8192]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/psr7\/src\/Stream.php","line":218,"function":"fread","args":[null,1048576]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/psr7\/src\/functions.php","line":419,"function":"read","class":"GuzzleHttp\\Psr7\\Stream","type":"->","args":[1048576]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Signature\/SignatureV4.php","line":164,"function":"GuzzleHttp\\Psr7\\hash","args":[{"__class__":"GuzzleHttp\\Psr7\\Stream"},"sha256"]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Signature\/S3SignatureV4.php","line":22,"function":"getPayload","class":"Aws\\Signature\\SignatureV4","type":"->","args":[{"__class__":"GuzzleHttp\\Psr7\\Request"}]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/Middleware.php","line":126,"function":"signRequest","class":"Aws\\Signature\\S3SignatureV4","type":"->","args":[{"__class__":"GuzzleHttp\\Psr7\\Request"},"*** sensitive parameter replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/FulfilledPromise.php","line":39,"function":"Aws\\{closure}","class":"Aws\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\FulfilledPromise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[true]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/3rdparty\/aws\/aws-sdk-php\/src\/S3\/S3ClientTrait.php","line":33,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/files\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php","line":77,"function":"upload","class":"Aws\\S3\\S3Client","type":"->","args":["files-yurikonanaocloud-second","urn:oid:24147",null,"private",{"mup_threshold":524288000,"part_size":524288000}]},{"file":"\/var\/www\/files\/lib\/private\/Files\/ObjectStore\/ObjectStoreStorage.php","line":449,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:24147",null]},{"file":"\/var\/www\/files\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null,null]},{"file":"\/var\/www\/files\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null,null]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Connector\/Sabre\/File.php","line":169,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[null]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":316,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3",null]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":130,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->","args":[{"__class__":"OCA\\DAV\\Upload\\FutureFile"},{"__class__":"OCA\\DAV\\Connector\\Sabre\\Directory"},"15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":161,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Damillora\/1702689971\/.file","files\/Damillora\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":642,"function":"move","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Damillora\/1702689971\/.file","files\/Damillora\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"]},{"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMove"],[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MOVE",[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/files\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/files.yurikonanao.cloud\/remote.php\/dav\/uploads\/Damillora\/1702689971\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/files\/apps\/dav\/lib\/Server.php","line":298,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/files\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/files\/remote.php","line":163,"args":["\/var\/www\/files\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/files\/apps\/dav\/lib\/Upload\/AssemblyStream.php","Line":123},"CustomMessage":"Could not create object urn:oid:24147 for files\/Music\/Nogizaka46\/Hadashi de Summer\/15-03C \u767d\u7c73\u69d8 (\u3055\u3086\u308a\u3093\u3054\u8ecd\u56e3).mp3"},"userAgent":"Mozilla\/5.0 (Linux) mirall\/2.5.0git (Nextcloud)","version":"15.0.0.7"}

Browser log

Browser log

[ptrunk]

Same problem here. In my case every file upload is broken but only files with more than 10mb produce error messages. Other files seem to be uploaded fine but if you check your s3 backend storage or want to download those files they are not complete (3.2mb instead of the original size).

I have also noticed a strange behavior because it is working fine if the default encryption module app is activated (but not enabled in the admin settings). If it is disabled (like in my 14.x.x installation before) all uploads are totally broken.

I am sorry for not providing any logs but i don't have any. I have increaed the log level but still not logs in nextcloud. Maybe you can help me to give you some more information.

[jkroepke]

Got this error message with an nextcloud client on Nextcloud 15 w/ Amazon S3, too.

I have also noticed a strange behavior because it is working fine if the default encryption module app is activated (but not enabled in the admin settings).

I can confirm, active the app but let it disabled resolve this error. But Group Folder are not working with this anymore.

I'm using the offical nextcloud 15 fpm docker container.

[Damillora]

@ptrunk I can confirm that my uploaded files are also broken and only files more than 10mb gives out errors.

In my case files are cut off at roughly half the file size (54%)

I can also confirm that enabling the default encryption module but not enabling encryption works around the problem.

I'll edit the issue to reflect this new information.

[srgvg]

Not sure if this is related, but I started having sync issues since upgrading to NC15. I use S3 (Digitalocean) as primary storage.

A typical error I get (from the client log) is

The item is not synced because of previous errors: Error downloading https://XXXX.XXXXX>XXX/remote.php/dav/uploads/YYYY/193679685/.file - server replied: Internal Server Error (A sha256 checksum could not be calculated for the provided upload body, because it was not seekable. To prevent this error you can either 1) include the ContentMD5 or ContentSHA256 parameters with your request, 2) use a seekable stream for the body, or 3) wrap the non-seekable stream in a GuzzleHttp\Psr7\CachingStream object. You should be careful though and remember that the CachingStream utilizes PHP temp streams. This means that the stream will be temporarily stored on the local disk.

Server side I get this in the logs:

[objectstore] Error: Aws\Exception\CouldNotCreateChecksumException: A sha256 checksum could not be calculated for the provided upload body, because it was not seekable. To prevent this error you can either 1) include the ContentMD5 or ContentSHA256 parameters with your request, 2) use a seekable stream for the body, or 3) wrap the non-seekable stream in a GuzzleHttp\Psr7\CachingStream object. You should be careful though and remember that the CachingStream utilizes PHP temp streams. This means that the stream will be temporarily stored on the local disk. at <<closure>>

 0. /var/www/html/3rdparty/aws/aws-sdk-php/src/Signature/S3SignatureV4.php line 22
    getPayload(GuzzleHttp\Psr7\Request {})
 1. /var/www/html/3rdparty/aws/aws-sdk-php/src/Middleware.php line 126
    signRequest(GuzzleHttp\Psr7\Request {}, "*** sensitive parameter replaced ***")
 2. /var/www/html/3rdparty/guzzlehttp/promises/src/FulfilledPromise.php line 39
    Aws\{closure}("*** sensitive parameters replaced ***")
 3. /var/www/html/3rdparty/guzzlehttp/promises/src/TaskQueue.php line 47
    GuzzleHttp\Promise\{closure}("*** sensitive parameters replaced ***")
 4. /var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php line 246
    run(true)
 5. /var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php line 223
    invokeWaitFn()
 6. /var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php line 267
    waitIfPending()
 7. /var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php line 225
    invokeWaitList()
 8. /var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php line 62
    waitIfPending()
 9. /var/www/html/3rdparty/aws/aws-sdk-php/src/S3/S3ClientTrait.php line 33
    wait()
10. /var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php line 77
    upload("ginsys-cloud", "urn:oid:623737", null, "private", {mup_threshold:  ... 0})
11. /var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php line 449
    writeObject("urn:oid:623737", null)
12. /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 630
    writeStream("files/Applicati ... e", null, null)
13. /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 630
    writeStream("files/Applicati ... e", null, null)
14. /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 630
    writeStream("files/Applicati ... e", null, null)
15. /var/www/html/apps/dav/lib/Connector/Sabre/File.php line 169
    writeStream("files/Applicati ... e", null)
16. /var/www/html/apps/dav/lib/Connector/Sabre/Directory.php line 156
    put(null)
17. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 316
    createFile("Nextcloud-2.5.1-x86_64.AppImage", null)
18. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 130
    copyNode(OCA\DAV\Upload\FutureFile {}, OCA\DAV\Connector\Sabre\Directory {}, "Nextcloud-2.5.1-x86_64.AppImage")
19. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 161
    copy("uploads/serge/2907701815/.file", "files/serge/App ... e")
20. /var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 642
    move("uploads/serge/2907701815/.file", "files/serge/App ... e")
21. <<closure>>
    httpMove(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
22. /var/www/html/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
    call_user_func_array([Sabre\DAV\CorePlugin {},"httpMove"], [Sabre\HTTP\Requ ... }])
23. /var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php line 479
    emit("method:MOVE", [Sabre\HTTP\Requ ... }])
24. /var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php line 254
    invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
25. /var/www/html/apps/dav/lib/Server.php line 298
    exec()
26. /var/www/html/apps/dav/appinfo/v2/remote.php line 35
    exec()
27. /var/www/html/remote.php line 163
    require_once("/var/www/html/a ... p")

MOVE /remote.php/dav/uploads/serge/2907701815/.file
from 104.37.191.11 by serge at 2018-12-20T21:18:45+00:00

and

[webdav] Fatal: Sabre\DAV\Exception: A sha256 checksum could not be calculated for the provided upload body, because it was not seekable. To prevent this error you can either 1) include the ContentMD5 or ContentSHA256 parameters with your request, 2) use a seekable stream for the body, or 3) wrap the non-seekable stream in a GuzzleHttp\Psr7\CachingStream object. You should be careful though and remember that the CachingStream utilizes PHP temp streams. This means that the stream will be temporarily stored on the local disk. at <<closure>>

 0. /var/www/html/apps/dav/lib/Connector/Sabre/File.php line 209
    convertToSabreException(Aws\Exception\Co ... {})
 1. /var/www/html/apps/dav/lib/Connector/Sabre/Directory.php line 156
    put(null)
 2. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 316
    createFile("VID_20181103_110125.mp4", null)
 3. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 130
    copyNode(OCA\DAV\Upload\FutureFile {}, OCA\DAV\Connector\Sabre\Directory {}, "VID_20181103_110125.mp4")
 4. /var/www/html/3rdparty/sabre/dav/lib/DAV/Tree.php line 161
    copy("uploads/serge/3234493661/.file", "files/serge/Imm ... 4")
 5. /var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 642
    move("uploads/serge/3234493661/.file", "files/serge/Imm ... 4")
 6. <<closure>>
    httpMove(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
 7. /var/www/html/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
    call_user_func_array([Sabre\DAV\CorePlugin {},"httpMove"], [Sabre\HTTP\Requ ... }])
 8. /var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php line 479
    emit("method:MOVE", [Sabre\HTTP\Requ ... }])
 9. /var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php line 254
    invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
10. /var/www/html/apps/dav/lib/Server.php line 298
    exec()
11. /var/www/html/apps/dav/appinfo/v2/remote.php line 35
    exec()
12. /var/www/html/remote.php line 163
    require_once("/var/www/html/a ... p")

MOVE /remote.php/dav/uploads/serge/3234493661/.file
from 104.37.191.11 by serge at 2018-12-20T21:20:05+00:00

I also get lots of unresolved conflicts.

[drphr4ud]

Same issue as OP. Only difference to OP's environment is that is my back-end is minio.

I installed and enabled the default encryption module but did not enable encryption as per ptrunk's suggestion and this error seems to have gone away.

[Nutomic]

Same problem here, on a fresh setup with docker-compose and Nextcloud 15 and Minio as primary storage. In our case, small files (under 10 MB) always work fine, only big files can't be uploaded. Enabling the encryption module also fixes the problem for large files.

Happy to help if there is any more info (or testing) needed.

[Vaelatern]

Can verify with PHP 7.0 on debian, Nextcloud 15.0.0, that when I enabled encryption from the Settings->Security, then enabled the Default Encryption App in Apps, then disabled Encrypt Home, then ran php occ maintenance:singleuser --on; occ encryption:disable; occ maintenance:singleuser --off, my problem was as far as I can tell solved. Wasabi (fully s3 compatible) is my primary storage.

[srgvg]

@Vaelatern tried your suggestion. I'm using the official docker images (tag 15.0.0, which are AFAIK Debian based). However occ maintenance:singleuser doesn't exist, at least on my setup, the correct command would be occ maintenance:mode --on/off.

Also, FYI, it's implied by @Vaelatern, but to make it explicit: whilst encryption is in the end disabled in admin setting, it seems important that the app is kept enabled.

However that doesn't (immediately) solve my issues. At first it seemed, but then I got other error's first, like

[no app in context] Error: Could not find mount point, can't keep encryption keys

DELETE /remote.php/dav/files/blah/.IMG_3875.JPG.~72f28efa

Which mask the former issue, which re-appair after I again disable the default encryption app.
But during debugging I sometimes noticed it was important to re-connect or log out/in again. So here, restarting the nextcloud desktop client helped at that point.

At some point I also disabled, then re-enabled Group Folders, as skimming through other issues that seemed a bit related - by error messages I got - it was often mentioned to be related to Group Folders and encryption. As most of my data lives in Shared Group Folders, I gave that a try. Not sure if it helped, but it could have.

So, in the end, and for now, this seems to solve the issue at my end. I'll keep an eye on it in coming days, and keep following this issue here.

[Vaelatern]

So ultimately it's just something different about how the file makes its way to S3.

[ncodeyx]

Can confirm on the latest Nextcloud 15 version.

[cconover]

I'm seeing the same behavior on Nextcloud 15 with Wasabi (S3) backend. I have not done anything with encryption settings.

[lukasmu]

Can also confirm on Nextcloud 15.0.2 with DigitalOcean Spaces (S3).

[Levero]

Can also confirm on Nextcloud 15.0.2 with Amazon S3. Getting this error message:

A sha256 checksum could not be calculated for the provided upload body, because it was not seekable. To prevent this error you can either 1) include the ContentMD5 or ContentSHA256 parameters with your request, 2) use a seekable stream for the body, or 3) wrap the non-seekable stream in a GuzzleHttp\Psr7\CachingStream object. You should be careful though and remember that the CachingStream utilizes PHP temp streams. This means that the stream will be temporarily stored on the local disk.

[cconover]

I'm getting the same error message as @Levero

[Vaelatern]

It's definitely a thing that's happening. Has anybody traced what's throwing that error?

[srgvg]

Not sure what "needs triage" means and who can do that.
Is there any way to escalate this? This bug is definitely confirmed by now.

[ptrunk]

The problem can be solved by enabling the default encryption module. But in my case the module was disabled. And for me it is not clear why it is necessary to enable it to use S3 Backends.

I didn't realize this after the upgrade. So i had lot of corrupted data. And this is a big problem for a service like nextcloud. I really think this bug should be escalated.

[kesselb]

A new issue has the label "needs triage" by default. I tried to reproduce this issue yesterday with no luck. Any file i upload (either by web client or curl) is a stream and seekable and worked for me with s3 and minio. If you have the same problem please include at least the basic information from the issue template. You can use the issuetemplate app (which is included by default) for that.

[ptrunk]

@danielkesselberg Thanks for your feedback. Could you please try this test again with a disabled "default encryption module"? I think a lot of people are running into this issue because it is not clear that this module is required in Nextcloud 15. In all version before it was not.

[Nutomic]

@danielkesselberg Did you try this with a fresh installation with docker-compose, without any addons? I tried this a couple of times and could reproduce this every single time. Happy to provide more information if you need it.

[kesselb]

I used a fresh install on lemp like in https://github.com/nextcloud/server/issues/13062#issue-390777051. A good start would be server configuration & logs from issue template.

[cconover]

My instance is running using Docker Compose.

[jkroepke]

We are using the Docker Images inside Kubernetes.

I get the same errors from https://github.com/nextcloud/server/issues/13062#issue-390777051 unless I install the default encryption module.

[gcollura]

I have the same problem with nextcloud-snap, installed on version 13 and upgraded to 15. Storage is pointed at Amazon s3. I don't use encryption. If the file is reasonably big I get an error message on top of the page, otherwise if it's >5MB, but still reasonably small, it fails with Connection closed.

[kesselb]

My instance is running using Docker Compose.

We are using the Docker Images inside Kubernetes.

I get the same errors from #13062 (comment) unless I install the default encryption module.

I have the same problem with nextcloud-snap, installed on version 13 and upgraded to 15. Storage is pointed at Amazon s3. I don't use encryption. If the file is reasonably big I get an error message on top of the page, otherwise if it's >5MB, but still reasonably small, it fails with Connection closed.

If you have the same problem please include at least the basic information from the issue template. You can use the issuetemplate app (which is included by default) for that.

[lukasmu]

I haven't really had time to look into it in detail yet. I will try to upload a issue template later this week.
Unfortunately the issuetemplate app is not available in NC 15 yet...

What I could see already (using the webinterface):
There are several PUT requests send to
https://example.org/remote.php/dav/uploads/USERNAME/web-file-upload-SOMEID/ANOTHERID
which all are fine but then the MOVE request to
https://example.org/remote.php/dav/uploads/USERNAME/web-file-upload-SOMEID/.file
fails with the internal server error reported several times above ("A sha256 checksum could not be calculated for the provided upload body, because it was not seekable...").

[drphr4ud]

My instance is running using Docker Compose.

We are using the Docker Images inside Kubernetes.
I get the same errors from #13062 (comment) unless I install the default encryption module.

I have the same problem with nextcloud-snap, installed on version 13 and upgraded to 15. Storage is pointed at Amazon s3. I don't use encryption. If the file is reasonably big I get an error message on top of the page, otherwise if it's >5MB, but still reasonably small, it fails with Connection closed.

If you have the same problem please include at least the basic information from the issue template. You can use the issuetemplate app (which is included by default) for that.

issuetemplate app not showing in my installations of NC 15.0.2. https://apps.nextcloud.com/apps/issuetemplate doesn't indicate it exists for version 15.

FWIW a lot of "me too" posters here use Docker images. Let me just say I am using a straight install on Ubuntu LTS 18.04 minimal with nginx. No docker involved so it's not likely related to the docker image.

I ran into this issue because the Nextcloud documentation specifically states that the encryption module is NOT compatible with object based storage. When I set it up the first thing I did was disable the default encryption module because it is stated in the documentation that it won't work with my storage back-end anyway. So I disable it and that then breaks the checksumming of any uploads.

[kesselb]

issuetemplate app not showing in my installations of NC 15.0.2. https://apps.nextcloud.com/apps/issuetemplate doesn't indicate it exists for version 15.

:see_no_evil:

Can reproduce your issue (with and without docker). Regression from https://github.com/nextcloud/server/pull/12072. A CountReadStream object is passed to writeObject but CountReadStream does not implement seekable :boom: Not sure if CountReadStream is wrong here or seekable just missing @icewind1991 @rullzer :ping_pong:

[gcollura]

I can't use issuetemplate either. But my setup is just a nextcloud-snap with channel=15/candidate and storage on Amazon s3.
So, if I understand correctly, the workaround for now is to enable the default encryption module app, but without enabling the encryption from the admin panel, right?

[koehn]

Ditto. 15.0.2, DigitalOcean Spaces, Kubernetes, official Docker image. Enabling the encryption app didn’t help, although I didn’t turn it on for my personal account.

[rullzer]

mmm I'll look into this today. But I'm confused as last week I had a test setup that worked like a charm on S3.

Anyways I'll get back to you all.

[theopen-institute]

Thanks to everyone working on this. Can anyone recommend a short term fix? We didn't catch this bug when we tested v15 before general rollout, and now we're in a rough spot with a severely hamstrung system.

[rullzer]

Ok i could reproduce it.

We should think if it is possible to create a sha256 hash without going over the whole stream and feeding it to the S3 SDK. Because we could of course calculate this while reading the file.

Anyways. PR incomming

[rullzer]

The quick fix is in https://github.com/nextcloud/server/pull/13829
Then we can look into a way that still improves performance while not breaking later.

[theopen-institute]

Thanks @rullzer, this is tremendous. I manually hotfixed your PR, and my site is working again. Your quick work on this is greatly appreciated.

[srgvg]

Whilst I enabled the work-around, enabling encryption app, not enabling encryption, which solved most errors described in this issue, I still have issues with some files yielding the Could not find mount point, can't keep encryption keys error, both uploading new files, or deleting files/folders.

Not sure if that is still part of this issue here. Might be https://github.com/nextcloud/server/issues/13380

Any pointers to where/how to follow up on that welcome, thx.