Duplicati: Slow OneDrive backup speed

[StefanZi]

I have:

• [X ] searched open and closed issues for duplicates

---

Version info

Duplicati Version: 2.0.1.35_experimental_2016-12-13
Operating System: FreeNas 9.10.2-U2 Jail
Backend: OneDrive (1TB Office 365)

Bug description

The backup speed of OneDrive backend is only half of the achieved speed with a FTP backend.

Backup Job: 33.61GB (57538 files), AES encrypted, 250MB filesize
Upload speed: max 12Mbit/s (1.5 MB/s)

Uploading to OneDrive: Backup-Size: 30.15GB / Duration 14:15:24 (0.60 MB/s)
Uploading to FTP: Backup-Size: 30.57GB / Duration: 06:13:05 (1.4 MB/s)

In the backup logs I saw some retries in the OneDrive job like these. But I never saw a second retry (only 1 of 5). Next thing is a message about renaming some dblock file was successful.

• The remote server returned an error: (401) Unauthorized.
• Error getting response stream (ReadDone1): ReceiveFailure

Steps to reproduce

• Create two backup jobs with the same fileset with OneDrive and FTP backend
• Run the backup jobs and compare result

Actual result: FTP backend is twice as fast as OneDrive
Expected result: Both backends deliver nearly the same speed.

Screenshots

Loadprofile of my pfSense firewall when uploading

OneDrive

FTP

debug log

Log of the OneDrive backup Job result
DeletedFiles: 0 DeletedFolders: 0 ModifiedFiles: 0 ExaminedFiles: 57538 OpenedFiles: 57538 AddedFiles: 57538 SizeOfModifiedFiles: 0 SizeOfAddedFiles: 36084699558 SizeOfExaminedFiles: 36084699558 SizeOfOpenedFiles: 36084699558 NotProcessedFiles: 0 AddedFolders: 6256 TooLargeFiles: 0 FilesWithError: 0 ModifiedFolders: 0 ModifiedSymlinks: 0 AddedSymlinks: 4 DeletedSymlinks: 0 PartialBackup: False Dryrun: False MainOperation: Backup CompactResults: null DeleteResults: DeletedSets: [] Dryrun: False MainOperation: Delete CompactResults: null EndTime: 01/17/2017 11:40:09 BeginTime: 01/17/2017 11:40:01 Duration: 00:00:07.9699580 BackendStatistics: RemoteCalls: 272 BytesUploaded: 32372893173 BytesDownloaded: 266104903 FilesUploaded: 249 FilesDownloaded: 3 FilesDeleted: 0 FoldersCreated: 0 RetryAttempts: 18 UnknownFileSize: 0 UnknownFileCount: 0 KnownFileCount: 249 KnownFileSize: 32372893173 LastBackupDate: 01/16/2017 22:25:58 BackupListCount: 1 TotalQuotaSpace: 0 FreeQuotaSpace: 0 AssignedQuotaSpace: -1 RepairResults: null TestResults: MainOperation: Test Verifications: [ Key: duplicati-.....dlist.zip.aes Value: [], Key: duplicati-.....dindex.zip.aes Value: [], Key: duplicati-.....dblock.zip.aes Value: [] ] EndTime: 01/17/2017 11:41:23 BeginTime: 01/17/2017 11:40:30 Duration: 00:00:52.3923670 EndTime: 01/17/2017 11:41:23 BeginTime: 01/16/2017 21:25:58 Duration: 14:15:24.7119940 Messages: [ Renaming "duplicati-....dblock.zip.aes" to "duplicati-.....dblock.zip.aes", Renaming "duplicati-....dblock.zip.aes" to "duplicati-....dblock.zip.aes", Renaming "duplicati-....dblock.zip.aes" to "duplicati-....dblock.zip.aes", Renaming "duplicati-....dindex.zip.aes" to "duplicati-....dindex.zip.aes", Renaming "duplicati-....dindex.zip.aes" to "duplicati-....dindex.zip.aes", ... ] Warnings: [] Errors: []

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[fkappen]

Same here Upload is far below the limit - max 5mbit/s - Upload with original OneDrive Client run with 40mbit/s Upload

[JanSchr]

Same here too, 40 mbit/s Upstream but no more then 12 mbit/ used. When creating a sync job within in my qnap (using the qnap CloudSync App) the full 40 mbit/s upstream is used, so it's not network related.

[Bazzu85]

I have similar issue with ACD.
issue opened for these specific service..

[AnderssonPeter]

i only get a few kb/s per sec, initial backups take for ever!

[hifihedgehog]

Same here. My upload speeds are generally 5-10 MB/s with my web server, but I only get 1 MB/s with OneDrive from Duplicati. I know it isn't OneDrive, either, to blame because other tools for syncing to OneDrive see speeds around 5-10 MB/s. So what should be a 20-30 minute backup takes literally 2-3 hours. Yikes!

[fred-cardoso]

I am having the same issue. I am in the middle of a initial backup, and it got only now around 50% (I started two days ago :D)

Anyways, speed also capped around 12 Mbit/s.

I don't have now a comparison with another protocol (like FTP) because I don't have that kind of services. But I get easily 20Mbits/s on Speedtests, but usually around 30.