desktop 🚀 - Error when uploading changes on files with umlauts

Also affects Linux desktop client:

Client version: 2.5.2-20190319.015224
Operating system: Ubuntu 18.04
OS language: German
Qt version used by client package: Qt 5.9.5
Client package: from official Nextcloud PPA

danielrheinbay on 28 Apr 2019

👍1

I can confirm this issue aswell - having it since 2.5.X
It works fine with 2.3.3.

OS Language: German.
Client version: v2.5.2
OS: Windows 10 Pro 1809

valentingc on 3 May 2019

I have the same issue in French on Ubuntu 19.04 with file where name has accented characters, punctuation and blank space.

Client : 2.5.1git

PhBobo on 24 May 2019

Same issue with 2.5.2 on archlinux.
Syncing works with owncloud-client too, by the way.

marco44 on 25 May 2019

I have the same issue in French on Ubuntu 19.04 with file where name has accented characters, punctuation and blank space.

Client : 2.5.1git

It does not seems to be a problem with accented characters, as I have a lot of other files in this case and it works fine. The problem arise with big files, in my case mp4 files.

PhBobo on 26 May 2019

Same issue when edditing docx file at > 11,9MB incl German Umlaut in file name like "Mündliche Prüfung Testkandidat.docx" with MS-Word 2019 under Windows Pro 1809.

No issues encountered using the NC Web app and Document Server by Onlyoffice.
Nextcloud Client: git-Revision 56c905 auf Mar 19 2019, 13:34:10 verwendet Qt 5.11.1, OpenSSL 1.0.1h 5 Jun 2014

Ubuntu Server 18.04
locale: LANG=de_DE.UTF-8 ....
nginx 1.15.7
MariaDB 10.3.11 (utf-8, big int)
PHP 7.2.10 (utf-8, 512 MB)
Nextcloud 16.0.1

Serverfox on 4 Jun 2019

Same issue. I'm out of sync since a few weeks now.
Could it have been caused by upgrade to Nextcloud 16? I also remember by seeing @Serverfox comment that I did the bigint conversion at approximately the time the issue began appearing.

I believe we must many others affected by this bug, can this be prioritized?

Server:
nginx 1.16.0
LANG=fr_FR.UTF-8
MariaDB utf8mb4 10.2.x
PHP 7.2.x
Nextcloud 16.0.1

Client:
Version 2.5.2
Linux (Gentoo)

papjul on 7 Jun 2019

I have the same issue even without umlauts. Manually removing the local (client) sync DB most of the time fixes it for a while. The file is hidden in the top-most Nextcloud directory (e.g. C:/Users/MyUser/Nextcloud/xyz.db). Deleting it forces a re-scan of the Nextcloud-directory and usually a bunch of files are updated that have not been synced...

0x47 on 16 Jun 2019

👍1

I have the same issue even without umlauts.

I'm French, issues seems to be with special characters in general.

Manually removing the local (client) sync DB most of the time fixes it for a while. The file is hidden in the top-most Nextcloud directory (e.g. C:/Users/MyUser/Nextcloud/xyz.db). Deleting it forces a re-scan of the Nextcloud-directory and usually a bunch of files are updated that have not been synced...

Don't do this. This will not actually sync, this will just duplicate all files that were updated since the issue began, in conflict. Both conflicted versions stay on your local PC and don't actually sync. Nextcloud Desktop will say sync is up-to-date but should show you a "there are conflicted files" banner. Also, all files you deleted will reappear in your folders, so it's really not a good idea.

papjul on 16 Jun 2019

Thanks, that's good to know. I thought it was equivalent to setting up a new account and choosing the option "keep local files" in the desktop client, letting Nextcloud compare the offline and online folder structure.

So, how can this be achieved then without deleting the Files in .../AppData/Local/Nextcloud etc.? This requires to completely setup the account again, including app password. After all, there is no "repair sync database" option in the client.

0x47 on 16 Jun 2019

This issue is 100% reproducible, 100% out of NC specification and I want this to be taken more seriously!!!

This bug prevents me to install NC in a productive environment. I've spend time and money to set up NC properly and I really do not want to be forced to get back Google Drive, Dropbox etc. Neither of these are suffering from such an issue.

Serverfox on 16 Jun 2019

I know :( This is a critical regression bug and we did not receive a single reply from Nextcloud developers, I don't know how we can raise awareness among the hundreds of issues here…

@marco44 Can you confirm you safely replace Nextcloud client with ownCloud desktop client with success? How did you proceed? Thanks!

@0x47 don't lose time setting up your Nextcloud desktop app for the "first time", the issue is not in the database. It will become out-of-sync again as soon as you put newer versions of files containing a special character in its name. In case of emergency, you can upload newer versions of files from the web app, but that's not convenient at all especially if you have numerous and big files. You also need to track the changes manually.

papjul on 16 Jun 2019

@papjul : It's been working flawlessly for 3 weeks. I simply uninstalled nextcloud-client, installed owncloud-client, and re-setup all the shares from scratch... As only a few files were out of sync, there weren't so many conflicts. I even have the feeling the owncloud client is faster, but of course it may only be an impression. The only downside is that I have for now is a warning each time the client starts, telling me there may be incompatibilities.

On a side note, just a reminder, this is free/open source software. If you want a prompt response, you'd get it from a support contract from the nextcloud company. Not cheap, but we're not their target. We're getting this for free. I'm sad we don't get an answer either, but nobody's owing us anything. We could diagnose the problem ourselves, we have the code. Which I may do when I have spare time, which is not now :(

marco44 on 17 Jun 2019

❤1

I installed the ownCloud Client on one of my friends PC and it works flawlessly so far. Thanks for the tip! It was already kind of embarrassing having this issue after I convinced friends and family to switch over to Nextcloud from proprietary software...

0x47 on 17 Jun 2019

@Ox47 I feel the same. NC on Ubuntu is a real alternative to Dropbox, Google Drive and the like.

Shame on the Windows client that prevents the useage in family, friends.

I was alsoup to install NC as a save platform to exchange data with my students in school. Without a stable Windows client I can't proceed.

Would you be so kind to keep us up to date on your experience with the Owncloud client?

Thank you

Serverfox on 18 Jun 2019

Why don't you guys use the Nextcloud Client v2.3.3?
This one does not suffer from this issue, at least in my case.

valentingc on 18 Jun 2019

👍2

@valentingc Found v2.3.3.1 under https://download.nextcloud.com/desktop/releases/Windows/
and it works fine with or without Umlaut. Thank you very much for your help.

Are there known limitations in this version?

Serverfox on 19 Jun 2019

@Serverfox Not as far as I know. I've been using it daily basically since it was released (tried the new versions out in between but made a downgrade because of this issue). Works fine for me with the latest stable Nextcloud version (16.0.1). However, this is of course only a temporary fix in the long run.

valentingc on 19 Jun 2019

@valentingc Let's cross fingers.

Serverfox on 19 Jun 2019

@Serverfox Since switching to the ownCloud client I have had no issues on Windows as well.

Because of this issue, I tried the ownCloud client on Linux as well to tackle #1205. Since then, I don't have any crashes on Linux any more.

It seems that maybe choosing Nextcloud over ownCloud was a premature decision and I need to re-evaluate in the long-term.

0x47 on 19 Jun 2019

Seems as these issues are related to the client v2.5.x.

BTW the NC server is running without fault since 1st installation. That's the reason I'm disapointed by the client, no by NC as such.

Serverfox on 19 Jun 2019

Why don't you guys use the Nextcloud Client v2.3.3?
This one does not suffer from this issue, at least in my case.

Works perfectly since a couple of weeks, Thanks much!!

Serverfox on 5 Jul 2019

I don't recommend using an old version of NextCloud Client released almost 2 years ago, unless you are compiling it from sources. NextCloud Client relies on libraries such as OpenSSL 1.1.x, QtKeychain, Qt 5.x.x and zlib, which may contain severe vulnerabilities discovered since 2017 (or even in NextCloud Client itself). Your best option is to use latest Owncloud Client if you care about security.

papjul on 5 Jul 2019

@papjul Thanks for your advice. I swapped over to the Owncloud client, too. Works fine so far. Hoping the incompatibility warning concerning the NC Server won't do any harm on the long term.

Serverfox on 5 Jul 2019

I'm also hit by this problem. What I found is, that moving the file causing the error out of the folder (on to the desktop i.e.) waiting for the sync to go through and then putting the same file back, makes the sync work again. Until it's modified again and the problem starts over...

In case it helps I uploaded a Sync Client Log showing the issue
nextcloud-log-obfuscated-1907022.log

FreeMinded on 22 Jul 2019

@karlitschek can you have someone look at this? This is a serious problem. Thanks!

FreeMinded on 22 Jul 2019

Hi @FreeMinded I was made aware of your request. I am trying to reproduce it first. Will let everyone know if I find the problem.

camilasan on 23 Jul 2019

Hi @camilasan thanks a lot! Let me know if I can help you with something, testing, providing logs, samples, test account...

FreeMinded on 23 Jul 2019

Hi @FreeMinded, I couldn't reproduce the problem. I tried against Nextcloud 16 and 17. Could you provide a testing account and one of the big files that are not getting synced (if that is possible at all)? Thanks.

camilasan on 24 Jul 2019

@camilasan thanks! I just sent you an email.

FreeMinded on 25 Jul 2019

@camilasan If you're not able to reproduce after help from Freeminded, let me know. I can provide some reproducible files.

@papjul is correct, the issues seem to be related to special characters in general, not just umlauts.

Some additional details:

I'm having an almost identical issue with this filename:

Fejká - Moonlight.mp3

This occurs on both Linux and Windows with the same exact file for me.

On the off chance everyone else is using external storage, I'm utilizing an S3 Backend.

A potentially related issue is:

https://github.com/nextcloud/server/issues/7762

rtizzy on 25 Jul 2019

@camilasan If you're not able to reproduce after help from Freeminded, let me know. I can provide some reproducible files.

Please, that would be helpful.

On the off chance everyone else is using external storage, I'm utilizing an S3 Backend.

What do you mean? The problem occur with everyone using external storage?

camilasan on 25 Jul 2019

I'm using local storage and I am affected by this issue.

danielrheinbay on 25 Jul 2019

👍4

@camilasan I thought I'd mention my backend on the off chance others are using external backends (Or S3 in particular)

Sounds like others with local back ends are having the issue as well.

I just sent an email your way with access credentials and a reproducible file, let me know if you have any questions.

rtizzy on 25 Jul 2019

❤1 👍1

Hi All, I am sorry, but at the moment the error is not reproducible yet. I will have to get back to it later.

camilasan on 13 Aug 2019

@camilasan If you have difficulty reproducing this, you may consider syncing a huge file, e.g., >1G over a relatively slow network (e.g., 500KB/s). This is close to my case.

jeffli678 on 16 Aug 2019

👍1

@jeffli678 I've hit this issue with a 80MB file and a 200Mbps upload speed internet ;)

r3pek on 16 Aug 2019

@r3pek Then I guess there might be multiple ways to trigger this issue.

Moreover, I searched the code base for the error message (Server is unable to maintain the header compression context for the connection) and it is not there. Does it actually come from the server?

Update: It seems the "Server is unable to maintain the header compression context for the connection" error message comes from the HTTP/2, not the nextcloud. (https://httpwg.org/specs/rfc7540.html)

However, interestingly, if you google this error message, you find nextcloud is bothered the most.

jeffli678 on 16 Aug 2019

Now that you are all talking about files with big sizes, issue occured to me mainly with music files.
@rtizzy also gave an example with .mp3 extension, so we have 4 different examples happening when files have a big size (let's say >1MB).
Someone would have to test with a simple text file to see if it can be reproduced with smaller files.

Upload speed doesn't seem to matter, I don't have a slow connection (though it could be considered slow by American standards).

papjul on 16 Aug 2019

A bit extra information: after I connect to my nextcloud server via a proxy, the issue is gone and the files are synced properly! The interesting bit is, the proxy server is also my nextcloud server, so I suggest this issue is network in nature.

jeffli678 on 16 Aug 2019

i have same issue

vgdh on 26 Aug 2019

👎1

I can chime in on the issue.
I have 1 Mac client and 7 windows clients.
the Mac do not have any issues, and 6 out of 7 Windows clients are having issues with no sync and getting the error "An error occured while opening a folder" and sync status just halt with "waiting", I am unable to see if the folder is located on the client or the server. I have been running Nextcloud the last 4 years, and this issue started during the summer. First thing i did, was to upgrade the server from 13.x to the latest 16.0.4. I am running with the Danish language which has a bunch of special characters. Maybe its that, or maybe its the fact that I am having a NGINX reverse proxy serving the nextcloud to both LAN and WAN. Help is very much appreciated.

kulmosen on 27 Aug 2019

same issue occured when overwriting a photo files which are at least 30MB and contains chinese characters, the desktop log says follows with no error found on server side.

By the way, 2.5.0 works fine
Changing DNS is not useful

[OCC::AbstractNetworkJob::start     OCC::MoveJob created for "https://[REMOVED]" + "" "OCC::PropagateUploadFileNG"
[OCC::ComputeChecksum::start    Computing "" checksum of "[REMOVED]/TIF/20180929-杨以后地铁人像-003.tif" in a thread
[OCC::AccessManager::createRequest  6 "PROPFIND" "https://[REMOVED]/remote.php/dav/uploads/[REMOVED]/344475665" has X-Request-ID "[REMOVED]"
[OCC::AbstractNetworkJob::start     OCC::LsColJob created for "https://242c.cc" + "" "OCC::PropagateUploadFileNG"
[OCC::WebFlowCredentials::slotFinished  request finished
[OCC::AbstractNetworkJob::slotFinished  QNetworkReply::ProtocolFailure "Server is unable to maintain the header compression context for the connection" QVariant(Invalid)
[OCC::WebFlowCredentials::stillValid    QNetworkReply::ProtocolFailure

Milokita on 6 Sep 2019

Same problem here (Archlinux, nextcloud-client 2.5.3-2) with pdf files greater than 10 MB.
Replacing nextcloud-client with owncloud-client (v 2.5.4.11654-1), partially solves the problem, but owncloud-client complains of conflicting files. It renames these files (the same as under nextcloud-client) "xxxxxx (conflicting copy 2019-06-18 133750) ...". Deleting these conflicting files resolves the problem.
Replace owncloud-client with nextcloud-client: the problem is gone! I just have to wait to see if this solution is sustainable ...

nicoroud on 18 Sep 2019

False joy ! with nextcloud-client, as soon as the big files are modified, the error returns.
I de-install and reinstall owncloud-client...

nicoroud on 18 Sep 2019

Did anyone try the new version 2.6.0?

Serverfox on 30 Sep 2019

Did anyone try the new version 2.6.0?

Tried, the client will leave the upload process hanging. sometimes it report server stopped accepting new stream

Milokita on 1 Oct 2019

@camilasan
here is a example file name for you
vu西雅图二哥人像-001.png
just change the number and create several big files. MySQL backend. overwrite the file with diff file with same name when upload is not complete will trigger it for sure

Milokita on 1 Oct 2019

TL;DR: Issue is from a difference of implementation between HTTP 1.1 and HTTP 2.0 in Qt Network library.

Full details:
I have little knowledge of this, I just tried to understand what was going on with various links, feel free to correct me if I'm wrong!

There is a "bug" in HTTP2 in Qt, as discussed in [1] and [2].
Owncloud put a workaround in v2.5.4 that I believe everyone on this thread installed.
Also, I remember reading that Owncloud client used HTTP 1.1 by default until recent versions, while Nextcloud client uses HTTP/2 since a longer time (I believe the working Nextcloud Client v2.3.3 uses HTTP 1.1).

After that, the bug was patched partially in [3] with Qt 5.12.4, however there is still missing some patches, hence why some people here reported having cancelled operations/stream issues with v2.6.0.
This should however be fixed in Owncloud client v2.6.0 as you can see in [6].

A 2nd Qt patch was pushed for Qt 5.12.5 in [2]. I'm not sure if it helps our case.

The bug may or may not be definitely fixed in Qt 5.15 to be released in Nov 2020, or a later Qt 5.12.x, as seen in [5].
However the Qt version depends on the machine compiling the Nextcloud client, so the Nextcloud team needs to bump the minimum version to the correct Qt version as soon as it is availabe, and update compile tools for the binary versions.
In any case, if HTTP/2 must be used, it should be with Qt v5.12.4+ and a backported patch from owncloud-client.

As an immediate working workaround, I disabled HTTP/2 on my server.
This results in slow folder scanning, and poor sync performances, but Nextcloud client is working now (or at least I was able to confirm that a few files that never synced before are able to sync now).
I still have some timeouts but this is probably due to poor connection.

I did not manage to make Owncloud Client 2.6.0rc1 work with HTTP/2 despite the patch.
This may be due to incompatibility between server and client, or because this version is not stable yet (there are some patches pending for rc2 release).
I received the same issues as noted by @Milokita.

Sources:
[1] https://github.com/owncloud/client/issues/7020
[2] https://github.com/owncloud/client/issues/7174
[3] https://bugreports.qt.io/browse/QTBUG-73947 Initial bug fixed in Qt v5.12.4+
[4] https://bugreports.qt.io/browse/QTBUG-77852 Second patch released in Qt v5.12.5
[5] https://bugreports.qt.io/browse/QTBUG-75573 It will apparently be fixed in 5.15 since it is considered feature request
[6] https://github.com/owncloud/client/commit/677e44dc532ef2035f6dc5b3f45c527f6d1b0be2 (workaround for 2nd bug in owncloud client, needs Qt 5.12.4+)

Mentioning @camilasan to let them know that patches can be backported from owncloud client.
However, as mentioned above, this didn't work for me but I only lightly tested it.

So what I would do first is to switch back to HTTP 1.1 when Qt < 5.12.4 is used and/or provide a way to disable HTTP/2 (owncloud did this). Because disabling HTTP/2 on server side means that you need to disable HTTP/2 for all websites you host (you cannot have different HTTP versions on the same port), so it's better to disable it from client side.

papjul on 1 Oct 2019

TL;DR: Issue is from a difference of implementation between HTTP 1.1 and HTTP 2.0 in Qt Network library.

Full details:
I have little knowledge of this, I just tried to understand what was going on with various links, feel free to correct me if I'm wrong!

There is a "bug" in HTTP2 in Qt, as discussed in [1] and [2].
Owncloud put a workaround in v2.5.4 that I believe everyone on this thread installed.
Also, I remember reading that Owncloud client used HTTP 1.1 by default until recent versions, while Nextcloud client uses HTTP/2 since a longer time (I believe the working Nextcloud Client v2.3.3 uses HTTP 1.1).

After that, the bug was patched partially in [3] with Qt 5.12.4, however there is still missing some patches, hence why some people here reported having cancelled operations/stream issues with v2.6.0.
This should however be fixed in Owncloud client v2.6.0 as you can see in [6].

A 2nd Qt patch was pushed for Qt 5.12.5 in [2]. I'm not sure if it helps our case.

The bug may or may not be definitely fixed in Qt 5.15 to be released in Nov 2020, or a later Qt 5.12.x, as seen in [5].
However the Qt version depends on the machine compiling the Nextcloud client, so the Nextcloud team needs to bump the minimum version to the correct Qt version as soon as it is availabe, and update compile tools for the binary versions.
In any case, if HTTP/2 must be used, it should be with Qt v5.12.4+ and a backported patch from owncloud-client.

As an immediate working workaround, I disabled HTTP/2 on my server.
This results in slow folder scanning, and poor sync performances, but Nextcloud client is working now (or at least I was able to confirm that a few files that never synced before are able to sync now).
I still have some timeouts but this is probably due to poor connection.

I did not manage to make Owncloud Client 2.6.0rc1 work with HTTP/2 despite the patch.
This may be due to incompatibility between server and client, or because this version is not stable yet (there are some patches pending for rc2 release).
I received the same issues as noted by @Milokita.

Sources:
[1] [owncloud/client#7020](https://github.com/owncloud/client/issues/7020)
[2] [owncloud/client#7174](https://github.com/owncloud/client/issues/7174)
[3] https://bugreports.qt.io/browse/QTBUG-73947 Initial bug fixed in Qt v5.12.4+
[4] https://bugreports.qt.io/browse/QTBUG-77852 Second patch released in Qt v5.12.5
[5] https://bugreports.qt.io/browse/QTBUG-75573 It will apparently be fixed in 5.15 since it is considered feature request
[6] [owncloud/client@677e44d](https://github.com/owncloud/client/commit/677e44dc532ef2035f6dc5b3f45c527f6d1b0be2) (workaround for 2nd bug in owncloud client, needs Qt 5.12.4+)

Mentioning @camilasan to let them know that patches can be backported from owncloud client.
However, as mentioned above, this didn't work for me but I only lightly tested it.

So what I would do first is to switch back to HTTP 1.1 when Qt < 5.12.4 is used and/or provide a way to disable HTTP/2 (owncloud did this). Because disabling HTTP/2 on server side means that you need to disable HTTP/2 for all websites you host (you cannot have different HTTP versions on the same port), so it's better to disable it from client side.

thanks for the detailed study, since I have unistall NC client so I can't test by disable http2 yet. Browsing through apache2's log however shows that the client is talking http1.1
"PROPFIND /remote.php/dav/files/[REMOVED] HTTP/1.1" 207 4615 "-" "Mozilla/5.0 (Windows) mirall/2.5.4

Milokita on 2 Oct 2019

thanks for the detailed study, since I have unistall NC client so I can't test by disable http2 yet. Browsing through apache2's log however shows that the client is talking http1.1
"PROPFIND /remote.php/dav/files/[REMOVED] HTTP/1.1" 207 4615 "-" "Mozilla/5.0 (Windows) mirall/2.5.4

What do you mean by "however"?
Owncloud Client 2.5.4 is using HTTP/1.1 and it is not working?
HTTP/1.1 has no issue, it's HTTP/2 (which NC Client is using).

papjul on 2 Oct 2019

Sorry, I mean that NC 2.5.4 is not working. Though I have the HTTP2 enabled, the log shows NC us still working in HTTP1.1 OC is working very well ------------------ 原始邮件 ------------------
发件人: "Julien Papasian"notifications@github.com
发送时间: 2019年10月2日(星期三) 下午2:01
收件人: "nextcloud/desktop"desktop@noreply.github.com;
抄送: "XNG"flipwing@qq.com;"Mention"mention@noreply.github.com;
主题: Re: [nextcloud/desktop] Error when uploading changes on files withumlauts (#1182)

thanks for the detailed study, since I have unistall NC client so I can't test by disable http2 yet. Browsing through apache2's log however shows that the client is talking http1.1
"PROPFIND /remote.php/dav/files/[REMOVED] HTTP/1.1" 207 4615 "-" "Mozilla/5.0 (Windows) mirall/2.5.4
What do you mean by "however"?
Owncloud Client 2.5.4 is using HTTP/1.1 and it is not working?
HTTP/1.1 has no issue, it's HTTP/2 (which NC Client is using).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Milokita on 2 Oct 2019

Sorry, I mean that NC 2.5.4 is not working. Though I have the HTTP2 enabled, the log shows NC us still working in HTTP1.1 OC is working very well

It is a log from your Owncloud Client.
There is no Nextcloud Client 2.5.4, latest in 2.5 series was 2.5.3.
This confirms your Owncloud Client works with HTTP 1.1.

papjul on 2 Oct 2019

Sorry, I mean that NC 2.5.4 is not working. Though I have the HTTP2 enabled, the log shows NC us still working in HTTP1.1 OC is working very well

It is a log from your Owncloud Client.
There is no Nextcloud Client 2.5.4, latest in 2.5 series was 2.5.3.
This confirms your Owncloud Client works with HTTP 1.1.

You are correct, OC 2.5.4 have HTTP2 disabled due to that bug which OC 2.6.0RC1 added a patch & HTTP2 enabled. I will try the new version.

NC 2.5.0 using Qt 5.11.1 SSL 1.0.1h, does not have the problem; apache log says it uses HTTP1.1
"GET /ocs/v2.php/apps/notifications/api/v2/notifications?format=json HTTP/1.1" 304 461 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)

OC 2.6.0RC1 uses Qt5.12.4 SSL 1.1.1. It runs HTTP2 and it's working
[02/Oct/2019:18:18:12 +0800] "PROPFIND /remote.php/dav/files/[REMOVED] HTTP/2.0" 207 373 "-" "Mozilla/5.0 (Windows) mirall/2.6.0rc1 (build 12357)"

NC 2.6.0 uses Qt 5.12.5 SSL 1.1.1d, also can run HTTP2 but stucks again.
"PROPFIND /remote.php/dav/files/REMOVED/ HTTP/2.0" 207 495 "-" "Mozilla/5.0 (Windows) mirall/2.6.0stable-Win64 (build 20190927) (Nextcloud)"
when I switch off http2, the NC 2.6.0 reports
QNetworkReply::ProtocolFailure "invalid frame size"
and server log constantly reports
127.0.0.1 - - [02/Oct/2019:18:44:22 +0800] "PRI * HTTP/2.0" 400 3760 "-" "-"

Milokita on 2 Oct 2019

Sorry, didn't see you edited your comment.

OC 2.6.0RC1 uses Qt5.12.4 SSL 1.1.1. It runs HTTP2 and it's working
[02/Oct/2019:18:18:12 +0800] "PROPFIND /remote.php/dav/files/[REMOVED] HTTP/2.0" 207 373 "-" "Mozilla/5.0 (Windows) mirall/2.6.0rc1 (build 12357)"

Then, it works because it contains the patch that NC doesn't have. Mine doesn't work but maybe because I have Qt 5.12.5 which contains an additional patch that is messing with the OC patch.

NC 2.6.0 uses Qt 5.12.5 SSL 1.1.1d, also can run HTTP2 but stucks again.
"PROPFIND /remote.php/dav/files/REMOVED/ HTTP/2.0" 207 495 "-" "Mozilla/5.0 (Windows) mirall/2.6.0stable-Win64 (build 20190927) (Nextcloud)"
when I switch off http2, the NC 2.6.0 reports
QNetworkReply::ProtocolFailure "invalid frame size"
and server log constantly reports
127.0.0.1 - - [02/Oct/2019:18:44:22 +0800] "PRI * HTTP/2.0" 400 3760 "-" "-"

You need to close NC after switching HTTP2 to HTTP 1.1 on the server. NC still thinks he is using HTTP2 otherwise.

papjul on 6 Oct 2019

On NC 17.0.0 logs show this:
Sabre\DAV\Exception\BadRequest: Expected filesize of 10000000 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1048576 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

r3pek on 7 Oct 2019

On NC 17.0.0 logs show this:
Sabre\DAV\Exception\BadRequest: Expected filesize of 10000000 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1048576 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

@r3pek I didn't install Nextcloud 17.0 yet.
Are you saying that the workaround described above doesn't work anymore with Nextcloud 17.0?
The filesize of "10000000 bytes" looks like an issue with a configuration size limit, though.

papjul on 7 Oct 2019

@papjul haven't tested the HTTP2/1.1 issue (yet), but the workaround mentioned earlier (moving the file to another folder and moving it back again) does still work.
The thing is, the files I'm testing this issue with, none of them have 10M(i)B. They are all around 80 to 110 MiB

r3pek on 7 Oct 2019

Sorry, didn't see you edited your comment.

OC 2.6.0RC1 uses Qt5.12.4 SSL 1.1.1. It runs HTTP2 and it's working
[02/Oct/2019:18:18:12 +0800] "PROPFIND /remote.php/dav/files/[REMOVED] HTTP/2.0" 207 373 "-" "Mozilla/5.0 (Windows) mirall/2.6.0rc1 (build 12357)"

Then, it works because it contains the patch that NC doesn't have. Mine doesn't work but maybe because I have Qt 5.12.5 which contains an additional patch that is messing with the OC patch.

NC 2.6.0 uses Qt 5.12.5 SSL 1.1.1d, also can run HTTP2 but stucks again.
"PROPFIND /remote.php/dav/files/REMOVED/ HTTP/2.0" 207 495 "-" "Mozilla/5.0 (Windows) mirall/2.6.0stable-Win64 (build 20190927) (Nextcloud)"
when I switch off http2, the NC 2.6.0 reports
QNetworkReply::ProtocolFailure "invalid frame size"
and server log constantly reports
127.0.0.1 - - [02/Oct/2019:18:44:22 +0800] "PRI * HTTP/2.0" 400 3760 "-" "-"

You need to close NC after switching HTTP2 to HTTP 1.1 on the server. NC still thinks he is using HTTP2 otherwise.

thanks for the help anyway, I moved to OC for now and pretty happy about it.

Milokita on 9 Oct 2019

❤5

It appears that the new 2.6.1-rc1 have fixed the problem. @papjul

Milokita on 18 Oct 2019

As @Milokita already pointed out, the current rc was built with Qt5.12.5. and should fix this issue. The same applies for the daily builds at https://download.nextcloud.com/desktop/daily/ , which can also be tested agains this. Qt5.12.5 is the standard all our CD builds now.

DominiqueFuchs on 31 Oct 2019

As @Milokita already pointed out, the current rc was built with Qt5.12.5. and should fix this issue. The same applies for the daily builds at https://download.nextcloud.com/desktop/daily/ , which can also be tested agains this. Qt5.12.5 is the standard all our CD builds now.

By far I tried to upload with some 50MB zip files which is in total 1.5GB containing Chinese characters with daily build on 20191031 Qt version 5.12.5 and it works fine.
However, making the change to those file and the new sync would stuck and the end, could be my server's problem

Milokita on 1 Nov 2019

As @Milokita already pointed out, the current rc was built with Qt5.12.5. and should fix this issue. The same applies for the daily builds at https://download.nextcloud.com/desktop/daily/ , which can also be tested agains this. Qt5.12.5 is the standard all our CD builds now.

By far I tried to upload with some 50MB zip files which is in total 1.5GB containing Chinese characters with daily build on 20191031 Qt version 5.12.5 and it works fine.
However, making the change to those file and the new sync would stuck and the end, could be my server's problem

No, this is normal.
As explained before:

if HTTP/2 must be used, it should be with Qt v5.12.4+ and a backported patch from owncloud-client.

You're missing the 2nd condition, the "backported patch" from here: https://github.com/owncloud/client/commit/677e44dc532ef2035f6dc5b3f45c527f6d1b0be2
Or we can wait for a Qt patch here: https://bugreports.qt.io/browse/QTBUG-75573
But this might take a while…
Please also note that the version of OC client including this "new" patch (with Qt 5.12.5) is 2.6.0 but this didn't work with me (although this might because I didn't try enough, or a different issue).

This patch resumes uploading when it fails. In other words, without it, you're stuck at the same position as before, although there may be some files that succeed to upload in one shot after some retries. However, this is very painful, and I highly suggest to follow my advice to disable HTTP 2 on your server. I do this since I suggested it and it works like a charm. Performances are actually not that worse, unlike what I originally stated.

If you need more details, please re-read my comment:
https://github.com/nextcloud/desktop/issues/1182#issuecomment-537246257

papjul on 1 Nov 2019

As @Milokita already pointed out, the current rc was built with Qt5.12.5. and should fix this issue. The same applies for the daily builds at https://download.nextcloud.com/desktop/daily/ , which can also be tested agains this. Qt5.12.5 is the standard all our CD builds now.

By far I tried to upload with some 50MB zip files which is in total 1.5GB containing Chinese characters with daily build on 20191031 Qt version 5.12.5 and it works fine.
However, making the change to those file and the new sync would stuck and the end, could be my server's problem

No, this is normal.
As explained before:

if HTTP/2 must be used, it should be with Qt v5.12.4+ and a backported patch from owncloud-client.

You're missing the 2nd condition, the "backported patch" from here: owncloud/client@677e44d
Or we can wait for a Qt patch here: https://bugreports.qt.io/browse/QTBUG-75573
But this might take a while…
Please also note that the version of OC client including this "new" patch (with Qt 5.12.5) is 2.6.0 but this didn't work with me (although this might because I didn't try enough, or a different issue).

This patch resumes uploading when it fails. In other words, without it, you're stuck at the same position as before, although there _may_ be some files that succeed to upload in one shot after some retries. However, this is very painful, and I highly suggest to follow my advice to disable HTTP 2 on your server. I do this since I suggested it and it works like a charm. Performances are actually not that worse, unlike what I originally stated.

If you need more details, please re-read my comment:
#1182 (comment)

FYI, the patch is applied and the build is successful, waiting for merge into master
https://github.com/nextcloud/desktop/pull/1573

Milokita on 1 Nov 2019

🎉1

My understanding was that the problem is resolved in the 2.6.1 build of NC client? I am still experiencing the same issue. Anyone else too? FYI: I am using NC behind a traefik reverse proxy.

kosli on 8 Nov 2019

I'm seeing this as well, using the same setup @kosli is reporting, though I'm on client 2.6.0, and the autoupdater is reporting that I'm up to date. The file in question has a circumflex in the file name (ê), not an umlaut though.

tstackhouse on 8 Nov 2019

@tstackhouse try to update the client manually by downloading from nextcloud.com -> I think I never saw the client showing me an update is being available.
Whereas it seems I was able to resolve the issues with the 2.6.1 client -> i had to move the affected files out of the NC directory and back into it and since then I don't see that issue anymore. Let's see if that is not just a temporary fix...

kosli on 8 Nov 2019

👍1

@kosli Manual update did get me to 2.6.1, but it's still happening. My server is on 16.0.5, and I even tried changing my hosts file to bypass Traefik, but it seems like it might still be doing it.

tstackhouse on 10 Nov 2019

Also adding that I have a path that has an ellipsis character (…) in it, that is also seeing this same issue.

tstackhouse on 11 Nov 2019

I just tested this issue with client 2.6.1 on Windows 10 an NC 16.0.5. It's still happening...
Files with umlauts in name and/or path fail to upload after modification to the file. Once the exact same file is moved out and back in to the synchronized folder, the upload works fine.

Error message
"[OCC::ActivityWidget::slotItemCompleted Item "path/filename" retrieved resulted in "Server is unable to maintain the header compression context for the connection"

If there is anything I can do to help fix this issue, let me know.

FreeMinded on 12 Nov 2019

👍3

@camilasan is there any efforts going on to fix this issue? Could you please give us a heads up. Thanks!

FreeMinded on 25 Nov 2019

I can confirm this problem, this is also an issue on OSX systems. Moving the files outside a sync folder and back in "solves" the issue, but that's not quite how it's supposed to be.

msbt on 27 Nov 2019

I confirm this issue on NC 17.0.1 on NGINX on arch (thus all latest release versions).

I saw a red server error (sorry, cannot remember the error text) for that file in the sync client. After setting loglevel to 4, I see this in nexcloud.log:

BadRequest Expected filesize of ##### bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) ##### bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

I couldn't believe that this should have to do with Umlauts but just renaming the "ä" in the filename to "ae" led to the client syncing it fine. Renaming it back also worked (presumably because no data had to be written). It seems that it only happens for files for a certain size, though. My file in question was a .odp of 12MB.

Strange:

Before renaming anything, the PDF export of that presentation with the same "ä" in its file name (1.4MB) initially didn't sync but did after a while without me doing anything.
After the rename/sync/rename-back, I created a local coply of that file. It contained just the same "ä" but was synced immediately without any issues.

nursoda on 29 Nov 2019

Same issue here.
We are using Windows 10 (current version) with up-to-date Nextcloud server & client.

derritter88 on 10 Dec 2019

Same issue here with 2.6.1 on Arch Linux. Renaming the file worked fine though...

cunio-martin on 13 Dec 2019

Might that be some multibyte correlation? My mariadb was set up only 3 months ago, with multibyte support.

nursoda on 15 Dec 2019

I'm also using utf8mb4 on my MariaDB database. There definitely is/was an issue with HTTP2 in Qt though since it's not reproducible with HTTP 1, so it might just be with any character not in ASCII table.

If you're still affected by the issue, I recommend that you use the workaround to rename with another filename then back with the original name, or just use HTTP 1.x. Also, if you have lot of (conflicted copy), use regexp to replace them back to the original file, in bulk. Unless you have a PR ready, that's the best thing to do because no one fixed our (critical) issue for months when there was no workaround, so now there is one, just use it.

papjul on 15 Dec 2019

We're observe the same problem on our OSX clients using the web interface. I suspect that this might be related to the way Apple filesystems encode Umlauts. Apple is using NFD (Normalization Form Canonical Decomposition) while everybody else is using NFC (Normalization Form Canonical Composition). This is a problem I've also observed with SFTP transfers between OSX and Unix clients.

Cybso on 14 Jan 2020

Ubuntu 19, NextCloud client 2.6.2-20191224.111326~eoan1.

The problem is still here, I cannot sync my data successfully.

Any updates on the possible fix?

It doesn't seem to be server-side since Windows client works just fine. Even with enormous files (1GB+).

dodancs on 1 Feb 2020

Having similar problems with "ö" or "ä" or "ü" when updating files via sync client.

macOS 10.14.6 using NextCloud Client 2.6.2 and 2.7 Beta
Server version 17 (update to 18 did not fix it)
Server seems not affected. Webupload also unaffected. So just a client problem.
Server log shows nothing.

Client responds with "Error transferring https://DOMAIN/remote.php/dav/uploads/USER/4191192125/.file - server replied: (File with name //Auftrge could not be" and below the Folder path "Aufträge/XY - FOLDER NAME/2.1 Entwürfe/"

All the special characters like "-" or "." and space work fine.
It is just the Umlauts "äöü" causing problems.

found a temporal fix: downgrade client to 2.5.3
This ist the last unaffected version.

ArchiPOS on 1 Feb 2020

Hey everyone.
I gave this another shot and have been able to safely reproduce the issue where changing a file with umlauts results in an GUI error, not syncing the file and spawning log messages regarding the final MOVE instruction of the change synchronization. This was a bug with the header definition/encoding when pushing out the corresponding DAV request and is fixed in PR #1768

As it is well possible that this issue contains more than one bug or circumstances where umlauts can lead to failures, testing is necessary and would be very welcome. However, this specific bug was squashed.

Edit: Forgot to mention that this PR is now in review - if everything is fine, I guess we're backporting this to stable very soon.

DominiqueFuchs on 2 Feb 2020

Good catch!
It is already part of Owncloud: https://github.com/owncloud/client/commit/87a6e039a734f0bf17a5e8188ba2f1f82fae6492#diff-f9f22ece9a01d2e1cd06e3d8a088209b
But it was used to fix a bug when uploading large files with a "%" in its name: https://github.com/owncloud/client/issues/7176
Not sure it covers all cases like you said, but we will see ;)

papjul on 2 Feb 2020

owncloud reported bug in Qt again causing "Server stopped accepting new streams before this stream was established" & disabled HTTP2.0
https://github.com/owncloud/client/issues/7610

Milokita on 5 Feb 2020

@Milokita Please lets not make this a everything-HTTP2 thread, as I can‘t see any relationship between the owncloud issue you linked and the umlaut sync issue this one cares about.

Do you experience the issue regarding network stream blocking mentioned in the OC link? If yes, please use a separate issue for this. Thanks!

DominiqueFuchs on 5 Feb 2020

The link between the two issues is that the first fix was to move to Qt v5.12.4 and integrate the 1st PR from @Milokita. It was supposed to fix the issue but a regression seems to have been introduced in Qt 5.12.5, through what appears to be a bugfix, making the fix useless (the Qt bug was mentioned earlier in this thread).
Everything is connected together and I came to the same conclusion as made in Owncloud bug referenced: disabling HTTP2 makes everything work flawlessly.

So yes, we can make a separate bug more generic on all HTTP2 issues on which this bug can depend since it's not reproduceable with HTTP 1.

Everything is tied together around HTTP 2 and Qt. Quoting from the owncloud bug:

the QT implemrntation of H2 has been broken since forever. There's always: "ah, there's this bug, which will be fixed in QT x.x.x", then it takes forever to switch the client to use that QT version and then it starts all over.

Which sums up approximately what we've been doing the past year on our bug (which is more like a "sync issues when HTTP2 is enabled" bug).

@Milokita Can you open a new bug with your findings? I disabled HTTP2 long time ago, it will not be very helpful if I open it myself

papjul on 5 Feb 2020

I understand the issue about HTTP/2, what I'm saying is that this thread (titled 'Error when uploading changes on files with umlauts') hinted on missing code fixes (#1768) that are directly tied to _changes_ on existing files no matter the size and HTTP/2. This is what I'd like to review here as soon as the PR is merged and backported. Thanks for creating a new issue regarding the latter topic!

DominiqueFuchs on 5 Feb 2020

Having the "Could either be a network problem on the sending side or a problem writing to the storage on the server side." issue on a HTTP/1.1 server. Server 17.0.3, client 2.6.2 on macOS. The client thinks the server is HTTP/2 when I look at the SSL padlock info. Curl verifies that the server listens on HTTP/1.1.

realies on 16 Feb 2020

As the http2 will be disabled by default in next release (manual override is possible) I think this problem would be fixed by now. https://github.com/nextcloud/desktop/pull/1825

Milokita on 2 Mar 2020

As the http2 will be disabled by default in next release (manual override is possible) I think this problem would be fixed by now. #1825

When is the new release available for download, I struggling syncing most of my pictures because of syncing EXIF data by Digikam. This triggered 30.000 pictures do be uploaded, some folders with umlaute and some panorama files bigger than 20MB.

Thanks

feutl on 2 Mar 2020

As the http2 will be disabled by default in next release (manual override is possible) I think this problem would be fixed by now. #1825

When is the new release available for download, I struggling syncing most of my pictures because of syncing EXIF data by Digikam. This triggered 30.000 pictures do be uploaded, some folders with umlaute and some panorama files bigger than 20MB.

Thanks

As the fix have been merged into master branch, you can try the latest daily build https://download.nextcloud.com/desktop/daily/

Milokita on 3 Mar 2020

@Milokita
Thanks, I looked there yesterday already, but because they are all 2.7 and the Milestone is 2.6.4 - I was not sure if this would work.
Thanks I am going to test this right now :)

feutl on 3 Mar 2020

I tested versions from today and yesterday on Ubuntu via AppImage, they do start but I am unable to get to the GUI to see any outcome or results.

feutl on 3 Mar 2020

Hey @feutl,

could you try a special 2.6 build I've just created? (Current state of the stable-2.6 branch, including the HTTP/2 fix.):

https://download.nextcloud.com/desktop/daily/Linux/Nextcloud-stable-2.6.20200303-test-1-x86_64.AppImage

misch7 on 3 Mar 2020

Hey @feutl,

could you try a special 2.6 build I've just created? (Current state of the stable-2.6 branch, including the HTTP/2 fix.):

https://download.nextcloud.com/desktop/daily/Linux/Nextcloud-stable-2.6.20200303-test-1-x86_64.AppImage

Testing right now, the AppImage starts with GUI so I am able to start the sync. I report back after a while.

One thing is quite curious, the AppImage asked for big folders if they should be synced, even they were being set to be synced with the previous official 2.6 build. lets see how this works out.

feutl on 3 Mar 2020

It looks quite promising already, around 300GB (10 hours) to go and 95000 files.

Lots of files raise the "file has changed since discovery" notification in the activity log, I need to wait until all of this gets resolved, but the build is already more stable than the official 2.6 build.

feutl on 3 Mar 2020

Thanks for testing @feutl 👍

I'll build the 2.6.4 release then finally :)

misch7 on 3 Mar 2020

The sync worked well the last hours but for some reason it created thousands of . (hidden) files like .IMG_7974-export.jpg.~6279f317 and filled up my disk. I have no clue why, could that be related to the client or just bad luck?
But those files are only stored locally, nothing on Nextcloud itself, also no files got deleted so far on the NC Server, if I can rely on the NC "trash" :)

feutl on 3 Mar 2020

Still have some connection closed errors, but much much much more stable.
What I synced last few hours was 100times more than with the official 2.6 in a few days.

feutl on 3 Mar 2020

The sync worked well the last hours but for some reason it created thousands of . (hidden) files like .IMG_7974-export.jpg.~6279f317 and filled up my disk. I have no clue why, could that be related to the client or just bad luck?
But those files are only stored locally, nothing on Nextcloud itself, also no files got deleted so far on the NC Server, if I can rely on the NC "trash" :)

These files are the temp files created when syncing, it should be gone after the file is synced with server

Milokita on 4 Mar 2020

The 2.6.4 release is out:
https://github.com/nextcloud/desktop/releases/tag/v2.6.4

@feutl Strange issue with the temp files; did you try to reboot the system just to be sure?

misch7 on 4 Mar 2020

@Milokita
Could be possible they got created by the old Nextcloud Desktop Client which struggled downloading the files and lost the connection much too often.

@misch7
I am still seeing Connection closed (skipped due to earlier error) errors in my setup, but it also says retrying in about 50 minutes. I have the log output window open, anything specific I should look at to resolve the last pieces?

update: added some debug information
searched for one file which raises the above error and this is what I found

[_csync_detect_update   file: path2017-08-21T16_05_32.jpg, instruction: INSTRUCTION_NEW <<=
[_csync_merge_algorithm_visitor     INSTRUCTION_CONFLICT           server file: path/2017-08-21T16_05_32.jpg
[OCC::SyncEngine::checkErrorBlacklisting    blacklist entry for  "path/2017-08-21T16_05_32.jpg"  has expired!
[OCC::SyncEngine::deleteStaleDownloadInfos  Deleting stale temporary file:  "local path/.2017-08-21T16_05_32.jpg.~6fcdc4a5"
[OCC::PropagateItemJob::scheduleSelfOrChild     Starting INSTRUCTION_CONFLICT propagation of "path/2017-08-21T16_05_32.jpg" by OCC::PropagateDownloadFile(0x557aecd33b30)
[OCC::AccessManager::createRequest  2 "" "https://cloud.feutl.com/remote.php/dav/files/user/path/2017-08-21T16_05_32.jpg" has X-Request-ID "a240f0fa-cd60-4d60-a988-fee0622d0903"
[OCC::AbstractNetworkJob::start     OCC::GETFileJob created for "https://cloud.feutl.com" + "path/2017-08-21T16_05_32.jpg" "OCC::PropagateDownloadFile"
[OCC::SyncJournalDb::setErrorBlacklistEntry     Setting blacklist entry for  "path/2017-08-21T16_05_32.jpg" 3 "Connection closed" 1583308327 625 1538211017 "4492465e42c59f923ad970c8d7f430d7" "" 0
[OCC::blacklistUpdate   blacklisting  "path/2017-08-21T16_05_32.jpg"  for  625 , retry count  3
[OCC::PropagateItemJob::done    Could not complete propagation of "path/2017-08-21T16_05_32.jpg" by OCC::PropagateDownloadFile(0x557aecd33b30) with status 10 and error: "Connection closed"
[OCC::ActivityWidget::slotItemCompleted     Item  "path/2017-08-21T16_05_32.jpg"  retrieved resulted in  "Connection closed"
[OCC::ActivityWidget::slotItemCompleted     Item  "path/2017-08-21T16_05_32.jpg"  retrieved resulted in error  "Connection closed"

feutl on 4 Mar 2020

I fixed most of the above errors manually by replacing the files reported with a version from my backup. still quite confusing why this happened, some files seemed broken, could have been generated by the errors with the old 2.6 release and the connection losses I had.

All seems good now on the Linux Client.
Windows Client updated as well, sync works great, some Connection closed errors which I try to fix after the rest was synced successfully.

feutl on 4 Mar 2020

👍1

Windows Client also synced 100GB without having problems. Thanks for finding the issue and providing a fix :+1:

feutl on 4 Mar 2020

🚀2

Desktop: Error when uploading changes on files with umlauts

Expected behaviour

Actual behaviour

Steps to reproduce

Client configuration

Most helpful comment

All 103 comments

Some additional details:

Related issues