Desktop: Issues with special chars on Windows and other operating systems

[fabianfrz]

The sync client may have issues with special chars like ":" on Windows. The character is perfectly valid on Linux/BSD systems and allowed by all file systems I know.

The sync client may respect OS limitations in file names by renaming them locally on Windows clients:

For reference - see: opnsense/core#2498

File name template I use in OPNsense for config files: config-<Hostname>.<Domain Name>-<Date>_<Time>.xml

Currently there seems to be no error message on Windows for this, see #1723.

[TwiggyWan]

How about having one list of disallowed characters for all the supported OS and enforcing that?

[fabianfrz]

bad idea, that blocks automation of legit files. there will be always one OS that causes troubles and sometimes you do not even know that os. In my opinion the desktop client should replace all occurrences of special characters by a replacement character like an underscore and handle that internally with a mapping file for example when using json for a .nextclound_filemappings.json in each directory which will be hidden on all operating systems if possible:

{
  "test:1234.json": "test_1234.json"
}

[TwiggyWan]

there will be always one OS that causes troubles and sometimes you do not even know that os
What does that mean? doesn't this project support a finite list of OSes?

Where is it written what characters are forbidden on the server?
I create a demo instance, go in the brower to create a file :

Okay, slashes forbidden, that makes sense.

but now i create 'test\iss'(or 'test\iss', same behavior), i do not get the same pop up telling me the file can't be named that way, but i get an error :

So this file name should be legit because it is legit on linux but is somehow forbidden on the server already, when 'test:iss' is allowed. So from my perspective this looks like character sanitization is already performed.

[Aldaris1985]

Maybe just one thing to add which I mentioned in #1723 : the issue is also that those forbidden characters are not properly reported. At least in the past version, files with the forbidden character had been ignored but the sync was shown as successful (please have a look at the issue, the log file is there). Although it is visible in the log files but which user apart from us freaks is digging through that? ;) It would be very helpful to point non-synced filed clearly to the user including the ones not synced due to forbidden character.

[fabianfrz]

@TwiggyWan you can use NextCloud with any operating system, that has a working TCP stack and supports http. That means that there are also operating systems like OS/2 (for example used at ATMs), FreeDOS (yes that supports networking), Symbian (old nokia phone OS), Tizen (for example used on smart TV), and much more.

All of them can upload files to the server but the sync client needs to be able to handle all of them in some way. Also if there are users of different countries, you might have even some special characters in the file name. Maybe you have a Chinese filename and want to write that to a file system that is not Unicode capable. The only system that has that information is the host running the sync client.

[TwiggyWan]

Okay. AFAIK there is a notion of server capabilities in case of a sync up but nothing regarding client capability in case of a sync down :
https://github.com/nextcloud/desktop/blob/78fa448c7eb34f35099e8b9140c4f1b60124c089/src/libsync/syncengine.cpp#L1094-L1114
And this is probably why there is no error message given to the user in #1723 as the error seems to be raised only for sync up.

According to what is in place, there is no notion of replacing characters valid on the client and invalid on the server, the sync just fails. What you suggested above @fabianfrz would mean that for sync down, we would perform renaming, but not for sync up. Wouldn't that just be confusing for the users?

[fabianfrz]

@TwiggyWan in case of sync up I think this can be handled under the hood by the server. Nobody says that the filename on the real file system must match the one stored on disk. For example the server running on windows could just accept a file called test:file.txt, store it as {b0693f9e-9ea8-4886-8c57-7823cd564677} and in the directory listing and on access it can offer it as test:file.txt again. Usually the servers are running on Linux or BSD so usually they have some modern file systems that can handle strange file names. In general, there should be anything allowed that is compliant to the WebDAV standard and any common operating system (no file you can create on Linux, Windows or Mac OS should be rejected because of the file name).

[claell]

I edited the description to reflect what @Aldaris1985 mentioned (no error message on Windows - #1723).

Also changed the title, since this issue seems to be treated like some meta issue now for any special character related problems on any operating system. I'd rather suggest not to do that, since the underlying problems might be pretty different, but for now at least the title correctly reflects again what this issue is about (now).

Also added the labels high (because there is no notificaiton of a failed sync on Windows in that case) and technical debt (since dealing with special characters should have been done directly when implementing, not afterwards).

[Githopp192]

When i downloaded those Files with COLON ":" directly from the NC-Web Gui - Nextcloud did a rename in that Nextcloud Web Gui did replace the ":" (Colon) with "_" (underscore).

The question is ..why not to trigger this behave to the Agent also ?

(in my case Level 1 should had do the job --> replace ":" with "_" on the iPhone App Level)

At least - as this is not fixed - the Nextcloud Agent on Windows (or anywhere else) should report an Error.

(as claell also already commented)

But this will not happen - Windows Agent is ok - so the users will not be aware of that issue and this could be a risk, if users or application will expect those files properly synchronized.

[fabianfrz]

@Githopp192 this is not done by nextcloud. This is behaviour by for example Firefox.

[Githopp192]

Thx Fiabanfrz .. yes ..this could be true. In my case it is MS Edge