Toml: Toml parser does not work for windows path

[parkavi11]

When I have a Windows path in the toll, it is not parsed and I am unable to read files in Windows.

My path is as follows:
{ path = "..\TestProject2\target\balo\test1-2019r3-any-0.1.0.balo" }

[eksortso]

Backslashes in basic strings are escape characters, so when you use double quotes to wrap your string, those backslashes will do strange things, or they just won't parse. Basic strings are like that. You could double up your backslashes, but there's an easier way.

In order to treat your backslashes as literal backslashes, you'll need to use literal strings. In other words, use _single quotes_ around Windows file paths, like so:

path = '..\TestProject2\target\balo\test1-2019r3-any-0.1.0.balo'

Now, what you're reading here is the TOML standard, which is a separate project from the parser you're using. If you have further questions about your particular parser, I can't answer them. But hopefully, I was able to give you enough information so you could use any Windows file path in your TOML documents.

[marzer]

@parkavi11 Does @eksortso's comment resolve your issue? If so you should close it.

[parkavi11]

Sorry for not responding to your instant reply. However, using single quotes around Windows file paths, didn't work in my case. Is there any other solution that is compatible with all the OS, not just for Windows? I mean, is it possible to call the path in the same way that I call in Mac?

[marzer]

@parkavi11 No idea. Using single quotes should work since single-quoted strings treat \ characters literally. Sounds like the parser you're using has a bug.

Do note that this repository is for the TOML _language_, not for any particular platform or implementation. I suggest you find the repository for the parser you're using and file an issue there instead.

[parkavi11]

Thank you team for your prompt response. I will check with the parser. Let's close the issue.

[parkavi11]

Is there any other ways where I can give the Windows path inside double quotes?
I tried like this way (mentioned below) and some characters are escaped.

[dependencies]
"parkavi2/test1" = {path = "..\\TestProject\\target\\balo\\test1-2020r1-any-0.1.0.balo"}

In this case b makes backspace the previous character and \t does tab, hence manipulating the actual path as below:

/TestProject/ targebalo/ test1-2020r1-any-0.1.0.balo

[marzer]

@parkavi11 The example you've provided _should_ work the way you want it to, so if it's escaping the characters _after_ the \\ (instead of the first \ escaping the second one), then the TOML parsing library you're using has a bug. This isn't an issue with the TOML language spec itself.

Otherwise, no, there'd be no other way to have Windows paths inside double quotes, unless you could use forward-slashes and convert them in the program.

[marzer]

Also, I've no idea why you're so stuck on needing to use double-quotes. Single-quoted strings are unquestionably better for your use-case, you should use those instead. If they don't work, then again, the TOML library you're using is broken and you should file a bug report with them, or switch to a different library.

[abelbraaksma]

@parkavi11, windows paths only use backslashes externally. The file system is totally capable of using forward slashes, and in fact, that's how they're stored in NTFS. There are, however, some programs that don't know this and manipulate paths by hand, leading to errors (like: splitting a path in segments by searching for the \ character: don't do that). Any program on windows that uses the BCL or the Win32 api should not have problems when passing it paths with forward slashes. The moral: just change each backward slash to a forward slash.

Added benefit of always using forward slashes is that your program, or config file, will work on Linux-based systems as well.

Finally, it's entirely possible that your toml parser has a bug (without knowing which of the dozens parsers you use it's not possible to help), or that you expect the toml parser to resolve your relative path to an absolute path. But TOML doesn't do that, it's ignorant of the type of the string. Any TOML parser merely merely parses the config, and it's up to the receiving application to interpret a string as a (relative or absolute) path, as a URI, a mail address, a color, anything, really, or as an ordinary string with no additional meaning.

[parkavi11]

@marzer yes this has to be reported in the library and hence close the issue here and get the feedback from the toml library team. Further from our side, it is expected to use double quotes and that is the reason for sticking with that.

@abelbraaksma as you said the path is perfectly working on Linux based systems and hence I am trying to work the same on Windows.

[marzer]

@parkavi11

Further from our side, it is expected to use double quotes

Why is it expected, though? Because someone prefers them? Limiting yourselves to only one type of string is unnecessary and doesn't make sense. You should change that guideline.

[abelbraaksma]

as you said the path is perfectly working on Linux based systems and hence I am trying to work the same on Windows

@parkavi11 Perhaps you misunderstood what I was saying. I meant to say that you _do not need to use backslashes on windows_. Just change your example to use forward slashes, and it will work seamlessly on Windows and Linux.

I mean, this works on Windows:

[dependencies]
"parkavi2/test1" = {path = "../TestProject/target/balo/test1-2020r1-any-0.1.0.balo"}

In this case b makes backspace the previous character and \t does tab, hence manipulating the actual path as below:

This is incorrect. \\b creates the string \b (one backslash followed by the character 'b'). The 'b' is not escaped here. If it is, it's a bug in the TOML parser you use. Or you are reinterpreting the output, which indeed could lead to the behavior you're showing.

Talking of which, can you please answer those questions, to help us help you?

• what parser do you use, or did you write your own?
• how do you process the result of that parser (programming language, code)
• why can't you use single quotes, does your parser not understand them?
• what is the actual resulting string that your parser sees (use a breakpoint in your code), when given either the escaped string, or the single quoted string?

And don't forget: TOML has no notion of what a path is. It only sees strings. It does not resolve the “.. “ in your string, for instance.

[parkavi11]

I mean, this works on Windows:

[dependencies]
"parkavi2/test1" = {path = "../TestProject/target/balo/test1-2020r1-any-0.1.0.balo"}

Yes, exactly this works in Windows too.

And don't forget: TOML has no notion of what a path is. It only sees strings. It does not resolve the “.. “ in your string, for instance.

Yes, we parse the string of path and then convert it as a Path type in order to read the file in the path separately.

[abelbraaksma]

Yes, exactly this works in Windows too.

Good, so you have a solution.

Yes, we parse the string of path and then convert it as a Path type

Then is is likely where your problem is, your re-parser may treat \ as an escape too, which would require you to double-escape the input string.

[parkavi11]

Answering your questions @abelbraaksma

1. com.moandjiezana.toml:toml4j:0.7.2

2. The content of the toml is read and distinguished as per the requirements. Therefore in the dependency we parse the path.

3. We have a requirement, in case a user tries a path with double quotes also, it has to be compatible. Single quote is already compatible in this case.

4. Below is the resulting string:

TestProject/       argealo/        est1-2020r1-any-0.1.0.balo

[abelbraaksma]

We have a requirement, in case a user tries a path with double quotes also,

Can you elaborate? What does a user do? Give the path in a popup window, pass it on the command line, write it in the toml config file? In what way do you test it?

Below is the resulting string:

If that's what you see after parsing it with toml, and before re-parsing it as file, and the toml file contents was read as a whole, than the issue is indeed with the com.moandjiezana.toml:toml4j:0.7.2 parser.

Here's what can happen with escaping;

• inside a toml file, double quotes, with double backslash, leads to single backslash in resulting value after parsing.
• if you take this value and paste it in Java, put quotes around it, you'll basically re-parsing the string. \\t became \t and has now become [tab].
• if you parse a literal string through toml in any programming language, you'll also have to deal with this double escaping. I.e. string x = MyTomlParser.Parse("path=\"you\\too\"") will give toml the string path="you\too". This will lead to a tab character again.
• if, after these steps, there's another parser that interprets a string as file, this parser may also treat a backslash as an escape. This would require a third escape level.

To overcome all these issues, only test with actual files that contain toml data, and if a value is re-parsed, make sure to re-escape the string.

[abelbraaksma]

The parser you use seems to be this one: https://github.com/mwanji/toml4j. If true, there hasn't been activity in a while. However, at first look it appears to do the escaping correctly (that is, double quoted strings require escaped backslashes, which is also suggested by this bug report, which isn't a bug: https://github.com/mwanji/toml4j/issues/53).

[parkavi11]

Thank you @abelbraaksma for your quick replies. We have decided to use single quotes to specify the paths. I will report a bug regarding the escape character issue in the toml library and will close this issue. Thank you @marzer for your explanations.