Mpv: Lua: subprocess and spaces in paths?

[DeadSix27]

mpv version and platform

Running a windows build from git master (by shinchiro).

Description

I am unable to use explorer.exe through subprocess_detached to open directories with spaced paths.

Maybe it's an obvious mistake on my end, but seems like this should work this way yet it doesn't?

Reproduction steps

Create 2 folders, e.g:

"C:\path space\"
"C:\path_nospace\"

Then execute this code in a lua plugin:

local command = { "explorer.exe", 'C:\\path space\\' }
utils.subprocess_detached({args = command})
command = { "explorer.exe", 'C:\\path_nospace\\' }
utils.subprocess_detached({args = command})

I did try adding additional quotes, e.g '"C:\path space\"' or "\"C:\path space\"".
I also tried forward slashes, same result.
And both paths work fine in command line, with and without quotes:

>explorer C:\path_nospace\
>explorer C:\path space\
>explorer "C:\path space\"
>explorer "C:\path_nospace\"

Expected behavior

It should open both folders.

Actual behavior

It only opens paths without a space in them.
EDIT: It will default to "Documents" if the path is incorrect

Log file: https://0x0.st/soiN.txt

[selsta]

Escape the space?

[DeadSix27]

@selsta On windows? How? EDIT I searched for that and on windows there is "^" for space escaping apparently, that doesn't work either.

[selsta]

Backslash

[DeadSix27]

@selsta So like on linux: 'C:\\path\\ space\\', doesn't work either.

[selsta]

'C:\\path\ space\\'

But I don’t use Windows so no idea.

[DeadSix27]

@selsta That isn't even valid lua syntax, you have to escape the backslash.

[selsta]

you can try out something like [["C:\\path space\\"]]

edit: tried it in a VM, it’s like @garoto said.

[garoto]

Remove the trailing backslashes, e.g.:

local command = { "explorer.exe", 'C:\\path space' }

[DeadSix27]

@garoto Indeed, that works. I also noticed that io.popen() works without doing that.
Any idea why this happens?

[garoto]

Zero clues, sorry. But I'm not a programmer. :)

But in any case, procmon (from sysinternals) shows explorer is invoked with this parameter when using trailing double backslashes with path with spaces:

"explorer.exe" "C:\path space\\"

And for the one with no spaces in it but with double backslashes at the end, this is what is passed to explorer.exe:

"explorer.exe" C:\path_nospace\

So yeah, that's what I got.

[rossy]

This is because of the Windows path escaping logic in mpv. In Linux, programs accept a list of strings as their arguments, so there is no need for escaping when launching a program as a subprocess. In Windows, programs only accept a single string as their command-line. The argv[] array is faked by the C runtime by parsing the command-line, so strings that contain special characters (space, tab and ") must be escaped to be treated as a single argument.

mpv's mp.subprocess uses escaping rules that are compatible with the MSVC runtime (including C programs like the Python interpreter) and .NET, which work as follows:

• If the argument does not contain spaces, tabs, or " characters
  
  
  
  • Use the argument verbatim
  
  
• If the argument contains spaces, tabs, or " characters:
  
  
  
  • Replace all \ characters that precede a " with \\
  
  
  • Replace all \ characters at the end of the argument with \\
  
  
  • Leave other \ characters alone
  
  
  • Replace " with \"
  
  
  • Surround the argument with " characters
  
  

These escaping rules are automatically applied to the args array passed to mp.subprocess, so you shouldn't try to escape the string yourself, otherwise the final command line will be "double escaped."

So for example:

• { 'explorer.exe', 'C:\\path_nospace\\' } becomes explorer.exe C:\path_nospace\
• { 'explorer.exe', 'C:\\path space\\' } becomes explorer.exe "C:\path space\\"

With this scheme, you can pass any string of unicode characters to compatible MSVC-runtime or .NET programs. However, some special programs like explorer.exe use their own escaping syntax which is not compatible. I'm not sure of the specifics, but it seems like explorer.exe treats the second backslash in C:\path space\\ as a literal backslash rather than an escape character.

I think @garoto's solution is the easiest way to handle this. explorer.exe is a special case, but since the argument is a path, you can just remove the trailing slash and it will still be a valid path.

We could also add a "verbatim" mode to mp.subprocess where it accepts a single string rather than an array for the args parameter (Python's Popen can do that as well,) but I think I want to avoid doing that as long as it's not absolutely necessary.

EDIT I searched for that and on windows there is "^" for space escaping apparently, that doesn't work either.

@DeadSix27 Yeah, ^ won't work here. ^ is the escape character used by the command interpreter (cmd.exe,) but mpv launches subprocesses directly, without cmd.exe. cmd.exe's escaping rules are much more complicated and IIRC it's not possible to escape certain sequences of characters, so it's best avoided.

I also noticed that io.popen() works without doing that. Any idea why this happens?

@DeadSix27 Lua's built-in io.popen() uses the shell (cmd.exe on Windows,) which is why it accepts a single string as its argument, rather than a list of strings.

[garoto]

@rossy, this same escaping logic applies to run statements in input.conf right?

And thanks for taking the time to write all that, much appreciated.

[rossy]

Yep. That should be the case after run started using mp_subprocess_detached().

And thanks for taking the time to write all that, much appreciated.

No problem. I hope it's helpful. Windows/MSVC escaping rules are weird and not well documented.

[DeadSix27]

@rossy Thanks a lot for explaining it so detailed :) and yeah, removing trailing separators seems like the way to go for this edge case.

[rossy]

@DeadSix27 Glad to help. I'll close this issue now if the explorer.exe issue is fixed, but feel free to reopen it or create a new issue if there are other argument-escaping oddities. I _think_ removing the trailing slash will work for all paths, but I'm not totally sure. I might have to implement a "verbatim" mode for utils.subprocess sooner or later and let Lua scripts do their own argument escaping.