Vscode: Support expanding input parameters into multiple arguments

[anatolyburakov]

Currently, it is possible to specify debug arguments from input, using this syntax:

        "inputs": [
                {
                        "id": "mystr",
                        "type": "promptString",
                        "description": "",
                }
        ]

And having this prompt appear in launch.json:

        "configurations": [
                {
                        "name": "myexe",
                        "type": "cppdbg",
                        "request": "launch",
                        "program": "myexepath",
                        "args": [
                                "${input:mystr}"
                        ]
                }
        ]

This will cause a prompt to appear, asking for arguments to pass to the application. However, it is only possible to pass one argument that way.

It would be nice if it was possible to expand this:

"-a 5 -b 10 -c 15 -d \"some string\""

into this:

["-a", "5", "-b", "10", "-c", "15", "-d", "some string"]

This would make opportunistic debugging (testing with different parameters quickly) much easier, and will avoid polluting the launch.json with targets that are used only once and then discarded.

[isidorn]

This is coming from the configurationResolverService thus forwarding to @alexr00
Also fyi @weinand

[weinand]

@anatolyburakov I understand the usefulness of your feature.

But how should it actually work?

Today the input prompting mechanism is based on variable substitution: a specific pattern ${variable} that appears inside a string is substituted by the variable's value.

With your feature the variable's value becomes an array. So the pattern that occurs in a string now needs to be replaced by by an array. How do you envision would the syntax look like and how would you explain the semantics?

[vscodebot[bot]]

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.

Happy Coding!

[anatolyburakov]

@anatolyburakov I understand the usefulness of your feature.

But how should it actually work?

Today the input prompting mechanism is based on variable substitution: a specific pattern ${variable} that appears inside a string is substituted by the variable's value.

With your feature the variable's value becomes an array. So the pattern that occurs in a string now needs to be replaced by by an array. How do you envision would the syntax look like and how would you explain the semantics?

The easiest way i can think of is add another input type (instead of it being a "string" or a "choice", also add something like "argument list"), and split it into an array the way shell does (something every programmer should be familiar with). This would be the most sensible. The launch config does not allow specifying full command strings anyway and mandates making an "args" array for arguments - so array expansion inside this array shouldn't be too hard (i'm not familiar enough with JS, but it works find in Python with one line of slicing magic).

[weinand]

@anatolyburakov thanks, but I understand how to make a variable returning an array. That's not the problem.

The question is: how would the launch config look like that allows for substituting a variable by an array.

Your example syntax does not work because the variable ${input:mystr} sits inside a string and substitution cannot "extend" beyond that string (so the string cannot be replaced by a set of strings).

                {
                        "name": "myexe",
                        "type": "cppdbg",
                        "request": "launch",
                        "program": "myexepath",
                        "args": [
                                "${input:mystr}"
                        ]
                }

And something like this cannot work either because its is no longer valid JSON:

                {
                        "name": "myexe",
                        "type": "cppdbg",
                        "request": "launch",
                        "program": "myexepath",
                        "args": ${input:mystr}
                }

[anatolyburakov]

@weinand i'm not well versed in either JS or in internals of VSCode, so i wouldn't know how to do that. however, with Python, it's done trivially:

strs = ["a", "b", "toreplace", "c"]
idx = strs.index('toreplace')
strs = [:idx] + expanded_values + [idx+1:]

i'm sure there's probably a better way to write this, but the point is, i don't understand the problem with implementing such a simple array transform. perhaps i'm missing something?

[alexr00]

The problem is that launch.json is json, not a programming language. The examples that @weinand give in https://github.com/microsoft/vscode/issues/83678#issuecomment-553442628 illustrate this very nicely. You either have invalid json or a string.

[anatolyburakov]

@alexr00 yes, but whatever _handles_ that JSON presumably _is_ a programming language?

[alexr00]

In the example we have:

"args": [
    "${input:mystr}"
]

So replace ${input:mystr} with an array? ${input:mystr} is still in quotes (like "${input:mystr}"), so what does an array in quotes look like?

[anatolyburakov]

@alexr00 basically yes, replace that with an array. if it matches without being the entire string (i.e. "something ${input:mystr} something else"), simply throw an error (or don't replace).

EDIT: to be more precise, remove it, and insert an array in the middle. "replacing" with an array will just yield an array within an array...

the replacement is already being performed - so, regardless of whether or not "JSON is not a programming language", there presumably is some kind of pre-processing already being performed on this JSON data that involves "a programming language" (unless JSON has that built-in somehow? please excuse my ignorance on this matter). i'm simply suggesting adding one more step, albeit a slightly more complex one than simply string replacement.

[weinand]

We are all programmers, so we don't have to discuss how to implement this feature.

The issue is that there is an "end user aspect": How does the syntax look like that an end user would use and what exactly is the semantics of that syntax.

This "end user aspect" is completely independent from implementation details.

Let me try a proposal "for end users":

Variable substitution is only performed in strings and the syntax is "... ${variable name} ... ${another variable} ...".
As long as all the variables evaluate to a string type, their values are used to replace the variable patterns in the enclosing string.

If the type of at least one variable is a not string, its value will be used to replace the entire enclosing string (and all the other variables are ignored).

So if the variable "BAR" is the number 123, the JSON expression

  "foo": "hello ${BAR} world"

would effectively become:

  "foo": 123

Any surrounding text or additional variables inside the original string are dropped.

If the value of "BAR" is an array [ "a", "b", "c" ] the JSON expression becomes:

  "foo": [ "a", "b", "c" ]

---

This syntax and semantics is a bit strange for my taste....

@alexr00 @isidorn @roblourens @dbaeumer what do you think?

[anatolyburakov]

@weinand my apologies, it seems that i was consistently misunderstanding the nature of the questions i'm getting.

speaking of your proposal, that works only if you do simple replacement. you don't have to do that, and it would limit usefulness of this feature, because it would now be not possible to have some predefined arguments _and_ custom ones at the same time. in a perfect world, i would like something like:

"foo": [
   "a", "b", "${bar}", "d"
]

turning into

"foo": [
   "a", "b", "c", "e", "f", "d" // insert "c", "e", "f" where "${bar}" was but leave everything else in place
]

That's the main goal of this proposal.

What happens when ${bar} does not enclose the entire string (such as " ${bar} " or "a ${bar} b")? Frankly, i don't much care, as i consider this case to be user error. In my view, it would be rejected as invalid. Build/debug tasks already can throw errors (such as if the command itself is invalid), so why not on this?

[alexr00]

While I see how this could be useful, I agree with @weinand that the semantics of this are a bit strange. We have taught users that ${something} is replaced with a string, and that is a convention used in places other than VS Code too. This array concept takes something that was very simple ("It gets replaced by a string") and makes it complex + introduces additional user error.

[anatolyburakov]

Well, i'm open to suggestions on how to get this functionality without building it into VSCode. when introducing VSCode to my coworkers, this is one of the first things that i have to explain away, and make known. we're all used to running gdb with arguments, The nature of software we're working on is such that there is a huge number of command-line parameters, so creating a launch configuration for every permutation gets a bit tedious.

[anatolyburakov]

@alexr00 @weinand is this kind of thing possible to do with an extension? As in, intercepting all calls to debug targets, and modifying arguments before actually running debug target? Or maybe there is a better way to achieve this? This really sounds like a good excuse to try my hand at VSCode extension development, as it seems simple enough to get it fixed quickly, and annoys me strongly enough to give me motivation to fix it... :)

[Kenobi-the-2nd]

Good morning,

I have found the input feature very useful in the context of debugging. However, as @anatolyburakov has mentioned, there are many cases when it would be very useful to type in an entire list of args that is passed to the "args" debug parameter as an array of strings. For example, I do much of my development in Perl, and the ability to pass multiple arguments to a script is essential. The current functionality using string substitution will not work in Perl when passing more than one argument since it treats the entire string as a single argument. Being able to type in the args and have them substituted as an array would solve the issue.

Concerning how this particular feature would be added, we could do it along the lines of the existing framework, as mentioned in the previous discussions:

{
    "type": "perl",
    "request": "launch",
    "name": "Debug INPUT",
    "console": "integratedTerminal",
    "program": "${workspaceFolder}/${relativeFile}",
    "args": ["${input:argsPrompt}"]
}

...

"inputs": [
        {
          "id": "argsPrompt",
          "description": "Enter list of args...",
          "default": "",
          "type": "promptStringList",
        }
    ]

The new type, promptStringList, (or something similar) would replace the variable with a string list. In this way, we have a different input type for the array of strings. However, I understand from the above comments by @alexr00 and @weinand that simply introducing a new type while still keeping the same ${input:var} syntax would be misleading to some users. While the approach is workable, I agree that it is not the optimal design.

To remedy this, I suggest a new type of variable with the format @{} ( instead of ${}), such as @{inputList:argsPrompt}, for example. This variable would be of array type--meaning, in essence, that wherever a variable of array type occurs, it will be replaced by a list of strings intended to be used inside of a json array. To differentiate inputs in array format from inputs in string format, we would simply define a new input type, perhaps called inputLists. An example configuration with this new format would be similar to above, except with the noted changes:

{
    "type": "perl",
    "request": "launch",
    "name": "Debug INPUT",
    "console": "integratedTerminal",
    "program": "${workspaceFolder}/${relativeFile}",
    "args": ["@{inputList:argsPrompt}"]
}

...

"inputLists": [
        {
          "id": "argsPrompt",
          "description": "Enter list of args...",
          "default": [],
          "type": "promptStringList",
        }
    ]

In the debug configuration above, say that we enter the following argument list in the input box:

a b c

The new array type variable, "@{input:argsPrompt}" (including the double quotes), would then be substituted by three strings:

"a", "b", "c"

With this new variable type, we thus remove the confusion between string substitution and array substitution. The string variable convention remains in place without being disturbed by a change in semantics.

The usefulness of this feature would extend past debug configurations and into tasks, where arrays of strings would be useful for passing multiple distinct arguments. I do not think this feature would be hard to implement and believe that it would turn out to be very useful. In my case, it would enable input of multiple args to my Perl scripts, which is currently not possible.

If you would like more/alternate suggestions and/or details regarding this feature, please let me know. I am happy to contribute however I can to the development of this idea, and I very much hope that this feature will be added.

Thank-you for listening. Your time and patience is much appreciated.

[weinand]

The problem is "args": [ "${input:argsPrompt}" ] is an array with one element: a string. Variable substitution occurs within the string. Variable substitution is not a macro mechanism that works on the level of the JSON. So the result of the variable substitution can only be of type 'string', it can not be another JSON element (e.g. an array or an integer or another full JSON).

Imagine how difficult it would be to understand the result of something like "${input:x}${input:y}${input:z}" where 'x' is a string, 'y' is an array, and 'z' is another JSON structure.

Introducing new syntax like "@{inputList:argsPrompt}" does not change anything: variable substitution occurs within the string.

If your really want to pass a command line via the "input" variables to your program, ask the debugger author to support an alternative launch configuration attribute "commandline" that avoids the array:

{
    "type": "perl",
    "request": "launch",
    "name": "Debug INPUT",
    "console": "integratedTerminal",
    "program": "${workspaceFolder}/${relativeFile}",
    "commandline": "-foo -bar /hello/world -o -p"
}

[Kenobi-the-2nd]

That makes sense. I was unsure of how the substitution worked in json. These are all very good points. I will mention your suggestion to the extension author.

Again, thank-you very much for looking into it. Your time is much appreciated sir.

[anatolyburakov]

@weinand i still think this should be part of VSCode itself and not up to each and every debugger to implement their own way. that way, at least VSCode can provide a facility for parsing the command line into a set of tokens that works consistently across debuggers, rather than each individual debugger having to implement (possibly buggy) code to parse the command-line with all of its intricacies such as escaping/quotation handling.

[weinand]

@anatolyburakov agreed, but I do not see a way to accomplish this in VS Code with the current string-based variable substitution mechanism, e.g.:

  "args": [ "${variable}" ]

The substitution mechanism only works within strings but the overall syntax is valid JSON.

To solve the issue at hand we could use a substitution mechanism that works outside of JSON, e.g.:

  "args": ${variable}

Here variable could resolve to an array or any other type easily. But the syntax is no longer JSON.

There are no plans to change the substitution mechanism to the latter.

[vscodebot[bot]]

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!

[weinand]

Another idea:

VS Code does not own the JSON schema for launch configs, so it does not know anything about args.

But VS Code collects all schemas and merges them into a single combined schema for the launch.json. While doing so, it could look at the individual schema contributions and find all "string arrays". For every string array it could (blindly) inject a schema for a corresponding "string" property. E.g. for args: string[ ] it would inject an _args: string. (Ideally VS Code could inject a schema that allows only the string array or the string but not both). With this a user could either write "args": [ "-o", "abc" ] or "_args": "-o abc" (but not both).

When starting a new debug session VS Code would process the launch config after variable substitution and before passing it to the debug adapter and convert _args into an args[] by parsing it.

@alexr00 @isidorn what do you think?

[alexr00]

I am a bit concerned about escaping in the "_args": "-o abc" case. If someone quotes and spaces in there they might expect certain escaping to happen. Tasks has some cases like that already, and the escaping is a nightmare.

[anatolyburakov]

@alexr00 i believe it would be a benefit, not a hindrance. see my previous response:

@weinand i still think this should be part of VSCode itself and not up to each and every debugger to implement their own way. that way, at least VSCode can provide a facility for parsing the command line into a set of tokens that works consistently across debuggers, rather than each individual debugger having to implement (possibly buggy) code to parse the command-line with all of its intricacies such as escaping/quotation handling.

so, while implementing such a code is indeed nightmare, it is better for VSCode team to suffer through this nightmare once, than 1) every debugger extension developer having to suffer through this, and 2) every user of every debugger extension out there dealing with quirks and problems of escaping implementation in different debuggers due to 1). there's probably already ten different npm packages implementing command line parsing/escaping, so it might not even be that much of a nightmare...

[felipecrs]

@weinand why not support something like what you suggested, commandline, but in VS Code itself?

It could be also commandLineArgs, argsString, argsLine. And there is no need for ditching the old args array. We could either pass argsString after the args when runnning the program, or maybe respecting their order written in the JSON.

[felipecrs]

I am a bit concerned about escaping in the "_args": "-o abc" case. If someone quotes and spaces in there they might expect certain escaping to happen. Tasks has some cases like that already, and the escaping is a nightmare.

For example, "_args": "-o abc \"file name\"" should be treated as "args": ["-o", "abc", "file name"].

Or maybe, better yet if there is how, pass the entire string, not converted to an array, to the program to execute and leave the expansion for the shell. (But I suppose there is no shell in this case).

[weinand]

@felipecassiors the properties of the debug configurations in launch.json are not owned (or defined) by VS Code. They are owned by the debugger extension and are contributed by a JSON schema. If VS Code would introduce "commandLineArgs" all debugger extensions would have to adopt them.

My proposed solution works transparently and would not require adoption by debugger extensions.

In addition, the feature request we are discussing here is not to introduce a property that represents a command line. We are trying to solve the issue that the "input variable" mechanism can only return a string to the variable substitution mechanism. This limitation makes it impossible to use an "input variable" for the "args" (or any other) array.

My proposed solution works around that issue by introducing an alternative representation for a JSON array, namely a string that gets automatically parsed into the array behind the scenes.

[marvin1up]

I'm definitely +1 on @weinand suggestion to inject (e.g.) _args for string array - it is creative and provides a good approach for different uses including inputs. Some people would even just prefer, for the args case, to use _args to simplify cutting and pasting arg strings into a configuration. Personally, I would keep the space delimited parsing completely basic (no handling of any quoting) and document that. The actual vector (arg, e.g) handles that for all current uses. Then, if desired, create a second issue to decide WHETHER and HOW to support fancier quote parsing for the _var form.

[felipecrs]

By the way, the parsing string back into an array, dealing with quoting, should be the easiest part. See shell-quote's parse:

[felipecrs]

Just to say, I implemented argsString for vscode-bash-debug via https://github.com/rogalmic/vscode-bash-debug/pull/139. So that, prompt for arguments was made possible:

[anatolyburakov]

@felipecassiors that looks great! now, if only it worked with other debuggers (hint hint!)...

[felipecrs]

I wrote a new feature request (enable history in input prompt) https://github.com/microsoft/vscode/issues/97655 which would benefit this a lot. It's a backlog candidate, so if you like it please cast your vote with a 👍.

[Kenobi-the-2nd]

I think the solution proposed by @weinand above is a great idea and would very much like to see this added.