Powershell: Command line arguments with a dollar sign

[Tibblez]

Command line arguments sent to a file do not send in full if the argument contains a dollar sign. I have tried every combination of escape character and quote configuration that I can think of. I am not certain if this is specific to the dollar sign but I was able to use single quotes with an exclamation mark in the argument and that worked fine.

I was not able to find any explanation for this in other open issues or the known issues but I apologize if I'm missing something obvious here.

Steps to reproduce

Create a new ps1 file called test.ps1 with this snippet.

param(
[string]$foo
)
write-host $foo

Then calling it from the command line

powershell ./test.ps1 -foo 'Before$After'

Expected behavior

Before$After

Actual behavior

Before

Environment data

> $PSVersionTable
Name                           Value                                                                           
----                           -----                                                                           
PSVersion                      6.0.0-beta                                                                      
PSEdition                      Core                                                                            
BuildVersion                   3.0.0.0                                                                         
CLRVersion                                                                                                     
GitCommitId                    v6.0.0-beta.2                                                                   
OS                             Linux 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016                 
Platform                       Unix                                                                            
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                         
PSRemotingProtocolVersion      2.3                                                                             
SerializationVersion           1.1.0.1                                                                         
WSManStackVersion              3.0                                                                             

[SteveL-MSFT]

I believe this is by design. Solution is to use the backtick to escape:

powershell ./test.ps1 -foo 'Before`$After'

[lzybkr]

I'm not sure it's by design.

Compare:

#1 PS>  ./test.ps1 -foo 'Before$After'
Before$After
#2 PS> echoargs ./test.ps1 -foo 'Before$After'
arg 0: <./test.ps1>
arg 1: <-foo>
arg 2: <Before$After>

CommandLine:
"EchoArgs.exe" ./test.ps1 -foo Before$After

#3 PS> powershell -file  ./test.ps1 -foo 'Before$After'
Before$After
#4 PS> powershell -command  ./test.ps1 -foo 'Before$After'
Before

Notice in 1, 2, and 3, $After is preserved. Only in 4 does $After disappear.

I would say we need some more digging before calling this by design - but it does provide a cleaner workaround - use -file.

[mklement0]

To summarize the difference in behavior:

• -File treats its arguments as _literal strings_, without further interpretation by PowerShell.

• -Command interprets its arguments _as if they'd been specified inside PowerShell_.

In the case at hand:

• With -File, Before$After is retained as-is, as a _literal string_.

  • Currently, this categorically prevents passing _Boolean_ values as arguments, because a [bool] / [switch] parameter doesn't accept _string_ values (e.g., $True is passed as if '$True' had been specified) - see #4036.

• With -Command, Before$After is interpreted as an _expandable string_ - as if you had passed it as an argument in argument mode from _within_ PowerShell - and - given that variable $After is not defined - it expands to just Before.

[SteveL-MSFT]

Getting -File for .ps1 and -Command to parse args the same as -Command (so that variables are string literals) is working. Need to figure out how to get scripts without .ps1 extension to work. With the change I'm proposing, "Before$After" needs to escape the $ otherwise expectation is that PowerShell interprets it as the value of the $After variable.

[mklement0]

@SteveL-MSFT: I think it's vital that we align the behavior of -Command with that of -c in POSIX-like shells, which, in short, means:

• The _first_ argument following -Command should be interpreted like a _script_.

  • If that script needs access to the subsequent arguments, it must use $args or a param() statement to access them.

    • Note: POSIX-like shells bind the first subsequent argument to $0 (rather than $1), whose PowerShell equivalent would be $MyInvocation.MyCommand.Name, but I think it's sensible _not_ to do that in PowerShell.

  • Note that this is a fundamental, breaking change from how -Command functions today, which basically reassembles a PowerShell command line from all the arguments by mere string concatenation (with spaces) and then invokes the result based on PowerShell rules.

• Any _subsequent_ arguments should be treated as _literals_ - just as with -File.

In other words: what follows an ad-hoc script (-Command) or a script file (-File, or by default), are _arguments to pass to that ad-hoc script/script file_, and they should be processed _the same_:

• as _literals_ (after potential _up-front_ expansion, depending on the calling shell) - except that $true and $false must also be recognized as Boolean literals (which is currently not the case - see #4036), just as, say 22 is recognized as an [int], if bound to a parameter of that type.

Conversely, anything that should be interpreted according to PowerShell-internal rules, must go directly into the ad-hoc script passed to -Command.

To illustrate the proposed difference:

# Already works this way.
> powershell -noprofile -command '"$HOME"'
/home/jdoe

# This is how it *should* work (currently breaks).
> powershell -noprofile -command '$args' '$HOME'
$HOME 

[SteveL-MSFT]

@PowerShell/powershell-committee reviewed this and although we agree the argument handling isn't technically correct, it has been this way since inception and would be a big breaking change without significant benefit, so the current behavior is 'by design'.

[mklement0]

Thanks for letting me know, @SteveL-MSFT.

I sincerely wish you revisited this issue, however:

While I get that breaking changes are very problematic, I think:

• that the current behavior is fundamentally broken _in itself_ - not just because it fundamentally differs from the behavior of POSIX-like shells.

• that this broken behavior will become more evident - and be a perennial pain point - when exposed to the more command-line-savvy, quoting-aware Unix crowd.

• the switch from -Command to -File already was a big breaking change; v6 is an opportunity to get _all_ aspects of the CLI right.

Consider the following calling-from-bash examples, which highlight the _fundamental_ problem (I'm leaving the aspect of how the _first_ argument should be treated as a mini-script to which the remaining arguments should be passed via $Args aside for now):

# Breaks, because literal `don't` is interpreted as a bareword by PowerShell.
$ powershell -noprofile -command Write-Output "don't" # !! BREAKS
The string is missing the terminator: '. 

#'# With a *literal*, you can work around that, but in addition to the
# outer quoting - WHICH SHOULD BE ENOUGH - it requires TWO EXTRA LAYERS OF 
# QUOTING: 1 for bash (\`) and 1 for PowerShell (the ` passed through).
$ powershell -noprofile -command Write-Output "don\`'t" # works, but extremely cumbersome.
don't

#'# Using a NOT-KNOWN-IN-ADVANCE VALUE is SIMILARLY CUMBERSOME AND COMPLEX:
$ v="don't"; powershell -noprofile -command Write-Output "'${v//\'/\'\'}'" # works, but extremely cumbersome.
don't

Cramming it into the _first_ argument is an option, but equally cumbersome:

$ powershell -noprofile -command "Write-Output  \"don't\""
don't

$ v="don't"; powershell -noprofile -command "Write-Output  '${v//\'/\'\'}'"
don't

PowerShell's current reassemble-all-arguments-into-a-command-line-and-then-reinterpret-it approach severely hampers the ability to pass arguments _as-is, as data_ to the command.

Interpretation as PS source code should be limited to the _first_ argument - the "mini-script".

Separately (as previously stated), so as to align with POSIX-like shells, the remaining arguments should be passed via $Args, which would also provide consistency with how -File handles its arguments; applied to the example above:

# WISHFUL THINKING; note how $v is passes via $Args
$ v="don't"; powershell -command 'Write-Output $Args[0]' "$v"
don't

[SteveL-MSFT]

@mklement0 as part of the fix for the other related issue #4036 I intend to also make a doc update to hopefully clarify this for the user on the differences between -File and -Command. The general statement is that the user needs to be aware of how the "outer shell" handles escaping (Bash in your examples) and what gets passed to PowerShell and what PowerShell passes to the command. I think that if this behavior becomes a big customer sticking point, we could consider adding another switch that affects the arg passing behavior, but it was considered a bigger impactful breaking change than the switch of -File and -Command where you would at least get a reasonable error message.

[mklement0]

@SteveL-MSFT: I'm glad to hear it's getting documented.

it was considered a bigger impactful breaking change than the switch of -File and -Command

Undoubtedly.

if this behavior becomes a big customer sticking point, we could consider adding another switch that affects the arg passing behavior

Just be aware that you're closing the door on allowing -c to work the same way as in POSIX-like shells; while -e would be the next best thing, the existence of -Command that works very differently will sow confusion.

The general statement is that the user needs to be aware of how the "outer shell" handles escaping

Absolutely, that's a must in any scenario.

But requiring _double_ escaping for things that should be passed as _literals_ (e.g., "don\'t"`) is both cumbersome and confusing.

To repeat myself briefly, interpreting _individual arguments_ passed to a command as part of the command's _source code_ subverts fundamental notions of how arguments are passed in the Unix world.

The saving grace is perhaps that wanting to pass literal data arguments to a single command string that functions as an _ad-hoc_ script is not that common (e.g., sh -c 'echo "$1"' - '$HOME' printing $HOME).

In the absence of passing such arguments, if the current behavior is retained, I suggest documenting that the best use of -Command is to pass the entire command as a _single string_ for conceptual clarity; e.g., instead of:

$ powershell -noprofile -command 'Write-Output' '`$HOME'  # confusing: 2 args 

using:

$ powershell -noprofile -command 'Write-Output `$HOME' # better: 1 arg

[SteveL-MSFT]

Cc @BrucePay

[mklement0]

@SteveL-MSFT:

When it comes to documenting the _current_ behavior of -Command perhaps the following summary can be helpful to readers with a Unix background:

• -Command _only_ accepts an ad-hoc PowerShell script (which may be a single statement) and has _no mechanism for passing arguments_ to that script.

• While you can _technically_ pass additional arguments after the 1st -Command argument, they simply become part of the ad-hoc script by string concatenation with spaces.

• Thus, in the absence of argument-passing, shell variable-based values to be used in the ad-hoc script must be "baked into" it and therefore require additional quoting (escaping) by PowerShell rules in order to be treated as literals.

• For conceptual clarity and to avoid additional quoting headaches, it is easier to always pass the entire ad-hoc script as a _single_ argument following -Command - at least in the Unix world.

  • On Windows, the scarcity of shell metacharacters and $ and ' having no special meaning to cmd.exe makes this less of a problem.

[SteveL-MSFT]

@mklement0 please take a look at my PR for the doc update and add any suggestions on language https://github.com/PowerShell/PowerShell-Docs/pull/1430

[mklement0]

Thank you, @SteveL-MSFT.

To [half-mis-]quote Dame Edna, I mean the following in a caring way:

• Like most PowerShell documentation, the one you link to falls somewhere along the spectrum of light introduction to confusing to incorrect/misleading.

• I honestly wouldn't know how to fit the subtleties presented here into the existing format, so I trust you to do the right thing with the information provided here.

Truthfully, I think PowerShell's documentation needs a serious overhaul, and my potential contribution to this particular help topic would feel like rearranging the deck chairs on the Titanic.

[SteveL-MSFT]

@mklement0 I don't think you should feel like you are constrained to the style that exists in the current docs. They were written years ago. I prefer to avoid a lot of detail as I feel people tend not to read that much content. I'd rather add more examples that show the differences. If you have thoughts on some specific ones, please share :)

[mklement0]

@SteveL-MSFT: I appreciate the invitation, but it feels overwhelming to me.

A few more meta observations:

• On a general note, based on my previous comment, I think that the existing documentation needs a serious overhaul (see the problems with the current documentation on error handling as a notable example).

  • I get the tension between wanting to be concise and simple (less tech-savvy sysadmins) and wanting to be comprehensive (for the benefit of developers).

  • Thus, keeping the documentation concise and to-the-point is important, but, conversely, if the actual behavior is too complex to be described concisely, perhaps the complexity of the behavior is the true problem.

    • If you need more evidence of the intractability that is the existing -Command behavior, see this SO answer of mine.

• The Unix world is used to _speed_, which PowerShell cannot compete with - PowerShell's strength is _abstraction_.
  Therefore, PowerShell's success in the Unix world will hinge on these factors:

  • From _inside_ PowerShell, the ability to call external Unix utilities as seamlessly as possible.

    • Aside from proper support for POSIX-like globbing (whose current trouble spots are overeagerness and inability to include hidden items), expressive constructs such as && and || are needed, and, ideally, also process substitution and brace expansion.

  • Arguments must be passed as-is (after potential interpretation by PowerShell), as an _array_ of _literal tokens_, as is customary in the Unix world (no command-line reconstruction shenanigans behind the scenes and _predictable_ "requoting" on Windows)

  • From the _outside_ (e.g., from bash), PowerShell must present a _sane CLI_ to also encourage ad-hoc use of PowerShell, as described above and more generally in #3743.