PowerShell 🚀 - 'Get-Command ?' returns % (ForEach-Object alias) and other unrelated CommandInfo objects

I'm assuming that's because ? is being interpreted as the single-character wildcard when searching for commands.

vexx32 on 6 Apr 2019

👍5

This seems related to some other things that @mklement0 has brought up in other wildcard-escaping issues. You actually have to escape the wildcard _twice_ to get it to register what you're looking for:

PS C:\Users\Joel> gcm ?

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           % -> ForEach-Object
Alias           ? -> Where-Object
Alias           h -> Get-History
Alias           r -> Invoke-History


PS C:\Users\Joel> gcm `?

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           % -> ForEach-Object
Alias           ? -> Where-Object
Alias           h -> Get-History
Alias           r -> Invoke-History


PS C:\Users\Joel> gcm ``?

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           ? -> Where-Object

vexx32 on 6 Apr 2019

👍2

FYI

PS C:\> gcm "?"

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           % -> ForEach-Object
Alias           ? -> Where-Object
Alias           h -> Get-History
Alias           r -> Invoke-History

PS C:\> gcm "`?"

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           % -> ForEach-Object
Alias           ? -> Where-Object
Alias           h -> Get-History
Alias           r -> Invoke-History

PS C:\> gcm "``?"

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           ? -> Where-Object

PS C:\> gcm '?'

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           % -> ForEach-Object
Alias           ? -> Where-Object
Alias           h -> Get-History
Alias           r -> Invoke-History

PS C:\> gcm '`?'

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           ? -> Where-Object

PS C:\> gcm '``?'
# No output

mkht on 6 Apr 2019

Thanks for the input. You guys are probably right about it being the single character wildcard, I did not think of this. Unfortunately, as a consumer of Get-Command I do not want to write my own logic (especially since the escaping does not work if the -Name value is a cmdlet like Install-Module instead) depending on what comes in as a user input (which can be anything in PSSA). Maybe a -LiteralName would be better (and faster) for such cases. I can understand that PowerShell was designed to be as helpful as possible and tries to not have complicated cmdlet parameters but every once in a while this can create annoying issues of PowerShell trying to be too clever.
From an architectural perspective, would it be helpful to have a LiteralString type that derives from System.String and means to PowerShell cmdlets that the string should be interpreted literally as-is? @SteveL-MSFT

bergmeister on 6 Apr 2019

👍1

That would be really nice.

vexx32 on 6 Apr 2019

Shall we close this issue after maintainers have seen this and possible create a new issue for the proposal of the LiteralString type or parameter?

bergmeister on 6 Apr 2019

Let me shed some more light on the current situation:

It's clear that we have a problem with treating wildcards consistently, as the sheer number of open issues related to them attests (title search only): https://github.com/PowerShell/PowerShell/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+in%3Atitle+wildcard
Also, it's not obvious:
- what parameters support wildcards:
  - #4715 asks for the syntax diagrams to reflect wildcard support.
  - #4716 asks for reliable programmatic discoverability of parameters with wildcard support
  - #2490 asks for auto-generated help / syntax diagrams to reflect wildcard support (and default values).
- how to use escaping in their arguments.

Currently, all wildcard-supporting parameters are [string]-typed.

The [SupportsWildcards()] attribute is purely informative - no behavior is enforced, no support is provided. Commands can implement wildcard support without indicating that fact via this attribute, which is problematic.

Because wildcard-supporting parameters are [string]-typed, the only way to opt out of interpretation of a value as a _pattern_ (i.e., to opt into treatment as a _literal_), is to use escaping _embedded in the string value_, using _embedded_ ` as the escape character.

Therefore, for escaping to be effective, the ` _must reach the target command_, i.e., must be _embedded in the string_.

The conceptually clearest expression of that is the use of single-quoting:

Get-Command '`?' # use of '...' ensure that ` is passed through

What makes this confusing is that PowerShell's _expandable-string_ `-escaping can get in the way:

(In argument mode) `? and "`?" are both treated as expandable strings, and ` is the escape char. in expandable string that has _syntactic function_ and therefore gets _removed_ during parsing, and Get-Command see just ?. In other words: passing `? or "`?" is no different than passing just ?

To put it yet another way: lone ` instances in expandable strings are "eaten" during _string parsing_, and _do not become part of the string_.

By contrast, `` does effectively escape an ` in an expandable string, which then becomes part of the string, which is why the following work too:

Get-Command ``?   # string parsing eats 1 ` and Get-Command sees literal `?
Get-Command "``?"  # ditto

As for ensuring the literal treatment of arguments provided _via expressions or variables_, you must use [WildcardPattern]::Escape()

$name = '?'
Get-Command ([WildcardPattern]::Escape($name)) # OK - the method call yields literal `?

mklement0 on 6 Apr 2019

I think that 99% of the time, Get-Command is used with wildcards to search and resolve a command. I'm not against adding a new parameter or switch for programmatic use.

A side topic, in general, it would be nice if we have some custom types derived off [string] so we have more context of the intended use. For example, [wildcardedString] or [filesystemPath] so we can have more intelligent tab completion and self documentation. In the latter case, if we ensure all appropriate parameters have the [SupportsWildCards()] attribute, we could do something with PSReadLine to show that the parameter supports wild cards (for example).

SteveL-MSFT on 8 Apr 2019

👍1

@SteveL-MSFT: indeed, with Get-Command the confusion between a literal and a wildcard pattern is an edge case, and the ability to use wildcards without jumping through syntax hoops is generally a benefit.

With _file paths_, however, it is a recurring pain point, and it has led to the deplorable -Path / -LiteralPath schism.

The problem hasn't come up often in other contexts - such as with Get-Command here - but it's a generic problem.

Implementing a -<name> / Literal<name> parameter pair _everywhere_ wildcards are supported is not practical.

What is needed is a concise, obvious way to _opt out_ of treating an argument as a wildcard for a _single_ parameter that can accept _either_ a wildcard or a literal, namely for both:

_literal_ use (currently required: _embedded_ `-escaping, e.g., '`?')
_variable/expression_-based used (currently required: ([WildcardPattern]::Escape($name)))

Let me suggest the following way forward, which would only require changes to the parameter binder; however, it would be a breaking change:

In POSIX-like shells such as Bash, it is _explicit quoting_ that disambiguates wildcard patterns from string literals (leaving aside that wildcard handling is fundamentally different there and that unquoted use implies others so-called shell expansions).

Applied to PowerShell:

# WISHFUL THINKING

# Literal syntax
Get-Command ?   # wildcard
Get-Command '?'  # literal
Get-Command "?"  # literal

# Variable/expression-based syntax
$name = '?'
Get-Command $name    # wildcard
Get-Command "$name"  # literal

# Pipeline use
Write-Output ? | Get-Command  # literal - pipeline input is always literal
Write-Output ? | % { Get-Command $_ } # wildcard - if really needed

The parameter binder could then act as follows:

On binding _arguments_:
- If the target parameter has the SupportsWildcards attribute:
  - If the argument is _unquoted_, pass it / its evaluated value through _as-is_.
  - If the argument is _quoted_, apply [WildcardPattern]::Escape() to it / its evaluated value and bind that.
- Otherwise, no distinction is made between quoted and unquoted use.
On binding _from the pipeline_:
- As implied by the examples above, pipeline inputs would always be treated as _literals_.

From an implementing cmdlet / function's perspective, all that is needed is to consistently decorate wildcard-supporting parameters with the SupportsWildcards attribute, and the parameter binder would do the rest: literal inputs would arrive automatically escaped.

A potential new hybrid parameter type such as [wildcardedstring] could also tie into this: instead of _escaping_ a literal value, the parameter binder could simply construct the [wildcardedstring] in a way that indicates that the string is a _literal_, which would make performant implementations simpler: the implementer will easily know whether the argument is a literal, and can skip matching operations (you can currently do the same thing manually with [WildcardPattern]::ContainsWildcardCharacters())

Additionally, parsing of compound tokens as arguments would have to change to allow direct concatenation of quoted and unquoted tokens:

# WISHFUL THINKING

# Quote only the literal parts of a path - those parts that need it.
# The wildcard char. must be unquoted in order to be recognized as such.
Get-ChildItem "/foo/name with spaces"/*

Currently, such a compound token results in _two_ arguments - which, honestly, never made much sense - see #6467):

# CURRENT BEHAVIOR

# Quoted string first:
# !! 2 arguments
PS> Write-Output "foo bar"baz
foo bar
baz

# Unquoted string first:
# !! 1 argument
PS> Write-Output baz"foo bar"
bazfoo bar

As for what would break:

The big issue: wildcard-based paths with explicit quoting would no longer work; e.g., Get-ChildItem "*.txt"
Unlikely to be a problem: Code that relied on compound strings such as "foo"bar getting broken into multiple arguments.

As for what we would gain:

The continued convenience of being able to use wildcards without special syntax - e.g., Get-Command *-Csv, Get-ChildItem *.txt
A simple, consistent opt-out-of-wildcard-matching mechanism - if needed - to ensure literal use.

mklement0 on 8 Apr 2019

I agree that some syntactical difference between wildcarded and literal strings is a good idea. I _don't_ think that just forbidding all wildcard patterns in enclosed strings is the answer, however... Rather, I think that this is an opportunity to make a meaningful distinction between the two string parsing modes.

Single-quoted strings are _already widely understood_ to be literal strings, in that they don't expand variables and subexpressions, etc.

Why not simply expand that already-understood behaviour and make single-quoted strings the de facto standard for non-wildcard parameter input?

In other words, in your brief example:

PS> Get-Command ? # wildcard
PS> Get-Command "?" # wildcard
PS> Get-Command '?' # literal

Alternately, we could make use of a different enclosing character(s) entirely? Some possible suggestions:

//?//
<?>
\\?\\

Also, your suggestion @mklement0 would result in being _fundamentally unable_ to pass wildcard patterns via variable, which I think is very silly. 😄

vexx32 on 8 Apr 2019

@vexx32

Also, your suggestion @mklement0 would result in being fundamentally unable to pass wildcard patterns via variable

To quote from my proposal above:

# Variable/expression-based syntax
$name = '?'
Get-Command $name    # wildcard

That is, the suggestion was to interpret the values of variables or expressions _that aren't enclosed in "..." as _wildcards_ and, conversely, only treat them as literals when "..."-enclosed.

Conversely, your single-quotes-as-literals-only proposal prevents use of variables / expressions as _literals_, which I think is very silly. 😄

Re new sigils / enclosing delimiters such as //...//:

I intentionally tried to _avoid_ introducing those, because PowerShell already has a problem with additional sigils such as @ and { at the start of unquoted arguments having special meaning compared to other shells.

Any additional exceptions to literal parsing are problematic (not just from a backward-compatibility perspective), as you'll always have to be on guard as to whether your unquoted argument will be taken as a literal or as an expression.

Also note that we want to retain _wildcard_ interpretation as the default, which means that the new syntax would have to express opt-in to _string-literal_ interpretation.

Building on your suggestion to leverage the well-established semantics of '...' to mean _literal_, perhaps the following could work:

$name = "?"
Get-Command ''$name

That is, '' preceding a variable name or expression would signal the intent to use its value _literally_.

(Currently, this results in _two_ arguments: the empty string, followed by the value of $name - again, it's unlikely that existing code relies on this behavior.)

This would then allow use to also treat "..."-enclosed tokens as wildcards, as you suggest.

mklement0 on 8 Apr 2019

Conversely, your single-quotes-as-literals-only proposal prevents use of variables / expressions as literals, which I think is very silly. 😄

You can have the parser assign a _different type_ ([literalstring] perhaps?) to the string, or an attribute to the variable that stores it, which flags it explicitly as a literal string. 😄

vexx32 on 8 Apr 2019

@vexx32: Assign a different type _based on what_? What signals the distinction between wanting to use $var's value in Get-Command $var / Get-Command "$var" as a wildcard vs. a literal?

mklement0 on 8 Apr 2019

$literal = 'literal'
$nonliteral = "nonliteral"

If, instead, you did as you mention:

Get-Command $literal # literal
Get-Command "$literal" # converted to non-literal
Get-Command $nonliteral # nonliteral
Get-Command "$nonliteral" # nonliteral

vexx32 on 8 Apr 2019

How about making hashtable and string implicitly convertible to WildcardPattern?

Then you could set it up so the parameter binder (indirectly I suppose, through type conversions) can handle these:

Get-Command ImplicitlySupportsWildcard*
Get-Command @{ Value = 'Does not support wildcards'; IsLiteral = $true }

# Or with extras in the parser. Picture some better form of syntax here instead.
Get-Command "No wildcards"/@

So then cmdlets could type the parameter as WildcardPattern and handle both literals and wildcards with pattern.IsMatch(string). WildcardPattern already creates a delegate to use for matching, so you could have it create one like (input) => pattern.Equals(input, StringComparison.InvariantCultureIgnoreCase) for literals.

Maybe also add properties Pattern and IsLiteral to WildcardPattern so the cmdlet author can use them directly if needed.

Edit: Fixed culture in example delegate, ty @mklement0

SeeminglyScience on 8 Apr 2019

👍4

@vexx32:

What you're proposing is highly obscure and I strongly recommend against it.

Once a string has been created, its content is by definition a _literal_, and we don't want to create implicit string subtypes that carry information that is entirely unrelated to strings per se.

Instead, we're looking for a syntax form that _in the context of wildcard matching only_ conveys the distinction between something meant to be a pattern and something to be used as-is - because wildcard patterns currently happen to be implemented as strings.

In other words: we're trying to convey information that is unrelated to strings per se, and only matters if strings happen to be used as - potential - wildcard patterns.

Piggybacking on _string_-quoting semantics is only an option in the context of _argument parsing_, where use of _unquoted_ strings is permitted and can thus serve as a distinction, but it's important to understand that string-literal parsing and interpretation of a string's _content_ as a wildcard expression are unrelated things.

My own proposal is guilty of blurring these lines too, but only in said constrained scenario - nowhere else, notably not in expression mode.

In short, what we want to express is: "Whatever this _string_ expands to (or not), treat its _resulting content_ either as _pattern_ or use it _as-is_."

@SeeminglyScience:

Great idea to adapt the existing [WildcardPattern] type and to give it an IsLiteral property (for consistency with -eq, the string-comparison mode should be InvariantCultureIgnoreCase, though) - doing so obviates the need for a potential [wildcardedstring] type.

As for the literal argument syntax:

@{ Value = 'Does not support wildcards'; IsLiteral = $true } works unambiguously and introduces no new syntax, but it is obviously very verbose.
"No wildcards"/@ strikes me as obscure and would be the first case of a _postpositional_ sigil.

However, conceivably we could take a page out of C#'s book:

# All literal uses
Get-Command @'?'
Get-Command @"?"
$name = '?'; Get-Command @$name
$name = '?'; Get-Command @$($name)

# Note that the following would NOT work:
Get-Command @?  #  reserved for splatting.
Get-Command @($name) #  array subexpression operator

The problem is that @ already has multiple unrelated or only loosely related meanings (splatting, here-strings, array-subexpression operator).

While a separate sigil - @ - potentially makes it clearer that a dimension that is independent of string quoting is being expressed, the fact that the same sigil is used for here-strings again muddies the waters.

Another problem is that @ cannot be followed by an _unquoted_ token (because it would then be the _splatting_ syntax), and the ability to use unquoted tokens is an important convenience in a shell.

As for backward compatibility:

@' and @" are currently interpreted as the start here-strings, whose content must start on the _next_ line, so there'd be no problem there.
@$name is currently the same as "@$name", however, so it's conceivable that existing code relies on that.

_[Doesn't work, because something like '''name' is syntactically a regular single-quoted string with an escaped ']_ At the end of the day I like @vexx32's suggestion in the abstract: to piggyback on '...' semantics, but perhaps _only_ with a special argument-parsing-mode only '' _prefix_ to a string, irrespective of the latter's own quoting style:

# Pattern - irrespective of string-quoting style
Get-Command name
Get-Command 'name'
Get-Command "name"
Get-Command $name
Get-Command $('name' + '?')

# [DOES NOT WORK]
# Literal - only with '' prefix
Get-Command ''name 
Get-Command '''name'
Get-Command ''"name"
Get-Command ''$name
Get-Command ''$($name + '?')

mklement0 on 8 Apr 2019

@mklement0

"No wildcards"/@ strikes me as obscure and would be the first case of a postpositional sigil.

Oh, to be honest I misread part of one of your comments. I hastily included that as an example because I thought you proposed it. I don't actually have any thoughts on what the syntax should be or even whether or not there should be one at all. That's what I get for skimming 🙂

SeeminglyScience on 8 Apr 2019

😄1

@SeeminglyScience:

even whether or not there should be one at all.

Indeed we don't _need_ a syntax for that; even with the current behavior, you can always disambiguate with:

Get-Command '`?'  # string-literal: embedded escaping of metachars.
$name = '?'; Get-Command ([WildcardPattern]::Escape($name)) # escape programmatically

Based on what prompted creation of this issue and that the subsequent discussion it's clear that these options are obscure in the former case, and additionally cumbersome in the latter.

In the case of -Path / -LiteralPath, you can sidestep the issue by choosing -LiteralPath for literal input paths, forgoing the convenience of _positional_ argument binding.

In the majority of cases you don't have that option.

Perhaps the need to force literal interpretation rarely arises (I've tripped over Get-Command ? myself, for instance), but when it does, it's currently far from obvious how to handle it, because:

about_Wildcards doesn't talk discuss escaping at all.
Similarly, the [WildcardPattern] class isn't part of the end-user documentation.
As stated, discovering which parameters support wildcards is quite cumbersome at the moment (see #4715, #4716, and #2490).

_At the least_, these issues should be addressed - even if nothing else changes.

Additionally, I think that a concise syntax that works with both literals and variables/expressions for on-demand treat-as-literal semantics is a worthwhile addition for convenience.

mklement0 on 8 Apr 2019

A quick meta note, @vexx32:

which I think is very silly. 😄

Regrettably, I echoed that phrase of yours in my response, but what I should have said is this:

Sticking a 😄 at the end of an insult doesn't nullify it.
(Again, it would be the first instance of a _postpositional_ sigil.)

Perhaps you meant to be playful, but we don't have a personal relationship, and intent is hard to read in its absence and with written communication in general.

I value your helpful contributions to this community and the constructive discussions you and I generally have; please don't sour that with flippant responses such as the above.

To be clear: if there is a problem with a proposal or argument, I absolutely want to hear about it - just not packaged that way (leaving aside the fact that this particular objection was unfounded).

mklement0 on 8 Apr 2019

👍1

Understood. I'll make an effort to be more clear and steer away from the flippant. 🙂

vexx32 on 8 Apr 2019

👍2

# Or with extras in the parser. Picture some better form of syntax here instead.
Get-Command "No wildcards"/@

Interesting idea. I have really disliked the whole Path vs LiteralPath schism from day one. And what, looks like we'd need a -LiteralCommand to solve this the "PowerShell" way.

I'd love to see the need for a LiteralPath parameter go away in PS 7. But you'd need a way to indicate to a command that wildcard chars should not be interpreted as wildcards for a parameter. Perhaps the above suggestion (or one like it) is the way to go but it would need to be able to handle expressions in general:

Get-Command $name/@
Get-ChildItem (Join-Path $PSScriptRoot "foo[bar]")/@  # < Hmm that syntax is problematic I think

I'd venture a guess that there are a ton of latent bugs in PowerShell scripts out in the wild because folks write script like this Test-Path $path not ever thinking that the user might supply a path with [] in it.

So I'm not sure how you solve this exactly but yeah, it would be so nice to be able to write a command for PS7 that doesn't require a LiteralPath parameter. That would also reduce parameterset complexity by eliminating one parameterset completely.

rkeithhill on 8 Apr 2019

👍3

Agreed on all points, @rkeithhill, except the details of the new syntax (see bottom).

folks write script like this Test-Path $path not ever thinking that the user might supply a path with [] in it.

Yes, it's unfortunate that -Path is the default, but, conversely, if -LiteralPath were, the convenience of Test-path *.txt would go away, because, at least with positional use you would then have to explicitly signal the desire to treat the argument as a pattern.

At least we now have alias -lp to make use of -LiteralPath less verbose.

But I agree that ditching the -Path / -LiteralPath schism and promoting a dedicated treat-this-potential-pattern-as-a-literal syntax is the way forward, not least because it can then be applied to _any_ (existing) wildcard-supporting parameter - such as Get-Command's -Name in the case at hand, without burdening implementers and end users with separate parameter sets.

On to bikeshedding:

As stated, I think that introducing a postpositional sigil is problematic - it differs from all other sigils, and you have to visually scan to the _end_ of the token to discern intent.

_[Doesn't work, because something like '''name' is syntactically a regular single-quoted string with an escaped ']_ ~~Using '' as a prefix is not the prettiest but~~:

~~it's unlikely to break existing usage~~
~~conveys the notion of literalness better~~
~~works equally with string literals (wether with literal or expandable content) and variables / expressions:~~

Get-Command ''name
Get-Command ''$name
Get-ChildItem ''$(Join-Path $PSScriptRoot "foo[bar]") # perhaps just ''(...) too.

mklement0 on 8 Apr 2019

P.S.:

I suggest combining the new syntax with @SeeminglyScience's proposed enhancement to the [WildcardPattern] class.

Future cmdlets can then ditch the current combo of [string]-typed parameters with informational-only [SupportsWildcards()] attribute for a [WildcardPattern]-typed parameter that is both self-documenting and provides useful functionality.

The parameter binder would then have to do extra work in both cases when encountering the new syntax:

When binding to [string]-typed parameters with [SupportsWildcards()], bind the result of [WildcardPattern]::Escape()
When binding to [WildcardPattern] parameters, construct an instance appropriately.

mklement0 on 8 Apr 2019

👍1

A small implementation recommendation:

It's likely that existing cmdlets would be reworked to accept WildcardPattern instead, if this is the case then compatibility with existing scripts could still be kept by implementing LiteralPath parameters like this

public string LiteralPath
{
    get => Path != null && Path.IsLiteral ? Path.Pattern : null; // or string.Empty
    set => Path = new WildcardPattern(value, isLiteral: true);
}

SeeminglyScience on 8 Apr 2019

👍2

I generally support @mklement0 proposal, but against that using '' as prefix.
Because '' is widely recognized by users as expression of literal strings, the following expressions look very weird to me and may cause confusion.

# Literal - only with '' prefix
Get-Command ''name 
Get-Command '''name'
Get-Command ''"name"

If anything, it might be better to use the @ as a prefix.
The @ is also used in C# as a prefix for escape, and I think that we will be familiar with it.

# Literal - only with @ prefix
Get-Command @'?'
Get-Command @"?"

However, it may be a problem in the following situations. It will need to be discussed.

$names = '?', 'gci', '*Firewall*'
Get-Command $names  # All elements as non-literal

# How to express $names as literal?
Get-Command @($names)           # array subexpression
Get-Command @$names             # best way?
$names | % { Get-Command @$_}   # not bad but a bit verbose
$names | % { Get-Command ([WildcardPattern]::Escape($_))}   # current solution

mkht on 9 Apr 2019

I agree that '' is ugly, @mkht, but more importantly, I've just realized that it won't work: something like '''name' is a valid regular single-quoted string with literal content 'name.

The problem I see with @ is its that it already has several distinct uses and, in the context of _strings_, it is used for _here-strings_, which in PowerShell do not imply literalness the way they do in C# (though perhaps the association with C# would be strong enough to suggest it in single-line use).
Also, an important convenience on the command line is the ability to use _unquoted_ tokens, which is _not_ an option with @, because @name is the syntax for _splatting_.
And it also means continued conflation of string literalness with pattern literalness.

The problem is that I don't think it's an option to introduce another sigil in argument-parsing mode, so we're stuck with the set of characters that are special at the start of a token: $ @ { ' " (

At the end of the day, whatever form we agree on that technically works and is concise enough should be fine, but I've since realized that there are more _conceptual_ issues, discussed below.

In light of them, I'm again leaning toward using the quoted / unquoted distinction as used in POSIX-like shells, but it's clearly not an ideal solution.

Splatting with wildcard parameters:

My hope was that we'd get away with a special syntax in _argument mode only_ (which, on second thought, is probably not a good idea in and of itself - though argument mode does have unique features), but due to _splatting_ we have to deal with _expression mode_ too.

The fundamental problem is the coupling of wildcard patterns with strings we cannot escape (anymore):

Strings moonlight as pattern literals in PowerShell (for _regex_ patterns too, but it's less problematic there), which is conceptually backwards: In JavaScript, for instance, you use explicit syntax /.../ for pattern literals (regexes, in this case), and using strings instead signals the intent to use a string's content literally; e.g., 'f.1'.replace('.', '@') // literal vs. 'f1'.replace(/./, '@') // pattern (regex)
As stated, however, we don't want to give up the convenience of using _unquoted_ pattern literals on the command line, as in Get-ChildItem *.txt.
- POSIX-like shells (such as Bash) use _unquoted_ use of arguments - even as _part_ of compound strings that also have _quoted_ parts - to disambiguate patterns from a strings; in other words: the special syntax for creating a pattern is _not to quote_.
  - However, it is _the shell itself_ that expands patterns (globs, which are limited to filesystem paths), and the target command only ever sees the _results_ - which is not how PowerShell does it.
  - My original proposal suggested using the unquoted / quoted distinction as well, which would work in _argument mode_, but not in expression mode with splatting: since quoting strings is a syntactic necessity in expression mode, the desire to create a pattern would have to be indicated explicitly _by way of a cast_ (in the absence of special syntax):

# Literal use
$htArgs = @{
  Path = 'ab[]'    # literal - consistent with proposed argument mode behavior
}

# Pattern use
$htArgs = @{
  Path = [WildcardPattern] 'ab[]'  # pattern - clumsy, and still involves a string
}

Also, while unquoted arguments in POSIX-like shells _consistently_ have special meaning, in PowerShell they'd only be special if they happen to bind to wildcard parameters.

Further, the need to enclose _variable references_ and _expressions_ in "..." in order to opt out of pattern matching _situationally_ would be obscure:

Get-ChildItem "$name"  # force literal use with quoting
Get-ChildItem "$($PSScriptRoot + '/name')"  # ditto

When binding to non-wildcard string parameters, just $name and ($PSScriptRoot + '/name') are enough.

Pipeline binding with wildcard parameters:

Earlier I proposed treating inputs from the pipeline as literals, but that's not how things currently work:

'ab[]' > paths.txt
Get-Content paths.txt | Get-ChildItem # FAILS, because the paths are treated as wildcards.

^{If you're wondering about the awkward sample filename: it was specifically chosen so that it results in an error when interpreted as a pattern, which not all invalid-as-a-pattern strings currently do - see #6733.}

With pipeline input we have no way of indicating the intent - pattern vs. literal - and while defaulting to pattern interpretation is consistent, it's also the less common use case: my sense is that _typically_ you'll want to feed _literal_ paths from a file. That is, to robustly process paths stored in a file - even now - you have to use Get-Content paths.txt | ForEach-Object { Get-ChildItem -lp $_ } rather than just Get-Content paths.txt | Get-ChildItem. I'm unsure what the right solution is here.

Also, when piping Get-ChildItem / Get-Item output, we definitely want to ensure _literal_ use, which could be done by making PSPath an alias name for parameter Path (as is currently the case with LiteralPath) and in the simplest form by providing an ArgumentTransformationAttribute that turns the path into a literal pattern, similar to what is shown in https://github.com/PowerShell/PowerShell/issues/6057#issuecomment-400107991.

A couple of asides re wildcard handling with file paths - but _only_ file paths:

@rkeithhill has suggested a higher-level PathParameter attribute that would let PowerShell handle the wildcard expansion - though the proposal is still framed in the context of the -Path / -LiteralPath bifurcation: https://github.com/PowerShell/PowerShell/issues/8819#issuecomment-460102241
- @Jaykul has experimented with a similar approach in https://github.com/Jaykul/Path
also, cmdlets that accept filesystem paths should arguably accept them _directly as System.IO.FileSystemInfo objects_, as output by Get-ChildItem / Get-Item or similar commands, to avoid the need for reparsing path strings - see https://github.com/PowerShell/PowerShell/issues/6057#issuecomment-399286074

mklement0 on 9 Apr 2019

🚀1 👍1

To help with the discussion, let me clear up some potential terminology confusion:

_Literal_ can mean two things:

Literal _notation_: "Honey, I'm $HOME." and 'This is not a pipe.' are both _string literals_.
Literal _content_: Only the content of 'This is not a pipe.' - a single-quoted string literal - is taken literally, or, to take a page out of C#'s terminology book, is used _verbatim_.

Therefore, what we're looking for here is a new, concise, bareword-friendly (not requiring quoting) notation for _verbatim patterns_ - i.e., something that when passed to a wildcard-supporting parameter is interpreted not as a pattern, but verbatim.

mklement0 on 15 Apr 2019

I think you would need to enhance WildcardPattern dramatically such that:

WildcardPattern would need to store the "OriginalString" somewhere (like URLs do) so it can implement ToString() usefully so that functions or cmdlets using [WildCardPattern] as a parameter can still use it as a string internally when necessary.
This means WildcardPattern would need to store whether it was literal so you could tell the difference between escaped and literal text -- which probably means the WildcardOptions enum needs a IsLiteral value so you can construct a real WildcardPattern
WildcardPattern would need to support casting hashtables (looks for Value and Literal or IsLiteral)
WildcardPattern would need to cast non-string objects with respect for a Literal or IsLiteral property...

Jaykul on 17 Apr 2019

👍2

Good suggestions, @Jaykul; such enhancements could help prevent bugs such as #9475.

Returning to the idea of a syntax for verbatim (escaped) patterns (which the parameter binder could translate into (enhanced) [WildcardPattern] instances for parameters declared with that type):

As an aside: The unquoted (glob) / quoted (verbatim) distinction already _is_ implemented, namely when _calling external programs_ on Unix, to emulate POSIX-like shell behavior; e.g., /bin/echo *.txt results in expansion to all matching filenames, whereas /bin/echo '*.txt' prints *.txt verbatim.

Given that @ already serves many different purposes, we can stick with it as follows and embrace the logic of conceiving of verbatim strings as _escaped patterns_:

@`... to create escaped wildcard expressions
@\... to create escaped regexes (obviates the need for [regex]::Escape() calls.

Note the prettiest, but concise, and I think the syntax hints at the intent.

Examples:

# Wishful thinking

# With a bareword (wouldn't be supported in expression mode)
Get-Command @`?
Get-Item @`file[1].txt

# With a variable
$file = 'file[1].txt'; Get-Item @`$file

# With a (sub)-expression
$ndx = 1; Get-Item @`('file[{0}].txt' -f $ndx)

# With -replace, in expression mode
'a * is born' -replace @\'*', 'star'
$replace = '*'; 'a * is born' -replace @\$replace, 'star'

mklement0 on 9 May 2019

This should be easy enough to implement with the code I added for the execution path. Where we prefer literal matches before wildcard matches.

TravisEz13 on 30 May 2019

Powershell: 'Get-Command ?' returns % (ForEach-Object alias) and other unrelated CommandInfo objects

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Most helpful comment

All 30 comments

Splatting with wildcard parameters:

Pipeline binding with wildcard parameters:

Related issues