Powershell: Some scripts / files with wildcard metacharacter [ in the file name or path cannot be invoked or redirected to

[mklement0]

^{Edited for the current status as of PowerShell Core 7.0.0-preview.5}

Note:

• Originally, this issue was also about the inability to _execute_ files whose literal names happen to be syntactically invalid wildcard patterns.

• This execution aspect has been fixed in #9202 (but wasn't back-ported to the currently stable version, v6.2.3)

Seemingly, all external-executable invocation methods ( ., &, Start-Process, powershell -File) as well as redirections > / >> mistakenly interpret script / file paths as _wildcard patterns_, and if a path that contains [ happens not contain a syntactically valid range expression, invocation and redirection break.

• In other words: The problem occurs, unless each instance of [ happens to be part of a a _valid_ range expression; e.g., [ab] (valid) vs. [a (invalid).

Fundamentally, external-executable / redirection output paths should be treated as _literals_ in these situations.

Steps to reproduce

Run the following Pester tests.

Describe "Redirection with and invocation of literal file paths that are invalid as wildcard patterns" {

  BeforeAll {
    Push-Location testdrive:\
    $f1 = './t[.ps1'
    $f2 = './t[.txt'
    New-Item -Type File $f1 -Value "'hi'"
  }

 # Passes now, due to fix in #9202
  It "Script file can be dot-sourced." {
    . $f1 | Should -Be 'hi'
  }

 # Passes now, due to fix in #9202
  It "Script file can be called." {
    & $f1 | Should -Be 'hi'
  }

 # Passes now, due to fix in #9202
  It "Script file can be called via CLI with -File" {
    pwsh -noprofile -file $f1 | Should -Be 'hi'
  }

  It "Redirection works with new file (>)" {
    & { 'hi' > $f2; Get-Content -LiteralPath $f2 } | Should -Be 'hi'
  }

  It "Redirection works with existing file (>)" {
    New-Item -Force -Type File $f2
    & { 'hi' > $f2; Get-Content -LiteralPath $f2 } | Should -Be 'hi'
  }

  It "File path is passed correctly to external programs" {
    if ($IsWindows) {
      cmd /c type ($f1 -replace '/', '\') | Should -Be "'hi'"
    } else {
      /bin/cat $f1 | Should -Be "'hi'"
    }
  }

  AfterAll {
    Pop-Location
  }
}

Expected behavior

All tests should pass.

Actual behavior

The > tests fail on all platforms.

In v6.2.3, the invocation tests fail too.

Environment data

PowerShell Core 7.0.0-preview.5

Originally reported for PowerShell Core v6.0.0-beta.7.

[iSazonov]

It seems we should escape (try tab completion):

. './t`[`].ps1'

[mklement0]

No, I think there is no reason to attempt wildcard pattern matching at all in all these scenarios - the paths should be taken as literals, which obviates the need for escaping.

[iSazonov]

I discover this in #4699 and we can see this in https://github.com/iSazonov/PowerShell/blob/c1b300caeef8e242e484eb827667931fde714796/test/powershell/Host/TabCompletion/TabCompletion.Tests.ps1#L752

[mklement0]

When tab-completing for _parameters that support wildcards_, the escaping you mention makes sense:

# -Path implied, which supports wildcards -> escaping
Get-Item './t`[`].ps1'  # The result of typing Get-Item ./t<tab> *on Unix*

# -LiteralPath specified, which does NOT support wildcards -> NO escaping
Get-Item -LiteralPath ./t[].ps1  # The result of typing Get-Item -LiteralPath ./t<tab>

Since there is no good reason to treat the file path in scenarios such as . <file-path> as a wildcard pattern:

• _tab completion_ should work the same way it does with Get-Item -LiteralPath - treat it as a literal.

• and _execution_ of such a path should obviously succeed.

[iSazonov]

Here can be a root of the bug.

[daxian-dbw]

@mklement0 The file redirection is broken for this scenario, but the invocations seem working fine on beta.7:

PS:4> '"Hi!"' | Set-Content -literalpath './t[].ps1'
[F:\tmp]
PS:5> ./t[].ps1
Hi!
[F:\tmp]
PS:6> . './t[].ps1'
Hi!
[F:\tmp]
PS:7> & './t[].ps1'
Hi!
[F:\tmp]
PS:8> E:\pscore\powershell.exe -noprofile -file './t[].ps1'
Hi!
[F:\tmp]
PS:9>

The $PSVersion:

PS:9> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta
PSEdition                      Core
GitCommitId                    v6.0.0-beta.7
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Am I missing something?

[mklement0]

@daxian-dbw:

Sorry, I didn't realize that the behavior varies with similar filenames present in the same directory.

I've updated the original post with scenarios that should fail predictably with initially empty directories, on both Windows and Unix platforms.

The key is to use something like t[.ps1 - something truly invalid if interpreted as a wildcard pattern.

[daxian-dbw]

@mklement0 Thanks for the clarification!

[TravisEz13]

@mklement0 This issue is about redirection. Which occurs at execution time, but it is not about what you execute. I think this would drop it out of the Defense in Depth category and into a functional issue. If you believe this is a security issue, Please report this issue to [email protected] per the contribution guidelines and not via GitHub. If they determine that the issue, can be disclosed publicly, feel free to post security issues here.

[mklement0]

@TravisEz13, quoting from the OP (emphasis added):

Seemingly, all external-executable invocation methods ( ., &, Start-Process, powershell -File) as well as redirections > / >> mistakenly interpret script / file paths as wildcard patterns,

Similarly, the repro steps show _execution_ commands.

So, as stated in #9202, the latter therefore addresses _part_ of this issue, correct?

As for the _other_ part of this issue, redirection: no, I don't think it is security-critical.

[mklement0]

(The execution issue is _framed_ differently here, because my angle was the _inability_ to execute, but my understanding is that #9202 will fix that too.)

[TravisEz13]

Yes... sorry... I was reading only the parts, not address by #9202 . I'm in a bit of a rush to get that PR done and some of the suggested changes have broken the PR. I'm trying to track back which change broke the PR.

[mklement0]

Thanks for letting me know, @TravisEz13. I just wanted to make sure we keep track of what has and hasn't been fixed yet. I've added a note to the OP.

[TravisEz13]

@mklement0 I updated you comment. We use Defense in Depth rather than security and especially critical for this type of change as it would likely require social engineering or an existing exploit to be useful.

[mklement0]

Thanks, @TravisEz13 - didn't know the customary lingo.

[TravisEz13]

I haven't found a great current document, but PowerShell has had two types of issues that I'm aware of.

Defense in Depth and Security Boundary Bypass.

This document defines Defense in Depth and Security Boundary: https://www.microsoft.com/en-us/msrc/windows-security-servicing-criteria

It also has a link to the bar to call an issue critical near the top called meet the bar for servicing.

[mklement0]

I appreciate the pointers, @TravisEz13.

[msftrncs]

Attempting to escape the wildcard characters from a redirection target argument is still broken. I ran in to this issue testing file path completion scenario's.

[arberg]

I suspect this bug I found is this issue, but since I'm not sure I think its valuable if I add the details:

Somehow create the file: c:\temp\ivydata-[2.4,).properties
piping to the file in powershell does not work, it gives the same error.

> pwsh -version
PowerShell 6.2.3

C:\temp> Test-Path -PathType Leaf "c:\temp\ivydata-[2.4,).properties"
Test-Path : The specified wildcard character pattern is not valid: ivydata-[2.4,).properties
At line:1 char:1
+ Test-Path -PathType Leaf "c:\temp\ivydata-[2.4,).properties"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Test-Path], WildcardPatternException
+ FullyQualifiedErrorId : RuntimeException,Microsoft.PowerShell.Commands.TestPathCommand

C:\temp> dir \temp\*.properties | %{test-path $_}
Test-Path : Cannot retrieve the dynamic parameters for the cmdlet. The specified wildcard character pattern is not valid: ivydata-[2.4,).properties
At line:1 char:28
+ dir \temp\*.properties | %{test-path $_}
+                            ~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Test-Path], ParameterBindingException
+ FullyQualifiedErrorId : GetDynamicParametersException,Microsoft.PowerShell.Commands.TestPathCommand

ps: I found the file under c:\users\.ivy2

[mklement0]

Thanks, @arberg, but the behavior you're seeing is by design, because _positionally_ passing a path to Test-Path implicitly binds to the -Path parameter, which by design interprets its argument as a wildcard expression. If you use Test-Path -LiteralPath "c:\temp\ivydata-[2.4,).properties", the problem goes away.

By contrast, the issue at hand is that when you use > / >> you (a) don't get a choice about how the path is interpreted and (b) the implied behavior should be the same as with -LiteralPath, i.e. interpretation as a _literal_ path.

In the past, the same problem existed for _invoking_ files, but that has been fixed in #9202 (though not back-ported to 6.2.3).

I've updated the OP with the current bug status and Pester tests.

[msftrncs]

Here is the code for the redirection:

https://github.com/PowerShell/PowerShell/blob/d58a82ad19fbfad81e85778c8b08cb1b28f58fce/src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs#L1153-L1173

Of course, changing this, makes it a breaking change.

[msftrncs]

Also, it really seems like its Out-File that is broken. Its not processing wildcard paths correctly, when they have been escaped. Doesn't seem to be something specific to Out-File, as it calls the LocationGlobber to handle wildcards ...

[mklement0]

Thanks for the pointer, @msftrncs:

As for broken escaping in wildcards - see #7999

As for making > / >> bind to Out-File's -LiteralPath parameter instead being a breaking change:

Technically, yes, but the current behavior is unexpected, always fraught and mostly useless:

• It is fraught, because the file getting targeted after matching the wildcard may not be the one you intended to target. (And, of course, if _no_ file matches, you'll get an error.)

• It is mostly useless in that you can't target _multiple_ files with a wildcard pattern.

  • If that worked, it would definitely be fraught in itself; in fact, it currently does work with Set-Content, which I feel should switch to treating its paths as _literals_ too - see #6729 and #9588