Powershell: Regression: Get-ChildItem -File -Filter *. no longer finds extension-less files on Windows

[mklement0]

in Windows PowerShell and v6.1.3 you can match files that don't have a filename extension with provider filter -Filter *..

This doesn't work anymore.

Note that -Filter ultimately uses the underlying Windows API for wildcard matching, which has legacy quirks, including the ability to match extension-less filenames with *.; it is _not_ the same wildcard language as _PowerShell's own_ (and *. wouldn't work that way with, say, -Include).

Steps to reproduce

Run the following test on Windows:

Describe "Extension-less file filtering"  {
  It "-Filter *. finds extension-less files" -Skip:(!$IsWindows) {
      '' > TestDrive:/foo
      (Get-ChildItem -File -LiteralPath TestDrive:/ -Filter *.).Name | Should -Be foo
  }
}

Expected behavior

The test should pass.

Actual behavior

The test fails.

Environment data

PowerShell Core v6.2.0 (6.2.0) on Microsoft Windows 10 Pro (64-bit; Version 1803, OS Build: 17134.706)

[vexx32]

🤔 if there's no extension, there's no period either, typically, no? So I can sort of see why it would be the way it currently is. I wasn't aware that Windows PowerShell could use such a pattern & why it might have been changed (if the change was at all intentional).

But I guess that depends on how you look at it? Weird...

[mklement0]

@vexx32:

Yes, there's no period, and if you use _PowerShell's_ wildcard language - Get-ChildItem * -Include *. - _nothing_ matches.

However, the -Filter argument is by design ultimately interpreted _by the Windows API_, and its wildcard language has legacy quirks - including the ability to find extension-less files with *. - try cmd /c dir *., for instance. This quirk happens to be useful, and people may have come to rely on it.

See https://stackoverflow.com/a/17739503/45375 for background information.

[msftrncs]

I wouldn't classify it as a quirk, its actually intentional, and comes from the DOS days when all _files_ had an inherent period at the end of their names. At that time, the period was a field separator, separating the name from the extension (and the period was not stored on disk), and of course only one period was allowed, the separator. Today in Windows, the last period in any pattern is considered to be that same separator between name and extension, and so receives special treatment, but yet does not stop it from matching other patterns that might also include a . before the actual extension, because in some ways you might consider the extension of a file to have multiple periods.

Note that DIR *. in CMD will match any file or directory either doesn't have a '.' or that the only '.' is at the beginning of the file name, as a period at the beginning of a file name is not considered to be defining an extension, since the name field would be empty, and that is not allowed.

[mklement0]

Good to know, @msftrncs. Undoubtedly, *. serves a useful purpose and is a handy way to find extension-less files (and directories).

As you state, the . isn't actually part of the filename, and by default you can't even create a file whose actual name ends in .; e.g., ab.; if you use the \\?\ prefix (invariably with an _absolute_ path), however, you can (New-Item ('\\?\' + $pwd.ProviderPath + '\ab.')), and cmd /c dir *. matches that too.

On a related note, if you don't use the \\?\ prefix, any _trailing spaces or periods_ your specify as part of a path are quietly ignored (Get-Item -Path ab..... will find file ab, for instance, with both -Path and -LiteralPath).

[roller]

Just to re-iterate what @mklement0 wrote, "trailing spaces or periods your specify as part of a path are quietly ignored". This is much bigger effect than just matching. You won't be able to open or interact with the file at all without using a \\?\ prefix.

Easy ways to create such a file:

• Create a Scheduled Task in Task Scheduler with a naming ending in a period
• Linux subsystem for Windows (eg, touch a.file.)

Note that powershell Core for linux running on WSL doesn't have the same restriction, so we can't really blame powershell. Also, Microsoft Docs recommend against creating such a file.

Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not.

[mklement0]

Thanks, @roller.

Yes, unfortunately the behavior is implemented at the level of the Windows API.

Even though there's now an opt-in to ignore the legacy 259-char. max-path-length constraint, there seems to be no analogous opt-in for supporting "irregular" file names (trailing spaces, periods).

It's implied by your quote from the docs, but just to spell it out: File Explorer, as part of the Windows (GUI) shell, _shows_ such file names, but cannot _act on them_.

Therefore, unfortunately, even with the long-path opt-in in place, you still need _always_ need prefix \\?with an absolute path to access such files.

[sba923]

Similarly, -Filter 2*.* in PowerShell Core 6.2.0 will not find items with names like 20190504, whereas it will in Windows PowerShell 5.1. Is there a place where all "breaking" changes in PowerShell Core (vs. Windows PowerShell) are documented? If there isn't, where would such a piece of document best be hosted?

[SteveL-MSFT]

Looks like this is the result of a change in corefx default behavior.

[sba923]

Looks like ;-) you don't want to document _that_ particular change in behavior, but rather remove it!

This is clearly going in the direction of making 7.0 as backwards-compatible with Windows PowerShell 5.1 as possible, which is the right thing to do IMVHO.

But what about documenting (in a single place) all areas where differences _will_ remain?

[SteveL-MSFT]

@sba923 for changes in behavior we are aware of, we do document them in our changelog as Breaking Change, but I don't believe we have a single place capturing all the differences which makes sense to have perhaps as an about topic cc @sdwheeler

[fMichaleczek]

@SteveL-MSFT @sba923 This page is not updated but contains a good start for all new PS6 users Breaking Changes for PowerShell 6.0

[SteveL-MSFT]

We should have a consolidated page for PowerShell 7 cc @joeyaiello