Powershell: Location completion should be case-insensitive

[vors]

Steps to reproduce

On Linux or OS X

• create a folder Foo
• type cd foo and hit Tab

Expected behavior

Completion will correct cd foo into cd ./Foo

Actual behavior

Nothing happens

Environment data

> $PSVersionTable
Name                           Value                                                                                                        
----                           -----                                                                                                        
PSVersion                      5.1.10032.0                                                                                                  
PSEdition                      Linux                                                                                                        
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                      
BuildVersion                   3.0.0.0                                                                                                      
GitCommitId                    v0.5.0                                                                                                       
CLRVersion                                                                                                                                  
WSManStackVersion              1.0                                                                                                          
PSRemotingProtocolVersion      2.3                                                                                                          
SerializationVersion           1.1.0.1   

Reasoning

• we already do that with / and \ on all systems.
• fish shell does it and it's very convinient.

[andschwa]

GNU Readline (and hence Bash) does this do with set completion-ignore-case on. I think most Linux developers probably use this (I certainly do).

[andschwa]

@vors do you think this would be a good Hacktoberfest project? It seems pretty reasonable and non-breaking, as it's an improvement to interactivity of PowerShell completion. I'm not certain, but I don't think this would actually require touching PSReadLine. I don't know much about the tab-completion system though.

[vors]

@andschwa yes, that's a very good proposition! I think that implementation would need to be in the engine, but the user preference should be exposed thru PSReadLine.

[lzybkr]

Yes, the implementation must be in the engine. As for non-breaking, I agree, but the implementation likes needs to use the file globber that is used for everything else, so care is required.

The engine already has some options which are passed in via a Hashtable.

TabExpansionPlusPlus exposes setting those options here

It's reasonable to also add an option to PSReadline though, or really, just make it the default and not configurable, because it's really not that useful for tab completion to be case sensitive.

[vors]

I agree. Can hardly imagine anybody wanting switch to case-sensitive behavior

[joeyaiello]

I agree that this needs to be moved out to PSReadline. In any case, given that no one else has complained, I think people actually prefer the case-sensitive behavior. For now, I'm moving it out to 6.1.0 (with the intention of making an issue in the PSReadline repo). I'm also adding it to the Linux/Mac UX project so we can triage it against everything else.

[lzybkr]

PSReadline does not generate completions, it just displays them. This is a core PowerShell issue.

[joeyaiello]

Weird, I don't know how I read the thread and saw "people want this out on PSReadline".

Everything else remains, though. Let's solve this later.

[jackfruh]

I want case insensitive as well - I've already set it for my shell on OSX - is there a way to set it in powershell? I'm ok if it's not a default so long as I can configure it.

[iSazonov]

@lzybkr Please comment for Up-for-Grabs - what is a right code place (line) for the fix?

[lzybkr]

I think this is the relevant code: https://github.com/PowerShell/PowerShell/blob/99236b18bbfeeeb4165c69028698b9ef92c29965/src/System.Management.Automation/engine/CommandCompletion/CompletionCompleters.cs#L4126

Because we use Resolve-Path (which we pretty much need because of PSDrives), maybe we need a new parameter to allow case insensitive resolution.

It's definitely Up-for-Grabs.

[mrsauravsahu]

The issue seems to be the Dir method in the FileSystemProvider.cs file. There are calls to EnumerateDirectories and EnumerateFiles which compare names of the files and directory against the wilcard in the location. Unfortunately, they do not ignore case. I'm having trouble implement this.

https://github.com/PowerShell/PowerShell/blob/cfe173ac0e90d1bc189c469464fd589f3f16ac3b/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L1611

(sorry, new to markdown)

[iSazonov]

@saurav-sahu You can left click on line number and GitHub will be generate URL for the line.

@lzybkr As @saurav-sahu mentioned we use CoreFX enumerations - they is case-sensitive on Unix without switch option. We could make a tricky workaround for Unix to fix the Issue or close as "by design" but It seems other Unix shells has an option to switch globally case-sensitivity (/cc @mklement0 ) - What is our way?

[jackfruh]

@iSazonov has a good point - I wonder if looking at source code from one of the existing Unix shells would be helpful?

[mrsauravsahu]

For Unix, the match for filename is done using the fnmatch function. Calls to which are here:

https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem/src/System/IO/UnixFileSystem.cs#L644

and

https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem/src/System/IO/UnixFileSystem.cs#L644

(The preview doesn't show up if it is a different repository)

If you look up the fnmatch function's signature, the last parameter flag does take the value FNM_CASEFOLD, which does case insensitive filename matching. All we have to do is implement an overloaded function in the corefx UnixFileSystem.cs to pass the FNM_CASEFOLD flag and the issue should be fixed.

[mklement0]

I can't offer spelunking in the bash source code, but I can provide some additional context:

There are two distinct, but related aspects of filesystem case-sensitivity:

• Interactively: How _tab completion_ works.

• Programmatically: How _globbing_ works.

In bash the two aspects are controlled _separately_:

• Tab completion (programmable completion): As @andschwa notes, it is the readline library that controls the behavior. You can place set completion-ignore-case on in your ~/.inputc file (or even system-wide, in /etc/inputrc) to persistently activate case-insensitive completion, or, ad hoc, with a more limited scope, you can run bind "set completion-ignore-case on"

• Globbing (filename expansion): This is part of bash itself, and is controlled via shell option nocaseglob: shopt -s nocaseglob makes globbing case-INsensitive. (There's a related nocasematch option, but it applies to _string_ comparisons only.)

bash categorically defaults to case-sensitive behavior, even on platforms where it _typically_ makes no sense, such as macOS.
(Ultimately, whether case-sensitive behavior makes sense comes down to a given _filesystem_'s intrinsic [non]-case-sensitivity.)

An important thing to note that even though shopt -s nocaseglob takes effect globally for the _current_ shell, _scripts_ do _not_ see it, because bash runs scripts in _child processes_ that do not inherit the parent shell's state.

---

Returning to PowerShell:

For case-insensitive filesystems, you'll likely want both tab completion and globbing to be case-insensitive, such as on macOS.

For case-sensitive filesystems, such as on Linux, _conceivably_ you want insensitivity for tab completion, for convenience and discoverability, while retaining sensitivity in globbing for stringency.

Certainly with respect to _globbing_, we must not mask the case-sensitivity of an underlying filesystem, at least not by default, so making both behaviors _opt-in_ sounds sensible.

The flip side is that having to opt in every time on a platform where not doing so wouldn't make sense is a nuisance.

Having _platform-dependent defaults_ is an option (Windows, macOS: case-INsensitive; Linux: case-SENSITIVE) - though you could argue that _consistent_ default behavior across platforms is more important.

(As stated, you could argue that the behavior should depend on the specific _filesystem_ being targeted, _situationally_, but I think that would be too obscure and confusing.)

Another aspect to consider: In PowerShell, scripts run in-process (unless you use an executable, extension-less script file with a shebang line), so a setting analogous to shopt -s nocaseglob (say, via a new preference variable such as $ps:NoCaseGlobbing) would then affect all scripts invoked in the session too, unlike in bash.

(This fundamental difference in how scripts are run will likely cause confusion in other cases too.)

[pcgeek86]

Guys, can we prioritize this for 6.0? Looks like some effort has already gone into fixing this, but I'm ramming into this issue on nearly a daily basis. I really hope we can get this in for 6.0 release. @SteveL-MSFT @joeyaiello

[SteveL-MSFT]

@pcgeek86 given that the PR hasn't moved forward, this is not something that will make it into 6.0.0.

[iSazonov]

The PR is a solution and ready but it is only _slow_ workaround. We could merge it if we really want to have it now.

[SteveL-MSFT]

@iSazonov at this time, we are only taking targeted fixes for RC2 and risk averse to introducing regression. Given the extent of the changes in this PR, I don't think we should take it for 6.0.0.

[jazzdelightsme]

This may already be covered, but: in addition to tab completion and globbing, I would hope this could also fix the situation where I just pass the path directly. Example:

1. mkdir Foo
2. cd foo

Expected result:
Working directory ends in \Foo.

Actual result:
Working directory ends in \foo.

One way this can really trip you up is in git: if you cded using the wrong directory casing and then run git commands (like "git log ."), git will think you have no files/history there. :(

(that's how I ended up here today)

I should also note that the steps above work as expected in cmd.exe on Windows.

[iSazonov]

@jazzdelightsme If your scenario is on Unix, with the change you'll can do:

mkdir Foo
cd f<Tab>
cd Foo<Enter>

[jazzdelightsme]

@iSazonov Tab completion will also work that way on Windows. It's just if you enter the complete path yourself, PowerShell keeps it as-is.

I previously thought this problem belonged as part of this issue, but on second thought, this issue seems to more particularly track tab completion and globbing on non-Windows... so I'll open a separate issue for it: #5678

[pcgeek86]

@SteveL-MSFT Can we fix this for 6.0.1?

[SteveL-MSFT]

@pcgeek86 we're only taking critical fixes for servicing releases. We should have a 6.1.0-preview release in early Feb. However #4699 isn't merged yet.

[iSazonov]

4699 is "stupid" solution - I explore another way.

[iSazonov]

In .Net Core we'll have https://github.com/dotnet/corefx/issues/25873

[SteveL-MSFT]

Working with test build of dotnetcore 2.1-rc1, this just works on macOS. Will add tests.

[SteveL-MSFT]

Just tried it on Ubuntu and, as expected, will need to add code to make it work on a case sensitive filesystem. However, the experience on macOS is SO much better!

[pcgeek86]

This is great news. :)

[SRGOM]

This is now in Powershell 6.2 released last week.

[SRGOM]

Is there a way to disable this too?

[SteveL-MSFT]

@SRGOM Just to be clear, you want an option for tab-completion to be case-sensitive? If so, please open a new issue.

[lihuelworks]

I want exactly that