Powershell: Command precedence should be honored irrespective of whether an auto-loading module is currently already imported or not

[mklement0]

As described in about_Command_Precedence, cmdlets take precedence over external programs with the same name.

However, this currently only applies to cmdlets _whose containing module has already been imported_ into the session at hand.

This makes the behavior hard to predict.

Given that the commands exported by auto-loading modules are known in advance, before actual import, there's no reason to consistently give them precedence, even if that entails import on demand.

Steps to reproduce

Run the following on _Windows_.

 pwsh -noprofile -c '$null>$env:TEMP\Get-Service; $env:PATH += '';'' + $env:TEMP; (Get-Command Get-Service).Source; Remove-Item $env:TEMP\Get-Service' |
  Should -Not -BeLike "$env:TEMP\*"

Expected behavior

The test should succeed, because submitting Get-Service should find the _cmdlet_ of that name in the auto-loading Microsoft.PowerShell.Management module - even though that module hasn't been loaded yet - not the external file in $env:PATH by that name.

Actual behavior

The test fails, because the external file in $env:PATH unexpectedly takes precedence.

Environment data

PowerShell Core 7.0.0-preview.4 

[rjmholt]

This is a good find. The problem is of course that proving a negative takes longer. We should look into fixing this, but it's probably a breaking change from Windows PowerShell and would also cause significant command invocation slowdown.

[BrucePay]

The whole point of the cmdlet cache is to automatically load cmdlets that aren't present and this works today. I therefore assume that the issue with Get-Command is that it's using a different code path than CommandDiscovery which is weird.

cause significant command invocation slowdown

I would much rather live with a minor, obscure inconsistency (if there is one) than take a big perf trying to fix it.

[mklement0]

it's probably a breaking change from Windows PowerShell

Technically, yes, but it's hard to imagine that someone actively relied on this behavior.

would also cause significant command invocation slowdown.

Why would there be a _significant_ slowdown?

The session already knows about the command names from not-yet-auto-loaded modules, so it sounds like you simply need to consider those too when looking up by name, which I wouldn't expect to have a tangible impact (though I know little about the plumbing).

Or are you specifically concerned that _immediately after session start-up_ it takes a while to collect all names from auto-loading modules, which would slow down CLI invocations with -Command?

[rjmholt]

Why would there be a significant slowdown?

Say you have a script called build.ps1 on the PATH.

When you run the command build, the proposed new behaviour must look for commands matching that before falling back to the script.

First we look for loaded commands. That's fast.

Then we look in the module analysis cache. Slower since we might have to touch the filesystem, but still acceptable. But still no result.

So now, we're forced to look on the module path for any module exporting a command or alias called build. This is because the cache might not capture modules newly installed or added. This is very slow, because:

• We must find every module on the module path
• We must inspect all manifests
• In the cases where manifests use wildcards, we must do module analysis (where we analyse script modules to see if we can work out their exports).

Only after we have looked at every module on the module path can we fall back to invoking the script file on the PATH. And because the filesystem can always have a new module plonked onto it without going through PowerShell (imagine extracting a zip onto the module path in explorer.exe), the cache can never prove the negative, so we always must look. Meaning script invocation now gets much much slower.

TLDR: Every script invocation without a slash in it takes a Get-Module -ListAvailable performance hit.

[rjmholt]

But I also agree that autoloading should be transparent and that it shouldn't affect command precedence. Invoking a script without the slash in it is something I personally wouldn't recommend and haven't seen much of. But I can imagine not everyone agrees with that second point.

[rjmholt]

I therefore assume that the issue with Get-Command is that it's using a different code path than CommandDiscovery which is weird.

We first try to resolve the command with CommandSearcher and then try autoloading:

https://github.com/PowerShell/PowerShell/blob/49e906bc3bd302e54ab1f5ad00c5bda8f8fba07f/src/System.Management.Automation/engine/CommandDiscovery.cs#L793-L822

CommandSearcher looks like this:

https://github.com/PowerShell/PowerShell/blob/49e906bc3bd302e54ab1f5ad00c5bda8f8fba07f/src/System.Management.Automation/engine/CommandSearcher.cs#L76-L243

[mklement0]

I appreciate the background information, @rjmholt - sounds like there is indeed a serious performance concern; let me mull that over a bit.

@BrucePay:

inconsistency (if there is one)

If the OP leaves any doubt as to whether there is an inconsistency, let me know.

[iSazonov]

How many name conflicts do we have? It seems it is an edge case.

For interactive session it is not critical too - user can fix his mistake by adding extension, or path to the command name, or install/import a module.

For reliable scripts we would use full qualified cmdlet names and full path to scripts to avoid name conflicts. Otherwise, there is always a way to break the script. (Even just changing PATH env variable)

[mklement0]

Given the performance implications, it makes sense to live with this inconsistency and simply _document_ it - see https://github.com/MicrosoftDocs/PowerShell-Docs/issues/4976 - so I'm closing this.

@iSazonov, @rjmholt

I don't expect it to be common either, but naming conflicts strike me as most likely with *.ps1 scripts placed in $env:PATH.

I use this technique personally to avoid having to cram too many functions into my $PROFILE (and when creating a whole module would be overkill), but, more importantly, the technique makes sense for convenient invocation of PSGallery-installed scripts (Install-Script).

With other kinds of external executables, naming conflicts with cmdlets are unlikely, though more so with functions with non-standard names and especially aliases.

Yes, you can work around the issue, but the points is that _you may not be aware of the need to do so_, especially given that the behavior differs _situationally_.

[iSazonov]

I don't expect it to be common either, but naming conflicts strike me as most likely with *.ps1 scripts placed in $env:PATH.

We could continue (async) search after we found first command and write warnings about possible name conflicts.

[mklement0]

Thanks, @iSazonov - I personally think it's sufficient to document the issue.

[rjmholt]

I'll make an edit just so there's no jumping around required