Powershell: Get-Module with a non-wildcard module name that doesn't match a loaded / available module quietly returns nothing

[mklement0]

Follow-up from #7434 and related to #7521.

Note that this is a longstanding, albeit undocumented, behavior, and changing it now may break existing scripts, so there are two possible resolutions:

• Conclude that changing this would break too many scripts and therefore make do with _documenting_ the surprising / inconsistent behavior.

• Conclude that this is a Bucket 3 change and make the change nonetheless.

Steps to reproduce

Ask for a nonexistent (loaded) module with a name that _doesn't include wildcard characters_:

Get-Module NoSuchModule

Expected behavior

A non-terminating error indicating the specified module isn't _loaded_.
$? should indicate $False

Analogously, with -ListAvailable, the error message should state that a module by that name isn't _available_ (in any of the dirs. listed in $env:PSModulePath).

Actual behavior

Nothing ([System.Management.Automation.Internal.AutomationNull]::Value) is output, and $? reflects $True.

While this behavior may make sense for _wildcard_-based arguments, it doesn't for literal ones.

Compare this to the behavior of cmdlets such as Get-Item, Get-ChildItem, Get-Content, ..., which all fail if a given non-wildcard path doesn't identify an _existing_ file.

Environment data

PowerShell Core v6.1.0-preview.4 on macOS 10.13.6
PowerShell Core v6.1.0-preview.4 on Ubuntu 16.04.4 LTS
PowerShell Core v6.1.0-preview.4 on Microsoft Windows 10 Pro (64-bit; Version 1803, OS Build: 17134.165)
Windows PowerShell v5.1.17134.165 on Microsoft Windows 10 Pro (64-bit; Version 1803, OS Build: 17134.165)

[BrucePay]

As a matter of _convention_, we usually generate an error for a non-wildcard query e.g. get-item nosuchitem returns a error, but get-module nosuchmodule quietly returns nothing.

[Jaykul]

@BrucePay It doesn't seem to me that this has to do with wildcards, but simple the possibility of multiple outputs

[SteveL-MSFT]

In the Get-Module case, using -Name is really a filter and not literal since you can get multiple versions of the same module. So in that case, it should act the same as a wildcard.

[mklement0]

@Jakul: The output behavior is incidental. What matters is whether the -Name argument is designed to uniquely identify an existing entity using a literal (non-wildcard) name.

As such, @SteveL-MSFT, you're right that the case of Get-Module is _ambiguous_:

• (a) On the one hand, you could argue that its -Name argument identifies _multiple entities_, if you conceive of the _versions_ of a given module as _distinct entities in their own right_.

• (b) On the other hand, you can conceive of a module with a given name to be an _abstract, yet distinct_ entity, and conceive of its _versions_ as its _specific incarnations_.

• (c) As an aside: Even the unambiguous case of asking for an _exact module version_ via the RequiredVersion key of a hashtable representing a fully qualified module module passed -FullyQualifiedModuleName currently quietly returns nothing if the specific version is not found - something that should also be fixed.

---

The (a) interpretation is at odds with the parameter name: if the argument is really a _filter_, it shouldn't be named -Name, since in most cmdlets -Name unambiguously identifies a specific, existing entity.

Viewed through the lens of (b), -Name can still be thought of unambiguously identifying the module _in the abstract_, irrespective of which and how many of its versions are loaded.

Perhaps needless to say, (b) makes more sense to me in the grand scheme of things (leaving backward compatibility aside).

Overall, separating the aspects of _retrieving_ items from _testing their existence_ is indeed the way to go.

[SteveL-MSFT]

It bothers me that FullyQualifiedModuleName can be a filter, but it is what it is and we may just have to live with those inconsistencies for backwards compatibility sake. I think the best we can do is ensure that new cmdlets follow the intended guidelines to have consistency as well as promote more Test- cmdlets removing the need for Get- to be multi-purpose.

[SteveL-MSFT]

Created https://github.com/PowerShell/PowerShell/issues/7562 for Test-Module

[essentialexch]

This is not a Bucket 3 Change. At best, it's a Bucket 2 Change (for those who want the change); and from my perspective would require a "Breaking of Contract". This behavior of Get-Module was used and documented by The Scripting Guys at least as far back as 2010 (https://blogs.technet.microsoft.com/heyscriptingguy/2010/07/11/hey-scripting-guy-weekend-scripter-checking-for-module-dependencies-in-windows-powershell/). I've got my version of their function Get-myModule all over my published scripts and I've widely recommended that resource over the years for people with similar needs.

Is it easy to fix? Sure. Is there any way to fix tens of thousands of already deployed scripts? No.

There is a middle-of-the-road compromise: change $? but don't emit an error. And, of course, document the behavior. This does smack of being a kluge and rather ad-hockery.

[TimCurwick]

This proposal is just wrong. I put up with this bad behavior where it already exists for backwards compatibility, but spreading it across the ecosystem in the name of consistency is not appropriate.
Finding nothing when there is nothing to find is not an error. It is the correct answer. It is the correct result.

[rkeithhill]

Finding nothing when there is nothing to find is not an error.

Not that I'm necessarily arguing in favor of this breaking change but if I'm trying to get a file and that file doesn't exist, I expect a non-terminating error - particularly in the interactive console:

PS> Get-Item xyzzy
Get-Item : Cannot find path 'C:\Users\hillr\xyzzy' because it does not exist.

There is a lot of shell history that says this should be an error e.g.:

# Bash
~$ ls xyzzy
ls: cannot access 'xyzzy': No such file or directory
~$ echo $?
2

and

# CMD
C:\Users\hillr>dir /b xyzzy
File Not Found

C:\Users\hillr>echo %ERRORLEVEL%
1

I "get" that dir/list could be considered a bit different than get, but get is what we've got in PowerShell for listing dirs. :-)

[essentialexch]

As far as I know, to the best of my knowledge, we are no longer discussing the general case, but instead the specific case about "Get-Module".

[rkeithhill]

In the Get-Module case, using -Name is really a filter and not literal since you can get multiple versions of the same module.

The same thing applies to Get-Process - try Get-Process svchost on Windows (Get-Process init on Linux) and you'll see. Yet Get-Process xyzzy reports a non-terminating error. Sigh...

The equivalent (sort of) on Linux also errors:

~$ ps 8675309
  PID TTY      STAT   TIME COMMAND
~$ echo $?
1

Of course, ps doesn't spew a sea of red text either.

[Jaykul]

This command is the tip of the iceberg. It's the _ultimate test case._

If we decided to change this -- _there are no more limits to what can be changed_. Get-Module is one of the most common commands ... and as far as most users are concerned, the current behavior _is correct_, if only because it is how it has always behaved.

The REALITY is that Get-Module Foo is how you test if a module is loaded. It is a combination function: we use it to test, and we use it to get -- but most people use it to test.

A decade worth of code has been written based on this _expected_ behavior. In fact, he _get_ functionality isn't what people usually care about. Search _just on GitHub_ for if(Get-Module and you'll find over 148,000 results. Compare that to a search for just Get-Module and there's only about 2,000 more.

As a result, any attempt to "correct" this will be annoying to all current users, in addition to being a breaking change for compatibility with Windows PowerShell.

--

If we start producing errors from core commands where there have been none before, the experience for users is going to be a lot like a disaster: broken scripts and modules everywhere -- things that used to work, but don't work anymore (and nobody understands why, because they didn't write this).

Writing Test commands doesn't solve the problem of breaking the existing command -- changing the command requires rewriting hundreds of thousands of lines of code, and the Test command simply won't be available on Windows PowerShell.

[ChrisLGardner]

Let's take @Jaykul's data and extrapolate a little from there. If we assume that 10% of the people writing PowerShell scripts and modules are using Github (a very high %, I'd be surprised if the number using any source control at all is around that % nevermind using github) then that's at 1.48 million scripts that will break with this change. Or to put it another way, that's 1.48 million scripts that need to be rewritten before they can be used on PS v6+.

This change may only produce a non-terminating error but how many script writers know that there are even non-terminating errors? How many people using scripts written by others are likely to know? I'd imagine most of them will see red text and assume a big problem has occurred, because for the vast majority of users there is no difference between the error types.

Personally I don't want to have to rewrite all of my scripts to have -ErrorAction SilentlyContinue on every non-wildcard Get-Module call (or wrap them in try/catch with -ea stop to make sure it's just a module not found exception). I've only got a few hundred scripts that I'd need to change but that's still a good week of effort to rewrite them and at least that much time again to test them all (a lot written before I started using Pester).

I'd also definitely argue against it being a Bucket 3 change, this is at least a Bucket 2 change or higher. This has been documented by a variety of blog posts and a lot of code samples all over the internet, including "official" Microsoft sources like Hey Scripting Guy linked above, it just might not be officially documented and that's a very easy fix to make.

[markekraus]

I think this change complicates things beyond just adding -ea SilentlyContinue. If this pattern is adopted en mass, users will be forced to redo large sections of code to accommodate where Get- commands currently return nothing when nothing is found but do produce errors whenever other conditions occur (such as the queried service being unavailable). Where before the presence of an error indicated a failure and no results indicated not found, now the user must do error inspection. This pattern will result in error message scraping because many authors don't make error generation easy to consume.. because error generation in PowerShell is neither intuitive nor easy.

Does adding a ton of test- commands fix that? well, somewhat. But now I need more code to do the same thing I have been doing for yeas and I need to make massive adjustments to my existing Windows PowerShell code to make it work with core. If I'm a small shop, this would just add another reason to not move to core and stay on Windows PowerShell.

Was the original design wrong? Maybe. But it has resulted in a unique-to-PowerShell shortcut that has been widely used as a best practice recommendation. The current behavior doesn't break anything. It doesn't stand in the way of progress. The new behavior, if adopted from the start, would have built a body of work and recommendations that would have maintained it. However, now a decade of examples, text books, blog posts, help documents, course materials, etc are all seeds of confusion and frustration for new users who come in to this after the behavior is changed.

What does changing this behavior solve and at what cost?

[potatoqualitee]

Please, no.

[FriedrichWeinmann]

I've got to chime in on the _DON'T DO THIS!_ faction.

I see no problem being solved by this change. I see lots of problems being created.
Do I like being clean and consistent? Yes, totally! I'm a German neat-freak: My indentation being wrong will cause me discomfort until I fix it.
But there has to be a reasonable balance between benefit and cost, and this is way off.

Benefit:

• More neatness

Cost

• Decades of confusion for beginners, with dated advice not consistently working
• Millions of scripts that need fixing _now_
• Millions of scripts more that _will_ need fixing, because someone was following old guidance or ignoring new advice because he currently has no business need to switch
• Hurt adoption, because common examples will already drown the users in a sea of red
• Add continuing maintenance cost to keep code compatible across editions

Also, I do not believe standardizing this will make it easier to learn PowerShell to begin with, even if a user only learns solely by new documentation: I have yet to find a beginner to powershell - outside of (very) senior programmers - to think in that level.
Virtually everybody will accept that using:

if (Get-Module somemodule)

Is a valid way to test for a module being imported and react accordingly.
Complaining about it not throwing a non-terminating exception will simply not occur to them (They'll be more like "Yay! My code finally works!")

So: No, please don't do this.

[SteveL-MSFT]

Seems like there's some confusion about what the @PowerShell/powershell-committee actually concluded. To summarize:

• the intent is for a literal to return a non-terminating error, but wildcards/filtering to return nothing
• we do not plan to accept changes to existing behavior to make them consistent, but any new cmdlets should adhere to this
• even without changing the existing cmdlets, we still want to introduce Test-* cmdlets and make that the norm and best practice
• if not documented, we need to document existing cmdlet behavior where it deviates from the intent

[TimCurwick]

Steve,

1. Are you confirming that Get-Module will not be changed?
2. I strongly disagree with the conclusions of the committee. The behavior (for example) of Get-Module -Name $ModuleName should not change based which characters are present in the value of $ModuleName. And the appropriate behavior in the case of a null result is to return a null result.
3. While there may be exceptions, this is the preferred behavior for almost all future Get commands, as well. What makes PowerShell great to work with is how simple it is to do simple things. Dynamic type conversion makes working with nulls trivial. Forcing us to add unnecessary complexity to our code in order to handle unnecessary errors with no value added is unnecessary and serves no purpose.

[RamblingCookieMonster]

For what it's worth, here's an expectation that I would imagine more than myself subscribe to:

• Error doing a thing? Error.
• Success doing a thing, no data found? No data returned.

Get-ADUser follows the other pattern, erroring out when AD is successfully queried for a specific identity, when nothing is found. It's not pleasant coding around this

I'm not quite as savvy a developer, maybe this is a pattern found all over that I'm just unaware of - that said, PowerShell tends to aim towards convenience and doing what a sysadmin would expect, rather than adding technical hoops you might find in other languages and shells

Cheers!

[RamblingCookieMonster]

Oh, and on -ErrorAction SilentlyContinue - imho that is never a valid workaround to this. This ignores any actual errors doing a thing. Someone might handle a data not found scenario differently than a error querying data source scenario...

[joeyaiello]

I can only fully speak for myself, but AFAIK, no one on the @PowerShell/powershell-committee was ever proposing that we change Get-Module. We were just using it as a foil to Get-FormatData/Get-TestData to create a more thougtful rationale around the issue of returning (or not returning) a non-terminating error in the Get-SpecificThing foo case.

So yes, to be very explicit: I am not in favor of changing Get-Module.

That being said, my rationale is slightly different than most of what I'm seeing in this thread.

Sometimes, Really Important Things™ get to break consistency with all the other stuff in a language. This might be because it's hard to maintain consistency all the way to your basest of cases, but often it's because these Really Important Things have to be built (and left untouched) very early on in the development of a language, before standard patterns and practices are well defined. Very rarely does it make sense to change Really Important Things, even if you have new standards of consistency going forward.

In all cases of breaking changes decisions we've made thus far, there are two competing forces at play:

• The benefit that a breaking change would bring to the ecosystem
• The negative impact of making that change, including the scope or volume of scripts/users impacted, and the amount of work required by users to address that impact

One example of a Very Important Thing is our return semantics. For better or for worse (and let's please not have this conversation here, it's just an example 😊), PowerShell returns everything written to the pipeline. We could never change this. The benefit is, I'll say, debatable at best, and the negative impact would be huge and difficult to correct. But, we didn't carry that baggage forward into PowerShell classes. And given that more dev-y users are likely to be the ones utilizing PowerShell classes, there was benefit to breaking consistency with the old thing and having more traditional return semantics.

What we did say on the call is that we want to revert the original Get-FormatData change in 6.1 so that we can fully enumerate other cmdlets that we might want to change (e.g. Get-TypeData) all at once in 6.2.

But we're still going to do an assessment of which ones we should change, and Get-Module doesn't pass muster for me. (Get-TypeData probably does given it's lower usage, though...)

[SteveL-MSFT]

@PowerShell/powershell-committee reviewed this, in this case Get-Module -Name is a filter even though it implies a singleton as you can get multiple outputs in the case of multiple versions of a script module loaded. So this is currently by-design.