Powershell: `Select-Object` writes errors when multiple patterns match the same property

[GeeLaw]

Steps to reproduce

New-Object PSObject | Select-Object ab | Select-Object *a*, *b*

Expected behavior

ab
--

Actual behavior

Select-Object : The property cannot be processed because the property "ab" already exists.
At line:1 char:42
+ New-Object PSObject | Select-Object ab | Select-Object *a*, *b*
+                                          ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (@{ab=}:PSObject) [Select-Object], PSArgumentException
+ FullyQualifiedErrorId : AlreadyExistingUserSpecifiedPropertyNoExpand,Microsoft.PowerShell.Commands.SelectObjectCommand

ab
--

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.2
PSEdition                      Core
GitCommitId                    v6.0.2
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Explanation

When user provides multiple patterns for Select-Object, he merely wants to select the disjunction of all these properties. The reproduction step here is just an example. In the real world, I was trying to take a summary of Outlook Contact object by $contacts.Items | select *email*,*name* and it Writes-Error because there are EmailXDisplayName properties for X = 1, 2, 3.

Select-Object should simply ignore properties already included in the output object if it is matched again. I.e., selecting a property twice makes no effect. This specifically implies that an error should not be written if one selects -property a,a -- this makes the life easier when the list of properties actually comes from merging two arrays.

This issue also reproduces in Windows PowerShell 5.1. Once confirmed, please also direct this issue to Windows PowerShell team.

[BrucePay]

@GeeLaw

Select-Object should simply ignore properties already included in the output object if it is matched again

In your example, you are clearly asking for _two_ properties when there is only one possible match. Continuing is obviously an error since it would only return one property when you explicitly asked for two. (I assume you asked for two because you actually expected two properties). So it seems to me that Select-Object is exhibiting the correct behaviour.

[mklement0]

@BrucePay:

I think @GeeLaw's point is that if you use wildcard-based property-name matching, multiple wildcard expression may overlap in the property names they match and that Select-Object should simply _ignore_ such accidental duplicates among the resulting list of property names.
(His real-life example was the Email* / *Name* wildcard-expression pair, one designed to match email-related properties, the other (presumably) person-related properties, yet they just happen to have overlapping matches.)

In other words: turn any resulting _multiset_ of property names into a _set_ for convenience.

This makes sense, given that property names must be unique, so no one should be surprised if duplicates are ignored.

Conversely, the current behavior of noisily complaining about duplicates is disruptive - and not always easy to anticipate.

[BrucePay]

@mklement0 He _explicitly specified_ that he wanted two properties when there was only one. That's clearly an error and should be treated as such. And you get the error spew because we don't require heterogeneous collections.

current behavior of noisily complaining about duplicates is disruptive - and not always easy to anticipate.

Errors are, by their nature, unanticipated. And error messages are supposed to be disruptive because, well, there's an _error_.

[mklement0]

@BrucePay:

He explicitly specified that he wanted two properties when there was only one.

No; from what I can tell the very point is to output _one_ property, _ignoring the duplicates_ in the _effective_ list of property names (even though the current sample command is flawed).

We can let @GeeLaw clarify, but that is certainly what _I_ have been talking about.

The real-life example in the original post tells a better story, which I've tried to recap in my previous comment.

And you get the error spew because we don't require heterogeneous collections.

What do you mean by _heterogeneous collections_?

Errors are, by their nature, unanticipated.

If you're using wildcards, you're playing in "fuzzy land", by definition - otherwise you'd use explicit names.

Wildcards _not_ matching anything already _is_ silent - as opposed to an error condition; e.g., Get-Item / | Select-Object foo* or, in the context of the file system, Get-ChildItem *nosuch*.

By analogy, two wildcards that happen to produce overlapping results too should not produce an _error_ - the sensible resolution _in the given context_ is to _quietly weed out duplicates_ :

• As stated, no one should be surprised at that, given that it should be understood that property names on output objects must be unique.

  • The upside is that wildcard-based property-name matching can be applied safely, without needing to worry about accidental overlap.

[GeeLaw]

In reply to @mklement0 (1st reply to the issue): Thanks for your reply. I mistyped the second command in the first snippet. It should have been Select-Object ab, and it's now fixed.

In reply to @BrucePay (2nd reply): See "Explanation" for a real-world example. The "Steps to reproduce" consists of a minimal reproduction. If you don't use Outlook object model, I'll write down a more familiar scenario:

# Let's say the REST method returns JSON, which gives the following properties:
# FirstName, LastName, FullName,
# Email1Address, Email2Address, Email3Adddress,
# Email1DisplayName, Email2DisplayName, Email3DisplayName,
# Birthday, Notes, SignificantOther, Company, Anniversary
$result = Invoke-RestMethod <# parameters ignored #>;
$result | Select-Object -Property *name*, *email* | Out-GridView;

What I would like to do is to inspect properties related to name and emails. Since Email1DisplayName matches both pattern, Select-Object tries to add it to the resulting object twice.

If you are talking about select a, a case, I would suggest the cmdlet not detect such case (ignore the error, too) because of logical simplicity and the possible scenario of merging two sets of properties to select. If we were to detect error (not taking list merging into consideration), we must keep track of each property, whether it is selected by the following possible combinations:

• none;
• non-wildcard once;
• non-wildcard multiple times;
• wildcard once;
• wildcard multiple times;
• non-wildcard once, wildcard multiple times;
• non-wildcard multiple times, wildcard once;
• non-wildcard multiple times, wildcard multiple times.

And we must define which of these cases produce an error. It'll be cumbersome in the code and for people. Instead, I suggest we keep it simple by ignoring re-selection.

P.S. What I meant by list merging is:

$mandatoryProps = @('a', 'b', 'c')
$userProps = Get-PropertiesUserWantsToSee;
$toSelect = $mandatoryProps + $userProps;
$sourceObjects | Select-Object $toSelect | Out-GridView;

It is possible to use [System.Collections.Generic.HashSet[string]] as a workaround, but you need to be careful -- construct the object with case-insensitive comparator or .ToLowerInvariant() when inserting the elements.

In reply to @mklement0 (3rd and 5th reply): Yes, your interpretation gets what I intended to convey.

[frankfuu]

@GeeLaw thanks for posting this example. I was trying to ask the same thing but all i can think of was contrived examples like why isn't gci | select Name, Name possible? 😢

I think the ability to do this makes reporting and discoverability of data alot clearer and convenient.

What was your workaround?

[GeeLaw]

@frankfuu I didn't try finding a workaround -- I suppose this is only an issue for interactive usage. For scripting scenarios, it would be better to know in advance the exact schema, eliminating the need to use a wildcard and even Select-Object to select properties.

If you really hate the ErrorRecords, temporarily suppress them by -ErrorAction Ignore. Note that Ignore is better than SilentlyContinue if you are sure that the command would run through without problem if the issue were resolved, since Ignore does not put ErrorRecords into the $Error automatic variable.

[frankfuu]

@GeeLaw this is great! I'll stick with temporary supression for interactive usage from now.