Powershell: Powershell7 - Select-String with -AllMatches by default in favor of highlighting

[copdips]

Summary of the new feature/enhancement

Powershell 7 introduces the Select-String highlighting by default, but it highlights only the first match of each line. IMHO, a common user experience is to see all the matches be highlighted.

Proposed technical implementation details (optional)

Set -AllMatches to $true by default. Maybe other solutions, if the default switch value should be $false.

[iSazonov]

It is very popular cmdlet and changing the default would huge breaking change.

[copdips]

Yes I know, but as v7 is considered as a new major version, could we accept some breaking changes ?

We've already added a breaking change to the cmdlet Export-Csv by adding -IncludeTypeInformation, and making -NoTypeInformation as default. Maybe we can do the same for Select-String this time.

I really hope we can make Powershell more and more simpler to use and human friendly.

[iSazonov]

I think it is Bucket 1. It will breaks both scripts and code.

[jhoneill]

Actually it need not break anything

_"it highlights only the first match of each line. IMHO, a common user experience is to see all the matches be highlighted."_

The formatted output could highlight all matches in a line without changing the matches returned.

The way matchinfo objects format needs fixing. Try running select-string remotely or in a job and serializing/deserializing means it doesn't output because the format XML relies on to string doing custom things depending what was specified when select-string ran. If someone cleaned that up (and I've looked) adding highlight all matches would be simple to add..

[SeeminglyScience]

The formatted output could highlight all matches in a line without changing the matches returned.

In order to do that, the formatter would need to rerun the regex every time the MatchInfo is formatted. That's pretty heavy for a formatting. The bigger issue with that imo is that it implies you would be able to access the Match objects from MatchInfo like you can when AllMatches is specified.

[jhoneill]

The formatted output could highlight all matches in a line without changing the matches returned.

In order to do that, the formatter would need to rerun the regex every time the MatchInfo is formatted. That's pretty heavy for a formatting. The bigger issue with that imo is that it implies you would be able to access the Match objects from MatchInfo like you can when AllMatches is specified.

True ... or at least not obviously wrong. IIRC the formatter relies on a custom .Tostring() method which is why serialized/deserialized match info objects don't output. It's not very efficient as-is.
This
select-string -Pattern conditional -Path *.ps1 | % {$_.line -replace "($($_.pattern))", "e[7m$1e[0m" }`
isn't visibly slow.

[SeeminglyScience]

True ... or at least not obviously wrong. IIRC the formatter relies on a custom .Tostring() method which is why serialized/deserialized match info objects don't output.

It does, here's the method used for the emphasized string. That said, the serialization issue is probably because the Matches property gets serialized into string[] instead of Match[]. The serializer doesn't handle nested objects well (most likely by design).

That problem is more difficult to solve than it probably appears. If you generate the string at MatchInfo creation, then you're doing a whole lot of extra work for any that never make it to formatting.

If you wait until formatting to generate the display string, then you're either stuck with it breaking after serialization, or ditching the Match class from the BCL. Match won't serialize well because it has other nested objects that it depends on, and doesn't have a public constructor so you can't really rehydrate it.

Anyway that's just a guess, I haven't looked into it.

It's not very efficient as-is.

Eh, it's not as efficient as it could be but we're mostly talking about micro benchmarks I think. It's mostly fine. Regex on the other hand is significantly more likely to be expensive.

This
select-string -Pattern conditional -Path *.ps1 | % {$_.line -replace "($($_.pattern))", "e[7m$1e[0m" }`
isn't visibly slow.

That's a pretty simple pattern though. Add some backtracking and run it on a computer with less resources, you'll definitely see it.

[copdips]

I have one suggestion, but it would be a completely new powershell feature. That's to let Powershell to accepts some global settings at startup (profile.ps1). These settings can guide Powershell to change the default behaviors of some cmdlets.

Which means, for exmaple, a setting:

``powershell Set-CmdletDefaultBehavior Select-String Select-String -AllMatch ```` is telling Powershell to useSelect-Stringwith-AllMatch` by default, as this is an user defined setting in profile.ps1, it won't break any compatibility.

With this feature, we can solve many other similar problems. (Out-String -Stream for me:) ).
100 users have 100 preferences, just let them choose their favorite one.

BTW, I've seen a blog about the proxy functions, very interesting, but it's not easy to use.

[jhoneill]

That problem is more difficult to solve than it probably appears.

I had a look because there are bunch of things I don't like about the way formatting works with select string ... Roundabout V2. I wrote a wrapper to make select-string recursive and stuck it my profile and I type whathas xxx as many times as I use cd ... highlighting adds some problems - basically if you redirect the output you get escape codes , and the escape codes are function of how select-string was run, not what you do at output time. And .... in the end it came down "for pity's sake why don't they do this at output time" (Answer: if you want context before and after the formatting XML becomes the stuff of nightmares). So yes I get this is something superficially trivial with a more complexity beneath - a lot more.

Match won't serialize well because it has other nested objects that it depends on, and doesn't have a public constructor so you can't really rehydrate it.

... yes , and that was why I didn't dig any deeper. You need to change this, which means you need to change that, which in turn ... and a tiny change to do highlighting better means you're re-writing heaven only knows what.

It's not very efficient as-is.

Eh, it's not _as_ efficient as it could be but we're mostly talking about micro benchmarks I think. It's mostly fine. Regex on the other hand is significantly more likely to be expensive.

"Mostly fine" Overtones of Douglas Adams' and "mostly harmless".
I looked at it and thought how could something based on regex could do a job like that so much string slicing ... but that specific way of formatting is baked in to the object in way it really shouldn't be.

That's a pretty simple pattern though. Add some backtracking and run it on a computer with less resources, you'll definitely see it.

Yes. It was a simple example. I don't have an sensible feel for how complex the regex would have to be slow things down below unacceptably; and how often that might happen.

[SeeminglyScience]

I have one suggestion, but it would be a completely new powershell feature. That's to let Powershell to accepts some global settings at startup (profile.ps1). These settings can guide Powershell to change the default behaviors of some cmdlets.

I've got some good news for you :)

$PSDefaultParameterValues['Select-String:AllMatches'] = $true

[copdips]

I have one suggestion, but it would be a completely new powershell feature. That's to let Powershell to accepts some global settings at startup (profile.ps1). These settings can guide Powershell to change the default behaviors of some cmdlets.

I've got some good news for you :)

$PSDefaultParameterValues['Select-String:AllMatches'] = $true

thx a lot !

[iSazonov]

I think we have a conclusion for the discussion that users can use $PSDefaultParameterValues['Select-String:AllMatches'] = $true as needed without a breaking changing PowerShell.
Also users can customize output formatting because formatting system is extensible.

[vexx32]

@copdips that's already a feature, actually! 🙂

$PSDefaultParameterValues['Select-String:AllMatches'] = $true

Add that line to your profile.ps1 script, and it will always be applied as long as the profile is in effect. You can still disable it when calling the cmdlet by specifying -AllMatches:$false

[copdips]

I think we have a conclusion for the discussion that users can use $PSDefaultParameterValues['Select-String:AllMatches'] = $true as needed without a breaking changing PowerShell.
Also users can customize output formatting because formatting system is extensible.

Well, more or less, knowing this feature is very useful for me, thanks for sharing the tips. But I consider it as a workarond, not a solution.

For people just start to use powershell, they will feel frustrated when they see the default highlighting is only for the first match of each line, and this might lead some serious mistakes as they will think what they're seeing are all what are matched by Select-String.

[SeeminglyScience]

For people just start to use powershell, they will feel frustrated when they see the default highlighting is only for the first match of each line, and this might lead some serious mistakes as they will think what they're seeing are all what are matched by Select-String.

For certain use cases I can definitely see how that would be the case. Changing the default now, while I can't think of a way that it would technically be a breaking change, would still be potentially dangerous. Any script currently using Select-String without the AllMatches switch would be hit with an exponentially longer execution time (in most scenarios) and resource consumption. A change like that could be more dangerous than one that breaks a script out right.

Edit: I'm talking about making AllMatches the default specifically. My opinion is that formatting should reflect the object, so if AllMatches isn't specified then it shouldn't highlight everything. I wish AllMatches was the default from the start, but I don't think it's wise to change it now.

[jhoneill]

AllMatches switch [can result in] an exponentially longer execution time

My opinion is that formatting should reflect the object, so if AllMatches isn't specified then it shouldn't highlight everything.

I'll advance the contrary opinion...
Select-String has two distinct uses.

1. "Find me lines which match some _thing_ from some set of files and dump them to the screen, telling me where you found them."
2. "Give my code a set of matching things in this text"
   In the first Pre-7 all matches made no difference you got the line, whether it matched once or 10 times
   -Context (for example) is mostly for scenario 1. -AllMatches for scenario2

Because the formatting is done with a method which uses string slicing (see above) which depends on found matches it needs -AllMatches to not do "Highlight _Some_ of what I asked for". There really isn't a case I can think of where someone would want "Highlight some" and it finds 20 lines and leads the eye to 20 matches but the required one isn't in those 20, the highlight is distracting the user from what they are looking for. Although in most scenarios the highlight is helpful there are some cases where adding it a net negative.

[SeeminglyScience]

Select-String has two distinct uses.

It's use case is just about anything you'd use regex. One example off the top of my head is parsing a file. You may need to get the first occurrence of a pattern and use it's position as a basis for the next pattern.

There really isn't a case I can think of where someone would want "Highlight _some_" and it finds 20 lines and leads the eye to 20 matches but the required one isn't in those 20, the highlight is distracting the user from what they are looking for. Although in most scenarios the highlight is helpful there are some cases where adding it a net negative.

Here's two:

1. Testing regex. If I use Select-String to see the results of my pattern, and it returns more than I expect, then I have to figure out if formatting is lying or if my pattern is wrong.

2. Only the first match is relevant. There's been plenty of cases for me where I'm specifically looking for the first occurrence. Everything after that is what I want to read, but all other matches to the pattern are just noise.

I'm not disagreeing that AllMatches should have been the default (with a First parameter instead), but there are plenty of usages of the behavior out there.

Also I realized making AllMatches default would be a breaking change, since sls pattern | % { $_.Matches.Index } for example would now return an array (Matches is already an array, it's likely that folks expecting a single object would use member enumeration).

[jhoneill]

Select-String has two distinct uses.

It's use case is just about anything you'd use regex. One example off the top of my head is parsing a file. You may need to get the first occurrence of a pattern and use it's position as a basis for the next pattern.

That is one of the two. ("Get stuff and show it to me" or "Get stuff and use it as the basis of something else in following code) - in this case the second. I think the "find & show" tends toward the quick/simple (work first time) expressions where the "use it further down" tends towards complex (two or 3 goes to get right) expressions. That's why I think the overhead of using a regex during pretty-printing is probably OK.

There really isn't a case I can think of where someone would want "Highlight _some_" and it finds 20 lines and leads the eye to 20 matches but the required one isn't in those 20, the highlight is distracting the user from what they are looking for. Although in most scenarios the highlight is helpful there are some cases where adding it a net negative.

Here's two:

1. Testing regex. If I use Select-String to see the results of my pattern, and it returns more than I expect, then I have to figure out if formatting is lying or if my pattern is wrong.

Well not really. You had that in the existing version, and you're looking at lines returned. Did "1 line or 10 screens full". How much is highlighted does matter as much...

1. Only the first match is relevant. There's been plenty of cases for me where I'm specifically looking for the first occurrence. Everything _after_ that is what I want to read, but all other matches to the pattern are just noise.

I think the OP is probably the majority case. There it _is_ harmful to miss items and the desired one might be the 1st or the nth, but I'm not sure the first is needed massively more than the last. (where the 2nd or 3rd of 4 would be _odd)_ or how bad the distraction is when you're not interested in the 2nd, 3rd etc... but they're highlighted anyway.

The formatting behavior needs so many things fixed.
If I do select-string -path ..\foo -pattern it displays the full path which takes up half the line
but If I do $s = select-string -path ..\foo -pattern it displays the full path which takes up half the line and the do cd .. and $s (not necessarily one after the other).
And a select-string files fond from a (slow) crawl can't be run as a job because when receive job gets the item back it lacks the to tostring() needed to make it work etc.

IMHO , it needs someone to book an issue in for 7.1 to go through the whole thing properly. It's unlikely that one of these discussions will find the perfect answer and even if it did, the window for 7.0 is closing.

[SeeminglyScience]

That's why I think the overhead of using a regex during pretty-printing is probably OK.

For most patterns on most machines, probably. Though if you are specifically writing a pattern for first match, it's not too difficult to run into infinite back tracking.

# Don't run this in a session you want to keep
$string = '1z2345678901234567890123456789123456789'
$pattern = '(\d+)*z'

# Works
$string | Select-String -Pattern $pattern

# Does not finish
$string | Select-String -Pattern $pattern -AllMatches

Also my main objection is that the formatting wouldn't actually reflect the object it's representing.

Well not really. You had that in the existing version, and you're looking at lines returned. Did "1 line or 10 screens full". How much is highlighted does matter as much...

The old formatting did the best it could without a good way to color text, but yeah it was often mostly useless. Old things being bad doesn't mean we should make new things worse.

I think the OP is probably the majority case. There it _is_ harmful to miss items and the desired one might be the 1st or the nth, but I'm not sure the first is needed massively more than the last. (where the 2nd or 3rd of 4 would be _odd)_ or how bad the distraction is when you're not interested in the 2nd, 3rd etc... but they're highlighted anyway.

I'm not 100% which thing you're specifically advocating for so I'm just going to sum up my thoughts:

• If you're asking for Select-String to have AllMatches specified by default

  • That would change the behavior of existing code in ways that may out right fail or worse silently behave deferential. Plus potentially significant performance implications.

• If you're asking for the default behavior not to be change, but instead have the formatting system re-run the regex with -AllMatches:

  • The formatting would not reflect the object that it is formatting. This would be misleading. Folks would see the formatting, go to use the Matches contained on the object, and only see the first match.

[jhoneill]

For most patterns on most machines, probably. Though if you are specifically writing a pattern for first match, it's not too difficult to run into infinite back tracking.

True.

Also my main objection is that the formatting wouldn't actually reflect the object it's representing.

Data doesn't have colour or highlight or format. Formatting a date in US format (especially at the start of the year) will be read wrongly every other country. Format in the sense of trying to make it better for humans to took at is arbitrary. If I search for "Counter" does showing me

$counter = 1
$counter= $top - $counter - $height
$nextLine = $Counter

Serve me better by not emphasising the third occurrence ? The formatter does me a dis-service because it makes me think it is highlighting what I searched for when it isn't
BTW with -simplematch you can't highlight all matches with-allmatches there's no way to see the third match above

If you're asking for the default behavior not to be change, but instead have the formatting system re-run the regex with -AllMatches:

• The formatting would not reflect the object that it is formatting. This would be misleading. Folks would see the formatting, go to use the Matches contained on the object, and only see the first match.

That is the case I'm thinking of. I think the current formatting is a mess and needs to be re-done .

• Not being able to show output when run as a job or in a remote session should have resulted in the whoever did the current design being asked to do it again, properly.
• The current file name formatting is a result of trying to be too clever. Output of a variable should not change if I change directory. When only checking one file or one directory displaying the full path is actively unhelpful.
• Writing ansi escape sequences in the output stream (as opposed to in a prompt, a or message going to conHost) is generally bad, and needs to be done with a lot more thought than seems to be the case here
• As General principle users should not need to know the inner workings of a function. A user without a knowledge of regex should be able use -simplematch [system.and see all matches for it, without having to (a) use a regex with "[system." and (b) know that only the left-most regex is highlighted.

Since it is too late for 7.0 to make wholesale changes, these things should be looked at, but in 7.1.
I think of all these things as "it would be better if", rather than "it absolutely must", and benefiting from more discussion. :-)

[vexx32]

Not following your $counter example at all there @jhoneill. Since they're all on separate lines, Select-String would show them all.

Did something get rearranged during Markdown parsing?

[copdips]

Maybe I should reformat my issue.

By introducing highlighting by default in Powershell 7 and meanwhile with only the first match in each line, we're introducing some kind of potential user expectation error.

As I mentioned previously, this behavior might provoke some serious production mistakes for people newly in powershell, or even for people who are familiar with Powershell, we cannot grantee that they can remember to add -AllMatches each time.

Suppose we're facing an issue with an app on a remote server, we need to check the app logs by using Select-String after PsRemoting. Maybe the valuable keyword to the root cause is not the first match of the line, and unfortunately we forget to add the -AllMatches switch, then ...

There was no such error in previsous versions of powershell because whether SingleMatch or AllMatches, both display the line without any difference, but with highlighting since v7, it's no more the same experience.

I love the highlighting by default, I'm aware that making -AllMatches by default is a big breaking change, but it indeed introduces some new issues, we need to find a solution.

[jhoneill]

Not following your $counter example at all there @jhoneill. Since they're all on separate lines, Select-String would show them all.

Did something get rearranged during Markdown parsing?

That was my point. The third occurrence is on line 2. The first, second and fourth are highlighted, and the eye goes past the 3rd one.

[SeeminglyScience]

For most patterns on most machines, probably. Though if you are specifically writing a pattern for first match, it's not too difficult to run into infinite back tracking.

True.

Also my main objection is that the formatting wouldn't actually reflect the object it's representing.

Data doesn't have colour or highlight or format. Formatting a date in US format (especially at the start of the year) will be read wrongly every other country. Format in the sense of trying to make it better for humans to took at is arbitrary.

The formatted date still accurately represents the object. A closer analogy would be Get-ChildItem rerunning itself with Recurse.

If I search for "Counter" does showing me

$counter = 1
$counter= $top - $counter - $height
$nextLine = $Counter

Serve me better by not emphasising the third occurrence ? The formatter does me a dis-service because it makes me think it is highlighting what I searched for when it isn't

Yeah, because it's showing you exactly what you searched for, the first occurrence. If it highlighted everything, you'd expect Select-String | % { $_.Matches } to contain all of the matches, but it won't. It'll only contain the first one. You're just moving the confusion to a place that is more difficult to notice if you ever intend to use the cmdlet in a script.

• Not being able to show output when run as a job or in a remote session should have resulted in the whoever did the current design being asked to do it again, properly.

Yeah certainly a bug that needs fixing. But also one that is very easy to miss, and difficult to fix as we've discussed.

• Writing ansi escape sequences in the output stream (as opposed to in a prompt, a or message going to conHost) is generally bad, and needs to be done with a lot more thought than seems to be the case here

Not sure what you mean here. Select-String does not return the highlighed string, it returns a MatchInfo object. Unless you're piping Select-String to Out-String, the formatted string won't be output, it'll go directly to the host.

I skipped the bits that weren't relevant to the discussion.

[SeeminglyScience]

Suppose we're facing an issue with an app on a remote server, we need to check the app logs by using Select-String after PsRemoting. Maybe the valuable keyword to the root cause is not the first match of the line, and unfortunately we forget to add the -AllMatches switch, then ...

What if you forget to add -Recurse to Get-ChildItem? Or -Force for hidden files? What if you only want files but forget to add -File?

But also keep in mind that Select-String not showing results in a remote session is a separate bug. Not sure if that's the problem you're referring to here, but if so, that's not by design.

[copdips]

Suppose we're facing an issue with an app on a remote server, we need to check the app logs by using Select-String after PsRemoting. Maybe the valuable keyword to the root cause is not the first match of the line, and unfortunately we forget to add the -AllMatches switch, then ...

What if you forget to add -Recurse to Get-ChildItem? Or -Force for hidden files? What if you only want files but forget to add -File?

Hmmm, I should say you're right :)

But also keep in mind that Select-String not showing results in a remote session is a separate bug. Not sure if that's the problem you're referring to here, but if so, that's not by design.

No, it's for saying that in a PsRemoting session, the workaround $PSDefaultParameterValues['Select-String:AllMatches'] = $true in profile.ps1 doesn't work, otherwise I should write a custom PsRemoting cmdlet to inject this command after PsRemoting.

[copdips]

Suppose we're facing an issue with an app on a remote server, we need to check the app logs by using Select-String after PsRemoting. Maybe the valuable keyword to the root cause is not the first match of the line, and unfortunately we forget to add the -AllMatches switch, then ...

What if you forget to add -Recurse to Get-ChildItem? Or -Force for hidden files? What if you only want files but forget to add -File?

Hmmm, I should say you're right :)

Well, I think there's still a little difference... It's all about the common habit. People are used to add -Recurse to Get-ChildItem, same for Unix guys with ls -R, they all know the recurse is not default. On the contrary for Select-String which highlights only the first match by default, IMHO this might not be a common user habit. Maybe we can keep the current behavior and let people to give the feedback later once the v7 is released and widely used.

[jhoneill]

Data doesn't have colour or highlight or format. Formatting a date in US format (especially at the start of the year) will be read wrongly every other country. Format in the sense of trying to make it better for humans to took at is arbitrary.

The formatted date still accurately represents the object

Depends what you call accurate. What does 02/01/2020 mean then ? Because unless you know I wrote it in Europe with least-significant first, or I know you wrote it in the US with least significant in the middle, neither of us knows if means January or February.

If I search for "Counter" does showing me
$counter = 1
$counter= $top - $counter - $height
$nextLine = $Counter
Serve me better by not emphasising the third occurrence ? The formatter does me a dis-service because it makes me think it is highlighting what I searched for when it isn't

Yeah, because it's showing you exactly what you searched for, the first occurrence. If it highlighted everything, you'd expect Select-String | % { $_.Matches } to contain all of the matches

I'd get that output with -simplematch (whether or not I specified -all matches) and then Select-String | % { $_.Matches } contains no matches. But also I'd expect select-string > file.txt to contain the line which matched. It doesn't: it contains the line with \e[7m and \e[0m inserted in it

• Writing ansi escape sequences in the output stream (as opposed to in a prompt, a or message going to conHost) is generally bad, and needs to be done with a lot more thought than seems to be the case here

Not sure what you mean here. Select-String does not return the highlighed string, it returns a [MatchInfo] object. Unless you're piping Select-String to Out-String, the formatted string won't be output, it'll go directly to the host.

do select-string "something" > file.txt and edit the file. So not only can't you get output from a remote session, if you send it to a file and copy that the results are corrupt. That's why ansi to std-out is _bad_ but to write-host ansi_stuff doesn't get redirected so that's OK.

[SeeminglyScience]

Depends what you call accurate. What does 02/01/2020 mean then ? Because unless you know I wrote it in Europe with least-significant first, or I know you wrote it in the US with least significant in the middle, neither of us knows if means January or February.

It's formatted based on the current culture.

I'd get that output with -simplematch (whether or not I specified -all matches) and then Select-String | % { $_.Matches } contains no matches. But also I'd expect select-string > file.txt to contain the line which matched. It doesn't: it contains the line with \e[7m and \e[0m inserted in it

...

Not sure what you mean here. Select-String does not return the highlighed string, it returns a [MatchInfo] object. Unless you're piping Select-String to Out-String, the formatted string won't be output, it'll go directly to the host.

do select-string "something" > file.txt and edit the file. So not only can't you get output from a remote session, if you send it to a file and copy that the results are corrupt. That's why ansi to std-out is _bad_ but to write-host ansi_stuff doesn't get redirected so that's OK.

The redirection operator is lumped in with Out-String/Out-File. You're essentially redirecting output meant for the host to something else. You don't really get anything usable when you do gci > file.txt either.

[iSazonov]

We have many (all?) cmdlets for which coloring would be useful. Discussing and fixing only one Select-String makes no sense.
We have an issue for tracking PSMore. With implementing this idea we will get very flexible modern formatting system. We can simultaneously make colored, user-friendly output to the console, log-friendly output to an file and service-friendly output to an Azure service.
Also we have an issue for tracking color schema support.

[jhoneill]

@iSazonov . Personally I think "prettying up" stuff by splashing colour around isn't great. "Lets put table headings in a nice tone of Green" "Directory headings should be blue". "Shade memory or CPU use in a process listing". No. Don't. It needs to be done extremely selectively, with very clear standards about what means what. To date we have a error/warning/verbose which have approximated Red/Amber/Green (or cyan isn't quite green). The new error formatting, which _does_ work, PSreadline doing syntax colouring and Select-String picking out [some] text it was asked to find. This one command has done a very good of highlighting some of the pitfalls which hopefully will a better overall product in the future. How to do highlighting _properly_ in Select-String is informative for a bigger discussion rather than a problem which must be solved this minute. @SeeminglyScience makes the point that when you redirect something which would have plopped out onto the console (to a file or piping to a non PS command) things go through out-string, so prettified stuff doesn't "play nice" with unix tools ... It's a much wider question than one cmdlet. So you're absolutely right to say _fixing only one makes no sense._

(and @SeeminglyScience the "formatted based on current culture" is fine IF you know what the current culture is, and whether what you are talking to heeds your culture settings or not. Some cloud services decided based on my preference, some look at where my IP address appears to come from, and some -Azure devops for example, always use least-significant in the middle. _There is a fair question to asks about how much knowledge of the configuration and inner workings of the system is it reasonable to expect the user to have._ I've seen problems occur in one place because servers all used the same time regardless of location, and in a different place I've seen equal and opposite problems because the time on the server was what a clock outside the server room would say. Never mind problems I get where excel uses "." as 'local decimal point character' and transforms it to "," in France. Anyone want to use the importExcel module in France has to write "0.00" to get a number to display as "3,14" .... I hate that they need to know details of how dates are stored inside and XLS or XLSX file in order to use the thing )

[copdips]

Hi @jhoneill @SeeminglyScience

As per your comments, I tested Select-String | Out-File $file, I would say that here there's a real breaking change with the current release.

Previously before the introduction of the default highlight, it just saves the matched lines to the $file, whereas in the current release, it saves also some kind of special color metadata (, ). We can confirm that by Compare-Object, Get-FileHash, git diff or just use vim $file to check the content.

I don't think this extra metadata is expected that's why I said there's a real breaking change. But maybe I should open a new issue to address it.

BTW, as a positive side effect, this is the first time I've seen that we can save color metadata in text file, and display it by Get-Content in console, even cat in linux works the same, it's nice to know that.

[SeeminglyScience]

I don't think this extra metadata is expected that's why I said there's a real breaking change. But maybe I should open a new issue to address it.

Formatting isn't part of the change contract, that's why it can do things like use the user's current culture. In other words, piping objects to Out-File should typically only be done with basic primitives like string/int or objects whose formatting you control. Instead, you should craft the string you want to send to a file. In this case, piping to ForEach-Object { $_.ToString() } | Out-File would have kept the original behavior as public methods are part of the change contract.