Powershell: Why do headers only appear once?

[Liturgist]

Why does the header line ("Name" and "Value") only appear the first time?

PS C:\src\powershell> Get-Content .\headers.ps1
$h = @{1 = 'a'; 2 = 'b'}
$h
Write-Host '======='
$h
Write-Host '======='
$h
PS C:\src\powershell> .\headers.ps1

Name                           Value
----                           -----
2                              b
1                              a
=======
2                              b
1                              a
=======
2                              b
1                              a

[ealexjordan]

Hopefully I can get some clarification on what the issue is here. The 'name' and 'value' header are printed only once because it is printing a table of your hashtable.
How many times are you expecting to see them printed?

[Liturgist]

The hashtable if printed three (3) times. Only the first has headers.

[thezim]

I see this behavior in Powershell 5.1 (Windows 10) as well.

[vors]

Short answer: that's just how PowerShell formatting system works.
Long answer will take too much time, so here is _medium-size compromise_ version

Let's take a look at this code

$h = @{1 = 'a'; 2 = 'b'}
$h
Write-Host '======='
$h
Write-Host '======='
$h

There are two separate routes in this code that both are ending up on the screen:

1. Objects output (three $h expression invocations)
2. Write-Host invocations.

Note that these two methods are deeply different in Powershell.
For example, let's tweak the code this way

PS C:\src\powershell> $a = ./headers.ps1                                                                                                              
=======
=======
PS C:\src\powershell> $a

Name                           Value                                                                                                                                  
----                           -----                                                                                                                                  
2                              b                                                                                                                                      
1                              a                                                                                                                                      
2                              b                                                                                                                                      
1                              a                                                                                                                                      
2                              b                                                                                                                                      
1                              a                                                                                                                                      

As you can see, you can manipulate $h objects, but everything from Write-Host goes straight to the screen. As I side-note, here is a great post from @jpsnover about Write-Host usage.

So we can take Write-Host out from the equation.
We are ending up with this

$h = @{1 = 'a'; 2 = 'b'}
$h
$h
$h

Now, we need to talk about PowerShell formatting system.

Every object type has formatting rules. They are applied in two cases:

• When objects are written to the console (and user see the output).
• When they are formatted with Out-String cmdlet.

There are two most used formatting groups:

• Tables
• Lists

It's important to point out, that although every object has default formatting, it's not locked just to this formatting.
Default formatting goal is to provide a good interactive experience to all users across the board.
Here you can find documentation about changing object representation.
You can also use Update-TypeData command to globally change default formatting for types, if you don't like them. All these changes will only result in the view change, objects would still be the same.

We are one step away from finishing the explanation. The last step is "why there is only one header for table formatting"?
I found another great post (again from @jpsnover) that explains it

Now, if there is not a registered view for a datatype, then Out-Default looks at the FIRST OBJECT IN THE STREAM to determine how many properties the object has 5 or more properties, it send the ENTIRE STREAM to Format-List, otherwise it sends the ENTIRE STREAM to Format-Table. When it sends the stream to Format-Table, that command needs to generate columns. It does this by looking at the properties of the FIRST OBJECT – those become the columns. If the first Object has 2 properties, you’ll get a 2 column table even if all the other objects in the stream have 10 properties.

@Liturgist does it answer your question?

[Liturgist]

@vors - It sounds like if the next object to be formatted is the same type as the previous object, then the formatter does not emit any headers. Is that right? If so, then the formatters do not actually go into what many people think of as the pipeline; stdout.

There are many Write-* cmdlets. Would the following items seem to be similar for someone coming from a traditional C/*NIX environment? Are there other streams you would add here? How many default streams does PowerShell have?

Write-Error - stderr
Write-Host - stdout
Write-Output - stdobj

If I were to want to have headers written each time the formatter is invoked, what kind of things might I do? I tried changing Write-Host to Write-Output, but have the same result. Having a [string] in the pipeline did not reset the formatter to produce headers. Does Write-Output go through the formatter?

PS C:\src\powershell> Get-Content .\headers.ps1
$h = @{1 = 'a'; 2 = 'b'}
$h
Write-Output '======='
$h
Write-Output '======='
$h

PS C:\src\powershell> $r = .\headers.ps1
PS C:\src\powershell> $r

Name                           Value
----                           -----
2                              b
1                              a
=======
2                              b
1                              a
=======
2                              b
1                              a

PS C:\src\powershell> $r.GetType()
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

PS C:\src\powershell> $r | % {$_.GetType()}
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    Hashtable                                System.Object
True     False    String                                   System.Object
True     False    Hashtable                                System.Object
True     False    String                                   System.Object
True     False    Hashtable                                System.Object

Just found what appears to be a great article about PowerShell and streams. https://blogs.technet.microsoft.com/heyscriptingguy/2014/03/30/understanding-streams-redirection-and-write-host-in-powershell/

[vors]

I cannot come up with a good use-case, but if you really want to have headers, you can do ForEach-Object {$_ | Format-Table}, like

> @{'foo' = '1'}, @{'2' = '2'} | ForEach-Object {$_ | Format-Table}


Name                           Value                                                                                                                                  
----                           -----                                                                                                                                  
foo                            1                                                                                                                                      



Name                           Value                                                                                                                                  
----                           -----                                                                                                                                  
2                              2  

Formatter emits only one header for table, regardless of the types of objects.
For example

PS > New-Object -TypeName psobject -Property @{'foo' = 'foo'; 'bar' = 'bar'};  New-Object -TypeName psobject -Property @{'foo' = 'foo'; 'baz' = 'baz'}                                                                                                                                                      

bar foo
--- ---
bar foo
    foo

It may seems that information is lost in this example (the baz column), but it's not. It's just that formatting uses only the first objects for columns. That's exactly what @jpsnover covered in great details in https://blogs.msdn.microsoft.com/powershell/2006/04/29/how-powershell-formatting-and-outputting-really-works/

There are 6 streams, i.e.

PS > 'foo' 6> 1 
foo


PS > 'foo' 7> 1                                                                                                                      
At line:1 char:7
+ 'foo' 7> 1
+       ~~
Unexpected token '7>' in expression or statement.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : UnexpectedToken

1 - stdout
2 - stderr
3 - verbose
4 - information
5 - debug
6 - ??

I could mess up 3, 4, 5 maybe they have a different order. :)

The one that you named stdobj is the pipeline one. I would not call it a separate stream. When PowerShell and OS interacts (i.e. to file redirection) it actually acts like stdout.

[Liturgist]

From https://connect.microsoft.com/PowerShell/feedback/details/297055/capture-warning-verbose-debug-and-host-output-via-alternate-streams Is this the way it is, or the way Keith Hill is suggesting it should be?

0 - stdin
1 - stdout (write-output, normal cmdlet output, non captured expression output)
2 - stderr (write-error, throw, cmdlet non-terminating errors)
3 - warnings (write-warning, cmdlet warning)
4 - verbose (write-verbose and cmdlet -verbose output)
5 - debug (write-debug and cmdlet debug output)
6 - host (write-host, read-host, explicit out-host use)
9 - combined (all output combined into a single - easy to redirect stream)

[kilasuit]

The streams are as follows

*   All output
1   Success output
2   Errors
3   Warning messages
4   Verbose output
5   Debug messages
6   Informational message

Which I took from the about_redirection help file - Get-Help about_redirection or also online at
https://technet.microsoft.com/en-us/library/hh847746.aspx

The Connect Item was from 2007 which would be in the v1 - v2 time line and v2 had a number of functional updates over v1

[thezim]

Is this behavior documented in help system?

[vors]

@thezim yes https://technet.microsoft.com/en-us/library/hh847746.aspx

@Liturgist it almost works as described in the Connect

PS /Users/vors> Write-Warning 'foo' 3>1 ; cat 1                                                                                                                        
foo                                                                                                                                                                    
PS /Users/vors> Write-Verbose -Verbose 'foo' 4>1 ; cat 1                                                                                                               
foo                                                                                                                                                                    
PS /Users/vors> Write-Debug 'foo' 5>1; cat 1                                                                                                                           
PS /Users/vors> Write-Information 'foo' 6>1 ; cat 1                                                                                                                    
foo                                                                                                                                                                    
PS /Users/vors>

Write-Debug is the only one that doesn't seems to follow this, but it's likely because it requires some additional massaging (like -Verbose in the Write-Verbose case).

In general, streams concepts are not that central in PowerShell compare to the object pipeline. Particularly, all streams higher than 2 are one or another form of information logging. The stream number really represents severity (less is more important).
I wrote thousands of PowerShell lines without using Write-Debug, Write-Information or any stream higher than 2 directly.

I'd like to quote Jeffrey one more time

Using Write-Host is almost always wrong.

[thezim]

@vors I was referring to the formatter behavior. Sorry for not being clear.

[lzybkr]

You can use Out-Host if you want to make sure the headers are written, compare:

#83 PS> ps -id $pid; ps -id $pid

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   1018      61    97220      87128      14.72  10800   1 powershell
   1020      61    97248      87668      14.72  10800   1 powershell


#84 PS> ps -id $pid | out-host ; ps -id $pid

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   1053      61    98336      88740      14.75  10800   1 powershell



Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   1020      61    98400      88856      14.77  10800   1 powershell

[Liturgist]

@lzybkr - I would not really want to have a header and header separator for each and every line.

[lzybkr]

You wouldn't get headers for every line - you'll just get headers as though no more objects are being written to the pipe. This way, the next object won't try and apply the same formatting, the formatter will choose the default formatting for this new object.

In the following example, in 113, notice how the csrss lines look like they belong to the table but have no headers. In 114 (with Out-Host on the first command, the service objects are formatted as a table instead of like a list in 113, and in 115, the csrss lines have the header because both previous commands use Out-Host.

#113 PS> ps -name pow*; gsv ws*; ps -name csr*

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    575      39   141720      55164       3.53   7196   1 powershell
    647      36    90336     108912      12.39  13984   1 powershell

Status      : Running
Name        : wscsvc
DisplayName : Security Center


Status      : Running
Name        : WSearch
DisplayName : Windows Search

    642      19     1776       2828               592   0 csrss
    700      29     2240       2824               688   1 csrss


#114 PS> ps -name pow* | Out-Host; gsv ws*; ps -name csr*

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    575      39   141720      55164       3.53   7196   1 powershell
    627      36    90336     108916      12.45  13984   1 powershell



Status   Name               DisplayName
------   ----               -----------
Running  wscsvc             Security Center
Running  WSearch            Windows Search

Id      : 592
Handles : 645
CPU     :
SI      : 0
Name    : csrss


Id      : 688
Handles : 700
CPU     :
SI      : 1
Name    : csrss



#115 PS> ps -name pow* | Out-Host; gsv ws* | Out-Host; ps -name csr*

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    575      39   141720      55164       3.53   7196   1 powershell
    660      36    90336     108924      12.53  13984   1 powershell



Status   Name               DisplayName
------   ----               -----------
Running  wscsvc             Security Center
Running  WSearch            Windows Search



Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    645      19     1776       2832               592   0 csrss
    700      29     2240       2824               688   1 csrss

[andschwa]

The original question appears now well-answered, and I do not see anyone waiting for a follow-up, so I'm closing the issue. Thanks for participating @Liturgist!

[felixfbecker]

This tripped me up when I wanted to do polling:

while ($true) {
  Clear-Host
  &$Command
  Start-Sleep 1
}

The first time the headers are rendered, the second time and following they disappear. This works around it:

while ($true) {
  Clear-Host
  &$Command | Format-Table
  Start-Sleep 1
}

I believe Clear-Host should reset the state of whether headers were shown

[SteveL-MSFT]

@felixfbecker please open that as a separate issue