Powershell: Additional Parameters for Get-Content

[scotthardwick]

It would be useful for Get-Content to have
-Skip
-SkipLast
in the same fashion that Select-Object has.

This would allow you to skip over initial lines you did not want, rather than take the performance and memory hit of loading the entire file into an initial variable and then doing a select.
$Content = Get-content $myTextFile | select-object -skip 500

Currently both Get-Content and Select-Object have a -First and -Last. But only Select-Object has -Skip and -SkipLast.

[bpayette]

Hi @scotthardwick, by default, Get-Content reads a file line by line and streams each line into the pipeline. Since the lines are streamed, Get-Content | Select -first 10 will only read 10 lines from file so it more or less already does what you want.

[mklement0]

It's true that there's no _memory-usage_ concern with combining Get-Content with Select-Object, but there is a _performance_ concern, due to the processing overhead that the pipeline introduces.

I presume that performance concerns led to the implementation of the -Head aka -First and -Tail aka -Last parameters _directly on Get-Content_ to begin with - even though they duplicate Select-Object functionality.

It sounds like you, @scotthardwick, are looking to make this for-performance duplication of functionality _complete_, by also implementing -Skip and -SkipLast directly on Get-Content, correct?

Here's an example that shows that the performance difference can be dramatic:

try { 
  # Create a large (temporary) file.
  $tempFile = [IO.Path]::GetTempFileName()
  1..1e5 > $tempFile

  # Warm up the cache.
  Get-Content $tempFile | Out-Null

  (Measure-Command { $lines = Get-Content -Last 5 $tempFile }).TotalSeconds
  (Measure-Command { $lines = Get-Content $tempFile | Select-Object -Last 5 }).TotalSeconds

} finally {
  Remove-Item $tempFile
}

Sample results from my machine (seconds):

0.046115   # Get-Content -Last
2.2028236  # Get-Content | Select-Object -Last

That is, with a text file of 100,000 (very short) lines, Get-Content -Last outperformed Get-Content | Select -Last by a factor of about 47(!).

(As @vexx32 points out below, the discrepancy isn't just explained by pipeline overhead; -Last as part of Get-Content is presumably efficiently implemented by reading from the _end_ of the file.)

[scotthardwick]

@mklement0

It sounds like you, @scotthardwick, are looking to make this for-performance duplication of functionality complete, by also implementing -Skip and -SkipLast directly on Get-Content, correct?

Correct!

[vexx32]

Doesn't Get-Content -Tail also read the file differently? I think it's more than just the pipeline overhead there. But yeah if we can add parameters to Get-Content in order to reduce what we actually need to read from disk, it will indeed be much faster to read.

[mklement0]

Another Select-Object parameter worth porting: -Index (line selection by index / indices).