Logstash: How to parse .gz (compressed file) using logstash

[jotsnadasari]

Hi All,

Does latest logstash support parsing .gz files using logstash?

We have a requirement to get .gz files dynamically every hour. We do not know the file names and content. All we know is the file is compressed in .gz format.

We are using logstash version 5.6.0 and using gzip_lines codec plugin but logstash is not able to parse .gz files. Also using grok filter for parsing/extracting fields.

Sample code

input {
file {
type => "test-xyz"
path => "/tmp/*.gz"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => "gzip_lines"
}
}

I have seen some old threads saying logstash does not support .gz files.
Wanted to check whether there is any work around for dynamic .gz files we receive.

Please provide your inputs on this.

Thanks and Regards,
Jotsna

[guyboertje]

@jotsnadasari
This is known not to work. The file input reads 16K blocks from the gz file and the gzip_lines code is expecting a complete compressed string.