Beats: libbeat-logp: incorrect file naming using interval rotator

[michalpristas]

when interval rotator is used it decides on a name format in the following manner:

• for the interval in seconds: yyyy-dd-mm-hh-MM-ss
• for the interval in minutes: yyyy-dd-mm-hh-MM
• for the interval in hours: yyyy-dd-mm-hh

funny thing happens when we use N-{seconds/minutes/hours}
For example we want to use 2h rotation:

What we see:

filebeat-2019-03-19-02-00-00-1
filebeat-2019-03-19-04-00-00-1
filebeat-2019-03-19-06-00-00-1

What we should see:

filebeat-2019-03-19-02-1
filebeat-2019-03-19-04-1
filebeat-2019-03-19-06-1

[leopucci]

I'm analyzing this one

[leopucci]

Maybe related #4569

@michalpristas on which operating system are you testing this?
This will be OS related.
On CentOs 7:

1m:

filebeat-2019-03-22-14-03-1
filebeat-2019-03-22-14-03-2
filebeat-2019-03-22-14-04-1
filebeat-2019-03-22-14-05-1

1 hour

filebeat-2019-03-22-14-1
filebeat-2019-03-22-15-1
filebeat-2019-03-22-16-1

[michalpristas]

Hey @leopucci i'm using Ubuntu 18.04

[leopucci]

Tested on Ubuntu 16.04.6

1m

filebeat-2019-03-25-10-17-1
filebeat-2019-03-25-10-17-2

Can you share the output of the command timedatectl? Maybe only happens with timezone enabled..

[michalpristas]

Local time: Po 2019-03-25 14:27:49 CET
Universal time: Po 2019-03-25 13:27:49 UTC
RTC time: Po 2019-03-25 07:21:48
Time zone: Europe/Prague (CET, +0100)
System clock synchronized: no
systemd-timesyncd.service active: yes
RTC in local TZ: no

[michalpristas]

hey @leopucci it should be ok with 1m or 1h try with 2m

[leopucci]

timedatectl
Local time: Mon 2019-03-25 10:38:01 -03
Universal time: Mon 2019-03-25 13:38:01 UTC
RTC time: Mon 2019-03-25 13:37:56
Time zone: America/Sao_Paulo (-03, -0300)
Network time on: yes
NTP synchronized: yes
RTC in local TZ: no

[leopucci]

Caught something odd on centos... indeed there is a bug... related to the rotation i guess (https://github.com/elastic/beats/issues/11364)

Edit: Forget it... it used the file timestamp to rotate, maybe not wrong.

[leopucci]

Got it with 2m
filebeat-2019-03-25-10-54-00-1

2hours
filebeat-2019-03-25-09-00-00-1

[leopucci]

This issue happens because the interval log rotation was only available for fixed times 1h, 24h, 724h, 3024h, and 365*24h
Any value different from those would add some 00-00-00 to the filename and would not rotate correctly.
2h was going to rotate each hour.

The pull request changes this behavior

Point of attention for the seconds,
Log rotation is based on logging.metrics.period. So if you put a time of 10s but logging.metrics.period default is 30s, it will rotate logs each 30s.

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic[bot]]

This issue doesn't have a Team:<team> label.