Beats: ERR Failed to publish events caused by: write tcp 127.0.0.1:40390->127.0.0.1:5044: write: connection reset by peer

[szkrawcz]

Hello
Filebeat struggling to send csv file to logstash. It struggles in few first attempts . At the end of the day there is success but in log constantly I see errors and no confirmation that it succeed .

Configuration is straight forward . All on one host with using loop back IP 127.0.0.1

This is dump of filebeats in debug mode.
2017-01-25T11:22:24-08:00 DBG output worker: publish 20 events
2017-01-25T11:22:24-08:00 DBG Try to publish 20 events to logstash with window size 15
2017-01-25T11:22:24-08:00 DBG handle error: write tcp 127.0.0.1:40386->127.0.0.1:5044: write: connection reset by peer
2017-01-25T11:22:24-08:00 DBG 0 events out of 20 events sent to logstash. Continue sending
2017-01-25T11:22:24-08:00 DBG close connection
2017-01-25T11:22:24-08:00 DBG closing
2017-01-25T11:22:24-08:00 ERR Failed to publish events caused by: write tcp 127.0.0.1:40386->127.0.0.1:5044: write: connection reset by peer
2017-01-25T11:22:24-08:00 INFO Error publishing events (retrying): write tcp 127.0.0.1:40386->127.0.0.1:5044: write: connection reset by peer
2017-01-25T11:22:24-08:00 DBG close connection
2017-01-25T11:22:24-08:00 DBG send fail
2017-01-25T11:22:25-08:00 DBG connect
2017-01-25T11:22:25-08:00 DBG Try to publish 20 events to logstash with window size 7
2017-01-25T11:22:25-08:00 DBG update current window size: 7
2017-01-25T11:22:25-08:00 DBG set to max ok window size: 10
2017-01-25T11:22:25-08:00 DBG 7 events out of 20 events sent to logstash. Continue sending
2017-01-25T11:22:25-08:00 DBG Try to publish 13 events to logstash with window size 10
2017-01-25T11:22:25-08:00 DBG 10 events out of 13 events sent to logstash. Continue sending
2017-01-25T11:22:25-08:00 DBG Try to publish 3 events to logstash with window size 10
2017-01-25T11:22:25-08:00 DBG 3 events out of 3 events sent to logstash. Continue sending
2017-01-25T11:22:25-08:00 DBG send completed
2017-01-25T11:22:25-08:00 DBG Events sent: 22
2017-01-25T11:22:25-08:00 DBG Processing 22 events
2017-01-25T11:22:25-08:00 DBG Registrar states cleaned up. Before: 9, After: 9
2017-01-25T11:22:25-08:00 DBG Write registry file: /home/szymon/Downloads/filebeat-5.1.2-linux-x86_64/data/registry
2017-01-25T11:22:25-08:00 DBG Registry file updated. 9 states written.
2017-01-25T11:22:26-08:00 DBG End of file reached: /home/szymon/log/test8.log; Backoff now.

Filebeat config
filebeat.prospectors:

• input_type: log
  paths:
  
  
  
  • /home/szymon/log/*.log
  
  

output.logstash:

Logstash
hosts: ["localhost:5044"]

input {
beats {
port => 5044 }
}
output {
stdout { codec => rubydebug }
}

For confirmed bugs, please report:

• Version: Filebeat 5.1.2 Logstash 5.1.2 . Java openjdk version "1.8.0_111" ( also Oracle Java had same results)
• Operating System: Ubuntu 16.04
• Steps to Reproduce:

[ruflin]

We try to reserve Github for confirmed bugs, for questions please use discuss. The above is a common question and quite a few threads related to it can be found here: https://discuss.elastic.co/search?q=ERR%20Failed%20to%20publish%20events In case non of the threads solves your problem feel free to open a new topic on discuss.

[cgough]

I have searched all the threads listed above and there are no solutions.

[chintan1112]

Working on Logstash version : logstash-7.2.0
Try Adding > client_inactivity_timeout => 1200 parameter.

e.g

Logstash

input {
beats {
client_inactivity_timeout => 1200
port => 5044
}
}

output {
stdout { codec => rubydebug }
}

Reference : https://discuss.elastic.co/t/solved-tcp-rst-reset-causing-filebeat-logstash-connection-reset/82343/2

[aqelkp]

I have been also trying to find a solution for this issue for two days. None of the thread in discuss solves my issue