Fluentd: out_forward: failed forward v0.14 to v0.12 after upgrade to v0.14.9

Created on 16 Nov 2016  路  6Comments  路  Source: fluent/fluentd

I'm testing a system that forward logs from fluentd v0.14 to v0.12.
Successed until 0.14.8, but upgraded to 0.14.9 did not work.

Is this a change in specifications?

Environment

  • CentOS 7.2.1511
  • ruby 2.3.1p112
fluentd v0.14      fluentd v0.12
 out_forward   -->  in_forward

Log

[SUCCESS] v0.14.8 to v0.12.29

v0.14.8 out_forward log:

2016-11-16 16:45:54 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:45:54 +0900 [info]: starting fluentd-0.14.8
2016-11-16 16:45:54 +0900 [info]: spawn command to main: /home/t-osakada/.rbenv/versions/2.3.1/bin/ruby -Eascii-8bit:ascii-8bit  -rbundler/setup /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/bin/fluen
td -c fluent.conf --log ./log --under-supervisor
2016-11-16 16:45:55 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:45:55 +0900 [info]: starting fluentd-0.14.8 without supervision
2016-11-16 16:45:55 +0900 [info]: gem 'fluentd' version '0.14.8'
2016-11-16 16:45:55 +0900 [info]: adding match in @in_dummy pattern="log.**" type="forward"
2016-11-16 16:45:55 +0900 [info]: [Fluent::ForwardOutput] adding forwarding server '127.0.0.1:24224' host="127.0.0.1" port=24224 weight=60 plugin_id="object:3f89fdae6d54"
2016-11-16 16:45:55 +0900 [info]: adding source type="dummy"
2016-11-16 16:45:55 +0900 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2016-11-16 16:45:55 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type dummy
    @label @in_dummy
    dummy {"message":"hello world"}
    tag "log"
  </source>
  <label @in_dummy>
    <match log.**>
      @type forward
      @log_level "trace"
      heartbeat_type udp
      require_ack_response
      buffer_type "memory"
      flush_interval 1s
      <server>
        host "127.0.0.1"
        port 24224
      </server>
      <buffer tag>
        flush_mode interval
        retry_type exponential_backoff
        @type memory
        flush_interval 1s
      </buffer>
    </match>
  </label>
</ROOT>
2016-11-16 16:45:55 +0900 [debug]: [Fluent::ForwardOutput] buffer started instance=69861904399820 stage_size=0 queue_size=0
2016-11-16 16:45:55 +0900 [debug]: [Fluent::ForwardOutput] rebuilding weight array lost_weight=0
2016-11-16 16:45:55 +0900 [debug]: [Fluent::ForwardOutput] enqueue_thread actually running
2016-11-16 16:45:55 +0900 [debug]: [Fluent::ForwardOutput] flush_thread actually running
2016-11-16 16:45:55 +0900 [debug]: [Fluent::ForwardOutput] enqueueing all chunks in buffer instance=69861904399820
2016-11-16 16:45:56 +0900 [trace]: [Fluent::ForwardOutput] adding metadata instance=69861904399820 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:56 +0900 [trace]: [Fluent::ForwardOutput] writing events into buffer instance=69861904399820 metadata_size=1
2016-11-16 16:45:56 +0900 [trace]: [Fluent::ForwardOutput] sending heartbeat host="127.0.0.1" port=24224 heartbeat_type=:udp
2016-11-16 16:45:56 +0900 [debug]: [Fluent::ForwardOutput] enqueueing all chunks in buffer instance=69861904399820
2016-11-16 16:45:57 +0900 [trace]: [Fluent::ForwardOutput] adding metadata instance=69861904399820 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:57 +0900 [trace]: [Fluent::ForwardOutput] writing events into buffer instance=69861904399820 metadata_size=1
2016-11-16 16:45:57 +0900 [trace]: [Fluent::ForwardOutput] sending heartbeat host="127.0.0.1" port=24224 heartbeat_type=:udp
2016-11-16 16:45:57 +0900 [debug]: [Fluent::ForwardOutput] enqueueing all chunks in buffer instance=69861904399820
2016-11-16 16:45:57 +0900 [debug]: [Fluent::ForwardOutput] enqueueing chunk instance=69861904399820 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] adding metadata instance=69861904399820 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] writing events into buffer instance=69861904399820 metadata_size=1
2016-11-16 16:45:58 +0900 [debug]: [Fluent::ForwardOutput] dequeueing a chunk instance=69861904399820
2016-11-16 16:45:58 +0900 [debug]: [Fluent::ForwardOutput] chunk dequeued instance=69861904399820 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:58 +0900 [debug]: [Fluent::ForwardOutput] trying flush for a chunk chunk="54166435aeb5e33bb56300d23a6df622"
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] adding write count instance=69861899136340
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] executing sync write chunk="54166435aeb5e33bb56300d23a6df622"
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] write operation done, committing chunk="54166435aeb5e33bb56300d23a6df622"
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] committing write operation to a chunk chunk="54166435aeb5e33bb56300d23a6df622" delayed=false
2016-11-16 16:45:58 +0900 [debug]: [Fluent::ForwardOutput] purging a chunk instance=69861904399820 chunk_id="54166435aeb5e33bb56300d23a6df622" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:58 +0900 [debug]: [Fluent::ForwardOutput] chunk purged instance=69861904399820 chunk_id="54166435aeb5e33bb56300d23a6df622" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:45:58 +0900 [trace]: [Fluent::ForwardOutput] done to commit a chunk chunk="54166435aeb5e33bb56300d23a6df622"

v0.12.29 in_forward log:

2016-11-16 16:44:41 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:44:41 +0900 [info]: starting fluentd-0.12.29
2016-11-16 16:44:41 +0900 [info]: gem 'fluentd' version '0.12.29'
2016-11-16 16:44:41 +0900 [info]: adding match pattern="log.**" type="stdout"
2016-11-16 16:44:41 +0900 [info]: adding source type="forward"
2016-11-16 16:44:41 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    port 24224
    @log_level trace
  </source>
  <match log.**>
    @type stdout
  </match>
</ROOT>
2016-11-16 16:44:41 +0900 [info]: listening fluent socket on 0.0.0.0:24224
2016-11-16 16:45:58 +0900 [trace]: accepted fluent socket from '127.0.0.1:59260': object_id=70348667967600
2016-11-16 16:45:56 +0900 log: {"message":"hello world"}
2016-11-16 16:45:57 +0900 log: {"message":"hello world"}
2016-11-16 16:45:58 +0900 [trace]: sent response to fluent socket
2016-11-16 16:45:58 +0900 [trace]: closed socket
2016-11-16 16:46:00 +0900 [trace]: accepted fluent socket from '127.0.0.1:59262': object_id=70348667960300
2016-11-16 16:45:58 +0900 log: {"message":"hello world"}
2016-11-16 16:45:59 +0900 log: {"message":"hello world"}
2016-11-16 16:46:00 +0900 [trace]: sent response to fluent socket
2016-11-16 16:46:00 +0900 [trace]: closed socket
2016-11-16 16:46:02 +0900 [trace]: accepted fluent socket from '127.0.0.1:59264': object_id=70348667934960
2016-11-16 16:46:00 +0900 log: {"message":"hello world"}
2016-11-16 16:46:01 +0900 log: {"message":"hello world"}

[FAIL] v0.14.9 to v0.12.29

v0.14.9 out_forward log:

2016-11-16 16:35:33 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:35:33 +0900 [info]: starting fluentd-0.14.9
2016-11-16 16:35:33 +0900 [info]: spawn command to main: /home/t-osakada/.rbenv/versions/2.3.1/bin/ruby -Eascii-8bit:ascii-8bit  -rbundler/setup /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/bin/fluen
td -c fluent.conf --log ./log --under-supervisor
2016-11-16 16:35:34 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:35:34 +0900 [info]: starting fluentd-0.14.9 without supervision
2016-11-16 16:35:34 +0900 [info]: gem 'fluentd' version '0.14.9'
2016-11-16 16:35:34 +0900 [info]: adding match in @in_dummy pattern="log.**" type="forward"
2016-11-16 16:35:34 +0900 [info]: [Fluent::Plugin::ForwardOutput] adding forwarding server '127.0.0.1:24224' host="127.0.0.1" port=24224 weight=60 plugin_id="object:3fc76180213c"
2016-11-16 16:35:34 +0900 [info]: adding source type="dummy"
2016-11-16 16:35:34 +0900 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2016-11-16 16:35:34 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type dummy
    @label @in_dummy
    dummy {"message":"hello world"}
    tag "log"
  </source>
  <label @in_dummy>
    <match log.**>
      @type forward
      @log_level "trace"
      heartbeat_type udp
      require_ack_response
      <server>
        host "127.0.0.1"
        port 24224
      </server>
      <buffer tag>
        @type "memory"
        flush_interval 1s
      </buffer>
    </match>
  </label>
</ROOT>
2016-11-16 16:35:34 +0900 [debug]: [Fluent::Plugin::ForwardOutput] buffer started instance=70125566117840 stage_size=0 queue_size=0
2016-11-16 16:35:34 +0900 [debug]: [Fluent::Plugin::ForwardOutput] rebuilding weight array lost_weight=0
2016-11-16 16:35:34 +0900 [debug]: [Fluent::Plugin::ForwardOutput] enqueue_thread actually running
2016-11-16 16:35:34 +0900 [debug]: [Fluent::Plugin::ForwardOutput] flush_thread actually running
2016-11-16 16:35:34 +0900 [debug]: [Fluent::Plugin::ForwardOutput] enqueueing all chunks in buffer instance=70125566117840
2016-11-16 16:35:35 +0900 [trace]: [Fluent::Plugin::ForwardOutput] adding metadata instance=70125566117840 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:35:35 +0900 [trace]: [Fluent::Plugin::ForwardOutput] writing events into buffer instance=70125566117840 metadata_size=1
2016-11-16 16:35:35 +0900 [trace]: [Fluent::Plugin::ForwardOutput] sending heartbeat host="127.0.0.1" port=24224 heartbeat_type=:udp
2016-11-16 16:35:35 +0900 [debug]: [Fluent::Plugin::ForwardOutput] enqueueing all chunks in buffer instance=70125566117840
2016-11-16 16:35:36 +0900 [trace]: [Fluent::Plugin::ForwardOutput] adding metadata instance=70125566117840 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:35:36 +0900 [trace]: [Fluent::Plugin::ForwardOutput] writing events into buffer instance=70125566117840 metadata_size=1
2016-11-16 16:35:36 +0900 [trace]: [Fluent::Plugin::ForwardOutput] sending heartbeat host="127.0.0.1" port=24224 heartbeat_type=:udp
2016-11-16 16:35:36 +0900 [debug]: [Fluent::Plugin::ForwardOutput] enqueueing all chunks in buffer instance=70125566117840
2016-11-16 16:35:36 +0900 [debug]: [Fluent::Plugin::ForwardOutput] enqueueing chunk instance=70125566117840 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:35:37 +0900 [trace]: [Fluent::Plugin::ForwardOutput] adding metadata instance=70125566117840 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:35:37 +0900 [trace]: [Fluent::Plugin::ForwardOutput] writing events into buffer instance=70125566117840 metadata_size=1
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] dequeueing a chunk instance=70125566117840
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] chunk dequeued instance=70125566117840 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag="log", variables=nil>
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] trying flush for a chunk chunk="541661e5762163ca06206dcfd2719937"
2016-11-16 16:35:37 +0900 [trace]: [Fluent::Plugin::ForwardOutput] adding write count instance=70125566828860
2016-11-16 16:35:37 +0900 [trace]: [Fluent::Plugin::ForwardOutput] executing sync write chunk="541661e5762163ca06206dcfd2719937"
2016-11-16 16:35:37 +0900 [warn]: [Fluent::Plugin::ForwardOutput] node closed the connection. regard it as unavailable. host="127.0.0.1" port=24224
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] taking back chunk for errors. plugin_id="object:3fc76180213c" chunk="541661e5762163ca06206dcfd2719937"
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] taking back a chunk instance=70125566117840 chunk_id="541661e5762163ca06206dcfd2719937"
2016-11-16 16:35:37 +0900 [debug]: [Fluent::Plugin::ForwardOutput] chunk taken back instance=70125566117840 chunk_id="541661e5762163ca06206dcfd2719937" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=n
il, tag="log", variables=nil>
2016-11-16 16:35:37 +0900 [warn]: [Fluent::Plugin::ForwardOutput] failed to flush the buffer. plugin_id="object:3fc76180213c" retry_time=0 next_retry=2016-11-16 16:35:38 +0900 chunk="541661e5762163ca06206dcfd2719
937" error_class=Fluent::Plugin::ForwardOutput::ConnectionClosedError error="node 127.0.0.1:24224 closed connection"
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/out_forward.rb:477:in `send_data'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/out_forward.rb:215:in `block in write'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/out_forward.rb:209:in `times'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/out_forward.rb:209:in `write'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/output.rb:994:in `try_flush'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/output.rb:1194:in `flush_thread_run'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin/output.rb:394:in `block (2 levels) in start'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v14/vendor/bundle/ruby/2.3.0/gems/fluentd-0.14.9/lib/fluent/plugin_helper/thread.rb:66:in `block in thread_create'
2016-11-16 16:35:37 +0900 [trace]: [Fluent::Plugin::ForwardOutput] sending heartbeat host="127.0.0.1" port=24224 heartbeat_type=:udp

v0.12.29 in_forward log:

2016-11-16 16:35:26 +0900 [info]: reading config file path="fluent.conf"
2016-11-16 16:35:26 +0900 [info]: starting fluentd-0.12.29
2016-11-16 16:35:26 +0900 [info]: gem 'fluentd' version '0.12.29'
2016-11-16 16:35:26 +0900 [info]: adding match pattern="log.**" type="stdout"
2016-11-16 16:35:26 +0900 [info]: adding source type="forward"
2016-11-16 16:35:26 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    port 24224
    @log_level trace
  </source>
  <match log.**>
    @type stdout
  </match>
</ROOT>
2016-11-16 16:35:26 +0900 [info]: listening fluent socket on 0.0.0.0:24224
2016-11-16 16:35:37 +0900 [trace]: accepted fluent socket from '127.0.0.1:59200': object_id=69849397257480
2016-11-16 16:35:37 +0900 [warn]: emit transaction failed: error_class=MessagePack::MalformedFormatError error="invalid byte" tag="log"
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/event.rb:149:in `feed_each'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/event.rb:149:in `each'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/out_stdout.rb:33:in `emit'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/event_router.rb:90:in `emit_stream'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:165:in `on_message'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:301:in `call'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:301:in `block in on_read_msgpack'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:300:in `feed_each'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:300:in `on_read_msgpack'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:286:in `call'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:286:in `on_read'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/cool.io-1.4.5/lib/cool.io/io.rb:123:in `on_readable'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/cool.io-1.4.5/lib/cool.io/io.rb:186:in `on_readable'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/cool.io-1.4.5/lib/cool.io/loop.rb:88:in `run_once'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/cool.io-1.4.5/lib/cool.io/loop.rb:88:in `run'
  2016-11-16 16:35:37 +0900 [warn]: /home/t-osakada/test/fluentd/v12/vendor/bundle/ruby/2.3.0/gems/fluentd-0.12.29/lib/fluent/plugin/in_forward.rb:107:in `run'
2016-11-16 16:35:37 +0900 [error]: forward error error=#<MessagePack::MalformedFormatError: invalid byte> error_class=MessagePack::MalformedFormatError
  2016-11-16 16:35:37 +0900 [error]: suppressed same stacktrace
2016-11-16 16:35:37 +0900 [trace]: closed socket
2016-11-16 16:35:44 +0900 [trace]: accepted fluent socket from '127.0.0.1:59202': object_id=69849397218900
2016-11-16 16:35:44 +0900 [warn]: emit transaction failed: error_class=MessagePack::MalformedFormatError error="invalid byte" tag="log"
  2016-11-16 16:35:44 +0900 [warn]: suppressed same stacktrace
2016-11-16 16:35:44 +0900 [error]: forward error error=#<MessagePack::MalformedFormatError: invalid byte> error_class=MessagePack::MalformedFormatError
  2016-11-16 16:35:44 +0900 [error]: suppressed same stacktrace
2016-11-16 16:35:44 +0900 [trace]: closed socket
2016-11-16 16:35:52 +0900 [trace]: accepted fluent socket from '127.0.0.1:59204': object_id=69849397192320
2016-11-16 16:35:52 +0900 [warn]: emit transaction failed: error_class=MessagePack::MalformedFormatError error="invalid byte" tag="log"
  2016-11-16 16:35:52 +0900 [warn]: suppressed same stacktrace
2016-11-16 16:35:52 +0900 [error]: forward error error=#<MessagePack::MalformedFormatError: invalid byte> error_class=MessagePack::MalformedFormatError
  2016-11-16 16:35:52 +0900 [error]: suppressed same stacktrace
2016-11-16 16:35:52 +0900 [trace]: closed socket
document
Source
picture t-osakada
👍1

Most helpful comment

@t-osakada Specify time_as_integer true in your configuration of out_forward in v0.14.
See "Sub-second time": http://www.fluentd.org/blog/fluentd-v0.14.0-has-been-released

picture tagomoris on 16 Nov 2016
👍3

All 6 comments

@t-osakada Specify time_as_integer true in your configuration of out_forward in v0.14.
See "Sub-second time": http://www.fluentd.org/blog/fluentd-v0.14.0-has-been-released

tagomoris picture tagomoris on 16 Nov 2016
👍3

This should be included in the release note of v0.14.9. @repeatedly ?

tagomoris picture tagomoris on 16 Nov 2016

That solved it. Thanks!

t-osakada picture t-osakada on 16 Nov 2016

That's very nice to hear. Thank you to remind us to note it on release notes!

tagomoris picture tagomoris on 16 Nov 2016

I have posted v0.14.9 release note.

repeatedly picture repeatedly on 16 Nov 2016

I'm seeing the same error being logged. The logs are being forwarded from td-agent version 0.14.11 to fluentd version 0.12.26.

I added time_as_integer true in my configuration of out_forward in v0.14.11 td-agent, but the same MalformedFormatError error is being reported in fluentd v0.12.26.

<match **>
  type forward
  time_as_integer true
  heartbeat_type tcp
  flush_interval 3s
  max_retry_wait 1024s
  retry_limit 2000
  expire_dns_cache 30

  buffer_type file
  buffer_path /var/log/msi/buffer/forward
  buffer_chunk_limit 8m
  buffer_queue_limit 4096
  num_threads 4

  <server>
    name fluentd
    host fluentd
    port 24224
  </server>

  # in case fluentd server is not available
  <secondary>
    type file
    path /var/log/msi/buffer/secondary
    #compress gzip
  </secondary>
</match>
jcstanaway picture jcstanaway on 24 Jan 2017
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

robert-7 picture robert-7  路  4Comments
aniamin picture aniamin  路  7Comments
rolindroy picture rolindroy  路  7Comments
aleks-mariusz picture aleks-mariusz  路  7Comments
amread picture amread  路  5Comments