Logstash: cloning a LogStash::Timestamp results in an empty java object

[jsvd]

2.3.0 :012 > LogStash::Timestamp.now.to_java
 => #<Java::OrgLogstash::Timestamp:0x48110259> 
2.3.0 :013 > LogStash::Timestamp.now.clone.to_java
 => nil 

My suspicion is that the initialization of a LogStash::Timestamp requires the call to setTimestamp on a new instance. When doing a clone this set operation isn't called so the resulting cloned object doesn't contain the org.logstash.Timestamp instance

[original-brownbear]

Fixed all the way back to 5.6 via #8889 #8890 and #8891

[fbaligand]

Great !
Thanks @original-brownbear !

[rolodato]

Hi @original-brownbear,

I'm still able to reproduce this on 6.2.3 with this config:

input {
    stdin { }
}

filter {
  mutate {
    copy => {
      "@timestamp" => "@timestamp2"
    }
  }
}

output {
   stdout { codec => "rubydebug" }
}

Then, run logstash and enter anything into stdin. This is the result:

$ echo "foo" | /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/npe-repro.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-04-17 14:15:24.598 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[INFO ] 2018-04-17 14:15:24.604 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[WARN ] 2018-04-17 14:15:24.891 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-04-17 14:15:25.026 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.3"}
[INFO ] 2018-04-17 14:15:25.197 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-04-17 14:15:25.653 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-04-17 14:15:25.701 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2d19c28f@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 sleep>"}
[INFO ] 2018-04-17 14:15:25.720 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}
Exception in thread "Ruby-0-Thread-8@[main]>worker0: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:384" java.lang.NullPointerException
    at org.logstash.ext.JrubyTimestampExtLibrary$RubyTimestamp.ruby_to_iso8601(JrubyTimestampExtLibrary.java:112)
    at org.logstash.ext.JrubyTimestampExtLibrary$RubyTimestamp.ruby_inspect(JrubyTimestampExtLibrary.java:106)
    at org.logstash.ext.JrubyTimestampExtLibrary$RubyTimestamp$INVOKER$i$0$0$ruby_inspect.call(JrubyTimestampExtLibrary$RubyTimestamp$INVOKER$i$0$0$ruby_inspect.gen)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:318)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:131)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:339)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:193)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:193)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:84)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:134)
    at org.jruby.ir.runtime.IRRuntimeHelpers.yieldSpecific(IRRuntimeHelpers.java:419)
    at org.jruby.ir.instructions.YieldInstr.interpret(YieldInstr.java:74)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.processOtherOp(StartupInterpreterEngine.java:187)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:111)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:186)
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
    at org.jruby.runtime.Block.yield(Block.java:165)
    at org.jruby.RubyArray.collectBang(RubyArray.java:2499)
    at org.jruby.RubyArray.map_bang(RubyArray.java:2518)
    at org.jruby.RubyArray$INVOKER$i$0$0$map_bang.call(RubyArray$INVOKER$i$0$0$map_bang.gen)
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:498)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:298)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:318)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:131)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:339)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:318)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:131)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:339)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:318)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:131)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:339)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.RubyClass.finvoke(RubyClass.java:531)
    at org.jruby.RubyBasicObject.send19(RubyBasicObject.java:1670)
    at org.jruby.RubyKernel.send19(RubyKernel.java:1986)
    at org.jruby.RubyKernel$INVOKER$s$send19.call(RubyKernel$INVOKER$s$send19.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:77)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:144)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:130)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
    at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:56)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:318)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:131)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:339)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.RubyMethod.call(RubyMethod.java:115)
    at org.jruby.RubyMethod$INVOKER$i$call.call(RubyMethod$INVOKER$i$call.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:186)
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
    at org.jruby.runtime.Block.yield(Block.java:165)
    at org.jruby.RubyArray.each(RubyArray.java:1734)
    at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:498)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:298)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:84)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:134)
    at org.jruby.ext.thread.Mutex.synchronize(Mutex.java:148)
    at org.jruby.ext.thread.Mutex$INVOKER$i$0$0$synchronize.call(Mutex$INVOKER$i$0$0$synchronize.gen)
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:498)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:298)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:186)
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
    at org.jruby.runtime.Block.yield(Block.java:165)
    at org.jruby.RubyHash$12.visit(RubyHash.java:1362)
    at org.jruby.RubyHash$12.visit(RubyHash.java:1359)
    at org.jruby.RubyHash.visitLimited(RubyHash.java:662)
    at org.jruby.RubyHash.visitAll(RubyHash.java:647)
    at org.jruby.RubyHash.iteratorVisitAll(RubyHash.java:1319)
    at org.jruby.RubyHash.each_pairCommon(RubyHash.java:1354)
    at org.jruby.RubyHash.each(RubyHash.java:1343)
    at org.jruby.RubyHash$INVOKER$i$0$0$each.call(RubyHash$INVOKER$i$0$0$each.gen)
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:498)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:298)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)
    at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)
    at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:73)
    at org.jruby.runtime.Block.call(Block.java:124)
    at org.jruby.RubyProc.call(RubyProc.java:289)
    at org.jruby.RubyProc.call(RubyProc.java:246)
    at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)
    at java.lang.Thread.run(Thread.java:748)

[original-brownbear]

@rolodato strange, will take a look :)

[rolodato]

Thanks for the quick response! Forgot to mention:

$ ruby --version
jruby 9.1.13.0 (2.3.3) 2017-09-06 8e1c115 OpenJDK 64-Bit Server VM 25.161-b14 on 1.8.0_161-b14 +jit [linux-x86_64]

[fbaligand]

Hi all,

I confirm I have the same issue with Logstash 6.2.2

[original-brownbear]

@jsvd maybe you're more qualified to give an opinion here than me (tricky Ruby stuff).

The problem here is that we're not, in fact, using .clone or .dup to copy the timestamp.
Instead, we're using:

Marshal.load(Marshal.dump(o))

in LogStash::Util.deep_clone (mutate calls that) and that fails to catch the Java typed (TimeStamp) field in the RubyTimestamp.

Any reason we're using the marshalling and not .clone there?

[jsvd]

in ruby a .clone (and .dup) is shallow:

jruby-9.1.12.0 :001 > arr = [Object.new]
 => [#<Object:0x6c130c45>] 
jruby-9.1.12.0 :002 > arr.first.object_id
 => 2000 
jruby-9.1.12.0 :003 > arr.clone.first.object_id
 => 2000 

and in most situations we want to ensure that we aren't passing object references around while we're mutating the clones, so we want to do a deep clone. The only "clean" way in ruby to do a deep clone is using marshalling, which saves us from the trouble of implementing a true deep_clone with all the insane traversal necessary:

jruby-9.1.12.0 :004 > Marshal.load(Marshal.dump(arr)).first.object_id
 => 2002 

[original-brownbear]

@jsvd Thanks!
I guess the best thing we can do then, is to add a special case for TimeStamp to deep_clone when we can't marshal Java fields. (that should, in fact, be fine, since I fixed that .clone method here earlier and it's ok to shallow clone its immutable fields). Any reason not to do that?

[jsvd]

This could also probably be solved by implementing the marshalling strategy for this particular class.

I'm good with either solution. Eventually we'll rewrite most of the LogStash::Utils and the mutate field anyway.

[xSAVIKx]

I'm also receiving the same stack trace on docker.elastic.co/logstash/logstash-oss:6.2.4.

[original-brownbear]

@jsvd thanks, went for .clone because I already fixed the clone method here before :) so that was a one-liner in https://github.com/elastic/logstash/pull/9405 :)

[original-brownbear]

fixed in https://github.com/elastic/logstash/pull/9405 :)

[ebuildy]

This bug also occurred if you want to copy @metadata generated by dead_letter_queue:

input
{
  dead_letter_queue
  {
      path => "/usr/share/logstash/dead_letter_queue"
      commit_offsets => true
      pipeline_id => "main"
  }
}

filter
{
  mutate
  {
      copy => {"@metadata" => "_logstash_metadata"}
  }
}