Logstash: Invalid FieldReference: `variables[XRAY_ENABLED]`

[max-wittig]

• Version: logstash-oss:7.4.0
• Operating System: none, docker based image

Since the 7.4.0 update, we got a lot of warnings related to XRAY.
Nothing related to XRay was configured.

org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: variables[XRAY_FIXVERSION]
logstash_1 | at org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:283) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.FieldReference.parse(FieldReference.java:184) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.FieldReference.parseToCache(FieldReference.java:175) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.FieldReference.from(FieldReference.java:107) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedMap.put(ConvertedMap.java:75) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:55) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:152) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.convert(Valuefier.java:74) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:55) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:152) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.convert(Valuefier.java:74) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedList.newFromList(ConvertedList.java:27) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.lambda$initConverters$14(Valuefier.java:153) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.convert(Valuefier.java:74) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:55) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:152) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.Valuefier.convert(Valuefier.java:74) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:55) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.initializeFallback(JrubyEventExtLibrary.java:303) ~[logstash-core.jar:?]
logstash_1 | at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_initialize(JrubyEventExtLibrary.java:77) ~[logstash-core.jar:?]

Any ideas?

[yaauie]

Something in your pipeline is attempting to use an invalid field reference when working with the event. From what I can tell in your backtrace, it is in an codec, which means that the raw data being handed to the codec contains field names that are not supported.

The strict-mode field reference parser was introduced as an opt-in feature in ~6.5, and made the default in 7.0; where the former field reference parser allowed ambiguous inputs, the strict-mode parser requires valid field references.

I get that this is frustrating. My advice would be to downgrade to 6.8 (which by default will merely _warn_ when it encounters ambiguous references)

See:

• https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references
• https://www.elastic.co/guide/en/logstash/7.4/field-references-deepdive.html

[max-wittig]

@yaauie Thanks for the response. These are actually also just warnings and everything is working fine. The thing is that there is no field XRAY something in my logs.

[max-wittig]

Is there some setting that was added in 7.4?

[colinsurprenant]

Opened #11608 to discuss this.