Elasticsearch: Ingest processor fails if `host.ip` contains an array of IPs

[rwaight]

Elasticsearch version: 7.3.1 running on Elastic Cloud

Description of the problem including expected versus actual behavior:
When attempting to enrich events with geoIP information in beats, with netinfo.enabled: true defined, the Elasticsearch Ingest processor fails if host.ip contains an array of IPs. The expected behavior is that an array of IPs can be ingested into Elasticsearch. The current workaround is to disable netinfo (i.e., netinfo.enabled: false)

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

1. Configure adding the geoip-info ingest pipeline in order to enrich events with geoIP information

   1. Configure the Elasticsearch output to use the geoip-info pipeline (i.e. output.elasticsearch.pipeline: geoip-info)

   2. Configure a top-level processor in *beats (this impacts auditbeat, filebeat, heartbeat, metricbeat, packetbeat, and winlogbeat) to add geo information, with netinfo.enabled: true
      

      configuration from packetbeat.yml

processors:
  - add_host_metadata:
      netinfo.enabled: true
  - add_cloud_metadata: ~
  - add_fields:
      when.network.source.ip: 10.100.10.0/24
      fields:
        source.geo.location:
          lat: 42
          lon: -93
        source.geo.continent_name: North America
        source.geo.region_iso_code: US-IA
        source.geo.country_iso_code: US
        source.geo.region_name: Iowa
      target: ''
  - add_fields:
      when.network.destination.ip: 10.100.10.0/24
      fields:
        destination.geo.location:
          lat: 42
          lon: -93
        destination.geo.continent_name: North America
        destination.geo.region_iso_code: US-IA
        destination.geo.country_iso_code: US
        destination.geo.region_name: Iowa
      target: ''

1. Start packetbeat, it is unable to publish events to Elasticsearch, when not in DEBUG mode, the only response generated is:

ERROR    pipeline/output.go:121    Failed to publish events: temporary bulk send failure

Provide logs (if relevant):

elasticsearch/client.go:533    Bulk item insert failed (i=0, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}

more of the same Bulk item insert failed logs

2019-08-30T10:04:26.038-0500    DEBUG   [elasticsearch] elasticsearch/client.go:339 PublishEvents: 35 events have been published to elasticsearch in 130.0856ms.
2019-08-30T10:04:26.038-0500    DEBUG   [tcp]   tcp/tcp.go:177  tcp flow id: 0xc000e29900
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=0, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:216  flow id flags: 3089
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=1, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:94   get flow
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=2, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:96   lookup flow: {3089 0 255 255 255 12 255 255 255 255 255 20 24 1 0 1} => [124 173 116 146 217 104 248 177 86 165 135 119 104 198 38 169 10 100 10 20 187 1 193 226 3 0 0 0 0 0 0 0]
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:89   unlock flows
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=3, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [sniffer]   sniffer/sniffer.go:209  Packet number: 49991
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=4, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:173  decode packet data
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:84   lock flows
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=5, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:157 flowid: add eth
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=6, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:246  IPv4 packet
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=7, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=8, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:163 flowid: add ipv4
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=9, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:287  TCP packet
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:230 flowid: add tcp
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=10, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:216  flow id flags: 1041
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:94   get flow
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=11, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:96   lookup flow: {1041 0 255 255 255 12 255 255 255 255 255 20 255 1 0 1} => [0 80 182 150 119 253 248 177 86 165 135 119 10 100 10 15 10 100 10 20 21 194 61 13]
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=12, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=13, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:89   unlock flows
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=14, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=15, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [sniffer]   sniffer/sniffer.go:209  Packet number: 49992
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=16, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:173  decode packet data
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flows.go:84   lock flows
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=17, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=18, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=19, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:157 flowid: add eth
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=20, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:246  IPv4 packet
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:163 flowid: add ipv4
2019-08-30T10:04:26.039-0500    DEBUG   [decoder]   decoder/decoder.go:287  TCP packet
2019-08-30T10:04:26.039-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=21, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.039-0500    DEBUG   [flows] flows/flowid.go:230 flowid: add tcp
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=22, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [decoder]   decoder/decoder.go:216  flow id flags: 1041
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=23, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flows.go:94   get flow
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=24, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flows.go:96   lookup flow: {1041 0 255 255 255 12 255 255 255 255 255 20 255 1 0 1} => [0 80 182 150 119 253 248 177 86 165 135 119 10 100 10 15 10 100 10 20 21 194 61 13]
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=25, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flows.go:89   unlock flows
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=26, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [sniffer]   sniffer/sniffer.go:209  Packet number: 49993
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=27, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [decoder]   decoder/decoder.go:173  decode packet data
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flows.go:84   lock flows
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=28, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flowid.go:157 flowid: add eth
2019-08-30T10:04:26.040-0500    DEBUG   [decoder]   decoder/decoder.go:246  IPv4 packet
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=29, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=30, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flowid.go:163 flowid: add ipv4
2019-08-30T10:04:26.040-0500    DEBUG   [decoder]   decoder/decoder.go:287  TCP packet
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=31, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flowid.go:230 flowid: add tcp
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=32, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flowid.go:236 flowid: add tcp connection id
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=33, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [tcp]   tcp/tcp.go:177  tcp flow id: 0xc000e29900
2019-08-30T10:04:26.040-0500    DEBUG   [elasticsearch] elasticsearch/client.go:533 Bulk item insert failed (i=34, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]","caused_by":{"type":"illegal_argument_exception","reason":"field [host.ip] of type [java.util.ArrayList] cannot be cast to [java.lang.String]"}},"header":{"processor_type":"geoip"}}
2019-08-30T10:04:26.040-0500    DEBUG   [decoder]   decoder/decoder.go:216  flow id flags: 3089
2019-08-30T10:04:26.040-0500    DEBUG   [flows] flows/flows.go:94   get flow
2019-08-30T10:04:26.040-0500    INFO    [publisher] pipeline/retry.go:155   Drop batch

When not in DEBUG mode, the only response generated is:

ERROR    pipeline/output.go:121    Failed to publish events: temporary bulk send failure

[elasticmachine]

Pinging @elastic/es-core-features

[martijnvg]

So the ip addresses are then provided as json array in the source field (the field setting in the pipeline config)? If that is the case then is the expected behaviour is that the geo information is appended in the configured target_field (in the order matching to the order of the ip addresses in the configured source field)?

[webmat]

I think an acceptable implementation would be to loop over addresses, and simply decorate the event based on the first IP that has a match in the DB.

The most likely scenario this will happen is in situations specifically like host.ip, where a host has a bunch of IP addresses for real and virtual network interfaces. So most of them would be private, and potentially one of them would be a public address. If there's more than one public address assigned to the host, only the first one would be geolocated. The other ones would likely have yielded a similar result anyway.

[sspreitzer]

I am opting for a geoip feature where the ingest would iterates over an array of IPs and enrich the document's geoip field with an array of geoip information.

(Furthermore opening the possibility to point to point paths in visualizations. As well opening up the possibility for computed searches shortest paths, triangulations, etc..)

[rwaight]

Hi @probakowski, thank you for opening the PR! Would it be possible to backport to 7.4?

[jbaiera]

@rwaight Now that 7.5 is released I doubt there will be any further releases of the 7.4 branch

[rwaight]

Thanks for clarifying this with me, @jbaiera! I'll keep an eye on this for release in 7.6.

[robertdeheer]

I'm still seeing a similar issue in 7.6.0.

[probakowski]

@robertdeheer would you mind sharing exact error/stacktrace?
I've just tested it on 7.6.0 with:

PUT _ingest/pipeline/testpipe
{
  "processors": [
    {
      "geoip": {
        "field": "ip",
        "target_field": "geo",
        "ignore_missing": true
      }
    }
  ]
}

POST _ingest/pipeline/testpipe/_simulate
{
  "docs": [
    {
      "_source": {
        "ip": ["192.168.1.1", "151.101.2.217"]
      }
    }
  ]
}

and it works as expected

[robertdeheer]

Yes, the code you provided worked. I realized that my issue is probably
related to locating hosts in a private network and visualizing them in
kibana. That’s a different issue.

Thanks,
Rob

On Tue, Mar 10, 2020 at 12:49 PM Przemko Robakowski <
[email protected]> wrote:

@robertdeheer https://github.com/robertdeheer would you mind sharing
exact error/stacktrace?
I've just tested it on 7.6.0 with:

PUT _ingest/pipeline/testpipe
{
"processors": [
{
"geoip": {
"field": "ip",
"target_field": "geo",
"ignore_missing": true
}
}
]
}

POST _ingest/pipeline/testpipe/_simulate
{
"docs": [
{
"_source": {
"ip": ["192.168.1.1", "151.101.2.217"]
}
}
]
}

and it works as expected

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch/issues/46193?email_source=notifications&email_token=AD22AX3N46Q6XSOIOJZHFZLRG2KWJA5CNFSM4ISPSR6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOM4PGQ#issuecomment-597280666,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD22AXY3V5W6LO3ZZTUF5DLRG2KWJANCNFSM4ISPSR6A
.