Pfelk: [OPNsense] ipv6-icmp _grokparsefailure

[rrosson]

Describe the bug
parse failure occurring with ipv6-icmp and maybe other related ipv6

Screenshots

@timestamp | Apr 24, 2020 @ 13:10:29.479
-- | --
  | @version | 1
  | _id | b4eZrXEBLHxnhfQdLlTn
  | _index | pf-2020.04.24
  | _score | -
  | _type | _doc
  | event.created | Apr 24 13:10:29
  | event.id | 134
  | event.original | <134>Apr 24 13:10:29 myfirewall.mydomain.org filterlog: 22,,,0,igb1,match,pass,in,6,0x00,0x00000,255,ipv6-icmp,58,32,fe80::1698:7dff:fe06:9bdb,2603:300b:d08:bb00:230:18ff:fece:f114,
  | host | 192.168.0.1
  | labels | filterlog
  | observer.name | myfirewall.mydomain.oneinsane.org
  | observer.type | firewall
  | pf_message | 22,,,0,igb1,match,pass,in,6,0x00,0x00000,255,ipv6-icmp,58,32,fe80::1698:7dff:fe06:9bdb,2603:300b:d08:bb00:230:18ff:fece:f114,
  | tags | pf, _grokparsefailure, dns

Firewall System (please complete the following information):

• OPNsense
• Version: 20.1.5

Operating System (please complete the following information):

• OS:

Linux 4.15.0-96-generic x86_64
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Elasticsearch, Logstash, Kibana (please complete the following information):

• Version of ELK:

ii  elasticsearch                         7.6.2                                           amd64        Distributed RESTful search engine built for the cloud
ii  kibana                                7.6.2                                           amd64        Explore and visualize your Elasticsearch data
ii  logstash                              1:7.6.2-1                                       all          An extensible logging pipeline

Elasticsearch, Logstash, Kibana logs:
Nothing found related to issue

[a3ilson]

I do not have IPv6 from my ISP and currently do not use it. I understand IPv6 filtering is needed and seeking contributions for IPv6 filtering/GROK patterns.

Thanks!

[revere521]

I also have it disabled, as it interferes with my VPN

@rrosson you could try throwing a greedydata field at it, to see what it collects - and then we could try to build out a pattern

in the pfelk.grok:

# IPv6
PF_IPv6_VAR %{WORD:Type},%{WORD:Option},%{WORD:Flags},%{WORD:Flags}
PF_IPv6_ICMP %{GREEDYDATA:[icmptest]}

then report back what you get in the new icmptest field

[a3ilson]

@rrosson - and anyone else with IPv6 running. Please post/provide the pf_message raw text field. We'll refine the GROK pattern, post back on here and test for future inclusion.

[rrosson]

currently I only have the gateway configured with IPV6 and am still playing with provider on getting it to work properly. Here are the ipv6-icmp messages:

event.original
<134>Apr 25 07:05:54 myfirewall.mydomain.org filterlog: 22,,,0,igb1,match,pass,in,6,0x00,0x00000,255,ipv6-icmp,58,32,fe80::1698:7dff:fe06:9bdb,fe80::230:18ff:fece:f114,
pf_message
22,,,0,igb1,match,pass,in,6,0x00,0x00000,255,ipv6-icmp,58,32,fe80::1698:7dff:fe06:9bdb,fe80::230:18ff:fece:f114,
event.original
<134>Apr 25 07:05:54 myfirewall.mydomain.org filterlog: 32,,,0,igb1,match,pass,out,6,0x00,0x00000,255,ipv6-icmp,58,24,fe80::230:18ff:fece:f114,fe80::1698:7dff:fe06:9bdb,
pf_message
32,,,0,igb1,match,pass,out,6,0x00,0x00000,255,ipv6-icmp,58,24,fe80::230:18ff:fece:f114,fe80::1698:7dff:fe06:9bdb,

[TehMuffinMoo]

Whilst I'm helping out here...

This looks like it's simply because the [protocol][type] field in the IPv6 Grok filter is set to WORD which I'd imagine would fit every other use case here, but this is ipv6-icmp which won't match due to the hyphen.

I also don't use IPv6.. but for the few events I do get the protocol.type is a single word. (TCP/UDP/Options/etc.)

Changing this to DATA would fix it.

[revere521]

I'm not sure how much ipv6 traffic i see, but i'll test that - looks reasonable

[TehMuffinMoo]

Sweet! Thanks

For those who may need this clarifying!

I know some of my field names may not match in the images above as some I’ve changed for my own purposes. But this should sort it in patterns file (pfelk.grok);

Change this;

PF_IPv6_SPECIFIC_DATA (?<[network][type]>(6)),%{BASE16NUM:IPv6_Flag1},%{WORD:IPv6_Flag2},%{WORD:flow_label},%{WORD:[protocol][type]},%{INT:[protocol][id]},

To this;

PF_IPv6_SPECIFIC_DATA (?<[network][type]>(6)),%{BASE16NUM:IPv6_Flag1},%{WORD:IPv6_Flag2},%{WORD:flow_label},%{DATA:[protocol][type]},%{INT:[protocol][id]},

[revere521]

@rrosson could you also test this to see if it solves your issue?

[rrosson]

@rrosson could you also test this to see if it solves your issue?

Sorry for the delay. I was on vacation when this came in. Just made the change. will report back when I get something.

[revere521]

no worries at all

[ShagoY]

Its work, no more _grokparsefailure for ipv6-icmp

[a3ilson]

@rrosson Let me know if the provided solution from @TehMuffinMoo suffices and/or if your still having failures and if so can you provide those.

[rrosson]

This can be closed. Thanks for all the time

[a3ilson]

No worries...let me know if you have any more ipv6 issues. I'll be enriching ipv6 data next year (moving) and will have accessibility to ipv6 through my isp.