Pfelk: cant parse the data in firewall

[keshif]

pfsense 2.5
ubuntu 20.04
sudo tcpdump -vvv -A -i any port 5140 can see the data
but dashboard firewall cant parse it

[gokhansezen]

pfsense 2.5
Ubuntu 20.4.2 LTS

[gokhansezen]

[a3ilson]

@keshif & @gokhansezen - Please check your logstash log. I suspect the issue was the result of an added grok pattern and revision to pfelk.grok (i.e. openvpn grok patterns were removed and an openvpn.grok file was created).

Logstash is likely not running as it is unable to define the openvpn grok pattern as a result of the changes noted above.

Solution:
download the openvpn.grok file and restart logstash

• sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/patterns/openvpn.grok -P /etc/pfelk/patterns/
• systemctl restart logstash.service

Please check back to confirm the proposed solution reminded the issue. Updates were made in the manual install instructions and script.

[EverWin42O]

Hi a3ilson,

Facing same issue, above fix didn't worked for me.

(pfSense 2.5.0 & ubuntu 20.04), it was working before updating with script v20.3a.

[a3ilson]

Awe shucks... can you run the error script and provide the associated log?

[EverWin42O]

error.pfelk.log

Thanks for your quick response.
PFA.

[a3ilson]

error.pfelk.log

Thanks for your quick response.
PFA.

Thanks! Found the issue..will fix here in a few

[a3ilson]

Redownload the openvpn.grok and restart logstash.

🐛 The issue or error:

,

🔧 The fix:
Amended Line 42 of openvpn.grok from:

OPENVPN_MGT(%{OPENVPN_MGT_CLIENT}|%{OPENVPN_MGT_CMD})

to:

OPENVPN_MGT (%{OPENVPN_MGT_CLIENT}|%{OPENVPN_MGT_CMD})

added a space between OPENVPN_MGT and the applied GROK

[EverWin42O]

updated openvpn.grok and restart logstash. Still no logs displayed in Discovery > firewall, suricata, dhcp (none).
error.pfelk.log

[a3ilson]

updated openvpn.grok and restart logstash. Still no logs displayed in Discovery > firewall, suricata, dhcp (none).
error.pfelk.log

Thanks @EverWin42O !

Had another typo (e.g. OEPNVPN_OTHER vs OPENVPN_OTHER on line 52).

Reload openvpn.grok and restart Logstash.

[gokhansezen]

It's the same with me :(

[EverWin42O]

Updated, Didn't worked.
error.pfelk.log

[a3ilson]

Updated, Didn't worked.
error.pfelk.log

No errors were noted...I would recommend purging (deleting) your indices then check to see if they are recreated and the document counts increase (logs).

[EverWin42O]

Tried deleting; it got created after reboot with yellow mark. Still no data.

[gokhansezen]

DHCPD DHCP(%{DHCPD_DISCOVER}|%{DHCPD_DUPLICATE}|%{DHCPD_OFF>

[a3ilson]

@EverWin42O - I rebuilt my instance (loaded a snapshot of Ubuntu and installed pfelk)... everything appears to be working. Can you check to see if Logstash is logging any of your logs (this would be denoted within the index [i.e. document count])

[a3ilson]

DHCPD DHCP(%{DHCPD_DISCOVER}|%{DHCPD_DUPLICATE}|%{DHCPD_OFF>

@gokhansezen - please elaborate and/or provide context

[a3ilson]

@EverWin42O - what are you running (pfSense or OPNsense) and what version?

[EverWin42O]

pfsense 2.5.0

[a3ilson]

@EverWin42O - Let's give this a try:

Within pfSense try changing the output of your syslog (i.e. from BSD to Syslog or vice versa). The format was recently troubleshoot (#246) and was testing/working. However, if nothing is being logged and no errors are present, this would be the most logical issue.

1. Change pfSense logging output
   
   
   
   • Does problem still exist?
   
   

[EverWin42O]

Yes, Tried that switching BSD to Syslog and vice versa.
Tried clean installation and auto script.
clean ubuntu install and doing manual installation.

[a3ilson]

Yes, Tried that switching BSD to Syslog and vice versa.
Tried clean installation and auto script.
clean ubuntu install and doing manual installation.

Alright, let me know if it works out. If you still have the same results, I'd be curious to your logging and could further troubleshoot/assist. Just provide a sanitized (amend IPs as needed/desired) by providing a sample of logs via tcpdump (sudo tcpdump -vvv -A -i any port 5140)

[EverWin42O]

Thank you so much for your support. Finally it started working. auto script worked perfectly. pfsense BSD log works.

[a3ilson]

@gokhansezen & @keshif - After updating the openvpn.grok file, let me know if you still have the same issue.

@EverWin42O - thanks for troubleshooting and providing the necessary logs to identify and correct.

[keshif]

firewall and ubound started working !
Thanks

[gokhansezen]

Many many thanks @a3ilson
Now automated install script running is perfectly.