Pfelk: Firewall Dashboard Error set fielddata=true

[InvokerFury]

Hi Sir,

thanks for make this greate application for logging PFsense, I have an issue when trying to access my firewall-dasboard, its empty and error, I don't know how to make it running, this is the log from kibana.

{ "request": {}, "response": {}, "body": { "statusCode": 400, "error": "Bad Request", "message": "[status_exception] error while executing search", "attributes": { "error": { "type": "status_exception", "reason": "error while executing search", "caused_by": { "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "pfelk-firewall-2020.10.20", "node": "efb914wxRTafTepEl7TxnQ", "reason": { "type": "illegal_argument_exception", "reason": "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [@timestamp] in order to load field data by uninverting the inverted index. Note that this can use significant memory." }

Pls Help Thanks

[a3ilson]

@InvokerFury - There isn't enough data to isolate the error. Would you be able to provide the pfELK log? This would be located at /etc/pfELK/error-data.sh if utilizing the scripted installer. Otherwise, please download here and provide the log.

[InvokerFury]

Hi Sir,

thanks for your response, i installed manually because I am using amazon linux. here I attached the log
error.pfelk.log

Thanks

[a3ilson]

@InvokerFury - Thanks!

This one is on me...I didn't test the OpenVPN grok pattern. There is an error on line 624. You'll need to do the following:

• Re-download the pfelk.grok - I corrected the issue (i.e. missing brackets on the vpn.client field).

or

• Amend pfelk.grok pattern line 624 from ...[%{DATA:vpn][client}\]... to ...[%{DATA:[vpn][client]}\]...

One you have correct the mistake simply restart logstash (e.g. systemctl restart logstash.service)

Thanks!

[InvokerFury]

Hi Sir,

Your welcome sir, but my firewall-dasboard on kibana still not running, 2 days ago its ok, but when I update I got this error, here I attached ss.

[a3ilson]

Did you purge the previously saved objects, templates and indices prior to upgrading? Unfortunately, there wasn't a clean upgrade and the changes broke prior instances/installations.

[InvokerFury]

Hi Sir,

I try to reinstall today for a fresh install, but the same thing still happened. Would you help, please? because 2 days ago it's running well..

Thanks

[a3ilson]

Did you install the required templates?

You should see them within the index management section, as depicted below:

[InvokerFury]

Hi Sir,

this is mine, maybe I do something wrong, would you help, please?

[a3ilson]

Were the firewall logs being sent prior to importing the templates and dashboards?

[InvokerFury]

yes, sir, i configure logstash first and sending the log to elasticsearch, and then i import the dashboard

[a3ilson]

Are you receiving parsed/enriched logs? Check Kibana>>Discover

[InvokerFury]

i receive the parsed log from the firewall, only firewall-dashboard not working sir

[a3ilson]

Ok, so you have other Dashboards that are working?

[InvokerFury]

yah sir, my wazuh dashboard running well, when trying to access pfelk-dashboard i got notice all shard failed 'search_phase_execution_exception
all shards failed'

[a3ilson]

After you corrected the vpn.client bracket error and after restarting logstash - are there any logstash issues/errors? I'm assuming not, if the Discover is working.

One of the elements within the Firewall Dashboard is discover, can you confirm this is accessible and function?

[InvokerFury]

yes sir, discover its working well, i don't know why dashboard still not function, its always said 400 bad request, set fieltdata=true

[InvokerFury]

Iam sorry sir, its working now, i think because memory issue on my ec2, its 4gb ram
thanks for your advice, I ll upgrade my instance first and tuning elasticsearch.

[a3ilson]

After looking at your previous response (i.e. the screenshot of discover), it didn't appear you had received enough log. I'm guessing you only log blocked events vs all hence the limited logs.

I would let it run a bit and check back.

Feel free to share your Wazuh dashboard and conf files. That'd be a nice plugin to share.

[InvokerFury]

Hi Sir,

the wazuh iam using right now is from the official site sir, here the link for wazuh that I use https://wazuh.com/. really appreciate your help, from Indonesia with love.

Thanks & regards,
Invoker

[a3ilson]

Thanks and glad we were able to resolve this!