Beats: How to disable xpack check in Filebeat

[xiaowangwindow]

hello, I am a newbie to use ES and Kibana in Amazon Elasticsearch Service, which do not support xpack plugin.
I use Filebeat to send nginx log to ES directly. However, when run sudo filebeat setup -e, error happen like below:

ERROR   instance/beat.go:691    Exiting: 2 errors: Error checking if xpack is available: 401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."}; Error checking if xpack is available: 401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."}

the error happened because elasticsearch in Amazzon Elasticsearch Service don't support xpack plugin, and user can't install manually.

So, I add below config to /etc/filebeat/filebeat.yml in order to disable xpack and forbidden filebeat to check xpack availability. But the error still exists.

xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.reporting.enabled: false

Is there any way to disable filebeat xpack check?

[ph]

@xiaowangwindow This question is more appropriate to our discuss forum; we keep the issue tracker for bug only.

I am not sure how you run into this issue, I've tested it with the latest release of Filebeat and by default xpack monitoring is disabled you have to enable it explicitly. So by default, it should not try to connect to a remote cluster.

I do not know which version of Filebeat you are running, but I believe you are hitting an old behavior described in https://github.com/elastic/beats/pull/6627.

I suggest you update to the latest version of FB which is 6.4.0.

I will close this issue if its still a problem for you create a discuss post.

[coden-ata]

I'm experiencing this same issue. No luck in the discussion boards.

[ph]

@coden-ata Can you add your configuration to this issue?

[coden-ata]

See attached config file:

filebeat.txt

[coden-ata]

The problem seems to be that the xpack check occurs whether or not Xpack and ML are disabled:
https://github.com/elastic/beats/blob/6.5/filebeat/fileset/modules.go#L356

[marcelcamargo]

The problem seems to be that the xpack check occurs whether or not Xpack and ML are disabled:
https://github.com/elastic/beats/blob/6.5/filebeat/fileset/modules.go#L356

Any solution?
Here, same problem = /

[christianlusardi]

Any solution?
Here, same problem = /

[Overbryd]

Here is my solution for running filebeat-oss (notice the oss, it stands for OPEN source.)
The following command is required to boot up filebeat without touching any of the wonderful x-pack-upsell-code.

$ filebeat setup --pipelines --template --dashboards -c /etc/filebeat.yml

Here is an excerpt on how to integrate that with an initContainer in Kubernetes:

      initContainers:
      - name: filebeat-init
        image: docker.elastic.co/beats/filebeat-oss:7.1.1
        envFrom:
        - configMapRef:
            name: elk-internal
        securityContext:
          runAsUser: 0
        command: ['filebeat', 'setup', '--pipelines', '--template', '--dashboards', '-c', '/etc/filebeat.yml']

<rant>

I mean, Elastic get to terms with AWS... This is getting more than silly.

All that Elastic wants is people pouring into their Elastic Cloud service.

All that AWS wants is people pouring into their Elastic Search service.

In the end the customers are the ones loosing, by the ignorance of both companies. Elastic deliberately ignores compatibility with one of the largest OSS service installations out there, and spits in the face of these users.

There are numerous reasons why this is a bad idea, the most important one, is that Elastic only ever cares about its own customers.
OSS or other 3rd party providers are left outside the door, and their users must use hacky solutions to get around the foul breadcrumbs that Elastic is dropping everywhere.

Thank you all great people at Elastic. OSS is GREAT, and you are such an adorable supporter of it all. As long as we become your customer of course.

Yea... I am looking at YOU Elastic. Don't act silly.
</rant>

[wdullaer]

I tried the workaround described here, but it's not working for me on 7.1.1, 7.2.1 and 7.3.1
Apparently beats insists on doing a check for ILM.
When searching for that error, you end up in this function:
https://github.com/elastic/beats/blob/281aab166269002cde4f4b15318ba1e43885d65e/libbeat/idxmgmt/ilm/client_handler.go#L210

The code there only assumes the open source version of elasticsearch if it gets 400, however AWS elasticsearch return 401 when accessing /_xpack

This check seems overly restrictive. I agree with the above rant this starts to smell like bad faith.
It would make much more sense to assume that the proprietary paid feature is not available on any error and continue with that assumption in the function which is meant to autoconfigure the software

In any case, the problem can be resolved by explicitly disabling ILM in your config

setup.ilm.enabled: false

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic[bot]]

This issue doesn't have a Team:<team> label.

[timwsuqld]

Not stale

[TunerBuilder]

Any updates?
I am also encountering this issue with winlogbeat

[murali2904]

Hi All,

Even after disabling it I still get error in logs

Failed to connect to backoff(elasticsearch(https://vpc-xxxx.eu-west-1.es.amazonaws.com:443)): Connection marked as failed because the onConnect callback failed: request checking for ILM availability failed: 401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."}

output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:443}']
setup.ilm.enabled: false
xpack.monitoring.enabled: false
username: ""
password: ""

I am using OSS version of filebeat .
containers:
- name: filebeat
image: docker.elastic.co/beats/filebeat-oss:7.9.1
args: [
"-c", "/etc/filebeat.yml",
"-e",
]

Not able to find any more info on this issue.

Thanks
Murali

[murali2904]

Same issue with Metricbeat as well

2020-09-24T16:41:15.091Z INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.7.0
2020-09-24T16:41:22.748Z ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://vpc-xxxx.es.amazonaws.com:443)): Connection marked as failed because the onConnect callback failed: request checking for ILM availability failed: 401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."}

[digitaldavenyc]

Experiencing this is as well. I cannot believe this is unresolved since 2018!

[cunninghamba]

I was able to get around this issue by adding the following to my filebeat.yml config file:
setup.ilm.enable: false
I hope this helps you

[digitaldavenyc]

I figured out what my issue was, I had to disable the nginx module AND add setup.ilm.enable: false to the filebeat.yml

I've had it with the pissing contest between Elastic and AWS, this would not be an issue otherwise.

[cschyma]

Using filebeat-oss:7.9.2 the solution to me was to add the following lines to the filebeat.yml. setup.ilm.enable: false itself did not solve the problem.

#============================== X-Pack Monitoring ===============================
# filebeat can export internal metrics to a central Elasticsearch monitoring
# cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The
# reporting is disabled by default.

# Set to true to enable the monitoring reporter.
#monitoring.enabled: false
setup.ilm.enabled: false
setup.pack.security.enabled: false
setup.xpack.graph.enabled: false
setup.xpack.watcher.enabled: false
setup.xpack.monitoring.enabled: false
setup.xpack.reporting.enabled: false

Interestingly these lines are from a template configuration the AWS support provides but are not at the original documentation.

[wesklei]

I had this issue today and after setting all xpack options to false also had to run the setup like this

sudo filebeat setup --pipelines --index-management --dashboards -c /etc/filebeat/filebeat.yml

setup.ilm.enabled: false
setup.pack.security.enabled: false
setup.xpack.graph.enabled: false
setup.xpack.watcher.enabled: false
setup.xpack.monitoring.enabled: false
setup.xpack.reporting.enabled: false

[mdgreenwald]

If you're using the elastic helm chart for Filebeat with AWS Elasticsearch you may need something like this:

      filebeatConfig:
        filebeat.yml: |
          filebeat.inputs:
          - type: container
            paths:
              - /var/log/containers/*.log
            processors:
            - add_kubernetes_metadata:
                host: ${NODE_NAME}
                matchers:
                - logs_path:
                    logs_path: "/var/log/containers/"
          output.elasticsearch:
            host: '${NODE_NAME}'
            hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
          setup.ilm.enabled: false
          setup.pack.security.enabled: false
          setup.xpack.graph.enabled: false
          setup.xpack.watcher.enabled: false
          setup.xpack.monitoring.enabled: false
          setup.xpack.reporting.enabled: false

This was true as of version 7.9.3.

[hairongGao]

If you're using the elastic helm chart for Filebeat with AWS Elasticsearch you may need something like this:

      filebeatConfig:
        filebeat.yml: |
          filebeat.inputs:
          - type: container
            paths:
              - /var/log/containers/*.log
            processors:
            - add_kubernetes_metadata:
                host: ${NODE_NAME}
                matchers:
                - logs_path:
                    logs_path: "/var/log/containers/"
          output.elasticsearch:
            host: '${NODE_NAME}'
            hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
          setup.ilm.enabled: false
          setup.pack.security.enabled: false
          setup.xpack.graph.enabled: false
          setup.xpack.watcher.enabled: false
          setup.xpack.monitoring.enabled: false
          setup.xpack.reporting.enabled: false

This was true as of version 7.9.3.

Hi , running into the same issue , I tried your config with ES 7.8 but nu luck .
is there any extra step im missing ?