Beats: functionbeat: can not run "Exiting: error unpacking config data: more than one namespace configured accessing 'output'"

[lifeofguenter]

Yes, I know beats can not have multiple outputs, but this is not the case:

functionbeat.provider.aws.deploy_bucket: "functionbeat-deploy"

functionbeat.provider.aws.functions:
  # Define the list of function availables, each function required to have a unique name.
  # Create a function that accepts events coming from cloudwatchlogs.
  - name: cloudwatch
    enabled: false
    type: cloudwatch_logs

    # Description of the method to help identify them when you run multiples functions.
    description: "lambda function for cloudwatch logs"

    # Concurrency, is the reserved number of instances for that function.
    # Default is unreserved.
    #
    # Note: There is a hard limit of 1000 functions of any kind per account.
    #concurrency: 5

    # The maximum memory allocated for this function, the configured size must be a factor of 64.
    # There is a hard limit of 3008MiB for each function. Default is 128MiB.
    #memory_size: 128MiB

    # Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
    #dead_letter_config.target_arn:

    # Optional fields that you can specify to add additional information to the
    # output. Fields can be scalar values, arrays, dictionaries, or any nested
    # combination of these.
    #fields:
    #  env: staging

    # List of cloudwatch log group registered to that function.
    triggers:
      - log_group_name: /aws/lambda/functionbeat-cloudwatch
        #filter_pattern: mylog_

    # Define custom processors for this function.
    #processors:
    #  - dissect:
    #      tokenizer: "%{key1} %{key2}"

output:
  redis:
    hosts: ["REDIS_HOST"]
    key: "filebeat"
    db: 0
    timeout: 10

logging.level: "${logging.level:error}"

yet, if I specify that exact config:

./functionbeat run -c ./functionbeat.yml

I get the following error:

Exiting: error unpacking config data: more than one namespace configured accessing 'output'

[jsoriano]

Umm, yes, actually it seems that the elasticsearch output is configured in any case, try to add this to your configuration as workaround:

output.elasticsearch.enabled: false

[jsoriano]

With output.elasticsearch.enabled: false it won't start, elasticsearch output seems to be required.

[monicasarbu]

Hi @lifeofguenter. At the moment Functionbeat only supports Elasticsearch as an output and we don't have plans to add other outputs to Functionbeat. Functionbeat serves the use case of collecting logs from different Cloud services and then sending them to Elasticsearch or Elastic Cloud.

[reversefold]

Not having the ability to ingest the logs through our already-set-up pipelines is quite the deal-breaker. I would assume that any log-beat would support all of the outputs that filebeat does so we can send them through kafka for queueing and process them through logstash for transformation and filtering.

[kettunen]

I finally managed to install Functionbeat listening SQS (that's a different story) but now this :(

@monicasarbu is it really so that Functionbeat won't support logstash output even that Functionbeat package contains reference to it in functionbeat.yml. Also in https://www.elastic.co/guide/en/beats/functionbeat/current/functionbeat-getting-started.html it says "Logstash (optional) for parsing and enhancing the data.". If I set "output.elasticsearch.enabled: false" I'll have error "Exiting: could not create the license manager: no elasticsearch output configuration found, verify your configuration".

[kylegoch]

Wait. If it _only_ supports Elasticsearch then why does the reference yml file provided with Functionbeats have a Logstash Output section

[L-F-Escobar]

Am I reading this correctly?? Functionbeat does not support sending logs from cloudwatch to logstash??? Why is there a section in functionbeat.yml for logstash? This cant be.... Is this correct?

[imvieira]

./filebeat -e -c /etc/filebeat/filebeat.yml -d "publish" Exiting: error unpacking config data: more than one namespace configured accessing 'output' (source:'/etc/filebeat/filebeat.yml')

```grep "output." /etc/filebeat/filebeat.yml

output.

The cloud.id setting overwrites the output.elasticsearch.hosts and

The cloud.auth setting overwrites the output.elasticsearch.username and

output.elasticsearch.password settings. The format is <user>:<pass>.

Configure what output to use when sending the data collected by the beat.

-------------------------- Elasticsearch output ------------------------------

output.elasticsearch:

----------------------------- Logstash output --------------------------------

output.logstash:

Elasticsearch output are accepted here as well.

output configuration, so if you have the Elasticsearch output configured such```

Only works with a single output.*...