Beats: Update autodiscover docs after adding `container` input

[nevmerzhitsky]

• Version: Filebeat 7.2
• Operating System: Docker
• Discuss Forum URL: no

@exekias are you sure that the implementation of #12162 is finished? I try to use container as input for autodiscover Docker provider but the setup is not working:

filebeat:
  autodiscover.providers:
    - type: docker
      templates:
        - condition.contains:
            docker.container.image: nginx
          config:
            - module: nginx
              access:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              error:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
  inputs:
    - type: container
      paths:
        - '/var/lib/docker/containers/*/*.log'

But in documentation docker input is used for autodiscover Docker provider still and it works:

filebeat:
  autodiscover.providers:
    - type: docker
      templates:
        - condition.contains:
            docker.container.image: nginx
          config:
            - module: nginx
              access:
                input:
                  type: docker
                  containers.ids:
                    - "${data.docker.container.id}"
              error:
                input:
                  type: docker
                  containers.ids:
                    - "${data.docker.container.id}"
  inputs:
    - type: container
      paths:
        - '/var/lib/docker/containers/*/*.log'

but in this case I see deprecation warnings in logs:
{"level":"warn","timestamp":"2019-07-04T18:22:45.776Z","logger":"cfgwarn","caller":"docker/input.go:49","message":"DEPRECATED: 'docker' input deprecated. Use 'container' input instead. Will be removed in version: 8.0.0"}

Full config:

name: agent-docker@${MANAGER_HOSTNAME}
tags: ${BEAT_TAGS:[]}
processors:
  - add_cloud_metadata:
  - add_docker_metadata:
  - add_tags:
      tags: ${BEAT_ENV:unspecified}
      target: "environment"
filebeat:
  config:
    modules:
      path: ${path.config}/modules.d/*.yml
      reload:
        enabled: false
  autodiscover.providers:
    - type: docker
      templates:
        # Redis services
        - condition.or:
            - equals.docker.container.labels.com.docker.swarm.service.name: depth-history_hot-db
          config:
            - module: redis
              log:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              slowlog:
                var.hosts: ['${data.host}:${data.port}']
        # PostgreSQL services
        - condition.or:
            - equals.docker.container.labels.com.docker.swarm.service.name: trade-account_postgres
          config:
            - module: postgresql
              log:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
        # Nginx services
        - condition.contains:
            docker.container.image: nginx
          config:
            - module: nginx
              access:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
#                  type: docker
#                  containers.ids:
#                    - "${data.docker.container.id}"
              error:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
#                  type: docker
#                  containers.ids:
#                    - "${data.docker.container.id}"
        # ElasticSearch services
        - condition.contains:
            docker.container.image: elasticsearch
          config:
            - module: elasticsearch
              server:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              gc:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              audit:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              slowlog:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
              deprecation:
                input:
                  type: container
                  paths:
                    - '/var/lib/docker/containers/*/*.log'
  inputs:
    - type: container
      paths:
        - '/var/lib/docker/containers/*/*.log'
  shutdown_timeout: 5s
output:
  elasticsearch:
    hosts: ${ELASTIC_HOSTS}
    username: elastic
    # Read a value from filebeat.keystore
    password: ${ELASTIC_PASSWORD}
    ssl:
      verification_mode: none
      certificate_authorities:
        - /usr/share/filebeat/certs/ca/ca.crt
    compression_level: 1
setup:
  kibana:
    host: ${KIBANA_HOST}
    username: ${output.elasticsearch.username}
    password: ${output.elasticsearch.password}
    ssl:
      verification_mode: none
      certificate_authorities:
        - /usr/share/filebeat/certs/ca/ca.crt
  ilm:
    policy_name: filebeat
xpack:
  monitoring:
    enabled: true
logging:
  json: true
  metrics:
    enabled: true

(no one of these templates of autodiscover are work)

[exekias]

Hi,

You are right, we should update docs to make use of the new input. I saw you are using both static input and autodiscover at the same time, that's not a good idea as they will try to gather the same logs. The other issue is that you are using a wildcard (*/*) in paths, making it gather logs from all containers and not only the one you are interested on.

I would expect this to work:

filebeat:
  autodiscover.providers:
    - type: docker
      templates:
        - condition.contains:
            docker.container.image: nginx
          config:
            - module: nginx
              access:
                input:
                  type: container
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*.log"
              error:
                input:
                  type: container
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*.log"

        # fallback config when no condition match:
        - config:
            - type: container
              paths:
                - "/var/lib/docker/containers/${data.docker.container.id}/*.log"

[nevmerzhitsky]

@exekias damn sorry! I'm absolutely didn't get that I can't mix static and autodiscover cases 😨

[exekias]

Let's use this issue to clarify docs on this regard :+1:

[nevmerzhitsky]

Just for clarification: proposed config fix the issue with container input. Even more, some of my old warnings about input factoring disappeared from logs!
Thus the issue is only about update documentation now.

[dnmahendra]

My Config is as follows -

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    setup.dashboards.enabled: true
    setup.template.enabled: true
    setup.kibana.host: "192.168.82.162:5601"

    setup.template.settings:
      index.number_of_shards: 1
    setup.template.overwrite: true
    filebeat.overwrite_pipelines: true

    filebeat.config:
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false

    filebeat.autodiscover:
      providers:
        - type: kubernetes
          templates:
            - condition.and:
                - contains:
                    kubernetes.container.name: istio-proxy
                - contains:
                    kubernetes.labels.app: swift-client
                - contains:
                    kubernetes.labels.app: swift-server
              config:
                input:
                  type: container
                  tail_files: true
                  containers:
                    stream: stdout
                    ids:
                      - ${data.kubernetes.container.id}
            - condition.and:
                - contains:
                    kubernetes.container.name: swift-client
                - contains:
                    kubernetes.labels.app: swift-client
              config:
                - type: docker
                  multiline.pattern: '^WARN|^INFO|^DEBUG|^ERROR|^Error|^TypeError'
                  multiline.negate: true
                  multiline.match: after
                  containers:
                    ids:
                      - ${data.kubernetes.container.id}

    processors:
      - add_cloud_metadata: ~
      - add_kubernetes_metadata:
          in_cluster: true

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']

In the following condition if I change the type to container nothing is working. No logs are outputted but type docker is working without problems. I am using filebeat 7.2.0

            - condition.and:
                - contains:
                    kubernetes.container.name: swift-client
                - contains:
                    kubernetes.labels.app: swift-client
              config:
                - type: docker
                  multiline.pattern: '^WARN|^INFO|^DEBUG|^ERROR|^Error|^TypeError'
                  multiline.negate: true
                  multiline.match: after
                  containers:
                    ids:
                      - ${data.kubernetes.container.id}

[exekias]

ey @dnmahendra, container input has slightly different parameters, check https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-container.html for more details. I have opened https://github.com/elastic/beats/pull/12864 to update examples in docs