Prometheus: No token found

[GoWebProd]

What did you do?
I'm trying to parse metrics from Telegraf
What did you expect to see?
State up
What did you see instead? Under which circumstances?
State down and error: no token found
Environment
In docker prom/prometheus:latest

• Prometheus configuration file:

global:
  scrape_interval:     15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'default'
    static_configs:
      - targets: ['192.168.6.141:9273', '192.168.5.249:9273', '192.168.5.247:9273']

• Logs:

level=warn ts=2017-12-31T17:55:27.987211072Z caller=scrape.go:673 component="target manager" scrape_pool=default target=http://192.168.5.249:9273/metrics msg="append failed" err="no token found"

Metric: https://transfer.sh/15ceZy/metrics

[simonpasquier]

Checking your metrics file with promtool outputs the following error:

error while linting: text format parsing error in line 213: invalid label name for metric "docker_container_cpu_throttling_periods"

The offending line is:

docker_container_cpu_throttling_periods{08cd1981="",095e01a0="",37588df4="",49fc8029="",4fd2623f="",57e8c97c="",714ff794="",767a3995="",9a730256="",a089e9af="",a4476fd9="",annotation_io_kubernetes_container_hash="",annotation_io_kubernetes_container_restartCount="",annotation_io_kubernetes_container_terminationMessagePath="",annotation_io_kubernetes_container_terminationMessagePolicy="",annotation_io_kubernetes_pod_terminationGracePeriod="",annotation_io_rancher_cni_network="",annotation_io_rancher_cni_wait="",annotation_io_rancher_container_dns="",annotation_io_rancher_container_dnssearch="",annotation_io_rancher_container_ip="",annotation_io_rancher_container_mac_address="",annotation_io_rancher_container_name="",annotation_io_rancher_container_orchestration="",annotation_io_rancher_container_pull_image="",annotation_io_rancher_container_uuid="",annotation_io_rancher_scheduler_global="",annotation_io_rancher_service_deployment_unit="",annotation_io_rancher_service_launch_config="",annotation_io_rancher_stack_name="",annotation_io_rancher_stack_service_name="",annotation_kubernetes_io_config_seen="",annotation_kubernetes_io_config_source="",annotation_pod_alpha_kubernetes_io_init_containers="",annotation_pod_beta_kubernetes_io_init_containers="",c4d44529="",container_id="321ba5c7c273c511c3262730d165197f5f515f854ce4553e01cd169aa64e3f61",container_image="rancher/netes",container_name="r-kubernetes-support-kubelet-1-49bde3f8",container_version="v1.7.6-netes2-1",cpu="cpu-total",e91f139c="",eaee61ea="",engine_host="k8s-slave-1.novalocal",fce8991e="",host="default-stats-2-35f96a6e",io_kubernetes_container_logpath="",io_kubernetes_container_name="",io_kubernetes_docker_type="",io_kubernetes_pod_name="",io_kubernetes_pod_namespace="",io_kubernetes_pod_uid="",io_kubernetes_sandbox_id="",io_rancher_cni_link_mtu_overhead="",io_rancher_cni_network="",io_rancher_cni_wait="",io_rancher_container_agent_role="environment",io_rancher_container_agent_service_metadata="",io_rancher_container_agent_uuid="49bde3f8-2ea3-4a8b-9662-937d5877d794",io_rancher_container_create_agent="true",io_rancher_container_dns="false",io_rancher_container_dnssearch="",io_rancher_container_ip="",io_rancher_container_mac_address="",io_rancher_container_name="kubernetes-support-kubelet-1",io_rancher_container_orchestration="cattle",io_rancher_container_primary="",io_rancher_container_system="",io_rancher_container_uuid="49bde3f8-2ea3-4a8b-9662-937d5877d794",io_rancher_deployment_uuid="",io_rancher_network_arpsync="",io_rancher_network_cni_binary="",io_rancher_network_macsync="",io_rancher_revision="",io_rancher_scheduler_global="true",io_rancher_service_deployment_unit="afb1a395-725f-41ea-96de-41042ee4fcf2",io_rancher_service_launch_config="io.rancher.service.primary.launch.config",io_rancher_sidekicks="",io_rancher_stack_name="kubernetes-support",io_rancher_stack_service_name="kubernetes-support/kubelet"} 0

The first label name is 08cd1981 which doesn't follow the label specification.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.