Ingress-nginx: Enabling OpenTracing makes ingress controller unhealthy

Created on 8 Sep 2018  路  4Comments  路  Source: kubernetes/ingress-nginx

Is this a request for help? Yes I've asked for help in slack channel

What keywords did you search in NGINX Ingress controller issues before filing this one? OpenTracing, Jaeger


BUG REPORT

NGINX Ingress controller version: 0.17.1
Kubernetes version (use kubectl version): v1.10.6
Environment:

  • Cloud provider or hardware configuration: AWS
  • OS (e.g. from /etc/os-release): Ubuntu 16.04.5 LTS
  • Kernel (e.g. uname -a): 4.4.0-1065-aws

What happened:
I am trying to enable opentracing jaeger to my ingress controller, however the controller pod health check fails and it does not work. I am editing the controller configmap and then I delete one pod to test and tee latter health check fails.

configmap

enable-opentracing: "true"
jaeger-collector-host: jaeger-collector.tracing.svc.cluster.local
jaeger-collector-port: "14267"
jaeger-sampler-type: "probabilistic"
jaeger-sampler-param: "0.3"
jaeger-service-name: "ingress-nginx"

error on the logs

nginx_status.go:207] unexpected error obtaining nginx status info: unexpected error scraping nginx status page: unexpected error scraping nginx : Get http://0.0.0.0:18080/nginx_status: dial tcp 0.0.0.0:18080: connect: connection refused

The pod works fine after I disable opentracing, when I check the opentracing.json I see:

"reporter": {
    "localAgentHostPort": "jaeger-collector.tracing.svc.cluster.local:14267"
  }

What you expected to happen:
At least some error message describe any error about the opentracing config is properly set.

Thank you

Most helpful comment

@aledbf I was able to reproduce now and got Error: signal: segmentation fault (core dumped) the same as here https://github.com/kubernetes/ingress-nginx/issues/2856 so as you mentioned in that PR also it should be fixed in 0.19. Will reopen the issue in case the error persists, thank you

All 4 comments

nginx_status.go:207] unexpected error obtaining nginx status info: unexpected error scraping nginx status page: unexpected error scraping nginx : Get http://0.0.0.0:18080/nginx_status: dial tcp 0.0.0.0:18080: connect: connection refused

This is not an error. The reason for this message is the scraping of prometheus metrics start before the ingress controller. This is fixed in 0.19

thx for the quick reply! Interesting, it only happen when I enable the trace and the pod is never ready. I will update to the 0.19 version anyway

@aledbf I was able to reproduce now and got Error: signal: segmentation fault (core dumped) the same as here https://github.com/kubernetes/ingress-nginx/issues/2856 so as you mentioned in that PR also it should be fixed in 0.19. Will reopen the issue in case the error persists, thank you

As a reference, using zipkin config fields worked

Was this page helpful?
0 / 5 - 0 ratings