Zipkin: UI: Inject <base> into HTML configured via ENV var

[abesto]

As discussed in https://github.com/openzipkin/zipkin/issues/1731.

In situations where we run behind a reverse proxy with limited configurability (like the nginx ingress of Kubernetes), we can't rely on the reverse proxy injecting the required <base> tag to make the UI work on whatever URL path prefix it's assigned. The cleanest solution would be adding support to the Spring application serving the UI for adding the tag based on the presence and value of an ENV var.

I propose we call that ENV var ZIPKIN_UI_BASEPATH. Via @wdittmer, other applications all have custom names for it, there doesn't appear to be a standard:

For example, for Prometheus we need to configure:

• '--web.external-url=http://management.test.cbo:13370/prometheus'
• '--web.route-prefix=/'

For Grafana:

• name: GF_SERVER_ROOT_URL
  value: /grafana

For Kibana:

• name: SERVER_BASEPATH
  value: "/kibana"

[adriancole]

Works for me. If we do, we should update the webpack files to use the same
nomenclature or even have a comment with the variable chosen. That way grep
based troubleshooting is easier

https://github.com/openzipkin/zipkin/blob/master/zipkin-ui/webpack.config.js

On 25 Feb 2018 3:39 pm, "Zoltán Nagy" [e-mail] wrote:

As discussed in #1731 https://github.com/openzipkin/zipkin/issues/1731.

In situations where we run behind a reverse proxy with limited
configurability (like the nginx ingress of Kubernetes), we can't rely on
the reverse proxy injecting the required tag to make the UI work
on whatever URL path prefix it's assigned. The cleanest solution would be
adding support to the Spring application serving the UI for adding the tag
based on the presence and value of an ENV var.

I propose we call that ENV var ZIPKIN_UI_BASEPATH. Via @wdittmer
https://github.com/wdittmer, other applications all have custom names
for it, there doesn't appear to be a standard:

For example, for Prometheus we need to configure:

• '--web.external-url=http://management.test.cbo:13370/prometheus'
• '--web.route-prefix=/'

For Grafana:

• name: GF_SERVER_ROOT_URL
  value: /grafana

For Kibana:

• name: SERVER_BASEPATH
  value: "/kibana"

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin/issues/1930, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD61wZcciuaWUB49auCLKIpWNqpKRlRks5tYQ4-gaJpZM4SSM9S
.

[wdittmer]

Hey, how is it going with this?

[abesto]

Umm... Moderately well :D It's on my list of things to do, definitely on top of my Zipkin things to do. I'm hoping to get a (first?) implementation done this weekend.

[abesto]

Oh wow. I thought this'd be a quick Google search, adopting things a bit for the specifics of our case, open PR, all good. Nope, changing HTML is apparently not simple.

My first approach was extending the config.json we generate, and injecting the <base> tag from JS. The server side is clean, adding a new config value is trivial. BUT: of course the JS that'd load config.json won't know where to load the file from, because the <base> tag is not set. 🐔 and 🥚 , you see.

My second approach was to do exactly what was originally described: rewrite the HTML content. Apparently "the way" to do that is with ServletFilters, which... I tried to do, got something that _might_ work, but I don't see how I'd inject the new filter from the context of the autoconfigure module. Sounds like a horrible idea anyway.

Is the "right" way to just ... change https://github.com/openzipkin/zipkin/blob/master/zipkin-autoconfigure/ui/src/main/java/zipkin/autoconfigure/ui/ZipkinUiAutoConfiguration.java#L118 into a method returning ResponseEntity<String>, read the HTML file _like a peasant_, and do the HTML parsing / mutation, then pass a string to the ResponseEntity? (And of course cache the contents of the HTML file in memory, no need to hit the disk on each request)

... or pull in a full-fledged template system with proper Spring integration? That sounds overkill.

[adriancole]

if all that needs to change is index.html I'd be inclined to solve it that way

[wdittmer]

I upgraded tot openzipkin/zipkin:2.5.3 and added
- name: ZIPKIN_UI_BASEPATH
value: "/zipkin/zipkin"
to the deployment yaml and I have a working UI via the bastion via reverse proxy 👍

Thanks all!

[wdittmer]

A bit more information about the deployment:
We have a AWS VPC in which an EC2 instance runs that functions as the bastion service, to which we can setup our SSH tunnel. Then we have EC2 instance(s) with a kubernetes cluster. One of the pods is an nginx-ingress pod that reverse proxies the services in the cluster for multiple namespaces. One of the services is Zipkin in the monitoring namespace.

The ingress-nginx configuration:
```apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cbo-ingress-mgmt-elb
labels:
app: ingressservice-mgmt-elb
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: "/"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:

• host: management.test.cbo
  http:
  paths:
  
  
  
  • backend:
    
    serviceName: authentication-service
    
    servicePort: 8210
    
    path: /authentication
  
  
  • backend:
    
    serviceName: authorization-service
    
    servicePort: 8160
    
    path: /authorization
  
  
  • backend:
    
    serviceName: configuration-service
    
    servicePort: 8110
    
    path: /configuration
  
  

---

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cbo-ingress-mgmt-elb
namespace: monitoring
labels:
app: ingressservice-mgmt-elb
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: "/"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:

• host: management.test.cbo
  http:
  paths:
  
  
  
  • backend:
    
    serviceName: alertmanager
    
    servicePort: 9093
    
    path: /alertmanager
  
  
  • backend:
    
    serviceName: elasticsearch-proxy
    
    servicePort: 9200
    
    path: /elasticsearch
  
  
  • backend:
    
    serviceName: grafana
    
    servicePort: 3000
    
    path: /grafana
  
  
  • backend:
    
    serviceName: kibana
    
    servicePort: 5601
    
    path: /kibana
  
  
  • backend:
    
    serviceName: prometheus-monitoring
    
    servicePort: 9090
    
    path: /prometheus
  
  
  • backend:
    
    serviceName: zipkin-service
    
    servicePort: 9411
    
    path: /zipkin
  
  

And the Zipkin configuration:

apiVersion: v1
kind: Service
metadata:
namespace: monitoring
name: zipkin-service
spec:
ports:
- port: 9411
selector:
app: zipkin

---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
namespace: monitoring
name: zipkin
spec:
replicas: 1
selector:
matchLabels:
app: zipkin
template:
metadata:
labels:
app: zipkin
spec:
containers:
- name: zipkin
image: openzipkin/zipkin:2.5.3
env:
- name: STORAGE_TYPE
value: elasticsearch
- name: ES_HOSTS
value: http://elasticsearch-proxy:9200
- name: ES_DATE_SEPARATOR
value: .
- name: ZIPKIN_UI_BASEPATH
value: "/zipkin/zipkin"

The separate configurations result in one concatenated nginx.conf file.
The crucial thing is that we cannot change `nginx.ingress.kubernetes.io/rewrite-target: "/"` which is applied to all backends.

Now that this issue has been fixed it works.
If we set up the SSH tunnel with the bastion now, and curl to /zipkin, we get:

$ curl -vvv -L http://management.test.cbo:13370/zipkin

• Trying 127.0.0.1...
• TCP_NODELAY set
• Connected to management.test.cbo (127.0.0.1) port 13370 (#0)

GET /zipkin HTTP/1.1
Host: management.test.cbo:13370
User-Agent: curl/7.54.0
Accept: /

< HTTP/1.1 302 Found
< Date: Thu, 15 Mar 2018 10:32:57 GMT
< Location: ./zipkin/
< Server: nginx/1.13.8
< vary: origin
< X-Application-Context: zipkin-server:shared:9411
< Content-Length: 0
< Connection: keep-alive
<

• Connection #0 to host management.test.cbo left intact
• Issue another request to this URL: 'http://management.test.cbo:13370/zipkin/'
• Found bundle for host management.test.cbo: 0x7fdb19c21450 [can pipeline]
• Re-using existing connection! (#0) with host management.test.cbo
• Connected to management.test.cbo (127.0.0.1) port 13370 (#0)
  GET /zipkin/ HTTP/1.1
  Host: management.test.cbo:13370
  User-Agent: curl/7.54.0
  Accept: /

< HTTP/1.1 302 Found
< Date: Thu, 15 Mar 2018 10:32:57 GMT
< Location: ./zipkin/
< Server: nginx/1.13.8
< vary: origin
< X-Application-Context: zipkin-server:shared:9411
< Content-Length: 0
< Connection: keep-alive
<

• Connection #0 to host management.test.cbo left intact
• Issue another request to this URL: 'http://management.test.cbo:13370/zipkin/zipkin/'
• Found bundle for host management.test.cbo: 0x7fdb19c21450 [can pipeline]
• Re-using existing connection! (#0) with host management.test.cbo
• Connected to management.test.cbo (127.0.0.1) port 13370 (#0)
  GET /zipkin/zipkin/ HTTP/1.1
  Host: management.test.cbo:13370
  User-Agent: curl/7.54.0
  Accept: /

< HTTP/1.1 200 OK
< Cache-Control: max-age=60
< Content-Language: en-
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 15 Mar 2018 10:32:57 GMT
< Server: nginx/1.13.8
< vary: origin
< X-Application-Context: zipkin-server:shared:9411
< Content-Length: 1041
< Connection: keep-alive
<

ZipkinInvestigate system behavior

• Connection #0 to host management.test.cbo left intact
  ```