Kibana: Kibana 4.2.0 cannot be proxied under a directory any more

[BernhardBln]

We were playing around with Kibana 4.1 which we proxied under /kibana/* - that worked quite well.

Now, the Kibana app is under /app/kibana and tries to load resources via their absolute url, e.g. /bundles/kibana.bundle.js.

Could that be referenced relatively? Then proxying would still work...

[BernhardBln]

Probably a double of #5171? Just saw that now.

[PSiAU]

+1

[nxhack]

+1

[benediktarnold]

+1

[ave19]

+1

[jcneisius]

+1

[damm]

This is fairly easy with nginx.

    location ~ (/app/kibana|/bundles/|/kibana4|/status|/plugins) {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        rewrite /kibana4/(.*)$ /$1 break;
    }

[damm]

I admit it would be nice to set the route so Kibana 4 knows where it's mounted; but sometimes that's unrealistic. Kibana 4 is designed as a server; with that they are expecting to control the routes. We are deploying it where it is not dedicated.

I kind of expect the rough edges.

[BernhardBln]

Thanks for the workaround; I just don't really like it for my use case as we have another web app running under / and I wouldn't want them to interfere. With the previous version this was possible, so it's a regression, in my opinion :)

[damm]

@BernhardBln actually in this case I originally had it /kibana and now forced it to be at /app/kibana

It's kind of rough when you have / owned by something else and you deploy something like ES or Kibana and you want to be able to reverse proxy easily.

Reverse Proxy and ES Plugins is still a problem for me.

[kouki-o]

+1

[kouki-o]

worked with apache2. (please rewrite host 'proxy-dest.exaple.com')
But this config is simple way.

<Location /bundles>
  ProxyPass http://proxy-dest.exaple.com:5601/bundles
  ProxyPassReverse http://proxy-dest.example.com:5601/bundles
</Location>

<Location /app/kibana>
  ProxyPass http://proxy-dest.example.com:5601/app/kibana
  ProxyPassReverse http://proxy-dest.example.com:5601/app/kibana
</Location>

access to 'kibana.example.com' by browser.
works in as below. (if host 'kibana.example.com' running a proxy)

http://kibana.example.com/ -> DocumentRoot
http://kibana.example.com/(any) -> DocumentRoot
http://kibana.example.com/app/kibana -> Proxy to kibana
http://kibana.example.com/bundle -> Proxy to Kibana

Let me know if you know of any better way.

[PSiAU]

Using a vhost, all you really need are two lines in your vhost config. You shouldn't need to worry about /bundles/ and /app/kibana.

ProxyPass / http://127.0.0.1:5601/
ProxyPassReverse / http://127.0.0.1:5601/

And then going to http://kibana.example.com/ should just work. That's the way i'm having to do it at the moment until this directory issue is fixed.

[damm]

@PSiAU the problem is when Kibana is not the only thing deployed.

I have Sensu, Grafana, Kibana and a few other plaintext locations on a single domain/path. Thankfully the locations picked are unused by anything else :)

[PSiAU]

I hear you @damm. I do exactly the same thing and run everything on a single domain so I don't need separate SSL certs for each app's domain. I presumed (maybe incorrectly) that @kouki-o was dedicating kibana.domainname as a short term solution to the directory problem, as i've had to do.
So i've got everything running under the main domain, and just kibana split off onto it's own separate domain until this bug is resolved. My 2 line vhost config was for this scenario only.
Apologies if I misunderstood.

[pselibas]

+1

[yodog]

same problem here

[ThStock]

+1

[yuecen]

+1

[schmorgs]

I also had this problem, using the same Apache http server to proxy both Kibana 4 and Elasticsearch, but with the help of this thread and lots of digging, I got it working as follows :

  ProxyRequests Off

  ProxyPass /app/kibana http://127.0.0.1:5601/app/kibana
  ProxyPassReverse /app/kibana http://127.0.0.1:5601/app/kibana

  ProxyPass /bundles http://127.0.0.1:5601/bundles
  ProxyPassReverse /bundles http://127.0.0.1:5601/bundles

  ProxyPass /elasticsearch http://127.0.0.1:9200
  ProxyPassReverse /elasticsearch http://127.0.0.1:9200

It does mean I need to point my browser to "http://myserver/app/kibana" but I'm sure with a bit more playing I could fix that piece too.

[dfr0]

+1

[damm]

@schmorgs it's hardcoded in Kibana so you'd have to handle it with rewrites... :/

[fbejarano]

+1

[schmorgs]

@damm I don't have any rewrites in my config and it works just fine.

[dfr0]

My problems continue, i configured a reverse proxy for kibana on apache but i have another error.

My proxy configuration:

ProxyPass http://*_:5601/bundles
ProxyPassReverse http://__:5601/bundles
Order Deny,Allow
Deny from all
Allow from all
AuthType Basic
AuthUserFile /etc/_/htpasswd.controlusers
Require valid-user
AuthName "* log Server"

ProxyPass http://*_:5601/app/kibana
ProxyPassReverse http://__:5601/app/kibana
Order Deny,Allow
Deny from all
Allow from all
AuthType Basic
AuthUserFile /etc/__/htpasswd.controlusers
Require valid-user
AuthName "_* log Server"

Order Deny,Allow
Deny from all
Allow from **
AuthType Basic
AuthName "OKN ELS Server"
AuthUserFile /etc/okn/htpasswd.controlusers
Require valid-user

Only happens with the kibana 4.2 version due to the new routes

[Xel-Naga]

@dfr0
@schmorgs 's workground is good enough for me
and i think it should be working for you too
well , i made a little bit modification:

ProxyRequests Off

ProxyPass /app/kibana http://localhost:5601/app/kibana
ProxyPassReverse /app/kibana http://localhost:5601/app/kibana

ProxyPass /bundles http://localhost:5601/bundles
ProxyPassReverse /bundles http://localhost:5601/bundles

ProxyPass /elasticsearch http://localhost:9200
ProxyPassReverse /elasticsearch http://localhost:9200

[dfr0]

Sure, I don't know that elasticsearch needs your own proxy or location, i discovered that some hours ago with the chrome development tools, f*

But the status options don't work, do you've the same problem with this???

Thanx for everything.

[Xel-Naga]

@dfr0 maybe you should start a new thread for "status options" issue, i have no idea what's that about

[dfr0]

thx for the quick response.

Here show you screenshots from my testing virtual machine , on this machine kibana work correctly "almost :)" but on the cloud don't work the status app.

I'll to create a new issue for that, thanx so much

[schmorgs]

@dfr0 You'll need to more entries in your proxy config to cater for the status page as well :

ProxyPass /status http://127.0.0.1:5601/status
ProxyPassReverse /status http://127.0.0.1:5601/status
ProxyPass /api/status http://127.0.0.1:5601/api/status
ProxyPassReverse /api/status http://127.0.0.1:5601/api/status

If you use the Chrome's "Inspect Element" option and select the "Console" tab, you can see the URL's that Kibana is trying to connect to, and if one of the URL prefixes isn't in your Apache reverse proxy setting, just add it and bounce Apache.

[dfaropennetwork]

:D Thx so much, i don't saw the console tab, i only are seeing on the network tab.

[dfaropennetwork]

Work correctly thx.

[yodog]

my nginx balancer stopped working with kibana.
everything else works as expected.
this is the config that worked before the update:

/etc/nginx/conf.d/elastic-80.server.conf

upstream web {

    server elk-node-01;
    server elk-node-02;
    server elk-node-03;

    keepalive 5;
}

server {

    listen 80;

    location / {

        proxy_pass http://web;
        proxy_http_version 1.1;
        proxy_set_header Connection "Keep-Alive";
        proxy_set_header Proxy-Connection "Keep-Alive";
    }

    access_log /var/log/nginx/access-80.log;
    error_log  /var/log/nginx/error-80.log;
}

now it doesn't anymore.

[guydavis]

Unfortunately, I can confirm that server.basePath: "/kibana" in kibana.yml doesn't work as expected. Visiting http://myserver.com/kibana/ results in an HTTP 200 with only the JSON error text about 404 Not Found. This is with Kibana 4.3.1 (from official tarball) on CentOS 7.2. If I don't set server.basePath, Kibana works as expected. It just takes over the root context of my HAProxy however. This remains a show-stopper for our proxied, multi-app deployment.

[spalger]

@guydavis it sounds like your proxy is not removing /kibana from the request before it sends it to Kibana. It must to that in order for the feature to work properly.

[yodog]

@guydavis , i got it to work again. explained my config here: https://discuss.elastic.co/t/4-3-0-how-to-configure-your-nginx-balancer-and-apache-reverse-proxy/37351/2?u=yodog

[spartman]

I'm facing the same issue. I have a nginx setup for multiple app. I'm not able to make kibana work under a subfolder /kibana

What is the configuration for nginx when we use server.Path?

[ksou1]

I was able to get both kibana 4.3.1 and ES plugins working through Nginx, using the detail @damm posted on 10/30/2015, and a separate location directive to handle the plugins. I had to add "elasticsearch" as a path in the location directive for Kibana, as my logs revealed requests directed to that path, and without it i would not get any information in my Kibana screens). The second directive handles the ES plugins i use (particularly "hq"). The "kopf" plugin doesnt connect unless I have the "/" path in the directive. Here's nginx location directives:

location ~ (/app/kibana|/bundles/|/kibana4|/status|/plugins|/elasticsearch) {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
rewrite /kibana4/(.*)$ /$1 break;
access_log /opt/nginx-1.8.0/logs/kibana-access.log;
error_log /opt/nginx-1.8.0/logs/kibana-error.log;
}

location ~ (/|/_plugin) {
proxy_pass http://localhost:9217;
access_log /opt/nginx-1.8.0/logs/esplugin-access.log;
error_log /opt/nginx-1.8.0/logs/esplugin-error.log;
}

I listen on port 443 and access Kibana and ES plugins as:
Kibana: https://./app/kibana
ES Plugins: https://./_plugin/

Hope this helps somebody, cuz its a bear trying to get all this stuff working again every time Elastic decides to change direction.

[yodog]

@ksou1 you dont need separate locations if you follow the instructions posted here: https://discuss.elastic.co/t/4-3-0-how-to-configure-your-nginx-balancer-and-apache-reverse-proxy/37351/2?u=yodog

[dalenewman]

Setting _kibana.yml's_ server.basePath and making sure my Orchard CMS proxy forwards custom HTTP headers did the trick for me. I was getting a 400 (bad request) on the _elasticsearch/_mget_ posts until I passed the kbn_version header.

[t3chn0m4g3]

I am on Kibana 4.5.3, when setting server.basePath to /kibana you can use a modified version of @damm's NGINX location config:

location /kibana/ {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        rewrite /kibana/(.*)$ /$1 break;
    }

As already pointed out I'm also guessing the server.basePath setting does not work as expected, the rewrite should not be necessary. However the modified config works perfectly fine for me and does not break my other locations.

[pdostal]

I tried everything from this thread - but only @ksou1's solution worked for me.
I also added /kibana redirection to /app/kibana as a shortcut.

[linzhaolover]

location ~ (/app/kibana|/bundles/|/kibana4|/status|/plugins|/elasticsearch) {
    proxy_pass http://localhost:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    rewrite /kibana4/(.*)$ /$1 break;
}

I add /elasticsearch , it's ok

[FlorianHeigl]

@linzhaolover can you explain the reason for the 'Upgrade $http_upgrade' bit?

[cortopy]

@damm's suggestion nearly worked for me. I just needed to add /elasticsearch to the regex too. I'm also using http2, so this works better for me:

    location ~ (/app/kibana|/bundles/|/kibana4|/status|/plugins|/elasticsearch) {
        proxy_pass              http://10.23.251.150:5601;
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_set_header        X-Forwarded-Host $http_host;
    }

[saschalalala]

My updated location for kibana 5.2.2 is this:

location ~ (/app/kibana|/bundles|/es_admin|/plugins|/api|/ui|/elasticsearch) {
    proxy_pass              http://10.23.251.150:5601;
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        X-Forwarded-Host $http_host;    
}

I didn't find any issues so far so this seems to be enough for the current version. Or could it be that I am missing something?

[avoidik]

most of all solutions provided here isn't taking into account cases like additional services on a same server behind reverse proxy and thus doesn't work properly

for example solution with /status in location will cause issues to a Nexus, because its URL contains links like this:

https://fqdn/nexus/service/local/outreach/status?_dc=0

as you might see regular expression will match this case

another problem with /elasticsearch and also with Nexus, if you will upload elasticsearch artifact into Nexus, then you will not be able to download it, because URL will be matched by the regular expression

[chriswue]

@t3chn0m4g3 You don't need the re-write rule if you set proxy_pass correctly with a trailing slash:

location /kibana/ {
    proxy_pass http://localhost:5601/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
}

The difference is explained here: https://www.nginx.com/resources/admin-guide/reverse-proxy/