Kibana: Console: Client request error: self signed certificate in certificate chain

[nellicus]

Unable to use Console when using Own Root CA

Kibana version

Version 5.0.0-alpha2
Build   12279
Commit SHA  6d7bb14

Kibana config

abonuccelli@w530 /opt/elk/PROD/kibana $ egrep '^[^#]' kibana-5.0.0-alpha2-linux-x64/config/kibana.yml 
elasticsearch.url: "https://192.168.1.105:9200"
elasticsearch.username: "kibana"
elasticsearch.password: "xxxxxx"
elasticsearch.ssl.ca: /opt/elk/PROD/FS/kibana/cacert.pem
elasticsearch.ssl.verify: true
logging.verbose: true
xpack.security.encryptionKey: "xxxxxx"
xpack.security.skipSslCheck: true
xpack.security.useUnsafeSessions: true
xpack.security.kibana.password: "xxxxxx"

Root CA cert

generated as per docs

abonuccelli@w530 /opt/elk/PROD $ keytool -printcert -file /opt/elk/PROD/FS/secure/cacert.pem 
Owner: CN="Elasticsearch Test RootCA"
Issuer: CN="Elasticsearch Test RootCA"
Serial number: 9560e13f57abd61b
Valid from: Sun Nov 01 23:54:44 CET 2015 until: Mon Oct 31 23:54:44 CET 2016
Certificate fingerprints:
     MD5:  4C:43:3A:3E:DB:3D:B3:65:79:32:39:98:04:51:5F:74
     SHA1: 88:0D:4E:93:6C:8E:5C:6A:23:56:CD:8F:58:D6:EA:63:47:8E:86:69
     SHA256: 56:FD:07:00:1C:55:AD:16:B4:E2:8C:35:5F:9F:6B:23:AD:7B:9F:ED:61:A5:07:9D:1A:80:B3:53:7A:77:53:C1
     Signature algorithm name: SHA256withRSA
     Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: F6 74 D3 93 1B 3F 3D 63   E4 38 F4 84 1A 37 05 39  .t...?=c.8...7.9
0010: 10 26 03 52                                        .&.R
]
[CN="Elasticsearch Test RootCA"]
SerialNumber: [    9560e13f 57abd61b]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F6 74 D3 93 1B 3F 3D 63   E4 38 F4 84 1A 37 05 39  .t...?=c.8...7.9
0010: 10 26 03 52                                        .&.R
]
]

Request from curl

abonuccelli@w530 /opt/elk/PROD $ curl -XGET https://192.168.1.105:9200 -u elastic:xxxxxx --cacert /opt/elk/PROD/FS/secure/cacert.pem 
{
  "name" : "node1",
  "cluster_name" : "tony_prod",
  "version" : {
    "number" : "5.0.0-alpha2",
    "build_hash" : "e3126df",
    "build_date" : "2016-04-26T12:08:58.960Z",
    "build_snapshot" : false,
    "lucene_version" : "6.0.0"
  },
  "tagline" : "You Know, for Search"
}

Console browser output and debug log when requesting

Error connecting to 'https://192.168.1.105:9200/_search':

Client request error: self signed certificate in certificate chain

Kibana verbose logs

respons [14:52:35.241]  GET /api/console/proxy?uri=https%3A%2F%2F192.168.1.105%3A9200%2F_mapping&_=1462366354318 502 121ms - 9.0B
respons [14:52:35.244]  GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_1_0 200 112ms - 9.0B
respons [14:52:35.253]  GET /api/console/proxy?uri=https%3A%2F%2F192.168.1.105%3A9200%2F_aliases&_=1462366354319 502 123ms - 9.0B

Dev tools

request

Request URL:http://192.168.1.105:5601/api/console/proxy?uri=https%3A%2F%2F192.168.1.105%3A9200%2F_search
Request Method:POST
Status Code:502 Bad Gateway
Remote Address:192.168.1.105:5601

response

HTTP/1.1 502 Bad Gateway
content-type: text/plain; charset=utf-8
kbn-name: kibana
kbn-version: 5.0.0-alpha2
set-cookie: sid=Fe26.2**bc4cd89edc19ac8d28324b3edab0b6ccb055fc5a03ba4350fc91b88b02f0f177*KmNc6hYVhZzEn7T0Hc5zgw*34A1ifapomcWjEf2nu3-TSOJIm4nqwlEyVxpSuenZdRR6wzmzCNXb4-FJR1w4tqmyBtkvXLyFr2lCYx3IEnX2LA1_P_vIqyerUfSpp2ifrM**da1ae4f15be2286d72a957f0ec63c80a81d52fd0119deee45f3aa0bee603c661*pZD3kzoTLwCRAllDqhnomBCnNGWYgg001fQox_kC-x8; HttpOnly; Path=/
cache-control: no-cache
content-encoding: gzip
vary: accept-encoding
Date: Wed, 04 May 2016 12:53:49 GMT
Connection: keep-alive
Transfer-Encoding: chunked

Query String Parameters

uri=https%3A%2F%2F192.168.1.105%3A9200%2F_search

[ycombinator]

This is a guess but it looks like the proxy being uses by Console will need to set rejectUnauthorized to false to make this error go away. See https://github.com/hapijs/h2o2#options.

[Bargs]

I'm not super familiar with Console, but I believe this can be solved with sense.proxyConfig in kibana.yml: https://www.elastic.co/guide/en/sense/current/installing.html#configuration

[ycombinator]

@Bargs yeah, the config is the better way to go here.

[nellicus]

@Bargs @ycombinator thanks for your comments

I've tried both

from https://www.elastic.co/guide/en/sense/current/installing.html

sense.proxyConfig:
 - match:
    host: "*"
    port: "{9200..9202}"
   ssl:
    ca: "/opt/elk/PROD/FS/secure/cacert.pem"

and got

  log   [10:04:54.518] [warning][config] Settings for "sense" were not applied, check for spelling errors and ensure the plugin is loaded.

and also from https://www.elastic.co/guide/en/kibana/5.0/securing_console.html

sense.proxyConfig:
  - ^https?://(localhost|w530|w530\.nellicus\.com|192\.168\.1\.105\[::0\].*)$

and got same result

  log   [10:08:40.369] [warning][config] Settings for "sense" were not applied, check for spelling errors and ensure the plugin is loaded.

disabling validation (rejectUnauthorized: false) is not a solution, of course we do want to enforce cert chain validation, not disable it.

cacert.pem in use is a RootCA certificate generated as per our Shield documentation, there is no-self-signed certs in this configuration.

[nellicus]

also setting

elasticsearch.ssl.verify: false

doesn't help

[Bargs]

@nellicus apologies, it looks like the sense documentation hasn't been updated for 5.0 yet. Anywhere you see sense in kibana.yml config it should be updated to console.

So could you try

console.proxyConfig:
 - match:
    host: "*"
    port: "{9200..9202}"

   ssl:
    ca: "/opt/elk/PROD/FS/secure/cacert.pem"

And let us know how that works?

[nellicus]

@Bargs yes that did the trick! thank you

[gmoskovicz]

@Bargs did we finally added this documentation to the x-pack documentation? i do see this in the old sense configuration, not sure if we ended up exporting this to the x-pack one.

[Bargs]

It doesn't look like it, I would expect it to be in the kibana docs now that console is part of kibana and open source. Either here or here but it seems to be missing.