Kibana: start kibana error with 7.0.1

[boot-vue]

elasticsearch: 7.0.1
kibana: 7.0.1

curl http://localhost:9200/

{
  "name" : "localhost",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.0.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "e4efcb5",
    "build_date" : "2019-04-29T12:56:03.145736Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.7.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

kibana.yml

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name.  This is used for display purposes.
server.name: "kibana"

# The URLs of the Elasticsearch instances to use for all your queries.

elasticsearch.hosts: ["http://localhost:9200"]

# When this setting's value is true Kibana uses the hostname specified in the server.host
# setting. When the value of this setting is false, Kibana uses the hostname of the host
# that connects to this Kibana instance.
#elasticsearch.preserveHost: true

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
# kibana.index: ".kibana"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
#elasticsearch.startupTimeout: 5000

# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false

# Specifies the path where Kibana creates the process ID file.
#pid.file: /var/run/kibana.pid

# Enables you specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
#i18n.locale: "en"

error log

Kibana server is not ready yet

log   [09:48:45.081] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.082] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.082] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.082] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.082] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.083] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.083] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.083] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.083] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.083] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.084] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.084] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.084] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.084] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.085] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.085] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.085] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.511] [warning][browser-driver][reporting] Enabling the Chromium sandbox provides an additional layer of protection.
  log   [09:48:45.512] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml
  log   [09:48:45.528] [error][status][plugin:[email protected]] Status changed from uninitialized to red - [data] Elasticsearch cluster did not respond with license information.
  log   [09:48:45.572] [warning][reporting] The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports: Error: socket hang up
  log   [09:48:45.573] [warning][reporting] See Chromium's log output at "/home/battery/kibana/data/headless_shell-linux/chrome_debug.log"
 error  [09:49:15.082] [warning][process] UnhandledPromiseRejectionWarning: Error: Request Timeout after 30000ms
    at /home/battery/kibana/node_modules/elasticsearch/src/lib/transport.js:355:15
    at Timeout.<anonymous> (/home/battery/kibana/node_modules/elasticsearch/src/lib/transport.js:384:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
    at emitWarning (internal/process/promises.js:81:15)
    at emitPromiseRejectionWarnings (internal/process/promises.js:120:9)
    at process._tickCallback (internal/process/next_tick.js:69:34)
 error  [09:49:15.083] [warning][process] Error: Request Timeout after 30000ms
    at /home/battery/kibana/node_modules/elasticsearch/src/lib/transport.js:355:15
    at Timeout.<anonymous> (/home/battery/kibana/node_modules/elasticsearch/src/lib/transport.js:384:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  log   [09:49:15.515] [warning][reporting] Could not retrieve cluster settings, because of Request Timeout after 30000ms
  log   [09:49:15.545] [warning][task_manager] PollError Request Timeout after 30000ms
  log   [09:49:15.546] [warning][maps] Error scheduling telemetry task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!
  log   [09:49:15.547] [warning][telemetry] Error scheduling task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!

[jvalderrama]

I am experiment the same issue and I had not been able to fix the error so far

[elasticmachine]

Pinging @elastic/kibana-platform

[sidsinhad]

Is this a regression? I have the same issue. Simple config, but cannot have Kibana start up.

[sidsinhad]

Is this a regression? I have the same issue. Simple config, but cannot have Kibana start up.

both are version 7.0.1

[epixa]

Can you provide the output of GET http://localhost:9200/_xpack? Feel free to obfuscate the license.uid, though it should look like a uuid.

[boot-vue]

Can you provide the output of GET http://localhost:9200/_xpack? Feel free to obfuscate the license.uid, though it should look like a uuid.

curl http://localhost:9200/_xpack

{
  "build": {
    "hash": "e4efcb5",
    "date": "2019-04-29T12:57:49.344766Z"
  },
  "license": null,
  "features": {
    "ccr": {
      "description": "Cross Cluster Replication",
      "available": true,
      "enabled": true
    },
    "graph": {
      "description": "Graph Data Exploration for the Elastic Stack",
      "available": true,
      "enabled": true
    },
    "ilm": {
      "description": "Index lifecycle management for the Elastic Stack",
      "available": true,
      "enabled": true
    },
    "logstash": {
      "description": "Logstash management component for X-Pack",
      "available": true,
      "enabled": true
    },
    "ml": {
      "description": "Machine Learning for the Elastic Stack",
      "available": true,
      "enabled": true,
      "native_code_info": {
        "version": "7.0.1",
        "build_hash": "6a88928693d862"
      }
    },
    "monitoring": {
      "description": "Monitoring for the Elastic Stack",
      "available": true,
      "enabled": true
    },
    "rollup": {
      "description": "Time series pre-aggregation and rollup",
      "available": true,
      "enabled": true
    },
    "security": {
      "description": "Security for the Elastic Stack",
      "available": true,
      "enabled": false
    },
    "sql": {
      "description": "SQL access to Elasticsearch",
      "available": true,
      "enabled": true
    },
    "watcher": {
      "description": "Alerting, Notification and Automation for the Elastic Stack",
      "available": true,
      "enabled": true
    }
  },
  "tagline": "You know, for X"
}

the license is null, I can't use it for free? ?

[jvalderrama]

@boot-vue can you share your solution? I suppose that you set the x-pack to enable in your elasticsearch cluster. Am I wrong?

[epixa]

That is strange... a basic license should be automatically generated if you don't explicitly declare an alternative (paid) license. I'd also be interested in the steps you took to reconcile this.

@jvalderrama Is your license property null? @sidsinhad How about yours?

[boot-vue]

@boot-vue can you share your solution? I suppose that you set the x-pack to enable in your elasticsearch cluster. Am I wrong?

Sorry, I was wrong. yesterday I think I can start a trial license by this. https://www.elastic.co/guide/en/elasticsearch/reference/7.0/start-trial.html

but still wrong.

curl -X POST "localhost:9200/_license/start_trial?acknowledge=true"

{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}

@epixa No basic license is automatically generated. You can check the elastic linux64 zip file . https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-linux-x86_64.tar.gz

[sidsinhad]

Heres mine.
I also tried disabling xpack.security on kibana.yml and elasticsearch.yml. But same.
{ "tagline": "You know, for X", "features": { "watcher": { "enabled": true, "available": false, "description": "Alerting, Notification and Automation for the Elastic Stack" }, "sql": { "enabled": true, "available": true, "description": "SQL access to Elasticsearch" }, "ccr": { "enabled": true, "available": false, "description": "Cross Cluster Replication" }, "graph": { "enabled": true, "available": false, "description": "Graph Data Exploration for the Elastic Stack" }, "ilm": { "enabled": true, "available": true, "description": "Index lifecycle management for the Elastic Stack" }, "logstash": { "enabled": true, "available": false, "description": "Logstash management component for X-Pack" }, "ml": { "native_code_info": { "build_hash": "6a88928693d862", "version": "7.0.1" }, "enabled": true, "available": false, "description": "Machine Learning for the Elastic Stack" }, "monitoring": { "enabled": true, "available": true, "description": "Monitoring for the Elastic Stack" }, "rollup": { "enabled": true, "available": true, "description": "Time series pre-aggregation and rollup" }, "security": { "enabled": true, "available": false, "description": "Security for the Elastic Stack" } }, "license": { "status": "active", "mode": "basic", "type": "basic", "uid": "#####" }, "build": { "date": "2019-04-29T12:57:49.344766Z", "hash": "e4efcb5" } }

[sidsinhad]

heres the error I get:

log [05:22:46.610] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.611] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.611] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.611] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.612] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.612] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.612] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.612] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.612] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.613] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.613] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.613] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.613] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.614] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.614] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.614] [error][status][plugin:[email protected]] Status changed from yellow to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:46.998] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml log [05:22:47.029] [error][status][plugin:[email protected]] Status changed from uninitialized to red - [data] Elasticsearch cluster did not respond with license information. log [05:22:47.069] [warning][reporting] The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports: Error: Failed to launch chrome!

[jvalderrama]

@epixa my license property is null too for es v7.0.0. Also some of the commands like:

http://localhost:9200/_cluster/health
http://localhost:9200/_cat/nodes
http://localhost:9200/_cat/health

throws ans error like:

{ error: { root_cause: [ { type: "master_not_discovered_exception", reason: null } ], type: "master_not_discovered_exception", reason: null }, status: 503 }

But the command http://localhost:9200/ goes fine:

{ name: "localhost-master", cluster_name: "elastic-cluster-dual", cluster_uuid: "_na_", version: { number: "7.0.0", build_flavor: "default", build_type: "rpm", build_hash: "b7e28a7", build_date: "2019-04-05T22:55:32.697037Z", build_snapshot: false, lucene_version: "8.0.0", minimum_wire_compatibility_version: "6.7.0", minimum_index_compatibility_version: "6.0.0-beta1" }, tagline: "You Know, for Search" }

I downgrade to use 6.x es version and my cluster (elasticsearch+kibana) works fine (both with 6x version).

[epixa]

@sidsinhad @jvalderrama Are you both using linux-x86_64 for Elasticsearch as well?

[sidsinhad]

@epixa Yes. That is correct. x86_64

[jvalderrama]

@epixa x86_64, I think the problem is elasticsearch itself I was wondering solve this with the last release support given in https://github.com/elastic/ansible-elasticsearch/commits/master 4 days ago than support 7.x version, but I used this and the resulting cluster was the same:

http://localhost:9200/_xpack without license, this is null
http://localhost:9200/_cluster/health does not respond with the same error wrote in the comments before
http://localhost:9200/_cat/nodes does not respond with the same error wrote in the comments before
http://localhost:9200/_cat/health does not respond with the same error wrote in the comments before

http://localhost:9200/ goes fine:

{ name: "localhost-master", cluster_name: "cluster-dual-v7", cluster_uuid: "_na_", version: { number: "7.0.1", build_flavor: "default", build_type: "rpm", build_hash: "e4efcb5", build_date: "2019-04-29T12:56:03.145736Z", build_snapshot: false, lucene_version: "8.0.0", minimum_wire_compatibility_version: "6.7.0", minimum_index_compatibility_version: "6.0.0-beta1" }, tagline: "You Know, for Search" }

[jvalderrama]

I was able to solve the issue and now the entire stack is working in v7.0.1, the main problem was configuring the discovery and cluster formation setting in the ES cluster (elasticsearch.yml file). Nothing to do with kibana configuration or issues are related. The Kibana v7.01 was deployed and linked to ES succesfully.

Please refer here https://github.com/elastic/ansible-elasticsearch/issues/561 to see the detail.
@epixa Thanks a lot for your response and the support.

[hsnassaad]

@jvalderrama i did not understand can u explain more plz

[boot-vue]

I was able to solve the issue and now the entire stack is working in v7.0.1, the main problem was configuring the discovery and cluster formation setting in the ES cluster (elasticsearch.yml file). Nothing to do with kibana configuration or issues are related. The Kibana v7.01 was deployed and linked to ES succesfully.

Please refer here elastic/ansible-elasticsearch#561 to see the detail.
@epixa Thanks a lot for your response and the support.

yeah. cluster.initial_master_nodes shoule be config.

discovery.seed_hosts: ["127.0.0.1", "[::1]"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
#
# For more information, consult the discovery and cluster formation module documentation.

[epixa]

I’m glad you were able to sort this out! That’s a tricky problem to track down...

[sidsinhad]

Brilliant! that worked for me too boot-vue! Thanks all!

[markgg7788]

I'm use 7.3.0，you must config elasticsearch.yml
set
node.name='node-1'
and
cluster.initial_master_nodes: ["node-1"]
then restart es!
Enjoin!

[anmobor]

I had the same problem because I was using elasticsearch 7.2, in the moment I updated the version to 7.4 kibana started working perfectly

[elinaldosoft]

In my case, I resolved as follows, added this config in ElasticSearch.

Elasticsearch v7.4.1:

sudo vi /etc/elasticsearch/elasticsearch.yml

• node.name: node-1
• discovery.seed_hosts: ["127.0.0.1", "[::1]"]
• cluster.initial_master_nodes: ["node-1"]

Kibana v7.4.1:

sudo vi /etc/kibana/kibana.yml

• elasticsearch.hosts: ["http://host-external:9200"]

[leonardiwagner]

This issue is due to the ElasticSearch x86_64 distribution.

• I was using the .deb amd64 without any requirement to set discovery.seed_hosts and cluster.initial_master_nodes and Kibana was working well.
• Today I had to use the .rpm x86_64 and I got the same problem and setting these properties f fixed that.

[ethical20]

In my case, I resolved as follows, added this config in ElasticSearch.

Elasticsearch v7.4.1:

sudo vi /etc/elasticsearch/elasticsearch.yml

• node.name: node-1
• discovery.seed_hosts: ["127.0.0.1", "[::1]"]
• cluster.initial_master_nodes: ["node-1"]

Kibana v7.4.1:

sudo vi /etc/kibana/kibana.yml

• elasticsearch.hosts: ["http://host-external:9200"]

Many thanks for this!

I have Kibana V7.6.2, with the same issue and this fixed it.

You saved my day.

[elinaldosoft]

In my case, I resolved as follows, added this config in ElasticSearch.

Elasticsearch v7.4.1:

sudo vi /etc/elasticsearch/elasticsearch.yml

• node.name: node-1
• discovery.seed_hosts: ["127.0.0.1", "[::1]"]
• cluster.initial_master_nodes: ["node-1"]

Kibana v7.4.1:

sudo vi /etc/kibana/kibana.yml

• elasticsearch.hosts: ["http://host-external:9200"]

Many thanks for this!

I have Kibana V7.6.2, with the same issue and this fixed it.

You saved my day.

I'm glad!

[videotronix]

try this it worked for me https://www.youtube.com/watch?v=7gBDQLkwgHc when elastic search was set for red status after 15min or so on boot.