Kibana: Use multiple Kibana instances for api_integration suites
@elastic/kibana-stack-services would like to test that the Task Manager API works well when there are multiple Kibana servers connected to the same Elasticsearch cluster, and I think that's good coverage to get for all of our api_integration tests.
I'd like to propose that we start two Kibana servers, both connected to a single Elasticsearch node, and have the api_integration tests round-robin requests between the two nodes. This way all tests are verifying that they don't rely on any local state automatically. Thoughts?
spalger
All 8 comments
Pinging @elastic/kibana-operations
elasticmachine
on 23 Sep 2019
How would the round-robin work?
wayneseymour
on 23 Sep 2019
Does the task manager API have test coverage for handling this?
tylersmalley
on 23 Sep 2019
Instead of adding the complexity of another Kibana instance to all integration tests, how about take an approach similar to what we did for SAML and have a separate config? We will have to add support for specifying the number of Kibana instances.
tylersmalley
on 23 Sep 2019
How would the round-robin work?
We currently use a service for making requests to the Kibana server that knows the Kibana URL to use, the idea would be that we just rotate between the two Kibana instances for every other request.
spalger
on 23 Sep 2019
Does the task manager API have test coverage for handling this?
Nope, I'm looking into enabling it with this effort.
spalger
on 23 Sep 2019
Instead of adding the complexity of another Kibana instance to all integration tests, how about take an approach similar to what we did for SAML and have a separate config? We will have to add support for specifying the number of Kibana instances.
Hmm, I feel like the complexity comes from adding support for specifying the number of Kibana instances needed for a config, not from the fact that we'd be verifying all tests also work in this mode.
I saw the alternate solution as having the task manager folks implement this coverage as a jest integration test which manages the two instances itself rather than having it handled by the FTR framework.
spalger
on 23 Sep 2019
Hmm, I feel like the complexity comes from adding support for specifying the number of Kibana instances needed for a config, not from the fact that we'd be verifying all tests also work in this mode.
Maybe complexity was the wrong word - more there is overhead.
I saw the alternate solution as having the task manager folks implement this coverage as a jest integration test which manages the two instances itself rather than having it handled by the FTR framework.
This is probably the way to go. There are a handful of things which fall into the category of configuration, like SAML/Kuberos/PKI and Tribe/CCS, which should really take on the burden of testing in isolation. There might come a time where so much of Kibana functions in a multi-node environment it would make sense to treat this as a default like we have done with the basepath.
tylersmalley
on 24 Sep 2019
Related issues
Ginja
路
3Comments
socialmineruser1
路
3Comments
timroes
路
3Comments
MaartenUreel
路
3Comments
ynux
路
3Comments
Most helpful comment
Maybe complexity was the wrong word - more there is overhead.
This is probably the way to go. There are a handful of things which fall into the category of configuration, like SAML/Kuberos/PKI and Tribe/CCS, which should really take on the burden of testing in isolation. There might come a time where so much of Kibana functions in a multi-node environment it would make sense to treat this as a default like we have done with the basepath.