Kibana: Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/certificates·ts - Uptime app with generated data certificate page can navigate to cert page

Created on 2 May 2020 · 18Comments · Source: elastic/kibana

A test failed on a tracked branch

Error: retry.tryForTime timeout: Error: expected testSubject(uptimeCertificatesPage) to exist
    at TestSubjects.existOrFail (/dev/shm/workspace/kibana/test/functional/services/test_subjects.ts:62:15)
    at onFailure (/dev/shm/workspace/kibana/test/common/services/retry/retry_for_success.ts:28:9)
    at retryForSuccess (/dev/shm/workspace/kibana/test/common/services/retry/retry_for_success.ts:68:13)

First failure: Jenkins Build

uptime blocker failed-test skipped-test v7.9.0

Source

kibanamachine

All 18 comments

Pinging @elastic/kibana-test-triage (failed-test)

elasticmachine on 2 May 2020

Pinging @elastic/uptime (Team:uptime)

elasticmachine on 3 May 2020

New failure: Jenkins Build

kibanamachine on 3 May 2020

New failure: Jenkins Build

kibanamachine on 4 May 2020

Fixed by https://github.com/elastic/kibana/pull/65056

shahzad31 on 4 May 2020

New failure: Jenkins Build

kibanamachine on 5 May 2020

zube[bot] go home, you're drunk

spalger on 5 May 2020

😄1

New failure: Jenkins Build

kibanamachine on 6 May 2020

New failure: Jenkins Build

kibanamachine on 7 May 2020

New failure: Jenkins Build

kibanamachine on 13 May 2020

New failure: Jenkins Build

kibanamachine on 13 May 2020

For real though... what's going on here?

spalger on 13 May 2020

it reappeared after 4-5 days :(
i haven't been able to really debug why this is flaky, i have run flaky test runner for 3-4 times, and it passes without zero failures, @justinkambic @andrewvc can you please take a look at this test? i might be missing something obvious or tricky here.

shahzad31 on 13 May 2020

I don't know what is happening. The log says we click on the certificates page link no problem, and then it fails trying to assert that the certs page has loaded by looking for a data-test-subj attribute we render on a div on that page. It doesn't find this, because it's not there.

Looking at the HTML that failed this test, we never left the overview page, so clearly the click on the cert page link failed, or something weird happened, even though it says it worked. I don't know what would make this happen, but that's where we are with it. I'm not sure how to debug it further.

justinkambic on 13 May 2020

Sorry, I was talking to zubebot, but it would be great to figure out why this test is flaky.

spalger on 13 May 2020

6 failures in the last 7 days, across master, 7.x, and several different PRs against master and 7.x

If we look at the last 30 days we see 19 with similar diversity

Skipped

master: https://github.com/elastic/kibana/commit/5f9c4704ac511f31674a14ca8f5471ba42e9b984
7.x/7.9: https://github.com/elastic/kibana/commit/1aae674d634b04a0a8d020cd1e024c32acd96d0e

spalger on 13 May 2020

Sorry, I was talking to zubebot

I don't know if anyone can explain what that bot is on about with its weird behavior. Sometimes it closes issues that aren't resolved, and then it gets into weird loops like that.

justinkambic on 13 May 2020

👍1

I believe I've fixed this in https://github.com/elastic/kibana/pull/66349 which I ran 42 times on the flaky test runner.

I know we're supposed to always run the tests in order, but I rewrote the sweet to not depend on ordering. Navigating to the certs page was something that was order dependent, and now no longer is.

I'm going to close this for now, but feel free to reopen if need be.