Kibana: [test-failed]: Chrome X-Pack UI Functional Tests1.x-pack/test/functional/apps/lens/lens_reporting·ts - lens app lens reporting should not cause PDF reports to fail

[liza-mae]

Version: 7.9.0
Class: Chrome X-Pack UI Functional Tests1.x-pack/test/functional/apps/lens/lens_reporting·ts
Stack Trace:
Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
Wait timed out after 10014ms
at /var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
at process._tickCallback (internal/process/next_tick.js:68:7)
at onFailure (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/test/common/services/retry/retry_for_success.ts:28:9)
at retryForSuccess (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/test/common/services/retry/retry_for_success.ts:68:13)
_Test Report: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-kibana-tests/495/testReport/_

[elasticmachine]

Pinging @elastic/kibana-test-triage (failed-test)

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[stratoula]

I tried to reproduce it. I ran the test against a 7.9 cloud instance. Is not always reproducible, sometimes it fails, sometimes not.
When it fails, the report is not generated and the error I get is:

On the reports I get this:

I am not sure why this happens, is it a bug on reporting maybe? Any thoughts @flash1293 ?

[stratoula]

Update: It seems that when I change the time filter from 15 minutes to 5 years, the report is generated successfully. I will try to also change the time range on the test and check if it fails

[flash1293]

Is not always reproducible, sometimes it fails, sometimes not

Seems like this is a legit error on cloud - @elastic/kibana-reporting-services could you take a look at this one?

[joelgriffith]

Whats' the size/type of instance running Kibana? The Target closed error is almost always synonymous with insufficient resources to complete the job.

[flash1293]

@liza-mae Can we resize the instance used in the test to check whether that's the issue here? (if it's running in a 1GB container it's very likely it's related)

[stratoula]

My tests were indeed on a 1GB instance

[flash1293]

Ah, I forgot those screenshots were from your run, @stratoula . Could you try again with a 4gb or 8gb instance? As reporting is firing up a headless chromium, it needs a bit more memory to operate. Surprised you had successful runs at all :)

[stratoula]

Well, I have a 8GB instance and can't reproduce the problem - unless I am really lucky for 10 times on a row :)

[liza-mae]

I use 2GB in the testing ( I will double check though). @joelgriffith is 2GB the minimum for reporting to work? If we need 8GB seems like something we would need to look at to fix.

[flash1293]

Let's wait for Liza in that case, maybe she used a 1GB instance as well.
Edit: Race condition, please disregard.

[stratoula]

I also see that .x-pack/test/functional/apps/visualize/reporting·ts.Visualize Visualize Print PDF button downloaded PDF has OK status fails often on the cloud with the same error message.

[liza-mae]

Version: 7.9.0
Class: Chrome X-Pack UI Functional Tests1.x-pack/test/functional/apps/lens/lens_reporting·ts
Stack Trace:
Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"]) Wait timed out after 10036ms at /var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17 at process._tickCallback (internal/process/next_tick.js:68:7) at onFailure (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/test/common/services/retry/retry_for_success.ts:28:9) at retryForSuccess (/var/lib/jenkins/workspace/elastic+estf-cloud-kibana-tests/JOB/xpackGrp2/TASK/saas_run_kibana_tests/node/linux-immutable/ci/cloud/common/build/kibana/test/common/services/retry/retry_for_success.ts:68:13)
_Test Report: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-kibana-tests/589/testReport/_

[liza-mae]

I moved this test to run on 2GB - let's wait and see if it fixes it.

[joelgriffith]

Somewhere between 2-4GB is kind of the sweet spot for reporting in the cloud. 8GB is more bullet-proof but harder to recommend given the cost

[stratoula]

Thanx @joelgriffith, let's see if it continues to be flaky with 2GB

[tsullivan]

I want to emphasize that @elastic/kibana-reporting-services understands supporting screenshot reporting in 1GB is a big issue for out Cloud-first mindset. 1GB is the default configuration for Kibana deployments in Cloud.

There are a few things we have in-flight to address this

• A Screenshot service in Kibana to present an alternate page rendering for Reporting with a "lighter" UI
• Waiting for Kibana platform to improve performance and require lower memory consumption in headless Chromium.

Given that neither of these are guarantees that we can ever support screenshot Reporting on 1GB instances, we need to think of ways to message this problem with the out-of-the-box experience.

[stratoula]

After discussion with @liza-mae, It seems than on a 2GB instance the test passes so I will close this PR.