Robottelo: tests.foreman.ui.test_contentview test failing due to timeout on success locator

[san7ket]

These test's are failing on upgrade and fresh due to due to Timeout on alert-success locator after the publish/promote is done. Tests doesn’t seem to find the locator/alert notification. but the publish is done successfully.
Upgrade has 4 of 8 failing

2017-09-01 22:53:23 - robottelo.ui.locators - DEBUG - Accessing locator "alert.success_sub_form" by xpath: "//div[@ng-hide]//div[contains(@class, 'alert-success')]"
2017-09-01 22:53:35 - robottelo.ui.base - DEBUG - TimeoutException: Waiting for element '//div[@ng-hide]//div[contains(@class, 'alert-success')]' to display. Message: element //div[@ng-hide]//div[contains(@class, 'alert-success')] is not visible

Couldn't find a related issue, to filing one to be tracked.

[rplevka]

Perhaps not quite a cure for the problem, but I can only suggest the same as i did for the Sync Plan tests - let's assert on the publish/promote tasks using API first, then check the progress bars, etc.

This way we'll have a clear idea whether the tasks take too long or the UI got it wrong (locator issues, etc.)

[ntkathole]

Same issue with
tests/foreman/endtoend/test_ui_endtoend.py::EndToEndTestCase::test_positive_end_to_end

   common_locators['alert.success']
E   AssertionError: None is not true

https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/endtoend/test_ui_endtoend.py#L113

[rplevka]

@ntkathole - have you double-checked that the task has finished successfully so the error is bogus?

[ntkathole]

@rplevka yeah I checked...If we commented the assertion for success locator, test passed successfully!!

[ldjebran]

I think that this is a bug , the success message is not always displayed on recent browsers (FF 45 FF 46 FF 47 Chrome) , but always working on FF 38,

To reproduce the problem create an empty content view using chrome/FF45/FF 47 and publish
now do the same on FF 38

[san7ket]

@rplevka actually, this is happening for manifest upload, activation keys, repo sync(sometimes)

[svtkachenko]

As I can see from the last 6.2 run, almost all tests from 22 failed were failing on add repository step. And according to logs it is a timing issue, probably caused by server overload or some latency because of saucelabs usage. It was fixed for master in #5034

From logs:

1. Time when add repository button was clicked`. Alert appears almost immediately.

2017-09-11 22:13:57 - robottelo.ui.locators - DEBUG - Accessing locator "contentviews.add_repo" by xpath: "//button[contains(@ng-show, 'available') and contains(@ng-click, 'addRepositories')]"

1. Then helper returns to remove/list tab, that also requires some time.

2017-09-11 22:14:01 - robottelo.ui.locators - DEBUG - Accessing locator "contentviews.tab_repo_remove" by xpath: "//a[contains(@href, 'repositories') and contains(@ui-sref, 'list')]/span[@class='ng-scope' and contains(., 'List/Remove')]"

1. And finally actual check for alert is performed 8 seconds later, however it is visible only for 4 seconds.

2017-09-11 22:14:05 - robottelo.ui.locators - DEBUG - Accessing locator "alert.success_sub_form" by xpath: "//div[@ng-hide]//div[contains(@class, 'alert-success')]"

[rplevka]

@svtkachenko, thanks, that is so great :D
I think asserting by checking the task status would help here too
since this action does not generate a task, I think we should simply assert the action by API after 'wait for ajax' or a similar 'out of band' action. I'm not trying to suggest to fix this issue by applying my suggestion to only the tests affected by this issue.
I'd like to start applying similar approach on global level. This is for a broader discussion

[san7ket]

@svtkachenko The failures for last run improved +1 , 2 out of 8 are failing. But if it's a timing issue, then nothing much could be done for upgrade boxes as they are already in load due to existing data/features.

@rplevka But I also agree with the fact that, even if the UI time-outed , we simply shouldn't fail the test and try to go an check it via API.
Check via UI --> If timeouted --> log the timeout and go ahead and Check with api -->Pass if present.

[rplevka]

affecting
tests.foreman.ui.test_docker.DockerContentViewTestCase.test_positive_promote_multiple_with_docker_repo
too (and i guess all the subsequential tests using ContentViews in UI)

[rplevka]

ui.test_discoveredhost.DiscoveryTestCase.test_positive_manual_provision_host_with_rule
ui.test_activationkey.ActivationKeyTestCase.test_positive_add_multiple_aks_to_system