Cypress: Feature: Run until test fails (debug flakey/intermittent errors)

[MadLittleMods]

Add a way to keep re-running the test-suite until you run into a failure. When debugging flakey, intermittent failures, it would be nice to just keep retrying the tests until something fails. I don't want to keep pressing re-run manually.

This could be accomplished with cypress run and some outside scripting on top of it but I want to do this inside the cypress open test runner. Using cypress run is also tough to get useful info out of when a failure happens but this is being tracked by https://github.com/cypress-io/cypress/issues/448

There is a Run all tests button in the UI but it only re-runs the test suite once. I want to keep retrying the test suite until some test fails.

---

Related issues but mainly around retrying failed tests,

• https://github.com/cypress-io/cypress/issues/518
• https://github.com/cypress-io/cypress/issues/1599
• https://github.com/cypress-io/cypress/issues/4886

[gabbersepp]

Maybe you can take the great retry plugin and inverse its behaviour? Should not be too much efford i think

[gabbersepp]

Or open an issue in its repository. Maybe the owner introduces a flag to switch between both modes?

[MadLittleMods]

Thanks @gabbersepp! Created an issue over there, https://github.com/Bkucera/cypress-plugin-retries/issues/37

[jennifer-shehane]

There is an example in our recipes that kind of does something similar to this here: https://github.com/cypress-io/cypress-example-recipes/blob/master/test-repeat.js