Cypress: Allow to rerun only failed tests directly from the Test Runner UI

[sylvain-hamel]

Current behavior:

In the test runner, there is no option to only rerun failed tests. After investigating and fixing a failed test, I'd like the watcher to only rerun failed tests.

Desired behavior:

Add a rerun "Rerun failed tests only" checkbox to the tests runner UI.

[jennifer-shehane]

Some similiar requests:

• https://github.com/cypress-io/cypress/issues/1313
• https://github.com/cypress-io/cypress/issues/1599

[Superkunair]

There's a plugin that can execute your failed test
cypress-plugin-retries

[sylvain-hamel]

@jennifer-shehane and @Superkunair you both refer to issues and solutions to handle flaky tests by rerunning them. That's not what I'm looking for. I'm looking for a feature in the Test Runner UI.

The use case is:

• I change some code and then run the tests using the Test Runner UI.
• Running them takes 10 minutes and 3 tests fail.
• For the next several minutes I'll focus on these 3 tests, changing code and rerunning them many times.
• While I do this, I want the Test Runner UI to only watch/rerun these 3 tests, until they pass.
• When I'm done fixing the issue, I can rerun the whole batch before I commit and push.

So this request is really for an improved developer experience while investigating issues, not as a solution to handle flaky tests.

[brian-mann]

What you're referring to is what we internally call burn-down mode.

[jennifer-shehane]

Yes, just tagging similar requests - this is a new kind of request which is why the issue was left open.

[DavidGWheeler]

There's a plugin that can execute your failed test
cypress-plugin-retries

That is for re-running during execution to account for network issues? We want a button in the runner marked "re-run failed tests only) to allow for debugging. Don't confuse the issue.

[DavidGWheeler]

@jennifer-shehane and @Superkunair you both refer to issues and solutions to handle flaky tests by rerunning them. That's not what I'm looking for. I'm looking for a feature in the Test Runner UI.

The use case is:

• I change some code and then run the tests using the Test Runner UI.
• Running them takes 10 minutes and 3 tests fail.
• For the next several minutes I'll focus on these 3 tests, changing code and rerunning them many times.
• While I do this, I want the Test Runner UI to only watch/rerun these 3 tests, until they pass.
• When I'm done fixing the issue, I can rerun the whole batch before I commit and push.

So this request is really for an improved developer experience while investigating issues, not as a solution to handle flaky tests.

THIS! It is a really super simple and basic feature. Every IDE has it. Only Cypress runner fails to provide such a simple thing!

[DavidGWheeler]

Some similiar requests:

• #1313
• #1599

Neither of those issues relates to this issue in any way.

[Honigbaum]

I can agree with @DavidGWheeler, we're looking forward to see this feature too. Currently there isn't any way to rerun just failed test after fixing it.

[juni0r]

I agree this would be an immensely helpful feature.

[samtsai]

I've found two options:

• Use it.only(...) in your tests
• https://github.com/bahmutov/cypress-skip-and-only-ui
  

[LieAlbertTriAdrian]

This will be super helpful

[aesfer]

This is a must for test-driven development.

For me jest in watch mode (via vue-cli) has the ideal behaviour, it goes one step further and only re-runs the tests affected by code changes since the last time they ran (so that if there are 5 failing tests and I modify 1 of them only that one will re-run).

However I don't want to confuse the issue and as the OP suggested, an option for only re-running failing tests is a perfect first step.

[djaiss]

While I don’t agree with @DavidGWheeler 's statement about it being a feature that is both simple and easy (if it’s so easy, please submit a PR), I think this feature is really essential to the workflows on projects I’m working on.

I was really excited about the Cypress 5.0 when I read about the retry ability - but this is really not what I had in mind. I think having the ability to re-run a single failed test, or multiple failed tests in a row, would really help.

On a test suite of 130 tests that take 20 minutes to run, seeing 13 failed tests (completely separated) and re-running them is a nightmare once I changed the behavior to fix them.

[narze]

In some other testing frameworks I've used they have this feature built-in (RSpec, Cucumber)

When the test fails it will create a file (eg. .rspec_retry) with filenames with line of code number of failing tests, and the next test run will focus only these tests. You don't have to manually add focus like .only to any failing tests manually. When all tests are passing, the file will be deleted.

This feature is not about mitigating flaky tests, but for TDD or auto-focusing test for fixing purpose.