Cypress: Adding logic to the `fail` event per documentation negates test failure

[scottcrowe]

Is this a Feature or Bug?

Bug

Current behavior:

Documented code for Debugging when a test fails does not leave the test as a failure.
Referenced documentation https://docs.cypress.io/api/events/catalog-of-events.html#Catching-Test-Failures

Desired behavior:

Tests with failures still shown as failures in the log.

Steps to reproduce:

Add the following to support/index.js file:

Cypress.on('fail', (err, runnable) => {
  debugger
})

Create a test that fails and run it

Note: Adding runnable.emit('error', err); to the above event binding will have the error message show in log but does not mark the test as a failure.

Versions

OS: Windows 7
Browser: Chrome 66
Cypress: 3.1.0

[Strajk]

Confirm, I had to adjust the code for the test to fail

Cypress.on("fail", (err, runnable) => {
  debugger
  return err
})

cc @scottcrowe

[Strajk]

Also beware that if you use cypress-webpack-preprocessor, it's stripping debugger statements with default options

(spend more that little time debugging this)

https://github.com/cypress-io/cypress-webpack-preprocessor/issues/29

[jennifer-shehane]

I created a new issue in our docs to document how to fail a test after listening to 'fail' events here: https://github.com/cypress-io/cypress-documentation/issues/1045. Our documentation is open source and contributions are welcome. 😄

Since this seems to be a documentation issue and not an actual issue with Cypress's implementation, I will close this issue.