Cypress: Cypress Module API does not handle exit codes correctly as it does with cypress run in the CLI

[joshpzero]

Current behavior:

Cypress Module API currently does not handle exit codes correctly as it does with cypress run in the CLI.

I would expect to exit with error code 1 on failure of any tests and error code 0 for all passing tests. This means our Gitlab CI stages are giving incorrect results.

Steps to reproduce:

1. Run the default module code to execute tests
   cypress.run({ spec: './cypress/integration/examples/actions.spec.js' }) .then((results) => { console.log(results) }) .catch((err) => { console.error(err) })
2. node ./run-module.js

Expected that the correct exit code is recieved

Versions

3.1.2

[chrisbreiding]

When you use the module API, you're in control of the process and how it exits. It wouldn't be appropriate for Cypress to exit the process for you. There's no way for Cypress to know when you're finished handling/processing the results, so it's possible it would kill the process too early.

You can use process.exit() to exit with the appropriate code.

cypress.run({ 
  spec: './cypress/integration/examples/actions.spec.js' 
}) 
.then((results) => { 
  console.log(results) 
  process.exit(results.totalFailed)
})
.catch((err) => { 
  console.error(err) 
  process.exit(1)
})