Cypress: Cypress should set a mocha slow threshold for more reasonable reporting

Created on 9 Mar 2017 · 16Comments · Source: cypress-io/cypress

I realize that this is something I can set, and that test speeds will probably be all over the map depending on what I'm testing, but I assume right now that cypress is using the default slow threshold which prints that red (123ms) after a "slow" test. Right now our non-cypress users on our team look at CI and say, "wow your tests are having some issues"

So I guess I'm wondering if there's a better threshold to set it to where it might provide meaning for the general case.. and if so, can that be the default in cypress?

1️⃣ first-timers-only needs a test enhancement

Source

paulfalgout

👍7

Most helpful comment

This is finally about to be done for good ;-)

I know there is an existing PR, but we had to do it in a slightly different way - but the result is the same.

brian-mann on 9 Jan 2018

👍3 🎉2

All 16 comments

Agreed. Mocha slow threshold is completely inappropriate for e2e/integration testing. We'll file this away as "expose mocha to be configurable" either via cypress.json or as code in a support/index.js file.

brian-mann on 9 Mar 2017

-s, --slow <ms> "slow" test threshold in milliseconds [75] is the option in Mocha.

Perhaps we should raise this to a default of 5000ms and then expose something in cypress.json like slowTestThreshold.

It's probably possible to tell mocha about this right now in a root before / beforeEach hook with this.slow(...)

brian-mann on 15 Sep 2017

In addition, we need to lift off this property from runnables else this.slow(...) will not work.

Specifically when we pull runnable data to send to the backend server to mocha's reporter...

brian-mann on 15 Sep 2017

This is finally about to be done for good ;-)

I know there is an existing PR, but we had to do it in a slightly different way - but the result is the same.

brian-mann on 9 Jan 2018

👍3 🎉2

@jennifer-shehane @brian-mann what's the timeline on this? I'd love to see this go into 4.0 if possible. Minimize the red!

Bkucera on 22 Dec 2018

👍2

I feel like the first part of this was done, and the second part regarding increasing thresholds was never implemented. Would probably require looking at some averages we've collected and assigning a new threshold.

I'm not completely sure what @brian-mann had planned.

jennifer-shehane on 8 Jan 2019

This is finally about to be done for good ;-)

Did this ever happen? Using mochawesome with cypress leaves a lot of red in the console, and confuses users into thinking that tests are failing.

I tried adding the following code to support/index.js, but it didn't work:

beforeEach(function() {
  this.slow(30000);
});

fr0 on 19 Mar 2019

Sorry I am a bit confused by this thread. Was something implemented in the end? Do I have a way (any way) to move the threshold up? All my tests are 20s or more (and that's perfectly normal) so would be great if I could.