Cypress-cucumber-preprocessor: feature startup speed

[darrinholst]

Longtime cucumber user, but relatively new to cypress so first, thank you for this!

It seems that cypress resets the world between each feature file. That results in some noisy logging after each feature...

  Running: user-dropdown/logout.feature...                                               (24 of 25) 

  [▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬]

  2 passing (3s)


  (Results)

  ┌────────────────────────────────────────────┐
  │ Tests:        2                            │
  │ Passing:      2                            │
  │ Failing:      0                            │
  │ Pending:      0                            │
  │ Skipped:      0                            │
  │ Screenshots:  0                            │
  │ Video:        true                         │
  │ Duration:     3 seconds                    │
  │ Spec Ran:     user-dropdown/logout.feature │
  └────────────────────────────────────────────┘

I could live with the logging, but my main problem is that this more than doubles the execution time for the test run of all features.

The tests are pretty fast once they get going, but there is a noticeable lag to get them started.

Is there any way to organization features and/or scenarios in a way that it runs them all in the same "test run".

I am running the oldschool step definition structure and I noticed the bit about startup speed in the readme. Is that what I'm seeing here? We have 110 step definitions used across 25 feature files.

[lgandecki]

I think this might be a problem with how cypress works itself. If you do NOT use cucumber, and just write mocha spec files, would that be any faster? Would you like to check?

[darrinholst]

@lgandecki That was my assumption but thought I'd check to see if I was missing anything obvious. Is there anything debugging related here that would convert features to specs and write them out to disk?

[darrinholst]

Closing this. It's just the way cypress works. Their answer is probably parallelization.

It would be nice to have a way to create a "mega" feature with some extra gherkin syntax to be able to "import" features so we can still split them up by file, but allow them to run in the same mocha spec.

[darrinholst]

@lgandecki any interest in something like this. It adds a loader for *.features files which is just a marker file to get all the *.feature files in that directory and combine them into one cypress test. I've been using it for a week now and is working good. I could clean up the duplication and PR it if wanted.

[lgandecki]

Interesting.. speeding things up would be fantastic! but I don't get the code or your explanation yet.. Any chance of a simple repo showing how to use this? That would help figuring out how to implement this (or maybe something inspired by what you've done). If you don't have time, no worries, I'll understand and experiment myself eventually :)

On Jul 26, 2019, at 15:14, Darrin Holst [e-mail] wrote:

Reopened #189.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[darrinholst]

@lgandecki sorry for the delay. Here is an example repo using the features from this repo itself

https://github.com/dholst/cypress-cucumber-preprocessor-example.

I added an index.features file and specified which way I want to run them in the --spec config.

Here is the normal test run - https://gist.github.com/darrinholst/be3eaf9016aa63c094111e0744d56ca9 which took 4.25 minutes.

Here is the run with my changes - https://gist.github.com/darrinholst/80ea8820a12b0185e7d36f2ac7f03bb1 which took 22 seconds.

You'll notice that there was 1 failure there which is due to this project's take on isolated step definitions. Since all the features run in one context this way all the step definitions have to be loaded which kind of breaks the isolation. I think there is probably a way to fix it by passing all the required step definitions to the createTestsFromFeature function and requiring them in there...maybe.

I don't use that feature and I'd be curious to know what existing cucumber users think of it. The official stance from cucumber developers themselves is that it's an anti-pattern, but that's a discussion for another day. 😄

I hope the example repo and the results show what I was trying to do and the speed improvements that are possible.

[lgandecki]

now I have to appologize for the delay.
I get what you are doing. Very cool. I think we could use that. :)
The performance in the library case itself (when there is no DOM interactions) is amazing.

Do you want to create a PR?
We would need to DRY it up a bit (I can do that), and prepare documentation.
Also, I'm thinking if it would make more sense to use this trick with some ENV variable, for doing cypress run? Might be a bit cleaner. Then they could still do cypress run without specifying the spec files, same way they do currently.
For the cypress open - it would make sense to have the separate .feature file, for when someone wants to work on a specific one, but also have the abilitiy to click ".features". so we can run everything in the cypress GUI as well

[laiscoolblue]

This would be super nice to have!

I've tried running it to try it out with our tests, but as I understand the new version of the preprocessor is not compatible anymore with @darrinholst implementation.

Currently, in our CI the node in which Cypress in running is taking 10 minutes to run, but the actual execution of the tests takes 4 minutes, so it would be nice to have the option to bundle them.

I would also suggest having the option to bundle them by tags. I'm available to help with improving the documentation.

[lgandecki]

yeah, we will get it in. Possibly this weekend. I need to fix the hooks first and then this is the second thing I'll be looking at.

[darrinholst]

Added #217. I don't have a strong preference for how this is implemented, i.e. env variable or not. For the case of cypress open we have been overriding the testFiles so all feature files are available to click on in the gui...

    cypress.open({
        project: './test/acceptance',
        config: `testFiles=**/*.feature*,baseUrl=${process.env.APP_URL}`
    });

I think documentation will be key for this as it's not immediately obvious how it works (at least it wasn't for me)

[darrinholst]

oh, here's the default config that grabs only the bundled features...

{
    "chromeWebSecurity": false,
    "defaultCommandTimeout": 10000,
    "ignoreTestFiles": "*.js",
    "testFiles": "**/*.features"
}

[lgandecki]

Thanks for sharing these.

I think you are right - documentation will be key, and this is more of a one-time setup, as long as the default behavior just works it should be fine, even if the recommended way of starting things up would take some extra fiddling..

@laiscoolblue I think I will take you up on the offer if documentation is your superpower. ;) Do you want to work on that? You could create a PR matching Darrinholst work - I will work on simplyfing the code a bit, but the external behavior will be exactly the same, so documentation should not be affected. Part of the documentation could be a link to files in https://github.com/TheBrainFamily/cypress-cucumber-example that shows a setup similar to what darrinholst shared here.

[laiscoolblue]

Hey @lgandecki, sorry for the late reply, nice work!

I can work on the documentation today.

[lgandecki]

:tada: This issue has been resolved in version 1.15.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[amitguptagwl]

1. To use bundled file, should I create empty index.features file in features folder?

npx cypress run --spec \"**/*.features\"

Above command hangs.

1. What would be the syntax if I want to use while giving the list of tags to run

npx cypress-tags run -e TAGS='not @Ignore' --spec \"**/*.features\"