Cypress: `ignoreTestFiles` is not working on Cypress 3.0.2 in Windows

[heardy]

Current behavior:

On Windows 10 cypress detects all files under integrationFolder as spec files irrespective of ignoreTestFiles

Desired behavior:

Ignore files that match the glob set in ignoreTestFiles.

Note: tested on Mac and works correctly

Steps to reproduce:

Add any files to integration folder eg html files and intended test files ending in spec.js (eg ExampleTest.spec.js)

Set ignoreTestFiles in cypress.json to /**/*.html

Execute cypress run and see that the intended ignored files (in this case html files) are include as spec files.

I have also tested with ignoreTestFiles set to /**/!(*.spec.js) and has the same behaviour.

Versions

OS: Microsoft Windows [Version 10.0.17134.165]
Cypress: 3.0.2
Chrome: 67.0.3396.99 (Official Build) (64-bit)

[jennifer-shehane]

I wonder if this has to do with using / versus \. If that is the case, you can work around this by setting the config using our plugins API and use path.join to handle this case like so:

// cypress/plugins/index.js
module.exports = (on, config) => {
  console.log(config) // see what all is in here!

  // modify config values
  config.ignoreTestFiles = path.join('**', '*.html');

  // return config
  return config
}

If this is the case and this does fix your issue, Cypress should just normalize this for you based on your OS.

[heardy]

The issue was the path separator, which I thought the glob tool would handle but looking at the code the ignoreTestFiles is passed to minimatch which I gather doesn't handle slashes in windows correctly.

Your plugin solution above works, thanks Jennifer.

Also, I noticed when logging out the config that there is a config property testFiles. This looks like the exact config option that I was wanting so I could set the file path for test files. I have set it to "testFiles": "**/*.spec.js", and it works on both my mac and windows machines. Is this option ok to use? I can't find it in the configuration docs (https://docs.cypress.io/guides/references/configuration.html#Options)

Note: testFiles uses node-glob which handles windows/mac paths correctly - https://github.com/cypress-io/cypress/blob/0e2246785c3a2453dd0e9b9ca0e7daa5f2bceab4/packages/server/lib/util/specs.coffee#L109
Whereas ignoreTestFiles uses minimatch which doesn't seem to handle paths correctly - https://github.com/cypress-io/cypress/blob/0e2246785c3a2453dd0e9b9ca0e7daa5f2bceab4/packages/server/lib/util/specs.coffee#L90-L91

[jennifer-shehane]

We did not publicly expose testFiles because it had not been determined as a flag that would be useful to be exposed to users.

The ignoreTestFiles needs to be updated to handle glob paths correctly regardless of OS. Currently minimatch (or at least our version) doesn't handle paths correctly - https://github.com/cypress-io/cypress/blob/0e2246785c3a2453dd0e9b9ca0e7daa5f2bceab4/packages/server/lib/util/specs.coffee#L90-L91

[raelgc]

Thanks @heardy for mention testFiles: it was exactly what I was looking for after fight with the non-working ignoreTestFiles.

[kohlikohl]

@raelgc Great find of testFiles!

@jennifer-shehane Please, please, please document this. It is super useful when the project structure does not align with the suggested structure by cypress.

For example, we run a mono-repo which has multiple npm packages inside it, with E2E tests in multiple packages. The test runs are coordinated across all packages though. So being able to filter by the files that I actually want to test was super helpful.

[peterthim]

Still an issue in 3.2.0,

I had to do the following
module.exports = (on, config) => { config.ignoreTestFiles = path.join( config.integrationFolder, 'manual', '**', '*.js' ); return config; };

to remove all testcases from the manual folder (where i have failing TC that shouldn't fail the TA runs
without using absolut paths i didn't get it to work.

[jennifer-shehane]

This may be related to this PR fix https://github.com/cypress-io/cypress/pull/3880

[flotwig]

Looks like the original issue here was trying to glob using a full path: /**/*.html. It works as expected if you just do **/*.html. Since there's not a bug here, going to close this. #3880 will merge soon and allow ignoreTestFiles to be overridden from other places as well.