Nx: Cypress builder option `cypressConfig` not working

[alanyinjs]

Prerequisites

• [x] I am running the latest version
• [x] I checked the documentation and found no answer
• [x] I checked to make sure that this issue has not already been filed
• [x] I'm reporting the issue to the correct repository (not related to Angular, AngularCLI or any dependency)

Issue Summary

This issue contains a set of two related issues with Cypress builder options. In particular, the option for the path to configuration file (cypressConfig) was not working, as Cypress will always grab cypress.json as its configuration and ignore what's in the file pointed by cypressConfig.

The goal we wanted to achieve was to pass different environmental variables to Cypress builder through Angular CLI, so that we can run e2e tests in different environments (e.g. build, stage, etc.). In package.json we have the following scripts for running e2e tests:

• Build: "e2e:build:watch": "ng e2e op2-e2e -c build --watch",

• Stage: "e2e:stage:watch": "ng e2e op2-e2e -c stage --watch",

This way we can set configurations in angular.json to pass different config files to cypress builder for different environment. An example with our angular.json setting is as follows:

    "bazinga-e2e": {
      "root": "apps/bazinga-e2e",
      "projectType": "application",
      "prefix": "",
      "architect": {
        "e2e": {
          "builder": "@nrwl/builders:cypress",
          "options": {
            "cypressConfig": "apps/bazinga-e2e/cypress.json",
            "tsConfig": "apps/bazinga-e2e/tsconfig.e2e.json",
            "devServerTarget": "bazinga:serve:ci"
          },
          "configurations": {
            "prod": {
              "cypressConfig": "apps/bazinga-e2e/cypress.stage.json"
            },
            "stage": {
              "cypressConfig": "apps/bazinga-e2e/cypress.stage.json"
            },
            "build": {
              "cypressConfig": "apps/bazinga-e2e/cypress.json"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": "apps/bazinga-e2e/tsconfig.e2e.json",
            "exclude": ["**/node_modules/**"]
          }
        }
      }
    }

Here we expect cypress builder to read configuration files according to the environment configuration passed in by Angular CLI (e.g. if ng e2e bazinga-e2e -c stage is run, we expect Angular CLI to grab the path to cypress.stage.json and pass it to cypressConfig for Cypress builder: https://github.com/nrwl/nx/blob/master/packages/cypress/src/builders/cypress/cypress.impl.ts). This will enable different options for Cypress in different environments.

Alternatively, at a very basic level, one should expect cypressConfig in "options" to work(as documented here: https://nx.dev/angular/api/cypress/builders/cypress). e.g. if we have following configuration:

    "bazinga-e2e": {
      "root": "apps/bazinga-e2e",
      "projectType": "application",
      "prefix": "",
      "architect": {
        "e2e": {
          "builder": "@nrwl/builders:cypress",
          "options": {
            "cypressConfig": "apps/bazinga-e2e/cypress.stage.json",
            "tsConfig": "apps/bazinga-e2e/tsconfig.e2e.json",
            "devServerTarget": "bazinga:serve:ci"
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": "apps/bazinga-e2e/tsconfig.e2e.json",
            "exclude": ["**/node_modules/**"]
          }
        }
      }
    }

We expect Cypress builder to grab cypress.stage.json and use it as its configuration file.

However both above use cases are not supported. We found during our investigation that Cypress builder always tends to use cypress.json as its config file and grabs but completely ignores all configuration within any other configuration file which path is specified in cypressConfig. Furthermore, in the absence of a cypress.json file, it always tends to create an empty cypress.json with {} as its content and ignores whatever file path we specify in "options".

Although cypressConfig option does not have the correct behaviour. baseUrl options has exactly the above mentioned behaviour, which suggests there is a bug in applying configuration to Cypress.

Not supporting the above two use cases means that we are unable to set configurations based on environment.

Expected Behaviour

When file path to a Cypress configuration file is specified in cypressConfig option for @nrwl/builders:cypress (either in "options" or "configurations"), the file path should be used by Cypress to identify its configuration file, thus giving us the option to apply different set of configurations for Cypress in different environments.

Current Behaviour

The file path specified in cypressConfig will not used by Cypress builder to identify its configuration file. Cypress builder will always use cypress.json. In absence of cypress.json, it will create a new empty file named cypress.json and assume no configurations is being passed in.

Without looking further into the source code of Cypress builder, we doubt it's going to be a merging/overriding issue, as suggested by the behaviour in absence of cypress.json.

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. Set up different Cypress config files e.g. cypress.json, cypress.stage.json etc. Make sure you have different variables in those settings.
2. Set up angular.json and package.json as mentioned above.
3. Use Cypress.env() to access environmental variables in a test case and console.log it out to identify which config file has been used.
4. Run e2e test against both build and stage environments.
5. You will always find the environmental variables specified cypress.json are being used.

Context

Please provide any relevant information about your setup:

• version of Nx used: Not working on 7.4.1 and 7.8.7 (latest version that's compatible with Angular 7, will try again when we upgrade to Angular 8 this week)
• 3rd-party libraries and their versions

    "@angular/animations": "~7.2.0",
    "@angular/cdk": "~7.2.0",
    "@angular/common": "~7.2.0",
    "@angular/compiler": "~7.2.0",
    "@angular/core": "~7.2.0",
    "@angular/forms": "~7.2.0",
    "@angular/platform-browser": "~7.2.0",
    "@angular/platform-browser-dynamic": "~7.2.0",
    "@angular/router": "~7.2.0",
    "@angular/upgrade": "~7.2.0",
    "@auth0/angular-jwt": "^2.1.0",
    "@ngrx/effects": "^7.2.0",
    "@ngrx/entity": "^7.2.0",
    "@ngrx/router-store": "^7.2.0",
    "@ngrx/store": "^7.2.0",
    "@nrwl/nx": "^7.8.7",
    "angular": "1.6.6",
    "angular-froala-wysiwyg": "~2.9.6",
    "angular-mentions": "^0.9.1",
    "core-js": "^2.5.4",
    "d3": "^5.9.1",
    "date-fns": "^2.0.1",
    "file-saver": "^1.3.8",
    "iframe-resizer": "^4.1.1",
    "json2csv": "^4.5.2",
    "ng-bazinga (private package)": "^7.0.42",
    "ng2-tooltip-directive": "^2.1.8",
    "ngx-color": "^3.0.3",
    "ngx-infinite-scroll": "^7.1.0",
    "rxjs": "~6.3.3",
    "single-spa": "^4.3.1",
    "validate-currency-code": "^0.1.0",
    "xml-formatter": "^1.1.0",
    "zone.js": "^0.8.26"

  "devDependencies": {
    "@angular-devkit/build-angular": "~0.13.0",
    "@angular/cli": "~7.3.8",
    "@angular/compiler-cli": "~7.2.0",
    "@angular/language-service": "~7.2.0",
    "@ngneat/spectator": "~4.0.0",
    "@netbasal/spectator": "3.12.0",
    "@ngrx/store-devtools": "^7.2.0",
    "@nrwl/builders": "7.4.1",
    "@nrwl/schematics": "7.4.1",
    "@types/jasmine": "^3.4.0",
    "@types/jquery": "3.3.6",
    "@types/node": "~10.14.15",
    "@types/react": "^16.9.2",
    "@types/styled-components": "^4.1.18",
    "codelyzer": "~4.5.0",
    "cypress": "~3.4.1",
    "cypress-image-snapshot": "~3.1.1",
    "deep-object-diff": "^1.1.0",
    "jasmine-core": "~3.4.0",
    "jasmine-marbles": "~0.6.0",
    "jasmine-reporters": "~2.3.2",
    "karma": "~4.3.0",
    "karma-chrome-launcher": "~3.1.0",
    "karma-coverage-istanbul-reporter": "~2.1.0",
    "karma-jasmine": "~2.0.1",
    "karma-jasmine-html-reporter": "~1.4.2",
    "karma-junit-reporter": "~1.2.0",
    "karma-parallel": "~0.3.1",
    "karma-spec-reporter": "~0.0.32",
    "ncp": "^2.0.0",
    "ng-mocks": "~8.1.0",
    "ng-openapi-gen": "^0.5.2",
    "ng-swagger-gen": "^1.7.1",
    "ngrx-store-freeze": "0.2.4",
    "node-sass": "^4.12.0",
    "bazinga-swagger-client-generator (private package)": "^0.1.9",
    "prettier": "1.15.2",
    "react": "^16.9.0",
    "replace-in-file": "^3.4.3",
    "bazinga-ui-components (private package)": "0.46.2",
    "bazinga-widgets-integration (private package)": "0.46.2",
    "styled-components": "^4.3.2",
    "swagger-json-filter": "0.0.1",
    "ts-node": "~7.0.0",
    "tslint": "~5.11.0",
    "typescript": "~3.2.2",
    "webpack-bundle-analyzer": "^3.4.1"
  }

• and most importantly - a use-case that fails
  As mentioned in Issue Summary and Steps to Reproduce

A minimal reproduce scenario using allows us to quickly confirm a bug (or point out coding problem) as well as confirm that we are fixing the right problem.

Failure Logs

• It will just silently fail and grab whatever config is in cypress.json and use them. We can confirm that cypress builder was looking for those configuration files for different environment and actually grabbed them by changing the file name of those files. e.g. cypress.stage.json -> cypress1.stage.json.

Related Issues

1866

[skydever]

Hi @alanyinjs

Thx for this detailed writeup, I hope this is clarifying some things I may have missed in #1866

I already created a pr #1867 for this one, did you have the chance to look at it? I was not able to change the behavior of Cypress - it is always expecting a cypress.json in the project directory (the directory the option cypressConfig is pointing to), as you also mentioned here, so I came up with the following idea:

• put cypress base configuration in cypress.json
  
  
  
  • the file has to exist, so I took advantage of it and put my base config there
  
  
• environment specific configuration overrides are placed in cypress.stage.json (whatever name you put into angular.json)

The modified builder of the pr is reading both files and merging the configuration values (env. specific configuration has a higher priority to override base configuration) and passing the values directly to Cypress via the API (the values are listed as CLI args if you check the settings in the Cypress app).

What do you think about that solution, would it fit for your use case too?

[skydever]

... the only thing is that the cypress.json needs to exist in the same directory like the env. specific ones, and there is no way to enforce that I guess, it just has to be 🤔 I can add an error log message or warning ...

[skydever]

Hi @alanyinjs

I am updating the pr to be able to pass Cypress configuration values as builder options. This way there will only be one cypress.json and the overrides per configuration can be done at angular.json without creating multiple cypress.*.json files.

I re-read your issue because I am evaluating wich options are a candidate to be overwritten by different angular.json e2e configurations. When I got you right, you only want to pass different environment variables to Cypress. I think this is already possible with the env builder option, but the documentation for it is missing at the docs.

So you can do that at angular.json to overwrite the values:

    "bazinga-e2e": {
      "root": "apps/bazinga-e2e",
      "projectType": "application",
      "prefix": "",
      "architect": {
        "e2e": {
          "builder": "@nrwl/builders:cypress",
          "options": {
            "cypressConfig": "apps/bazinga-e2e/cypress.json", // has to be cypress.json
            "tsConfig": "apps/bazinga-e2e/tsconfig.e2e.json",
            "devServerTarget": "bazinga:serve:ci"
          },
          "configurations": {
            "staging": {
              "env": {
                "my-var": "staging-value"
              }
            }
          }
        }
      }
    }

Does this work for you?

I will also update the docs for the cypress builder. (note for myself: yarn documentation)

I need ignoreTestFiles and I will also add testFiles, but I think that's it - in my opinion there is no need for other values to be overwritten. What do you think?

[skydever]

... actually, there is the spec option already - I was able to handle my use case with it. I think there is no option left that needs to be added? 🤔

it seems like only the docs need an update:

• maybe a clear info about cypress.json for cypressConfig, that it has to be named cypress.json
• add the missing env option UPDATE: see this line concerning the script to autogenerate the docs. env is an object
• the spec option seems to have a formatting problem with the *'s at the example

@vsavkin @FrozenPandaz I tried to run yarn documentation as mentioned here but the output was missing the env option, and I got many new lines that did not exist previously (in the generated markdown files) - I guess because I am on Windows?

[prudhvidandamudi]

Hi guys
any update on this bug, I am in the same situation where I need to invoke different cypress.json files based on the environment.

I guess this fix should also help to solve my problem

https://stackoverflow.com/questions/58440173/how-to-set-environment-variables-in-cypress-json-file-through-command-line-ang

[skydever]

hi @prudhvidandamudi

doesn't it work to set the different environment settings in angular.json like described above and then run ng e2e your-app --configuration=staging?

[prudhvidandamudi]

@skydever
Thank you, I didn't check your last message. It worked like magic.

[pascalbe-dev]

Hey there,
any update here?

I need ignoreTestFiles and I will also add testFiles, but I think that's it - in my opinion there is no need for other values to be overwritten. What do you think?

This would be really great. Is there any workaround for now? I don't think I can set the testFiles via an env variable, right?

[skydever]

Hi @pascalbe-dev

I am using the spec option to handle such use cases. I think there is no need for testFiles and ignoreTestFiles, no actions from my side ...

[pascalbe-dev]

Ahh. This works perfectly fine for my use case aswell.
I missed the spec option in your previous comment. Sorry about that. Thanks for your quick response.

[FrozenPandaz]

This now works! Fixed by #2460

[wyau7]

Hello, I'm new to Cypress and am not sure how I use the spec option in the Cypress Builder to allow me to execute my script based on settings per environment. Can someone provide assistance here?

[wyau7]

also, I can't seem to pass in "env" properties via angular.json