Cypress: Random failures over Regexp: Invalid regular expression

[houshuang]

I get very strange non-deterministic errors when opening my website in Cypress. It complains about a failing Regexp, but the weird thing is that this never causes a problem in a normal browser, and in fact, even within the Cypress browser, if I open a new tab and reload the code, it works perfectly. And I also added on Cypress.on('uncaught.exception') to my code, which does not seem to help.

Here's my test script:

const uuid = require('cuid');
Cypress.on('uncaught:exception', (err, runnable) => {
  return false;
});

describe('My First Test', function() {
  it('finds the content "type"', function() {
    const id = uuid();
    // cy.wait(30000 * Math.random());
    cy.visit(`https://icchilisrv3.epfl.ch:7000/w948?login=${id}`);

    cy.get('video', { timeout: 90000 });
    cy
      // .wait(10000)
      .then(() => cy.wrap(Cypress.$('video')[0].networkState).should('eq', 2));
    cy.wait(180000);
  });
});

Here you see two tabs of the same browser, consistently showing completely different behaviour (even when reloading etc). What's very strange is that when I deploy this in 30 different browsers using Kubernetes, I get half of them failing with this error, and the other half working normally! Perhaps this is a timing issue (the reason for the non-determinism) with some interaction between my code and Cypress test code?

Here you see examples of failing and successful Cypress runs - these are exactly the same code in different Kubernetes containers:

(in this case, I don't get the error exceptions, because of the uncaught.exception setting, but the behaviour is exactly the same as in interactive mode, making me believe that the problem is identical).

Reference to Cypress dashboard run

[jennifer-shehane]

We've seen this Invalid regular expression before, so I suspect it is some situation on our part. It needs investigating.

[rufushonour]

@houshuang Not sure if you fixed this issue or not but I've managed to get around it by adding "modifyObstructiveCode": false to my cypress.json

I think this might be related to #1396 ?

[Francismb]

I can confirm, adding "modifyObstructiveCode": false to your config fixes the issue.

[Uzlopak]

FakeNews... still getting garbage regex error

[flotwig]

I believe that #4698 might fix this issue - browsers infer the character encoding of subresources (partially) based on the character encoding of the consuming HTML document, so fixing the character encoding of HTML in Cypress should cascade to this.

I tried using the example in the OP, but that website is not available. If anyone has a reproducible example of this happening, I'll test it against #4698 to see if this fixes the problem.

[SeanBE]

Ran this example https://github.com/cypress-io/cypress/issues/2112#issuecomment-510899430 with the #4698 build and I still get invalid regular expression.

[flotwig]

@SeanBE This doesn't seem like an encoding issue, rather, a browser compatibility error. When I add that RegEx, the app no longer works in Firefox:

It's because you're using negative lookbehind in the RegEx:

• Firefox doesn't support negative lookbehind
• Chrome >=62 does, which is why it works when you use cypress open with Chrome 75
• The version of Chrome bundled with Cypress (61) does not support negative lookbehind, which is why it fails in cypress run.

Read more about the development of this feature.

Try using cypress run with --browser=chrome, you'll find that it does work now because you're using the Chrome bundled with your system.

---

We're working on upgrading the Chrome/Electron version bundled with Cypress: #3568 For now, you'll need to run tests in Chrome to support negative lookbehind, or use some other method of pattern-matching to support your needs.

---

I am still looking for a way to reproduce the issue in the OP, which is a character encoding corruption causing an invalid RegEx.

[Halt001]

I have a similar issue where unicode characters in vendor.js are not properly encoded. Problem with creating a repo to reproduce this is that even renaming the project path is enough to get it to behave properly.

So I have a repo which can reproduce the problem but only on my machine and only on the exact same path. Creating a new test project on that exact same path also produces the error so it must be caused by something outside the project directory.

Installing a previous version of Cypress (3.2.0) fixes the problem.
Setting "modifyObstructiveCode": false also fixes the problem.

Is it possible that there is something cached somewhere that uses the project path as key?

[flotwig]

@Halt001 Are you using 3.4.1? That release contained some fixes for character encoding.

Is your Javascript UTF-8-encoded? What headers is it being sent with from your web server?

[Uzlopak]

I have to correct myself. It did fix our issue.

Best Regards

[Halt001]

Yes, I'm using 3.4.1

The Javascript file I'm talking about is the vendor.js file created by Angular, and it contains UTF8 characters.

Request headers as seen in the Chrome instance started by Cypress:

GET http://localhost:4200/vendor.js HTTP/1.1
Host: localhost:4200
Proxy-Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Sec-Fetch-Mode: no-cors
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
Accept: */*
Sec-Fetch-Site: same-origin
Referer: http://localhost:4200/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

Response headers as seen in the Chrome instance started by Cypress:

HTTP/1.1 200 OK
x-powered-by: Express
access-control-allow-origin: *
accept-ranges: bytes
content-type: application/javascript; charset=UTF-8
etag: W/"3d0d98-V90I6hrR5NqasgimxMHD6dWanEk"
date: Mon, 12 Aug 2019 16:44:56 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Connection: keep-alive
Transfer-Encoding: chunked

After renaming my project directory the problem goes away. For completeness I'll copy the headers below for this working situation.

Request headers as seen in the Chrome instance started by Cypress when it all works:

GET http://localhost:4200/vendor.js HTTP/1.1
Host: localhost:4200
Proxy-Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Sec-Fetch-Mode: no-cors
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
Accept: */*
Sec-Fetch-Site: same-origin
Referer: http://localhost:4200/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

Response headers as seen in the Chrome instance started by Cypress when it all works:

HTTP/1.1 200 OK
x-powered-by: Express
access-control-allow-origin: *
accept-ranges: bytes
content-type: application/javascript; charset=UTF-8
etag: W/"3d12ae-UQQQNJC5AvB8VbY23IHRMFJdox8"
date: Mon, 12 Aug 2019 17:00:07 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Connection: keep-alive
Transfer-Encoding: chunked

[flotwig]

What is the name of your project folder when it works, and when it is broken?

[Halt001]

/Users/Tim/Dev/bagger/Nx/myorg is the path where it doesn't work.

Rename any part of the path, (not necessarily the myorg part) and everything works fine.

As there is nothing special about this path and throwing the project away and recreating it from scratch in the exact same location causes the same problem again I suspect that Cypress uses something that is cached beyond the life of the project and also outside the project folder.

The project is the Nx dev tutorial until step 2: Tutorial.

This is the Repo that contains the tutorial on my machine in the state that it fails. But I'm pretty certain it won't fail anywhere else because as I said, the problem seems to be bound to a specific path. (Run e2e with: ng e2e todos-e2e --watch ornpm run e2e -- --watch)

Also note that when the server is running and Cypress is failing, opening another Chrome browser manually and opening the same url shows that vendor.js is then correctly served without the utf8 problem.

Edit:
The problem occurs in line 52438 of vendor.js
const interpolatedValue = ��ɵinterpolation5(prefix, v0, i0, v1, i1, v2, i2, v3, i3, v4, suffix);
When everything works it is:
const interpolatedValue = ɵɵinterpolation5(prefix, v0, i0, v1, i1, v2, i2, v3, i3, v4, suffix);

[Halt001]

Very interesting observation: It is not the value of the project path that makes the difference but the length. I can rename al I want but as long as it stays the same length it will fail, adding or removing characters in the path will make it work.

So this rules out my theory of the path being some kind of key in a cache but it may be possible to reproduce on other machines.

Edit:
I can confirm that this problem is now reproducible on another machine with above mentioned repo as long as the project path is exactly the same length.

The path name is also in the file contents so the path length impacts the location of the UTF8 characters in vendor.js. This is a long file, could it be that something goes haywire when chunking this file, maybe a byte count vs character count goes wrong when calculating a new chunk?

[jpike88]

Just ran into this issue.

"modifyObstructiveCode": false" fixes the issue. I encourage the cypress team to treat this issue as more of a priority, as it appears to be breaking core Angular code.

[CSchulz]

We encountered the same issue on nearly all developer machines.

Generated javascript assets of Angular gets encoding issues with UTF-8 signs like ɵ sometimes.
https://puu.sh/DOFhM/1dd12f8b4a.png
(credits for the screenshot to @alfaproject)

Any ETA when #4984 will be merged?

[cypress-bot[bot]]

The code for this is done in cypress-io/cypress#4984, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[cypress-bot[bot]]

Released in 3.5.0.