Testcafe: Browser get's disconnected when using testcafe remote inside docker

[rafalsep]

What is your Test Scenario?

We're running Create React App inside a docker container. Adding testcafe and trying to run the tests against local environment fails with error message The following browsers disconnected: Chrome 79.0.3945.130 / macOS 10.14.5. Tests will not be run.
Running the same tests outside of docker works without any issues.

What is the Current behavior?

Tests doesn't run because browser get's disconnected.

What is the Expected behavior?

Tests should run inside docker the same way as without it.

What is your web application and your TestCafe test code?

The issue can be easily reproduced with Create React App and trying to run simple testcafe test against it inside docker.
Please find the sample project under this url

Your website URL (or attach your complete example):

https://github.com/rafalsep/testcafe_docker_remote_issue

Your complete test code (or attach your test files):

 fixture('test').page(`http://localhost:3000/?showLogin`);

test('initial', async t => {
  await t.expect('ok').eql('ok');
});

Your complete configuration file (if any):

.testcaferc.json

{
  "src": ["./tests/**/index.js"],
  "port1": 5080,
  "port2": 5081,
  "hostname": "localhost"
}

docker-compose.yaml

version: "3.1"

services:
  ui:
    container_name: testcafe-test
    image: test/testcafe-test
    restart: always
    volumes:
      - .:/usr/app
    environment:
      PORT: 3000
      HOST: 0.0.0.0
    ports:
      - 3000:3000
      - 5000:5000
      - 5080:5080
      - 5081:5081

Dockerfile

FROM node:12.14.0
WORKDIR /usr/app
EXPOSE 3000
EXPOSE 5080
EXPOSE 5081
EXPOSE 5000
CMD ["tail", "-f", "/dev/null"]

Your complete test report:

root@c00421a22268:/usr/app# npm run e2e

> [email protected] e2e /usr/app
> testcafe remote

Connecting 1 remote browser(s)...
Navigate to the following URL from each remote browser.
Connect URL: http://localhost:5080/browser/connect
CONNECTED Chrome 79.0.3945.130 / macOS 10.14.5
ERROR The following browsers disconnected: Chrome 79.0.3945.130 / macOS 10.14.5. Tests will not be run.

Type "testcafe -h" for help.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] e2e: `testcafe remote`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] e2e script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-02-27T10_24_27_777Z-debug.log

Screenshots:

Steps to Reproduce:

1. Git clone example repo from https://github.com/rafalsep/testcafe_docker_remote_issue
2. Follow reproduction steps from README.md

Your Environment details:

• testcafe version: 1.8.2
• node.js version: 12.14.0
• command-line arguments: testcafe remote
• browser name and version: Chrome 79.0
• platform and version: macOS 10.14.5
• other:

[Dmitry-Ostashev]

Thank you for your report and complete example. I've reproduced this issue. We'll investigate it and notify you as soon as we have any results.

[nelhu]

I run into this issue and 100% same with rafalsep

[nelhu]

I run into this issue and 100% same with rafalsep

@Dmitry-Ostashev could you please help with this question?

[zodman]

The same happened on WSL with remote
i run

npx testcafe -L --ports 8000,8000 remote test1.js

the client gets disconnected.

But after a time it fix without doing nothing. (RARE!!!), here the output

npx testcafe remote  test1.js
Connecting 1 remote browser(s)...
Navigate to the following URL from each remote browser.
Connect URL: http://192.168.56.1:8000/browser/connect
CONNECTED Chrome 80.0.3987.149 / Windows 10
ERROR The following browsers disconnected: Chrome 80.0.3987.149 / Windows 10. Tests will not be run.

// test1.js
import { Selector } from 'testcafe';

fixture `Getting Started`
    .page `http://devexpress.github.io/testcafe/example`;

test('My first test', async t => {
    // Test code
});

when i run with DEBUG=* here the log:

http://paste.ubuntu.com/p/2DGKR4JGBM/

There are some kind of recursion i seed.

when i run with live mode:

Stopping TestCafe live mode...

Make changes to the source files or press Ctrl+R to restart the test run.
TypeError: Cannot read property 'tests' of null
    at LiveModeRunner._finishPreviousTestRuns (/home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:149:38)
    at LiveModeRunner.runTests (/home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:36:21)
    at /home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:111:30
ERROR The following browsers disconnected: Chrome 80.0.3987.149 / Windows 10. Tests will not be run.

[nelhu]

The same happened on WSL with remote
i run

npx testcafe -L --ports 8000,8000 remote test1.js

the client gets disconnected.

But after a time it fix without doing nothing. (RARE!!!), here the output

npx testcafe remote  test1.js
Connecting 1 remote browser(s)...
Navigate to the following URL from each remote browser.
Connect URL: http://192.168.56.1:8000/browser/connect
CONNECTED Chrome 80.0.3987.149 / Windows 10
ERROR The following browsers disconnected: Chrome 80.0.3987.149 / Windows 10. Tests will not be run.

// test1.js
import { Selector } from 'testcafe';

fixture `Getting Started`
    .page `http://devexpress.github.io/testcafe/example`;

test('My first test', async t => {
    // Test code
});

when i run with DEBUG=* here the log:

http://paste.ubuntu.com/p/2DGKR4JGBM

There are some kind of recursion i seed.

when i run with live mode:

Stopping TestCafe live mode...

Make changes to the source files or press Ctrl+R to restart the test run.
TypeError: Cannot read property 'tests' of null
    at LiveModeRunner._finishPreviousTestRuns (/home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:149:38)
    at LiveModeRunner.runTests (/home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:36:21)
    at /home/zodman/wor/cds-integration-testing/node_modules/testcafe/src/live/test-runner.js:111:30
ERROR The following browsers disconnected: Chrome 80.0.3987.149 / Windows 10. Tests will not be run.

I wish if a team member from testcafe would resolve this issue

[aleks-pro]

Hello @nelhu ,

I run into this issue and 100% same with rafalsep

We will research the problem and update this thread as soon as we have any news.

[aleks-pro]

Hello @zodman ,

The same happened on WSL with remote
...

Thank you for sharing your scenario with us. We will take it into account.

[nelhu]

Hello @nelhu ,

I run into this issue and 100% same with rafalsep

We will research the problem and update this thread as soon as we have any news.

how is it now? [/smile]

[AlexanderMoiseev]

Hello,
This issue has not been researched yet. Continue to follow this issue.

[Kuzirashi]

Hello! More and more developers jump on WSL environment, even some used-to-be Linux fanboys (like me). I can confirm the issue on trying to run WSL via: testcafe --live remote tests/test-example.js gives me exactly the same error.

[alexey-lin]

@Kuzirashi Thank you for the information. We will take it into account.

[ovidiuc]

Hi,

I have a different configuration (testcafe runs in Debian VM on Vmware, with browser on Windows 10 host), but I was getting the same error as @zodman and @nelhu, with the same test and command, so I've researched the issue and found the bug.

I think it might affect all remote attempts.

The actual error is:

NativeBinaryHasFailedError: The /home/web1/.config/yarn/global/node_modules/testcafe-browser-tools/bin/linux/glibc-64/find-window process failed with the 101 exit code.
    at Object.execFile (/home/web1/.config/yarn/global/node_modules/testcafe-browser-tools/src/utils/exec.js:132:15)
    at BrowserConnection._runBrowser (/home/web1/.config/yarn/global/node_modules/testcafe/src/browser/connection/index.ts:152:10) {
  code: 'E1038',
  data: [
    'remote:',
    'NativeBinaryHasFailedError: The /home/web1/.config/yarn/global/node_modules/testcafe-browser-tools/bin/linux/glibc-64/find-window process failed with the 101 exit code.\n' +
      '    at Object.execFile (/home/web1/.config/yarn/global/node_modules/testcafe-browser-tools/src/utils/exec.js:132:15)'
  ]
}

Was able to get it by editing /browser/connection/index.ts. Add err as param and console.log it.

In the file /src/browser/provider/built-in/remote.js, the variable canDetectLocalBrowsers is set to true by default, but never set to false (which would prevent the browser open attempt).

There is an attempt to do it on /src/runner/bootstrapper.ts, but that code never gets to run. remote.js runs before bootstrapper.ts#L420, which is only for Mac anyway.

The bug was introduced in commit https://github.com/DevExpress/testcafe/commit/83eafed7a2fd97f0c24d87866a9cc97185d6e712 , so downgrading to version 1.6.1 is a workaround for me. It would be great if people with other configurations (such as Docker or WSL) are able to confirm this.

Edit: I see the problem was reported before in #4900

[AlexKamaev]

@ovidiuc
It seems that your issue is related to https://github.com/DevExpress/testcafe/issues/4652 and https://github.com/DevExpress/testcafe/issues/4742
Please refer to these issues for details.

The cause of the issue is that you try to use a non-headless browser with TestCafe running in a Linux system without a GUI.

If you start the browser in the same Docker image as TestCafe, it's always better to start tests in headless mode.

If you connect a remote browser to a Linux-like system (including WSL), you can install and run Xvfb in your host or container as a workaround.

[ovidiuc]

@AlexKamaev maybe I'm misunderstanding how Testcafe is supposed to be working.

But shouldn't the remote flag prevent any automatic attempts to open a browser window? Headless or not. Since the browser is on a remote machine.

This is also what I understand from the documentation. https://devexpress.github.io/testcafe/documentation/recipes/basics/test-on-remote-computers-and-mobile-devices.html There is no mention of the fact that the browser type is taken into consideration.
Thanks.

[AndreyBelym]

@ovidiuc TestCafe doesn't try to open a browser window, it tries to find a native handle of a browser window. It is required to provide functionality such as screenshot capturing and window resizing. TestCafe does it since some users tried to start browsers on the same machine with TestCafe, connected them as remotes, and expected full functionality.

As described in #4742, we use the findWindow function from the testcafe-browser-tools package to look up a native window handle. The 2.0.0 version of this function started throwing exceptions instead of returning null if a handle cannot be found due to missing X11 subsystem on Linux. We didn't modify our code properly to cover this change, but definitely implement correct handling in a future release.

It's true that the workaround with headless mode won't help in your case. I didn't explain well the situation when discussed the issue with @AlexKamaev. But I now I have updated his answer to cover more possible cases.

If you start the browser in the same Docker image as TestCafe, it's always better to start tests in headless mode.

If you connect a remote browser to a Linux-like system (including WSL), you can install and run Xvfb in your host or container as a workaround.

Since #4742 caused issues reported in this thread, I close this issue as a duplicate. Please use #4742 to track and discuss our progress in these issues.

[lock[bot]]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.

[lock[bot]]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.