Testcafe: Testcafe and Webpack cause an ECONNRESET

[Neatoro]

Are you requesting a feature or reporting a bug?

Reporting a bug

What is the current behaviour?

After updating my Webpack and Testcafe version I am getting an error named ECONNRESET. Testcafe runs against a webpack-dev-server an executes the tests, at least some of them. After some time testcafe dies with this error message:

Error: read ECONNRESET
    at TCP.onread (net.js:660:25)
Emitted 'error' event at:
    at Socket.onerror (_stream_readable.js:687:12)
    at Socket.emit (events.js:182:13)
    at Socket.EventEmitter.emit (domain.js:442:20)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
### How would you reproduce the current behaviour (if this is a bug)?

I already found an issue (#2649) but it does not help me. Seems to me like there are sometimes issues with closing a connection when ending a fixture.
I already tried using the Node version v10.5.0 but does not help so far.

What is the expected behavior?

The tests run through.

Specify your

• operating system: macOS High Sierra v10.13.6
• testcafe version: v0.21.0
• node.js version: v10.8.0

[sambegin]

Go the same problem :

I was able to get the log in chrome directly when this happened :

I tried with testcafe version 0.21.1 and rolled back to 0.20.5 and got the same problem (seems to be a free game tho).

Specify your
operating system: macOS High Sierra v10.13.6
testcafe version: v0.21.1 or 0.20.5
node.js version: v10.3.0

Do we have any news on this ?

[MarinaRukavitsyna]

Hello guys.
To check the issue locally, please send us a link to your web page (or a small online example) and test for research.

[sambegin]

I don't have a quick example to give right now, I'm not sure I'll have time to provide it to you today but quick question tho :

With the log I saw in chrome debug tool "Cannot read property body of undefined", does this PR is fixing the problem ?

[sambegin]

@MarinaRukavitsyna
I think you will be able to reproduce the problem easy enough by lunching a small app lunched by webpack (with node 10.5 and webpack 4 in my case).

If you need any information just tell me. I'll provide it to you real quick. We actually rely on testcafe in our CI pipeline and we are stuck right now because of this.

[MarinaRukavitsyna]

@sambegin
Thank you for your clarification. I have reproduced the issue based on your description. We need some additional time to research it.

[DaAitch]

Here are my test results on Ubuntu.
To get a feeling how we use TestCafe:

• in fixture before(async t => ... we do a setup taking about 10s creating DB and starting server
• no concurrency
• using screenshots
• about 8 fixtures, 15 Tests at all
• for this test, I only run one fixture with 9 Tests

Test Results

Result  Node    Debug               Browser         TestCafe
failed  10.7.0  chrome:headless     68.0.3440.75    0.20.0
failed  10.9.0  chrome:headless     68.0.3440.75    0.20.0
failed  10.7.0  chrome              68.0.3440.75    0.20.0
failed  10.9.0  chrome              68.0.3440.75    0.20.0
ok      10.7.0  firefox:headless    61.0.1          0.20.0
ok      10.9.0  firefox:headless    61.0.1          0.20.0
ok      10.7.0  firefox             61.0.1          0.20.0
ok      10.9.0  firefox             61.0.1          0.20.0
failed  10.7.0  chrome:headless     68.0.3440.75    0.21.1
failed  10.9.0  chrome:headless     68.0.3440.75    0.21.1
failed  10.7.0  chrome              68.0.3440.75    0.21.1
failed  10.9.0  chrome              68.0.3440.75    0.21.1
ok      10.7.0  firefox:headless    61.0.1          0.21.1
ok      10.9.0  firefox:headless    61.0.1          0.21.1
ok      10.7.0  firefox             61.0.1          0.21.1
ok      10.9.0  firefox             61.0.1          0.21.1

• error rate of failed was about 80%-100%
• error rate of ok was 0%

Further observations

• it's not a stable issue
• it's more likely to crash with chrome without headless mode
• visually: TCP error only occurred on TestCafe starting page ...x.x.x.x:xyz/about:blank. Whenever it navigates to e.g. ...x.x.x.x:xyz/http://localhost:3000 in the test it continues running through
• TCP error occurs immediately after test-done of the test before at first or second TestCafe command
• can't remember TCP error happening in between a test
• as far as I remember TCP error never occurred at the first tests, but maybe Test 5-8 out of 9
• did maybe 30-40 tests switching browser, node version and test cafe versions: never got a TCP error with firefox
• firefox sometimes get's disconnected to server at startup, when not having focus, but no TestCafe crash

Conclusion

• use Firefox if possible
• we should investigate on the time slot when switching tests only in Chrome

[miherlosev]

Fixed in https://github.com/DevExpress/testcafe/pull/2743

[AndreyBelym]

Hi @sambegin @DaAitch! The fix is released in [email protected], you can install it with npm i testcafe@latest and try it on your machine.

[aszegedi]

Hi,

I'm still getting this issue:

events.js:167
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:111:27)
Emitted 'error' event at:
    at Socket.onerror (_stream_readable.js:690:12)
    at Socket.emit (events.js:182:13)
    at Socket.EventEmitter.emit (domain.js:442:20)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
ERROR: "test-runner" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
TestCafe tests have done

against:

Google Chrome run options: {"path":"/usr/bin/google-chrome","cmd":"--disable-infobars --disable-session-crashed-bubble --no-first-run --new-window --enable-automation --enable-logging --v=1 --no-sandbox --disable-background-networking --disable-software-rasterizer --user-data-dir=\"browser-data/google-chrome\" --disable-web-security --disable-extensions --disable-default-apps --disable-notifications --disable-sync --disable-translate --disable-save-password-bubble --disable-session-crashed-bubble --hide-scrollbars --metrics-recording-only --mute-audio --no-first-run --safebrowsing-disable-auto-update --password-store=basic --ignore-certificate-errors --allow-insecure-localhost","macOpenCmdTemplate":"open -n -a \"{{{path}}}\" --args {{{pageUrl}}} {{{cmd}}}"}
TestCafe test project: e2e
TestCafe run options: {"speed":0.7,"selectorTimeout":80000,"assertionTimeout":120000,"pageLoadTimeout":60000,"skipUncaughtErrors":true}
 Running tests in:

Chrome 69.0.3497 / Linux 0.0.0

Node version:  v10.11.0
Yarn version:  1.10.1
Typescript version:  Version 3.1.1  and the TSC path:  /usr/local/bin/tsc
TestCafe version:  0.22.0

The issue seems the same as reported above.

Could you help me out on this? Should I report this as a new and separate issue?

P.S.:
Our Docker image for executing GUI tests:

hortonworks/cloud-web-testcafe:latest

docker run -i --privileged --rm --net=host --name testcafe-e2e-runner --env-file ./utils/e2e-testenvironment -v /var/lib/jenkins/jobs/gui-e2e-testcafe/workspace/web/****-ui/test:/testcafe/project -v /dev/shm:/dev/shm hortonworks/cloud-web-testcafe:latest yarn test

[LavrovArtem]

The fix is published in 0.22.1-alpha.3.

[aszegedi]

Thanks, Artem! Do you know a release date candidate for 0.22.1?

[LavrovArtem]

We have not established any exact dates yet. Any personal estimate may be misleading, so I cannot currently tell you the precise date.

[cmbirk]

I'm having this same issue with testcafe ^0.23.3 and node v10.15.0. Should I open a new issue to track? @LavrovArtem

[LavrovArtem]

@cmbirk, I see that you've created the new issue. You are right. Let us continue the conversation there.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs or feature requests. For TestCafe API, usage and configuration inquiries, we recommend asking them on StackOverflow.