Testcafe: Allow configurable timeouts for page and XHR requests
Are you requesting a feature or reporting a bug?
Bug
What is the current behavior?
Any XHR that takes longer than 20 seconds will make tests depending on parsing the response fail
What is the expected behavior?
Either the page load ltimeout or another timeout should be available to make testcafe not fail when XHR take longer than 20 seconds
How would you reproduce the current behavior (if this is a bug)?
Run the below test which loads twp page that retrieve a simple JSON from a remote service using XHR. The first page XHR response returns in less than 1 sec. The second page XHR response returns in 25 seconds. The first succeeds but the second fails because testcafe gives up on XHR that take over 20 seconds.
import { Selector } from 'testcafe';
fixture `Testcafe XHR support POC`
test('the label jsonDisplay should show up the fast XHR response', async t => {
// The below succeeds
await t.navigateTo('http://www.nestorurquiza.com/xhr.html');
const jsonDisplay = Selector('#jsonDisplay').addCustomDOMProperties({
innerHTML: el => el.innerHTML
});
await t.expect(jsonDisplay.visible).ok();
await t.expect(jsonDisplay.innerHTML).contains('hello');
});
test('the label jsonDisplay should show up the slow XHR response', async t => {
// The below fails
await t.navigateTo('http://www.nestorurquiza.com/slowXHR.html');
const jsonDisplay = Selector('#jsonDisplay').addCustomDOMProperties({
innerHTML: el => el.innerHTML
});
await t.expect(jsonDisplay.visible).ok();
await t.expect(jsonDisplay.innerHTML).contains('hello');
});
Using the following command the test ends with the below error:
$ ./node_modules/.bin/testcafe --selector-timeout 60000 --assertion-timeout 60000 --page-load-timeout 60000 chrome tests/slow-xhr.test.js
Running tests in:
- Chrome 69.0.3497 / Linux 0.0.0
XHR
✓ the label jsonDisplay should show up the fast XHR response
✖ the label jsonDisplay should show up the slow XHR response
1) AssertionError: expected 'Result of request should appear here ...' to include 'hello'
Browser: Chrome 69.0.3497 / Linux 0.0.0
17 | await t.navigateTo('http://www.nestorurquiza.com/slowXHR.html');
18 | const jsonDisplay = Selector('#jsonDisplay').addCustomDOMProperties({
19 | innerHTML: el => el.innerHTML
20 | });
21 | await t.expect(jsonDisplay.visible).ok();
> 22 | await t.expect(jsonDisplay.innerHTML).contains('hello');
23 |});
24 |
at contains (/home/nurquiza/workspace/lms-e2e/tests/slow-xhr.test.js:22:41)
1/2 failed (1m 05s)
Note that the command sets to 1 minute all the available timeouts. Optionally and for the record, see below the html of this page in case you want to host it yourself. Change the mocky-delay param to have a page to render in 1ms and another in 25s (If you host it you will need to either have the XHR service in the same sub-domain or provide CORS support for the remote service):
<html>
<head>
<title>XHR example</title>
</head>
<body>
<label id="jsonDisplay">Result of request should appear here ...</label>
<script>
const url = 'https://www.mocky.io/v2/5185415ba171ea3a00704eed?mocky-delay=1ms';
const jsonDisplay = document.querySelector('#jsonDisplay');
fetch(url).then(function(response) {
response.text().then(function(text) {
jsonDisplay.innerHTML = text;
});
});
</script>
</body>
Provide the test code and the tested page URL (if applicable)
See above. Note that this is an issue that will result in tests suddenly not passing after a release just because the responses from the database are slower for instance. The test failure in that case becomes a distraction, a false positive.It would be ideal to have a configurable timeout for XHR responses or use the same page load timeout to fix the currently hardcoded 20 seconds landmark.
Specify your
- operating system:
- testcafe version:
- node.js version:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
$ ./node_modules/.bin/testcafe --version
0.20.3
$ node --version
v8.9.3
nestoru
All 13 comments
I was able to reproduce the issue and prepared an example without calling any external sites' API.
Test page:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="jsonDisplay"></div>
<script>
const url = 'http://127.0.0.1:8081?test=1';
const jsonDisplay = document.querySelector('#jsonDisplay');
fetch(url).then(function(response) {
response.text().then(function(text) {
jsonDisplay.innerHTML = text;
});
});
</script>
</body>
</html>
Test code
import { Selector } from 'testcafe';
const DestinationRequest = require('../../../../../../node_modules/testcafe-hammerhead/lib/request-pipeline/destination-request');
const http = require('http');
fixture `fixture`;
test('test', async t => {
// DestinationRequest.TIMEOUT = 60000;
const server = http.createServer((req, res) => {
res.writeHead(200, {
'Content-Type': 'text/json',
'Access-Control-Allow-Origin': '*',
});
setTimeout(() => {
res.write('hello');
res.end();
}, 40000);
});
server.listen(8081);
await t.navigateTo(`http://127.0.0.1:8080/index.html`);
await t.expect(Selector('#jsonDisplay').visible).ok();
await t.expect(Selector('#jsonDisplay').textContent).contains('hello');
// DestinationRequest.TIMEOUT = 25000;
});
You are right; the issue occurs when the server doesn't respond for a long period. This constant is defined in the testcafe core module testcafe-hammerhead https://github.com/DevExpress/testcafe-hammerhead/blob/master/src/request-pipeline/destination-request/index.js#L185
There is no public option to modify it, but it still can be modified. To make the example work, please uncomment all lines in the test file.
AlexKamaev
on 8 Oct 2018
I've discussed the issue with my colleagues and we decided that would be useful to have public options to set the timeout, so I'll mark the issue as proposal.
AlexKamaev
on 8 Oct 2018
@AlexKamaev Thanks for the quick turnaround. Really appreciated.
nestoru
on 8 Oct 2018
One more request (TestCafe Studio): https://www.devexpress.com/Support/Center/Question/Details/T726255/test-failures-failed-to-complete-a-request-within-the-timeout-period
Farfurix
on 22 Mar 2019
Any update on this at all? Very keen to see this implemented.
cericoli
on 17 May 2019
@cericoli we haven't decided about the interface that can be used to modify such intrinsic values. We don't want to overwhelm regular users with excessive configuration details. Maybe we can introduce a section in the configuration file structure. But this problem still needs o lot of planning and development, so I can't give you any estimates.
AndreyBelym
on 20 May 2019
Team, are we going to have resolutions on that?
zhkostadinov
on 21 Jan 2020
Currently, we do not have any news regarding this.
AlexKamaev
on 22 Jan 2020
In this case, any workaround is highly appreciated.
zhkostadinov
on 22 Jan 2020
@zhkostadinov,
Did you try Alex' workaround from the first comment?
alexey-lin
on 23 Jan 2020
The link which is posted leads to Page not found.
I suppose, he mentioned increasing the _defineProperty(DestinationRequest, "XHR_TIMEOUT", 2 * 60 * 1000); in file testcafe-hammerhead/lib/request-pipeline/destination-request/inex.js will solve that?
I already change the 2 * 60 * 1000 part to 200 * 60 * 1000, but it doesn't help me.
I'm using cli --assertion-timeout 150000, but it fails on the ~1 minute with that error.
The interesting thing is that, the error is observed when the test is running in parallel on Chrome(latest) and Firefox(latest). When I run the same tests only in Chrome, for example from 10 runs 3 will fail, the other will succeed.
zhkostadinov
on 23 Jan 2020
@zhkostadinov
Hello,
Yes, you can change these timeouts:
lib/request-pipeline/destination-request/index.js
_defineProperty(DestinationRequest, "TIMEOUT", 25 * 1000);
_defineProperty(DestinationRequest, "XHR_TIMEOUT", 2 * 60 * 1000);
I'm using cli --assertion-timeout 150000, but it fails on the ~1 minute with that error.
The assertion timeout increase cannot fix the issue with the request.
Could you please share your simple project (or a public URL), so that we can reproduce the issue? We will examine it and check for a suitable solution.
Farfurix
on 24 Jan 2020
Hello @Farfurix , thanks for the reply. Unfortunately, I'm not able to share that information, because it is for internal usage, and doesn't have a public part.
zhkostadinov
on 24 Jan 2020
Related issues
Turkirafaa
·
3Comments
inikulin
·
3Comments
calisven
·
3Comments
darkowic
·
3Comments
AndreyBelym
·
3Comments
Most helpful comment
Any update on this at all? Very keen to see this implemented.