Newman: Content length not set in raw mode

[chandruxp]

when using postman generated collections with Post http method and raw body (json data), the content length is not set automatically in Newman. But in postman and runner, it is set. So exported collections don't work in Newman. Unless, of course, you set the content length explicitly in postman before exporting.

[RangerDan]

I'm running into something like this, but I'm unable to get around it by setting the content length. Every request is timing out with ETIMEDOUT:

RequestError: [223f1c83-1bb6-b40c-acc7-d90a2dd4e4ce] 'HB Heart Beat' terminated. Complete error:
Error: ETIMEDOUT
at null._onTimeout (~/.nvm/versions/node/v0.12.9/lib/node_modules/newman/node_modules/request/request.js:808:15)
at Timer.listOnTimeout (timers.js:119:15)

I've included more detail here: http://stackoverflow.com/q/34502312/1985310?sem=2

[abhijitkane]

@RangerDan To configure the timeout for newman, use the --requestTimeout option (https://www.npmjs.com/package/newman)

[RangerDan]

I did that previously and it made no difference.

[abhijitkane]

What's the timeout that you're specifying?

[czardoz]

@RangerDan, can you share the exact command you used?

[RangerDan]

I appreciate the help. I know there was a bug fixed in how newman handles raw content in 1.1.12, and this seemed familiar so that's why I latched onto this thread.

The timeout was tried between 10 and 10000ms. The timeout appears to work. Short times come back immediately with the timeout error. Longer ones take longer. Here's the command I use to launch my collection:

newman -c Test.json.postman_collection --requestTimeout 10000

Specifically, I'm trying to hit a server with a raw body. The Content-type has been tried as raw (no content-type), text/xml, or application/xml. The Content-Length was tried as set or not. I've attached the simplest test collection I've tried. Other collections work just fine when they are not raw.

Test.json.postman_collection.txt

I cannot see any incoming connections to the remote server which makes me think it is never attempting to reach out from node/newman before timing out. Newman is v1.2.27. I've tried Newman using Node v5.3 and rolled back to v0.12 recently to give it a shot. The issue affects both. I manage my node versions using nvm. Newman is installed globally. The base OS is Fedora.

[czardoz]

Can you try specifying the charset in your Content-Type header?

Something like:

application/xml; charset=utf-8

Is there any load balancer or proxy between Newman and the Server?

[RangerDan]

The charset had no effect.

The two machines are on the same subnet. I can hit it with the identical request through command-line curl. I also hit the server with node-libcurl and got a response. Something is up with newman as far as I can tell.

[RangerDan]

Unlike what this bug says, the content length is being set by newman for raw bodies. Perhaps migration to a different library for handling requests fixed this issue?

Regarding my issue: It appears the server I am connecting has case-sensitive header comparison. I was able to grab the requests off the wire using tcpdump and compare them. Postman sends 'Content-Length' and Newman sends 'content-length'. The server wasn't handling length zero content correctly. Thanks to everyone who chimed in helping me troubleshoot.

I know the server itself shouldn't care (RFC 2616: 4.2). As the behavior is inconsistent between Postman and Newman, is there a chance of getting the library used between the two solutions modified so it is more consistent?

[czardoz]

@RangerDan, Postman uses Chrome's xhr and Newman uses Node's request library. I'll see if there are ways to keep header case consistent :)

[DaisySu]

@czardoz Same here. Is there one solution for this issue?

[czardoz]

@DaisySu, does your server require case sensitive headers?

[DaisySu]

@czardoz Thanks for your reply. I have resolved it with "-k" in the command.

[johnmckinzie]

@czardoz I'm having a similar issue. The collection runs fine in Postman, but I get a We are in the prototyping phase and not using SSL/TLS, so the "-k" did not solve our issue.

RequestError: [68dddcdb-c48c-b411-5d17-2ac73457b85f] 'Tests' terminated. Complete error: 
Error: ESOCKETTIMEDOUT
    at ClientRequest.<anonymous> (/usr/lib/node_modules/newman/node_modules/request/request.js:825:19)
    at ClientRequest.g (events.js:273:16)
    at emitNone (events.js:80:13)
    at ClientRequest.emit (events.js:179:7)
    at Socket.emitTimeout (_http_client.js:582:10)
    at Socket.g (events.js:273:16)
    at emitNone (events.js:80:13)
    at Socket.emit (events.js:179:7)
    at Socket._onTimeout (net.js:328:8)
    at tryOnTimeout (timers.js:224:11)

The collection are calling some very small services. I upped the time out to 30 seconds, but that did nothing. The API returns in under a second through Postman and browser.

[willrichboa]

Is it possible to do better error handling on these request timeout errors. They are not captured by -O. The request and response just are skipped entirely in that report. -t and -H both record if an assigned test fails, but show a statusCode.code of "0" and a responseBody of "undefined". Which is widely unhelpful.

I set a -r 10000 and had to look at my Jenkins console log to find this error:
RequestError: [c2efcd5d-cf46-7ed6-ee3a-ff6abe333d29] 'requesttName' terminated. Complete error:
Error: ETIMEDOUT
at null._onTimeout (/scratch/jenkins/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/Newman_install/lib/node_modules/newman/node_modules/request/request.js:808:15)
at Timer.listOnTimeout (timers.js:92:15)

[rodgeve]

I solved my problem, I was getting ESOCKETTIMEOUT errors from Newman for the same (GET) request that was succeeding in Postman. After comparing the Postman requests to Newman and an equivalent curl command in Wireshark I determined the difference to be the 'Connection' header values. I'm using Newman with Node.js v5.x (the same problem is present in Node.js v4.x).

The fix:

1. Set the 'Connection' header for the (GET) request that is failing in Newman to the value 'keep-alive' in the Postman collection.
2. Re-export your collection for running in Newman

This seems to be a possible bug in Node.js handling of the Connection header value in the GET request.

[davenewza]

@rodgeve @johnmckinzie I'm having the same ESOCKETTIMEOUT issue with newman using Node.js 4.x. I've tried the 'Connect' header but still the problem persists. This only appears to be happening on GET requests.

[davenewza]

Working! The header is 'Connection', not 'Connect'.

[rodgeve]

Thanks for the correction. I've updated my post.

On Tue, Apr 19, 2016 at 9:53 AM, Dave [email protected] wrote:

Working! The header is 'Connection', not 'Connect'.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/postmanlabs/newman/issues/249#issuecomment-211931250

[shamasis]

We need to re-examine this on our v3.x beta branch.

[xs2bharat]

I am still getting error after adding "Conenction" header 'keep-alive'. Error is coming POST request while running newman v 2.1.2 from jenkins.

[39mError: connect ETIMEDOUT
at Object.exports._errnoException (util.js:1007:11)
at exports._exceptionWithHostPort (util.js:1030:20)
at TCPConnectWrap.afterConnect as oncomplete
 × Status code is 200

[kunagpal]

@xs2bharat Could you re-try with the latest version of Newman?

[kunagpal]

Closing issue on our end due to inactivity. Please let us know if this still persists.

[johndoethe1st]

I just installed version 5.1.3 and it seems this is still a current issue. When I send out a POST request with raw data then I don't get a 'Content-Length' header automagically. If I already create it but leave it empty the value is not being updated. When I try to manually set the value I get nothing but timeouts.

[prakharjoshi]

@johndoethe1st can you confirm the Newman version newman -v? Also, can you share the collection and environment files (excluding the sensitive data) for which you are getting the error.

You mentioned version 5.1.3, I am assuming this is the postman App version :).

[prakharjoshi]

Closing issue on our end due to inactivity. Please let us know if this still persists.