Artillery: Bug: Getting ECONNRESET error when running simple test with http loop

[NivLipetz]

Ever since the pull request https://github.com/artilleryio/artillery/pull/557 merged into master, simple tests with configs that have http.pool configured are failing with error: Error: socket hang up: ECONNRESET

Test file:

{
    "config": {
        "target": "https://httpstat.us",
        "http": {
            "pool": 3
        },
        "phases": [{
            "duration": 1,
            "arrivalRate": 3
        }]
    },
    "scenarios": [{
        "name": "Get response code 200",
        "flow": [{
            "get": {
                "url": "/200",
                "forever": true
            }
        }]
    }]
}

Results:

[hassy]

Thanks for the bug report @NivLipetz! I can confirm this is a bug, and a fix is incoming for the next release.

[NivLipetz]

My pleasure. Something to consider when you are looking at a fix for this bug is the version of request.js npm package. When running artillery tests using request.js version 2.87, I could reach 1500 RPS testing a simple server that returned only 200 locally on my machine. However, when I used the most up to date version of request.js, 2.88, I could reach only ~750 RPS. I ran these tests with a pool of 100 sockets where I let request.js handle the logic of the sockets (basically by reverting the commit that introduced this bug https://github.com/artilleryio/artillery/issues/621)

It's important to take into account this significant difference in performance between the two request.js versions

[bmvalente]

@hassy: any date on the next release? thank you

[NivLipetz]

@bmvalente This issue has been fixed here: https://github.com/artilleryio/artillery/commit/33128406a2d5782d0e74b10943ee7a5233187e2a
Correct me if i am wrong.
Thanks @hassy !

[hassy]

thanks for the bug report @NivLipetz; re performance differences between request.js versions -- that's very interesting and something to look into, I'll create a separate ticket for it