Artillery: Replacing Request.js with Got

[hassy]

As you probably know Request.js has been officially deprecated since Feb 11, 2020. Request.js is the HTTP library that Artillery uses under the hood.

Work on moving to Got.js is already underway and is expected to complete soon.

Why Got

• Lots of features, see the feature comparison table for details. HTTP/2 support is especially promising.
• Actively developed & maintained.
• Migration looks to be quite straightforward.

When

Soon™️. A preview release should be available in the next few days.

Update 10/05 master is now fully-migrated to Got.js with full-backwards compatibility. The plan is to increase test coverage before marking the work as done.

[hassy]

New version of Artillery's HTTP engine using Got is now on master.

[enudler]

hi @hassy
did you check the performance of this version with got vs previous version of request?
I could get up to 400 rps using pool of 250

using request.js I could hit 1200 rps.

[hassy]

@enudler thanks for the info! I haven't run any benchmarks or profiled the code yet, but it's definitely something we'll look at before releasing the new engine. (Request.js will also still be available as a "legacy" engine for any users relying on Request.js-specific behavior.)

How complex is the scenario that you tested? i.e. are you using capture, custom hooks etc, or was it a simple request-in-a-loop type of scenario?

[enudler]

thanks, @hassy
it was simple get request
I will try to provide an example of the test and compare the vesrion

had also another issue when I received gzip responses
had to use decompress=false in got.js options.

[enudler]

test file:

{
  "config": {
    "http": {
      "pool": 250
    },
    "target": "http://localhost:3001/webhooks",
    "phases": [{
      "duration": 60,
      "arrivalRate": 1000
    }]
  },
  "scenarios": [
    {
      "name": "Scenario 1",
      "weight": 100,
      "flow": [
        {
          "get": {
            "url": "http://localhost:3001/webhooks"
          }
        }
      ]
    }
  ]
}

master:

All virtual users finished

Summary report @ 14:50:38(+0300) 2020-03-31
Total time: 2 minutes, 18 seconds
http.request_rate: ....................... 435/sec avg
http.response_rate: ...................... 435/sec avg
scenarios.created.Scenario 1: ............ 60000
scenarios.created.total: ................. 60000
scenarios.completed: ..................... 60000
http.requests: ........................... 60000
http.codes.200: .......................... 60000
http.responses: .......................... 60000
scenarios.duration:
min: ................................... 2
max: ................................... 469.9
median: ................................ 5
p95: ................................... 9
p99: ................................... 17
http.response_time:
min: ................................... 2
max: ................................... 430.9
median: ................................ 5
p95: ................................... 7
p99: ................................... 13

1.6.0-29
All virtual users finished
Summary report @ 14:23:27(+0300) 2020-03-31
Scenarios launched: 60000
Scenarios completed: 60000
Requests completed: 60000
RPS sent: 914.91
Request latency:
min: 0.2
max: 99.4
median: 1.8
p95: 2.7
p99: 3.9
Scenario counts:
Scenario 1: 60000 (100%)
Codes:
200: 60000

[hassy]

Nice one, thank you @enudler. WRT gzip responses specifically - 100% compatibility is the goal, but not there yet.

[hassy]

@enudler Some work has been done to optimize the performance of the new HTTP engine. I am seeing 50-70% improvements in RPS on master vs v1.6.0-29 with the following simple test:

artillery quick -c 10 -n 5000 http://localhost

http://localhost is nginx serving a static page. The server is a baremetal c1.small.x86 machine from packet.net - https://www.packet.com/cloud/servers/c1-small/ - 4 physical cores @ 3.5 GHz so there's no contention for CPU between artillery and nginx (or anything else on the machine).

If you have time to re-run some of your tests that would be great. I'd love to hear if you see similar improvements.

[enudler]

hi @hassy
sorry for the late response
indeed, much better

Summary report @ 20:02:14(+0300) 2020-06-05
Total time: 1 minute, 0 seconds
http.request_rate: ....................... 988/sec avg
http.response_rate: ...................... 988/sec avg
scenarios.created.Scenario 1: ............ 60000
scenarios.created.total: ................. 60000
scenarios.completed: ..................... 60000
http.requests: ........................... 60000
http.codes.200: .......................... 60000
http.responses: .......................... 60000
scenarios.duration:
min: ................................... 2
max: ................................... 127.6
median: ................................ 3
p95: ................................... 5
p99: ................................... 9
http.response_time:
min: ................................... 2
max: ................................... 126.3
median: ................................ 1
p95: ................................... 5
p99: ................................... 7

[philjones88]

Breaking change for those of us using the custom JS for before and after hooks is the response no longer contains the body of the response. We used this to pull out the CSRF token in a HTML page.

The docs literally say "body" in them https://artillery.io/docs/http-reference/#function-signatures

[danjamesmay]

Will Got be backwards compatible with the proxy helper HTTP_PROXY=<http_scheme>://<IP>:<PORT> that used to be supported in requests.js?

[hassy]

@danjamesmay Yep, HTTP_PROXY env vars are still supported.

[hassy]

@philjones88 that should be fixed on master now

[hassy]

As far as the test suite is concerned, everything is green, so the switch from Request to Got can be marked as completed.

We may find issues with compatibility still - those will be addressed as separate issues.