Meshery: Add wrk2 support for HTTP Load Test

[sakomws]

Feature Epic Description
Extend Meshery to bring in the capability to work with multiple load generators as built-in libraries through an interface and load generator-specific implementation.
Through Meshery UI, be able to choose and configure the two supported load generators: _Fortio_ and _wrk2_ for HTTP load testing.

This feature epic is comprised of these enhancement issues:

• #209 M0: Extend Load Generation Build and Packaging
• #210 M1: Extend Load Generation w/wrk2
• #211 M2: General Interface for Load Generators
• #212 M3: Incorporate wrk2 as a Load Generator
• #213 M4: Expose Load Generator settings in UI

Resources:
https://github.com/wg/wrk
https://github.com/giltene/wrk2

[leecalcote]

@girishranganathan have you sent out a call for review on the design spec? It'd be good to get @t-lo's input early in this process.

[t-lo]

Happy to help! Our extensions to wrk2 are currently here: https://github.com/kinvolk/wrk2 . We added a feature to spread RPS load over multiple endpoints as well as extended the LUA API to close / re-open connections from a LUA script (necessary for the RPS load spread feature).

We aim to create PRs to @giltene 's original wrk2 work located here: https://github.com/giltene/wrk2 but this is on the back burner for now because of higher priority work.

(Please note that I'm on vacation until the 8th of July with very limited internet, but I'll of course be available when I'm back).

[girishranganathan]

Thanks a lot @t-lo. Have a great vacation. The WIP doc can be found here. Please feel free to chime in.

[girishranganathan]

Hello @t-lo, it was very nice to meet you in person again at kubecon.
We have a functional version of a fork of your wrk2 integrated in Meshery, like we had told you before.
Meshery, up until now, has been fortio focused. It was challenging to process & present WRK2 data using our existing infra.

I have tried to include a snap of the charts Meshery has generated with fortio and wrk2 for the same set of endpoint, concurrent requests, QPS, duration combo. I have also included the JSON we are using to render the charts, if you are interested.

Fortio

Here is an example of a test run with fortio:

Raw JSON results

{
"status": "success",
"result": {
"meshery_id": "00000000-0000-0000-0000-000000000000",
"name": "No mesh_1575320652669",
"runner_results": {
"AbortOn": 0,
"ActualDuration": 30048740204,
"ActualQPS": 1.9967559236314665,
"DurationHistogram": {
"Avg": 0.1501899647833334,
"Count": 60,
"Data": [{
"Count": 9,
"End": 0.045,
"Percent": 15,
"Start": 0.042565906
}, {
"Count": 8,
"End": 0.05,
"Percent": 28.333333333333332,
"Start": 0.045
}, {
"Count": 3,
"End": 0.06,
"Percent": 33.333333333333336,
"Start": 0.05
}, {
"Count": 4,
"End": 0.12,
"Percent": 40,
"Start": 0.1
}, {
"Count": 6,
"End": 0.14,
"Percent": 50,
"Start": 0.12
}, {
"Count": 4,
"End": 0.16,
"Percent": 56.666666666666664,
"Start": 0.14
}, {
"Count": 3,
"End": 0.18,
"Percent": 61.666666666666664,
"Start": 0.16
}, {
"Count": 3,
"End": 0.2,
"Percent": 66.66666666666667,
"Start": 0.18
}, {
"Count": 13,
"End": 0.25,
"Percent": 88.33333333333333,
"Start": 0.2
}, {
"Count": 3,
"End": 0.3,
"Percent": 93.33333333333333,
"Start": 0.25
}, {
"Count": 2,
"End": 0.35000000000000003,
"Percent": 96.66666666666667,
"Start": 0.3
}, {
"Count": 1,
"End": 0.4,
"Percent": 98.33333333333333,
"Start": 0.35000000000000003
}, {
"Count": 1,
"End": 0.461091598,
"Percent": 100,
"Start": 0.45
}],
"Max": 0.461091598,
"Min": 0.042565906,
"Percentiles": [{
"Percentile": 50,
"Value": 0.14
}, {
"Percentile": 75,
"Value": 0.21923076923076923
}, {
"Percentile": 90,
"Value": 0.2666666666666667
}, {
"Percentile": 99,
"Value": 0.45443663920000005
}, {
"Percentile": 99.9,
"Value": 0.46042610212
}],
"StdDev": 0.09553582756575404,
"Sum": 9.011397887000005
},
"Exactly": 0,
"HeaderSizes": {
"Avg": 0,
"Count": 60,
"Data": [{
"Count": 60,
"End": 0,
"Percent": 100,
"Start": 0
}],
"Max": 0,
"Min": 0,
"Percentiles": null,
"StdDev": 0,
"Sum": 0
},
"Labels": "No mesh_1575320652669 -_- https://github.com",
"NumThreads": 1,
"RequestedDuration": "30s",
"RequestedQPS": "2",
"RetCodes": {
"200": 39,
"429": 21
},
"RunType": "HTTP",
"Sizes": {
"Avg": 89909.2,
"Count": 60,
"Data": [{
"Count": 21,
"End": 10000,
"Percent": 35,
"Start": 9269
}, {
"Count": 39,
"End": 133331,
"Percent": 100,
"Start": 100000
}],
"Max": 133331,
"Min": 9269,
"Percentiles": null,
"StdDev": 59173.729224152616,
"Sum": 5394552
},
"SocketCount": 0,
"StartTime": "2019-12-02T21:04:13.063402723Z",
"URL": "https://github.com",
"Version": "1.3.1-pre"
}
}
}

WRK2

Here is an example of a test run with WRK2:

Raw JSON results

{
"status": "success",
"result": {
"meshery_id": "00000000-0000-0000-0000-000000000000",
"name": "No mesh_1575320867175",
"runner_results": {
"AbortOn": 0,
"ActualDuration": 30048889000,
"ActualQPS": 2.03,
"DurationHistogram": {
"Avg": 0.1628496,
"Count": 61,
"Data": [{
"Count": 1,
"End": 0.042175,
"Percent": 1,
"Start": 0
}, {
"Count": 1,
"End": 0.042175,
"Percent": 2,
"Start": 0.042175
}, {
"Count": 1,
"End": 0.042175,
"Percent": 3,
"Start": 0.042175
}, {
"Count": 2,
"End": 0.043199,
"Percent": 4,
"Start": 0.042175
}, {
"Count": 2,
"End": 0.043199,
"Percent": 5,
"Start": 0.043199
}, {
"Count": 2,
"End": 0.043199,
"Percent": 6,
"Start": 0.043199
}, {
"Count": 3,
"End": 0.043743000000000004,
"Percent": 7,
"Start": 0.043199
}, {
"Count": 3,
"End": 0.043743000000000004,
"Percent": 8,
"Start": 0.043743000000000004
}, {
"Count": 4,
"End": 0.043807,
"Percent": 9,
"Start": 0.043743000000000004
}, {
"Count": 4,
"End": 0.043807,
"Percent": 10,
"Start": 0.043807
}, {
"Count": 4,
"End": 0.043807,
"Percent": 11,
"Start": 0.043807
}, {
"Count": 5,
"End": 0.043871,
"Percent": 12,
"Start": 0.043807
}, {
"Count": 5,
"End": 0.043871,
"Percent": 13,
"Start": 0.043871
}, {
"Count": 6,
"End": 0.043999,
"Percent": 14,
"Start": 0.043871
}, {
"Count": 6,
"End": 0.043999,
"Percent": 15,
"Start": 0.043999
}, {
"Count": 6,
"End": 0.043999,
"Percent": 16,
"Start": 0.043999
}, {
"Count": 7,
"End": 0.044063000000000005,
"Percent": 17,
"Start": 0.043999
}, {
"Count": 7,
"End": 0.044063000000000005,
"Percent": 18,
"Start": 0.044063000000000005
}, {
"Count": 8,
"End": 0.044127,
"Percent": 19,
"Start": 0.044063000000000005
}, {
"Count": 8,
"End": 0.044127,
"Percent": 20,
"Start": 0.044127
}, {
"Count": 8,
"End": 0.044127,
"Percent": 21,
"Start": 0.044127
}, {
"Count": 9,
"End": 0.046111,
"Percent": 22,
"Start": 0.044127
}, {
"Count": 9,
"End": 0.046111,
"Percent": 23,
"Start": 0.046111
}, {
"Count": 10,
"End": 0.046655,
"Percent": 24,
"Start": 0.046111
}, {
"Count": 10,
"End": 0.046655,
"Percent": 25,
"Start": 0.046655
}, {
"Count": 10,
"End": 0.046655,
"Percent": 26,
"Start": 0.046655
}, {
"Count": 11,
"End": 0.050975,
"Percent": 27,
"Start": 0.046655
}, {
"Count": 11,
"End": 0.050975,
"Percent": 28,
"Start": 0.050975
}, {
"Count": 12,
"End": 0.051999000000000004,
"Percent": 29,
"Start": 0.050975
}, {
"Count": 12,
"End": 0.051999000000000004,
"Percent": 30,
"Start": 0.051999000000000004
}, {
"Count": 12,
"End": 0.051999000000000004,
"Percent": 31,
"Start": 0.051999000000000004
}, {
"Count": 13,
"End": 0.088063,
"Percent": 32,
"Start": 0.051999000000000004
}, {
"Count": 13,
"End": 0.088063,
"Percent": 33,
"Start": 0.088063
}, {
"Count": 14,
"End": 0.09132699999999999,
"Percent": 34,
"Start": 0.088063
}, {
"Count": 14,
"End": 0.09132699999999999,
"Percent": 35,
"Start": 0.09132699999999999
}, {
"Count": 14,
"End": 0.09132699999999999,
"Percent": 36,
"Start": 0.09132699999999999
}, {
"Count": 15,
"End": 0.096255,
"Percent": 37,
"Start": 0.09132699999999999
}, {
"Count": 15,
"End": 0.096255,
"Percent": 38,
"Start": 0.096255
}, {
"Count": 16,
"End": 0.11731100000000001,
"Percent": 39,
"Start": 0.096255
}, {
"Count": 16,
"End": 0.11731100000000001,
"Percent": 40,
"Start": 0.11731100000000001
}, {
"Count": 16,
"End": 0.11731100000000001,
"Percent": 41,
"Start": 0.11731100000000001
}, {
"Count": 17,
"End": 0.124223,
"Percent": 42,
"Start": 0.11731100000000001
}, {
"Count": 17,
"End": 0.124223,
"Percent": 43,
"Start": 0.124223
}, {
"Count": 18,
"End": 0.139391,
"Percent": 44,
"Start": 0.124223
}, {
"Count": 18,
"End": 0.139391,
"Percent": 45,
"Start": 0.139391
}, {
"Count": 18,
"End": 0.139391,
"Percent": 46,
"Start": 0.139391
}, {
"Count": 19,
"End": 0.141055,
"Percent": 47,
"Start": 0.139391
}, {
"Count": 19,
"End": 0.141055,
"Percent": 48,
"Start": 0.141055
}, {
"Count": 20,
"End": 0.14924700000000002,
"Percent": 49,
"Start": 0.141055
}, {
"Count": 20,
"End": 0.14924700000000002,
"Percent": 50,
"Start": 0.14924700000000002
}, {
"Count": 20,
"End": 0.14924700000000002,
"Percent": 51,
"Start": 0.14924700000000002
}, {
"Count": 21,
"End": 0.14924700000000002,
"Percent": 52,
"Start": 0.14924700000000002
}, {
"Count": 21,
"End": 0.14924700000000002,
"Percent": 53,
"Start": 0.14924700000000002
}, {
"Count": 22,
"End": 0.15129499999999999,
"Percent": 54,
"Start": 0.14924700000000002
}, {
"Count": 22,
"End": 0.15129499999999999,
"Percent": 55,
"Start": 0.15129499999999999
}, {
"Count": 22,
"End": 0.15129499999999999,
"Percent": 56,
"Start": 0.15129499999999999
}, {
"Count": 23,
"End": 0.161279,
"Percent": 57,
"Start": 0.15129499999999999
}, {
"Count": 23,
"End": 0.161279,
"Percent": 58,
"Start": 0.161279
}, {
"Count": 24,
"End": 0.177791,
"Percent": 59,
"Start": 0.161279
}, {
"Count": 24,
"End": 0.177791,
"Percent": 60,
"Start": 0.177791
}, {
"Count": 24,
"End": 0.177791,
"Percent": 61,
"Start": 0.177791
}, {
"Count": 25,
"End": 0.184959,
"Percent": 62,
"Start": 0.177791
}, {
"Count": 25,
"End": 0.184959,
"Percent": 63,
"Start": 0.184959
}, {
"Count": 26,
"End": 0.18854300000000002,
"Percent": 64,
"Start": 0.184959
}, {
"Count": 26,
"End": 0.18854300000000002,
"Percent": 65,
"Start": 0.18854300000000002
}, {
"Count": 26,
"End": 0.18854300000000002,
"Percent": 66,
"Start": 0.18854300000000002
}, {
"Count": 27,
"End": 0.19340700000000002,
"Percent": 67,
"Start": 0.18854300000000002
}, {
"Count": 27,
"End": 0.19340700000000002,
"Percent": 68,
"Start": 0.19340700000000002
}, {
"Count": 28,
"End": 0.213119,
"Percent": 69,
"Start": 0.19340700000000002
}, {
"Count": 28,
"End": 0.213119,
"Percent": 70,
"Start": 0.213119
}, {
"Count": 28,
"End": 0.213119,
"Percent": 71,
"Start": 0.213119
}, {
"Count": 29,
"End": 0.225663,
"Percent": 72,
"Start": 0.213119
}, {
"Count": 29,
"End": 0.225663,
"Percent": 73,
"Start": 0.225663
}, {
"Count": 30,
"End": 0.22643100000000002,
"Percent": 74,
"Start": 0.225663
}, {
"Count": 30,
"End": 0.22643100000000002,
"Percent": 75,
"Start": 0.22643100000000002
}, {
"Count": 30,
"End": 0.22643100000000002,
"Percent": 76,
"Start": 0.22643100000000002
}, {
"Count": 31,
"End": 0.232063,
"Percent": 77,
"Start": 0.22643100000000002
}, {
"Count": 31,
"End": 0.232063,
"Percent": 78,
"Start": 0.232063
}, {
"Count": 32,
"End": 0.253695,
"Percent": 79,
"Start": 0.232063
}, {
"Count": 32,
"End": 0.253695,
"Percent": 80,
"Start": 0.253695
}, {
"Count": 32,
"End": 0.253695,
"Percent": 81,
"Start": 0.253695
}, {
"Count": 33,
"End": 0.255871,
"Percent": 82,
"Start": 0.253695
}, {
"Count": 33,
"End": 0.255871,
"Percent": 83,
"Start": 0.255871
}, {
"Count": 34,
"End": 0.26470299999999997,
"Percent": 84,
"Start": 0.255871
}, {
"Count": 34,
"End": 0.26470299999999997,
"Percent": 85,
"Start": 0.26470299999999997
}, {
"Count": 34,
"End": 0.26470299999999997,
"Percent": 86,
"Start": 0.26470299999999997
}, {
"Count": 35,
"End": 0.29158300000000004,
"Percent": 87,
"Start": 0.26470299999999997
}, {
"Count": 35,
"End": 0.29158300000000004,
"Percent": 88,
"Start": 0.29158300000000004
}, {
"Count": 36,
"End": 0.30412700000000004,
"Percent": 89,
"Start": 0.29158300000000004
}, {
"Count": 36,
"End": 0.30412700000000004,
"Percent": 90,
"Start": 0.30412700000000004
}, {
"Count": 36,
"End": 0.30412700000000004,
"Percent": 91,
"Start": 0.30412700000000004
}, {
"Count": 37,
"End": 0.317183,
"Percent": 92,
"Start": 0.30412700000000004
}, {
"Count": 37,
"End": 0.317183,
"Percent": 93,
"Start": 0.317183
}, {
"Count": 38,
"End": 0.34636700000000004,
"Percent": 94,
"Start": 0.317183
}, {
"Count": 38,
"End": 0.34636700000000004,
"Percent": 95,
"Start": 0.34636700000000004
}, {
"Count": 38,
"End": 0.34636700000000004,
"Percent": 96,
"Start": 0.34636700000000004
}, {
"Count": 39,
"End": 0.433407,
"Percent": 97,
"Start": 0.34636700000000004
}, {
"Count": 39,
"End": 0.433407,
"Percent": 98,
"Start": 0.433407
}, {
"Count": 40,
"End": 0.453887,
"Percent": 99,
"Start": 0.433407
}, {
"Count": 40,
"End": 0.453887,
"Percent": 99.9,
"Start": 0.453887
}, {
"Count": 40,
"End": 0.453887,
"Percent": 99.99,
"Start": 0.453887
}, {
"Count": 40,
"End": 0.453887,
"Percent": 99.999,
"Start": 0.453887
}, {
"Count": 40,
"End": 0.453887,
"Percent": 100,
"Start": 0.453887
}],
"Max": 0.453632,
"Min": 0.042144,
"Percentiles": [{
"Percentile": 50,
"Value": 0.14924700000000002
}, {
"Percentile": 75,
"Value": 0.22643100000000002
}, {
"Percentile": 90,
"Value": 0.30412700000000004
}, {
"Percentile": 99,
"Value": 0.453887
}, {
"Percentile": 99.99,
"Value": 0.453887
}, {
"Percentile": 99.999,
"Value": 0.453887
}],
"StdDev": 0.10981846,
"Sum": 0
},
"Exactly": 0,
"HeaderSizes": null,
"Labels": "No mesh_1575320867175 -_- https://github.com",
"NumThreads": 1,
"RequestedDuration": "30s",
"RequestedQPS": "2.000000",
"RetCodes": null,
"RunType": "HTTP",
"Sizes": null,
"SocketCount": 0,
"StartTime": "2019-12-02T21:07:47.415779933Z",
"URL": "https://github.com:443",
"Version": ""
}
}
}

Let's get our conversation started with these charts 😁.