Artillery: arrivalRate lower than 1

[ivoputzer]

ideally it would be nice to have the ability to make request over a given duration with fewer concurrent virtual users than 1 per second. it looks like it is not possibile to specify an arrivalRate lower than 1.

[hassy]

What's your use-case @ivoputzer? Can it be solved by using arrivalCount? An arrivalRate of less than 1 doesn't really make sense intuitively.

[ivoputzer]

we are currently trying to load-test our cluster with artillery and it works pretty well though in our case single services take long running tasks and we need to stress both the cluster and individual service for given phases and scenarios.

@hassy the problem can indeed be solved by tweaking both arrivalCount and duration properties each time and this is what we currently do though it feels quite strange at least to me doing the math each time again when trying to adjust the suite's throughput better expressed with an arrivalRate property. we've tried the previous patch #280 on our suites and the behaviour seems stable. just to make an example an arrivalRate of .5 would represent one request each 2 seconds and this once over a consistent duration.

[ivoputzer]

any update on the subject @hassy?

[hassy]

Not keen on supporting arrival rates of < 1 simply because it doesn't make sense intuitively (half a user every second? what would 0.3 mean? 0.7?)

The equivalent behavior could be expressed with inter arrival times, something like:

phases:
  - duration: 10
    interArrivalTime: 1

But that would need to be implemented as a new feature.

I appreciate you taking the time to read through the source code, make a change and submit a PR, however it's always advisable to open an issue to discuss changes first (for any changes that aren't a bug fix or an internal improvement).

[ivoputzer]

@hassy we currently use that patch because it simply does make sense to us. artillery displays rps rates lower than zero itself within its reports, thats why one would think it is just mathematically acceptable hence arrivalRate translates into request per second.

i appreciate your time and hope it just might be of help to some folks out there ;) cheers.

[hassy]

Yes but RPS is not the same as virtual user arrivals, and there isn't necessarily a linear relationship between the two.

What would the expected behaviour be if arrivalRate was set to 0.7?

[ivoputzer]

Yes but RPS is not the same as virtual user arrivals, and there isn't necessarily a linear relationship between the two.

@hassy both values do express a rate though even if there isn't a linear relationship between the two:

• requests per second
• virtual users per second

given a linear throughput on the web-server there would be a relationship between arrivalRate and rps but thats another story; what i want to say is, if rps can be lower than zero how come arrivalRate doesn't? moreover, why not allow decimal values in general?

What would the expected behaviour be if arrivalRate was set to 0.7?

0.2req/s  -> 1req every 5s
0.25req/s -> 1req every 4s
0.3req/s  -> 1req every 3.333333333s
0.5req/s  -> 1req every 2s
0.7req/s  -> 1req every 1.428571429s
1.0req/s  -> 1req every 1.0s
1.4req/s  -> 1req every 0.7142857143s
2req/s    -> 1req every 0.5s, 2req every 1s
2.5req/s  -> 1req every 0.4s, 2req every 1s
3req/s    -> 1req every 0.3s, 2req every 1.5s, 3req every 1s

[hassy]

RPS reported by Artillery is calculated after the fact, not set as a
target for when the test is run. So 5 requests in 10 seconds is 0.5 RPS.

If you set a phase's duration to 10 and arrivalCount to 5 you'd have an
average arrival rate of 0.5 afterwards too.

[ivoputzer]

@hassy i just think it's not very handy to change duration and arrivalCount each time when actually trying to adjust the suite's throughput aka arrivalRate. arrivalCount is based on the suite's duration, arrivalRate is not!

If you set a phase's duration to 10 and arrivalCount to 5 you'd have an
average arrival rate of 0.5 afterwards too.

given suites of different duration arrivalRate is consistent (n virtual users each second for a given amount of time), arrivalCount needs to be adjusted based on the current duration of the suite or phase (if duration is changed the math is gone!). it's just curious i can say i want 5 users each second but there is no way to say i want fewer users than one per second without changing property and doing the math based on the suite's duration.

it has very little to do with doing things before or after the fact, if both are rates why don't they behave the same mathematically? it's just kinda limiting and annoying, no big deal though.

[hassy]

Happy to take a PR which implements interArrivalTime or something like that, which would allow for a use-case like yours to be expressed, but non-integer arrivalRates don't make intuitive sense IMO.