Locust: On the same server，jmeter can run 20000rps，but locust only 5000rps

[yanchunhuo]

Description of issue / feature request

On a same server，as a loader，jmeter can run 20000rps，but locust(Distributed in use) only 5000rps ，why？with locust CPU is in high load，but jmeter is not

Expected behavior

no so bad rps with locust

Actual behavior

so bad rps with locust

Environment settings (for bug reports)

• OS:Ubuntu14.04
• Python version:2.7.6
• Locust version:0.8.1

Steps to reproduce (for bug reports)

TBD - example code appreciated

[heyman]

Without seeing the test scripts that was used, or the general setup and machine specs, it's impossible to tell if 5000 RPS is reasonable or not. For example, did you run multiple slave processes? If not, only one CPU core was utilized.

Locust uses Python-Requests for making HTTP requests. The reason for this is that the Requests lib is really nice to work with from a developer perspective (they go out of their way to handle all weird edge-cases), but it comes at a performance cost. In many cases this is worth it, since in general, developer cost is much higher than hardware cost. However, we have a much faster HTTP client implementation in the works (https://github.com/locustio/locust/pull/713) and if your test script is mostly just throwing HTTP-requests, you're likely to see ~5x improvement using that client.

Here's the branch with the new FastHttpLocust:
https://github.com/locustio/locust/tree/geventhttpclient

[yanchunhuo]

@heyman
一、the machine（run locust）:
CPU: 8CPU 2.3GHz
MEM: 16GB
二、Already used slave processes
三、script
1、my.py

#!/usr/bin/python
#-*- coding:utf-8 -*-
from locust import HttpLocust,TaskSet,task
class UserAction(TaskSet):
    @task
    def axbbind(self):
        self.client.get('/')

class User(HttpLocust):
    task_set=UserAction
    min_wait=0
    max_wait=1

2、startmaster.sh
locust -f my.py --host=http://192.168.8.17 --master --no-reset-stats

3、startslave.sh

for((i=0;i<8;i++))
do
   locust -f my.py --host=http://192.168.8.17 --no-reset-stats --slave --master-host=127.0.0.1 & 
done

[yanchunhuo]

@heyman
i used the FastHttpLocust，get the same result.
my service is just a simple nginx

[heyman]

i used the FastHttpLocust，get the same result.

That sounds strange, and seems to indicate that the bottleneck is not CPU on the locust machine in that case. Did you update the test scripts to use the FastHttpLocust class (not only use the geventhttpclient branch)?

What OS are you running on the test machine? Does the CPU usage max out during the test?

[yanchunhuo]

@heyman
i test again,use the l FastHttpLocust class,can reach 14000 rps.but the CPU is always in high load.CPU idle resources are almost 0.the load average always is 8.use jmeter can reach 20000 rps.the CPU only use 20%,load average only 2.the Server CPU is idle during test
Loader:
OS: Ubuntu14.04
CPU: 8CPU 2.3GHz
MEM: 16GB
Server:
OS: centos6.5
CPU: 8CPU 2.3GHz
MEM: 16GB

[heyman]

14k RPS on a single machine sounds pretty reasonable to me (and should correspond to a pretty high simultaneous user count in most real world scenarios).

Locust's goal isn't to squeeze out as many RPS with as little hardware as possible. In that case, python would have been a bad choice of programming language. Instead it provides a way to define user behaviour using code, so that you can create realistic load on a system by swarming it with those simulated users (running distributed on multiple machines if needed). In most cases hardware is cheaper than developer time.

If you're only interested in throwing as many RPS as possible against a few endpoints, I'd recommend using Apache Bench, or some similar tool. Or if JMeter solves your problem, you should stick with that.

[yanchunhuo]

@heyman
OK,thanks!

[cgoldberg]

I'd trade 30% in throughput for the ability to program clients in Python any day!

[tobeczm]

I have the same problem:
on the same machine, jmeter can run 4600+ rps, but locust only 3300+. I'm using FastHttpUser and 3 workers. the bottleneck is CPU. The CPU is pretty high (100%). But Jmeter only require about 40% CPU. The advantage of locust is that it use much less memory than Jmeter.

[heyman]

@tobeczm Locust and JMeter are two different programs with quite different feature sets. If you've identified that the bottleneck is CPU, you could add more machines/workers.