Locust: Simulating real browser behaviour

Created on 22 Dec 2013 · 6Comments · Source: locustio/locust

Hey,

A real browser will download not only the content of the page but also all js scripts, css, images etc. I was wondering how this is supported in locust. All those extra resources can easily amount for most of the web traffic but also server's ability to handle simultaneous users. It seems to me, however, that locust's (or requests') _get_ gets only the main content. Is this the case? If so, how accurate are the locust measures when predicting real-life performance of your setup?

Thanks,

wontfix

Source

krychu

👍1

Most helpful comment

This page has a lot of helpful info for doing this and more
https://tag1consulting.com/blog/drupal-loadtest-locust

sreed101 on 17 Nov 2017

👍6

All 6 comments

Hi!

Scaling up the serving of static files is very simple compared to scaling up a dynamic website. Also, high traffic websites often use some kind of CDN for static files. If not a CDN, a load balancer with a couple of nginx machines should be able to handle A LOT of traffic.

Locust focuses on simulating the traffic from dynamic requests. You could make Locust load static files by parsing responses and then issuing new calls to _client.get()_, but since the serving of static content should be totally separated from dynamic requests, I would recommend to have a separate load test that tests static files.

heyman on 22 Dec 2013

Hey,

_Techniques_ of scaling up the serving of static files or dynamic components have nothing to do with the task of load testing. More, some of the linked resources in HTML might require computation on the server side so could be considered dynamic. More, serving static content will consume CPU and bandwidth which can have significant effect on the server's efficiency in relation to dynamic content, so it would appear that you can't simply focus on dynamic parts without considering static parts. Also, resources such as CDN or a couple of nginx machines don't come for free so their availability should not be taken by granted. I think these are some strong assumptions you make about how people should build their apps or what they can afford.

Perhaps this is something you will want to consider. It would be fantastic to see such functionality in locust. Thanks for taking time to respond, I appreciate it.

krychu on 22 Dec 2013

👍3

More, some of the linked resources in HTML might require computation on the server side so could be considered dynamic

Then you should write locust tasks for them.

There are a multitude of other tools that are suitable for loadtesting static content, and you can run them at the same time as you run a locust test. The strength of Locust is to fetch dynamic content and act on the response.

However, using BeautifulSoup to fetch all images/css/js from every HTML-response in your Locust test and download those should be fairly trivial to implement yourself.

mthurlin on 29 Dec 2013

@mthurlin We also want to load test something that will be hit by real browsers, so we need this functionality too. I just researched on this, and it would require implementing all a, img, link and other tags that can potentially reference things that a browser would download. While it isn't hard to do, it is certainly quite a bit of work and not something that can be done in 5 minutes and it's also easy to miss something. Most advice places like stackoverflow recommend simply using selenium instead, but of course that has probably way too much overhead for use in a load test.

Therefore, this seems to be a real gap of locust and I recommend reopening this ticket and reconsidering putting something in like this eventually, even if it's just a beautifulsoup script as some sort of contrib resource which people can reuse.