Locust: use_as_lib.py example getting stuck when running

[naphta]

Describe the bug

Running the as a library example gets stuck looping over empty stats. It appears that from the traceback it's stuck running the stats and that the gevent is somehow blocking

I've tested with Python 3.7.3 and Python 3.8, as well as Locust 1.2.2 and Locust 1.2.1.

2020-08-25T10:38:16Z
Traceback (most recent call last):
  File "test.py", line 38, in <module>
    gevent.spawn(stats_history(env.runner))
  File "/Users/jhill/.local/share/virtualenvs/node-service-loadtesting-grpc-jUoOf3CB/lib/python3.7/site-packages/locust/stats.py", line 773, in stats_history
    gevent.sleep(HISTORY_STATS_INTERVAL_SEC)
  File "/Users/jhill/.local/share/virtualenvs/node-service-loadtesting-grpc-jUoOf3CB/lib/python3.7/site-packages/gevent/hub.py", line 163, in sleep
    hub.wait(t)
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 55, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_waiter.py", line 151, in gevent._gevent_c_waiter.Waiter.get
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
KeyboardInterrupt

Expected behavior

The Web UI should come up, show stats, and have a test running when loaded.

Actual behavior

The Web UI is blank, a test is never started.

Steps to reproduce

Copy the use_as_lib.py example, run with python from the command line.

Environment

• OS: macOS 10.15.5
• Python version: 3.8.5, 3.7.3
• Locust version: 1.2.2, 1.2.1
• Locust command line that you ran: python use_as_lib.py
• Locust file contents (anonymized if necessary):

import gevent
from locust import HttpUser, task, between
from locust.env import Environment
from locust.stats import stats_printer, stats_history
from locust.log import setup_logging

setup_logging("INFO", None)


class User(HttpUser):
    wait_time = between(1, 3)
    host = "https://docs.locust.io"

    @task
    def my_task(self):
        self.client.get("/")

    @task
    def task_404(self):
        self.client.get("/non-existing-path")

# setup Environment and Runner
env = Environment(user_classes=[User])
env.create_local_runner()

# start a WebUI instance
env.create_web_ui("127.0.0.1", 8089)

# start a greenlet that periodically outputs the current stats
gevent.spawn(stats_printer(env.stats))

# start a greenlet that save current stats to history
gevent.spawn(stats_history(env.runner))

# start the test
env.runner.start(1, spawn_rate=10)

# in 60 seconds stop the runner
gevent.spawn_later(60, lambda: env.runner.quit())

# wait for the greenlets
env.runner.greenlet.join()

# stop the web server for good measures
env.web_ui.stop()

• Packages installed:

certifi==2020.6.20
chardet==3.0.4
click==7.1.2
ConfigArgParse==1.2.3
Flask==1.1.2
Flask-BasicAuth==0.2.0
gevent==20.6.2
geventhttpclient==1.4.4
greenlet==0.4.16
grpcio==1.31.0
grpcio-tools==1.31.0
idna==2.10
itsdangerous==1.1.0
Jinja2==2.11.2
locust==1.2.2
MarkupSafe==1.1.1
msgpack==1.0.0
protobuf==3.13.0
psutil==5.7.2
pyzmq==19.0.2
requests==2.24.0
six==1.15.0
urllib3==1.25.10
Werkzeug==1.0.1
zope.event==4.4
zope.interface==5.1.0

[cyberw]

For now, just remove these lines:

# start a greenlet that save current stats to history
gevent.spawn(stats_history(env.runner))

@taojy123 can you have a look at this? Should it just be removed from the lib-example, or should the method be reworked to be more similar to stats_printer?

We could also add running the lib-example to the unit tests.

[naphta]

I can see that makes the example work fine, unfortunately I'm doing something a little bit more involved and removing that line does very little to help.

I'm not entirely sure what's going on here but basically when I remove that line in my code the test begins and starts making requests absolutely no problem but there are absolutely no stats stored. It might be because I'm using an old locustfile which I've quickly upgraded to 1.0 and the advice may have changed but I have a gRPC client running which when it makes a request it either runs locust.events.request_failure.fire() or locust.events.request_success.fire().

This locustfile does run absolutely fine when ran normally through the locust CLI I'm just wondering what's holding this back specifically.

[taojy123]

Sorry, I make a mistake in the example code.
Please change gevent.spawn(stats_history(env.runner)) to gevent.spawn(stats_history, env.runner)
I'll make a PR for it.

[naphta]

@taojy123 Such an obivous mistake now that I see it. Thanks for clearing it up!