Locust: How I share auth cookie with the rest of tasks only for current locust user?

[alexanderdevm]

I am looking into to moving my multi-threaded python script to locust.

A simple explanation of what my script does is:

Create a thread per user
In each thread authenticates user and get auth cookie
With that auth cookie perform various api calls at a set interval
When i started looking into locust, I have noticed that the only way to perform each task at its own specific interval, I would need to create a taskset per task.

This brought up an issue of how do i share the auth cookie for the given spawned user between task sets? Since in the long run I also need to share response data between taskset for the given spawned user as it differs between spawned users.

In the sample code below, all of the users spawned by locust, share the same "storage.cookie". Is there a way to keep storage.cookie unique per user, share it with all tasks sets for the given spawned user by locust ? Does locust report on which user is currently executing the task?

from __future__ import print_function
from locust import Locust, TaskSet, task, HttpLocust
import json


def auth(l):
    payload = {"username":"some_username","password":"some_password"} 
    resp = l.client.post('/auth', data = json.dumps(payload))
    storage.cookie = # get auth cookie from resp

def do_i_auth(l):
    if len(storage.cookie) == 0:
        auth(l)

class storage(object):
    cookie == ''

class first_call(TaskSet):
    def on_start(self):
        do_i_auth(self)

    @task
    def get_api_a(self):
        headers = {"Cookie":storage.cookie}
        self.client.get('/api_a', headers)

class second_call(TaskSet):
    def on_start(self):
        do_i_auth(self)

    @task
    def get_api_b(self):
        headers = {"Cookie":storage.cookie}
        self.client.get('/api_b', headers)

class api_A(HttpLocust):
    task_set = first_call
    min_wait = 5000
    max_wait = 5000    

class api_B(HttpLocust):
    task_set = second_call
    min_wait = 10000
    max_wait = 10000

Stack overflow copy: https://stackoverflow.com/questions/48739300/locust-how-i-share-auth-cookie-with-the-rest-of-tasks-only-for-current-locust-u

[cgoldberg]

storage.cookie = # get auth cookie from resp

you never created an instance of storage

[alexanderdevm]

@cgoldberg i did try although i am getting the same results, Would you be able to share an example!

[cgoldberg]

class storage(object):
    cookie == ''

Your code doesn't define a class correctly. It's a plain old Python class... so it needs an __init__ method to be instantiated.

So maybe:

class Storage(object):
    def __init__(self):
        self.cookie = None

You will first need to instantiate the Storage() object, then you can assign a value to the .cookie property of each instance. Basic OO.

[alexanderdevm]

@cgoldberg I have tried you suggestion and, the result is same, as in s.cookie is the same for all locust users. It seems that storage is initialized on locust start and not on each user/client/thread.

I have created a dummy locust script to make it easier to see the results:

from __future__ import print_function
from locust import Locust, TaskSet, task, HttpLocust
import random as r

class storage(object):
    def __init__(self):
        self.cookie = None

s = storage()

def auth(l):
    s.cookie = r.randint(9,50)

def do_i_auth(l):
    if s.cookie is None:
        auth(l)

class first_call(TaskSet):
    min_wait = 5000
    max_wait = 5000

    def on_start(self):
        do_i_auth(self)        

    @task
    def get_api_a(self):
        print(s.cookie)

class second_call(TaskSet):
    min_wait = 10000
    max_wait = 10000

    def on_start(self):
        do_i_auth(self)

    @task
    def get_species(self):
        print(s.cookie)

class api_A(HttpLocust):
    task_set = first_call

    host = 'localhost'


class api_b(HttpLocust):
    task_set = second_call

    host = 'localhost'

[cgoldberg]

s = storage()

You are creating an instance at the module level, so it is shared. You need to create it inside your Locust class, so an instance is created for each user.

https://docs.locust.io/en/latest/writing-a-locustfile.html#the-locust-class

[alexanderdevm]

I did look at the document, although i did not find a way how to share a single instance of s = storage() between first_call and second_call classes.

[alexanderdevm]

@cgoldberg I've been playing around trying various approaches, and I am not getting any good results.

I have simplified the approach to see if I get a new cookie per user spawned per user. I have included the code below.

I have started locust with 2 users and 2 hatch rate with the execution log below.

I was expecting to see:

• wait 5 seconds
• 2 calls to api_a with 2 different random values
• wait 5 seconds
• 2 calls to api_a with the same 2 values from step 2
• 2 calls to api_b with 2 different random values
• repeat from step 1

Although what I saw was:

• call to api_b, call to api_a
• wait 5 seconds
• call to api_a same value as step 1, api_b, api_a same value as step 1
• wait 5 seconds

Something is off:

• new random value in "s.cookie" does not get generated per user, although it should
• the sequence and frequency in which api_a and api_b should be called does not seems to what it supposed to be
• I am not looking yet at sharing the s.cookie between api_a and api_b as everything else does not seem to be work as expected

from __future__ import print_function
from locust import Locust, TaskSet, task, HttpLocust
import random as r
import datetime as d
import time as t

class storage(object):
    def __init__(self):
        self.cookie = None

class first_call(TaskSet):
    min_wait = 5000
    max_wait = 5000
    s = storage()

    def on_start(self):
        self.s.cookie = r.randint(9,50)

    @task
    def get_api_a(self):
        print('a %s' % self.s.cookie)
        print('time %s' % d.datetime.now().time())

class second_call(TaskSet):
    min_wait = 10000
    max_wait = 10000
    s = storage()

    def on_start(self):
        self.s.cookie = r.randint(9,50)

    @task
    def get_species(self):
        print('b %s' % self.s.cookie)
        print('time %s' % d.datetime.now().time())

class api_A(HttpLocust):
    task_set = first_call    
    host = 'localhost'


class api_b(HttpLocust):
    task_set = second_call
    host = 'localhost'

output:

[2018-02-14 02:37:13,289] linux_box.local/INFO/locust.main: Starting web monitor at *:8089
[2018-02-14 02:37:13,289] linux_box.local/INFO/locust.main: Starting Locust 0.8.1
[2018-02-14 02:37:27,779] linux_box.local/INFO/locust.runners: Hatching and swarming 2 clients at the rate 2 clients/s...
[2018-02-14 02:37:27,780] linux_box.local/INFO/stdout: b 10
[2018-02-14 02:37:27,780] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:27,780] linux_box.local/INFO/stdout: time 02:37:27.780321
[2018-02-14 02:37:27,780] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:28,281] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:28,281] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:28,282] linux_box.local/INFO/stdout: time 02:37:28.282035
[2018-02-14 02:37:28,282] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:28,783] linux_box.local/INFO/locust.runners: All locusts hatched: api_A: 1, api_b: 1
[2018-02-14 02:37:28,783] linux_box.local/INFO/locust.runners: Resetting stats

[2018-02-14 02:37:33,287] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:33,287] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:33,287] linux_box.local/INFO/stdout: time 02:37:33.287857
[2018-02-14 02:37:33,287] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:37,785] linux_box.local/INFO/stdout: b 10
[2018-02-14 02:37:37,785] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:37,786] linux_box.local/INFO/stdout: time 02:37:37.786040
[2018-02-14 02:37:37,786] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:38,288] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:38,289] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:38,289] linux_box.local/INFO/stdout: time 02:37:38.289153
[2018-02-14 02:37:38,289] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:43,289] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:43,289] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:43,290] linux_box.local/INFO/stdout: time 02:37:43.290018
[2018-02-14 02:37:43,290] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:47,789] linux_box.local/INFO/stdout: b 10
[2018-02-14 02:37:47,790] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:47,790] linux_box.local/INFO/stdout: time 02:37:47.790242
[2018-02-14 02:37:47,790] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:48,290] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:48,290] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:48,291] linux_box.local/INFO/stdout: time 02:37:48.291067
[2018-02-14 02:37:48,291] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:53,296] linux_box.local/INFO/stdout: a 31
[2018-02-14 02:37:53,296] linux_box.local/INFO/stdout: 
[2018-02-14 02:37:53,296] linux_box.local/INFO/stdout: time 02:37:53.296873

[cgoldberg]

you still aren't creating the storage instance in your Locust class (the classes based on HttpLocust).

as for your other questions/issues, I really have no idea what you are trying to do... and this entire thread is getting confusing.

I'm closing it, since there is no Locust issue.

[alexanderdevm]

I will review my approach, and if needed create another ticket with details specifically to the issue as you have mentioned that this thread is getting confusing.

[MatrixManAtYrService]

Why have separate classes at all? If you encapsulate the whole flow for each user on a single class then you can store the cookies on objects of that type.

This worked for me:

https://gist.github.com/MatrixManAtYrService/1d83abd54adc9d4181f9ebb98b9799f7

[nityasantoshi]

@alexanderdevm, I want to generate the token at load generation time means I want to use the same token for all locust instances.But If I follow your code, it is not working. Still it is generating the different token for every locust instance.

Please can any one help me on this