Gunicorn: Custom Worker Causing Issues

[Abraxos]

Hello,

I am trying to set a custom worker to retrieve client cert information from the server and then get it into my Flask app like so:

gunicorn_options.py:

import multiprocessing
import os
from mypackage.client_auth_worker import CustomSyncWorker

def config_dir():
    return os.path.dirname(os.path.realpath(__file__))

bind = "0.0.0.0:8000"
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = CustomSyncWorker
timeout = 30
ca_certs = os.path.join(config_dir(), 'ca.cert.pem')
certfile = os.path.join(config_dir(), 'server.cert.pem')
keyfile = os.path.join(config_dir(), 'server.key.pem')
cert_reqs = 2
do_handshake_on_connect = True

custom_auth_worker.py:

from gunicorn.workers.sync import SyncWorker

class CustomSyncWorker(SyncWorker):
    def handle_request(self, listener, req, client, addr):
        subject = dict(client.getpeercert().get('subject')[0])
        headers = dict(req.headers)
        headers['X-USER'] = subject.get('commonName')
        print(headers)
        req.headers = list(headers.items())
        super(CustomSyncWorker, self).handle_request(listener, req, client, addr)

I keep getting the following exception:

File "/Users/me/.virtualenvs/mypackage/lib/python3.5/site-packages/gunicorn/arbiter.py", line 162, in start
    self.log.info("Using worker: %s", self.cfg.worker_class_str)
  File "/Users/me/.virtualenvs/mypackage/lib/python3.5/site-packages/gunicorn/config.py", line 68, in __getattr__
    raise AttributeError("No configuration setting for: %s" % name)
AttributeError: No configuration setting for: worker_class_str

I tried multiple things to fix this, for example, I tried setting a worker_class_str in the gunicorn_options.py file, that did not resolve the issue. I then started digging in the code of the actual package and I found that when I modified the gunicorn/arbiter.py file to comment out the line: self.log.info("Using worker: %s", self.cfg.worker_class_str) everything works perfectly fine, the CustomSyncWorker is loaded just fine and does its job as it is supposed to setting the X-USER header and everything.

Is this a bug? Or am I just doing something drastically wrong? I figure I should not be taking it upon myself to modify the code locally just to force things to work...

[berkerpeksag]

I think you need to pass a string to worker_class. So changing

worker_class = CustomSyncWorker

to

worker_class = 'mypackage.client_auth_worker.CustomSyncWorker'

would solve your problem.

[berkerpeksag]

And note that this is already documented at http://docs.gunicorn.org/en/stable/settings.html#worker-class

Optionally, you can provide your own worker by giving Gunicorn a Python path to a subclass of gunicorn.workers.base.Worker. This alternative syntax will load the gevent class: gunicorn.workers.ggevent.GeventWorker.

Of course, we could always improve the documentation if you have a better suggestion.

Thank you for the report!

[Abraxos]

I believe I had tried that, and it did not work, however, I am not certain and I may have written the string differently (like in a "mypackage.client_auth_worker:CustomSyncWorker" format). I will have to check when I get home later today.

Aside from that I would recommend putting a direct example into the documentation or at least somewhere else (wherever you think it's appropriate) simply because I had seen that documentation and it wasn't clear that that's what I should've done in the case of a python configuration file. I ended up doing a bunch of trial-and-error which resulted in me getting a working version that I posted after commenting out the printout.

Thanks for getting back to me so quickly!

[Abraxos]

OK, confirmed, that worked. Thank you for the help!