Gunicorn: Configuration file requires .py extension since 20.0.1

[tari]

With gunicorn 20.0.1 and 20.0.2 running on PyPy3.6 (7.1.1), I get startup failures like this (some paths sanitized):

gunicorn[9371]: Failed to read config file: /home/.../gunicorn.cfg
gunicorn[9371]: Traceback (most recent call last):
gunicorn[9371]:   File "/home/.../virtualenv/site-packages/gunicorn/app/base.py", line 100, in get_config_from_filename
gunicorn[9371]:     mod = importlib.util.module_from_spec(spec)
gunicorn[9371]:   File "<frozen importlib._bootstrap>", line 558, in module_from_spec
gunicorn[9371]: AttributeError: 'NoneType' object has no attribute 'loader'

In a quick scan I couldn't find the commit that changes it, but I think this is caused by "fixed the way the config module is loaded. __file__ is now available" in 20.0.1. Downgrading to 20.0.0 allows my server to start up normally.

[benoitc]

@tari which command line are you using? what return the check-config command line?

[benoitc]

@tari if possible please share your config file. You can send it of my mail if you want to keep it private.

[tari]

Command line looks like this (again slightly modified to anonymize):

gunicorn -c gunicorn.cfg --capture-output -n myapp --bind unix:/run/myapp/gunicorn.sock --pid /run/myapp/gunicorn.pid -u myapp -g myapp myapp.wsgi

(The WSGI app is Django.)

This is my config:

import multiprocessing

from prometheus_client import multiprocess

workers = multiprocessing.cpu_count()
worker_class = 'gthread'
threads = 2

def child_exit(_server, worker):
    multiprocess.mark_process_dead(worker.pid)

[tilgovi]

Should be fixed if you rename gunicorn.cfg to gunicorn.cfg.py. The recent change made it so that Python determines what loader to use based on the file, and it does not know how to interpret that.

[tilgovi]

Apologies for the breaking change. I don't think it was intentional. It does mean that loading a config from bytecode or extension modules would work, though! Or any other thing that Python introduces in the future.

[gnarvaja]

Confirming the same problem here, and the fix. Just renamed my configuration file from .conf to .conf.py and it worked.

I think "python" should be the default loader to keep backward compatibility. Here is a quick and dirty fix:

diff --git a/gunicorn/app/base.py b/gunicorn/app/base.py
index cb1bd97..d94dfc9 100644
--- a/gunicorn/app/base.py
+++ b/gunicorn/app/base.py
@@ -97,6 +97,11 @@ class Application(BaseApplication):
         try:
             module_name = '__config__'
             spec = importlib.util.spec_from_file_location(module_name, filename)
+            if spec is None:  # Non-standard extension, not recognized fallback with python source loader
+                python_src_loader = importlib.util.spec_from_file_location(
+                    module_name, os.path.realpath(__file__)
+                ).loader
+                spec = importlib.util.spec_from_file_location(module_name, filename, loader=python_src_loader)
             mod = importlib.util.module_from_spec(spec)
             sys.modules[module_name] = mod
             spec.loader.exec_module(mod)

[benoitc]

@tilgovi good catch!

@gnarvaja I think your solution is good enough. Can you make a PR for it?

[benoitc]

@tilgovi do you think we need the patch above? We can make a 20.0.3 today with it.

[benoitc]

fixed in latest master