Gunicorn: Nuitka compiler can not be used with gunicorn

Created on 28 Jul 2017 · 5Comments · Source: benoitc/gunicorn

for example, i have a wsgi app(myapp.py):

def app(environ, start_response):
        data = b"Hello, World!\n"
        start_response("200 OK", [
            ("Content-Type", "text/plain"),
            ("Content-Length", str(len(data)))
        ])
        return iter([data])

then, i use nuitka, Nuitka is a Python compiler. after compiled, myapp.py change into myapp.exe file.

finally, i start gunicorn:

$ gunicorn -c config.py myapp.exe:app

Cause an error：

Traceback (most recent call last):
  File "/usr/local/bin/gunicorn", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 74, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 203, in run
    super(Application, self).run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 231, in run
    self.halt(reason=inst.reason, exit_status=inst.exit_status)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 344, in halt
    self.stop()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 393, in stop
    time.sleep(0.1)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 244, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 524, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

logfile:

[2017-07-28 06:34:49 +0000] [6621] [INFO] Booting worker with pid: 6621
[2017-07-28 06:34:49 +0000] [6621] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 578, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 135, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 356, in import_app
    raise ImportError(msg % (module.rsplit(".", 1)[0], obj))
ImportError: Failed to find application, did you mean 'myapp:app'?
[2017-07-28 06:34:49 +0000] [6621] [INFO] Worker exiting (pid: 6621)
[2017-07-28 06:34:49 +0000] [6616] [INFO] Shutting down: Master
[2017-07-28 06:34:49 +0000] [6616] [INFO] Reason: Worker failed to boot.

I use nuitka compiler because of the protection of the source code.

what should i do， Hope to receive your reply～

Source

aixiamomo

Most helpful comment

Compiling a python module into an executable binary and then attempting to import it from gunicorn is not going to work.

You might be able to write an application that invokes gunicorn programatically (imports gunicorn and uses its APIs directly, even if they aren't meant to be used that way) and that also includes your application code and then use Nuitka to compile that whole thing into an executable. It's possible that might work. But I'm not aware of that ever having been attempted.

Alternatively, you may be able to use the --module form of Nuitka compilation to compile your module or package. That should produce a standard shared object extension module that if you arrange to be on sys.path gunicorn should be able to find and import. This should be the same as compiling your module using Cython, and I would expect that to work under limited circumstances.

jamadden on 28 Jul 2017

👍3

All 5 comments

Compiling a python module into an executable binary and then attempting to import it from gunicorn is not going to work.

jamadden on 28 Jul 2017

👍3

thank you @jamadden

today, I found that there was such a section in the document

I'm trying to

aixiamomo on 31 Jul 2017

Then can we agree this is not a gunicorn issue, and close this ticket?

jamadden on 31 Jul 2017

Hello,
Sorry to re-open this but I use this method (compile a module with nuitka) and gunicorn gives immediatly a segfault if --threads=X with X>1. Any idea?

invenis-paris on 31 Oct 2017

@invenis-paris feel free to open a new issue if you think you have a new problem, and provide as much detail as you can. Specifically, example app, build instructions, and run command would help. Easiest is to maybe post an example repo. Thanks!