Gunicorn: Gunicorn not properly communicating with nginx

Created on 28 Apr 2017 · 6Comments · Source: benoitc/gunicorn

I am working on a deploying a rest API generated by Swagger in python flask and I am running into issues with gunicorn communicating with nginx properly. Through nginx, I can access the content of the webapp, however everything seems to be in test/plain MIME and the error codes are not passed properly from gunicorn to nginx and nginx displays every response from gunicorn including (400s and 500s responses) with a 200 status code.

I currently have the entire swagger UI and my python flask app properly running though gunicorn. I use the following command to start the app. The json and html MIME types are correct on the URIs and the status codes are passed properly.

gunicorn swagger_server:app -b 0.0.0.0:8000

I then switch gunicorn to bind to a internal unix socket using the following command

gunicorn swagger_server:app -b unix:/tmp/gunicorn.sock

and restart nginx with a conf file very similar to the nginx configurartion suggested on gunicorn docs page. My conf file is the following:

  worker_processes 1;
  events {
    worker_connections 1024; # increase if you have lots of clients
    accept_mutex off; # set to 'on' if nginx worker_processes > 1
    # 'use epoll;' to enable for Linux 2.6+
    # 'use kqueue;' to enable for FreeBSD, OSX
  }


  http {
    include mime.types;
    # fallback in case we can't determine a type
    default_type application/octet-stream;
    access_log /tmp/nginx.access.log combined;
    sendfile on;

    server_names_hash_bucket_size 64;

    upstream app_server {
      # fail_timeout=0 means we always retry an upstream even if it failed
      # to return a good HTTP response

      # for UNIX domain socket setups
      server unix:/tmp/gunicorn.sock fail_timeout=0;

      # for a TCP configuration
      # server 172.16.132.91:8000 fail_timeout=0;
    }

    server {
      # if no Host match, close the connection to prevent host spoofing
      listen 80 default_server;
      return 444;
    }

    server {
      # use 'listen 80 deferred;' for Linux
      # use 'listen 80 accept_filter=httpready;' for FreeBSD
      listen 80 deferred;
      client_max_body_size 4G;

      # set the correct host(s) for your site
      server_name 172.16.132.91;


      keepalive_timeout 5;

      # path for static files
      root /path/to/app/current/public;

      location / {
        # checks for static file, if not found proxy to app
        try_files $uri @proxy_to_app;
      }

      location @proxy_to_app {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # enable this if and only if you use HTTPS
        # proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Host $http_host;
        # we don't want nginx trying to do something clever with
        # redirects, we set the Host: header above already.
        proxy_redirect off;
        proxy_pass http://app_server;
      }

      error_page 500 502 503 504 /500.html;
      location = /500.html {
        root /path/to/app/current/public;
      }
    }
  }

I have only removed

  user nobody nogroup;
  # 'user nobody nobody;' for systems with 'nobody' as a group instead
  pid /tmp/nginx.pid;
  error_log /tmp/nginx.error.log;

from the conf file.

At thins point when I try to access the app over port 80 on chrome browser or using cURL commands, I receive a plain text version of the response which can be a json or a html doc and it always reports error code 200.

For example, below is the response from directly accessing gunicorn on a URI with a json error page and a 404 status.

now here is accessing the same URI through nginx.

as you can see the http header is attached to the json, and the status code is wrongly reported as 200.

Another example is

Gunicorn:

nginx

Again gunicorn proprly handles the response. However, for nginx the html code is treated as plain text and the http header is again added to the body.

What am I missing here?

P.S.

My nginx is version 1.8.1

I have tried serving the website through localhost tcp instead of unix socket. and the behaviors are exactly the same.

Also, I have tried the same URIs over cURL and firefox and again same behaviors .

Source

momemarian

Most helpful comment

This issue turned out to be a nginx problem. This setup was sitting on an ARM processor and when we cross compiled nginx, GCC -o3 optimization resulted in a bad binary that had issues properly understating the http headers coming from gunicorn.

Here is a link to a very similar issue trac.nginx.org/nginx/ticket/899

momemarian on 17 May 2017

👍3