Gunicorn: Upgrading to 19.7.0 means can't bind anymore

[hartym]

Upgraded to latest gunicorn and could not bind ip address anymore (simple wsgi app, ipv4 or ipv6 same story).

foo@xxx:~$ cd current
foo@xxx:~/current$ . env/bin/activate
foo@xxx:~/current$ gunicorn my.wsgi.package
[2017-03-10 18:50:46 +0000] [9242] [INFO] Starting gunicorn 19.7.0
[2017-03-10 18:50:46 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:47 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:48 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:49 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:50 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:51 +0000] [9242] [ERROR] Can't connect to ('127.0.0.1', 8000)

Reverting to 19.6.0 fixes the problem.

System version : Linux xxx 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u1 x86_64 GNU/Linux

Let me know if I can provide more info, 19.6.0 is acceptable solution for me but I guess that's a regression you wanna at least understand ?

[lee-pai-long]

Adding to @hartym comment using --log-level debug I get the following:

[2017-03-13 15:49:39 +0100] [7175] [INFO] Starting gunicorn 19.7.0
[2017-03-13 15:49:39 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:39 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:40 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:40 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:41 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:41 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:42 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:42 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:43 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:43 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:44 +0100] [7175] [ERROR] Can't connect to ('0.0.0.0', 8000)

[tilgovi]

It's very hard to imagine why this would be. Are you positive that nothing else was bound on the port at that time?

[tilgovi]

Hmm. It does seem you're binding to 0.0.0.0 now, when you used to be using 127.0.0.1. Does this make a difference?

[tilgovi]

Oh, I see. Both logs are from 19.7.0.

[lee-pai-long]

@tilgovi For my part I double checked, on the VM and try on another "empty" VM and the result is the same, in both case using 19.6 worked perfectly.

[tilgovi]

I suspect this is due to SO_REUSEPORT issues.

We probably need to make a change like this one in thriftpy: https://github.com/eleme/thriftpy/pull/264

[tilgovi]

Any chance you can test this?

The change would be in gunicorn/sock.py in the set_options method of BaseSocket class.

[tilgovi]

I've pushed a branch for this.

[benoitc]

@hartym @lee-pai-long can you test it ? If OK we will bump a 0.17.1 .

[tilgovi]

@hartym @lee-pai-long branch for testing is over at #1483.

[hartym]

Woops, sorry i did not see the thread before. I'm going to see if I can pull something quickly to test the suggested change.

[hartym]

Works for me.

I bumped the gunicorn version number back to 19.7.0, deployed to check the breakage was still there, it was indeed broken, applied the patch, restarted the server and the port bind now works again.

Thank you, will bumb to 19.7.1 when it will be available.

[tilgovi]

19.7.1 should be available now

[RazZziel]

19.7.1 is still broken for me:

root@server:/var/www/service# .venv/bin/gunicorn wsgi -c ../gunicorn.conf.py
[2017-04-25 08:56:45 +0000] [1234] [INFO] Starting gunicorn 19.7.1
Traceback (most recent call last):
  File "/var/www/service/.venv/lib/python3.5/site-packages/gunicorn/sock.py", line 44, in set_options
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
OSError: [Errno 92] Protocol not available

[benoitc]

master should fix it. can you test tit? A new release will land this week
anyway.

• benoit

On 25 April 2017 at 11:10:52, Ismael Barros² ([email protected])
wrote:

19.7.1 is still broken for me:

root@server:/var/www/service# .venv/bin/gunicorn wsgi -c ../gunicorn.conf.py
[2017-04-25 08:56:45 +0000] [1234] [INFO] Starting gunicorn 19.7.1
Traceback (most recent call last):
File "/var/www/service/.venv/lib/python3.5/site-packages/gunicorn/sock.py", line 44, in set_options
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
OSError: [Errno 92] Protocol not available

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/benoitc/gunicorn/issues/1480#issuecomment-296968404,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAA4opi2owK8jn0i6mSP4cEPMuUppM--ks5rzbicgaJpZM4MZsIC
.

[petermorlion]

I can verify the master works for me (while 19.7.1 doesn't).