Flask-socketio: [ERROR] Socket error processing request.

[Bekt]

I get the following error every now and then when running with gunicorn. I am not sure why or when this happens.

[2015-10-18 19:56:52 -0500] [17657] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File ".../venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 64, in handle
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File ".../venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 52, in handle
    self.handle_request(listener_name, req, client, addr)
  File ".../venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 127, in handle_request
    six.reraise(*sys.exc_info())
  File ".../venv/lib/python2.7/site-packages/gunicorn/workers/async.py", line 113, in handle_request
    resp.write(item)
  File ".../venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 324, in write
    self.send_headers()
  File ".../venv/lib/python2.7/site-packages/gunicorn/http/wsgi.py", line 320, in send_headers
    util.write(self.sock, util.to_bytestring(header_str))
  File "...venv/lib/python2.7/site-packages/gunicorn/util.py", line 300, in write
    sock.sendall(data)
  File ".../venv/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
    tail = self.send(data, flags)
  File ".../venv/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
    total_sent += fd.send(data[total_sent:], flags)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 174, in _dummy
    raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor

[miguelgrinberg]

Do things continue to work after this error?
Could this be happening when you close a browser tab that was connected to the server?

[Bekt]

Yea, everything continues to work fine. This happens when I only have one connection open as well.

I thought it may be because of multiple gunicorn workers so I tried with only one worker. Same thing happens.

[miguelgrinberg]

This usually happens when a connection is closed by the client. This can happen when running long-polling HTTP requests if you close the tab that the application was on, or also if you hit refresh. The problem is that when the long request ends gunicorn tries to write the response to the socket, but finds that the socket is gone because it was closed by the client.

If you think this description matches the problem, then I would not worry about it. It may be a good idea to report it to gunicorn, if you find a way to reliably reproduce it.

[Bekt]

I see that's the long-polling request is finished and the connection is upgraded to websocket. And whenever a client leaves, I see that disconnect event is firing properly. Would your response still be the case in this scenario?

[miguelgrinberg]

Well, it's really hard to be sure. The stack trace clearly points to an HTTP request, not WebSocket. I can't really tell if this is a long-polling request or a regular request, but you can see that it comes from the wsgi module. The failure occurred when gunicorn was trying to write the HTTP response of this request to the socket, in particular while it tried to write the headers.

I can't really say more than that from the stack trace, this is clearly internal to gunicorn. If you say that the connection was upgraded to WebSocket before this happened, then it's odd that there are still HTTP request flying (assuming your application doesn't still do some Ajax on the side that is).

So as I said before, it would be good to try to identify what actions are you taking that trigger this crash.

[Bekt]

:+1:

I'll look into this more later. In the meantime, here's my code:

venv/bin/gunicorn --worker-class eventlet app:app

flask-socketio==1.0b4
eventlet

from flask import Flask, render_template
from flask_socketio import SocketIO, emit

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, resource='ws')

@app.route('/')
def index():
    return render_template('index.html')

@socketio.on('connect')
def connect():
    print('connected')
    emit('my response', {'data': 'Connected'})

@socketio.on('disconnect')
def disconnect():
    print('disconnected')

print('starting')

<!doctype html>
<html>
<head>
  <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/socket.io/1.3.5/socket.io.min.js"></script>
  <script type="text/javascript">
    var url = 'http://' + document.domain + ':' + location.port;
    var socket = io(url, {path: '/ws'});
    socket.on('connect', function() {
        console.log('client connected');
    });
  </script>
</head>
<body>
  sup
</body>
</html>

[NotSureAbout]

Bekt - did you ever get to the bottom of this? Im having the same problem.

[kamal8]

I am having the same problem

2016-03-06 13:02:20 [12012] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 45, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 102, in handle_request
resp.close()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/http/wsgi.py", line 369, in close
self.send_headers()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/http/wsgi.py", line 288, in send_headers
util.write(self.sock, util.to_bytestring(header_str))
File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 298, in write
sock.sendall(data)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 383, in sendall
tail = self.send(data, flags)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 377, in send
return self._send_loop(self.fd.send, data, flags)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 364, in _send_loop
return send_method(data, *args)
File "/usr/lib/python2.7/socket.py", line 170, in _dummy
raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor

[Bekt]

I never had a chance to dig into this more, but I did deploy a small app that seems to be working even with the error messages.

[miguelgrinberg]

These are benign errors that I believe to be caused by gunicorn, eventlet and gevent. The error means that the server tried to write or read from a socket that out of nowhere got closed. The issue occurs when the client abruptly leaves a HTTP or WebSocket connection, without properly closing it. The typical case is when you close the browser window. In many cases the web server detects this condition and suppresses the "broken pipe", "bad file descriptor" or similar error that results from it, but there are several situations where this error is not capture, so you see it in your log.

[miguelgrinberg]

Several issues that are possibly related to this problem were fixed since this was reported. I'm going to assume this is fixed. Please reopen if this problem persists.

[geasyheart]

hello,I am having the same problem,

C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\Scripts\python.exe C:/Users/geasy/PycharmProjects/tmp/flask-chat/manage.py
(6120) wsgi starting up on http://127.0.0.1:80
(6120) accepted ('127.0.0.1', 56135)
(6120) accepted ('127.0.0.1', 56136)
(6120) accepted ('127.0.0.1', 56137)
(6120) accepted ('127.0.0.1', 56138)
(6120) accepted ('127.0.0.1', 56139)
(6120) accepted ('127.0.0.1', 56140)
(6120) accepted ('127.0.0.1', 56144)
ce05223a46814122a8c11782b2419f85
ce05223a46814122a8c11782b2419f85
Traceback (most recent call last):
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\greenpool.py", line 88, in _spawn_n_impl
    func(*args, **kwargs)
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\wsgi.py", line 734, in process_request
    proto.__init__(sock, address, self)
  File "C:\Users\geasy\AppData\Local\Programs\Python\Python35\lib\socketserver.py", line 683, in __init__
    self.finish()
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\wsgi.py", line 651, in finish
    greenio.shutdown_safe(self.connection)
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\greenio\base.py", line 479, in shutdown_safe
    return sock.shutdown(socket.SHUT_RDWR)
OSError: [WinError 10038] 在一个非套接字上尝试了一个操作。
29a7aa9376414bd5aa8e3425d441b5e3
(6120) accepted ('127.0.0.1', 56146)
Traceback (most recent call last):
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\greenpool.py", line 88, in _spawn_n_impl
    func(*args, **kwargs)
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\wsgi.py", line 734, in process_request
    proto.__init__(sock, address, self)
  File "C:\Users\geasy\AppData\Local\Programs\Python\Python35\lib\socketserver.py", line 683, in __init__
    self.finish()
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\wsgi.py", line 651, in finish
    greenio.shutdown_safe(self.connection)
  File "C:\Users\geasy\PycharmProjects\tmp\flask-chat\venv\lib\site-packages\eventlet\greenio\base.py", line 479, in shutdown_safe
    return sock.shutdown(socket.SHUT_RDWR)
OSError: [WinError 10038] 在一个非套接字上尝试了一个操作。

the log consists of two parts, the first part is the normal exit, the second part is directly close the browser.

class Joined(Namespace):
    def on_connect(self):
        print(request.sid)
        room = session.get('room')
        count = redis_db.incr("count")
        emit('count', {'count': count}, room=room)

    def on_joined(self, message):
        name = session.get('name')
        room = session.get('room')
        join_room(room)

        emit('status', {'message': '{}:enter  room.'.format(name)}, room=room)

    def on_admin(self, message):
        room = session.get('room')
        emit('message', {"message": message['message']}, room=room)

    def on_broad(self, message):
        emit('message', {"message": "broadcast message:{}".format(message['message'])}, broadcast=True)

    def on_left(self, message):
        room = session.get('room')
        leave_room(room)
        print(request.sid)
        emit('status', {'message': '{} leave room.'.format(session['name'])}, room=room)

venv:

appdirs==1.4.0
click==6.7
enum-compat==0.0.2
eventlet==0.20.1
Flask==0.12
Flask-SocketIO==2.8.4
greenlet==0.4.12
itsdangerous==0.24
Jinja2==2.9.5
MarkupSafe==0.23
packaging==16.8
pyparsing==2.1.10
python-engineio==1.2.2
python-socketio==1.7.1
redis==2.10.5
six==1.10.0
Werkzeug==0.11.15

looking forward to your reply,thanks!

[miguelgrinberg]

As I mentioned above, these errors are benign, and for the most part can be ignored. They happen because the web server (Eventlet in this case) do not catch the broken pipe or bad descriptor error that occur when the client leaves the connection.

[geasyheart]

Bingo ! Thank you!

[Jolly23]

Thank you