Flask-socketio: Requests not returning when using eventlet or gevent

[tjppires]

I have an app that receives messages using socketio and then queries a service (dialogflow) and takes different actions depending on the answer. Everything works fine when I am using async_mode="threading", but if I switch to eventlet (even using monkey_patch) or gevent, the query to dialogflow (post) never returns an answer.

How do I fix this?

[miguelgrinberg]

I don't have enough information to help you, unfortunately. If you are making an HTTP request out to a 3rd party service, and you have properly monkey patched the standard library for eventlet, then it should not be a problem. Something in your application isn't the way you think it is. Do you want to share some code?

[tjppires]

Hi!

I managed to find the problem: there was a library using the threading module. Replacing it by eventlet solved the issue. Also, I had to use eventlet.patcher.monkey_patch(select=True, socket=True) instead of gevent.monkey_patch().

Now I just have one small problem: after adding eventlet as the async_mode, I stopped seeing the logs from the webserver, but I still can see the logs from other external libraries. If I add debug=True as an argument to socketio.run, I get the logs, but the part of my code that uses threading (now eventlet) stops working. Why does this happen?

Thanks!

[miguelgrinberg]

If you are using eventlet, you need to monkey patch as follows:

import eventlet
eventlet.monkey_patch()

This should, I think, help make the problem you have with debug mode go away.

[tjppires]

That does indeed get the logs back. But then the rest of the program (the part that used threading, and now uses eventlet) stops working 😞

Also, if I use eventlet.monkey_patch() instead of eventlet.patcher.monkey_patch(select=True, socket=True), even without logs, the threading part doesn't work.

[miguelgrinberg]

That makes me think that this thing you call the "threading part" is not compatible with eventlet. I can't really help you because I don't know what that is, but if monkey patching makes that code break, then that's a problem, because using normal threading alongside eventlet is also likely to give you trouble.

[tjppires]

The "threading part" is just a simple thread (now a eventlet thread) that is continuously listening to a queue, and may send websocket messages when something in the queue triggers it. It calls an external library that makes requests to a webservice, but this part is outside my control.

But thanks a lot for your time, not having the logs shouldn't be that much of an issue, at least for now.

[miguelgrinberg]

It calls an external library that makes requests to a webservice

Is this Python code as well? You need to make sure that this call isn't blocking. Monkey patching should do this, if you are still in Python. The queue should be fine, because after monkey patching you will get the eventlet queue.

[tjppires]

Yes, it's still python. I think the call may be blocking. I put the monkey patching before I import it, and it didn't solve the problem.

[miguelgrinberg]

What package is this?

[tjppires]

This one: dialogflow

[miguelgrinberg]

I don't see anything that would break with eventlet, to be honest, it's just HTTP requests. Maybe you should debug this a bit more to see if we can figure out the problem. Is the thread blocking completely? If your thread has a loop inside which you wait on the queue and/or send some requests, then everything should be fine, the problem happens if you have long periods of time during which there is no I/O activity.

[tjppires]

I managed to debug this a bit more. Dialogflow stops working if I add the flag thread=True to eventlet.patcher.monkey_patch(.), and I get the exception: google.api_core.exceptions.ServiceUnavailable: 503 Deadline Exceeded (after some seconds waiting for a response).

[miguelgrinberg]

The thread=True option is what makes threads become eventlet compatible. Without that option, you are using normal threads, which may or may not work, but is usually not recommended for eventlet/gevent applications.

[tjppires]

Ok, thank you. I will ping the dialogflow maintainers to see if they can figure out why this happens.

[abhishmitra]

Hey @tjppires! Any solution here?

[Horsetoast]

Having the same problem with dialogflow and eventlet

[vitoresende]

Hi @tjppires, do you have any news about it? I'm facing the same problem with eventlet and dialogflow.

[assafelovic]

That does indeed get the logs back. But then the rest of the program (the part that used threading, and now uses eventlet) stops working 😞

Also, if I use eventlet.monkey_patch() instead of eventlet.patcher.monkey_patch(select=True, socket=True), even without logs, the threading part doesn't work.

this solved the issue for me thank you @tjppires

[funsholaniyi]

@tjppires can you share how you eventually configured it to work, I am also using dialogflow and I am hitting 503 deadline exceeded error, any other external endpoint works fine except that of dialogflow

[funsholaniyi]

@tjppires can you share how you eventually configured it to work, I am also using dialogflow and I am hitting 503 deadline exceeded error, any other external endpoint works fine except that of dialogflow

I later fixed it by delegating dialogflow calls to celery task runner, since I am not using eventlet at the worker instance

And your main app that uses eventlet, you can decide to patch just the socket using eventlet.monkey_patch(select=True, socket=True) or just patch all using eventlet.monkey_patch()

Hope this helps someone

[tjppires]

Hi all,

I worked on this a couple of years ago, at my previous job. I solved the problem back then, but meanwhile, I left the company, and I don't have access to the code to be able to help you, nor do I remember how I solved the problem. My apologies, hope you figure it out.

[funsholaniyi]

Google core libraries do not seem to support eventlet base library - libevent at this time, see reference Link

Another fix is to use gevent and patch it with

from gevent import monkey

monkey.patch_all()
import grpc.experimental.gevent

grpc.experimental.gevent.init_gevent()

[riverscuomo]

Hi Miguel, I have the same problem as the others here. I built my app with eventlet only to find that Google's Dialogflow won't respond to the app after it's deployed.

I'm going to try one or both of the solutions mentioned by @funsholaniyi.

First, I'm trying to switch to gevent, but I'm having some problems getting the app to work at all. When I run locally, I get these warnings:

C:\Dropbox (RC)\Apps\riverscuomo_com\application.py:9: MonkeyPatchWarning: Monkey-patching outside the main native thread. Some APIs will not be available. Expect a KeyError to be printed at shutdown. monkey.patch_all() C:\Dropbox (RC)\Apps\riverscuomo_com\application.py:9: MonkeyPatchWarning: Monkey-patching not on the main thread; threading.main_thread().join() will hang from a greenlet monkey.patch_all()

When I deploy to Heroku, it builds successfully but then immediately shows errors like:

heroku[router]: at=error code=H13 desc="Connection closed without response" method=POST path="/socket.io/?EIO=3&transport=polling&t=1601148400299-678&sid=b18272b681f0463e9101babaf917a909" host=riverscuomo.com request_id=613e9e1a-bc3a-430c-9537-cc8fee6d7902 fwd="47.154.22.108,172.69.33.85" dyno=web.1 connect=2ms service=12835ms status=503 bytes=0 protocol=http

Any help would be much appreciated. This is how I got here: I put this block at the top of my code in place of the eventlet import.

from gevent import monkey
monkey.patch_all()
import grpc.experimental.gevent
grpc.experimental.gevent.init_gevent()

And this in my Heroku Procfile:

web: gunicorn -k gevent -w 1 application:app

I'm running Python 3.6 with the following requirements (among others):

eventlet==0.25.2
gevent==20.6.2
gevent-websocket==0.10.1
greenlet==0.4.16

[miguelgrinberg]

Like most Google APIs, Dialogflow is incompatible with eventlet and gevent, and Google hasn't shown interest in adding support. Sorry.

[riverscuomo]

I see. Thank you for responding. Do you have any thoughts about @funsholaniyi 's other strategy? "I later fixed it by delegating dialogflow calls to celery task runner, since I am not using eventlet at the worker instance". I've never used Celery before but that would be my next effort.

[miguelgrinberg]

@riverscuomo I have no experience with that, so I do not know if that works.