Ptvs: Support for gevent in ptvsd

[william-gr]

Currently ptvsd cannot run in a gevent/greenlet script, it just hungs:

from gevent import monkey
monkey.patch_all()
import ptvsd

ptvsd.enable_attach("test", address=('0.0.0.0', 3000))
ptvsd.wait_for_attach()
ptvsd.break_into_debugger()

It would be great if we could debug gevent apps.

[zooba]

Agreed. Do you have any information on what needs to be changed to make this work?

Also, have you tried importing ptvsd first? Both packages do monkey-patching, so it may be an ordering issue that is easy to resolve.

[william-gr]

Unfortunately I don't have much.
I tried importing first but no luck there:

Press Enter to continue . . . Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ptvsd/visualstudio_py_debugger.py", line 1670, in loop
    inp = read_bytes(conn, 4)
  File "/usr/lib/python3.6/site-packages/ptvsd/visualstudio_py_util.py", line 137, in read_bytes
    received_data = conn.recv(count - len(b))
  File "/usr/lib64/python3.6/site-packages/gevent/_socket3.py", line 337, in recv
    self._wait(self._read_event)
  File "/usr/lib64/python3.6/site-packages/gevent/_socket3.py", line 157, in _wait
    self.hub.wait(watcher)
  File "/usr/lib64/python3.6/site-packages/gevent/hub.py", line 651, in wait
    result = waiter.get()
  File "/usr/lib64/python3.6/site-packages/gevent/hub.py", line 899, in get
    return self.hub.switch()
  File "/usr/lib64/python3.6/site-packages/gevent/hub.py", line 630, in switch
    return RawGreenlet.switch(self)
gevent.hub.LoopExit: ('This operation would block forever', <Hub at 0x7f3a587bbd58 epoll pending=0 ref=0 fileno=8>)

I think it is related to socket and threads. If thread, threading and socket modules are reloaded (or not patched), it works.

[zooba]

Hmm... guess we'll need to find some time to invest into digging into this. gevent does have quite a different socket model, and we use sockets for communication between the debuggee and debugger, so it's entirely possible that everything will break down to the point where this may be impossible.

[william-gr]

I would think it is possible to use the native socket and threading instead of gevent. ptvsd needs to be aware of gevent and use the original modules if thats the case:

e.g.
gevent.monkey.get_original('socket', 'socket')

Maybe it all needs to run in a low level thread. Just some ideas, I don't know enough of either to know if that would actually work.

[robertaistleitner]

I'm also running into this problem and I tried to figure out where we'd have to use the original (unpatched) versions of socket, threading, _thread, etc but unfortunately I wasn't able to fix it. It just doesn't respond at some point.

I'm using https://github.com/kennethreitz/flask-sockets which is based on gevent-websocket.

I also tried to get it working by some gist code I found here and claims to solve a similar issue with django, but no luck as well: https://gist.github.com/azinman/f201d97a1d4bf9e8496d66b0ee505cfe

I also had a look on other debugging solutions which work pretty well together with gevent like plain pdb or https://github.com/romanvm/python-web-pdb. Nevertheless it would be awesome to have the debugger in the VS Code to be more productive.

Any help is greatly appreciated!

[layzerar]

I think this script should work for most users, cheers.

from gevent import monkey
monkey.patch_all()

def setup_ptvsd(host='0.0.0.0', port=5678):
    import sys
    saved_modules = {}
    try:
        green_modules = set([
            'socket', 'ssl', 'select', 'urllib',
            'thread', 'threading', 'time', 'logging',
            'os', 'signal', 'subprocess', 'requests',
        ])
        for modname in list(sys.modules.keys()):
            if modname.partition('.')[0] in green_modules:
                saved_modules[modname] = sys.modules.pop(modname)

        import ptvsd
        ptvsd.enable_attach(address=(host, port), redirect_output=False)
    finally:
        sys.modules.update(saved_modules)

setup_ptvsd()

[carlegbert]

Something similar to https://github.com/Microsoft/PTVS/issues/2390#issuecomment-421613027 worked for me.

[Anakin-Hao]

@layzerar Hi. I see you code reloads the modules. Would that undermine monkey.patch_all()?

Would your entire process still using monkey patched modules after that?

[layzerar]

@Anakin-Hao Yes, It will break monkey.patch_all() temporary, and then try to restore it.

The above script should not be used for production purpose.

[cyberw]

The workaround by @layzerar used to work perfectly for me but now has issues.

At first I thought it had hung on patch_all(), but some debug printouts show that stepping forward the debugger advances my script, but VS doesnt show which line I am at and the call stack is empty.

[cyberw]

I found this: https://github.com/Microsoft/vscode-python/issues/127

Adding "gevent": true to my launch config made it work again! (with the layzerar's workaround still applied)