Kitty: test_multiprocessing_spawn not passed in 0.18.2

Created on 28 Jul 2020 · 7Comments · Source: kovidgoyal/kitty

Describe the bug
test_multiprocessing_spawn not passed inf 0.18.2 on some arches:

test_multiprocessing_spawn (kitty_tests.tui.TestTUI) ... /usr/lib64/python3.9/multiprocessing/resource_tracker.py:96: UserWarning: resource_tracker: process died unexpectedly, relaunching.  Some resources might leak.
  warnings.warn('resource_tracker: process died unexpectedly, '
ERROR
test_word_trie (kitty_tests.unicode_input.TestUnicodeInput) ... ok
======================================================================
ERROR: test_multiprocessing_spawn (kitty_tests.tui.TestTUI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/kitty-0.18.2/kitty_tests/tui.py", line 54, in test_multiprocessing_spawn
    test_spawn()
    self = <kitty_tests.tui.TestTUI testMethod=test_multiprocessing_spawn>
    test_spawn = <function test_spawn at 0x7f990d6133a0>
  File "/builddir/build/BUILD/kitty-0.18.2/kitty/multiprocessing.py", line 62, in test_spawn
    x = q.get(timeout=2)
    ctx = <multiprocessing.context.SpawnContext object at 0x7f990d69e730>
    get_context = <bound method DefaultContext.get_context of <multiprocessing.context.DefaultContext object at 0x7f990d6b2e20>>
    p = <SpawnProcess name='SpawnProcess-1' pid=896215 parent=896197 stopped exitcode=255>
    q = <multiprocessing.queues.Queue object at 0x7f990e7946d0>
  File "/usr/lib64/python3.9/multiprocessing/queues.py", line 114, in get
    raise Empty
    block = True
    deadline = 4362487.813879826
    self = <multiprocessing.queues.Queue object at 0x7f990e7946d0>
    timeout = 1.9999742964282632
_queue.Empty
----------------------------------------------------------------------
Ran 77 tests in 2.532s
FAILED (errors=1, skipped=2)

To Reproduce
Build Fedora RPM:

Task
Log

Expected behavior
Successful build as before, 0.18.1.

Environment details
OS: Fedora Rawhide (33)

bug

Source

tim77

All 7 comments

Works fine for me on 64bit arches with python 3.8nd there were no
changes to this code since 0.18.1

I'm guessing you are running it on python 3.9??

kovidgoyal on 28 Jul 2020

@kovidgoyal weird, tried already also for F32 where python 3.8.5 version and got the same error.

tim77 on 28 Jul 2020

The test is skipped on 32bits.

kovidgoyal on 28 Jul 2020

I can reproduce this test failure when building with nix on macOS and on Linux. I bisected it. ee2fdb3d25b2a9aad534357388fcaa9213f4d6b3 works and 50414b333a42e4b8901def497423d87148b4b7bc shows the test failure.
If you like, I can tell you how to install Nix and get this exact environment to build kitty in.

I once managed to get a different error message

======================================================================
ERROR: test_multiprocessing_spawn (kitty_tests.tui.TestTUI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/path/to/kitty/kitty_tests/tui.py", line 54, in test_multiprocessing_spawn
    test_spawn()
    self = <kitty_tests.tui.TestTUI testMethod=test_multiprocessing_spawn>
    test_spawn = <function test_spawn at 0x103e19ca0>
  File "/path/to/kitty/kitty/multiprocessing.py", line 59, in test_spawn
    q = ctx.Queue()
    ctx = <multiprocessing.context.SpawnContext object at 0x103ea0940>
    get_context = <bound method DefaultContext.get_context of <multiprocessing.context.DefaultContext object at 0x103c980d0>>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/context.py", line 103, in Queue
    return Queue(maxsize, ctx=self.get_context())
    Queue = <class 'multiprocessing.queues.Queue'>
    maxsize = 0
    self = <multiprocessing.context.SpawnContext object at 0x103ea0940>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/queues.py", line 42, in __init__
    self._rlock = ctx.Lock()
    ctx = <multiprocessing.context.SpawnContext object at 0x103ea0940>
    maxsize = 32767
    self = <multiprocessing.queues.Queue object at 0x103cf1430>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
    Lock = <class 'multiprocessing.synchronize.Lock'>
    self = <multiprocessing.context.SpawnContext object at 0x103ea0940>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
    ctx = <multiprocessing.context.SpawnContext object at 0x103ea0940>
    self = <Lock(owner=unknown)>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/synchronize.py", line 80, in __init__
    register(self._semlock.name, "semaphore")
    _after_fork = <function SemLock.__init__.<locals>._after_fork at 0x103e99c10>
    ctx = <multiprocessing.context.SpawnContext object at 0x103ea0940>
    i = 0
    kind = 1
    maxvalue = 1
    name = 'spawn'
    register = <bound method ResourceTracker.register of <multiprocessing.resource_tracker.ResourceTracker object at 0x103e1e820>>
    self = <Lock(owner=unknown)>
    sl = <_multiprocessing.SemLock object at 0x103e18cf0>
    unlink_now = False
    value = 1
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/resource_tracker.py", line 147, in register
    self._send('REGISTER', name, rtype)
    name = '/mp-l_x4dj8b'
    rtype = 'semaphore'
    self = <multiprocessing.resource_tracker.ResourceTracker object at 0x103e1e820>
  File "/nix/store/xs2039rla1yc3sigz9mird3mc3i04md5-python3-3.8.3/lib/python3.8/multiprocessing/resource_tracker.py", line 160, in _send
    nbytes = os.write(self._fd, msg)
    cmd = 'REGISTER'
    msg = b'REGISTER:/mp-l_x4dj8b:semaphore\n'
    name = '/mp-l_x4dj8b'
    rtype = 'semaphore'
    self = <multiprocessing.resource_tracker.ResourceTracker object at 0x103e1e820>
BrokenPipeError: [Errno 32] Broken pipe