Spleeter: [Bug] RuntimeError: dictionary changed size during iteration

[FranzSkuffka]

Description

I get a RuntimeError on OSX 10.15.4

Step to reproduce

Install using pip, try example.

Output

(env) ❯ spleeter separate -i spleeter-example.mp3  -p spleeter:2stems -o output
Traceback (most recent call last):
  File "/Users/janwirth/blog/env/bin/spleeter", line 10, in <module>
    sys.exit(entrypoint())
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/spleeter/__main__.py", line 54, in entrypoint
    main(sys.argv)
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/spleeter/__main__.py", line 36, in main
    enable_logging()
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/spleeter/utils/logging.py", line 60, in enable_logging
    tf_logger = get_tensorflow_logger()
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/spleeter/utils/logging.py", line 27, in get_tensorflow_logger
    from tensorflow.compat.v1 import logging
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow/__init__.py", line 99, in <module>
    from tensorflow_core import *
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow_core/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "<frozen importlib._bootstrap>", line 1019, in _handle_fromlist
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow/__init__.py", line 50, in __getattr__
    module = self._load()
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow/__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow_core/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/tensorflow_core/core/framework/graph_pb2.py", line 7, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/google/protobuf/__init__.py", line 37, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/__init__.py", line 83, in <module>
    __import__('pkg_resources.extern.packaging.requirements')
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 9, in <module>
    from pkg_resources.extern.pyparsing import stringStart, stringEnd, originalTextFor, ParseException
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/extern/__init__.py", line 43, in load_module
    __import__(extant)
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 4756, in <module>
    _escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1])
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1284, in setParseAction
    self.parseAction = list(map(_trim_arity, list(fns)))
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1066, in _trim_arity
    this_line = extract_stack(limit=2)[-1]
  File "/Users/janwirth/blog/env/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1050, in extract_stack
    frame_summary = traceback.extract_stack(limit=-offset+limit-1)[offset]
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 211, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 363, in extract
    f.line
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 285, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 48, in getlines
    for mod in sys.modules.values():
RuntimeError: dictionary changed size during iteration

(env) ❯ python --version
Python 3.7.3

Environment

| | |
| ----------------- | ------------------------------- |
| OS | OSX 10.15.4 |
| Installation type | pip |
| RAM available | 32g |

[dizzi90]

Issue replicated

env

MacBook Pro 13-inch 2020
OSX Catalina 10.15.5
Python 3.7.3
ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers

spleeter separate -i spleeter-example.mp3 -p spleeter:2stems -o output --verbose

Traceback (most recent call last):
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/bin/spleeter", line 8, in <module>
    sys.exit(entrypoint())
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/spleeter/__main__.py", line 54, in entrypoint
    main(sys.argv)
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/spleeter/__main__.py", line 36, in main
    enable_logging()
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/spleeter/utils/logging.py", line 60, in enable_logging
    tf_logger = get_tensorflow_logger()
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/spleeter/utils/logging.py", line 27, in get_tensorflow_logger
    from tensorflow.compat.v1 import logging
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow/__init__.py", line 99, in <module>
    from tensorflow_core import *
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow_core/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "<frozen importlib._bootstrap>", line 1019, in _handle_fromlist
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow/__init__.py", line 50, in __getattr__
    module = self._load()
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow/__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow_core/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/tensorflow_core/core/framework/graph_pb2.py", line 7, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/google/protobuf/__init__.py", line 37, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
    __import__('pkg_resources.extern.packaging.requirements')
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 9, in <module>
    from pkg_resources.extern.pyparsing import stringStart, stringEnd, originalTextFor, ParseException
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/extern/__init__.py", line 43, in load_module
    __import__(extant)
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 4756, in <module>
    _escapedPunc = Word( _bslash, r"\[]-*.$+^?()~ ", exact=2 ).setParseAction(lambda s,l,t:t[0][1])
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1284, in setParseAction
    self.parseAction = list(map(_trim_arity, list(fns)))
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1066, in _trim_arity
    this_line = extract_stack(limit=2)[-1]
  File "/Users/dizzi90/.local/share/virtualenvs/karaography-cP0VS7vQ/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1050, in extract_stack
    frame_summary = traceback.extract_stack(limit=-offset+limit-1)[offset]
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 211, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 363, in extract
    f.line
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 285, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 48, in getlines
    for mod in sys.modules.values():
RuntimeError: dictionary changed size during iteration

[jaywad]

@FranzSkuffka @dizzi90 I have done a workaround to fix the above issue and it works.

Change line 48 in the below file from for mod in sys.modules.values(): to for mod in list(sys.modules.values()):

File location:
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py

[dizzi90]

@jaywad Do you mean that the bug is upstream?

[jayhill90]

@FranzSkuffka @dizzi90 I have done a workaround to fix the above issue and it works.

Change line 48 in the below file from for mod in sys.modules.values(): to for mod in list(sys.modules.values()):

File location:
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py

yeah that totally broke my Max4Live synth Oscillot. Thanks.....

@dizzi90 I'd wager since it broke Oscillot it is not a bug upstream.

[mmoussallam]

Hi @FranzSkuffka @jayhill90

We did not manage to replicate this on our side but it seems to be a very low-level python-tensorflow interaction failure. We've just moved to TF2 which is arguably more stable on recent python distrib. So please update spleeter and let us know if this is still happening.