Kombu: importlib_metadata is not Python 2 compatible.

[andyewen]

We've been getting the following error when trying to use celery 4.3.0 (kombu 4.6.4):

File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
    from celery.utils.threads import LocalStack
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 8, in <module>
    from .functional import memoize  # noqa
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/functional.py", line 10, in <module>
    from kombu.utils.functional import (LRUCache, dictfilter, is_list, lazy,
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 5, in <module>
    from .compat import fileno, maybe_fileno, nested, register_after_fork
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 10, in <module>
    import importlib_metadata
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 9, in <module>
    import zipp
  File "/usr/local/lib/python2.7/dist-packages/zipp.py", line 12, in <module>
    import more_itertools
  File "/usr/local/lib/python2.7/dist-packages/more_itertools/__init__.py", line 1, in <module>
    from more_itertools.more import *  # noqa
  File "/usr/local/lib/python2.7/dist-packages/more_itertools/more.py", line 340
    def _collate(*iterables, key=lambda a: a, reverse=False):
                               ^
SyntaxError: invalid syntax

The changelog mentions that importlib_metadata (mentioned in the traceback) was added in kombu 4.6.4.

Use importlib-metadata instead of pkg_resources for better performance

Pinning to kombu 4.6.3 seems to fix the problem for me.

[auvipy]

probably need to be revert that

[auvipy]

https://github.com/celery/kombu/pull/1071 is the PR can you comment there as well?

[davidszotten]

i wonder if this is similar to https://github.com/jaraco/zipp/issues/14

could you try (in a fresh virtualenv) to first upgrade pip (pip install -U pip) and then installing kombu again? (you need a new enough pip that knows to only install versions of packages that still have py2 support)

[matteius]

@andyewen I checked and more-itertools droped python 2 support when it went to version 6.0.0 -- it looks like 5.0.0 is the last version to support py2. Add a hard fixed requirement to your requirements in addition to where you have listed kombu:
more-itertools==5.0.0
See: https://pypi.org/project/more-itertools/5.0.0/ -- if you go to 6.0.0 release notes says "Python 2.7 is no longer supported. The 5.0.0 release will be the last version targeting Python 2.7."

If you still have issues with pinning versions of more-itertools then feel free to reopen this issue.

[andyewen]

Upgrading pip has fixed my problem. I was previously using the default pip version from Ubuntu 16.04.

[auvipy]

might be related https://github.com/celery/celery/issues/5699