Kombu: TypeError: unhashable type: 'Redis'

[subodh-malgonde]

I am using Celery with Django and Redis. I am on Mac, using Python 3.7 and redis server is installed (I tested using redis-cli ping). This is my configuration:

Django==2.2.7
celery==4.4.0
kombu==4.6.7
redis==3.4.0

This is my settings.py:

CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_CONN_MAX_AGE = 0

This is my project/celery.py:

from __future__ import absolute_import, unicode_literals
from celery import Celery

app = Celery('dhanwise', broker='redis://localhost:6379/0')
app.config_from_object('django.conf:settings', namespace='CELERY')

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

app.autodiscover_tasks()

This is my project/__init__.py:

from __future__ import absolute_import, unicode_literals
from .celery import app as celery_app

__all__ = ('celery_app',)

I run python manage.py runserver and then in a separate tab I run celery -A project worker -l info. I get this error:

 -------------- [email protected] v4.4.0 (cliffs)
--- ***** ----- 
-- ******* ---- Darwin-19.0.0-x86_64-i386-64bit 2020-01-31 15:29:28
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         project:0x10f9e7780
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . financial_planning.tasks.test_task

[2020-01-31 15:29:28,749: INFO/MainProcess] Connected to redis://localhost:6379/0
[2020-01-31 15:29:28,760: INFO/MainProcess] mingle: searching for neighbors
[2020-01-31 15:29:28,784: CRITICAL/MainProcess] Unrecoverable error: TypeError("unhashable type: 'Redis'")
Traceback (most recent call last):
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
    blueprint.start(self)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 40, in start
    self.sync(c)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 44, in sync
    replies = self.send_hello(c)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 57, in send_hello
    replies = inspect.hello(c.hostname, our_revoked._data) or {}
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/app/control.py", line 154, in hello
    return self._request('hello', from_node=from_node, revoked=revoked)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/app/control.py", line 106, in _request
    pattern=self.pattern, matcher=self.matcher,
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/celery/app/control.py", line 477, in broadcast
    limit, callback, channel=channel,
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/pidbox.py", line 352, in _broadcast
    channel=chan)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/pidbox.py", line 391, in _collect
    self.connection.drain_events(timeout=timeout)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/connection.py", line 323, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/transport/virtual/base.py", line 963, in drain_events
    get(self._deliver, timeout=timeout)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/transport/redis.py", line 374, in get
    self._register_BRPOP(channel)
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/transport/redis.py", line 311, in _register_BRPOP
    if not self._client_registered(channel, channel.client, 'BRPOP'):
  File "/Users/subodh/.local/share/virtualenvs/backend-oyMunorl/lib/python3.7/site-packages/kombu/transport/redis.py", line 306, in _client_registered
    (channel, client, cmd) in self._chan_to_sock)
TypeError: unhashable type: 'Redis'

[ank-s2k]

Same here:

[2020-01-31 04:17:41,250: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2020-01-31 04:17:41,255: DEBUG/MainProcess] | Worker: Building graph...
[2020-01-31 04:17:41,255: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Timer, Hub, Pool, Autoscaler, Beat, Consumer}
[2020-01-31 04:17:41,284: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2020-01-31 04:17:41,284: DEBUG/MainProcess] | Consumer: Building graph...
[2020-01-31 04:17:41,338: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Gossip, Tasks, Control, Agent, event loop}
[2020-01-31 04:17:41,408: DEBUG/MainProcess] | Worker: Starting Hub
[2020-01-31 04:17:41,408: DEBUG/MainProcess] ^-- substep ok
[2020-01-31 04:17:41,409: DEBUG/MainProcess] | Worker: Starting Pool
[2020-01-31 04:17:41,810: DEBUG/MainProcess] ^-- substep ok
[2020-01-31 04:17:41,811: DEBUG/MainProcess] | Worker: Starting Consumer
[2020-01-31 04:17:41,811: DEBUG/MainProcess] | Consumer: Starting Connection
[2020-01-31 04:17:41,828: INFO/MainProcess] Connected to ****
[2020-01-31 04:17:41,828: DEBUG/MainProcess] ^-- substep ok
[2020-01-31 04:17:41,828: DEBUG/MainProcess] | Consumer: Starting Events
[2020-01-31 04:17:41,837: DEBUG/MainProcess] ^-- substep ok
[2020-01-31 04:17:41,837: DEBUG/MainProcess] | Consumer: Starting Heart
[2020-01-31 04:17:41,840: DEBUG/MainProcess] ^-- substep ok
[2020-01-31 04:17:41,841: DEBUG/MainProcess] | Consumer: Starting Mingle
[2020-01-31 04:17:41,841: INFO/MainProcess] mingle: searching for neighbors
[2020-01-31 04:17:41,863: CRITICAL/MainProcess] Unrecoverable error: TypeError("unhashable type: 'Redis'")
Traceback (most recent call last):
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 318, in start
    blueprint.start(self)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/worker/consumer/mingle.py", line 40, in start
    self.sync(c)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/worker/consumer/mingle.py", line 44, in sync
    replies = self.send_hello(c)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/worker/consumer/mingle.py", line 57, in send_hello
    replies = inspect.hello(c.hostname, our_revoked._data) or {}
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/app/control.py", line 154, in hello
    return self._request('hello', from_node=from_node, revoked=revoked)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/app/control.py", line 98, in _request
    return self._prepare(self.app.control.broadcast(
  File "/home/dev/syncore/env/lib/python3.8/site-packages/celery/app/control.py", line 475, in broadcast
    return self.mailbox(conn)._broadcast(
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/pidbox.py", line 349, in _broadcast
    return self._collect(reply_ticket, limit=limit,
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/pidbox.py", line 391, in _collect
    self.connection.drain_events(timeout=timeout)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/connection.py", line 323, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 963, in drain_events
    get(self._deliver, timeout=timeout)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/transport/redis.py", line 374, in get
    self._register_BRPOP(channel)
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/transport/redis.py", line 311, in _register_BRPOP
    if not self._client_registered(channel, channel.client, 'BRPOP'):
  File "/home/dev/syncore/env/lib/python3.8/site-packages/kombu/transport/redis.py", line 306, in _client_registered
    (channel, client, cmd) in self._chan_to_sock)
TypeError: unhashable type: 'Redis'
[2020-01-31 04:17:41,867: DEBUG/MainProcess] | Worker: Closing Hub...
[2020-01-31 04:17:41,867: DEBUG/MainProcess] | Worker: Closing Pool...
[2020-01-31 04:17:41,867: DEBUG/MainProcess] | Worker: Closing Consumer...
[2020-01-31 04:17:41,867: DEBUG/MainProcess] | Worker: Stopping Consumer...

Solution: downgrade redis to redis==3.3.11

The latest release redis 3.4 broke this

[pawelad]

Duplicate of #1152

[auvipy]

pin your py-redis