Kombu: unhashable type: 'Redis'

Created on 31 Jan 2020 · 5Comments · Source: celery/kombu

Version:

django-redis==4.10.0
redis==3.3.11
celery==4.4.0

Platform: Python 3.7.4 on Calculate Linux 18.12

Description: If i use Redis as rpc for Celery, i have problem with method __equal__ in Redis class.

[2020-01-31 14:46:05,659: CRITICAL/MainProcess] Unrecoverable error: TypeError("unhashable type: 'Redis'")
Traceback (most recent call last):
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
    blueprint.start(self)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 40, in start
    self.sync(c)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 44, in sync
    replies = self.send_hello(c)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/worker/consumer/mingle.py", line 57, in send_hello
    replies = inspect.hello(c.hostname, our_revoked._data) or {}
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/app/control.py", line 154, in hello
    return self._request('hello', from_node=from_node, revoked=revoked)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/app/control.py", line 106, in _request
    pattern=self.pattern, matcher=self.matcher,
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/celery/app/control.py", line 477, in broadcast
    limit, callback, channel=channel,
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/pidbox.py", line 347, in _broadcast
    channel=chan)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/pidbox.py", line 386, in _collect
    self.connection.drain_events(timeout=timeout)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/connection.py", line 315, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/transport/virtual/base.py", line 963, in drain_events
    get(self._deliver, timeout=timeout)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/transport/redis.py", line 366, in get
    self._register_BRPOP(channel)
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/transport/redis.py", line 311, in _register_BRPOP
    if not self._client_registered(channel, channel.client, 'BRPOP'):
  File "/home/psayker/PycharmProjects/ee/py37/lib/python3.7/site-packages/kombu/transport/redis.py", line 306, in _client_registered
    (channel, client, cmd) in self._chan_to_sock)
TypeError: unhashable type: 'Redis'

This possibly because in Kombu we use __equal__ method of Redis class. In Redis 3.3.11, all work fine.
https://github.com/celery/kombu/blob/master/kombu/transport/redis.py#L306
https://github.com/andymccurdy/redis-py/blob/master/redis/client.py#L749

Upstream py-redis

Source

Psayker

👍17

Most helpful comment

After some research I noticed that Redis class has __eq__ implemented but not __hash__.
From the python docs(https://docs.python.org/3/reference/datamodel.html#object.__hash__)

if it defines __eq__() but not __hash__(), its instances will not be usable as items in hashable collections.

It worked before because models have __hash__ and__eq__ methods by default.

Domol on 31 Jan 2020

👍5 👀3 🎉2

All 5 comments

After some research I noticed that Redis class has __eq__ implemented but not __hash__.
From the python docs(https://docs.python.org/3/reference/datamodel.html#object.__hash__)