Kombu: KeyError async

[Sonictherocketman]

Was doing some late night builds when I noticed that none of my workers ran anymore.

After this version was pushed I started getting errors

AttributeError async
KeyError async

I think it has to do with some missing changes during this commit https://github.com/celery/kombu/commit/75695205f6e7af8e7e9178e010debc3871b19106

Downgrading to 4.1 solved the issue.

[nathan-jones]

Same here:

$ celery -A rtp worker -l info
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 123, in __getattr__
    return self[key]
KeyError: 'async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async

[pssuils]

Same here with kombu==4.2.0 and celery==4.1.0 however fixing the kombu==4.1.0 fixes the issue until version 4.2.0 is stable

[clokep]

I believe this is from #742 / #839.

Because of that change (+ a corresponding celery change), I believe kombu 4.2.0 is not compatible with celery 4.1.0 (and vice versa).

[auvipy]

could you check celery 4.1.1? it's a hotfix release

[alex-handley]

I think this might be related - https://github.com/celery/celery/pull/4679

My Airflow staging build has started failing with the same error

[auvipy]

this should be fixed by 4.1.1 check and report. we already got report airflow error was fixed by 4.1.1

[alex-handley]

Correct, upgrading and builds passing again. Thanks!

[auvipy]

thanks for updating!

[artinnok]

Same bug here:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 123, in __getattr__
    return self[key]
KeyError: 'async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async

[auvipy]

@artinnok please update to celery 4.1.1 or 4.2rc4

[clintonb]

This issue was introduced by #839; however, I think there's another issue: versioning. Kombu appears to follow semantic versioning, but that is not documented anywhere. If semantic versioning is followed, this backwards-incompatible change should have resulted in a bump to version 5.0.0.

The third issue lies in the fact that Celery's dependencies have no upper bound: https://github.com/celery/celery/blob/master/setup.cfg#L18-L21. Even if a major version change had gone into effect, our installations would have still been broken. Celery should have an upper bound to prevent problems like this in the future.

[auvipy]

i agree that should be pinned

[adrienjt]

As @clintonb mentioned, this is a versioning issue in Kombu. Rather than moving to asynchronous in Celery 4.1.1 and ask everybody to upgrade to it, Kombu 4.2.1 should have reinstated the async module, to delete it only in version 5. To avoid further damage (breaking builds in many projects pinned to Celery 4.1.0), I strongly recommend that Kombu reinstate the async module in a 4.2.2. Please re-open the issue.

[auvipy]

just pin your kombu and celery.

[thedrow]

@adrienjt I agree this was an oversight. We'll take more care in the future.