Celery: documation fix for django 1.7 "App registry isn't ready yet"

[thenewguy]

The celery app file needs to call django.setup in django 1.7 or you get an exception stating "App registry isn't ready yet" when a task is run.

See https://docs.djangoproject.com/en/dev/releases/1.7/#standalone-scripts

The documentation for first steps with django (http://docs.celeryproject.org/en/master/django/first-steps-with-django.html) need to be updated to reflect this.

[ask]

This is fixed in 3.1 branch, and will be part of 3.1.13

[ask]

Looking into it, this should already have been fixed in 3.1.12.

It definitely also works for me when starting the worker? Exactly when are you having a problem?

The proj/celery.py module should most definitely not call django.setup(), it's handled by the "django fixups" when the worker starts.

[thenewguy]

I think this was an issue with my virtualenv. I uninstalled and reinstalled celery and am not seeing the issue now after your last comment.

[mpasternak]

Looks like this is NOT fixed in 3.1.13

[ask]

What do you mean by looks?

[mpasternak]

Well, I am struggling with this problem now. I'm unable to access any models in my tasks.py file, because of AppRegistryNotReady . I am using 3.1.13 from PyPI.

One question, as I am debugging stuff... should celery.fixups.django.DjangoWorkerFixup.validate_models be called once when the celery daemon starts, or once per each worker?

It looks like it is called once and the stack looks like this by that time:

[mpasternak]

And yes, I am on Windows 8 + Python 2.7 32-bit.

[mpasternak]

As for 3.1.13, please re-open this bug.

Even if django.setup() procedure is called in django.py, I am getting this traceback.

[2014-07-18 16:15:35,117: ERROR/Worker-1] bpp.tasks.zaktualizuj_opis[8e93003c-fca4-4970-8be3-32eeede5fbb1]: Problem z pobraniem obiektu
Traceback (most recent call last):
  File "C:/Users/dotz/Desktop/django-bpp\bpp\tasks.py", line 79, in zaktualizuj_opis
    obj = klasa.objects.get(pk=pk)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\manager.py", line 92, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\query.py", line 345, in get
    clone = self.filter(*args, **kwargs)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\query.py", line 689, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\query.py", line 707, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\sql\query.py", line 1287, in add_q
    clause, require_inner = self._add_q(where_part, self.used_aliases)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\sql\query.py", line 1314, in _add_q
    current_negated=current_negated, connector=connector)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\sql\query.py", line 1138, in build_filter
    lookups, parts, reffed_aggregate = self.solve_lookup_type(arg)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\sql\query.py", line 1076, in solve_lookup_type
    _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\sql\query.py", line 1339, in names_to_path
    field, model, direct, m2m = opts.get_field_by_name(name)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\options.py", line 416, in get_field_by_name
    cache = self.init_name_map()
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\options.py", line 445, in init_name_map
    for f, model in self.get_all_related_m2m_objects_with_model():
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\options.py", line 561, in get_all_related_m2m_objects_with_model
    cache = self._fill_related_many_to_many_cache()
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\db\models\options.py", line 575, in _fill_related_many_to_many_cache
    for klass in self.apps.get_models():
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\utils\lru_cache.py", line 101, in wrapper
    result = user_function(*args, **kwds)
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\apps\registry.py", line 156, in get_models
    self.check_ready()
  File "c:\users\dotz\desktop\django-bpp\venv\lib\site-packages\django\apps\registry.py", line 119, in check_ready
    raise AppRegistryNotReady()
AppRegistryNotReady

[ask]

It's not the same issue though, under no circumstances should users add django.setup() to proj/celery.py.

Windows does not have fork, so it emulates it by starting a brand new process (like exec in posix). May very well be that the fix ups should be applied to the child processes as well.

[mpasternak]

What would be the best place to call django.setup() in worker code?

I added django.setup() celery.apps.worker.Worker.on_after_init , without any success.

[ask]

worker_process_init signal perhaps

[j7nn7k]

Same here. My setup Django 1.7c2, celery 3.1.13.

send_messages.delay(self) # <- self is a model_onstance

@shared_task
def send_messages(jmodel_instance):

    if job_model_instance.field_type == 'SM':
        do_stuff()

Error:

raise AppRegistryNotReady("Models aren't loaded yet.")

[hheimbuerger]

I just had the same issue with Django 1.7rc2 and celery 3.1.13 in a basically empty test project. I don't even have or want any models, so I can confirm that this in my tasks.py fixed the issue for me:

from celery.signals import worker_process_init
@worker_process_init.connect
def configure_workers(*args, **kwargs):
    import django
    django.setup()

[j7nn7k]

Doesn't work for me unfortunately.

[ask]

Are you all using Windows? Do you not get a traceback when this happens?

[j7nn7k]

I' using a Mac. Here's the complete trace.

    DecodeError: (AppRegistryNotReady("Models aren't loaded yet.",), <function model_unpickle at 0x10d44f668>, (('jobs', 'Job'), [], <function simple_class_factory at 0x10d44f5f0>))
    [2014-08-06 19:59:00,636: CRITICAL/MainProcess] Can't decode message body: DecodeError(AppRegistryNotReady(AppRegistryNotReady("Models aren't loaded yet.",), <function model_unpickle at 0x10d44f668>, (('jobs', 'Job'), [], <function simple_class_factory at 0x10d44f5f0>)),) [type:u'application/x-python-serialize' encoding:u'binary' headers:{}]

    body: '\x80\x02}q\x01(U\x07expiresq\x02NU\x03utcq\x03\x88U\x04argsq\x04cdjango.db.models.base\nmodel_unpickle\nq\x05U\x04jobsq\x06U\x03Jobq\x07\x86]cdjango.db.models.base\nsimple_class_factory\nq\x08\x87Rq\t}q\n(U\x0f_schedule_cacheNU\x0btemplate_idK\x01U\x11_created_by_cachecdjango.contrib.auth.models\nUser\nq\x0b)\x81q\x0c}q\r(U\x08usernameq\x0eX\x06\x00\x00\x00jannikU\nfirst_nameq\x0fX\x00\x00\x00\x00U\tlast_nameq\x10X\x00\x00\x00\x00U\tis_activeq\x11\x88U\x06_stateq\x12cdjango.db.models.base\nModelState\nq\x13)\x81q\x14}q\x15(U\x06addingq\x16\x89U\x02dbq\x17U\x07defaultq\x18ubU\x05emailq\x19X\x17\x00\x00\[email protected]\x0cis_superuserq\x1a\x88U\x08is_staffq\x1b\x88U\nlast_loginq\x1ccdatetime\ndatetime\nq\x1dU\n\x07\xde\x08\x05\x1255\x07x\x0ccpytz\n_UTC\nq\x1e)Rq\x1f\x86Rq... (1211b)
    Traceback (most recent call last):
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/messaging.py", line 586, in _receive_callback
        decoded = None if on_m else message.decode()
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/message.py", line 142, in decode
        self.content_encoding, accept=self.accept)
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/serialization.py", line 184, in loads
        return decode(data)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/serialization.py", line 59, in _reraise_errors
        reraise(wrapper, wrapper(exc), sys.exc_info()[2])
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/serialization.py", line 55, in _reraise_errors
        yield
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/serialization.py", line 184, in loads
        return decode(data)
      File "MYPROJECT/venv/lib/python2.7/site-packages/kombu/serialization.py", line 64, in pickle_loads
        return load(BytesIO(s))
      File "MYPROJECT/venv/lib/python2.7/site-packages/django/db/models/base.py", line 1471, in model_unpickle
        model = apps.get_model(*model_id)
      File "MYPROJECT/venv/lib/python2.7/site-packages/django/apps/registry.py", line 199, in get_model
        self.check_models_ready()
      File "MYPROJECT/venv/lib/python2.7/site-packages/django/apps/registry.py", line 131, in check_models_ready
        raise AppRegistryNotReady("Models aren't loaded yet.")
    DecodeError: (AppRegistryNotReady("Models aren't loaded yet.",), <function model_unpickle at 0x10d44f668>, (('jobs', 'Job'), [], <function simple_class_factory at 0x10d44f5f0>))

[ask]

Commit 0882ab5b7d7cfcd4cb43b7240352215425eb471d should fix this on Windows

[ask]

@j7nn7k You're 100% sure you don't use djcelery.setup_loader()?

[hheimbuerger]

I _am_ using djcelery.setup_loader(), why would I not?

It's in the instructions, and if I don't have it, then the BROKER_BACKEND = 'django' in my settings.py seems to be ignored and celery complains about being unable to connect to amqp://guest:**@127.0.0.1:5672/ (correct, because I don't have that) instead of using the Django backend.

[j7nn7k]

I don't use djcelery.setup_loader()

[j7nn7k]

Figured it out.
I made two mistakes.

1.) I didn't start celery with the app param (-A).

wrong:
celery worker

right:
celery worker -A proj # <- executed from manage.py dir

2.)
Didn't reference the correct django settings in celery.py. This is my config for development.

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings_dev')

Sorry guys and thanks for your help!

[ask]

That documentation is old and using setup_loader will disable the core celery integration. It will simply assume that you are using the old django-celery integration.

The canonical docs are here: http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
Don't mix and match the new and old tutorials, so if you insist on using the old django-celery integration,
you probably shouldn't have a celery.py either, and especially not one that sets the DJANGO_SETTINGS_MODULE environment.

If you use setup_loader then you have use manage.py celery and add djcelery to installed apps,
like in the old days.

[ask]

@j7nn7k Good to know it's working. the -A must be used so that it finds your Celery app instance, and when it imports the celery.py module it will also have the correct DJANGO_SETTINGS_MODULE set.
(-A proj is short for -A proj.celery:app which means the worker will do a from proj.celery import app)

[limpbrains]

I have similar error when I'm trying to use django.utils.translation.ugettext in my models.py
I'm on linux, Django 1.7 rc2, Celery v3.1.13

Everything accoding to http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
Trace:

(env)limpbrains@omg:~/dev/cc$ celery -A tst worker --loglevel=debug
Traceback (most recent call last):
  File "/home/limpbrains/dev/cc/env/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/base.py", line 307, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/celery.py", line 761, in handle_argv
    return self.execute(command, argv)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/celery.py", line 693, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/worker.py", line 179, in run_from_argv
    return self(*args, **options)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/base.py", line 270, in __call__
    ret = self.run(*args, **kwargs)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/bin/worker.py", line 212, in run
    state_db=self.node_format(state_db, hostname), **kwargs
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/worker/__init__.py", line 95, in __init__
    self.app.loader.init_worker()
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/loaders/base.py", line 128, in init_worker
    self.import_default_modules()
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/loaders/base.py", line 116, in import_default_modules
    signals.import_modules.send(sender=self.app)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/utils/dispatch/signal.py", line 166, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/amqp/utils.py", line 42, in __call__
    self.set_error_state(exc)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/amqp/utils.py", line 39, in __call__
    **dict(self.kwargs, **kwargs) if self.kwargs else kwargs
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/app/base.py", line 329, in _autodiscover_tasks
    self.loader.autodiscover_tasks(packages, related_name)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/loaders/base.py", line 251, in autodiscover_tasks
    related_name) if mod)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/loaders/base.py", line 272, in autodiscover_tasks
    return [find_related_module(pkg, related_name) for pkg in packages]
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/celery/loaders/base.py", line 291, in find_related_module
    return importlib.import_module('{0}.{1}'.format(package, related_name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/limpbrains/dev/cc/env/src/django-cc/cc/tasks.py", line 9, in <module>
    from cc.models import Wallet
  File "/home/limpbrains/dev/cc/env/src/django-cc/cc/models.py", line 18, in <module>
    class Wallet(models.Model):
  File "/home/limpbrains/dev/cc/env/src/django-cc/cc/models.py", line 19, in Wallet
    currency = models.CharField(_('Currency'), max_length=3, choices=currencies_coices())
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 83, in ugettext
    return _trans.ugettext(message)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 325, in ugettext
    return do_translate(message, 'ugettext')
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 306, in do_translate
    _default = translation(settings.LANGUAGE_CODE)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 209, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "/home/limpbrains/dev/cc/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 189, in _fetch
    "The translation infrastructure cannot be initialized before the "
django.core.exceptions.AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.

[hheimbuerger]

@ask I'm using setup_loader, I have djcelery in my installed apps, I don't have a celery.py and I'm using manage.py celery for all management commands. So as far as I can tell, I'm fully following the "old instructions" and not mix and matching.

[hheimbuerger]

FYI: the reason I had this issue only in the small test project and not in my main project is that there I'm using CELERYD_POOL = 'celery.concurrency.threads:TaskPool' for development.

[w-]

hi. This does not work for me
> cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"

django 1.7rc3
celery 3.1.13

I attempt to run
celery worker -A <project_name>

and I get

django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.

Is there something else i should be doing?

[webjunkie]

I have this error too, same as @limpbrains

The translation infrastructure cannot be initialized before the apps registry is ready

Did anyone find a solution?

[baoyiluo]

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application() can fix this problem! @webjunkie

[jijingg]

i add from django.contrib.auth.admin import User to models.py _(i'm sure the issue caused by this line)_
then python manage.py syncdb
django1.7 raised:

  File "/home/jijing/lab/django17/lib/python2.7/site-packages/django/db/models/options.py", line 577, in _fill_related_many_to_many_cache
    for klass in self.apps.get_models():
  File "/home/jijing/lab/django17/lib/python2.7/site-packages/django/utils/lru_cache.py", line 101, in wrapper
    result = user_function(*args, **kwds)
  File "/home/jijing/lab/django17/lib/python2.7/site-packages/django/apps/registry.py", line 168, in get_models
    self.check_models_ready()
  File "/home/jijing/lab/django17/lib/python2.7/site-packages/django/apps/registry.py", line 131, in check_models_ready
    raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.

[sreedharbukya]

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
upgraded django 1.7
This above hasn't fixed my below error for translation

The translation infrastructure cannot be initialized before the apps registry is ready
what other things I should change.

[koliyo]

I have the same problem, I updated to django 1.7 and now I get the AppRegistryNotReady exception. I'm using the old django-celery setup.

I am using celeryd daemon to launch the workers. Note that if I launch them manually using
python manage.py celery worker --loglevel=info
it does work. But when using the 'celery multi' command, which is used by celeryd, I get the AppRegistryNotReady exception.

[findsarfaraz]

I am also facing the same issue.
raise AppRegistryNotReady("Models aren't loaded yet.")

Celery documentation has many links which shows different ways to integrate celery with DJANGO. One says include celery.py in main project. Other has djcelery thing. Everyone thing just melt down to confusion.

I have documented my issue in the following link. Has anyone fixed it please reply.

https://groups.google.com/forum/#!topic/django-users/c_LOQ_nUvmw

[illagrenan]

I have same problem (AppRegistryNotReady: Models aren't loaded yet.) on Windows 8.1 and Django 1.7 using djcelery==3.1.16.

This worked for me:

$ (env) pip uninstall celery
$ (env) pip install celery

Btw I agree with @findsarfaraz – the documentation about integrating Celery with Django is confusing.

[mkuchen]

@ask I'm facing the same issue. I just upgraded Django from 1.6 to 1.7 (using Heroku/foreman for local development). Celery version is 3.1.17. I also have django-celery installed, version 3.1.16.

My wsgi.py file uses the proper application syntax:

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

...and my Procfile is as follows:

web: newrelic-admin run-program gunicorn -c gunicorn.py.ini wsgi:application
scheduler: newrelic-admin run-program celery beat --loglevel=INFO --app=api.celery:app
worker: newrelic-admin run-program celery worker -Q celery -E --loglevel=INFO --maxtasksperchild=200 --app=api.celery:app
periodic: newrelic-admin run-program celery worker -Q periodic -E --loglevel=INFO --maxtasksperchild=200 --app=api.celery:app

When I remove the scheduler, worker, and periodic lines, foreman start executes properly and runs without a hitch. But when I add the Celery commands back in, I get:

django.core.exceptions.AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.

However, when I run from my command line:

foreman run python manage.py celery worker -Q celery -E --loglevel=INFO --maxtasksperchild=200 --app=api.celery:app

It runs properly. Any thoughts here? It seems to have something to do with the way Celery is loading Django's translations module.

[mkuchen]

@limpbrains @sreedharbukya @ask I was able to fix the error by changing my celery.py file. It was originally:

# ...
app = Celery('api')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
# ...

I just switched those 2 lines and the issue was resolved, so:

# ...
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
app = Celery('api')
# ...

Hope this helps somebody!

[adambratt]

@mkuchen hahaha of all people to meet on a github comment :)

Well you just helped me!

[mkuchen]

@adambratt say whaaaat! What a coincidence. IIRC you were the one who introduced me to Celery in the first place. Talk about full-circle!

[barneyjackson]

Still experiencing this issue running Linux 14.04, Django 1.7, Celery 3.1.18. Is there a known fix/workaround/upgrade for this @ask? The change history in the docs implies 3.1.14/15 fixes it.

[nilliams]

Adding what worked for me in case it helps anyone:

None of the above solutions I've read in this and other threads worked for me. I'm running Django 1.8, Celery 3.1.18, django-celery 3.1.6 and starting workers via manage.py celery.

What appeared to magically fix the "Models aren't loaded yet." problem for me was disabling CELERYD_FORCE_EXECV as strongly recommended in the celery 3.1.7 release notes: http://docs.celeryproject.org/en/latest/changelog.html#version-3-1-17

(Even though the 2 things don't seem like they should be related).

[ask]

I don't think this is an actual issue anymore, more about some common pitfalls when integrating django/celery