Django-rest-framework: "AppRegistryNotReady: Apps aren't loaded yet" exception after 3.7.1 -> 3.7.2 update

[edmorley]

Steps to reproduce

1. Clone/checkout this PR (which updates django-rest-framework from 3.7.1 to 3.7.2): https://github.com/mozilla/treeherder/pull/2917
2. Push to a Travis-enabled GitHub repo

(Or to reproduce locally, clone/checkout, install Virtualbox/Vagrant then vagrant up && vagrant ssh then ./manage.py check inside the VM)

Expected behavior

Travis tests pass, like they do on 3.7.1 (which is on master):
https://travis-ci.org/mozilla/treeherder/builds/297971608

Actual behavior

The ./manage.py check step fails with:

Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/home/travis/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/travis/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute
    django.setup()
  File "/home/travis/venv/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/travis/venv/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/home/travis/venv/lib/python2.7/site-packages/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/opt/python/2.7.14/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/travis/venv/lib/python2.7/site-packages/hawkrest/__init__.py", line 17, in <module>
    from rest_framework.authentication import BaseAuthentication
  File "/home/travis/venv/lib/python2.7/site-packages/rest_framework/authentication.py", line 14, in <module>
    from rest_framework import HTTP_HEADER_ENCODING, exceptions
  File "/home/travis/venv/lib/python2.7/site-packages/rest_framework/exceptions.py", line 17, in <module>
    from rest_framework.utils.serializer_helpers import ReturnDict, ReturnList
  File "/home/travis/venv/lib/python2.7/site-packages/rest_framework/utils/serializer_helpers.py", line 8, in <module>
    from rest_framework.compat import unicode_to_repr
  File "/home/travis/venv/lib/python2.7/site-packages/rest_framework/compat.py", line 14, in <module>
    from django.contrib.auth import views
  File "/home/travis/venv/lib/python2.7/site-packages/django/contrib/auth/views.py", line 11, in <module>
    from django.contrib.auth.forms import (
  File "/home/travis/venv/lib/python2.7/site-packages/django/contrib/auth/forms.py", line 12, in <module>
    from django.contrib.auth.models import User
  File "/home/travis/venv/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/travis/venv/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
    class AbstractBaseUser(models.Model):
  File "/home/travis/venv/lib/python2.7/site-packages/django/db/models/base.py", line 110, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/home/travis/venv/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config
    self.check_apps_ready()
  File "/home/travis/venv/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

Plus the pytest step fails with the same exception and similar traceback too.

Full log:
https://travis-ci.org/mozilla/treeherder/jobs/297978305#L678

Versions:

• django-rest-framework 3.7.2
• Django 1.11.7
• Python 2.7.14

This appears to be a regression from #5510, CC @felixxm, @carltongibson

[xordoquy]

I don't know which option I'm for:

• move the view import from compat back to urls within DRF
• close the issue stating there shouldn't be DRF import in models / root files.

[carltongibson]

I think we have to handle this...

[xordoquy]

How high priority could this release be done ? I can make the release myself this afternoon.

[carltongibson]

@xordoquy Happy to release it today. As soon as we get a fix. (Tests...? 😱)

[xordoquy]

Let's got for it then.
I'm not sure we'd want test as it's a rather blur area.

[carltongibson]

Fine. I need 30mins or so before I can look at this. If there's a PR before then I'll merge and do a release immediately. 🙂

(Quickest thought was, could we just wrap the compat definitions in a callable to defer the imports?)

[xordoquy]

I'm currently committing the fix :)

[xordoquy]

Since we really don't need / want the views in the compat, let's just move them out of there. There might be project that just don't have django.contrib.auth at all (had one like that once).

[edmorley]

For this particular case, I'm presuming the problem is triggered by the from rest_framework.authentication import BaseAuthentication here:
https://github.com/kumar303/hawkrest/blob/1.0.0/hawkrest/__init__.py#L17

Is there something else that package should be doing instead, to avoid future similar cases like this? :-)

[xordoquy]

wooops, looks like I committed on master instead of my branch.
Let me know if you want me to revert & open a proper PR.

[rpkilby]

@xordoquy - you may need to anyway, the imports are broken in the current build.

[xordoquy]

Yeah, just noticed. Rolling back, sorry for the noise :(

[xordoquy]

more at #5566

[xeor]

Glad to see this being handled so quickly! :+1:
My build was failing as well. Looking forward for fix.

[xordoquy]

Should be fixed with #5567