Channels: [channels2] issues with raven

[brianmay]

I noticed that the raven library doesn't play nicely with channels2:

$ ./manage.py runserver
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x1100558c8>
Traceback (most recent call last):
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 228, in wrapper
    fn(*args, **kwargs)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    autoreload.raise_last_exception()
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 251, in raise_last_exception
    six.reraise(*_exception)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 228, in wrapper
    fn(*args, **kwargs)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/apps/registry.py", line 116, in populate
    app_config.ready()
  File "/Users/brianmay/tree/myro/channels/channels/apps.py", line 17, in ready
    monkeypatch_django()
  File "/Users/brianmay/tree/myro/channels/channels/hacks.py", line 10, in monkeypatch_django
    from .management.commands.runserver import Command as RunserverCommand
  File "/Users/brianmay/tree/myro/channels/channels/management/commands/runserver.py", line 4, in <module>
    from daphne.server import Server
  File "/Users/brianmay/tree/myro/daphne/daphne/server.py", line 5, in <module>
    asyncioreactor.install()
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/twisted/internet/asyncioreactor.py", line 322, in install
    installReactor(reactor)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/twisted/internet/main.py", line 32, in installReactor
    raise error.ReactorAlreadyInstalledError("reactor already installed")
twisted.internet.error.ReactorAlreadyInstalledError: reactor already installed

Just to be sure, I put an error condition in twisted/internet/reactor.py to see what stack trace I would get when it is first loaded:

$ ./manage.py runserver
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x1069a7bf8>
Traceback (most recent call last):
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 228, in wrapper
    fn(*args, **kwargs)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    autoreload.raise_last_exception()
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 251, in raise_last_exception
    six.reraise(*_exception)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/utils/autoreload.py", line 228, in wrapper
    fn(*args, **kwargs)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/__init__.py", line 54, in <module>
    from raven.base import *  # NOQA
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/base.py", line 37, in <module>
    from raven.conf.remote import RemoteConfig
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/conf/remote.py", line 36, in <module>
    DEFAULT_TRANSPORT = discover_default_transport()
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/conf/remote.py", line 18, in discover_default_transport
    from raven.transport.threaded import ThreadedHTTPTransport
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/transport/__init__.py", line 18, in <module>
    from raven.transport.registry import *  # NOQA
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/transport/registry.py", line 18, in <module>
    from raven.transport.twisted import TwistedHTTPTransport
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/raven/transport/twisted.py", line 16, in <module>
    from twisted.web.client import (
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/twisted/web/client.py", line 39, in <module>
    from twisted.internet import defer, protocol, task, reactor
  File "/Users/brianmay/.pyenv/versions/myrocc2/lib/python3.6/site-packages/twisted/internet/reactor.py", line 35, in <module>
    raise Meow()
NameError: name 'Meow' is not defined

To me, it looks like we have two libraries that both use twisted (not sure I understand why) and as a result clash.

I think raven uses twisted for async http sending stuff, although looking at the stack trace would suggest I would get the same error even if I hacked it to use HTTPTransport instead of TwistedHTTPTransport.

I have no idea what, if any, solution there is to this, or who to blame. However, I think this is going to be an important issue.

[andrewgodwin]

Twisted reactor installation is weird and needs to be done very early on - it's possible that if we make sure the reactor is installed as the very first thing this problem will go away. Could you try out having these lines in daphne.cli at the very top and see if it fixes it?

from twisted.internet import asyncioreactor
asyncioreactor.install()

[brianmay]

Unfortunately, doesn't appear to help :-(

[andrewgodwin]

Hm. I'll have to test this myself then to see how it can be fixed. It's weird that Raven uses Twisted reactors given it runs as a library...

[brianmay]

Yes, very weird. I suspect it might be related to the ThreadedHTTPTransport stuff, but I cannot begin to imagine how this might actually work (I kind of dread looking...).

Not that I use "./manage.py runserver" which might be affecting the results.

I just tried "daphne myrocc.base.asgi:application" (with raven removed and dapne unaltered), which curiously gives me the dreaded "django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet." - which I suspect is a completely different and unrelated problem (probably more related to the other ticket I filled).

[brianmay]

Ok, the two requirements for the crash seem to be:

• 'raven.contrib.django.raven_compat' in INSTALLED_APPS. No other configuration required.
• Use of ./manage.py runserver. Running daphne directly seems to work.

This is with no changes to daphne.

[andrewgodwin]

Ah, that makes sense. The bug will only happen when raven is imported before daphne. I think I can figure out a fix.

[andrewgodwin]

Fixed with 3d8d82be3528cc0150dac0c8ade1f6c306b412e4, provided you put Channels above Raven in the INSTALLED_APPS list, which is the best I can do without modifying Raven to lazy-load its twisted HTTP transport.

[brianmay]

Confirmed, that does seem to fix the problem. Thanks.

[babus]

Unfortunately that doesn't fix the problem for me.

Running Python3.6,

INSTALLED_APPS = [
    # Django channels
    'channels',

    # Auto expire session
    'session_security',

    # Elegant admin using grappelli and nested admin
    'grappelli.dashboard',
    'grappelli',
    'nested_admin',

    # Django contrib apps
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sites',
    'django.contrib.contenttypes',

    # For different storages
    'storages',

    # Rest framework and rest auth apps
    'rest_framework',
    'rest_framework.authtoken',
    'rest_auth',

    # Thumbnail app
    'sorl.thumbnail',

    # Django allauth apps for social authentication and user registration
    'allauth',
    'allauth.account',
    'rest_auth.registration',
    'allauth.socialaccount',
    'allauth.socialaccount.providers.facebook',
    'allauth.socialaccount.providers.google',
    # Custom user app to use email as the user identifier instead of username.
    'custom_user',
    # Form app for django admin
    'crispy_forms',
    # App for providing embed field for videos
    'embed_video',
    # Cache app which patches the ORM
    'cacheops',
    # Email backend which uses Celery
    'djcelery_email',
    # Enables admin interface for Celery.
    'django_celery_beat',
    # Subdomain hosting support.
    'django_hosts',
    # For creating robots.txt
    'robots',
    # For tagging
    'taggit',
    # For import export in django admin
    'import_export',
    # For CORS requests
    'corsheaders',
    # For ckeditor
    'ckeditor',

    # Django redis board
    # 'redisboard',

    # Comments
    'django_comments_xtd',
    'django_comments',

    # Sentry client
    'raven.contrib.django.raven_compat',
    # Opbeat client
    'opbeat.contrib.django',

    # Our apps
    'our_app',
]

Requirements.txt

asgi-redis[cryptography]==1.4.3
boto==2.48.0
bs4==0.0.1
channels==2.0.0
celery==4.1.0
coreapi==2.3.0
cssselect==1.0.1
daphne==2.0.0
django-allauth==0.35.0
django-cacheops==4.0.5
django-celery-email==2.0.0
django-celery-beat==1.1.0
django-ckeditor==5.4.0
django-comments-xtd==2.0.10
django-cors-headers==2.1.0
django-crispy-forms==1.7.0
django-custom-user==0.7
django-db-geventpool==2
django-embed-video==1.1.2
django-grappelli==2.11.1
django-hosts==3.0
django-import-export==0.7.0
django-nested-admin==3.0.12
django-redis==4.8.0
django-rest-auth==0.9.2
django-rest-swagger==2.1.1
django-session-security==2.5.1
django-storages==1.6.5
django-taggit==0.22.2
Django==2.0.2
djangorestframework==3.7.7
envdir==0.7
gevent==1.2.2
feedparser==5.2.1
hiredis==0.2.0
lxml
opbeat==3.6.1
Pillow==5.0.0
pip
psycopg2==2.7.3.2
pycountry==17.9.23
pyfcm==1.4.3
python-openid==2.2.5
python-dateutil==2.6.1
python-Levenshtein==0.12.0
python-rake==1.3.0
pytz==2017.3
raven==6.5.0
requests==2.18.4
roman
semantic-version==2.6.0
service-identity==16.0.0
six==1.11.0
slacker-log-handler==1.6.1
sorl-thumbnail==12.3
Unidecode==0.4.19
urllib3==1.22
uWSGI==2.0.15

# For django 2.0 compatibility
-e git+https://github.com/jazzband/[email protected]#egg=robots

manage.py

import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "our_app.settings")

    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

python manage.py check errors out

Traceback (most recent call last):
  File "manage.py", line 9, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 347, in execute
    django.setup()
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/apps/registry.py", line 89, in populate
    app_config = AppConfig.create(entry)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/apps/config.py", line 116, in create
    mod = import_module(mod_path)
  File "/home/vagrant/env3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/channels/apps.py", line 6, in <module>
    import daphne.server  # noqa
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/daphne/server.py", line 3, in <module>
    asyncioreactor.install()  # isort:skip
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/twisted/internet/asyncioreactor.py", line 322, in install
    installReactor(reactor)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/twisted/internet/main.py", line 32, in installReactor
    raise error.ReactorAlreadyInstalledError("reactor already installed")
twisted.internet.error.ReactorAlreadyInstalledError: reactor already installed

[andrewgodwin]

I've pushed an experimental fix in django/daphne@7949b24 - could you pull that and see if the problem is solved?

[babus]

Thank you for working on it and the quick patch! But there is another error now,

Traceback (most recent call last):
  File "manage.py", line 9, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 347, in execute
    django.setup()
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/apps/registry.py", line 89, in populate
    app_config = AppConfig.create(entry)
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/django/apps/config.py", line 116, in create
    mod = import_module(mod_path)
  File "/home/vagrant/env3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/channels/apps.py", line 6, in <module>
    import daphne.server  # noqa
  File "/home/vagrant/env3.6/lib/python3.6/site-packages/daphne/server.py", line 9, in <module>
    "finding the package that imports Twisted and importing it later on."
TypeError: 'str' object cannot be interpreted as an integer

[andrewgodwin]

Oh, whoops, I screwed up the string formatting after I tested it. Try now.

[babus]

Perfect! It works now. Thank you!

Insignificant but the formatting in warning has to be fixed.

/home/vagrant/env3.6/lib/python3.6/site-packages/daphne/server.py:9: UserWarning: Something has already installed a Twisted reactor. Attempting to uninstall it; you can fix this warning by importing daphne.server early in your codebase or finding the package that imports Twisted and importing it later on.
  "finding the package that imports Twisted and importing it later on."

[andrewgodwin]

That's not bad formatting, that's the fact that Python is showing you the last line of source code, which happens to also be in the string. I'll see if I can fool it into having something else there.

[babus]

Ah I see, Thanks @andrewgodwin .

[ademidun]

@andrewgodwin Thanks for the fix, it works well on my local host but when I push to Heroku its giving me problems:
My guess is that Heroku is not using the most recent channels release but I'm not sure.

My requirements.txt:
asgi-redis==1.4.3
asgiref==1.1.2
async-timeout==2.0.0
attrs==17.4.0
autobahn==17.10.1
Automat==0.6.0
beautifulsoup4==4.6.0
boto==2.48.0
certifi==2017.4.17
channels==2.0.0
chardet==3.0.3
click==6.7
constantly==15.1.0
coreapi==2.3.1
coreapi-cli==1.0.6
coreschema==0.0.4
daphne==2.0.1
dj-database-url==0.4.2
Django==1.11.2
django-cors-headers==2.1.0
django-extensions==1.9.8
django-multiselectfield==0.1.7
django-storages==1.6.5
djangorestframework==3.6.3
djangorestframework-jwt==1.11.0
fdfgen==0.16.0
gcloud==0.17.0
googleapis-common-protos==1.5.2
gunicorn==19.7.1
httpie==0.9.9
httplib2==0.10.3
hyperlink==17.3.1
idna==2.5
incremental==17.5.0
itypes==1.1.0
Jinja2==2.9.6
jws==0.1.3
MarkupSafe==1.0
msgpack-python==0.5.2
numpy==1.13.1
oauth2client==3.0.0
pbr==3.0.1
protobuf==3.4.0
psycopg2==2.7.1
pyasn1==0.3.6
pyasn1-modules==0.1.4
pycryptodome==3.4.3
Pygments==2.2.0
PyJWT==1.5.2
PyPDF2==1.26.0
pypdftk==0.3
Pyrebase==3.0.27
python-http-client==3.0.0
python-jwt==2.0.1
pytz==2017.2
raven==6.3.0
redis==2.10.6
requests==2.11.1
requests-toolbelt==0.7.0
rsa==3.4.2
selenium==3.6.0
sendgrid==5.3.0
six==1.10.0
stevedore==1.23.0
Twisted==17.9.0
txaio==2.8.2
typing==3.6.2
uritemplate==3.0.0
urllib3==1.21.1
virtualenv==15.1.0
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
whitenoise==3.3.1
zope.interface==4.4.3

Running python manage.py makemigrations on ⬢ xkcd .. up, run.4771 (Standard-1X)
Traceback (most recent call last):
File "manage.py", line 39, in
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 337, in execute
django.setup()
File "/app/.heroku/python/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/config.py", line 120, in create
mod = import_module(mod_path)
File "/app/.heroku/python/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 978, in _gcd_import
File "", line 961, in _find_and_load
File "", line 950, in _find_and_load_unlocked
File "", line 655, in _load_unlocked
File "", line 678, in exec_module
File "", line 205, in _call_with_frames_removed
File "/app/.heroku/python/lib/python3.6/site-packages/channels/apps.py", line 6, in
import daphne.server # noqa
File "/app/.heroku/python/lib/python3.6/site-packages/daphne/server.py", line 3, in
asyncioreactor.install() # isort:skip
File "/app/.heroku/python/lib/python3.6/site-packages/twisted/internet/asyncioreactor.py", line 322, in install
installReactor(reactor)
File "/app/.heroku/python/lib/python3.6/site-packages/twisted/internet/main.py", line 32, in installReactor
raise error.ReactorAlreadyInstalledError("reactor already installed")
twisted.internet.error.ReactorAlreadyInstalledError: reactor already installed

[ademidun]

Update: It might be an issue with my Procfile, there's some documentation on how to use Django Channels with Heroku but idk if its still valid, I think its pre 2.0.

https://blog.heroku.com/in_deep_with_django_channels_the_future_of_real_time_apps_in_django#1-procfile-and-process-types

Heroku's Sample procfile:
web: daphne chat.asgi:channel_layer --port $PORT --bind 0.0.0.0 -v2 worker: python manage.py runworker -v2

My current Procfile:
web: gunicorn server.wsgi --log-file=-

What should my Procfile look like if I want to run Django Channels on Heroku

[andrewgodwin]

Yeah, that blog post is for Channels 1. You'll want to look at our Deploying docs for the new info (http://channels.readthedocs.io/en/latest/deploying.html), but in short, you'd want:

web: daphne -p $PORT -b 0.0.0.0 -myproject.asgi:application

[SHxKM]

When running ./manage.py runserver with Channels 2, I'm seeing this warning twice:

UserWarning: Something has already installed a non-asyncio Twisted reactor. Attempting to uninstall it; you can fix this warning by importing daphne.server early in your codebase or finding the package that imports Twisted and importing it later on.

Even when "raven.contrib.django.raven_compat" is removed from INSTALLED_APPS or appears after/before channels, this warning shows up.

When running:

daphne my_app.asgi:application -p 8000 --bind 0.0.0.0

The warning doesn't show up.

@brianmay, @andrewgodwin: Any idea if I can proceed normally and just live with the annoying warning in my dev environment? can I assume my production environment will be OK?

[maxmalysh]

@SHxKM I have the same problem.

What's your experience? Is it safe to ignore the runserver warning?

Did you try switching to sentry-sdk?

[hyliker]

@maxmalysh The simple solution is just to import the daphe.server before the raven in the settings.py like this

import daphne.server # Fixed UserWarning from raven
import raven

[tusharsrivastava]

Great @hyliker you just helped a lot of people here. It really helped. Thanks