Hello,
I currently have a problem deploying my Django application with django-channels.
I'm trying to setup daphne with nginx and for that, I made this ASGI file:
import os
from channels.asgi import get_channel_layer
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings_production")
channel_layer = get_channel_layer()
But somehow, it keeps using myapp.settings (and thus, fails at the database connection)
I use this command to launch daphne: daphne myapp.asgi:channel_layer -v 2
Python version: 2.7.11+
Django version: 1.9.7
django-channels version: 0.15.0
Am I doing something wrong?
Do you have two different ASGI files and you're referencing the development one instead of the production one?
No, I only have one ASGI file.
I also tried to delete the pyc files, just in case. And I even tried to launch daphne by directly specifying DJANGO_SETTINGS_MODULE in the command line. Tried export as well.
so the issue seems to be with import...
when importing like
from channels.asgi import get_channel_layer
variable should be set before the import like
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings_production")
from channels.asgi import get_channel_layer
Reason being get_channel_layer refers django settings, so get_channel_layer is already imported in the py file with a different DJANGO_SETTINGS_MODULE (default django one)
I would say best approach is making asgi file like
import channels
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings_production")
channel_layer = channels.asgi.get_channel_layer()
Interesting. Changing the suggested import order is probably the right approach.
It didn't change anything, unfortunately.
@DarthLabonne Could you post the exact asgi.py file you're using (with the import order change if you can) and the daphne command line you're using?
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "routeradmin.settings_production")
from channels.asgi import get_channel_layer
channel_layer = get_channel_layer()
and:
daphne routeradmin.asgi:channel_layer -b 0.0.0.0 -v 2 -p 8001
Using nginx to proxy the requests to daphne. Here are the relative settings:
server {
listen 80;
server_name webui.kyotolabs.lan;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8001;
}
location /static {
autoindex on;
alias /usr/share/router-central-webui/static/public/;
}
I don't think I need to set the environment from nginx but maybe I'm wrong?
@DarthLabonne Things seems to work fine for me, There can be issue with your environment, I don't see any other problem in your code.
can you also paste the dhapne version you are using,
Also, I would suggest you to log the
os.environ.get("DJANGO_SETTINGS_MODULE")
from your asgi file and check this variable doesn't already have a value in your env.
Also I would suggest if you can paste the log of the whole error that you get, when you try to run server using daphne. May be that might help us to check where the problem is.
I use daphne 0.13.0 (Downloaded using pip)
I tried printing the DJANGO_SETTINGS_MODULE and before setting it manually, I get None.
The error I get is OperationalError, FATAL: Peer authentication failed for user "darthlabonne"
Django's traceback clearly indicates that it uses the dev settings file and not my production settings.
@DarthLabonne Seems like there can be some other issue in your code, can you try adding logs to your production and dev logs/print statement files at the top of the files and see if both the files or any one file is referenced only....
Can you also confirm that your server is not even starting? Also it would be really helpful if you can add the whole error traceback, That might help looking into the problem..
I tried updating settings.py (dev settings) to see if the code is correctly updated.
Weird thing: I did the same with settings_production.py and when I launch the server, the print statement does work so settings_production IS loaded at least at first.
It means that, somehow, it's not (or at least not entirely) daphne's mistake.
Maybe I DO need to SetEnv with nginx?
The error appears when I make a request (the "not production" settings are loaded) which is weird.
Here's the full traceback:
Environment:
Request Method: GET
Request URL: http://webui.kyotolabs.lan/
Django Version: 1.9.7
Python Version: 2.7.11
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'tastypie',
'channels',
'router',
'acs']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback:
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
149. response = self.process_exception_by_middleware(e, request)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/channels/handler.py" in process_exception_by_middleware
228. return super(AsgiHandler, self).process_exception_by_middleware(exception, request)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
147. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
22. if test_func(request.user):
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in <lambda>
46. lambda u: u.is_authenticated(),
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/utils/functional.py" in inner
204. self._setup()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/utils/functional.py" in _setup
351. self._wrapped = self._setupfunc()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/middleware.py" in <lambda>
22. request.user = SimpleLazyObject(lambda: get_user(request))
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/middleware.py" in get_user
10. request._cached_user = auth.get_user(request)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/__init__.py" in get_user
167. user_id = _get_user_session_key(request)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/auth/__init__.py" in _get_user_session_key
59. return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in __getitem__
50. return self._session[key]
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in _get_session
201. self._session_cache = self.load()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py" in load
33. expire_date__gt=timezone.now()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/manager.py" in manager_method
122. return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/query.py" in get
381. num = len(clone)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/query.py" in __len__
240. self._fetch_all()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/query.py" in _fetch_all
1074. self._result_cache = list(self.iterator())
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/query.py" in __iter__
52. results = compiler.execute_sql()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
846. cursor = self.connection.cursor()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/base/base.py" in cursor
231. cursor = self.make_debug_cursor(self._cursor())
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/base/base.py" in _cursor
204. self.ensure_connection()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/base/base.py" in ensure_connection
199. self.connect()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/utils.py" in __exit__
95. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/base/base.py" in ensure_connection
199. self.connect()
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/base/base.py" in connect
171. self.connection = self.get_new_connection(conn_params)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py" in get_new_connection
175. connection = Database.connect(**conn_params)
File "/usr/share/router-central-webui/env/local/lib/python2.7/site-packages/psycopg2/__init__.py" in connect
164. conn = _connect(dsn, connection_factory=connection_factory, async=async)
Exception Type: OperationalError at /
Exception Value: FATAL: Peer authentication failed for user "christophe"
@DarthLabonne Your Database settings seems wrong. Try checking connection to psql from command line using the same parameters that you have in production settings. And lets take this offline doesn't seems to the issue with channels :)
Yes they are, those are my local settings. The production settings are not correctly loaded, somehow. The sole fact it shows the debug page proves it.
@DarthLabonne, It's picking the right settings file for me even if I import the get_channel_layer function before setting default value of 'DJANGO_SETTINGS_MODULE' environment variable. I have tested it with channels==0.15.0 and daphne==0.13.0 and also the latest versions, ie. channels==0.16.0 and daphne==0.14.2.
@DarthLabonne If possible can you share your production settings masking all the sensitive info.... or simply the structure of the setting (specifying the imports location, your database settings and channels settings) .... I didn't find any issues like this running channels....
# settings.py
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'asgi_redis.RedisChannelLayer',
'CONFIG': {
'hosts': [('localhost', 6379)],
},
'ROUTING': 'routeradmin.wsroutes.routing'
}
}
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'routeradmin',
'USER': 'christophe',
},
'acs': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'acs',
'USER': 'christophe',
}
}
# settings_production.py
from settings import *
DEBUG = False
ALLOWED_HOSTS = [
'webui.superhost.com'
]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'routeradmin',
'USER': 'routeradmin',
'PASSWORD': 'awesomelysecurepassword',
'HOST': '127.0.0.1',
},
'acs': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'acs',
'HOSTNAME': 'acs.superhost.net',
}
}
And somehow, settings_production.py is loaded but never used.
Django clearly tells me it is using settings.py
By using a simple python manage.py runserver --settings=routeradmin.settings_production, the settings are correctly loaded (although it doesn't work great, as one would expect)
@DarthLabonne Ok one more question, You might be running your workers... daphne just provides the interface actual requests are processed by the workers... how are you staring your workers?... you might have to provide --settings=routeradmin.settings_production while starting the workers.... That might be the problem with your case....
python manage.py runworker --settings=routeradmin.settings_production
You can also update the manage.py file and add
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "routeradmin.settings_production") to avoid adding --settings args.
Hope this fixes your problem.
That actually fixed the problem!
I was just running python manage.py runworker.
Thank you!
1 year later, it seems like
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings_production")
actually does not change the value of DJANGO_SETTINGS_MODULE
os.environ['DJANGO_SETTINGS_MODULE'] = "myapp.settings_production"
seems to work better
How you run channels?
I'm using systemd to start daphne and related workers
Please provide used units, expected behavior and actual behavior :)
I think the docs are misleading https://channels.readthedocs.io/en/stable/deploying.html#run-interface-servers
since using your example asgi.py file we might run into an error, which can be explained by the fact using os.environ.setdefault("DJANGO_SETTINGS_MODULE", "back.settings") won't replace the actual environment variable DJANGO_SETTINGS_MODULE if already existing (at least when used in a systemd daemon) as you can see in the first lines of the system log
starting daphne via systemd:
[Unit]
Description=Daphne service
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
After=syslog.target
[Service]
ExecStart=####/bin/daphne -u /tmp/daphne.sock back.asgi.asgi:channel_layer
WorkingDirectory=####
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all
[Install]
WantedBy=multi-user.target
asgi.py
import os
import sys
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "back.settings")
sys.stdout.write(os.environ.get('DJANGO_SETTINGS_MODULE') + '\n')
os.environ["DJANGO_SETTINGS_MODULE"] = "back.settings"
sys.stdout.write(os.environ.get('DJANGO_SETTINGS_MODULE') + '\n')
raise Exception
import channels
channel_layer = channels.asgi.get_channel_layer()
systemctl status -n 100 daphne.service output after sudo service daphne restart
Jul 26 09:10:47 #### systemd[1]: Starting Daphne service...
Jul 26 09:10:47 #### daphne[26868]: settings
Jul 26 09:10:47 #### daphne[26868]: back.settings
( ...
Jul 26 09:10:47 #### daphne[26868]: Traceback (most recent call last):
Jul 26 09:10:47 #### daphne[26868]: File "####/bin/daphne", line 11, in <module>
Jul 26 09:10:47 #### daphne[26868]: sys.exit(CommandLineInterface.entrypoint())
Jul 26 09:10:47 #### daphne[26868]: File "####/lib/python3.5/site-packages/daphne/cli.py", line 144, in entrypoint
Jul 26 09:10:47 #### daphne[26868]: cls().run(sys.argv[1:])
Jul 26 09:10:47 #### daphne[26868]: File "####/lib/python3.5/site-packages/daphne/cli.py", line 174, in run
Jul 26 09:10:47 #### daphne[26868]: channel_layer = importlib.import_module(module_path)
Jul 26 09:10:47 #### daphne[26868]: File "####/lib/python3.5/importlib/__init__.py", line 126, in import_module
Jul 26 09:10:47 #### daphne[26868]: return _bootstrap._gcd_import(name[level:], package, level)
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap>", line 986, in _gcd_import
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap>", line 969, in _find_and_load
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap_external>", line 665, in exec_module
Jul 26 09:10:47 #### daphne[26868]: File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
Jul 26 09:10:47 #### daphne[26868]: File "./back/asgi/asgi.py", line 8, in <module>
Jul 26 09:10:47 #### daphne[26868]: raise Exception
... )
Jul 26 09:10:47 #### daphne[26868]: Exception
If you set DJANGO_SETTINGS_MODULE in the environment then it will correctly use the one you provided. the asgi.py one is just a default (please compare with django's manage.py https://github.com/django/django/blob/master/django/conf/project_template/manage.py-tpl)
Indeed, I just found the example misleading since it gives the impression you could change our settings file using os.environ.setdefault("DJANGO_SETTINGS_MODULE", XXX) and it seems @DarthLabonne fell for the same trap
I have the same problem and tried all of them above. my issue is when I have daphne, worker and django on ElasticBeanstalk, the worker is trying to use localhost Database even though I have specified the correct one in settings. So I am thinking worker is not loading the settings correctly.
Any thoughts?
here is the error:
2017-08-11 08:26:46,004 - ERROR - worker - Error processing message with consumer account.consumers.ws_disconnect:
Traceback (most recent call last):
File "/opt/python/run/venv/local/lib/python2.7/site-packages/channels/worker.py", line 119, in run
consumer(message, **kwargs)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/channels/sessions.py", line 68, in inner
session = session_for_reply_channel(message.reply_channel.name)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/channels/sessions.py", line 29, in session_for_reply_channel
instance._session.keys()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py", line 207, in _get_session
self._session_cache = self.load()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py", line 35, in load
expire_date__gt=timezone.now()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 374, in get
num = len(clone)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 232, in __len__
self._fetch_all()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1118, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 882, in execute_sql
cursor = self.connection.cursor()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
return self._cursor()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
self.ensure_connection()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect
self.connection = self.get_new_connection(conn_params)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
connection = Database.connect(**conn_params)
File "/opt/python/run/venv/local/lib64/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
my asgi.py:
import os
from channels.asgi import get_channel_layer
os.environ["DJANGO_SETTINGS_MODULE"] = 'hottooth.settings'
channel_layer = get_channel_layer()
my 'manage.py' if it helps:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ['DJANGO_SETTINGS_MODULE'] = 'hottooth.settings'
try:
from django.core.management import execute_from_command_line
except ImportError:
# The above import may fail for some other reason. Ensure that the
# issue is really that Django is missing to avoid masking other
# exceptions on Python 2.
try:
import django # noqa
except ImportError:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
)
raise
execute_from_command_line(sys.argv)
I am using supervisor to run daphne and workers, here are the commands:
Worker command: /opt/python/run/venv/bin/python manage.py runworker --settings hottooth.settings
related supervisor file:
[program:Daphne]
environment=PATH="/opt/python/run/venv/bin"
command=/opt/python/run/venv/bin/daphne -b 0.0.0.0 -p 5000 hottooth.asgi:channel_layer
directory=/opt/python/current/app
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/tmp/daphne.out.log
stderr_logfile=/tmp/daphne.err.log
[program:Worker]
environment=PATH="/opt/python/run/venv/bin"
command=/opt/python/run/venv/bin/python manage.py runworker --settings hottooth.settings
directory=/opt/python/current/app
process_name=%(program_name)s_%(process_num)02d
numprocs=4
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/tmp/workers.out.log
stderr_logfile=/tmp/workers.err.log
part of settings.py, database works fine since I can create new object using django. I notice the error is using different postgres backend than my settings, I tried to change to the error one, doesn't work.
if 'RDS_DB_NAME' in os.environ:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
}
else:
DATABASES = {
'default': dj_database_url.config(
default='postgres://localhost/hottooth',
conn_max_age=500
)
}
Try to add RDS_DB_NAME to the supervisor environment.
@wx2228 With beanstalk if you need to access the environment variable that you have set in the beanstalk console. You need to include the file located at "/opt/python/current/env"... this file contains all the environment values set in Beanstalk console.
Including the environment variable depends on how you are using your beanstalk config to start supervisor. You can use something like https://gist.github.com/StefanieD/5407df6b086b38c4f887#file-supervisor-config-L25
Hope this helps!
os.environ['DJANGO_SETTINGS_MODULE'] = "myapp.settings_production"
worked for me.
Don't create a directory called channels inside your project. It messes with the import statement import channels.asgi. It imports your project's channels directory. Whoops.
Why define channel_layer = get_channel_layer() in asgi.py, it should be defined in setttings.py
I had this problem because of celery:
# project/__init__.py
# This ensures that the app is loaded when Django starts
# so that the @shared_task decorator will use it.
from config.celery import app as celery_app
This import was causing an early import of django settings. I had to move this line to settings.py.
Most helpful comment
1 year later, it seems like
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings_production")actually does not change the value of DJANGO_SETTINGS_MODULE
os.environ['DJANGO_SETTINGS_MODULE'] = "myapp.settings_production"seems to work better