Channels: Seeing HTTP/WS send decode error after upgrading to channels 1.1.3

[sachinrekhi]

I just upgraded Django channels to 1.1.3 as well as the following related libraries:

channels==1.1.3
daphne==1.2.0
asgi-redis==1.3.0
asgiref==1.1.1

Everything ran fine in my dev environment. But once I pushed to production, installed the new libraries, and completely restarted all services, I immediately started seeing the following error:

ERROR: HTTP/WS send decode error: Cannot dispatch message on channel u'daphne.response.dUTtOgoynk!bNiNpvqevU'

The client was showing the following error:

WebSocket connection to 'wss://<url>.com/' failed: Error during WebSocket handshake: Unexpected response code: 403

When I downgraded to the previous library versions, everything worked fine again.

It appears the issue is during the websocket connect / initial handshake?

I'm using the enforce_ordering decorator on all my websocket consumers as well as the asgi redis back-end.

Any thoughts on what's going on?

[koole]

I'm having the exact same issue after updating to Channels 1.1.2. I am not using enforce_ordering though.
Library versions:
```channels==1.1.2
daphne==1.2.0
asgi_redis==1.2.1
asgiref==1.1.1
asgi_ipc==1.3.1

[andrewgodwin]

The printed error is harmless; it will be moved down to a DEBUG statement in the next release of Daphne, but it can be safely ignored until then.

@sachinrekhi Are you accepting the WebSocket in the websocket.connect handler?
@koole Are you getting both the 403 error and logged console error, or just the latter?

[koole]

@andrewgodwin I'm getting both the 403 error in the client and the HTTP/WS send decode error. However, it doesn't always happen. Most of the time the websocket connection doesn't get a 403 error and just works fine.

Edit: I'm not sure if it helps, but only after these changes (https://github.com/HelloLily/hellolily/commit/6807c9a9cf8b4ec225ab2489501857c4f8232ecd) these errors started occurring .

[andrewgodwin]

@koole The capacity changes would result in 503 if they had triggered a change, so it might well be a second, actual, error, though I can't reproduce it on my test stack locally.

Is there anything unusual about the conditions or your connect handler?

[koole]

@andrewgodwin No, it's a simple method that uses the http user to authenticate the user and add it to a few channel groups.

I've looked at the changelogs, but it doesn't seem like anything that has an impact on the specific code in the connect handler has changed.

[andrewgodwin]

@koole Hm, right. Nothing unusual there, and the close that your method sends also sends a text frame, meaning you'd see a WebSocket close code, not 403, if that was triggered.

If you can get it reproducing semi-reliably, could you try changing the connect handler to each of these and see what happens?

• Always sends {'accept': True} and immediately returns
• Always sends {'accept': False} and immediately returns
• Always sends {'text': 'testing', 'close': True} and immediately returns

There's a couple of different states inside Daphne that can happen, and the results of something like that would make it easier to debug. The same experiment could be done by anyone who has this issue.

[sachinrekhi]

Yes, I'm accepting the websocket in the websocket.connect handler implicitly by always sending a reply to the client on its reply channel. My websocket.connect handler only does 2 things: add the connection to a Group and send the reply channel name to the client:

Group('user.%s' % user.id).add(self.reply_channel)
self.send_reply({'type':'CLIENT_ID', 'client_id':self.reply_channel.name})

send_reply() is a small helper that just json encodes the message:

self.reply_channel.send({'text': json.dumps(message)}, True)

I'm consistently seeing the websocket handshake error on the client on first connecting, so seems like a real issue despite the red herring on the ignorable error message. It does seem like it subsequently connects though.

[andrewgodwin]

Right, it seems like we have a real bug here in connections being denied hidden under the noise of that error (which is triggered because the socket is disconnected and vanishes, and Daphne has nowhere left to send the rest of the responses the connect handler chucks at it).

Could I ask you to do the process I mentioned in https://github.com/django/channels/issues/595#issuecomment-292533570 and see if you can get any sort of consistent result?

[koole]

Unfortunately I can only randomly reproduce the error (about 1/50 connection attempts) on production and not locally, so I'm going to try deploying to somewhere else where I can test these changes to the connect handler. It will take some time though.

[andrewgodwin]

Alright. Any clues will help in this case - I will try and reproduce it myself too with the benchmark script, but it might take a few days as I am jumping around conferences.

[sachinrekhi]

I unfortunately couldn't repro the issue in my dev environment, even when using daphne with similar level of running workers as my prod environment.

[koole]

I've tested the different connect handlers, and this is the result:

{'accept': True}
Mostly 101, but sometimes the 403 occurs

{'accept': False}
Always 403

{'close': True}
Always 403

{'text': 'testing', 'close': True}
Mostly 101, sends 'testing' and then closes, but sometimes 403 also occurs.

Because it seems to happen so infrequently, I wrote some JS to connect to the websockets every second, disconnect if it succeeds, and then try again after 1 second. Generally, it just seems to happen randomly, but I've often seen times where it's just a sequence of 403-101-403-101's going on for a long time, until then suddenly it just starts giving 101's properly again. However, there are also times when I make 500 connections, and they all just work perfectly fine.

Edit:
It looks like as soon as I open a new window with our website and it starts loading, 403 errors in the window running websocket connections start occuring more often. When the other window is done loading, the amount of 403 errors drops again.

[pypetey]

I got the same error but in a bit unusual conditions.

I use external app called django-countries for having a list of countries. The case is that when I enter a view that contains form with select field for displaying a list of countries then my app freezes and I get below error a lot of times.

I don't really know how to reproduce it on clean project

ERROR - server - HTTP/WS send decode error: Cannot dispatch message on channel 'daphne.response.QauYEkZnEP!XNQMyGfygK' (unknown)

[slothyrulez]

Hi,

Same error here.

Note, no using websockets (http.request) and trying to send first to a group and then to the reply_channel.

The code is here

[slothyrulez]

If you need anything from my side, i'll be glad to help.

[pypetey]

@slothyrulez I've discovered what's the cause of the problem. It's django-debug-toolbar and django 1.11. It's especially visible when you use django countries or have more complex forms / widgets.

[slothyrulez]

@pypetey No debug-toolbar, and no django-countries here, only Django 1.11
Also, just the development setup, with the memory channels is showing the error.

[belongwqz]

I got the same error:
ERROR HTTP/WS send decode error: Cannot dispatch message on channel 'daphne.response.BAvJfuTGZY!wbAKiFAzGH' (unknown)

And return:
503 Service Unavailable
It's randomly occurring, especially at a large query in the server.

[slothyrulez]

daphne master 'fixes' the issue:
https://github.com/django/daphne/commit/bd03fabce6d110016365a528dd6185ee00ce71a7

But now, I'm wondering, once a channel is added to a group, can only be reached via that group?

def http_susbcribe(message):
    ...
    Group('a-group').add(message.reply_channel)
    ...

later on a view (note, at this point, the reply_channel is also part of the group):

class AView(View):
    def post(self, request, *args, **kw):
        ...
        Group("a-group").send({ ... stuff })
        request.reply_channel.send({ ... stuff })
        ....

[proofit404]

No, you can use reply channel as usual. Group will do broadcast and nothing else.

[slothyrulez]

Is on this case when I'm seeing the HTTP/WS send decode error: Cannot dispatch message on channel

[proofit404]

Could you provide full traceback and consumer code sample without ... ?

[slothyrulez]

@proofit404
The code is there > https://gitlab.com/commite/channels_examples
On the readme are instructions to setup the project.

The subscribe button goes to https://gitlab.com/commite/channels_examples/blob/master/channels_examples/sse_app/consumers.py#L6
And the send button goes to https://gitlab.com/commite/channels_examples/blob/master/channels_examples/sse_app/views.py#L25

[approxit]

Same error. It's getting more interesting here...

asgi-redis==1.3.0
asgiref==1.1.1
channels==1.1.3
daphne==1.2.0
django==1.10.1
django-debug-toolbar==1.7
django-channels-panel==0.0.5

@pypetey The idea is totally unrelated but it actually worked for me. I just removed django-debug-toolbar for a second and... everything fixed itself. It will be hard to figure out what's going on...

Error mentioned in this issue popped out after I upgraded channels to recent version (list from above) to investigate constantly 503 error on my development and production environment with a little older versions. Older versions was just silent 503 results with probably identical cause. First I thought it was linked with heavy loads, but development environment is mostly idling. Even single http query was getting 503. Looking into redis-cli --stat I was receiving constant ~+800 requests. After random self fix its ~+40 on idling and ~+600 on preparing http response.

Most frustrating thing about this is... I can't reproduce it after this random self fix.

[belongwqz]

Yes, when I remove django-debug-toolbar(and django-debug-toolbar-request-history django-debug-toolbar-template-profiler django-channels-panel), the problem is solved.

[lukeaus]

I was getting this error as well.

Relevant packages

asgi-redis==1.2.1
asgiref==1.1.1
channels==1.1.2
Django==1.10.7
daphne==1.2.0
Twisted==17.1.0

I managed to stop getting this error by using a list where I previously had a set

e.g. from the django-channels multichat example

def ws_connect(message):
    message.reply_channel.send({'accept': True})
    message.channel_session['rooms'] = []  # use a list not a set

[boris-savic]

Hi,

same error - 403 Access Denied, happens every so often. About every 10th request I'd say. Really hard to say what's wrong.

Here is the Daphne log:

[09/May/2017:08:42:57] "WSCONNECTING /sockets/license/232/" - -
[09/May/2017:08:42:57] "WSREJECT /sockets/license/232/" - -

Installed versions:

Django==1.10.5
daphne==1.2.0
asgi-redis==1.3.0
asgiref==1.1.1
Twisted==17.1.0
channels==1.1.3

[ollamh]

Hi,

I'm getting the same behavior, using nginx as a reverse proxy behind AWS ELB.
I thought it might be the issue with Twisted lib, returning from 17.1.0 to 16.6.0 seemed to fix it, but today I'm having the same issue again.

When I start channels and daphne using python manage.py runserver, it works perfectly, but when I start daphne using daphne -b 0.0.0.0 -p 8001 <my app>:asgi.channel_layer and channel worker python manage.py runworker I see returning {'accept': True} from worker, but daphne closes connection almost immediately. Daphne log shows:

[INFO 2017-05-12 09:01:05,293 pika.adapters.base_connection 212 base_connection: MainProcess] Connecting to 52.30.139.4:5672
127.0.0.1:51497 - - [12/May/2017:09:01:09] "WSCONNECTING /ws/" - -
127.0.0.1:51497 - - [12/May/2017:09:01:09] "WSREJECT /ws/" - -

some debug logs from worker:

[INFO 2017-05-12 09:01:01,770 pika.adapters.base_connection 212 base_connection: MainProcess] Connecting to 52.30.139.4:5672
2017-05-12 09:01:01,862 - INFO - runworker - Using single-threaded worker.
2017-05-12 09:01:01,863 - INFO - runworker - Running worker against channel layer default (quokky.qk_websocket.asgi_chat_rabbitmq.ChatRabbitmqChannelLayer)
2017-05-12 09:01:01,863 - INFO - worker - Listening on channels websocket.connect, websocket.disconnect, websocket.receive
[INFO 2017-05-12 09:01:09,280 quokky.qk_websocket.consumers 47 consumers: MainProcess] connect user <skip> with headers:{u'reply_channel': u'daphne.response.rxVNKsxhrf!YnalrFowrq', u'server': ['127.0.0.1', 8001], u'headers': [['upgrade', 'websocket'], ['accept-language', 'it-IT'], ['x-forwarded-host', '<host>'], ['x-forwarded-for', '<ip addr>'], ['sec-websocket-version', '13'], ['host', '<host>'], ['sec-websocket-key', '+bi2jEF7vsemXprMsqoAIw=='], ['connection', 'upgrade'], ['x-forwarded-proto', 'http']], u'client': ['127.0.0.1', 51497], u'query_string': u'http_authorization=<auth string>', u'path': u'/ws/', u'order': 0}, and reply channel: daphne.response.rxVNKsxhrf!YnalrFowrq
[DEBUG 2017-05-12 09:01:11,172 quokky.qk_websocket.consumers 52 consumers: MainProcess] sending accept to daphne.response.rxVNKsxhrf!YnalrFowrq
[DEBUG 2017-05-12 09:01:11,172 quokky.qk_websocket.consumers 54 consumers: MainProcess] sent accept to daphne.response.rxVNKsxhrf!YnalrFowrq

I'm using versions:

Python 2.7.6
channels==1.1.3
daphne==1.2.0
asgi-rabbitmq==0.4.1 <-- we're using rabbitmq instead of redis
asgi-redis==1.3.0
asgiref==1.1.1
Django==1.8.13
Twisted==16.6.0

UPD: Trying to downgrade to channels==1.1.2 returned Twisted==17.1.0 and I started to get
ERROR: HTTP/WS send decode error: Cannot dispatch message on channel .. error. Returning to previous Twisted, solved that, leaving only WSREJECT errors.

UPD2: Tried to downgrade autobahn to version 0.17.2 doesn't fix the issue, but makes WSREJECT happen more rarely

[proofit404]

@ollamh To be sure that isn't my fault can you provide layer configuration? Maybe you can reproduce this behavior with test_project inside channels repository?

[ollamh]

@proofit404 I'm using my class inherited from your RabbitmqChannelLayer with couple of additional methods for adding/discarding single queues. Also, I have my custom logic for Protocol.on_dead_letter method (launching celery task for deliver dead messages in another way).
This is my layer configuration.

CHANNEL_LAYERS = {
    "default": {
        'BACKEND': 'my_proj.my_app.asgi_chat_rabbitmq.ChatRabbitmqChannelLayer',
        "CONFIG": {
            "url": os.environ.get('QK_RABBITMQ_URL', 'amqp://guest:guest@localhost:5672/%2F'), <-- in staging it is Cloudamqp url
            'expiry': 10
        },
        "ROUTING": "my_proj.routing.channel_routing",
    },
}

I will try to reproduce this behaviour with test project in the nearest future.

[ollamh]

@proofit404 I've tried testproject with rabbitmq support from channels repo and default behavior is fine. But, I noticed that in default consumer there is no websocket connection handler, so, I put a function in consumers.py:

def ws_connect(message):
    import time
    time.sleep(2) # <-- this emulates some checks that I make in order to permit connection (e.g. database query for checking token)
    message.reply_channel.send({'accept': True})

And I started to see in Chrome js console the responses:

WebSocket connection to 'ws://localhost/' failed: Error during WebSocket handshake: Unexpected response code: 403

JS code to run test:

socket = new WebSocket("ws://localhost/");
socket.onmessage = function(e) {
    console.log(e.data); x = e.data;
}
socket.onopen = function() {
    socket.send(JSON.stringify({'text': 'test'}));
}
// Call onopen directly if socket is already open
if (socket.readyState == WebSocket.OPEN) socket.onopen();

Important thing is that when I make sleep equal to 1 second, all is fine again.

[proofit404]

Maybe it happens due to some timeout handling logic? Try to tune --websocket_timeout and --websocket_connect_timeout. Or maybe some twisted endpoint string argument.

If I understand correctly, 2 seconds is way more time that consumers should execute in regular conditions.

[wsYdd]

Hi,
I get same error -Error during WebSocket handshake: Unexpected response code: 403
ERROR HTTP/WS send decode error: Cannot dispatch message on channel u'daphne.response.weiRGMDDTT!zVXWKMcXYL' (unknown)

anyone solved this problem?

[ollamh]

@proofit404 Sorry, for a long response delay, I was out of computer.

Yes, I agree that 2 seconds is too much for production environment, it was set just for testing purposes.

I tried to define --websocket_connect_timeout parameter and set it to any value greater than 0 (by default I see daphne server sets it to 20), after that while making lots of requests, I couldn't reproduce error in my staging environment.

Removing this parameter leads to unstable behavior (1 in 3-6 attempts resulted in 403 handshake response).

The timeout is always less than a second:

[INFO 2017-05-15 08:10:37,567 MainProcess] connect user with headers:{...}, reply channel: daphne.response.MqPfMIMzZU!SgeyKITqvH
[DEBUG 2017-05-15 08:10:37,694 MainProcess] sending accept to daphne.response.MqPfMIMzZU!SgeyKITqvH
[DEBUG 2017-05-15 08:10:37,694 MainProcess] sent accept to daphne.response.MqPfMIMzZU!SgeyKITqvH

The same situation I see with testproject.

[matteing]

I stopped seeing these errors after I properly flushed and discarded the users from their groups, and everything started working again.

group_name = "notifications-%s" % message.user.username
Group("notifications-all").discard(message.reply_channel)
Group(group_name).discard(message.reply_channel)

So make sure you're doing this.

[matteing]

Wait, never mind; this is starting to affect me, and 503 just pops up randomly and freezes the whole server for a full 2-3 mins. I'm on pretty good rig, so it's not choking the consumers. Any solutions?

[andrewgodwin]

@ftxrc Are you definitely sure the consumers are not at maximum capacity? CPU load is not a good indicator (as if they have non-CPU-bound code they may still be maxing out).

[ollamh]

I tried to use latest master of daphne and channels and it seems that all problems are gone, maybe due to
https://github.com/django/daphne/issues/114#issuecomment-301531565

[matteing]

@andrewgodwin How would I measure this? I'm using runserver locally.

[andrewgodwin]

@ftxrc Are you hitting the server a lot? runserver only runs a single handler thread, so if you have anything that runs slow it is going to choke it. And all your consumers exit immediately, right? No external API calls, delays, sleeps, anything?

[matteing]

@andrewgodwin I'm doing a few DB calls, yeah... I would have to profile then. What profiling tool do you recommend?

[andrewgodwin]

I'd just log the individual DB calls with some timing around them as a first measure. You should also upgrade Channels to 1.1.5, there's a performance improvement in there that was catching a few people up that you might also be hitting.

[jgirdner]

We were getting http lag. Looked in daphne logs and noticed this error correlating with the lag. We restarted Daphne and the errors stopped. Moved half our workers to HTTP only and created the error again by hitting our socket page and refreshing. Now the HTTP lag is gone. It seems like the sockets are not closing themselves.

[ryanflores79]

@andrewgodwin, yes, I work with @jgirdner and we were receiving a massive amount of these errors in the daphne log and it was causing http requests to take up to 15 seconds to respond. No browser tabs had been opened to our page with WebSockets in over 12 hours. It seems like the sockets should "self-close". Any input is much appreciated.

[jgirdner]

@ryanflores79 thank goodness we can make workers http only :-) Helped solve our user lag problem.

[andrewgodwin]

@jgirdner @ryanflores79 Did you try upgrading to the more recent releases of channels and daphne?

[ryanflores79]

@andrewgodwin I just upgraded:

from channels==1.1.3 to channels==1.1.5
and
from daphne==1.2.0 to daphne==1.3.0

And that seems to have alleviated the problem.

Thanks for helping us n00bs!

[andrewgodwin]

OK, I'm going to close this and consider it fixed then. Thanks for getting back to me!

[vladimirmyshkovski]

I had the same error when I did not want to use the receive, but tried to send to the connect.

def my_function(message):
    message.reply_channel.send({"accept": True})
    Group("ws").add(message.reply_channel)
    Group("ws").send({
            "text": 'hello',
        })

The message was coming, but I was getting an error. That is, we can say that it was harmless.

But I admit 100% that a mistake on my part. On this other, be careful.
@andrewgodwin
Thanks for Channels !

[ValeriyG]

Hi!
same error - 403 Access Denied, happens every so often.

Installed versions

Python 3
Django==1.11.6
daphne==1.3.0
asgi-redis==1.4.0
asgiref==1.1.2
Twisted==17.9.0
channels==1.1.8
django-channels==0.7.0

Layers config

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'asgi_redis.RedisChannelLayer',
        "CONFIG": {
            "hosts": ['redis://localhost:6379/2'],
        },
        'ROUTING': 'customer_chatting.channels_routing.routing',
    }
}

Daphne logs

127.0.0.1:55896 - - [09/Oct/2017:14:54:16] "WSCONNECTING /chat/customer/13/live" - -
127.0.0.1:55896 - - [09/Oct/2017:14:54:17] "WSREJECT /chat/customer/13/live" - -

nginx logs

95.164.65.208 - - [09/Oct/2017:14:53:51 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:14:54:05 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:14:54:07 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:14:54:13 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:14:54:17 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:04:45 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:04:51 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 4 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:04:58 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:04:59 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:06:57 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:06:59 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:02 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:08 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:12 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:16 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:45 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 6 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:10:48 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:13:29 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:13:42 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:15:46 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:15:54 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:16:00 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 4 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:16:05 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 4 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:16:08 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:18:06 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 101 30 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:25:52 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"
95.164.65.208 - - [09/Oct/2017:15:25:56 +0300] "GET /chat/customer/13/live?session_key=xd9gl615ci7c45tutfr0q1atbvihcwrf HTTP/1.1" 403 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"

Using nginx behind cloudflare

In development environment all fine.

[ibrahimsn98]

I was getting "403 access denied error" when I tried to connect from android app, I have solved with adding Origin header to connect request

[ngaurav]

I was getting "403 access denied error" when I tried to connect from android app, I have solved with adding Origin header to connect request

This worked for me too. I was using OkHTTP. The complete code is here
Request.Builder().url(SOCK_ENDPOINT) .addHeader("Authorization", "Token $token") .addHeader("Origin", SOCK_ENDPOINT).build()

[milos-u]

Having similar issues with old version of channels and daphne, the fix that works for me is to assure that:

six (1.12.0)

or later version is installed (for those still using Python 2).