Ws: Websockets not working with HTTPS in an Apache Proxy (302 error?)

[DigitalLeaves]

Hello, first of all, thanks for your awesome library. I was able to setup and get WS running in a breeze. Thanks.

I have a problem with ws, I'm unable to make websockets work on an apache proxy through HTTPS. Websockets are working properly if no (apache) http(s) proxy is used.

My setup: I have an apache server with multiple virtual hosts. I have a HTTPS webpage for myserver.com and the HTTPS API with node/express/ws in api.myserver.com subdomain through the proxy, that redirects the requests to the node.js instance (multiple instances on PM2) running on port 3333.

This is my apache virtual host for the subdomain:

<VirtualHost *:443>
    ServerName api.myserver.com
    ServerAdmin [email protected]
    DocumentRoot /var/www/html/myserver/api
        Options -Indexes

    SSLEngine on                                                                
    SSLProtocol all -SSLv2                                                      
    SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM                

    SSLCertificateFile /etc/apache2/certs/STAR_myserver_co.crt
    SSLCertificateKeyFile /etc/apache2/certs/myserver_private_key.pem
    SSLCertificateChainFile /etc/apache2/certs/STAR_myserver_co.ca-bundle

    SSLProxyEngine On
    ProxyPreserveHost On
    ProxyRequests Off

    # This is for websocket requests
    ProxyPass /wss/ wss://localhost:3333/
    ProxyPassReverse /wss/ wss://localhost:3333/

        # This is for normal requests    
        ProxyPass / https://localhost:3333/
        ProxyPassReverse / https://localhost:3333/
</VirtualHost>

This works OK for redirecting the connections to the node express backend. I have installed mod_proxy, mod_proxy_http and mod_proxy_wstunnel.

This is the node.js API backend: first, I initialize express, sessions, etc.

// express, session and mongodb session storage
var express = require('express')
var session = require('express-session')
var MongoStore = require('connect-mongo')(session)
var app = express()
// configure sessionStore and sessions, mongodb, etc...

// Certificates and credentials for HTTPS server
var fs = require('fs')
var privateKey  = fs.readFileSync(__dirname + '/certs/myserver_private_key.pem', 'utf8')
var certificate = fs.readFileSync(__dirname + '/certs/myserver_cert.pem', 'utf8')
var ca = fs.readFileSync(__dirname + '/certs/myserver_ca.pem', 'utf8')
var credentials = {key: privateKey, cert: certificate, ca: ca}

app.enable('trust proxy')
app.set("trust proxy", 1)

And then I setup the HTTPS server securely, using the same certificates that in APACHE:

// Setup HTTPS server
var https = require('https')
var server = https.createServer(credentials, app)
server.listen(appPort, 'localhost', function () {
    // Server up and running!
    var host = server.address().address
    var port = server.address().port
    console.log('myserver listening at https://%s:%s', host, port)
})

Last, I setup the websocket connections:

// setup Websockets
wss = new WebSocketServer({ server: server })
wss.on('connection', function connection(ws) {
    var cookies = cookie.parse(ws.upgradeReq.headers.cookie)
    var sid = cookieParser.signedCookie(cookies["connect.sid"], myserver_secret)
    // extract user credentials and data from cookie/sid,

    // get the session object
    sessionStore.get(sid, function (err, ss) {
        ...
    })
})

Then my clients just try to connect to websockets securely (because, being a HTTPS app, I cannot use the ws:// insecure websockets connection):

window.WebSocket = window.WebSocket || window.MozWebSocket
webSocket = new WebSocket('wss://' + location.host + '/wss')

And then I get always the same error 302:

[Error] WebSocket connection to 'wss://api.myserver.com/wss' failed: Unexpected response code: 302

If I test on a local server directly to the node instance https://localhost:3333/ it's working perfectly and websockets work as they should.

Any idea of how to solve this? Is there a problem with ws redirections made by Apache proxy modules?

[lpinca]

Is there a problem with ws redirections made by Apache proxy modules?

Yeah this is most likely the issue, but the question is: why the proxy answer with a 302?

[lpinca]

Can you try to remove this directive?

ProxyPassReverse /wss/ wss://localhost:3333/

[DigitalLeaves]

Thanks @lpinca, tried it, but I got the same results.

[DigitalLeaves]

Is there something else I can do? I won't like to switch to a pulling mechanism in my app, but I need to get websockets to work in a HTTPS scenario. If I setup ws as a standalone service (not tied to express) I'm afraid I won't be able to bind the application user+session with the websocket connection.

[lpinca]

What is the version of Apache?
The first thing to find out is if it is expected for the mod_proxy_wstunnel to answer with a 302 status code.

[DigitalLeaves]

Server version: Apache/2.4.10 (Debian)

Well, to be honest, I didn't know of that module until websockets refused to work on HTTPS. I was using a simple mod_proxy with the ProxyPass and ProxyPassReverse until I had to make ws work through the proxy.

However, I disabled it and it made no difference. The problem here is that websockets are not really playing nice the with apache proxy redirection... Or Apache is unable to properly handle those redirections.

I am willing to try any other solution or way of having websockets working through HTTPS in my subdomain api.myserver.com. I guess this is a very common scenario, as node instances usually go through PM2 or similar to keep multiple alive instances, and nginx/apache is used as the webserver, thus needing some kind of proxy to tie node+ws to the https web service.

[lpinca]

Yes, I use a similar setup in production. The only difference is that I use NGINX instead of Apache and it works as a SSL terminating reverse proxy.

Apache 2.4 is kinda bad as it uses a thread for each WebSocket connection, but this doesn't mean that it shouldn't work. I remember I got it working for a client some time ago.

[DigitalLeaves]

If you could have a look sometime and maybe give me an insight of what I'm doing wrong, it would be greatly appreciated 😊

[lpinca]

@DigitalLeaves this super simple example works on my machine (no SSL)

$ httpd -v
Server version: Apache/2.4.23 (Fedora)
Server built:   Jul 18 2016 15:38:14

$ cat 00-proxy.conf 
# This file configures all the proxy modules:
LoadModule proxy_module modules/mod_proxy.so
LoadModule lbmethod_bybusyness_module modules/mod_lbmethod_bybusyness.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule lbmethod_bytraffic_module modules/mod_lbmethod_bytraffic.so
LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_express_module modules/mod_proxy_express.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
LoadModule proxy_fdpass_module modules/mod_proxy_fdpass.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so
LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so

Config:

<VirtualHost 192.168.0.30>
  ProxyPass /ws ws://192.168.0.197:3000/
  ProxyPassReverse /ws ws://192.168.0.197:3000/

  ProxyPass / http://192.168.0.197:3000/
  ProxyPassReverse / http://192.168.0.197:3000/
</VirtualHost>

The node server:

'use strict';

const WebSocket = require('ws');
const http = require('http');
const fs = require('fs');

const server = http.createServer((req, res) => {
  res.setHeader('Content-Type', 'text/html');
  fs.createReadStream(__dirname + '/index.html').pipe(res);
});

const wss = new WebSocket.Server({ server });

server.listen(3000, () => console.log('listening on port 3000'));

index.html:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
  </head>
  <body>
    <p>It works!</p>
    <script>
      (function () {
        var ws = new WebSocket('ws://' + location.host + '/ws');

        ws.onopen = function () {
          console.log('connected');
        };
      })();
    </script>
  </body>
</html>

---

Maybe it's the trailing slash on /wss/ in your config?

ProxyPass /wss/ wss://localhost:3333/

[DigitalLeaves]

Hi there! curiously enough, that fixed it 🎉. That extra "/" was really messing around with the Apache proxy configuration. I'm kind of pissed off of the lack of documentation on this existing on the internet.

Thank you very much for your support @lpinca, you really saved me from hours of banging my head against a wall. I was even thinking on dropping websockets altogether.

Thank you so very much!

[hazel0703]

hello guys how are you?? i am having a similar error and i do not know what to do?
Error during WebSocket handshake: Unexpected response code: 404, it does not connect to my app which is hosted at AWS - EC2 instance!! i would like to know if someone can help me?

[nitin992503]

https://github.com/websockets/ws/issues/893#issuecomment-326977639
@hazel0703
hey there have you been able to solve it yet.
I am in same the situation and i am unable to find a way out

[hazel0703]

@nitin992503 yes, at the end it was a problem with the IP address, you need to pass the public IP through the path (I have no idea what you are doing), also I have to start apache on the server side

[Jamesoncy]

how about doing this with ssl?

[icedcoke]

I solved the configuration problem of apache reverse proxy wss, and now publish the configuration parameters.

 LoadModule proxy_module libexec/apache2/mod_proxy.so
 LoadModule proxy_connect_module libexec/apache2/mod_proxy_connect.so
 LoadModule proxy_http_module libexec/apache2/mod_proxy_http.so
 LoadModule ssl_module modules/mod_ssl.so
 LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so

<VirtualHost *:443>

    DocumentRoot "D:/develop/Apache24/htdocs/test/public"
    Options -Indexes

    SSLEngine on   
    ServerName my.server.com:443

    SSLProxyEngine On
    ProxyPreserveHost On
    ProxyRequests Off

    ProxyPass / wss://127.0.0.1:1234/
    ProxyPassReverse / wss://127.0.0.1:1234/

    SSLProtocol all -SSLv2 -SSLv3

    SSLCipherSuite HIGH:!RC4:!MD5:!aNULL:!eNULL:!NULL:!DH:!EDH:!EXP:+MEDIUM

    SSLCertificateFile "D:/develop/Apache24/conf/cert/public.crt"

    SSLCertificateKeyFile "D:/develop/Apache24/conf/cert/1723000.key"

    SSLCertificateChainFile "D:/develop/Apache24/conf/cert/chain.crt"

<FilesMatch "\.(cgi|shtml|phtml|php)$">

    SSLOptions +StdEnvVars

</FilesMatch>

<Directory "D:/develop/Apache24/htdocs/test/public">

    AllowOverride All

    SSLOptions +StdEnvVars

</Directory>

</VirtualHost>  

[paragikjain]

@DigitalLeaves hy how you solved this problem actually I am facing the same issue. Which extra slash you removed. Could you please help me with that. I am struggling with this issue from last 3 days please help me with this.

[DigitalLeaves]

Yes, the extra tail at /wss/ was the problem.