Incubator-superset: redirects change https requests to http locations

[dennisobrien]

I'm running Caravel in AWS with this configuration:

• ELB terminates SSL (and accepts only https requests)
• Docker container runs gunicorn + caravel

Many requests hang in the browser because the https request is redirected to a http location.

$ curl -Ik https://caravel.example.com/
HTTP/1.1 302 FOUND
Content-Length: 239
Content-Type: text/html; charset=utf-8
Date: Thu, 18 Aug 2016 16:30:20 GMT
Location: http://caravel.example.com/caravel/welcome
Server: gunicorn/19.6.0
Connection: keep-alive

I'm not sure if this is an issue with Caravel or upstream in Flask or Flask-AppBuilder.

I tried setting PREFERRED_URL_SCHEME = 'https' in caravel_config.py hoping that would propagate to flask, but either it did not propagate, or it had no effect. (That config instructs flask what scheme to use when it cannot be determined.)

I think the right way to deal with this is to determine the protocol from the 'X-Forwarded-Proto' header. But I'm not sure if this is a bug in Caravel or Flask.

thanks,
Dennis

[mistercrunch]

We run behind a reverse proxy (nginx) that is https while the site is served by gunicorn on http and redirects are fine on our side.

Seems like a configuration issue in your reverse proxy.

[dennisobrien]

Thanks @mistercrunch
I'll try adding nginx to the mix and report back.

Just for kicks, I tried testing this to get an idea of where this might be breaking down. Here's a very simple app that just does a re-direct from '/a' to '/b'.

# app.py
from flask import Flask, redirect, url_for
app = Flask(__name__)

@app.route('/a')
def a():
    return redirect(url_for('b'))

@app.route('/b')
def b():
    return 'you made it!'

And testing this in straight-up Flask, the 'X-Forwarded-Proto' header is not recognized. (Notice the Location is https even though the header requested https.)

$ FLASK_APP=app.py flask run
...
$ curl -Ik -H "X-Forwarded-Proto: https" localhost:5000/a
HTTP/1.0 302 FOUND
Content-Type: text/html; charset=utf-8
Content-Length: 211
Location: http://localhost:5000/b
Server: Werkzeug/0.11.10 Python/3.5.2
Date: Sun, 21 Aug 2016 22:52:23 GMT

And running with gunicorn it _does_ handle the 'X-Forwarded-Proto' header. (Notice the Location is https now.)

$ gunicorn -b 127.0.0.1:5000 app:app
...
$ curl -Ik -H "X-Forwarded-Proto: https" localhost:5000/a
HTTP/1.1 302 FOUND
Server: gunicorn/19.1.0
Date: Sun, 21 Aug 2016 22:53:13 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 211
Location: https://localhost:5000/b

So gunicorn looks like it should be doing the right thing.

I think this can be closed since I believe the problem is not in Caravel. And I'll update this issue with whatever I learn.

thanks,
Dennis

[xrmx]

Thanks for the heads up, closing then.

[dennisobrien]

For the record, I found the cause of the problem and the fix. When gunicorn is run on a different machine from the load balancer (nginx or ELB), it needs to be told explicitly to trust the X-Forwarded-* headers sent. gunicorn takes an option --forwarded-allow-ips which can either be a comma separated list of ip addresses, or "*" to trust all.

I'm starting caravel with this command (with gunicorn running behind an ELB):

gunicorn \
  --error-logfile - \
  --access-logfile - \
  -w 8 \
  -k gevent \
  -b 0.0.0.0:8080 \
  --timeout 120 \
  --limit-request-line 0 \
  --limit-request-field_size 0 \
  --forwarded-allow-ips="*" \
  caravel:app

More details are in the gunicorn docs:
http://docs.gunicorn.org/en/stable/deploy.html

cheers,
Dennis

[rlei]

@dennisobrien In http://airbnb.io/superset/installation.html#configuration-behind-a-load-balancer

If the load balancer is inserting X-Forwarded-For/X-Forwarded-Proto headers, you should set ENABLE_PROXY_FIX = True in the superset config file to extract and use the headers.

Have you tried this setting?

[ecliptik]

I recently ran into this as well when trying to use a AWS load-balancer to a container and fixed it the following way.

1. superset v0.17.1 with docker image: python:3.6
2. Set ENABLE_PROXY_FIX = True in superset_config.py
3. Used a Application Load Balancer (ALB) instead of a classic ELB going to the container port over http and https

[Kumar1325]

@ecliptik I'm using the below configuration, running superset in python virtualenv

1. Superset v0.17.0 : python:2.7.12,
2. Set ENABLE_PROXY_FIX = True in superset_config.py
3. Using the ALB.

I'm able to access the application using the ALB endpoint, while logging in, it says "Invalid login", eventhough giving the valid credentials

could help me on resolving this.

[Jie-Yang]

I am having similar problem: ALB works find with HTTP, but when I binding HTTPS to the ALB, I get error: 502 Bad Gateway. I have tried to figure out more details about the error, but not much information logged in S3. Is there anywhere I can see more information about this error? There is no any error from the superset console. I suspect the problem happens in ALB, some headers not matched maybe?

and I have tried ENABLE_PROXY_FIX = True in superset_config.py and also environment variable FORWARDED_ALLOW_IPS=*, but not luck.

my Gunicon version is 19.7.1, superset is 0.18.5.

Any suggestion or tip is appreciated! Thanks

Jay

[Jie-Yang]

For people may come across the same problem, I have found the problem. By mistake, I set protocol of target group to HTTPS. When I change it to HTTP, everything works!

[jhmartin]

@Jie-Yang can you close the issue then?

[Maxwell2022]

@mistercrunch Can we actually re-open it? I'm using the latest 0.24.0 within docker (https://github.com/amancevice/superset). In front of this container, I have nginx where my SSL certificate is installed.

Everything is working fine, except that I have superset in an Iframe, and that as soon as the user login, it gets redirected to HTTP and the browser doesn't like it...

Here is the call I'm making to the login page with the user being login already, and how it gets redirected back an forth:

=> https://superset.example.com/login/
-> (301) http://superset.example.com
-> (302) https://superset.example.com
-> (301) http://superset.example.com/welcome/
-> (302) https://superset.example.com/welcome/

The 301 is happening because I have a rule in nginx to force HTTPS. However the issue is with the application redirect()

I tried all the config variables listed here, but still nothing:

ENABLE_PROXY_FIX = True
PREFERRED_URL_SCHEME = 'https'

[Maxwell2022]

So I slept over this issue and investigate a bit more about Flask application and how these are working behind reverse proxies. It's expecting the X-Forwarded-Proto header from the reverse proxy. So I just added it to my nginx config and all good, no more redirect to http. Here is my nginx config, if I can help anyone else:

upstream api {
  server superset-server:8088 max_fails=3;
}

map $http_upgrade $connection_upgrade {
  default       "upgrade";
  ""            "";
}

# Force HTTPS
server {
  listen 80;
  server_name superset.mydomain.com;
  return 301 https://$host$request_uri;
}

server {
  listen 443 ssl http2;
  server_name       superset.mydomain.com;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_certificate   /etc/nginx/letsencrypt/live/superset.mydomain.com/fullchain.pem;
  ssl_certificate_key /etc/nginx/letsencrypt/live/superset.mydomain.com/privkey.pem;
  ssl_trusted_certificate /etc/nginx/letsencrypt/live/superset.mydomain.com/chain.pem;

  access_log /var/log/nginx/superset.access.log main;
  error_log  /var/log/nginx/superset.error.log error;

  # Configuration for Lets Encrypt certificate renewal
  include /etc/nginx/snippet/acme-challenge.conf;

  location / {
    proxy_pass         http://api;
    proxy_redirect     off;

    proxy_set_header   Connection $connection_upgrade;
    proxy_set_header   Upgrade $http_upgrade;
    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Proto $scheme;
  }
}

[chenhaiyan]

@Maxwell2022 hello,I want to use superst in an inframe,I use https url，but redirect to http,can you tell me the file location that contain the config PREFERRED_URL_SCHEME?

[debojitkakoti]

This problem can be easily solved if the application is running behind AWS ALB.
Step 1. Create two listener in ALB, one running on port 80 and another running on 443 with certificate attached to it.
Step 2. Write a redirect rule in listener with port 80 for permanent redirect to port 443, your rule should look like below

https://#{host}:443/#{path}?#{query

So any redirect happens on the application level will redirect to https. Hope this helps.

[nowak-ninja]

Hey, I think redirects on ALB/Nginx level from 80 to 443 is a workaround, not a solution. I am not familiar with flask/gunicorn/whatever runs the Superset and anyway tried to force redirects go to https rather than http, but without success. I ended up with redirect solution on ALB, JUST for Superset. Is there ANY other way to force Superset to use https? Middleware or something?