Docker-gitlab: HTTPS issue after upgrading to 8.10.0

[e11it]

HI!

I have configuration:
docker: nginx-proxy <---> gitlab (vhost)

My docker-compose (8.9.6-1):

gitlab:
  container_name: gitlab
  image: sameersbn/gitlab:8.9.6-1
  links:
    - redis:redis
  external_links:
    - postgres:postgresql
  ports:
    - "22:22"
  environment:
    - VIRTUAL_HOST=
    - LETSENCRYPT_EMAIL=
    - LETSENCRYPT_HOST=
    - DB_HOST=postgresql
    - REDIS_HOST=redis
    - REDIS_PORT=6379
    - GITLAB_HOST=
    - GITLAB_SSH_PORT=22
    - GITLAB_HTTPS=true
  volumes:
    - /etc/localtime:/etc/localtime:ro
    - gitlab_data:/home/git/data

After upgrade gitlab to 8.10.0 it's become unable to logon: error 422.
The problem was in option: GITLAB_HTTPS=true.
So if you have nginx container in front of gitlab you have to disable option GITLAB_HTTPS.

Issue: some URL in UI become with http scheme(for example project url now ssl and http).

[lorenz]

Pretty much the same configuration here, I'm not getting 422s but I'm getting URLs with http://git.my-domain.org:443/, which obviously don't work.

[lorenz]

I have fixed my setup by reverting this line.

[solidnerd]

Okay, i use an nginx as fe for container as well but I got not the problems. For me it's seems your fe config is differently to mine. If have currently set GITLAB_HTTPS to true . You should pull

docker pull sameersbn/gitlab:8.10.0 

to have latest version of it.

[solidnerd]

Here is my nginx fe configuration:

upstream gitlab {
 server gitlab_gitlab_1:80 fail_timeout=60s;
}

server {
        listen 443 ssl;
        listen [::]:443 ssl;

        server_name git.example.com;
        access_log /var/log/nginx/git.example.com/access.log;

        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 24h;
        ssl_buffer_size 1400;
        ssl_session_tickets off;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

        ssl_ciphers AESGCM:HIGH:AES256+EECDH:AES256+EDH:!aNULL;
        ssl_prefer_server_ciphers on;

        ssl_certificate /etc/letsencrypt/live/git.example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/git.example.com/privkey.pem;
        ssl_dhparam /etc/nginx/ssl/dhparam.pem;

        ssl_stapling on;
        ssl_stapling_verify on;
        resolver 8.8.4.4 8.8.8.8 valid=300s;
        resolver_timeout 10s;

        add_header Strict-Transport-Security max-age=63072000;

    location / {
            proxy_pass http://gitlab;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP       $remote_addr;
            proxy_set_header X-Forwarded-Ssl on;
  }

  location /.well-known/ {
    root /var/www/git.example.com;
  }

}
server {
        listen 80;
        listen [::]:80;
        server_name git.example.com;
        return 301 https://$server_name$request_uri;  # enforce https
}

[lorenz]

My config is similar, login works here, but for example on https://git.example.org/admin/health_check I have http://git.example.org:443/health_check... URLs and I can't access build logs. I have the latest version.

[mgansler]

Same Issue with HAProxy, @lorenz thanks for your "fix", works for me as well.

My HAProxy Config:

frontend https
        bind IPv4:443 ssl crt domain.com/haproxy.pem
        bind IPv6:443 ssl crt domain.com/haproxy.pem
        mode http
        http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload;"
        http-request add-header X-Forwarded-Proto https
        http-request set-header X-Forwarded-Port 443
        http-request del-header Proxy # https://httpoxy.org
        use_backend bk_gitlab if { ssl_fc_sni gitlab.domain.com }

backend bk_gitlab
        server gitlab 172.50.0.1:8080

[lazyfrosch]

I fixed it by adding X-Forwarded-Ssl on as a header in my frontend proxy.

Weird issue.

[mgansler]

@lazyfrosch works for me, thanks!

[dominikschulz]

Having the same issues behind an AWS ELB where I can't inject headers, so please revert this change ASAP. It's really annoying.

[adamlc]

I'm also having this issue using jwilder/nginx-proxy. Unfortunately none of the "fixes" above seem to work for me :(

[thomasf]

Something like this seems to happen regardless if I provide valid credentials or nog...

Started POST "/users/sign_in" for 85.225.60.228 at 2016-07-25 08:26:32 +0000
Processing by SessionsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"......", "user"=>{"login"=>"thomasf", "password"=>"[FILTERED]", "remember_me"=>"0"}}
Can't verify CSRF token authenticity
Completed 422 Unprocessable Entity in 11ms (ActiveRecord: 1.7ms)

Which like people have said above points to a possible solution here

https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide#cant-verify-csrf-token-authenticitycant-get-past-login-pageredirected-to-login-page

It didnt work for me (still investigating) and it has worked up until the current version.

update: X-Forwarded-Ssl on probably did the trick after some more rearranging of my haproxy.cfg.

[andy-v]

@lazyfrosch works for me too, thanks!

[thomasf]

@lazyfrosch solution does not work for me using haproxy, not right now at least.

[lazyfrosch]

@thomasf im actually setting 3 headers:

  RequestHeader set X-Forwarded-Proto "https"  # (only had this before 8.10.0)
  RequestHeader set X-Forwarded-Port 443
  RequestHeader set X-Forwarded-Ssl on

[thomasf]

@lazyfrosch I think I just got it to work after rearraging my haproxy.cfg a bit more.. the X-Forwarded-Ssl on probably did the trick.

[adamlc]

It works now, I forgot to use proxy_set_header, I was using add_header before :). So I added the following to my nginx conf:

proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Forwarded-Port 443;

[sameersbn]

Sorry about this guys. The issue was because if this change https://github.com/sameersbn/docker-gitlab/commit/48cf19f9571cc5519908ee2d5ae0df71a9e5871e (line https://github.com/sameersbn/docker-gitlab/commit/48cf19f9571cc5519908ee2d5ae0df71a9e5871e#diff-f8d3d6b83d26bfed3a639be87877628dL105 in particular). Do you think i should revert the change. Or does it make more sense that the update is performed at the loadbalancer ?

[mgansler]

I googled a bit to find out if setting X-Forward-Ssl on is recommended or not. There are some tutorials which include it, while others do not. RFC 7239 does not mention X-Forward-Ssl.
Can anybody think of a situation where adding X-Forward-Ssl on could break something?

But whatever you decided, update the Readme ;)

[lazyfrosch]

@sameersbn I guess this really depends on how users want to use SSL, I'm doing this outside the container

[solidnerd]

For everyone who uses an own Load Balancer you can expose 8181 there is the plain gitlab workhorse that only speaks http. This is relatively new.

             GitLab Container
load balancer <-> nginx <-> workhorse

I would recommend this if you use your own load balancer for ssl instead of proxing it to the nginx interface. GitLab workhorse runs on 8181

[baschte]

@sameersbn i guess you should, cause many people use your gitlab docker in combination with nginx-proxy and let it do the SSL stuff. Would be very nice if it's reverted with the next gitlab image.

[westberliner]

yup, i'm one of them. gitlab-docker behind nginx proxy for handling ssl.

[solidnerd]

@sameersbn In my opinion we should revert this. Some people got fixed it with their own load balancer but some not. I have no clue how we could solve this on AWS ELB except we revert https://github.com/sameersbn/docker-gitlab/commit/48cf19f9571cc5519908ee2d5ae0df71a9e5871e .

[sameersbn]

Have reverted the changes. Could you guys please validate the latest image against your setup? Upon confirmation I'll create a new tag. Thanks.

[mgansler]

@sameersbn Works for me (HAProxy - removed X-Forwarded-Ssl header)

BTW: there is an update available :)

[baschte]

@sameersbn tested in combination with nginx-proxy and works like a charm :-) thank you!

[kurok]

Please increase ssl bufer size
ssl_buffer_size 1400; - > ssl_buffer_size 16k;
or comment it