Harbor: 用nginx做反向代理后无法push镜像

[kenee]

重现步骤：

1. 开启一个nginx绑定一个域名docker.xxx.com 监听80和443端口，443上挂letsencrypt上获取ssl证书
2. 登录https://docker.xxx.com 访问habor的web控制台正常
3. docker login docker.xxx.com正常
4. docker push docker.xxx.com/demo/xxx 报错 "unauthorized: authentication required"

网上查到的原因是docker login是走https端口443，docker push走http端口80，导致无法读取到docker login获取的token。

临时解决办法：客户机docker启动时候带上--insecure-registry=docker.xxx.com 强制docker login走http的80端口，就可以正常push了

[reasonerjt]

@kenee
There is a nginx in Harbor docker-compose.yml, so did you setup another nginx in front of that?

Based on your description seems the root cause is here:
https://github.com/docker/distribution/issues/1177

And based on my experience, it's due to some configuration issue in your proxy.
If you are setting up another nginx please show me the conf file.

[kenee]

yes,I am setting up another nginx, I have fix this problem as below step

1) add this configure in this nginx ssl section

client_max_body_size    0;
location /v2/ {
  proxy_pass http://192.168.xxx.xxxx:5000/v2/;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_buffering off;
  proxy_request_buffering off;

}

2) expose port 5000 in harbor Deploy docker-compose.yml

[xiaods]

we can add this tips to docs.

[reasonerjt]

re comment https://github.com/vmware/harbor/issues/37#issuecomment-200656952
For item 1), did you add this section to your own nginx conf? or the nginx conf in directory Deploy/config/?

If you are configuring you own nginx:
for item 2) I don't think it's needed to expose 5000 port as your nginx server should be talking to the nginx in harbor.

[reasonerjt]

re https://github.com/vmware/harbor/issues/37#issuecomment-200697979
We will write a guide to help user update the nginx conf of harbor to enable ssl connection.

[kenee]

1） in my own nginx conf
2) only add "proxy_set_header X-Forwarded-Proto $scheme;" will not working .

here is my conf

server {
  listen 80;
  server_name docker.xxx.com;

  client_max_body_size   0;

  location / {
        proxy_pass              http://192.168.xx.xxx:80;
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
  }

  access_log /var/log/nginx/docker_registry.log;
}

server
{
    listen     443 ;
    server_name docker.xxx.com;

    ssl on;

    ssl_certificate         /etc/letsencrypt/live/docker.xxx.com/fullchain.pem;
    ssl_certificate_key     /etc/letsencrypt/live/docker.xxx.com/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/docker.xxx.com/fullchain.pem;
    ssl_session_timeout 1d;
    #ssl_session_cache shared:SSL:50m;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    # Generate with:
    #   openssl dhparam -out /etc/nginx/dhparam.pem 2048
    ssl_dhparam /etc/nginx/dhparam.pem;

    # What Mozilla calls "Intermediate configuration"
    # Copied from https://mozilla.github.io/server-side-tls/ssl-config-generator/
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    #add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;


    client_max_body_size    0;

    location /v2/ {
      proxy_pass http://192.168.xx.xxx:5000/v2/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_buffering off;
      proxy_request_buffering off;
    }

    location / {
        proxy_pass              http://192.168.xx.xxx:80;
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        proxy_http_version      1.1;
     }
    access_log /var/log/nginx/docker_registry_ssl.log;
}

[EamonZhang]

Login Succeeded

unauthorized: authentication required

@reasonerjt
where is the guide please!!!!

just do follows
https://github.com/vmware/harbor/blob/master/docs/configure_https.md