Origin: Error: Status 503 trying to push repository

Created on 8 Feb 2017 · 12Comments · Source: openshift/origin

I've created a docker image containing my app, and I want to push it to remote docker-registry on OpenShift.

Version

oc v1.4.1+3f9807a
kubernetes v1.4.0+776c994
features: Basic-Auth

Server https://openshift.abcdefg.co:8443
openshift v1.4.1+3f9807a
kubernetes v1.4.0+776c994

Preconditions

I've OpenShift Origin installed on remote server https://openshift.abcdefg.co:8443
Default project is created and contains docker-registry pod, service and route
Created local docker image aaaaa/bbbbb:latest
Tagged image docker tag aaaaa/bbbbb:latest docker-registry-default.<ip-address>.xip.io/bbbbb
Added docker-registry-default.<ip-address>.xip.io/bbbbb to docker's insecure registries list

Steps To Reproduce

Login to remote OpenShift server
$ login oc login https://openshift.abcdefg.co:8443
Get session token
$ oc whoami -t
LOAfJAT4gf7CeR-o_1-LVoud_ABCD-8vtO1B0kNFPs0
Login to remote docker registry
$ docker login -u admin -p LOAfJAT4gf7CeR-o_1-LVoud_ABCD-8vtO1B0kNFPs0 docker-registry-default..xip.io
Login Succeeded
Push tagged image
$ docker push docker-registry-default..xip.io/bbbbb

Current Result

The push refers to a repository [docker-registry-default.<ip-address>.xip.io/bbbbb]
835e2cdde455: Retrying in 1 second 
be9e9528fd52: Retrying in 1 second 
6f7515f19096: Retrying in 1 second 
da07d9b32b00: Retrying in 10 seconds 
7cbcbac42c44: Retrying in 10 second
Error: Status 503 trying to push repository bbbbbb: "<html>\n  <head>\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n  <style type=\"text/css\">\n  /*!\n   * Bootstrap v3.3.5 (http://getbootstrap.com)\n   * Copyright 2011-2015 Twitter, Inc.\n   * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)\n   */\n  /*! normalize.css v3.0.3 | MIT License | github.com/necolas/normalize.css */\n  html {\n    font-family: sans-serif;\n    -ms-text-size-adjust: 100%;\n    -webkit-text-size-adjust: 100%;\n  }\n  body {\n    margin: 0;\n  }\n  h1 {\n    font-size: 1.9em;\n    font-weight: 400;\n    line-height: 1.3;\n    margin: 0.68em 0;\n  }\n  @media (min-width: 768px) { \n    h1 {\n      font-size: 2.15em;}\n  }\n  * {\n    -webkit-box-sizing: border-box;\n    -moz-box-sizing: border-box;\n    box-sizing: border-box;\n  }\n  *:before,\n  *:after {\n    -webkit-box-sizing: border-box;\n    -moz-box-sizing: border-box;\n    box-sizing: border-box;\n  }\n  html {\n    -webkit-tap-highlight-color: rgba(0, 0, 0, 0);\n  }\n  body {\n    font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n    line-height: 1.66666667;\n    font-size: 13px;\n    color: #333333;\n    background-color: #ffffff;\n    margin: 4em 3em;\n  }\n  p {\n    margin: 0 0 10px;\n    font-size: 13px;\n    font-weight: 300;\n    line-height: 1.4;\n  }\n  @media (min-width: 768px) {\n    p {\n      font-size: 14px;\n    }\n  }\n  </style>\n  </head>\n  <body>\n    <div>\n      <h1>Application is not available</h1>\n      <p>The application is currently not serving requests at this endpoint. It may not have been started or is still starting.</p>\n    </div>\n  </body>\n</html>\n"

Expected Result

Image should be pushed successfully.

componenimageregistry kinquestion prioritP2

Source

bhbv

👍2

Most helpful comment

same issue here pushing an image

received unexpected HTTP status: 503 Service Unavailable

kb7791 on 3 Jul 2019

👍3

All 12 comments

@legionus it looks like perhaps the ip address here is pointing to the router and that the route is either not created or not responding properly although the reproduction steps show the login was successful which is strange.

@bahadyr if you have set up a route for the registry service can you paste the route definition? Thanks!

pweil- on 8 Feb 2017

Thanks @pweil- Here is the route YAML.
Yes, I'm able to login the registry, but not able to push image.

apiVersion: v1 kind: Route metadata: name: docker-registry namespace: default selfLink: /oapi/v1/namespaces/default/routes/docker-registry uid: 7c5c6b2e-ec89-11e6-bc16-406186becd9d resourceVersion: '572857' creationTimestamp: '2017-02-06T16:29:52Z' labels: docker-registry: default annotations: openshift.io/host.generated: 'true' spec: host: docker-registry-default.176.-.--.--.xip.io to: kind: Service name: docker-registry weight: 100 port: targetPort: 5000-tcp wildcardPolicy: None status: ingress: - host: docker-registry-default.176.-.--.--.xip.io routerName: router conditions: - type: Admitted status: 'True' lastTransitionTime: '2017-02-06T16:29:52Z' wildcardPolicy: None

bhbv on 8 Feb 2017

@bahadyr You use a proxy?

legionus on 8 Feb 2017

@bahadyr looks like it is possibly that the route is not a passthrough route (see the termination type)

pweil- on 8 Feb 2017

@legionus no. @pweil- I just set it in YAML. Tried to push image, and getting 503 now.
``$ docker login -u admin -p gxmcPQSevLoV8hCEiv0Os_ZwUyB9dASDwLMn6Ww4Gs3fl0 docker-registry-default.176.-.--.--.xip.io Error response from daemon: Login: <html> <head> <meta name="viewport" content="width=device-width, initial-scale=1"> <style type="text/css"> ... </style> </head> <body> <div> <h1>Application is not available</h1> <p>The application is currently not serving requests at this endpoint. It may not have been started or is still starting.</p> </div> </body> </html> (Code: 503; Headers: map[Pragma:[no-cache] Cache-Control:[private, max-age=0, no-cache, no-store] Connection:[close] Content-Type:[text/html]])

bhbv on 8 Feb 2017

@bahadyr just for completeness can you delete and recreate the route as passthrough rather than editing it? Then ensure that the route is accepted (via the status conditions) before trying a login and push? A 503 usually means that the router got the request but didn't have the route to serve. If that doesn't work we can debug more.

pweil- on 8 Feb 2017

@pweil-
I've tried with tls:termination: passthrough and without it.
So with passthrough i'm not able even login.
Without passthrough I can login but still can not push.

$ docker push docker-registry-default.176.-.--.--.xip.io/bbbbb
The push refers to a repository [docker-registry-default.176.-.--.--.xip.io/bbbbb]
835e2cdde455: Preparing 
be9e9528fd52: Preparing 
6f7515f19096: Preparing 
da07d9b32b00: Retrying in 4 seconds 
7cbcbac42c44: Retrying in 4 seconds

bhbv on 8 Feb 2017

magick93 on 13 Feb 2017

It looks like the issue in idle state. Reopen if the problem is still relevant.

legionus on 6 Feb 2018

I realize this is closed, but I have the same issue now. anybody know how to resolve it?

spock123 on 22 May 2019

@kb7791: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.