Distribution: 401 Unauthorized for correct password - and correct htpasswd.

[luke-park]

I'm receiving the following error message when attempting to login to a private registry from my local device:

login attempt to https://registry.redacted.com/v2/ failed with status: 401 Unauthorized

The logs from the registry server provide the following information:

redacted_1 | time="2018-07-15T00:22:09Z" level=warning msg="error authorizing context: basic authentication challenge for realm \"redacted\": invalid authorization credential" go.version=go1.7.6 http.request.host=redacted http.request.id=8d55e659-21df-426c-a0e8-35b9fcd4c345 http.request.method=GET http.request.remoteaddr="redacted:61605" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2
redacted | redacted - - [15/Jul/2018:00:22:09 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))"
redacted | time="2018-07-15T00:22:10Z" level=error msg="error authenticating user \"redacted\": authentication failure" go.version=go1.7.6 http.request.host=redactedhttp.request.id=60ba2ab8-068c-4951-b09d-313221899b28 http.request.method=GET http.request.remoteaddr="redacted:61606" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2
redacted | time="2018-07-15T00:22:10Z" level=warning msg="error authorizing context: basic authentication challenge for realm \"redacted\": authentication failure" go.version=go1.7.6 http.request.host=redacted http.request.id=60ba2ab8-068c-4951-b09d-313221899b28 http.request.method=GET http.request.remoteaddr="redacted:61606" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2
redacted | redacted - - [15/Jul/2018:00:22:10 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))"

I have confirmed that the registry is both aware of and definitely accessing my htpasswd file, as an invalid entry or filepath produces a different error log. Also of note is that the registry is running on a remote server with a valid hostname.

I have generated my htpasswd file using the command provided in the documentation:
docker run --entrypoint htpasswd registry:2 -Bbn user password > htpasswd

I'm using the following Dockerfile to build the image, and the following docker-compose to run it:

# Pull the registry image.
FROM registry:2

# Add basic authentication.
RUN mkdir /auth
COPY ./htpasswd /auth/htpasswd

# Add TLS certs.
RUN mkdir /tls
COPY ./.crt /tls/.crt
COPY ./.key /tls/.key

# Expose 443.
EXPOSE 443

version: "3"
services:
    redacted:
        build: .
        environment:
            - REGISTRY_HTTP_ADDR=0.0.0.0:443
            - REGISTRY_HTTP_TLS_CERTIFICATE=/tls/.crt
            - REGISTRY_HTTP_TLS_KEY=/tls/.key
            - REGISTRY_AUTH=htpasswd
            - REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd
            - REGISTRY_AUTH_HTPASSWD_REALM=redacted
        ports:
            - "443:443"
        volumes:
            - /mnt/registry:/var/lib/registry



md5-56cbb474e75fa1dc1e13c061892a519d



redacted:$2y$05$67b9QGTVC5vKHTS81K4EVus0cR9PmViq9846fGJC3TqHwhWbAfi3y

Note the lack of newlines. The command used to generate the htpasswd file automatically appends two newlines to the end, but registry:2 complains if they are present, so I've removed them. The "y" is the last character in the file, no newlines.

The TLS pair in use is a child certificate of a private CA - but this CA is trusted by the machine running the docker client attempting to login. I don't think it is a TLS-related issue as we seem to complete TLS handshake fine to receive a HTTP 401 error.

I am definitely using the correct password to login. What could be the problem here?

[dmp42]

Can you copy your entire registry logs, from startup?

[dmp42]

I was able to successfully login using these exact steps:

mkdir auth
docker run --entrypoint htpasswd   registry:2 -Bbn testuser testpassword > auth/htpasswd
docker run -d   -p 5000:5000   --restart=always   --name registry   -v `pwd`/auth:/auth   -e "REGISTRY_AUTH=htpasswd"   -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm"   -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd     registry:2
curl -iv -H "Authorization: Basic $(echo -n "testuser:testpassword" | base64)" localhost:5000/v2/

Can you:

• confirm these exact steps are working for you
• start adding back your specifics one by one to figure out what is breaking / where?

[matutter]

I might be seeing the same issue. I'm deploying my registry with Ansible so this is jinja2 template for a compose file. I can actually docker login to the registry but I can't seem to use any other APIs. Every other API returns a 401.

docker-compose.yml

version: '3'
services:
  registry:
    image: "{{ image }}"
    ports:
    - "5000:5000"
    environment:
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
      REGISTRY_AUTH: htpasswd
      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
      REGISTRY_HTTP_TLS_KEY: /certs/domain.key
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/docker
    volumes:
      - "{{ volume_root }}/auth:/auth"
      - "{{ volume_root }}/certs:/certs"
      - "{{ volume_root }}/config.yml:/etc/docker/registry/config.yml"
      - "{{ registry_storage }}:/var/docker"
    restart: always

config.yml

Note that I'm using joxit/docker-registry-ui on my laptop. Hence the Access-Control-Allow-Origin

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
delete:
  enabled: true
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
    Access-Control-Allow-Origin: ["http://localhost:8004"]
    Access-Control-Allow-Methods: ['OPTIONS', 'DELETE', 'HEAD', 'GET']
    Access-Control-Allow-Headers: ['Authorization', 'Content-Type', 'Access-Control-Allow-Headers', 'X-Requested-With']
    Access-Control-Max-Age: [1728000]
    Access-Control-Allow-Credentials: [true]
    Access-Control-Expose-Headers: ['Docker-Content-Digest']
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

[dubo-dubon-duponey]

@matutter did you follow the suggestion from https://github.com/docker/distribution/issues/2645#issuecomment-414005389 ?

[matutter]

@dubo-dubon-duponey I just tried this now and found no issues doing this manually with curls. I don't think its likely an issue with the registry config.

There is a script and its output attached where I manually walked through deleting a tag just with curl without any issues.

The shell script: curls.txt
The output: out.txt

[edit] My problem was resolved - it was just a CORS issue with browser I was trying to do auth through.