Compose: docker-compose build --pull doesn't use osxkeychain

[jeffsawatzky]

Description of the issue

Getting the following error when doing docker-compose build --pull:

ERROR: compose.cli.main.main: Service 'my_app' failed to build: unauthorized: access to the requested resource is not authorized

The Dockerfile uses an image from quay.io, which I have authorized using:

docker login quay.io

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

Output of docker version

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:18:17 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:17:52 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Steps to reproduce the issue

1. docker-compose build --pull

Observed result

ERROR: compose.cli.main.main: Service 'my_app' failed to build: unauthorized: access to the requested resource is not authorized

Expected result

SUCCESS or something

Stacktrace / full error message

[jsawatzky@jsawatzky2019 my_app (VI-1035)]$ docker-compose --verbose build --pull
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: ['/Users/jsawatzky/.docker/config.json', '/Users/jsawatzky/.dockercfg']
docker.utils.config.find_config_file: Found file at path: /Users/jsawatzky/.docker/config.json
docker.auth.load_config: Found 'auths' section
docker.auth.parse_auth: Auth data for quay.io is absent. Client might be using a credentials store instead.
docker.auth.load_config: Found 'credsStore' section
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.22/version HTTP/1.1" 200 849
compose.cli.command.get_client: docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
compose.cli.command.get_client: Docker base_url: http+docker://localhost
compose.cli.command.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '19.03.1', 'Details': {'ApiVersion': '1.40', 'Arch': 'amd64', 'BuildTime': '2019-07-25T21:17:52.000000000+00:00', 'Experimental': 'false', 'GitCommit': '74b1e89', 'GoVersion': 'go1.12.5', 'KernelVersion': '4.9.184-linuxkit', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v1.2.6', 'Details': {'GitCommit': '894b81a4b802e4eb2a91d1ce216b8817763c29fb'}}, {'Name': 'runc', 'Version': '1.0.0-rc8', 'Details': {'GitCommit': '425e105d5a03fabd737a126ad93d62a9eeede87f'}}, {'Name': 'docker-init', 'Version': '0.18.0', 'Details': {'GitCommit': 'fec3683'}}], Version=19.03.1, ApiVersion=1.40, MinAPIVersion=1.12, GitCommit=74b1e89, GoVersion=go1.12.5, Os=linux, Arch=amd64, KernelVersion=4.9.184-linuxkit, BuildTime=2019-07-25T21:17:52.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('partnersync_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.22/networks/partnersync_default HTTP/1.1" 404 38
compose.project.build: db uses an image, skipping
compose.project.build: mq uses an image, skipping
compose.service.build: Building my_app
compose.cli.verbose_proxy.proxy_callable: docker build <- (path='/Users/jsawatzky/Code/my_company/my_app', tag='my_app_my_app', rm=True, forcerm=False, pull=True, nocache=False, dockerfile=None, cache_from=None, labels=None, buildargs={}, network_mode=None, target=None, shmsize=None, extra_hosts=None, container_limits={'memory': None}, gzip=False, isolation=None, platform=None)
docker.api.build._set_auth_headers: Looking for auth config
docker.auth._resolve_authconfig_credstore: Looking for auth entry for 'https://quay.io'
docker.api.build._set_auth_headers: Sending auth config ('quay.io', 'https://quay.io')
urllib3.connectionpool._make_request: http://localhost:None "POST /v1.22/build?t=my_app_my_app&q=False&nocache=False&rm=True&forcerm=False&pull=True HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker build -> <generator object APIClient._stream_helper at 0x1109cc200>
Step 1/2 : FROM quay.io/my_company/base-python:3.7-alpine3.9
ERROR: compose.cli.main.main: Service 'my_app' failed to build: unauthorized: access to the requested resource is not authorized

Additional information

Docker for Mac
I noticed that docker-compose pull works fine (it uses the creds from the osxkeychain)

[jeffsawatzky]

if I delete the quay.io entry from my ~/.docker/config.json it works fine.

[thzinc]

I've also experienced this with a private Harbor registry on macOS with the exact same docker/docker-compose versions. Thanks for the config.json workaround, @jeffsawatzky.

[camilo-globant]

Deleting the quay.io entry from config.json worked for me. Thanks @jeffsawatzky.

[bobvanmanen]

I'm running into the same issue where a Dockerfile is depending on a Docker container that's stored on a private Artifactory registry:

FROM private.host.com/path/to/container:label

Result:

ERROR: compose.cli.main.main: Service 'runtime' failed to build: manifest for private.host.com/path/to/container:label not found: manifest unknown: The named manifest is not known to the registry.

Deleting private.host.com and https://private.host.com from config.json fixes the issue.

[davidristovski]

@jeffsawatzky Works like a charm, many thanks!

[jdodds]

I had the same issue regardless of whether i was using --pull or not, deleting the auth entries worked.

[ndeloof]

Can you please check your ~/.docker/config.json is well set to use docker desktop credentials store ?

{
  "credsStore" : "desktop",
  ...
}

[ndeloof]

This issue might be a duplicate for https://github.com/docker/compose/issues/6713
in such case, a fix is on his way

[jdodds]

Can you please check your ~/.docker/config.json is well set to use docker desktop credentials store ?

mine was set to osxkeychain

[xeivieni]

Same here, if I delete the "credsStore" : "desktop" line in my config it just works fine. Also works on 1.23

[glensc]

(cross post https://github.com/docker/compose/issues/6686#issuecomment-560296898)

my current (better) workaround is to force use of buildkit, which does not invoke docker-credential helper without arguments:

$ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose up

$ docker --version
Docker version 19.03.5, build 633a0ea
$ docker-compose --version
docker-compose version 1.25.0-rc4, build 8f3c9c58

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.