Compose: Not able to use gcloud credential helpers with docker-compose build --pull on Windows

[cbuchacher]

Description of the issue

This is a resubmission of issue #5950 which was fixed by @shin- for _Linux_. I confirmed that this is indeed fixed on Linux, but same issue persists on Windows.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.23.0-rc3, build ea3d406e
docker-py version: 3.5.0
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.0.2o  27 Mar 2018

Output of docker version

Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:34 2018
 OS/Arch:           windows/amd64
 Experimental:      true

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker-compose config
(Make sure to add the relevant -f and other flags)

services:
  prometheus:
[...strip...]
version: '2.1'

Steps to reproduce the issue

1. gcloud auth configure-docker
2. Run docker-compose build -f compose.yml, where compose.yml contains a private image or a build context with a private image stored on the google container registry.

Observed result

The docker-compose build commands crashes when it tries to pull the image.

Expected result

The image is pulled successfully on demand.

Stacktrace / full error message

Step 16/19 : FROM eu.gcr.io/project/image:tag
ERROR: Service 'foo' failed to build: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication

Additional information

Windows 10

[shin-]

Thanks for the report! Can you share the output of docker-compose --verbose build? It'd help a lot in figuring out what part of the auth process is failing.

[cbuchacher]

Certainly. Thank you for looking into this.

# docker-compose.yml
version: '2.1'

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile

# Dockerfile
FROM eu.gcr.io/xps-ci/xavoteam/xavo-cloud-builder:8cb4222292553e2ffb5dcb9426041830211c146e

$ docker-compose --verbose build
compose.config.config.find: Using configuration files: .\docker-compose.yml
docker.utils.config.find_config_file: Trying paths: ['C:\\Users\\cbuchacher\\.docker\\config.json', 'C:\\Users\\cbuchacher\\.dockercfg']
docker.utils.config.find_config_file: Found file at path: C:\Users\cbuchacher\.docker\config.json
docker.auth.load_config: Found 'auths' section
docker.auth.parse_auth: Auth data for gcr.io is absent. Client might be using a credentials store instead.
docker.auth.parse_auth: Auth data for https://index.docker.io/v1/ is absent. Client might be using a credentials store instead.
docker.auth.load_config: Found 'credsStore' section
docker.auth.load_config: Found 'credHelpers' section
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/version HTTP/1.1" 200 549
compose.cli.command.get_client: docker-compose version 1.23.0-rc3, build ea3d406e
docker-py version: 3.5.0
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.0.2o  27 Mar 2018
compose.cli.command.get_client: Docker base_url: http+docker://localnpipe
compose.cli.command.get_client: Docker version: Platform={'Name': ''}, Components=[{'Name': 'Engine', 'Version': '18.06.1-ce', 'Details': {'ApiVersion': '1.38', 'Arch': 'amd64', 'BuildTime': '2018-08-21T17:29:02.000000000+00:00', 'Experimental': 'false', 'GitCommit': 'e68fc7a', 'GoVersion': 'go1.10.3', 'KernelVersion': '4.9.93-linuxkit-aufs', 'MinAPIVersion': '1.12', 'Os': 'linux'}}], Version=18.06.1-ce, ApiVersion=1.38, MinAPIVersion=1.12, GitCommit=e68fc7a, GoVersion=go1.10.3, Os=linux, Arch=amd64, KernelVersion=4.9.93-linuxkit-aufs, BuildTime=2018-08-21T17:29:02.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('dockercomposegcrtest_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/networks/dockercomposegcrtest_default HTTP/1.1" 404 61
compose.service.build: Building web
compose.cli.verbose_proxy.proxy_callable: docker build <- (path='\\\\?\\C:\\Users\\cbuchacher\\src\\docker-compose-gcr-test', tag='docker-compose-gcr-test_web', rm=True, forcerm=False, pull=False, nocache=False, dockerfile='Dockerfile', cache_from=None, labels=None, buildargs={}, network_mode=None, target=None, shmsize=None, extra_hosts=None, container_limits={'memory': None}, gzip=False, isolation=None, platform=None)
docker.api.build._set_auth_headers: Looking for auth config
docker.auth.resolve_authconfig: Using credentials store "gcloud"
docker.auth._resolve_authconfig_credstore: Looking for auth entry for 'gcr.io'
docker.auth.resolve_authconfig: Using credentials store "wincred"
docker.auth._resolve_authconfig_credstore: Looking for auth entry for 'https://index.docker.io/v1/'
docker.api.build._set_auth_headers: Sending auth config ('gcr.io', 'https://index.docker.io/v1/')
urllib3.connectionpool._make_request: http://localhost:None "POST /v1.24/build?t=docker-compose-gcr-test_web&q=False&nocache=False&rm=True&forcerm=False&pull=False&dockerfile=Dockerfile HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker build -> <generator object APIClient._stream_helper at 0x00000188C09F6570>
Step 1/1 : FROM eu.gcr.io/xps-ci/xavoteam/xavo-cloud-builder:8cb4222292553e2ffb5dcb9426041830211c146e
compose.cli.main.main: Service 'web' failed to build: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication

[shin-]

Thanks!

Looks like we're only finding gcr.io and the Docker Hub, but your image is on eu.gcr.io which we don't pass credentials for.

1. I assume you checked that docker build for that Dockerfile works properly?
2. Is there any mention of eu.gcr.io inside the following file? C:\Users\cbuchacher\.docker\config.json

[cbuchacher]

1. Yes, docker build works properly.
2. Yes, here it is:

{
  "credsStore": "wincred",
  "auths": {
    "gcr.io": {},
    "https://index.docker.io/v1/": {}
  },
  "stackOrchestrator": "swarm",
  "credHelpers": {
    "gcr.io": "gcloud",
    "us.gcr.io": "gcloud",
    "eu.gcr.io": "gcloud",
    "asia.gcr.io": "gcloud",
    "staging-k8s.gcr.io": "gcloud",
    "marketplace.gcr.io": "gcloud"
  },
  "HttpHeaders": {
    "User-Agent": "Docker-Client/18.05.0-ce (windows)"
  },
  "experimental": "enabled"
}

[cbuchacher]

The issue goes away if I add eu.gcr.io to auths:

   "credsStore": "wincred",
   "auths": {
     "gcr.io": {},
+    "eu.gcr.io": {},
     "https://index.docker.io/v1/": {}
   },
   "stackOrchestrator": "swarm",

[Iuiu1234]

Adding "credHelpers": {"gcr.io": "gcr.exe" }

solved for me