Azure-pipelines-tasks: Docker@0 build task doesn't respect proxy settings in config.json

[wpwoodjr]

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Docker@0

Environment

• Server - Azure Pipelines

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number: william.[email protected], SDI-K8s, Mandelbrot Dev-Docker container-CI, 20200218.46

• Agent - Hosted or Private:

  • If using private agent, provide the OS of the machine running the agent and the agent version: Ubuntu 18.04, agent v2.164.8

Issue Description

When running Docker, the agent uses the following command line:

/usr/bin/docker build -f /azp/agent/_work/1/s/mb-kubernetes/Dockerfile -t harbor.gsk.com/sdi/mandelbrot-dev:3.5.110 /azp/agent/_work/1/s/mb-kubernetes

The agent overrides my config.json file, located in /root/.docker/config.json, substituting its own config.json file, using the environment variable:

DOCKER_CONFIG=/azp/agent/_work/_temp/DockerConfig_1582085171955 link

My config.json file contains the proxy settings required for the RUN npm ci step in my Dockerfile.

By over-riding config.json and not including my proxy settings in the revised config.json, the RUN npm ci step fails in my Dockerfile.

Task logs

I'm not providing logs because the cause of my issue is evident.

[vithati]

@wpwoodjr , Task generates config file with auth details (container registry) mentioned in task and uses that config file. If you want to modify config file, use script task to update generated config file. You can try following steps as workaround for you.
1) Add docker task with login command. It generates config file with docker container registry mentioned in task. And sets config file path in DOCKER_CONFIG environment variable
2) Add script task to modify DOCKER_CONFIG file
3) Docker task with build command, here don't mention any container registry details in task, docker will use DOCKER_CONFIG file created in previous steps.

[wpwoodjr]

I mentioned the container registry in the build task to get the proper tagging. The work-around should work, but don't you think that the task should incorporate the users's config.json and not just ignore it?

[vithati]

@wpwoodjr , Task not alter user's config.json as it is not created by task. Task keeps separate config file with auth details metioned in the task and this config cleans up once job exection over.

[wpwoodjr]

@vithati That's correct. The Task does not alter my config.json. However it ignores it. The Task should take what's in my config.json and add it to the Task-created config.json. This is causing me a big headache TBH because I can't get through the proxy, and config.json is how you do that. I shouldn't have to rewrite the _Task's_ config.json!! The Task should incorporate _my_ config.json.

[vithati]

@wpwoodjr , Sorry for the inconvenience, presently task not supporting this scenario. I will add it not task backlog so that based on number requests for scenario we will add the change in task in future.

[wpwoodjr]

For others who run across this, I created a wrapper shell script for Docker. It adds proxy environment variables when building, creating, or running a container. Move the Docker executable to /usr/local/bin/docker-ce-cli and put this script at /usr/bin/docker and run chmod +x on it.

#!/bin/bash
# script to add proxies to docker build/run/create commands

cmd=/usr/local/bin/docker-ce-cli
args=()
i=0
add_arg() {
  args[i]="$1"
  i=$(($i+1))
}
add_proxies() {
  local proxies=(http_proxy HTTP_PROXY https_proxy HTTPS_PROXY ftp_proxy FTP_PROXY no_proxy NO_PROXY)
  for p in "${proxies[@]}"
  do
    add_arg "$1"
    add_arg "$p"
  done
}

for arg
do
  add_arg "$arg"
  if [[ "$arg" = "build" ]]
  then
    add_proxies "--build-arg"
  # don't inject proxies into running container (docker exec)
  elif [[ "$arg" = "run" || "$arg" = "create" ]]
  then
    add_proxies "--env"
  fi
done
exec $cmd "${args[@]}"

[vithati]

@wpwoodjr , Thanks for sharing the wrapper script.