Compose: Images are no longer loaded if there is a build entry in the file.

[lukas9393]

Description of the issue

If I have build and image entries in the same docker-compose config file, then I cannot load my images and get no feedback on why this is happening. I did not understand the problem until I loaded the master and started the debugger. Perhaps it would be better if the program indicated that the images are not loaded. In addition, it also prints the reason for this.

I build my images on a CI server using a docker-compose.yml file and load these images into the registry. I use the same file to load the images from the registry on my server.

So we are talking about the following command sequence:

CI-Server
Step 1: docker-compose build
Step 2: docker-compose push

Production Server
Step 3: docker-compose pull
Step 4: docker-compose up -d

Because of commit c6dd7da, a docker-compose pull ignores any service that contains build instructions in the config.

If this behaviour is included in the release 1.25.0, then I (and anyone else who uses the same file for building and pulling) will have to use two separate config files in the future.

E.g. docker-compose.yml + docker-compose.build.yml

With this bug issue I would like to point out that the fix of issue #6464 breaks with the original behavior. I'm in favor of either reverting the commit or adding a flag that also loads image even though it contains build instructions in the config file.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.25.0dev, build bc57a1bd
docker-py version: 4.0.1
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.1c  28 May 2019

Output of docker version

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:32:21 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker-compose config

services:
  web:
    build:
      context: /foo
    image: luuukas/nginx
    ports:
    - published: 80
      target: 80
version: '3.7'

Steps to reproduce the issue

1. docker-compose pull
// Nothing's happening
2. docker-compose up -d
// Wants to build the image

Observed result

docker-compose pull                                
// Nothing's happening

Expected result

docker-compose pull 
Pulling web ... done

Additional information

OS version / distribution, docker-compose install method, etc.

Darwin macbook-pro-092.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

[ndeloof]

Compose behaviour when a service do declare both image and build is well documented under https://docs.docker.com/compose/compose-file/#build and https://docs.docker.com/compose/compose-file/#image :
if you specify both build and image, then build will be executed until the image tag already exist on your engine.

The issue you describe about having both a "build" and "production " Dockerfile is the main reason for compose to support overrides

Also, using a single compose file for both dev and push is a key concept https://github.com/docker/app wants to address.

[justinmchase]

Can you explain why you would include a build context if you expect the image to be pull from a remote during pull instead?

Why not just run docker-compose build as well?

[ndeloof]

Based on various issues we received on this specific topic, it seems many people do share service image (maybe built from CI?) so that they don't have to run a full build on their own computer. On a large application with many (micro)services and with suboptimal Dockerfiles this can indeed be a huge time-consumer.

But when docker-compose file is used to define application stack to be deployed on a server, one don't want to rebuild from sources (probably sources are not present and only pull images, the same way it actually is implemented by docker stack deploy.

I can't see a way to reconcile those two usages without introducing some explicit tags, to explicitly ignore build when relevant.

[justinmchase]

Thanks I understand better.

[lukas9393]

Can you explain why you would include a build context if you expect the image to be pull from a remote during pull instead?

@justinmchase In my case we use 4 docker-compose files:
docker-compose(.dev|.ci|.prod)*.yml.
If we want to build containers for our production environment on our build server, we use the commands:

docker-compose build -f docker-compose.yml -f docker-compose.prod.yml
docker-compose push -f docker-compose.yml -f docker-compose.prod.yml

If we want to set up our production environment, we want to use the same files and the command:

docker-compose up -d -f docker-compose.yml -f docker-compose.prod.yml

We want to keep the build context within our standard files, because otherwise it would mean that we would need 8 docker-compose files.
docker-compose(.dev|.ci|.prod)*(.build)*.yml

I agree with you that the bug #6464 causes problem and must be solved. And https://github.com/docker/app is a tool I would like to include in our setup. But the Commit c6dd7da needs to be extended by a better feedback for the user. Currently you don't get any information. I only got the answer after cloning the project and starting the debugger.

If the image does not exist, Compose attempts to pull it, unless you have also specified build, in which case it builds it using the specified options and tags it with the specified tag.

https://docs.docker.com/compose/compose-file/#image

Although the behavior was described in the documentation, I didn't know anything about it. I expect it is more a bug than an intentional behavior.

Maybe the service can be extended by an entry that defines whether the image should be loaded or not.

version: "3.7"
services:
  webapp:
    build:
      context: ./dir
      dockerfile: Dockerfile-alternate
      pull: false
      args:
        buildno: 1

[justinmchase]

Thanks, I think I understand better. I think my case could be solved simply by not producing an error in the case of failing to pull an image which also has a build. I'm ok with it trying to pull so long as it considers it not an error to fail in the case of services that can also be built.