Compose: Docker compose pull doesn't respect local images

[antony]

During my searches I've seen this issue raised and closed a few times, but even using the latest build of docker-compose, this still doesn't behave the way I understand that it is expected to.

Scenario:
We're building a composed environment for our CI tests to run. We grab a database image, an api image, and a _local_ copy of the image containing application we are trying to test (it has been built in a previous CI step, and tagged with the build hash provided by our CI environment (CIRCLE_SHA1)

docker-compose.yml

api:
  image: mycompany/api

undertest:
  image: mycompany/undertest:${CIRCLE_SHA1}

db:
  image: mycompany/db

The commands we run then are as follows:

docker-compose pull # Because we definitely want the latest db, and api for a true test.
docker-compose up # compose the environment

Actual Result:
No matter what I do, docker compose always tries to pull my CIRCLE_SHA1 tagged version from docker hub. it doesn't exist there, I never want to push it (until it passes tests and is re-tagged as :latest and/or :release

I have a unique tag CIRCLE_SHA1 which only exists inside the build environment, meaning no confusion for docker-compose when it tries to pull, and yet, it seems to try to fetch it anyway, and fail even though that exact tag exists locally.

Expected Result:
I'd expect the fact that there is no remote build tagged with CIRCLE_SHA1 to cause docker-compose to use the local copy it finds. I need to do pull, because I want everything else to be the latest.

I'd suggest that if there is confusion where image: refers to a remote repository, then perhaps we could use local: instead, to reference a local image?

[decoomanj]

I'm facing similar issues. I use the --ignore-pull-failures option when I do a docker-compose pull. This basically ignores ALL errors. It is ok as a workaround. We do the same thing in our pipeline. I pull other production-images and start my freshly build release-candidate container locally. When the tests are ok, it's uploaded.

[johnharris85]

I don't see this is a issue / bug with current functionality. pull is doing exactly that, how would it know in your original compose file that 2 images should be pulled remotely but one should be grabbed locally (seeing as they all probably exist locally right, you just want to 'refresh' the other two?).

Perhaps a --filter option to pull? So something like:

docker-compose.yml
version: '2'
services:
  api:
    image: mycompany/api
    labels:
      - "refresh"
  undertest:
    image: mycompany/undertest:${CIRCLE_SHA1}
  db:
    image: mycompany/db
    labels:
      - "refresh"

Then:

docker-compose pull --filter "label=refresh"

@aanand @dnephin thoughts?

[dnephin]

--ignore-pull-failures was added for this very reason.

You could also do docker-compose pull db api to only pull the ones you wanted.

[johnharris85]

Both of those options seem kind of sub-optimal.

It's not really a 'failure' to ignore, more of a separate use case.

The second option means you're having to declare some behavior / state outside of the compose file. I.e. I have all of this automated but then want to change which images are pulled and which aren't, I probably want that to be reflected in my compose file as a single-point-of-truth, but I have to edit the pull command instead.

[antony]

I'd agree with @johnharris85 on --ignore-pull-failures not feeling like the option to use. Something like --prefer-local would work - but I'd still probably prefer a concrete local: my-org/my-image:tag to make for a clearer intent.

I didn't know you could pull specific images, so thanks for that @dnephin - the downside being, this would move knowledge about the composure of my stack outside the compose file, whereas for me a compose file is a one-stop encapsulation of my architecture.

[dnephin]

The second option means you're having to declare some behavior outside of the compose file.

Yes, and in this case I think that's correct. The behaviour that is external to the Compose file is not the behaviour of a container, but part of your project dev workflow. Compose is not a project workflow tool. It's a tool for orchestrating containers.

The Compose file should really only include configuration for those containers. How to pull an image is not container configuration, it's part of the workflow, and currently all of those options are commands and flags. I think this is an important distinction. Trying to mix both would make the Compose file very confusing.

The equivalent of --prefer-local is to just never run docker pull in the first place. Any missing images will be pulled before trying to run the container.

[JashaadGaines]

Problem: docker-compose -f docker-compose.yml up -> doesn't look locally for the image at all.
I agree @johnharris85 - These options are suboptimal. Docker is sort of acting like a dependency manager or at least one dependency resolver. It pulls images from a specified/default repo then/and only then, commences it's docker process. In the same way gradle, ant, maven, gulp, npm, etc. pull dependencies and define processes the user can run. The standard has been set for Dependency Mgr/Resolvers, look locally first (sometimes cache or something), then pull from repo. I'm trying to think of a really pressing reason why docker would want to force you to look at the repo first. Just like @antony & @decoomanj I don't want to push any images that don't even satisfy our lowest level of quality threshold: unit tests. Not to mention any other test suite.

Suggestion:
rancher-compose offers a --pull option to insure you're getting the latest image from the repo. It would be great if docker-compose looked for images locally first then to the repo. And if necessary there can be a --pull option to force docker to only search for images in the repo.

[martindariocernadas]

Any clues how to avoid this issue?

[smebberson]

I'd like to know how to avoid this issue too.

[izrobin]

I'd also like to use a locally built image, before building and publishing a real image to our registry. How would I go about doing that? Looking at this conversation, it seems impossible to use locally installed images?

[KamilKopaczyk]

@dnephin

The Compose file should really only include configuration for those containers. How to pull an image is not container configuration

It's not only for containers if docker-compose allows to pull and push.

Maybe it shouldn't be concern for docker-compose? Maybe docker should mark locally built, not-yet-pushed images and ignore it when trying to pull this particular image.

It's PITA since latest docker-compose releases, we're able to build docker images with concrete name specified in docker-compose.yml.
This way if you do docker-compose build --pull you end up with errors and images not being pulled and built. It's even worse - you can't use --ignore-pull-failures flag with docker-compose build.

[skehlet]

A hacky workaround to use your local images is to run your own registry:

docker run -d -p 5000:5000 --name registry registry:2
docker tag 4e6a104bad19 localhost:5000/stevek/my-container
docker push localhost:5000/stevek/my-container
# edit docker-compose file to use localhost:5000/stevek/my-container
docker-compose ...

[Constantin07]

+1 same issue

[karunapuri]

Try this './docker-env -c ./build --no-update'. This command works for running locally built images.

[joeyhub]

This kind of solution isn't really viable:

 --ignore-pull-failures was added for this very reason.

People want to ignore that an image is local. Not necessarily every other type of error.

Developers aiming to create robust systems will program strictly and abort on the first error. This breaks with that.

In nearly all my years of programming auto-fix, force and ignore options turn out useless if you program safely. They tend to only be genuinely useful in 0.0001% of cases for safe code.

If it's to be useful, then it should only fire when the error is non-critical (as in it has the image, only update failed). That's or there should be a variant that does that.

It should be possible to have something such as pull --missing-only. That would at least allow a pre-fetch.

It doesn't seem like docker stores enough information on images to know if they are local or remote.

The correct way to support it would be to add an option in the docker compose file that an image is built locally.

There's the question of why this happens at all if docker-compose can build images (and then knows that they are local).

For me it's because it isn't very good at that (for example it doesn't have a separate build section, instead it's crammed into services), I've made my own tool to manage and wrap that.

This then breaks pull because docker-compose doesn't know about them being built locally.

I can implement the functionality I need with my system. At that point docker-compose wont be used for build or pull at all for my, only service management (run, stop, up, down, start).

It would be nice though if docker compose fixed these issues at the core. It is also missing an image section where you should be able to specify images that are local and some management for private repositories.

If you have a repository section then you could specify images for those repositories (is left out maybe have it automatic, then some thought going into what happens when an image is in the global and one or more private).

repositories:
     local:
           url: ~ (maybe blank for local)
           images: [a, b, c] (simplest)
           namespace: company (works dependent on how you're tagging your images, matches company/db, can be more complex like a/b/c)

[noaho]

With the status quo, using —ignore-pull-failures and —include-deps together doesn’t actually work.. my dockerfile has “FROM: ubuntu:16.04”, but this dependency is never pulled because of the pull failure...

Of course it is pulled when I later build, but there is no way to force pull the latest version of every dependency, so it is left with the first version of the image forever...

I imagine there is a lot of out of date code in production for this reason..

[zuernBernhard]

--prefer-local (prefer newer image with same tag on local machine) would be great - than we can use the same compose file and workflow in local development and in production - and it is always possible to locally override images with ease for testing purpose

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[SuperSandro2000]

Bump

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[arkadyt]

Bump

[amackiewicz]

bump

[ndeloof]

On 1.25.1 service with a Build section won't break docker-compose build but report need to build image.

see https://github.com/docker/compose/pull/7052

[ndeloof]

@noaho as you use “FROM: ubuntu:16.04” in a Dockerfile this is not what compose consider a "dependency". dependencies is about services in your compose file, i.e if you run docker-compose <command> service1 --include-deps and service1 depends on service2 in your compose file, then command will also be ran on service2.

You're in charge for base image used by your Dockerfile. If you wish to get notified on updates/vulnerabilities reports, then check DockerHub features.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[thisismydesign]

Don't close this issue. This is an automatic message by Fresh - a bot against stale bots.

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[jpbochi]

bot wars!

[fgauer]

Hi chiming in here...this has had me stumped for days trying to stand up a new Docker based development environment using docker-compose and has led to much frustration and needless pain.

Coming from an environment where we have not used the image: feature in docker-compose.yml and now attempting to utilize it for CI/CD purposes for a new project we realized, after a significant amount of time, trial and error, that the docker-compose up always pulls from the repo defined in image: and does not give us a way to intuitively pull from a local image where/when we need to (i.e. local development).

We have the following goals with our setup:

1) We want to use the same set of docker-compose.yml files for CI/CD purposes in various environments, which includes local development, that should allow for a developer to make changes to the code base and immediately re-build the containers on their local machines in order to work with their changes.
2) In order to facilitate #1, in other words use the same set of scripts for various deployment environments, we use ENV variable substitutions in our docker-compose.yml files. This works fine until you throw in the local development scenario where we need to grab locally built images for development iteration.

Since the image: feature currently does not allow us to intuitively override ride it in such a way as to have it look for a local image, I believe we faced with going with one of the following options below. Please bear in mind that whereas the options below work, it was getting to this conclusion that was really quite painful due to the counterintuitive nature (at least this was my experience personally) of how the image: feature works without some kind of local override option.

1) Setup a completely separate docker-compose.yml file for local development purposes, or overlay multiple docker-compose.yml files to specify the environment’s image: tag in all environments except for local development. But with the ENV variable substitution features current working in docker-compose.yml for all of our other environments, why would we have to go here? It would be nice to substitute something in like ‘local:’ for the image name and have Docker Compose respond accordingly using ENV variables.

2) Setup a local ‘development’ repository on each developer’s local machine. We could do this, but it seems to rub against one of the fundamental advantages of using Docker for local development which is to minimize the amount of individual local environment setup for individual developers, and tracking/managing that whole circus.

[lucasbasquerotto]

@fgauer In my case I don't want to pull images if there is already a local image because that can be time consuming if there are lots of images (even if all of them are already pulled), furthermore it could give errors if there's no internet or if the connection fails.

What I did to solve it was to not use image: but build: in the docker-compose.yml file, because build pulls only when needed, and I use a Dockerfile that only defines the image I want to pull. I use build args to not have to create lots of Dockerfiles, like the following:

_Dockerfile:_

ARG IMAGE
ARG VERSION

FROM $IMAGE:$VERSION

docker-compose.yml:

services:
  my_service:
    build:
      context: .
      dockerfile: "path/to/Dockerfile"
      args:
        IMAGE: "alpine"
        VERSION: "latest"

Then I just run docker-compose build (and not docker-compose pull) and the problem is solved for me.

I don't know if it will work in your case, but you could give it a try.

[erdnaxeli]

To sum up, the problem is the following:

• we want to run docker-compose pull to pull external images
• we don't want the same command to fail on local only images, because that's not an actual error, that's the expected behavior (you can't pull an image not pushed on a registry)
• we don't want to have to specify the list of images to pull in the command line, because we want all the config to stay in the docker-compose file
• we don't want to ignore all errors, because errors can happen and we want to see them
• we don't want to have to run a local registry, that's useless and prone to security errors

Currently it fails because it does not know that the image is local only. Can't we add something in the docker-compose file to let it know it is local only and it should not try to pull it? Something like this would do the job:

services:
  my_service:
    image: some_external image
  my_private_service:
    image: some_local_only_image
    pull: false

This way I am explicitly telling docker compose that I don't want it to try to pull this image, and everyone is happy.

[SuperSandro2000]

I think this syntax is more logical.

services:
  my_service:
    image: some_external image
  my_private_service:
    image:
      name: some_local_only_image
      pull: false