Compose: 1.7.1 doesn't set network aliases when using `run` command

[tylerbrandt]

I noticed what appears to be a regression upon upgrading from 1.7.0 to 1.7.1, where the network "aliases" are not defined when running docker-compose run.

This is the simplest docker-compose.yml configuration I could create to illustrate the difference:

version: '2'

services:
  test:
    image: busybox
    networks:
      default:
        aliases:
          - busy
    command: sh

When running docker-compose run test on 1.7.1 the "Aliases" key is empty:

> docker-compose -v
docker-compose version 1.7.1, build 0a9ab35
> docker-compose run test
> docker inspect --format '{{.NetworkSettings.Networks.composetest_default.Aliases}}' $(docker-compose ps -q)
[]

On 1.7.0 I see the defined alias:

> docker-compose -v
docker-compose version 1.7.0, build 0d7bf73
> docker-compose run test
> docker inspect --format '{{.NetworkSettings.Networks.composetest_default.Aliases}}' $(docker-compose ps -q)
[busy]

I also tried using a defined network, but Aliases weren't created there either under 1.7.1. Aliases _are_ assigned as expected using docker-compose up -d test.

Is the 1.7.1 behavior expected, or a bug? In my case the new behavior is breaking a selenium test suite that I have created, which has two services which need bidirectional communication. My solution (under 1.7.0) was to link my "app" service to the "selenium" service, and define an alias for the app service (on the default network). Then I docker-compose run the "app" service in order to run a series of tests (I can attach the docker-compose.yml file for the actual service if necessary but it uses some proprietary images). Under 1.7.0 the containers can resolve each other using the defined aliases, but under 1.7.1 only "app" can see "selenium", but not the other way around.

[vdemeester]

@tylerbrandt given the acceptance tests (https://github.com/docker/compose/blob/master/tests/acceptance/cli_test.py#L1118), this is by design

[vdemeester]

(also seems like #3437 and this one are duplicates /cc @dnephin @aanand)

[tylerbrandt]

How does docker suggest I resolve this? I currently run my tests using a script that uses docker-compose run --rm e2e to start a service and its dependencies, run tests, and exit with the exit code of the test command. Do I need to replace it with something like this (not very graceful)?

# Start service(s) and run tests; this is needed instead of run to apply network alias
# Runs `npm run test-ci`
docker-compose up e2e

# Get exit code of stopped container
CONTAINER_ID=$(docker-compose ps -q e2e)
RESULT=$(docker inspect -f '{{ .State.ExitCode }}' $CONTAINER_ID)

# Stop service(s)
docker-compose down

# Return result
exit $RESULT

[dnephin]

docker-compose up --build -d
docker-compose run --rm e2e
status=$?
docker-compose down
exit $status

run will use the environment you created with `up.

You could use trap instead of capturing the exit status as well.

[robpblake]

Is there any work-around for this other than down-grading to 1.7.0 of compose? I need to give a container a network alias when running a docker-compose run command.

If no work-around, any idea when this is scheduled to be fixed?

[aanand]

docker-compose run is not designed for long-running tasks, so it doesn't make sense to add any aliases to the container it creates.

What are you trying to achieve?

[artemkaint]

this needs be linked with https://github.com/docker/compose/issues/3437

@aanand
I'd like to discribe my use cases. I run dev container with some wacher and web service and want to follow watcher's output and access this service by hostname (alias). After upgrade to 1.7.1 I have to run my development environment with
docker-compose up -d dev && docker-compose logs -f dev
But there are two points:

• service doesn't stop on Ctrl+C in termnal (I have to run down or kill commands)
• docker-compose logs doesn't support fully tty features (e.x. some kind of colors)

And therefore I have a question why feature "network aliases" have been cut from run interface?
I suppose this feature needs similar solution as in the case with docker-compose run --service-ports at least

[tylerbrandt]

@aanand I tried to explain in the original description, but basically: I have an app that uses the selenium/standalone-chrome image to run tests against a custom e2e service (it happens to both run the app with the tests and run the tests themselves, using webdriverio for orchestration).

The hostname of the e2e service needs to be resolvable by the selenium service (so it can navigate to the app), and the hostname of the selenium service needs to be resolvable by the e2e service (so webdriverio knows which selenium server to use).

It used to be that I could start the e2e service using docker-compose run e2e which would spin up the e2e service, the selenium service (which is a linked dependency), and run the test suite, allowing the selenium service to drive the browser against the e2e hostname (using the network alias):

e2e:
  links:
    - selenium
  networks:
    default:
      aliases:
        - e2e

Now I have to do the docker-compose up workaround, which is fine, but not ideal because it makes what used to be one line of shell code using docker-compose run into several lines of code with up/run/down, not to mention that the Docker images are not backwards compatible because the Dockerfile CMD command needs to be modified to make the up command work.

[aanand]

OK. Both of these sound like use cases outside of what docker-compose run is designed for, but they could also be potentially addressed by adding a flag to run for including the aliases defined on the service. Perhaps --service-aliases.

It definitely shouldn't be the default behaviour of run to add the aliases, because then you'd get situations where one-off containers are accessible via the same hostname as long-running containers for the same service, and part of the load-balancing group - which is almost never what you want.

[xfxf]

Hi, I'm currently hitting this issue as well, and am concerned this ticket (and similar ones) have been open for some time - is this something that is going to be altered?

I run my app's test suite with 'run' as I need it to be interactive sometimes (debugging), but 'docker-compose run' means that the hostname of the test suite's web server is never populated back to the selenium container, so selenium tests that try to talk back to my web container fail.

Using 'docker-compose up' results in a non-interactive local test suite which is a workflow showstopper if needing to debug.

[kyale]

I am experiencing the same issue on 1.8.1. @aanand I do not agree with your opinion about "run" not being designed for these use cases. In my opinion one off commands should be able to access related services in the same way as "up" otherwise it would make no sense to take depends_on and links into account when executing "run" either.

[mcronce]

I am having the same issue. My use case is similar to @tylerbrandt's. I have an "executor" service built from an image we control, which executes tests against a Selenium server (in the same compose file, service named "selenium" - using docker.io/selenium/standalone-firefox for the image).

Network topology requires me to spin up a tunnel to the system being tested. I'm doing this in the custom image with a simple SSH tunnel, but the Selenium container is unable to connect to the executor container by hostname. Extending Selenium's image to add the tunnel functionality to it is unpalatable, and the workaround seems to add needless complexity.

A flag to docker-compose --run to enable this would be very helpful.

[CryptidVulpes]

I'm having this same issue as well. With almost exactly the same use case as @tylerbrandt

I need to run my test containers with docker-compose run because I want the proper exit code to my shell.

[CryptidVulpes]

Managed to discover a workaround for this problem!

You can manually set a static IP address to your test container and add an extra_hosts entry pointing to that static IP. Settings like these will work:

version: '2'

services:
  service:
    networks:
      - test
    extra_hosts:
      - "test:172.16.0.42"

  test:
    depends_on:
      - service
    networks:
      test:
        ipv4_address: 172.16.0.42

networks:
  test:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 172.16.0.0/24
        gateway: 172.16.0.1

With a configuration similar to that docker-compose.yml, I am able to run docker-compose run --rm test and my service container can connect to the shortname test.

[gzur]

I've been googling myself gray-haired trying to figure this out. I must say that I find this to be pretty awful. You would expect to be able to runa container so that it is identical to up - but that is obviously not the case.

I run a stack comprised of a lot of microservices all running on docker, and I like to step into the containers when I am developing, to save me the trouble of having to generate appropriate payloads and shooting them at the container manually with curl or something - I had implemented my own DNS for doing this previously, but docker-compose was supposed to do away with my need of it. Or so I thought until I foolishly expected this to just work.

It sort of breaks the networking contract made by the docker-compose.yml file, that is, services should be available by name as long as they share a network.

For reference, I am on ubuntu 14.04, running docker-compose version 1.9.0, build 2585387

[xfxf]

I've since figured out a workaround for this, by using a named container (which populates using 'run' properly). This assumes a 'tests' container (which runs tests), which has a link to selenium:

docker-compose stop tests; docker-compose rm -f tests
docker-compose run --rm --name tests tests

The 'stop/rm' parts are there to ensure no old containers stick around, as container names must be unique.

[bitbier]

I would also like to express frustration with this feature going away as well. We use docker-compose to be able to stand up, test and teardown test environments in CI. We exclusively use docker-compose run (along with some docker-compose magic such as --name and -p to avoid clashing when running the same test on the CI host) to do this and rely on the alias being set correctly within its network. Now we have to build workarounds for this. Glad I found this issue; otherwise, I would have gone crazy trying to figure out why this isn't working anymore.

[coen-hyde]

Yeah this is pretty rough. It appears run was designed to operate this way. But as a user of docker compose this is breaking expected behavior for me. Why can service A talk with service B but service B can not talk with service A? Both these services are part of a network so they can physically talk with each other, it's just the alias's that are missing. Which makes this feel like a bug even if run was designed this way.

[mdmoss]

I found this choice very counter-intuitive. I also didn't see mention of it in the documentation (referring to https://docs.docker.com/compose/reference/run/). Even with #4811 I feel like many others will have the same issue.

[Vanuan]

In docker-compose 1.12 it works as expected for me.

[artemkaint]

it hasn't been working properly yet for me (ver 1.16)

[Vanuan]

So maybe it's a regression? Could you try 1.12.0?
BTW, I'm using 17.04.0-ce
Here's a test case:
https://github.com/docker/compose/issues/5147#issuecomment-326461774

[tagirb]

Not working in 1.16.1.

Looking through this issue and some others I often saw an opinion that docker-compose run is for one-off tasks, therefore, it's not a first-class service. While this is a righteous view, I don't see docker-compose up as an alternative to docker-compose run when running multiple versions of a particular service with varying arguments. docker-compose up doesn't provide a way of overriding command or am I wrong here?

It also clashes somehow with the way Docker implements this. In pure Docker you'd use docker run for whatever tasks you have, be it long-running services or one-time tasks. Why have it differently in docker-compose?

[Vanuan]

@tagirb Are you saying that the following example doesn't work:
https://github.com/docker/compose/issues/5147#issuecomment-326461774

[tagirb]

@Vanuan That particular example works, but as soon as I remove the depends_on statement it doesn't work anymore regardless of whether --name is defined:

> cat docker-compose.yml
version: '2'
networks:
    internal:
services:
    slave:
        command: "sleep 999"
        image: busybox
        networks:
            internal:
               aliases:
                 - slave1
    master:
        command: sh -c "sleep 1; ping slave1"
        image: busybox
        networks:
            internal:
               aliases:
                 - master1
> docker-compose run --rm -d slave && docker-compose run --rm master; docker-compose down
Creating network "test_internal" with the default driver
test_slave_run_1
ping: bad address 'slave1'
Stopping test_slave_run_1 ... done
Removing test_slave_run_1 ... done
Removing network test_internal
> docker-compose run --rm -d --name slave slave && docker-compose run --rm master; docker-compose down
Creating network "test_internal" with the default driver
slave
ping: bad address 'slave1'
Stopping slave ... done
Removing slave ... done
Removing network test_internal

Actually this whole DNS magic is not described in the documentation in any way. It's a bit tiring to try out various edge cases to verify the docker-compose behaviour, but looking through the source code to find out how it works is even more tedious, at least for me.

[Vanuan]

@tagirb
Why do you need multiple docker-compose run? Wouldn't docker-compose up work?

[tagirb]

@Vanuan Just as an example. In the end I refactored my use case to resolve the issue.