Compose: Set DOCKER_HOST_IP environment variable automatically

[msabramo]

My use case is that I'm running docker-compose on a Mac so when I do docker-compose port, by default it returns 0.0.0.0 as the IP address, which is not useful if you're on a Mac and trying to contact a service on the Docker host.

Example:

I have a docker-compose.yml that looks like this:

version: '2'
services:

  anonweb:
    ...
    ports:
      - ":8000"
    ...

Now I do:

[marca@marca-mac2 smdevstack]$ docker-compose up -d
Recreating smdevstack_anonweb_1

[marca@marca-mac2 smdevstack]$ docker-compose ps
        Name                      Command               State            Ports
---------------------------------------------------------------------------------------
smdevstack_anonweb_1   gunicorn --paste=/appinifi ...   Up      0.0.0.0:32777->8000/tcp

[marca@marca-mac2 smdevstack]$ docker-compose port anonweb 8000
0.0.0.0:32777

[marca@marca-mac2 smdevstack]$ http $(docker-compose port anonweb 8000)/status/pid

http: error: ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=32777): Max retries exceeded with url: /status/pid (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x10363fa50>: Failed to establish a new connection: [Errno 61] Connection refused',)) while doing GET request to URL: http://0.0.0.0:32777/status/pid

Connecting to 0.0.0.0:32777 of course from the Mac is not useful.

So instead I'd like to use make the container bind to the real IP address of the Docker machine. I change my docker-compose.yml to the following:

version: '2'
services:

  anonweb:
  ...
    ports:
      - "${DOCKER_HOST_IP}::8000"
    ...

This in itself accomplishes nothing because DOCKER_HOST_IP is not set:

[marca@marca-mac2 smdevstack]$ docker-compose up -d
WARNING: The DOCKER_HOST_IP variable is not set. Defaulting to a blank string.
Starting smdevstack_anonweb_1

[marca@marca-mac2 smdevstack]$ docker-compose ps
WARNING: The DOCKER_HOST_IP variable is not set. Defaulting to a blank string.
        Name                      Command               State            Ports
---------------------------------------------------------------------------------------
smdevstack_anonweb_1   gunicorn --paste=/appinifi ...   Up      0.0.0.0:32779->8000/tcp

[marca@marca-mac2 smdevstack]$ docker-compose port anonweb 8000
WARNING: The DOCKER_HOST_IP variable is not set. Defaulting to a blank string.
0.0.0.0:32779

[marca@marca-mac2 smdevstack]$ http $(docker-compose port anonweb 8000)/status/pid
WARNING: The DOCKER_HOST_IP variable is not set. Defaulting to a blank string.

http: error: ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=32779): Max retries exceeded with url: /status/pid (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x103789a50>: Failed to establish a new connection: [Errno 61] Connection refused',)) while doing GET request to URL: http://0.0.0.0:32779/status/pid

It works great if I arrange for DOCKER_HOST_IP to be set.

One nice way to do this is with direnv. Assuming I have direnv installed, I can do this:

[marca@marca-mac2 smdevstack]$ cat > .envrc
export DOCKER_HOST_IP=$(docker-machine ip)
direnv: error .envrc is blocked. Run `direnv allow` to approve its content.

[marca@marca-mac2 smdevstack]$ direnv allow
direnv: loading .envrc
direnv: export +DOCKER_HOST_IP

Now I get this:

[marca@marca-mac2 smdevstack]$ docker-compose up -d
Recreating smdevstack_anonweb_1

[marca@marca-mac2 smdevstack]$ docker-compose ps
        Name                      Command               State               Ports
----------------------------------------------------------------------------------------------
smdevstack_anonweb_1   gunicorn --paste=/appinifi ...   Up      192.168.99.101:32774->8000/tcp

[marca@marca-mac2 smdevstack]$ docker-compose port anonweb 8000
192.168.99.101:32774

[marca@marca-mac2 smdevstack]$ http $(docker-compose port anonweb 8000)/status/pid
HTTP/1.1 200 OK
Connection: close
Content-Length: 112
Content-Type: application/json; charset=UTF-8
Date: Sat, 13 Feb 2016 20:46:16 GMT
SM-Request-ID: 094fe0a4-605c-457f-a0b1-8a97764abcaa
Server: gunicorn/19.4.5

{
    "host": "7c8b1a192f27",
    "pid": 9,
    "reason": "/appenv/enabled.txt does not exist",
    "status": "SERVICE_DISABLED"
}

Beautiful!

But now if I want my little docker-compose project to be easily usable by others, I have to tell them to install direnv. Also my solution assumes folks are using docker-machine and maybe they aren't. Maybe they are using straight Docker without Docker Machine and so that docker-machine ip command won't work.

I am asking if docker-compose could simply set that environment variable automatically, perhaps by taking the existing DOCKER_HOST variable and massaging it into a simple IP address.

Then it just works out of the box without requiring direnv.

Reasonable?

Cc: @sudarkoff

[sudarkoff]

@msabramo Not exactly what you're asking for, but could you perhaps do something like the following?

version: '2'
services:

  anonweb:
  ...
    ports:
      - "${${DOCKER_HOST#tcp://}%:[0-9]*}::8000"
    ...

[msabramo]

@sudarkoff Thanks! That would be pretty cool. Unfortunately, docker-compose doesn't support fancy interpolations.

$ docker-compose stop
ERROR: Invalid interpolation format for "ports" option in service "anonweb": "${${DOCKER_HOST#tcp://}%:[0-9]*}::8000"

[sudarkoff]

Gotcha, makes sense.

Then I'd vote for just adding another line to the docker-machine env output:

...
export DOCKER_MACHINE_IP=<machine_ip_address>
...

[msabramo]

Yeah I came to the same conclusion. This should be addressed in docker-machine; not docker-compose

[msabramo]

See https://github.com/docker/machine/pull/3057

$ bin/docker-machine env | grep IP
export DOCKER_HOST_IP="192.168.99.101"

[msabramo]

Reopened this because https://github.com/docker/machine/pull/3057 was closed and so wondering if it might make sense to do something in docker-compose since it was deemed to be out of scope for docker-machine.

[dnephin]

It still feels like this is more appropriate for docker-machine. I'll see if we can get some more discussion going about it.

[msabramo]

@dnephin: I agree that docker-machine seems like the better place to solve this. Thanks for chiming in!

[abhinavkorpal]

@msabramo how should i set export DOCKER_HOST_IP in default env. I am not able to do

[msabramo]

What I've been doing lately is using direnv with the following line in my .envrc:

export DOCKER_IP=$(echo ${DOCKER_HOST:-tcp://127.0.0.1:2376} | cut -d/ -f3 | cut -d: -f1)

Could also use something like:

export DOCKER_HOST_IP=$(docker-machine ip ${DOCKER_MACHINE_NAME:-default})

[mattiLeBlanc]

I find it weird that I cannot do something like
command: export HOST_IP=$(host_ip) in my compose file.
I have an alias that returns the hostIp which I want as an environment var. Since my IP can change, having it in the docker-compose allows to to run a new build before I do an up. That way I don't have to first reset the ENV variable in my host and then run the build and up. Is this something that is possible or on the road map?