Compose: Inconsistent value of $PWD variable

[nomysz]

Description of the issue

$PWD variable sometimes returnes path with /host_mnt prefix and sometimes without prefix.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020

Output of docker version

Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 16:58:31 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker-compose config

networks:
  default:
    driver: bridge
    name: my_network
services:
  my_service:
    command: sleep 5
    image: alpine
    volumes:
    - my_code:/tmp/any:rw
version: '3.7'
volumes:
  my_code:
    driver: local
    driver_opts:
      device: /Users/me/Projects/project
      o: bind
      type: none

Steps to reproduce the issue

1. Run this config with docker-compose up

version: '3.7'

volumes:
    my_code:
        driver: local
        driver_opts:
            type: none
            device: $PWD
            o: bind


services:
    my_service:
        image: alpine
        command: sleep 5
        volumes:
            - my_code:/tmp/any


networks:
    default:
        driver: bridge
        name: my_network

1. Wait 5 seconds or stop Ctrl+C
2. Run again

Observed result

Error below

Expected result

Running instance

Stacktrace / full error message

ERROR: Configuration for volume my_code specifies "device" driver_opt /Users/me/Projects/project, but a volume with the same name uses a different "device" driver_opt (/host_mnt/Users/me/Projects/project). If you wish to use the new configuration, please remove the existing volume "my_code" first:

Additional information

Docker for Mac, MacOS Catalina 10.15.6

[thaJeztah]

I think compose needs to pick the $PWD up from the shell; are you sure the variable is set? There could be cases where it's not (also see https://github.com/docker/compose/issues/5003#issuecomment-314209681)

That said; from the compose file, I _think_ your intent is to make the bind-mount's source relative to the compose file's path (so to work similar to the shorthand volumes: - <local-path>:<container-path>)?

Perhaps we could use a special variable / substitution that could be used that would always point to the full path of the compose file.

/cc @ndeloof @ulyssessouza

[nomysz]

@thaJeztah

I think compose needs to pick the $PWD up from the shell; are you sure the variable is set?

It is set.

The point is not that it is not set. But that it returns different values. Once with /host_mnt prefix. Once without.

My intention is to have shared volume between two containers (while also being directory on my local machine) - while I was testing I got same behaviour with simpler config - that's why I'm posting simpler config with this bug. Real one would be with two services sharing same volume. That worked for me until recently $PWD broke and I get mentioned error message (same config - just docker stack update). I was updating docker with regular update tool for Docker for Mac.

[thaJeztah]

Hmmm.. possibly the /host_mnt is something that's set in the API proxy that's used on Docker Desktop to translate paths. @djs55 @StefanScherer do you know?

[nomysz]

I have found that comment of @djs55 here https://github.com/docker/for-mac/issues/4859 but like you see on attached screenshots it is not the case. With my versions it should work (in theory).

[StefanScherer]

At least I can reproduce the issue. I also have DD 2.4.0.0 Stable installed with gRPC FUSE setting active. It works when I switched back to osxfs setting.
Here's the output with gRPC FUSE:

$ docker-compose up
Creating network "my_network" with driver "bridge"
Creating volume "compose_my_code" with local driver
Pulling my_service (alpine:)...
latest: Pulling from library/alpine
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest
Creating compose_my_service_1 ... done
Attaching to compose_my_service_1
compose_my_service_1 exited with code 0
~/code/compose
$ docker-compose up
ERROR: Configuration for volume my_code specifies "device" driver_opt /Users/stefanscherer/code/compose, but a volume with the same name uses a different "device" driver_opt (/host_mnt/Users/stefanscherer/code/compose). If you wish to use the new configuration, please remove the existing volume "compose_my_code" first:
$ docker volume rm compose_my_code

The volume is created with /host_mnt prefix on first docker-compose up.

$ docker volume inspect compose_my_code
[
    {
        "CreatedAt": "2020-10-05T15:57:12Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "compose",
            "com.docker.compose.version": "1.27.4",
            "com.docker.compose.volume": "my_code"
        },
        "Mountpoint": "/var/lib/docker/volumes/compose_my_code/_data",
        "Name": "compose_my_code",
        "Options": {
            "device": "/host_mnt/Users/stefanscherer/code/compose",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

On the second call docker-compose seems to compare the existing value (with the prefix) with the new calculated $PWD and complains that these differ.

[nomysz]

Also looks like it's not only $PWD issue. I'm getting same error message for following config (double up to get error message):

version: '3.7'

volumes:
    my_code:
        driver: local
        driver_opts:
            type: none
            device: .
            o: bind

services:
    my_service:
        image: alpine
        command: sleep 5
        volumes:
            - my_code:/tmp/any

networks:
    default:
        driver: bridge
        name: my_network

[thaJeztah]

I see what's happening. It looks like the adjustments that the Docker Desktop API makes are not symmetrical; when _creating_ the volume, the API prepends /host_mnt to /Users/stefanscherer/code/compose, but when _inspecting_ the volume, it doesn't _remove_ the /host_mnt prefix. (I recall we had some discussions about similar changes that the API made for IP-addresses)

Compose therefore (rightfully) complains that the options do not match for the volume. Ideally such a check would be done by the daemon (I created a tracking issue for that https://github.com/moby/moby/issues/16068; I _think_ there's some checks already, but those may not yet cover volume _options_)

I think the first fix would be for the Docker Desktop API to strip the /host_mnt prefix when inspecting volumes

[nomysz]

@thaJeztah so is this a right place for bug report or should I create it in another repository?

[StefanScherer]

It's a known bug listed here https://docs.docker.com/docker-for-mac/release-notes/#known-issues for version 2.4.0.0
See also docker/for-mac#4859 A fix will land in next Edge release first.