Compose: docker-compose build ignoring .dockerignore?

[devinrsmith]

I've got a full example repo here:

https://github.com/devinrsmith/docker-compose-build-test

If I'm using docker-compose build incorrectly, please let me know!

[neciu]

You have problem with file format.

In your .dockerignore change subdir/ to subdir.

[devinrsmith]

Why does the .dockerignore work for docker build -t subdir ../ but not docker-compose build?

[fChristenson]

I am having the same issue running docker-compose 1.2.0.
Running vanilla docker build pulls in the ignore file and builds the image, np there.
However docker-compose does not use the ignore file, with or without the trailing slash.

[aanand]

Looks like a docker-py issue; a fix is being worked on in https://github.com/docker/docker-py/pull/604. Marking this as a bug to remind us to update our docker-py version once the fix is out.

[thaJeztah]

@aanand looks like https://github.com/docker/docker-py/pull/604 stalled and was removed from the milestone there?

[aanand]

@thaJeztah I've created https://github.com/docker/docker-py/pull/721 to continue the work.

[twillouer]

Hi,

I'm facing one strange problem with .dockerignore:

*.sh

!awesome/script.sh

with docker build all is fine, but docker-compose didn't see "awesome/script.sh"

Am I in the same issue ?

[aanand]

@twillouer that's a known bug, fixed by https://github.com/docker/docker-py/pull/721 - it'll be fixed on the Compose side in 1.5.0.

[twillouer]

thanks !

[etc0de]

This still seems to be broken even on latest git master on some machines. It doesn't work for me on a machine given this configuration:

Debian 8.2 with kernel 3.16.0 with lxc-docker version 1.7.1, build 786b29d
docker-compose is git master, HEAD is at dabf1e8657674014a5bc89f99edbf2fe0629bb71

The .dockerignore works fine for docker build (which runs instantly), but not for docker-compose (which uploads stuff it shouldn't upload for various minutes).

Also see #2100

[cruelbob]

I have same problem. docker-compose ignores ".dockerignore". "docker build" works fine.
System: windows 10

[xrstf]

I'm experiencing the same problem using the Docker Toolbox that was released yesterday:

• Windows 7 x64
• Docker version 1.9.0, build 76d6bc9
• Docker-Compose 1.5.0

[dnephin]

It sounds like we're still having problems with .dockerignore. If you're experiencing these problems with compose 1.5.0, please include a sample .dockerignore and directory structure so we can reproduce the error.

[xrstf]

Sorry for not provide a smaller usecase, but this is how I noticed the problem:

I have this package.json for my project:

{
  "dependencies": {
    "grunt-contrib-uglify": "^0.9.2"
  }
}

Plus I have this docker-compose.yml:

web:
  build: .

The Dockerfile looks like this:

FROM node:0.12

My .dockerignore contains one line (I tried adding a trailing slash, but the problem persisted):

node_modules

Now I can do

> npm install
  (...snip...)
> docker build .
Sending build context to Docker daemon  5.12 kB

Awesome, it works, only 5 KiB are sent (node_modules is roughly 10 MiB in total). With docker-compose in Q:\sites\test:

> npm install
  (...snip...)
> docker-compose up
Building web
Traceback (most recent call last):
  File "D:\opt\python\Scripts\docker-compose-script.py", line 9, in <module>
    load_entry_point('docker-compose==1.5.0dev', 'console_scripts', 'docker-compose')()
  File "D:\opt\python\lib\site-packages\compose\cli\main.py", line 54, in main
    command.sys_dispatch()
  File "D:\opt\python\lib\site-packages\compose\cli\docopt_command.py", line 23, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "D:\opt\python\lib\site-packages\compose\cli\docopt_command.py", line 26, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "D:\opt\python\lib\site-packages\compose\cli\main.py", line 170, in perform_command
    handler(project, command_options)
  File "D:\opt\python\lib\site-packages\compose\cli\main.py", line 583, in up
    detached=detached
  File "D:\opt\python\lib\site-packages\compose\project.py", line 313, in up
    detached=detached
  File "D:\opt\python\lib\site-packages\compose\service.py", line 404, in execute_convergence_plan
    container = self.create_container(do_build=do_build)
  File "D:\opt\python\lib\site-packages\compose\service.py", line 303, in create_container
    self.ensure_image_exists(do_build=do_build)
  File "D:\opt\python\lib\site-packages\compose\service.py", line 326, in ensure_image_exists
    self.build()
  File "D:\opt\python\lib\site-packages\compose\service.py", line 718, in build
    dockerfile=self.options.get('dockerfile', None),
  File "D:\opt\python\lib\site-packages\docker\api\build.py", line 48, in build
    context = utils.tar(path, exclude=exclude, dockerfile=dockerfile)
  File "D:\opt\python\lib\site-packages\docker\utils\utils.py", line 85, in tar
    t.add(os.path.join(root, path), arcname=path, recursive=False)
  File "D:\opt\python\lib\tarfile.py", line 1998, in add
    tarinfo = self.gettarinfo(name, arcname)
  File "D:\opt\python\lib\tarfile.py", line 1870, in gettarinfo
    statres = os.lstat(name)
WindowsError: [Error 3] Das System kann den angegebenen Pfad nicht finden: 'Q:\\sites\\test\\node_modules\\grunt-contrib-uglify\\node_modules\\maxmin\\node_modules\\pretty-bytes\\node_modules\\meow\\node_modules\\normalize-package-data\\node_modules\\validate-npm-package-license\\node_modules\\spdx-correct\\node_modules\\spdx-license-ids\\spdx-license-ids.json'

(I am using 1.5.0dev on this machine, but I got the exact same problem on my other machine with 1.5.0 final).

With node, I often encounter problems with too long paths and stuff on Windows, but the fact that Docker-Compose is attempting to tar up the node_modules is the indicator that the .dockerignore is ignored.

[etc0de]

FWIW, I am still having issues with this on Linux (Ubuntu) with docker-compose 1.5.0dev, so it's probably not isolated to Windows. However, it's on a production machine so I cannot easily put together a minimal test case (it works fine on my testing machine).

[Quantumplation]

I am also having this issue.

Directory structure:

docker-compose.yml
web
  + .dockerignore
  + Dockerfile
  + node_modules
    + ...

docker-compose.yaml

web:
  build: web
  tty: true
  ports:
    - 8081:5000

Dockerfile

FROM microsoft/aspnet

# Curl, node, npm, bower, grunt
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup | bash -
RUN apt-get install -y nodejs
RUN npm install -g bower
RUN npm install -g grunt-bower-cli
RUN npm install -g grunt
RUN npm install -g grunt-cli
RUN npm install -g grunt-bower-task

# Copy the project.json file first, then do a restore.
# This ensures that as long as project.json doesn't change, it will avoid
# doing a package restore
COPY project.json /app/
COPY bower.json /app/
COPY gruntfile.js /app/
COPY package.json /app/
WORKDIR /app

RUN ["dnu", "restore"]

# Then copy the rest of the files
COPY . /app

# Expose the port that the website listens on
EXPOSE 5000
# And start the website
ENTRYPOINT ["dnx", "-p", "project.json", "web"]

.dockerignore

node_modules

Result:

Pi@Ricci MINGW64 /d/proj/Repro
$ docker-compose build
Building web
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "C:\projects\compose\compose\cli\main.py", line 54, in main
  File "C:\projects\compose\compose\cli\docopt_command.py", line 23, in sys_dispatch
  File "C:\projects\compose\compose\cli\docopt_command.py", line 26, in dispatch
  File "C:\projects\compose\compose\cli\main.py", line 171, in perform_command
  File "C:\projects\compose\compose\cli\main.py", line 192, in build
  File "C:\projects\compose\compose\project.py", line 235, in build
  File "C:\projects\compose\compose\service.py", line 683, in build
  File "c:\projects\compose\venv\lib\site-packages\docker\api\build.py", line 48, in build
  File "c:\projects\compose\venv\lib\site-packages\docker\utils\utils.py", line 85, in tar
  File "c:\python27-x64\Lib\tarfile.py", line 2000, in add
  File "c:\python27-x64\Lib\tarfile.py", line 1872, in gettarinfo
WindowsError: [Error 3] The system cannot find the path specified: 'D:\\proj\\Repro\\web\\node_modules\\babel-preset-react\\node_modules\\babel-plugin-transform-react-jsx\\node_modules\\babel-helper-builder-react-jsx\\node_modules\\babel-types\\node_modules\\babel-traverse\\node_modules\\babel-code-frame\\node_modules\\js-tokens\\changelog.md'
docker-compose returned -1

[keatz55]

+1

[keatz55]

Did anybody find a workaround? I am trying to do everything in only the Dockerfile, but haven't had any luck yet.

[esc-rtn]

I'm getting this issue as well, could it be related to the node version everyone is running? Via this issue: https://github.com/npm/npm/issues/3697 .. I imagine upgrading node (more specifically to npm 3+) would fix it, but that's more of a nuclear option, and .dockerignore should still work.

[etc0de]

Again despite this being apparently ignored, I am seeing this on a Linux machine as well.

[esc-rtn]

I fixed this issue by removing an unnecessary volume reference in my docker-compose file. Compose ignores .gitignore for volumes.

I also had to upgrade my app to use npm 3+ .. but that may only apply to windows users.

[thaJeztah]

@esc-rtn the .dockerignore file only specifies what files should not be sent to the daemon during _build_. If you're using a bind-mounted volume during "run", it will simply mount all files in that location as a volume (all files that are present on the host).

[stekershaw]

I'm seeing similar too, on Windows: docker build works as expected, docker-compose not.

I've put a small example on GitHub, with notes in the README.md: https://github.com/stekershaw/docker-compose-ignore

[dnephin]

Another round of fixes for .dockerignore went into docker-py after Compsoe 1.5.2 was released. Could you try out the Compose 1.6.0 RC2 release to see if it's fixed now?

[stekershaw]

Just tried with docker-compose version 1.6.0rc2, build a7636be.

I have three test cases in my repo and with 1.5.2 two of them failed. Now with 1.6.0rc2 only one fails, when I have a directory and a file in the directory excluded, like so:

$ cat .dockerignore
files/test_dir
!files/test_dir/should_be_here_maybe

[aanand]

I get identical behaviour in that case between docker build and docker-compose build.

.dockerignore:

files/test_dir
!files/test_dir/should_be_here_maybe

docker-compose.yml:

test:
  build: .

Dockerfile:

FROM busybox
COPY . /context
CMD ["find", "/context"]

$ docker version
Client:
 Version:      1.10.0-rc1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   677c593
 Built:        Fri Jan 15 18:17:17 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.10.0-rc1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   677c593
 Built:        Fri Jan 15 18:17:17 2016
 OS/Arch:      linux/amd64

$ docker-compose version
docker-compose version 1.6.0rc2, build 695c692
docker-py version: 1.7.0-rc3
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

$ mkdir -p files/test_dir
$ touch files/test_dir/should_be_here_maybe

$ find .
.
./.dockerignore
./docker-compose.yml
./Dockerfile
./files
./files/test_dir
./files/test_dir/should_be_here_maybe

$ docker build --no-cache -t 1607-docker-build .
Sending build context to Docker daemon  5.12 kB
Step 1 : FROM busybox
 ---> 0cb40641836c
Step 2 : COPY . /context
 ---> 859e12600100
Removing intermediate container 9067b263098b
Step 3 : CMD find /context
 ---> Running in 1ddc0a573492
 ---> b1b3beacf5f2
Removing intermediate container 1ddc0a573492
Successfully built b1b3beacf5f2

$ docker run 1607-docker-build
/context
/context/docker-compose.yml
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here_maybe
/context/.dockerignore

$ docker-compose build --no-cache
Building test
Step 1 : FROM busybox
 ---> 0cb40641836c
Step 2 : COPY . /context
 ---> d86507051d6d
Removing intermediate container 0af2cbf69b17
Step 3 : CMD find /context
 ---> Running in 8533dae3af74
 ---> 1f736ecb2b38
Removing intermediate container 8533dae3af74
Successfully built 1f736ecb2b38

$ docker-compose run test
/context
/context/docker-compose.yml
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here_maybe
/context/.dockerignore

[stekershaw]

Hi @aanand, thanks for you response, I get the same result as you following your steps.

However, the test that I ran previously also had another file present in files/test_dir. If I add another file to files/test_dir then I see that it is not present (as I would expect) with docker build but it is present with docker-compose:

$ touch files/test_dir/should_not_be_here

$ docker build --no-cache -t 1607-docker-build .
Sending build context to Docker daemon  5.12 kB
Step 1 : FROM busybox
 ---> b175bcb79023
Step 2 : COPY . /context
 ---> a23d9645c21c
Removing intermediate container 8eb2bb23c4db
Step 3 : CMD find /context
 ---> Running in d9fef847acd8
 ---> e52ae84b1250
Removing intermediate container d9fef847acd8
Successfully built e52ae84b1250
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.

$ docker run --rm 1607-docker-build
/context
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here_maybe
/context/docker-compose.yml
/context/.dockerignore

$ docker-compose build --no-cache
Building test
Step 1 : FROM busybox
 ---> b175bcb79023
Step 2 : COPY . /context
 ---> 9df0cf4bfb69
Removing intermediate container 7820f982d59e
Step 3 : CMD find /context
 ---> Running in 06e1a0b89a45
 ---> 2c922dbc66d9
Removing intermediate container 06e1a0b89a45
Successfully built 2c922dbc66d9

$ docker-compose run test
ERROR: Interactive mode is not yet supported on Windows.
Please pass the -d flag when using `docker-compose run`.

$ docker-compose run -d test
dcitest_test_run_2

$ docker logs dcitest_test_run_2
/context
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_not_be_here
/context/files/test_dir/should_be_here_maybe
/context/docker-compose.yml
/context/.dockerignore

This is Windows (using git bash as my shell), checked with both with Docker compose 1.5.2 and 1.6.0rc2, docker 1.9.1.

Running the same on Ubuntu 14.04 with docker-compose 1.5.2 and docker 1.9.1 is fine:

# cat .dockerignore
files/test_dir
!files/test_dir/should_be_here_maybe

# find .
.
./.dockerignore
./docker-compose.yml
./files
./files/test_dir
./files/test_dir/should_not_be_here
./files/test_dir/should_be_here_maybe
./Dockerfile

# docker-compose build --no-cache
Building test
Step 1 : FROM busybox
 ---> b175bcb79023
Step 2 : COPY . /context
 ---> c533a0768d5e
Removing intermediate container 0c057fe8eb82
Step 3 : CMD find /context
 ---> Running in e8a0cf1f58d8
 ---> 175777486a25
Removing intermediate container e8a0cf1f58d8
Successfully built 175777486a25

# docker-compose run test
/context
/context/docker-compose.yml
/context/.dockerignore
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here_maybe
/context/Dockerfile

[bkcummins]

Still dealing with this most of the afternoon.

Docker version 1.10.1, build 9e83765
docker-compose version 1.6.0, build d99cad6

#docker-compose.yml
test:
  build: ./cdn

#Dockerfile
FROM busybox
COPY ["content/","/test/"]
RUN find /test/ -maxdepth 1

#.dockerignore
**/.DS_Store
**/.git
**/.bowerrc
**/bower_components
**/node_modules
**/npm-debug.log

Output is as expected with docker build .

docker-compose build --no-cache test copied over everything

[shin-]

2992 has some more .dockerignore fixes that should be included in 1.6.1

[stekershaw]

Thanks @shin- , unfortunately I still see different behaviour with docker-compose 1.6.2 from Docker Toolbox for Windows:

$ docker-compose --version
docker-compose version 1.6.2, build e80fc83

$ find .
.
./.dockerignore
./docker-compose.yml
./Dockerfile
./files
./files/test_dir
./files/test_dir/should_be_here_maybe
./files/test_dir/should_not_be_here

$ cat .dockerignore
files/test_dir
!files/test_dir/should_be_here_maybe

$ docker-compose build --no-cache
Building test
Step 1 : FROM busybox
latest: Pulling from library/busybox
f810322bba2c: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:97473e34e311e6c1b3f61f2a721d038d1e5eef17d98d1353a513007cf46ca6bd
Status: Downloaded newer image for busybox:latest
 ---> 3240943c9ea3
Step 2 : COPY . /context
 ---> 85a65d7f861c
Removing intermediate container 386d3103d8ab
Step 3 : CMD find /context
 ---> Running in e5e29b5746c4
 ---> 2c2d57a899ea
Removing intermediate container e5e29b5746c4
Successfully built 2c2d57a899ea

$ docker-compose run -d test
dcitest_test_run_1

$ docker logs dcitest_test_run_1
/context
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here_maybe
/context/files/test_dir/should_not_be_here
/context/.dockerignore
/context/docker-compose.yml

[shin-]

That's very odd. It definitely works for me.

$ docker-compose build --no-cache
Building web
Step 1 : FROM busybox
 ---> 3240943c9ea3
Step 2 : COPY . /context
 ---> 3619871879ad
Removing intermediate container 08432f688579
Step 3 : CMD find /context
 ---> Running in 5bbcf987c9e7
 ---> cf2bff2c1416
Removing intermediate container 5bbcf987c9e7
Successfully built cf2bff2c1416

$ docker-compose run -d web
Creating network "testdockerignore_default" with the default driver
testdockerignore_web_run_1

$ docker logs testdockerignore_web_run_1 
/context
/context/Dockerfile
/context/files
/context/files/test_dir
/context/files/test_dir/should_be_here
/context/docker-compose.yml
/context/.dockerignore

$ find .
.
./Dockerfile
./files
./files/test_dir
./files/test_dir/should_not_be_here
./files/test_dir/should_be_here
./docker-compose.yml
./.dockerignore

$ cat .dockerignore files/test_dir
!files/test_dir/should_be_here

What's the output of pip show docker-py in your environment?

[digizen]

I believe I also have the same issue.

 Docker build .

works for me. However, enclosing the same build in a docker-compose.yml file breaks my build. In my .dockerignore file, I'm excluding the node_modules directory for my node build. This is the relevant section from my docker-compose.yml:

web:
  build: ./app
  ports:
    - "8080:8080"
  links:
    - mongodb

I believe I'm running the most recent version of docker-compose:

>docker-compose version
docker-compose version 1.6.2, build e80fc83
docker-py version: 1.7.2
CPython version: 2.7.11
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015

I'm running Windows 10, x64.

[shin-]

WIndows-specific issue, maybe with path separators?

[digizen]

Don't know how to tell whether this is related to path separators.

Let me know if I can assist with testing or otherwise. Right now, I have a batch file that moves the "ignored" folders out of the project directory prior to the build, obviously ugly.

Thanks for your help.

[digizen]

For completeness' sake, this is my .dockerignore file:

**/node_modules

[digizen]

Aha! It's the glob pattern that's causing the problem. removing the "**/" fixed the issue for me.

[digizen]

While I have a workaround for my specific issue, the fact remains that there is a bug. At the very least the glob pattern **/ doesn't get properly parsed with docker-compose, but works fine with the regular docker build command. Other glob patterns may or may not work.

[etc0de]

@bfirsh and everyone else who might care:

I have an absolutely simple .dockerignore which looks like this:

livedata
readonly-data

The .dockerignore doesn't work either (those two folders in the build context folder get uploaded to the daemon and it takes _forever_) and I'm on Linux. I updated just to current 1.6.2, no change. Unless I'm seeing a different problem with the same symptom, let me reiterate _again_ that this doesn't appear to be windows-specific problem..

[nicbarker]

+1, also broken for me on Linux - don't think it's a windows specific problem.

[aanand]

@JonasT @nicbarker How are you confirming that the folders named in .dockerignore are being uploaded, other than the fact it's taking a long time? If you have a way of checking the tarball, I'd like to try and use it to reproduce the problem locally. As it is, I've added some debug to docker-py and am still unable to reproduce:

$ pip list | grep docker
docker-compose (1.7.0.dev0, /Users/aanand/work/docker/compose)
docker-py (1.8.0rc2, /Users/aanand/work/docker/docker-py)

$ find .
.
./.dockerignore
./docker-compose.yml
./Dockerfile
./include.txt
./livedata
./livedata/exclude.txt
./readonly-data
./readonly-data/exclude.txt

$ cat .dockerignore
livedata
readonly-data

$ cat Dockerfile
FROM busybox
COPY . /data

$ docker-compose --verbose build --no-cache
<unrelated output>
docker.utils.utils.tar: Writing tar file to <open file '<fdopen>', mode 'w+b' at 0x1090f26f0>
docker.utils.utils.tar:   Adding .dockerignore
docker.utils.utils.tar:   Adding Dockerfile
docker.utils.utils.tar:   Adding docker-compose.yml
docker.utils.utils.tar:   Adding include.txt
docker.utils.utils.tar: Done
<unrelated output>

[etc0de]

@aanand it takes a REALLY long time before it starts, and when I move the Dockerfile to ./context and use "build: ./context/" with no other change it is instantly blazingly fast. Also back then when this problem first appeared I tried docker build manually with "docker build" and it was also instant.

[dnephin]

That doesn't really confirm the issue. If there were another large directory in the project root, moving the build to ./context would make it faster.

[etc0de]

@dnephin there are no other folders in there except for "context" itself which I aded as a workaround, and three tiny text files..

[aanand]

@JonasT Is it possible you have a large .git directory that's getting included?

[etc0de]

Edit: NVM, it seems I was actually stupid enough to miss a directory because I checked on the wrong rsync'ed copy >.> sorry guys..

I did actually have an issue with this a while back on an older docker-compose version where I investigated it more deeply, but maybe that specific instance I had was actually fixed with some of the recent dockerpy updates.

[digizen]

Back to this being a Windows problem? :)

[ulrichSchreiner]

hi,
i have linux, compose 1.6.2, docker 1.10.3 ... exact the same problem. docker uses .dockerignore, compose ignores it.

[aanand]

@ulrichSchreiner Can you provide steps to reproduce?

[ulrichSchreiner]

hi,

here is a gist (https://gist.github.com/ulrichSchreiner/566815cea26ce55b95207e7795cf6962). the .dockerignore contains **/node_modules the Dockerfile adds a file in a t1/a/b/c/node_modules subfolder.

if you build this with "docker build ..." you get an error because there is no file to add (it is correctly ignored because of the pattern in .dockerignore). you can see the output in the last file of the gist.

but if you build it with the given "docker-compose.yml" it is successfully build --> docker-compose ignores the .dockerignore file.

and this is really a pain when your node_modules directory is hundreds of megs ....

i use

docker-compose version 1.6.2, build 4d72027

[aanand]

OK, I can reproduce this. Looks like docker-py's probably got a bug with **/ rules.

[shin-]

IIRC, lack of support for the ** syntax is a known limitation of our .dockerignore implementation. See https://github.com/docker/docker-py/pull/721#issuecomment-135065043

[mllanes]

Same issue here. .dockerignore is ignored when used with docker-compose
docker-compose version 1.6.2, build 4d72027 OSX

[lattwood]

./.dockerignore
./Dockerfile (symlink to ./server/docker/cms/Dockerfile)
./server/docker/docker-compose.yml

Contents of .dockerignore

./.git
./.vagrant

.vagrant is 16GB, so this is a "Big Deal(tm)" for us.

[lattwood]

Sorry,

docker-machine version 0.7.0, build a650a40

docker-compose version 1.7.0, build 0d7bf73
docker-py version: 1.8.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

Client:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Tue Apr 26 23:44:17 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:20 2016
 OS/Arch:      linux/amd64

[aanand]

@lattwood Looks like a bug - I've fixed it in https://github.com/docker/docker-py/pull/1065. For now, you can remove the leading ./ from your patterns and it should work.

[lattwood]

Thanks!

[agilgur5]

Running on Ubuntu 14.04, my dockerignore is simply **/always_restart.txt. On a regular docker build (v1.11.2), I don't see the file in the container, but with Compose (v1.7.1), I do.

Edit1: Seems like docker-py doesn't test for the double asterisk notation and I'm guessing likely doesn't handle it either: https://github.com/docker/docker-py/blob/1.9.0-release/tests/unit/utils_test.py#L721

Edit2: using the pattern */tmp/always_restart.txt works with Compose, so the ** pattern specifically is ignored

[aanand]

@agilgur5 Correct, this is a known issue with docker-py. I've created https://github.com/docker/docker-py/issues/1117 to track it.

[luckydonald]

Our ugly workaround in the DOCKERFILE:
Using git-check-ignore for parsing (thus needing git init), we just delete everything which should be ignored after copying.
Note, in our case that had to be after all COPYs (or ADDs), else we got unwanted files again.

RUN mkdir /app/
WORKDIR   /app/
# ...
# all the COPYs
# ...
RUN git init
COPY ./.dockerignore /app/
RUN mv .dockerignore .gitignore
RUN DEL_ME=$(find . -print0 | xargs -0) && echo Forcing deletion of .dockerignore files && git check-ignore --no-index $DEL_ME > tmp.txt
RUN DEL_ME=$(cat tmp.txt | xargs) && echo DELETING: $DEL_ME tmp.txt && rm -f -d -r $DEL_ME tmp.txt

To also remove files from the .gitignore (because why not)

COPY ./.gitignore      /app/
COPY ./.dockerignore   /app/
RUN DEL_ME=$(find . -print0 | xargs -0) && echo gitignore && git check-ignore --no-index $DEL_ME > tmp.txt
RUN rm .gitignore && mv .dockerignore .gitignore
RUN DEL_ME=$(find . -print0 | xargs -0) && echo dockerignore && git check-ignore --no-index $DEL_ME >> tmp.txt
RUN DEL_ME=$(cat tmp.txt | xargs) && echo DELETING: DEL_ME tmp.txt && rm -f -d -r $DEL_ME tmp.txt

[jameschao]

I think found a workaround for this by using data volumes in docker-compose.yml, by basically listing the container directories that you don't want to be overwritten. In our case, we wanted to preserve the node_modules directory in the container.

So, our docker-compose.yml looks like this:

  my_app:
    build: ./my_app
    volumes:
      - ./my_app:/app
      # prevent the mounting above from overwriting the following
      - /app/node_modules

The volumes value does duplicate what's in .dockerignore but it gets us going for now...

[cjerdonek]

I have a high-level suggestion for this issue:

I would recommend retitling this issue to something like, "[meta issue]: incorrect handling of .dockerignore rules" (or "[tracking issue]: incorrect handling ..."), or else simply closing this issue in favor of a collection of smaller, more actionable issues. This is because (1) it's not that Docker compose is "ignoring" the .dockerignore file (which is what the issue title currently says). It simply hasn't implemented the rules correctly. And (2), there are a number of parts to this issue. This isn't just one issue. For example, there is this one (** processing) and this new one I just filed (last-line precedence). And there very well may be more.

Because this issue thread is so long (because it covers a broad problem area), the individual problems that need to be solved are hard to locate within the thread. As opposed to a general ".dockerignore doesn't seem to work" issue, I'm suggesting filing well-defined issues that can be solved in isolation. As currently phrased, this issue may never be closed because the Python implementation might never have parity with the reference implementation.

[Chaoste]

I've used **/**/node_modules but figured out that this one was not working. So I defined each node_modules directory in my subprojects to fix this bug.

Before this hotfix I had an "Access Right Error" when installing npm dependencies because they were already there (caused by copying with a not working dockerignore file).

[podorozhny]

Same problem.

macOS Sierra 10.12.2
Docker version 1.12.5, build 7392c3b
docker-compose version 1.9.0, build 2585387

[shin-]

The ** pattern issue should be fixed in 1.11.2
There are possible outstanding issues like the one with last-line precedence, which are tracked separately in #3931 and #3886. Please refer to those for further updates.

[ekkis]

I'm not sure whether docker-compose is ignoring my .dockerignore or it's failing to interpret it correctly (unlike with docker which works just fine). this is how I landed here: http://stackoverflow.com/questions/42883596/equivalent-builds-dont-behave-the-same

and here's my .dockerignore file (in case you can spot what the issue may be):

$ cat .dockerignore 
*
!www
!app
!inc
*/node_modules
*/bower_components
**/*.log
**/Dockerfile
**/.gitignore

[aiomaster]

It seems that my docker-compose build ignores the .dockerignore file completely.
docker build works as expected.

$ docker-compose version                                                                                                                       
docker-compose version 1.14.0, build c7bdf9e
docker-py version: 2.3.0
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016

My .dockerignore looks like this

.bundle
.git
.gitignore
test
tmp
log

so nothing special here.
I am using Ubuntu 16.04 but I don't know if that matters.

BTW: Can someone explain, why docker-compose build does not use the same code as docker build to build an image using the Dockerfile and all that stuff? Why is there an obvious code duplication (that does not work really good as this issue shows)?

[ekkis]

why was this issue closed and how do we re-open it? the software is clearly broken and needs to be fixed

[shin-]

https://github.com/docker/compose/issues/1607#issuecomment-287177186

[ekkis]

ah. thanks shin. glad it will get addressed

[aiomaster]

I've read the issues #3931 and #3886 like suggested, but my .dockerignore is not special, so I could not map exactly my problem.
I doesn't work at all for me and it doesn't matter, whats in my .dockerignore.

[aiomaster]

So stupid, it was my mistake.:dizzy_face:
I mounted my code as a volume so it is clear, that there is nothing ignored. Sorry for that.

app:
  build: .
  container_name: my_app
  volumes:
    - .:/my_app

[NullVoxPopuli]

this seems unresolved?

version: '3.4'
services:
  ui:
    build:
      context: ./source/ui
      dockerfile: Dockerfile
      target: development
    command: bash -c "yarn dev"
    ports:
      - '9091:9091'
      - '9092:9092'
    expose:
      - '9091'
      - '9092'
    volumes:
      - ./source/ui:/app
      - node-modules:/app/node_modules

    environment:
      - PORT=9091
      - HMR_PORT=9092
      - NODE_ENV=development
      - API_HOST=api.docker:7001

volumes:
  node-modules:

in source/ui/.dockerignore:

node_modules
dist
```

but both folders end up getting created anyway...

[MartinMuzatko]

I had the same problem. just when I noticed that I'm on an old compose version. Updated to docker-compose version 1.22.0-rc1, build e7de1bc3.

The problem is, that the compose version shipped with debian/ubuntu is too old.

Now it works better, but still not the same like docker build

[shawnrushefsky]

This is still unresolved. Running Docker for Mac, latest version (Docker 18.03.1-ce-mac65).

My .dockerignore:

**/package-lock.json
**/node_modules

but both are getting passed into the container when using docker-compose.

[shin-]

https://github.com/docker/docker-py/pull/2065 is in 1.22.0 RC2 and should address that.

[rodrigobdz]

I'm using Docker for Mac Version 18.06.1-ce-mac73 (26764) and the bug is still there.

[wladyslaws]

I'm using:

docker-compose version 1.22.0, build f46880f
Docker version 18.06.1-ce, build e68fc7a

running on Ubuntu 16.04 Xenial and docker-compose build ignores the following .dockerignore:

**/*.jpg **/*.png **/*.pyc **/*.solverstate **/*.caffemodel **/*.tgz **/.pytest_cache **/*__pycache__* **/.git **/node_modules *.egg-info .eggs *Dockerfile* build dist
As it is supposed now to interpret double asterisks, I don't know what may be the cause. Anyone with the same issue?

[janhn]

As it is supposed now to interpret double asterisks, I don't know what may be the cause. Anyone with the same issue?

I'm having the same issue (with **/.tox).

[zymtx5g79k]

I had the same issue. I use different contexts.
docker-compose.yml is placed on app root path and looks like

services:
  api:
    build:
      context: ./docker/api

My files tree:

- app/
-- docker/
--- api/
---- .dockerignore <- it works!
-- docker-compose.yml
-- .dockerignore <- it does not work for contexts

Have a nice day!

[rodrigobdz]

@zymtx5g79k That's not working for me.

Directory structure:

- .dockerignore
- docker/
-- development/
--- docker-compose.yml  

docker-compose.yml

version: '3'
services:
  web:
    build:
      context: ../../.

[zymtx5g79k]

@rodrigobdz, check please:
docker-compose.yml:

services:
  api:
    build:
      context: ./../../
version: '3'

.dockerignore:

/docker

Files structure:

- docker/
-- api/
--- docker-compose.yml
- .dockerignore

[rodrigobdz]

Still the same outcome, the .git directory is in the container for example. I'll post here my .dockerignore. Maybe it has some issue itself.

# Custom
docker-compose.yml
docs/
livereload.js*
yarn-error.log
v8-compile-cache-0
# vim
*.swp

.git
.gitignore
README.md

[thaJeztah]

@rodrigobdz I'm not able to reproduce the issue (running docker compose version 1.22.0);

Prepare the test;

mkdir repro-1607 && cd repro-1607
mkdir -p docker/development/
mkdir -p ./.git/this-is-a-git-repo
echo "this is README.md" > README.md

cat > docker/development/docker-compose.yml -<<EOF
version: '3'
services:
  web:
    build:
      context: ../../.
EOF

cat > ./.dockerignore -<<EOF
# Custom
docker-compose.yml
docs/
livereload.js*
yarn-error.log
v8-compile-cache-0
# vim
*.swp

.git
.gitignore
README.md
EOF

cat > Dockerfile -<<EOF
FROM alpine
RUN apk add --no-cache tree
COPY . /foobar/
CMD tree -a /foobar/
EOF

Build using docker-compose;

cd docker/development/
docker-compose build --no-cache

Building web
Step 1/4 : FROM alpine
 ---> 11cd0b38bc3c
Step 2/4 : RUN apk add --no-cache tree
 ---> Running in 8767bc07dad9
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/1) Installing tree (1.7.0-r1)
Executing busybox-1.28.4-r0.trigger
OK: 5 MiB in 14 packages
Removing intermediate container 8767bc07dad9
 ---> 3916d3b689bb
Step 3/4 : COPY . /foobar/
 ---> 76ab68d75f88
Step 4/4 : CMD tree -a /foobar/
 ---> Running in 9891624b3cab
Removing intermediate container 9891624b3cab
 ---> d22b81d149f2
Successfully built d22b81d149f2
Successfully tagged development_web:latest

And verify that the content is not added;

docker run --rm development_web:latest
/foobar/
├── .dockerignore
├── Dockerfile
└── docker
    └── development
        └── docker-compose.yml

2 directories, 3 files

Now, rename the .dockerignore and perform the build again;

mv ../../.dockerignore ../../.dockerignore.disabled
docker-compose build --no-cache

Building web
Step 1/4 : FROM alpine
....

Verify that the content is no longer ignored, and added to the image:

docker run --rm development_web:latest
/foobar/
├── .dockerignore.disabled
├── .git
│   └── this-is-a-git-repo
├── Dockerfile
├── README.md
└── docker
    └── development
        └── docker-compose.yml

4 directories, 4 files

(edit: updated output of the first run, as I forgot to run tree with -a the first time)

[rodrigobdz]

@thaJeztah Thanks for taking the time to look into it. I found the issue.

My use case is similar to the one described in https://github.com/docker/compose/issues/2098#issue-108463351. I use a COPY command in my Dockerfile and additionally, mount the same directory as a volume.

I was expecting the excludes in .dockerignore to be applied to the volume, as described in https://github.com/docker/compose/issues/2098#issuecomment-143505943.

[thaJeztah]

I was expecting the excludes in .dockerignore to be applied to the volume, as described in #2098 (comment).

No, when bind-mounting a path from the host, docker won't be "in the middle" of that; on Linux, it's literally mounting that directory from the host inside the container. On Docker Desktop (Docker for Mac / Windows), there's some additional "magic" involved to make those files available inside the VM where the daemon (and container) runs, but essentially it's doing the same as on Linux after that.

.dockerignore is only used during build, and was intended for speeding up builds; to prevent having to send files to the daemon that are not used/desired in the image.

[rodrigobdz]

Thanks for the clarification! I consider that this explanation should be in the docs to avoid further misunderstanding in the community. Currently, the documentation is more focused on how to ignore files rather than on the scope of .dockerignore itself.

[thaJeztah]

That page of the documentation is describing the Dockerfile and docker build; from that perspective; wondering if it would make sense to describe that it does not work for other commands/uses.

[rodrigobdz]

Well, if it is a common source of misunderstanding, and evidently it is, it would certainly make sense to have it mentioned there.

A single sentence in the documentation would have saved us @thaJeztah, and all the users below, a lot of time.

Users misunderstanding the current documentation

Issue 1607

Issue 2098

[xvrqt]

Just wanted to +1 this issue.

I'm running docker-copose: 1.23.2 (newest I could find between Homebrew and PIP)
OS: Mac OS 10.14.2

It's still ignoring my .dockerfile while building. Going to have to copy things explicitly in the mean time.

[xvrqt]

I fixed this issue by removing an unnecessary volume reference in my docker-compose file. Compose ignores .gitignore for volumes.

This fixed it for me - docker-compose will ignore a .dockerignore file in a mounted volume.

[Harvzor]

@xvrqt I've probably wasted hours over the last week having random errors with my config file not being properly copied (my build process is supposed to copy it but it was being blocked by the .dockerignore file not being used). This is kinda unbelievable.

Thanks for highlighting that.

[LocoDelAssembly]

Hi! What is the status of this? I've prepared very simple docker-compose.yml scenarios that do not involve volumes and yet it fails to pick up .dockerignore when building from remote git repo.

https://github.com/LocoDelAssembly/docker-compose-dockerignore

Is the error I'm having at master branch expected? Any solutions other than building and tagging the image with vanilla docker and then using it in docker-compose.yml?

Thanks

[thaJeztah]

Any solutions other than building and tagging the image with vanilla docker and then using it in docker-compose.yml?

If you're running the current version of compose, you could use the COMPOSE_DOCKER_CLI_BUILD=1 (and DOCKER_BUILDKIT=1) options to make docker compose use the native docker build.

Your example looks to be different than the one being discussed here, so might be good to open a new ticket (if there's none for that yet)

[LocoDelAssembly]

Thanks @thaJeztah! Works with COMPOSE_DOCKER_CLI_BUILD=1, but if I also add DOCKER_BUILDKIT=1 it fails (for an unrelated reason).

https://travis-ci.org/LocoDelAssembly/docker-compose-dockerignore/builds/658351109

[thaJeztah]

Hm, I suspect that's a bug in the version of docker that's running there; docker 18.06 is pretty old (and EOL); that version of docker has quite an early version of BuildKit, which was not yet stable.

```
=> ERROR [internal] load metadata for docker.io/library/alpine:3.9.5 0.1s
218 => ERROR [1/5] FROM docker.io/library/alpine:3.9.5 0.0s
219 => => resolve docker.io/library/alpine:3.9.5 0.0s
220------
221 > [internal] load metadata for docker.io/library/alpine:3.9.5:
222------
223------
224 > [1/5] FROM docker.io/library/alpine:3.9.5:
225------
226failed to solve with frontend dockerfile.v0: failed to build LLB: failed to load cache key: docker.io/library/alpine:3.9.5 not found

[thaJeztah]

Oh, hm, I see your travis is installing docker as well (reading from my phone); does that replace the pre-installed version? If you add a docker info and docker version step after installing, does it show the correct (19.03.x) version of docker to be installed?

[LocoDelAssembly]

Here it is https://travis-ci.org/LocoDelAssembly/docker-compose-dockerignore/builds/658435194

Summary

Docker version for both failed builds

$ docker version
Client: Docker Engine - Community
 Version:           19.03.7
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        7141c199a2
 Built:             Wed Mar  4 01:22:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false
Server: Docker Engine - Community
 Engine:
  Version:          19.03.7
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       7141c199a2
  Built:            Wed Mar  4 01:21:08 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

docker info output is a bit large so not including, but Server Version: 19.03.7.

docker-compose versions

DOCKER_COMPOSE_VERSION=1.26.0-rc2

$ docker-compose version
docker-compose version 1.26.0-rc1, build 07cab513
docker-py version: 4.2.0
CPython version: 3.7.6
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

(Reported version is wrong, it is rc2, the build hash matches rc2)

DOCKER_COMPOSE_VERSION=1.25.4

$ docker-compose version
docker-compose version 1.25.4, build 8d51620a
docker-py version: 4.1.0
CPython version: 3.7.5
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

[Joshua2504]

Issue still existant in 19.03.8.

[LocoDelAssembly]

Issue still existant in 19.03.8.

Correct. Also updated my test repo with the newest 1.25 and 1.26 docker-compose versions but all continues to be as described in https://github.com/LocoDelAssembly/docker-compose-dockerignore

Build where the same kind of info I pasted above can be found: https://travis-ci.org/github/LocoDelAssembly/docker-compose-dockerignore/builds/673393656

[xpetit]

Have the same issue here, .dockerignore doesn't work in a reliable way with docker-compose