Docker-py: docker-compose fails at long paths on windows

[boukeversteegh]

When running docker-compose build in a project that contains long paths (the path below is exactly 260 chars, the maximum allowed in windows), building fails.

It is related to https://github.com/docker/compose/issues/2473 but it seems the solution lies in this repository, since the stack trace matches with these source files.

Let me know if it should be fixed in another repo.

Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "compose\cli\main.py", line 62, in main
  File "compose\cli\main.py", line 114, in perform_command
  File "compose\cli\main.py", line 835, in up
  File "compose\project.py", line 382, in up
  File "compose\service.py", line 305, in ensure_image_exists
  File "compose\service.py", line 727, in build
  File "site-packages\docker\api\build.py", line 55, in build
  File "site-packages\docker\utils\utils.py", line 95, in tar
  File "tarfile.py", line 2007, in add
  File "tarfile.py", line 1879, in gettarinfo
WindowsError: [Error 3] The system cannot find the path specified: 'C:\\Users\\johnd\\dev\\tmp\\foobar\\node_modules\\gulp-clean\\node_modules\\gulp-util\\node_modules\\dateformat\\node_modules\\meow\\node_modules\\normalize-package-data\\node_modules\\validate-npm-package-license\\node_modules\\spdx-correct\\node_modules\\spdx-license-ids\\LICENSE'
docker-compose returned -1

[shin-]

https://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx#maxpath

I guess there's some sort of hack we could implement. In the meantime this is something you could try:

A registry key allows you to enable or disable the new long path behavior. To enable long path behavior set the registry key at HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD). The key's value will be cached by the system (per process) after the first call to an affected Win32 file or directory function (list follows). The registry key will not be reloaded during the lifetime of the process. In order for all apps on the system to recognize the value of the key, a reboot might be required because some processes may have started before the key was set.

The registry key can also be controlled via Group Policy at Computer Configuration > Administrative Templates > System > Filesystem > Enable NTFS long paths.

[Howardbrooks1]

The Long path tool is the very best program for error, unlock solution.and it is very useful tool.

[numberz004]

For those on Windows 10 v1607 and later, the group policy value has been moved to Computer Configuration > Administrative Templates > System > Filesystem > Enable Win32 long paths

[sobelfall]

Run docker-compose with Docker Quickstart Terminal. It will solve your error

[sanchitbh]

Trying to run docker-compose and I keep getting this error:

c:\tmp>docker-compose up
Building web
Traceback (most recent call last):
  File "docker-compose", line 6, in <module>
  File "compose\cli\main.py", line 68, in main
  File "compose\cli\main.py", line 121, in perform_command
  File "compose\cli\main.py", line 938, in up
  File "compose\project.py", line 430, in up
  File "compose\service.py", line 317, in ensure_image_exists
  File "compose\service.py", line 918, in build
  File "site-packages\docker\api\build.py", line 148, in build
  File "site-packages\docker\utils\build.py", line 14, in tar
  File "site-packages\docker\utils\utils.py", line 100, in create_archive
  File "tarfile.py", line 1881, in gettarinfo
WindowsError: [Error 3] The system cannot find the path specified: u'c:\\tmp\\var\\node_modules\\axiom\\node_modules\\npm\\node_modules\\pacote\\node_modules\\make-fetch-happen\\node_modules\\http-proxy-agent\\node_modules\\agent-base\\node_modules\\es6-promisify\\node_modules\\es6-promise\\lib\\es6-promise\\promise\\all.js'
Failed to execute script docker-compose

Versions:

c:\tmp>docker -v
Docker version 17.09.0-ce, build afdb6d4

c:\tmp>docker-compose -v
docker-compose version 1.16.1, build 6d1ac219

I guess this is because of MAX_PATH limit on Windows for which I've tried enabling the Enable Win32 long paths but I still keeping getting this error.

Any ideas on how I can fix this? In the meantime is there an option to tell docker-compose to ignore this file and run the rest of it anyway?

[mbiebusch]

@sanchitbh Could you fix or workaround the problem in the meantime?

[mmalek06]

Guys, has anything changed? It's been more than a year and the problem still persists...

[shin-]

@mmalek06 The solution is to enable NTFS long paths as highlighted in my previous response. This is a Windows issue, not a Compose issue.

[mmalek06]

Well, actually that's not enough. I did it, but still the gpedit window says that in order for this to work, the app has to be manifested. In other words it has to tell windows it wants to use this feature. Docker is not manifested, therefore the issue persists.

[Puneeth-n]

I am getting it on my Jenkins server running on ubuntu.

+ make -C python_lib build

make: Entering directory `/var/lib/jenkins/workspace/PR-3284/python_lib'

Building ct-python-lib

Traceback (most recent call last):

  File "bin/docker-compose", line 6, in <module>

  File "compose/cli/main.py", line 68, in main

  File "compose/cli/main.py", line 121, in perform_command

  File "compose/cli/main.py", line 249, in build

  File "compose/project.py", line 343, in build

  File "compose/service.py", line 918, in build

  File "site-packages/docker/api/build.py", line 148, in build

  File "site-packages/docker/utils/build.py", line 14, in tar

  File "site-packages/docker/utils/utils.py", line 100, in create_archive

  File "tarfile.py", line 1881, in gettarinfo

OSError: [Errno 2] No such file or directory: '/var/lib/jenkins/workspace/PR-3284/terraform/services@tmp/durable-a9021a03'

Failed to execute script docker-compose

make: *** [build] Error 255

make: Leaving directory `/var/lib/jenkins/workspace/PR-3284/python_lib'

script returned exit code 2

[robertlamberson]

Still an issue with docker-compose on windows 10 build 1803. Group policy Enable win32 long paths and registry setting long paths both enabled. Please provide a work around

[8180] Failed to execute script docker-compose
Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose\cli\main.py", line 71, in main
File "compose\cli\main.py", line 127, in perform_command
File "compose\cli\main.py", line 282, in build
File "compose\project.py", line 373, in build
File "compose\service.py", line 1027, in build
File "site-packages\docker\api\build.py", line 154, in build
File "site-packages\docker\utils\build.py", line 31, in tar
File "site-packages\docker\utils\build.py", line 162, in create_archive
File "tarfile.py", line 1802, in gettarinfo
FileNotFoundError: [WinError 3] The system cannot find the path specified:

[shin-]

Working on a fix for this in Compose 1.23. in the meantime, installing Compose via pip seems to get rid of the issue

[robertlamberson]

Shin, I confirmed the work around of installing compose via pip and removing the earlier installed docker-compose resolved my build failures. Thanks

[BrunoZell]

adding node_modules to your .dockerignore can solve this issue too possibly