Vcpkg: Error creating directory `downloads/temp` (Windows)

[arthur-tacca]

Very often while installing a package that requires a download, the installation will fail with an error about creation of downloads/temp:

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:72 (file):
  file problem creating directory: C:/src/vcpkg/downloads/temp
Call Stack (most recent call first):
  scripts/cmake/vcpkg_from_github.cmake:106 (vcpkg_download_distfile)
  ports/mypkg1/portfile.cmake:15 (vcpkg_from_github)
  scripts/ports.cmake:72 (include)

After such a failure, downloads/temp does not exist. Running the same command again succeeds, but leaves an empty temp directory behind, ready to trip over the next run of the command. Often runs of vcpkg end up alternating between working and erroring as they alternate between deleting and recreating the directory.

This has been reported before in #3057. That user said the problem mysteriously cleared up for them, but they didn't say they deleted their downloads so I suspect that these were cached and so vcpkg just never attempted to recreate downloads/temp.

I suspected maybe my virus scanner was causing the directory to linger but I tried disabling it and running on another machine with a different virus scanner, but I still had the same problem in both cases. TortoiseGit was not installed (I mention this because of the confusingly similar #3398 I had also been having).

In vcpkg_download_distfile.cmake, perhaps it would be a good idea to insert a short sleep, like 200ms, between the calls to file(REMOVE_RECURSE "${DOWNLOADS}/temp") and file(MAKE_DIRECTORY "${DOWNLOADS}/temp"). I would give that a go on my machines and see if it fixes it for me, but I don't know how to make cmake do a sleep.

[Ulysses1337]

This was probably my single biggest complaint about Windows before 10 came along. For my own projects, both Python and native, I managed to find some insight in this answer: https://stackoverflow.com/questions/16373747/permission-denied-doing-os-mkdird-after-running-shutil-rmtreed-in-python

Anecdotally at least, I've had some success with the advice to rename before deletion.

[mengfan1102]

I have exactly the same issue, my vcpkg version: 0.0.113-nohash; OS version: Windows 10.0.17134.228

Thanks to @arthur-tacca and @Ulysses1337 's suggestions, I have solved the issue by modifying the file: scripts/cmake/vcpkg_download_distfile.cmake

change the following two lines:

file(REMOVE_RECURSE "${DOWNLOADS}/temp")
file(MAKE_DIRECTORY "${DOWNLOADS}/temp")

to:

if(IS_DIRECTORY "${DOWNLOADS}/temp")
    file(RENAME "${DOWNLOADS}/temp" "${DOWNLOADS}/temp0")
    file(REMOVE_RECURSE "${DOWNLOADS}/temp0")
endif()
file(MAKE_DIRECTORY "${DOWNLOADS}/temp")

Then everything works as expected, hope it helps.

[ras0219-msft]

Thanks for the investigation into what I'm sure is an incredibly frustrating issue!

I've applied the workaround suggested by @mengfan1102 to master -- I'm a bit more hesitant about adding a 200ms sleep since that does have a real, measurable performance impact for small libraries. I'd be happier with a solution that simply catches the download failure and _then_ sleeps and retries. This way the "happy path" isn't affected.

[arthur-tacca]

@ras0219-msft Agreed that mengfan1102's rename workaround is massively better than my suggestion of a short sleep. Great idea! I don't currently have access to the computer that this problem was worse on but I'm happy to believe this is fixed now, so I'll close this. Thanks everyone.

[andreadps]

It seems even the current workaround is not enough in some situations. I am building vcpkg from inside a Docker container. If I build my required ports in two steps like below:

RUN ./vcpkg install first_thing
RUN ./vcpkg install other_thing

The second build will fail like this:

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:79 (file):
  file RENAME failed to rename
    /tmp/workdir/vcpkg/downloads/temp
  to
    /tmp/workdir/vcpkg/downloads/temp0
  because: Invalid cross-device link

And this is vcpkg_download_distfile.cmake:

76:    # Works around issue #3399
77:    if(IS_DIRECTORY "${DOWNLOADS}/temp")
78:        file(REMOVE_RECURSE "${DOWNLOADS}/temp0")
79:        file(RENAME "${DOWNLOADS}/temp" "${DOWNLOADS}/temp0")
80:        file(REMOVE_RECURSE "${DOWNLOADS}/temp0")
81:    endif()
82:    file(MAKE_DIRECTORY "${DOWNLOADS}/temp")

So far, the solution has been to build all in the same step or to delete the temp dir before the seconds build:

rm -rf vcpkg/downloads/temp

I still have not figured out why this container behaves this way.

[lsparey]

I have the same issue. Could vcpkg_download_distfile.cmake not delete the temp directory as it's final step?

[Voskrese]

execute_process(COMMAND ${CMAKE_COMMAND} -E sleep 1 \\ 1 sec

function(sleep delay)
  execute_process(
    COMMAND ${CMAKE_COMMAND} -E sleep ${delay}
    RESULT_VARIABLE result
    )
  if(NOT result EQUAL 0)
    message(FATAL_ERROR "failed to sleep for ${delay} second.")
  endif()
endfunction(sleep)


file(REMOVE_RECURSE "${DOWNLOADS}/temp")
sleep(1)
file(MAKE_DIRECTORY "${DOWNLOADS}/temp")

sleep(0.225)